Ubuntu HowTo: Kafka tools won’t start on Ubuntu 18.04

Original Source Link

I Installed kafkatools from http://www.kafkatool.com/download.html in Ubuntu 18.04. The Installation is OK, but when I clicked on it, it does nothing. I installed multiple times on different devices, but I really don’t have any idea what is happening.

First I installed openjdk-8-jdk, kafkatools, and after that I installed zookeeper and kafka, and I removed kafkatools and reinstalled it, but the issue persists.

Kafka is a distributed streaming platform. It is useful for building real-time streaming data pipelines to get data between the systems or applications. Another useful feature is real-time streaming applications that can transform streams of data or react on a stream of data. This answer will help you to install Apache Kafka on Ubuntu 16.04 and later.

1. Install Java

Apache Kafka required Java to run. You must have Java installed on your system. Execute the below command to install default OpenJDK on your system from the official Ubuntu repositories. You can also install the specific version of from here.

sudo apt update
sudo apt install openjdk-8-jdk

2. Download Apache Kafka

Download the Apache Kafka binary files from its official download website. You can also select any nearby download mirror.

wget http://www-us.apache.org/dist/kafka/2.4.0/kafka_2.13-2.4.0.tgz

Then extract the archive file.

tar xzf kafka_2.13-2.4.0.tgz
mv kafka_2.13-2.4.0 /usr/local/kafka

3. Setup Kafka systemd unit files

Next create systemd unit files for the Zookeeper and Kafka service. This will help to manage Kafka services to start/stop using the systemctl command.

First, create systemd unit file for Zookeeper with below command:

vim /etc/systemd/system/zookeeper.service

Add the below content:

[Unit]
Description=Apache Zookeeper server
Documentation=http://zookeeper.apache.org
Requires=network.target remote-fs.target
After=network.target remote-fs.target
[Service]
Type=simple
ExecStart=/usr/local/kafka/bin/zookeeper-server-start.sh /usr/local/kafka/config/zookeeper.properties
ExecStop=/usr/local/kafka/bin/zookeeper-server-stop.sh
Restart=on-abnormal
[Install]
WantedBy=multi-user.target

Save the file and close it.

Next, to create a Kafka systemd unit file using the following command:

sudo nano /etc/systemd/system/kafka.service

Add the below content. Make sure to set the correct JAVA_HOME path as per the Java installed on your system.

[Unit]
Description=Apache Kafka Server
Documentation=http://kafka.apache.org/documentation.html
Requires=zookeeper.service
[Service]
Type=simple
Environment="JAVA_HOME=/usr/lib/jvm/java-1.11.0-openjdk-amd64"
ExecStart=/usr/local/kafka/bin/kafka-server-start.sh /usr/local/kafka/config/server.properties
ExecStop=/usr/local/kafka/bin/kafka-server-stop.sh
[Install]
WantedBy=multi-user.target

Save the file and close.

Reload the systemd daemon to apply the new changes.

systemctl daemon-reload

4. Start Kafka server

Kafka required ZooKeeper so first, start a ZooKeeper server on your system. You can use the script available with Kafka to get start a single-node ZooKeeper instance.

sudo systemctl start zookeeper

Now start the Kafka server and view the running status:

sudo systemctl start kafka
sudo systemctl status kafka

enter image description here

All done. The Kafka installation has been successfully completed. The next part of this answer will help you to work with the Kafka server.

5. Create a topic in Kafka

Kafka provides multiple pre-built shell script to work on it. First, create a topic named “testTopic” with a single partition with single replica:

cd /usr/local/kafka
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic testTopic
Created topic testTopic.

The replication-factor describes how many copies of data will be created. As we are running with a single instance keep this value 1.

Set the partitions options as the number of brokers you want your data to be split between. As we are running with a single broker keep this value 1.

You can create multiple topics by running the same command as above. After that, you can see the created topics on Kafka by the running the below command:

bin/kafka-topics.sh --list --zookeeper localhost:2181
testTopic
TecAdminTutorial1
TecAdminTutorial2

Alternatively instead of manually creating topics you can also configure your brokers to auto-create topics when a nonexistent topic is published to.

6. Send messages to Kafka

The “producer” is the process responsible for putting data into our Kafka. Kafka comes with a command-line client that will take input from a file or from standard input and send it out as messages to the Kafka cluster. The default Kafka sends each line as a separate message.

Let’s run the producer and then type a few messages into the console to send to the server.

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic testTopic

> Welcome to kafka
> This is my first topic

You can exit this command or keep this terminal running for further testing. Now open a new terminal to the Kafka consumer process on the next step.

7. Using Kafka consumer

Kafka also has a command-line consumer to read data from the Kafka cluster and display messages to standard output.

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic testTopic --from-beginning
Welcome to kafka
This is my first topic

Now If you have still running Kafka producer (Step 6) in another terminal, just type some text on that producer terminal. It will immediately be visible on consumer terminal. See the below screenshot of Kafka producer and consumer working:

enter image description here

Source: How to Install Apache Kafka on Ubuntu 18.04 & 16.04

Tagged : /

Ubuntu HowTo: Problems with Codeblocks

Original Source Link

I installed Codeblocks 17.12-1 and it’s not responding after building the file.
So I wanna try Codeblocks 16.01 from the terminal but it shows errors.

:~$ sudo apt install codeblocks
[sudo] password for salil: 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies: 
codeblocks : Depends: codeblocks-common (= 16.01+dfsg-2.1) but it is not 
going to be installed
E: Unable to correct problems, you have held broken packages.
:~$ 

Remove the codeblocks-common.

sudo apt-get remove codeblocks-common

then

sudo apt install -f 
sudo apt-get update.

Now try to reinstall. It should work now.

Tagged : /

Ubuntu HowTo: Microdia webcam not seen on Dell Laptop

Original Source Link

Using Xubuntu 18.04 on a Dell laptop that does not have a web camera.
I bought an external webcam that advertised as being compatible with ubuntu. Plugged in, lsusb sees it, but it will not show an image on cheese.

Bus 002 Device 013: ID 0c45:6366 Microdia 

I think the video driver for the camera is not present.

How do I get/make the appropriate service to use the web camera?
Or is these some other solution to the problem?

I too have a Microdia webcam, and mine works, albeit the colours seem very washed out compared to how the same camera appears on Windows.
Mine shows up as:
Bus 001 Device 005: ID 0c45:624e Microdia PC Camera (SN9C201 + SOI968)
You will most likely need the video4linux support libraries.
Check if you have them already. In terminal, do:

locate v4l1compat.so

If yours are in (most likely, and assuming you have a 64bit linux)
/usr/lib/x86_64-linux-gnu/libv4l/v4l1compat.so
Assuming you want to see the webcam output in cheese, create a file as follows:

#!/bin/bash
LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libv4l/v4l1compat.so cheese

save it as a script, mine is called ´cheesefix.sh´ (yes, cheesy!)
then look at the file permissions, make it executable.
finally do (in terminal)

./cheesefix.sh

or whatever you called it.

this is actually heavily based on

https://askubuntu.com/questions/418356/trust-webcam-on-ubuntu-12-04

Tagged : /

Ubuntu HowTo: How to fix “dpkg: error processing package systemd (–configure): [duplicate]

Original Source Link

I recently upgraded from 16.04 to 18.04, and I think something went wrong with the upgrade. When I run sudo apt-get install -f, it tries to upgrade udev, but then I get repeated messages about “unsafe symlinks” and this error message:

dpkg: error processing package systemd (--configure):
 installed systemd package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 systemd
E: Sub-process /usr/bin/dpkg returned an error code (1)

I was getting a similar message about systemd-shim earlier, but managed to fix it. I also had a problem with incorrect /etc/apt/sources.list, but I think that’s fixed as well.
I was getting various error messages during the upgrade process, and at the end I had to “destroy” something instead of finishing the upgrade properly (unfortunately I don’t remember exactly what happened.)
Any ideas as to what’s going on?

Check if you are the owner of all directories:

stat / /dev /var

If you aren’t the owner, run the following in terminal:

chown root DirectoryYouAreNotOwner

Note: replace DirectoryYouAreNotOwner with the directory that you aren’t owner of. So for example if the owner of / was unknown:

chown root /

Credit to this Ask Ubuntu post.

Tagged : / /

Ubuntu HowTo: Screen brightness in Ubuntu 18.04

Original Source Link

So at first even the brightness key (via Fn button) used to work in Ubuntu 18.04. After an update, i can’t even change it from the setting. And with the keys, it just show the icon of the screen brightness which is changing, and doesn’t actually do anything.

To be mentioned, with brightness-controller-simple which is suggested in this answer, the “actual” brightness doesn’t change, it just makes the colors more dark, with same amount of light coming out of the screen.
The xbacklight also doesn’t work; I mean, this command doesn’t do a thing xbacklight -inc 10, which is suggested here.

I finally could change the actual brightness, but from the terminal command which i read here. Which is this command:

sudo bash -c "echo 130 > /sys/class/backlight/intel_backlight/brightness;"

Is there any other way i could probably change the brightness?


I have ASUS model V502U. And results from lspci -v | less, video adapter part:

00:02.0 VGA compatible controller: Intel Corporation HD Graphics 520 (rev 07) (prog-if 00 [VGA controller])
Subsystem: ASUSTeK Computer Inc. Skylake GT2 [HD Graphics 520]
Flags: bus master, fast devsel, latency 0, IRQ 129
Memory at dd000000 (64-bit, non-prefetchable) [size=16M]
Memory at b0000000 (64-bit, prefetchable) [size=256M]
I/O ports at f000 [size=64]
[virtual] Expansion ROM at 000c0000 [disabled] [size=128K]
Capabilities: <access denied>
Kernel driver in use: i915
Kernel modules: i915

I found an answer, but not the best one!

Read this, to change the /etc/X11/xorg.conf file.

I did the solution from that post, and installed the new updates, then xbacklight worked. But still, I couldn’t change the brightness from setting. According to this answer, you can increase the screen brightness by 10% by this command:

xbacklight -inc 10

And decrease the brightness by 10% by this command:

xbacklight -dec 10

Then create a keyboard shortcut, for intsance, shift+f9 for command xbacklight -inc 10, or so. Unfortunately, it doesn’t let you use fn+f9 shortcut, since it is supposed to have a job itself primitively.

On Ubuntu 18.04 (with xfce desktop) my brightness problem finally solved.

I opened ‘Onboard Settings’ under the ‘Settings’ menu. If your desktop doesn’t have one, you can download it from Software Center.

On the ‘Onboard Preferences’ window, click the Layout menu, then choose the ‘Full Keyboard’ (Desktop keyboard with edit and function keys). Click here to view the Onboard Settings window

We require “xrandr” command to view and set the screen on Ubuntu 18.

[email protected]:/home/jarurote# xrandr

Screen 0: minimum 8 x 8, current 1600 x 900, maximum 32767 x 32767

eDP1 connected primary 1600×900+0+0 (normal left inverted right x axis y axis) 440mm x 250mm
1600×900 60.00*+ 59.82
1440×900 59.89
1400×900 59.96 59.88
1368×768 60.00 59.88 59.85
1360×768 59.80 59.96
1280×800 59.81 59.91
1152×864 60.00
1280×720 59.86 60.00 59.74
1024×768 60.00
1024×576 60.00 59.90 59.82
960×540 60.00 59.63 59.82
800×600 60.32 56.25
864×486 60.00 59.92 59.57
800×450 60.00
640×480 59.94
720×405 59.51 60.00 58.99
640×360 59.84 59.32 60.00

VGA1 disconnected (normal left inverted right x axis y axis)

VIRTUAL1 disconnected (normal left inverted right x axis y axis)

[email protected]:/home/jarurote#

See that the 3rd line of output command shows the output screen of “eDP1“.

To adjust the brightness of the screen, we can define the brightness of the screen in a range of [0.3, 1.0]. Suppose we need the brightness of 0.4, the command of xrandr should be like this.

xrandr –output eDP1 –brightness .4

That is it.

PS: Don’t adjust the brightness value higher than 1.

I had the same problem and the solution was changing the GRUB settings!
(to change GRUB you must have superuser access)

edit your GRUB by running:

sudo nano /etc/default/grub

add acpi_backlight=vendor to GRUB_CMDLINE_LINUX_DEFAULT, so it should look like something like this (if you don’t have additional parameters):

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash acpi_backlight=vendor"

then you should update your GRUB by this command:

sudo update-grub

and after all, you must reboot your system for the changes to take effect:

sudo reboot

In case the above suggestion doesn’t help you. Here I am suggesting which might help you. I have installed fresh Ubuntu 18.04 LTS and landed at this problem. In case, you have Nvidia Graphics Driver. Please do the following:
Edit (or create) inside /usr/share/X11/xorg.conf.d/10-nvidia.conf

Section "OutputClass"
    Identifier "nvidia"
    MatchDriver "nvidia-drm"
    Driver "nvidia"
    Option "AllowEmptyInitialConfiguration"
    Option "RegistryDwords" "EnableBrightnessControl=1;" # Add this if it exists
    ModulePath "/usr/lib/x86_64-linux-gnu/nvidia/xorg"
EndSection

Commands:

sudo vim /usr/share/X11/xorg.conf.d/10-nvidia.conf
Check with the above code.

Tagged : / /

Ubuntu HowTo: Caps lock stopped working after an update

Original Source Link

For some reason, Caps Lock stopped working on my machine. Not only on the laptop keyboard, but on any connected keyboard as well. I’m using 18.04 on a Dell XPS 15 9560.

It worked perfectly until a few days ago and it stopped working after rebooting when I just had done a regular system update.

Had similar problems with a KDE VM that had been working for years, before a VMWare update, and after trying MANY possibilities within KDE, to no avail, turned off the VMWare’s enhanced keyboard and everything returned to normal.

Tagged : / /

Ubuntu HowTo: Ubuntu 18.04 stopped working with NVIDIA drivers

Original Source Link

I am running Ubuntu 18.04 on a CLEVO notebook and have the nvida-390 drivers and Cuda installed. It worked fine for a month, but – suddenly today it stopped working. I can still access the login screen, but the screen remains purple.

I’ve already tried to purge and reinstall the nvidia-390 drivers (I’ve also tried the 396 drivers version) without success. After the login the computer always stops working. I cannot even access tty. After purging the drivers I can login and all works fine with nouveau, but I really need Cuda for my work.

My specs:

  • i7-6700HQ
  • 8GB RAM
  • Nvidia GeForce GTX 970M
  • Intel Wireless 8260

Can someone please help me? Thanks.

You might need to have Direct Rendering Manager Kernel Mode Setting enabled on system boot.
NVIDIA driver’s PRIME Synchronization support relies on DRM-KMS, which is disabled by default.
Find more comprehensive information in the discussion on the NVIDIA GPU Unix Graphics forum.

Execute sudo nano /etc/default/grub and add the parameter nvidia-drm.modeset=1 to the line GRUB_CMDLINE_LINUX_DEFAULT. Save the change you’ve made and run sudo update-grub.
Restart the Ubuntu operating system, and now everything should work properly, right as expected.

I was able to get it working with the latest Nvidia-418 drivers on a Lenovo W530 with the Nvidia GK107GLM Quadro K2000M:

  1. Edit the sudo nano /etc/default/grub and remove “quiet splash” from GRUB_CMDLINE_LINUX_DEFAULT and then sudo update-grub (that may be similar then adding nvidia-drm.modeset=1)
  2. Switch to LightDM: sudo dpkg-reconfigure lightdm (necessary so my second monitor was recognized)
  3. Install PPA with latest drivers: sudo add-apt-repository ppa:graphics-drivers/ppa and sudo apt-get update
  4. Install latest recommended driver (in my case Nvidia 418): sudo ubuntu-drivers autoinstall
Tagged : / /

Ubuntu HowTo: Brightness problem Ubuntu 18.04 LTS

Original Source Link

I have upgraded to ubuntu 18.04 LTS the display brightness controls are not working. It appears that the brightness is in its lowest level and I am not able to change display brightness at all. It was working on 16.04 LTS. My laptop uses Intel Graphics card.

I have tried to solve that by creating the file /usr/share/X11/xorg.conf.d/20-intel.conf with the following content. It worked the first time, but after one reboot it remains back to the previous problem.

Section "Device"
        Identifier  "card0"
        Driver      "intel"
        Option      "Backlight"  "intel_backlight"
        BusID       "PCI:0:2:0"
EndSection

What can I do?

I also had this problem, but on different distributions the solution was different.

Solution provided by Anas Elazhar worked well until I switched to Xubuntu:

Open the file /etc/default/grub using gedit or any other text editor.
Find below line.

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"  

Change above line to

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash acpi_backlight=vendor"

If it doesn’t work for you either try to change acpi_backlight value to video:

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash acpi_backlight=video"

Also create the file /usr/share/X11/xorg.conf.d/80-backlight.conf but content is slightly different from the file that you created:

Section "Device"
    Identifier  "Intel Graphics"
    Driver      "intel"
    Option      "AccelMethod"     "sna"
    Option      "Backlight"       "acpi_video0"
    BusID       "PCI:0:2:0"
EndSection

If combination acpi_backlight=video with this file doesn’t work change acpi_backlight back to vendor.

And do not forget to run sudo update-grub command in terminal every time you change /etc/default/grub!

Open the file /etc/default/grub using gedit or any other text editor. Find below line.

  GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"  

Change above line to

  GRUB_CMDLINE_LINUX_DEFAULT="quiet splash acpi_backlight=vendor"

This would ensure to load device specific drivers before default drivers in Linux.

Save the file and close text editor.

Open terminal and run below command to update grub.

  sudo update-grub

Reboot.

After reboot, try adjusting brightness using Laptop dedicated control keys. Good Luck.

Based on my current Ubuntu 18.04 installation (from today), I can adjust brightness at the login screen with the function keys (out of the box!). However, I am using i3 with Gnome, and after login, I am no more shown brightness controls in Gnome control centre.

Above answers didn’t unfortunately work for me: Instead, using Ducky’s xorg script together with Grub option made my i3 + Gnome session having undesired display behaviour and getting slower.

Anyway, at least for the case you go for a non-Gnome session (e.g. i3), I can point you to https://github.com/szekelyszilv/ybacklight
which (given you have meson and ninja installed) can be built by:

  • downloading, unpacking and changing into ybacklight main directory
  • mkdir build; meson build; cd build; ninja install

or, (without those tools and if you don’t want to install it in the system), by:

  • cd src; gcc ybacklight.c -o ybacklight; cp ybacklight to-your-pref-dir

and then be attached to function keys via WMs config. E.g. in i3wm, I added:

bindsym XF86MonBrightnessUp exec ybacklight -inc 10 
bindsym XF86MonBrightnessDown exec ybacklight -dec 10 

to my .config/i3/config.

Furthermore, setting brightness requires root privileges, e.g., achievable with “sudo ybacklight …” as well as an entry to /etc/sudoers: see How can I add a new user as sudoer using the command line?

I’ve recently updated to 18.04 and I have found that changing GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub to the following

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash acpi_backlight=video"

and adding,

Option         "RegistryDwords" "EnableBrightnessControl=1"

to /etc/X11/xorg.conf under

Section "Device"

fixed the control and brightness keys issues running Nvidia 390 driver.

I started the nvidia X server settings and in the x server display configuration changed the selection to X Screen0 and it worked.

The brightness control works now. Also before doing that I’ve changed the grub to acpibacklight=video0.

On Ubuntu 18.04 (with xfce desktop) my brightness problem finally solved.

I opened ‘Onboard Settings’ under the ‘Settings’ menu. If your desktop doesn’t have one, you can download it from Software Center.

On the ‘Onboard Preferences’ window, click the Layout menu, then choose the ‘Full Keyboard’ (Desktop keyboard with edit and function keys). Click here to view the Onboard Settings window

Installing Brightness Controller helped me in fixing up the issue

 sudo add-apt-repository ppa:apandada1/brightness-controller
 sudo apt update
 sudo apt install brightness-controller

You can access the Brightness Controller after download by searching for it.

I managed to get Brightness control working with nvidia drivers on Debian 10.

Here is my hardware :
– ThinkPad P51
– Nvidia Quadro M2200 GPU + Intel GPU

Here is how I did :

  • Go to Bios and configure display to Discrete graphics (only Nvidia GPU will be used)

  • Remove Nouveau driver if it is setup

    Add nouveau blacklist to /etc/modprobe.d/nvidia-installer-disable-nouveau.conf

    blacklist nouveau
    options nouveau modeset=0

    then run

    update-initramfs -u

    then reboot

  • Download and install latest Nvidia driver from Nvidia website (https://www.nvidia.com/Download/index.aspx?lang=en) Mine was version 430.50

    Open a tty console without any X Server launched (Ctrl+Alt+F3 at login page)

    Install linux headers if you don’t have ( apt-get install linux-headers-`uname -r` )

    Install driver as root ./NVIDIA-Linux-x86_64-430.50.run (erase libs if requested)

That’s it everyting working perfectly finally !!

Tagged : /

Ubuntu HowTo: Need help for extundelete

Original Source Link

Following is the output and command, I am trying to recover a shift deleted folder from home in sda1 , currently logged in via live CD Ubuntu 18.04

[email protected]:~$ sudo extundelete --restore-directory /home/ /dev/sda1
NOTICE: Extended attributes are not restored.
Loading filesystem metadata ... 1526 groups loaded.
Loading journal descriptors ... 31059 descriptors loaded.
Searching for recoverable inodes in directory /home/ ... 
2440 recoverable inodes found.
Looking through the directory structure for deleted files ... 
2440 recoverable inodes still lost.
No files were undeleted.

From the author of extundelete:

I guess that it is not finding a way to link the inodes with a file name, in which case the --restore-all method is your best shot at getting your files back.

https://sourceforge.net/p/extundelete/mailman/message/30159985/

So in your case, you would instead run:

sudo extundelete /dev/sda1 --restore-all

Although from my experience the output is deceptive because the files (inodes) that are still “lost” are for the whole partition and not just for the directory you specified, meaning that most of what it’s able to restore has already been restored. But it doesn’t hurt to try --restore-all since it may find some file fragments that it doesn’t know the path for.

Tagged : / /

Ubuntu HowTo: Updated users/Groups from Active Directory Integration with Ubuntu 18.04 LTS

Original Source Link

I set up the AD Integration for my Ubuntu 18.04 LTS workstation through a Virtual Machine using these two links:

https://discourse.ubuntu.com/t/service-sssd/11579

https://docs.microsoft.com/en-us/azure/active-directory-domain-services/join-ubuntu-linux-vm

And it seems to work fine as I am able to log in with my administrative account I used to join the machines. I am also able to search specific users on my Windows AD server although I have ran into some difficulties which am curious if anyone knows the answer to.

  1. If I create a new user on AD, it doesn’t seem to automatically transfer over to my Ubuntu. I can’t search that user using id as it doesn’t seem to exist.

  2. If I create a new group and add my user to that group, it also doesn’t transfer over…

Those are the main problems I have run into while setting up AD integration with Ubuntu. Am curious if there is something I am supposed to do on Ubuntu to “update” in the new users created on AD.

Edited: Edited: Basically is there a way to make sure from either /etc/sssd/sssd.conf or somewhere else that AD user list will automatically be updated onto my Ubuntu 18.04 instead of having to realm join again.

Tagged : /