I Installed kafkatools from http://www.kafkatool.com/download.html in Ubuntu 18.04. The Installation is OK, but when I clicked on it, it does nothing. I installed multiple times on different devices, but I really don’t have any idea what is happening.
First I installed openjdk-8-jdk, kafkatools, and after that I installed zookeeper and kafka, and I removed kafkatools and reinstalled it, but the issue persists.
Kafka is a distributed streaming platform. It is useful for building real-time streaming data pipelines to get data between the systems or applications. Another useful feature is real-time streaming applications that can transform streams of data or react on a stream of data. This answer will help you to install Apache Kafka on Ubuntu 16.04 and later.
1. Install Java
Apache Kafka required Java to run. You must have Java installed on your system. Execute the below command to install default OpenJDK on your system from the official Ubuntu repositories. You can also install the specific version of from here.
sudo apt update sudo apt install openjdk-8-jdk
2. Download Apache Kafka
Download the Apache Kafka binary files from its official download website. You can also select any nearby download mirror.
Then extract the archive file.
tar xzf kafka_2.13-2.4.0.tgz mv kafka_2.13-2.4.0 /usr/local/kafka
3. Setup Kafka systemd unit files
Next create systemd unit files for the Zookeeper and Kafka service. This will help to manage Kafka services to start/stop using the systemctl command.
First, create systemd unit file for Zookeeper with below command:
Add the below content:
[Unit] Description=Apache Zookeeper server Documentation=http://zookeeper.apache.org Requires=network.target remote-fs.target After=network.target remote-fs.target [Service] Type=simple ExecStart=/usr/local/kafka/bin/zookeeper-server-start.sh /usr/local/kafka/config/zookeeper.properties ExecStop=/usr/local/kafka/bin/zookeeper-server-stop.sh Restart=on-abnormal [Install] WantedBy=multi-user.target
Save the file and close it.
Next, to create a Kafka systemd unit file using the following command:
sudo nano /etc/systemd/system/kafka.service
Add the below content. Make sure to set the correct
JAVA_HOME path as per the Java installed on your system.
[Unit] Description=Apache Kafka Server Documentation=http://kafka.apache.org/documentation.html Requires=zookeeper.service [Service] Type=simple Environment="JAVA_HOME=/usr/lib/jvm/java-1.11.0-openjdk-amd64" ExecStart=/usr/local/kafka/bin/kafka-server-start.sh /usr/local/kafka/config/server.properties ExecStop=/usr/local/kafka/bin/kafka-server-stop.sh [Install] WantedBy=multi-user.target
Save the file and close.
Reload the systemd daemon to apply the new changes.
4. Start Kafka server
Kafka required ZooKeeper so first, start a ZooKeeper server on your system. You can use the script available with Kafka to get start a single-node ZooKeeper instance.
sudo systemctl start zookeeper
Now start the Kafka server and view the running status:
sudo systemctl start kafka sudo systemctl status kafka
All done. The Kafka installation has been successfully completed. The next part of this answer will help you to work with the Kafka server.
5. Create a topic in Kafka
Kafka provides multiple pre-built shell script to work on it. First, create a topic named “testTopic” with a single partition with single replica:
cd /usr/local/kafka bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic testTopic Created topic testTopic.
The replication-factor describes how many copies of data will be created. As we are running with a single instance keep this value 1.
Set the partitions options as the number of brokers you want your data to be split between. As we are running with a single broker keep this value 1.
You can create multiple topics by running the same command as above. After that, you can see the created topics on Kafka by the running the below command:
bin/kafka-topics.sh --list --zookeeper localhost:2181 testTopic TecAdminTutorial1 TecAdminTutorial2
Alternatively instead of manually creating topics you can also configure your brokers to auto-create topics when a nonexistent topic is published to.
6. Send messages to Kafka
The “producer” is the process responsible for putting data into our Kafka. Kafka comes with a command-line client that will take input from a file or from standard input and send it out as messages to the Kafka cluster. The default Kafka sends each line as a separate message.
Let’s run the producer and then type a few messages into the console to send to the server.
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic testTopic > Welcome to kafka > This is my first topic
You can exit this command or keep this terminal running for further testing. Now open a new terminal to the Kafka consumer process on the next step.
7. Using Kafka consumer
Kafka also has a command-line consumer to read data from the Kafka cluster and display messages to standard output.
bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic testTopic --from-beginning Welcome to kafka This is my first topic
Now If you have still running Kafka producer (Step 6) in another terminal, just type some text on that producer terminal. It will immediately be visible on consumer terminal. See the below screenshot of Kafka producer and consumer working: