Setting Up an Apache ZooKeeper Cluster on Ubuntu 18.04 LTS

Apache ZooKeeper serves as a centralized repository for managing configuration information, naming, distributed synchronization, and the provisioning of group services, all available as open-source software. Written in Java, ZooKeeper simplifies the process by allowing you to focus on core application logic without being burdened by the complexities inherent in distributed systems. Its architecture promotes high availability with redundant services, enabling distributed processes to coordinate through a hierarchical namespace of data registers. ZooKeeper stands out for its high throughput, low latency, high availability, and controlled access to Znodes. Designed to store status information, configurations, and location data, ZooKeeper offers features like reliability, simplicity in architecture, fast processing speeds, scalability, and more.

In this guide, we will learn how to set up a single-node Apache ZooKeeper cluster on Ubuntu 18.04.

Requirements

• A server running Ubuntu 18.04.
• A root password configured on your server.

Getting Started

First, update your system to the latest version by executing the following commands:

apt-get update -y
apt-get upgrade -y

Once updated, restart your system to apply the changes.

Install Java

Since ZooKeeper is written in Java, you need to install Java on your system. The latest version is not available in the default Ubuntu 18.04 repository. You can add the Java repository with the following command:

add-apt-repository ppa:linuxuprising/java

Update the repository and install Java using:

apt-get update -y
apt-get install oracle-java11-set-default

Confirm the Java installation by checking the version:

java --version

Expected output:

java 11.0.2 2018-10-16 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.2+7-LTS)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.2+7-LTS, mixed mode)

With Java installed, proceed to the next step.

Create a ZooKeeper User

Next, create a user for ZooKeeper to host its service:

useradd zookeeper -m
usermod --shell /bin/bash zookeeper

Set a password for this user:

passwd zookeeper

Add the ZooKeeper user to the sudo group to permit privileged command execution:

usermod -aG sudo zookeeper

Now, you can proceed to install ZooKeeper.

Install ZooKeeper

Create the necessary directory structure for ZooKeeper:

mkdir -p /data/zookeeper

Assign ownership of the directory to the ZooKeeper user:

chown -R zookeeper:zookeeper /data/zookeeper

Navigate to the /opt directory and download the ZooKeeper package:

cd /opt
wget https://archive.apache.org/dist/zookeeper/zookeeper-3.4.9/zookeeper-3.4.9.tar.gz

Extract the downloaded package:

tar -xvzf zookeeper-3.4.9.tar.gz

Rename the directory:

mv zookeeper-3.4.9 zookeeper

Assign the ZooKeeper user ownership of the ZooKeeper directory:

chown -R zookeeper:zookeeper /opt/zookeeper

Configure ZooKeeper

Create a configuration file for ZooKeeper:

nano /opt/zookeeper/conf/zoo.cfg

Insert these configurations:

tickTime=2500
dataDir=/data/zookeeper
clientPort=2181
maxClientCnxns=80

Save and close the file.

Start the ZooKeeper service:

cd /opt/zookeeper
bin/zkServer.sh start

Expected startup message:

ZooKeeper JMX enabled by default
Using config: /opt/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

Connect to the local ZooKeeper server:

bin/zkCli.sh -server 127.0.0.1:2181

Upon connecting, you should see:

[zk: 127.0.0.1:2181(CONNECTED) 1]

Type help to view available commands:

help
ZooKeeper -server host:port cmd args
    stat path [watch]
    set path data [version]
    ls path [watch]
    delquota [-n|-b] path
    ls2 path [watch]
    setAcl path acl
    setquota -n|-b val path
    history 
    redo cmdno
    printwatches on|off
    delete path [version]
    sync path
    listquota path
    rmr path
    get path [watch]
    create [-s] [-e] path data acl
    addauth scheme auth
    quit 
    getAcl path
    close 
    connect host:port

Type quit to exit.

To stop ZooKeeper, use:

bin/zkServer.sh stop

Expected shutdown message:

ZooKeeper JMX enabled by default
Using config: /opt/zookeeper/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED

Create a Systemd Service File for ZooKeeper

Create a systemd service file to control the ZooKeeper service:

nano /etc/systemd/system/zookeeper.service

Include the following configuration:

[Unit]
Description=Zookeeper Daemon
Documentation=http://zookeeper.apache.org
Requires=network.target
After=network.target

[Service]    
Type=forking
WorkingDirectory=/opt/zookeeper
User=zookeeper
Group=zookeeper
ExecStart=/opt/zookeeper/bin/zkServer.sh start /opt/zookeeper/conf/zoo.cfg
ExecStop=/opt/zookeeper/bin/zkServer.sh stop /opt/zookeeper/conf/zoo.cfg
ExecReload=/opt/zookeeper/bin/zkServer.sh restart /opt/zookeeper/conf/zoo.cfg
TimeoutSec=30
Restart=on-failure

[Install]
WantedBy=default.target

Save and close the file.

Reload systemd, then start and enable the ZooKeeper service:

systemctl daemon-reload
systemctl start zookeeper
systemctl enable zookeeper

Check the status of the ZooKeeper service:

systemctl status zookeeper

You should see something similar to:

? zookeeper.service - Zookeeper Daemon
   Loaded: loaded (/etc/systemd/system/zookeeper.service; disabled; vendor preset: enabled)
   Active: active (running) since Wed 2019-03-06 15:03:01 UTC; 5s ago
     Docs: http://zookeeper.apache.org
  Process: 3909 ExecStart=/opt/zookeeper/bin/zkServer.sh start /opt/zookeeper/conf/zoo.cfg (code=exited, status=0/SUCCESS)
 Main PID: 3926 (java)
    Tasks: 17 (limit: 1113)
   CGroup: /system.slice/zookeeper.service
           ??3926 java -Dzookeeper.log.dir=. -Dzookeeper.root.logger=INFO,CONSOLE -cp /opt/zookeeper/bin/../build/classes:/opt/zookeeper/bin/..

Mar 06 15:03:00 ubuntu1804 systemd[1]: Starting Zookeeper Daemon...
Mar 06 15:03:00 ubuntu1804 zkServer.sh[3909]: ZooKeeper JMX enabled by default
Mar 06 15:03:00 ubuntu1804 zkServer.sh[3909]: Using config: /opt/zookeeper/conf/zoo.cfg
Mar 06 15:03:01 ubuntu1804 zkServer.sh[3909]: Starting zookeeper ... STARTED
Mar 06 15:03:01 ubuntu1804 systemd[1]: Started Zookeeper Daemon.

Congratulations! You have successfully installed and configured a single-node ZooKeeper cluster on your Ubuntu 18.04 server. For production deployments, consider setting up a multi-node ZooKeeper cluster.

FAQ

What is the purpose of Apache ZooKeeper?

Apache ZooKeeper is used to manage configuration information, naming, and distributed synchronization for distributed applications, ensuring coordination and high availability.

Why do I need to create a separate user for ZooKeeper?

Creating a separate user for ZooKeeper enhances security by running the service with minimal required privileges, isolating it from other processes on the server.

Why should I configure ZooKeeper as a systemd service?

Configuring ZooKeeper as a systemd service allows ease of management, such as automatically starting the service on boot and provides a simple interface for starting, stopping, and checking the status of the service.

Can this tutorial be used to install ZooKeeper on other Ubuntu versions?

While this guide is tailored for Ubuntu 18.04, the steps should be applicable to other versions of Ubuntu with possible variations in package names or available Java versions.

How do I install a multi-node ZooKeeper cluster?

To set up a multi-node ZooKeeper cluster, you’ll need to configure multiple machines with similar steps, adjust the zoo.cfg for your nodes, and ensure proper network connectivity between the nodes.