RabbitMQ is a robust, open-source message-broker software originally built on the AMQP (Advanced Message Queuing Protocol). Over time, RabbitMQ has expanded its capabilities to support other protocols like STOMP (Streaming Text Oriented Messaging Protocol) and MQTT (Message Queuing Telemetry Transport).
Message-broker software like RabbitMQ enables seamless exchange of messages between distributed systems, applications, and services. Written in the Erlang programming language, RabbitMQ provides extensive support for client interfaces and libraries across major programming languages, including Python, NodeJS, Java, and PHP.
This comprehensive guide will walk you through setting up a RabbitMQ Cluster on Ubuntu 18.04 Server. We will use three Ubuntu servers to build the RabbitMQ Cluster, enable RabbitMQ Management, and establish a High Availability (HA) policy across all nodes.
Prerequisites
- Three or more Ubuntu 18.04 Servers:
- 10.0.15.21 hakase-ubuntu01
- 10.0.15.22 hakase-ubuntu02
- 10.0.15.23 hakase-ubuntu03
- Root privileges
What We Will Do
- Setup Hosts File
- Install RabbitMQ Server
- Enable Management Plugins
- Setup UFW Firewall
- Setup RabbitMQ Cluster
- Setup New Administrator User
- RabbitMQ Setup Queue Mirroring
- Testing
Step 1 – Setup Hosts File
Edit the ‘/etc/hosts’ file on all servers to map each server’s IP address to a hostname.
Use the vim editor to edit the ‘/etc/hosts’ file.
sudo vim /etc/hosts
Add the following configuration to the file:
10.0.15.21 hakase-ubuntu01 10.0.15.22 hakase-ubuntu02 10.0.15.23 hakase-ubuntu03
Save and exit.
Step 2 – Install RabbitMQ Server
Ensure that all repositories are up-to-date before installing RabbitMQ server.
Execute the following commands:
sudo apt update sudo apt upgrade
Install the RabbitMQ server packages from the Ubuntu repository:
sudo apt install rabbitmq-server -y
After installation, start the RabbitMQ service and enable it to launch on system boot:
sudo systemctl start rabbitmq-server sudo systemctl enable rabbitmq-server
The RabbitMQ Server is now installed on Ubuntu 18.04.
Step 3 – Enable RabbitMQ Management Plugins
Enable the RabbitMQ management plugins, which provide a web interface to monitor and manage the RabbitMQ server, accessible on the default TCP port ‘15672’.
Enable the plugins with this command:
sudo rabbitmq-plugins enable rabbitmq_management
If successful, restart the RabbitMQ service:
sudo systemctl restart rabbitmq-server
The RabbitMQ Management plugins are now enabled.
Step 4 – Setup UFW Firewall
Enable the Ubuntu UFW firewall. Open the necessary ports for RabbitMQ:
Add the SSH service and enable the firewall:
sudo ufw allow ssh sudo ufw enable
Include new RabbitMQ TCP ports ‘5672,15672,4369,25672’:
sudo ufw allow 5672,15672,4369,25672/tcp
Check the UFW firewall ports:
sudo ufw status
The UFW firewall configuration is complete, and we are ready to set up the RabbitMQ Cluster.
Step 5 – Setup RabbitMQ Cluster
Ensure the ‘.erlang.cookie’ file is identical on all nodes for the RabbitMQ cluster. Copy it from ‘hakase-ubuntu01’ to ‘hakase-ubuntu02’ and ‘hakase-ubuntu03’:
scp /var/lib/rabbitmq/.erlang.cookie root@hakase-ubuntu02:/var/lib/rabbitmq/ scp /var/lib/rabbitmq/.erlang.cookie root@hakase-ubuntu03:/var/lib/rabbitmq/
Ensure no errors occur during the transfer.
Setup ‘hakase-ubuntu02’ and ‘hakase-ubuntu03’ to join the cluster at ‘hakase-ubuntu01’.
Note: Run these commands on ‘hakase-ubuntu02’ and ‘hakase-ubuntu03’:
Restart RabbitMQ and stop the app:
sudo systemctl restart rabbitmq-server sudo rabbitmqctl stop_app
Join the cluster and start the app:
sudo rabbitmqctl join_cluster rabbit@hakase-ubuntu01 sudo rabbitmqctl start_app
Verify the RabbitMQ cluster status:
sudo rabbitmqctl cluster_status
The RabbitMQ Cluster is now established with ‘hakase-ubuntu01’, ‘hakase-ubuntu02’, and ‘hakase-ubuntu03’ as members.
Step 6 – Setup New Administrator User
Create a new admin user and remove the default ‘guest’ user from ‘hakase-ubuntu01’. This user will be replicated across the cluster:
Create the ‘hakase’ user:
sudo rabbitmqctl add_user hakase aqwe123@
Assign administrative privileges to ‘hakase’:
sudo rabbitmqctl set_user_tags hakase administrator
Grant ‘hakase’ full permissions:
sudo rabbitmqctl set_permissions -p / hakase ".*" ".*" ".*"
Remove the default ‘guest’ user:
sudo rabbitmqctl delete_user guest
List available users:
sudo rabbitmqctl list_users
A new RabbitMQ administrative user ‘hakase’ is now created, and the ‘guest’ user has been removed.
Step 7 – RabbitMQ Setup Queue Mirroring
Queue contents default to a single node. To enhance availability, we configure the ‘HA policy’ for queue mirroring across nodes. If a master node fails, the oldest mirror automatically becomes the new master:
Mirror queues on all nodes with the ‘ha-all’ policy:
sudo rabbitmqctl set_policy ha-all ".*" '{"ha-mode":"all"}'
For queues prefixed with ‘two.’, mirror them on two nodes with the ‘ha-two’ policy:
sudo rabbitmqctl set_policy ha-two "^two\." '{"ha-mode":"exactly","ha-params":2,"ha-sync-mode":"automatic"}'
Mirror ‘nodes.’ queues on ‘hakase-ubuntu02’ and ‘hakase-ubuntu03’ with ‘ha-nodes’ policy:
sudo rabbitmqctl set_policy ha-nodes "^nodes\." '{"ha-mode":"nodes","ha-params":["rabbit@hakase-ubuntu02", "rabbit@hakase-ubuntu03"]}'
List all H.A. policies:
sudo rabbitmqctl list_policies
Remove a specific H.A. policy:
sudo rabbitmqctl clear_policy ha-two
Step 8 – Testing
Access your RabbitMQ server via a web browser at port ‘15672’:
Log in using ‘hakase’ and ‘aqwe123@’.
The RabbitMQ admin dashboard should appear, showing all nodes running smoothly.
Click on the ‘Admin’ tab, then ‘Users’ to view ‘hakase’.
In the ‘Policies’ section, see all RabbitMQ H.A. policies created.
The RabbitMQ Cluster on Ubuntu 18.04 servers is now installed and configured successfully.
Reference
FAQ
- Why do we need to edit the Hosts File?Editing the hosts file ensures that each server in the cluster can be recognized by its hostname, which is crucial for network communication within the cluster.
- What is the purpose of enabling RabbitMQ Management Plugins?The management plugins provide a web-based interface to monitor, manage, and configure RabbitMQ servers, making server management more efficient.
- What is the ‘.erlang.cookie’ file used for?The ‘.erlang.cookie’ file is a security key that ensures all servers in a RabbitMQ cluster are authorized to communicate with each other.
- Why is queue mirroring important in RabbitMQ?Queue mirroring enhances fault tolerance by replicating queues across multiple nodes, ensuring availability even if one node fails.