Step-by-Step Guide: Installing OpenSearch on Ubuntu 22.04 Using Docker

by

OpenSearch is a community-driven project by Amazon that serves as a fork of Elasticsearch and Kibana. This open-source search engine and analytics suite are packed with features and innovative functionalities. OpenSearch consists of two primary components: OpenSearch, a fork of Elasticsearch, and OpenSearch Dashboards, a fork of Kibana. These components offer enterprise security, alerting, machine learning, SQL, index state management, and more.

Being 100% open-source and licensed under Apache 2.0, OpenSearch provides the capabilities to ingest, secure, search, aggregate, view, and analyze data for various use cases such as log analytics, application search, and enterprise search.

In this tutorial, you’ll learn how to install and configure OpenSearch and OpenSearch Dashboards using Docker on an Ubuntu 22.04 server. The guide will cover deploying an OpenSearch cluster with multiple containers and a single OpenSearch Dashboard using Docker Compose. Custom TLS certificates will be used to secure the deployment with authentication and authorization enabled.

A fresh Ubuntu server setup is necessary for this tutorial, so it includes steps to install Docker engine and Docker Compose on an Ubuntu 22.04 system.

Prerequisites

Ensure the following requirements are met before proceeding:

  • An Ubuntu 22.04 server with at least 4-8GB of RAM. The example references a server with the hostname ” and IP address ”.
  • A non-root user with sudo/root administrator privileges.

After fulfilling these requirements, you can start the OpenSearch installation.

Setting Up System

In this initial step, prepare your Ubuntu system for the OpenSearch deployment. You’ll disable SWAP and paging, then increase the max memory map through the ‘/etc/sysctl.conf’ file.

Execute the commands below to permanently disable swap and for the current session:

sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
sudo swapoff -a

Verify swap status by executing:

free -m

Expected Output:

disable swap

Next, modify the ‘/etc/sysctl.conf’ file to increase max memory maps:

sudo echo "vm.max_map_count=262144" >> /etc/sysctl.conf
sudo sysctl -p

Verify by running:

cat /proc/sys/vm/max_map_count

Expected Output:

setup sysctl

Installing Docker CE and Docker Compose

You can deploy OpenSearch either by traditional installation or a containerized environment. This guide focuses on using Docker and Docker Compose for deployment. Begin by installing Docker engine and Docker Compose from the official Docker repository.

Install Basic Dependencies

Start by installing essential dependencies. Confirm the installation process when prompted:

sudo apt install ca-certificates curl gnupg lsb-release

Expected Output:

install basic dependencies

Add Docker Repository

Add the Docker GPG key and repository:

sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

Expected Output:

setup repo

Install Docker and Docker Compose

Update your package index:

sudo apt update

Expected Output:

update repo

Install Docker and Docker Compose:

sudo apt install docker-ce docker-ce-cli containerd.io docker-compose-plugin

Expected Output:

install docker and docker compose

The Docker service should start automatically. Verify using:

sudo systemctl is-enabled docker
sudo systemctl status docker

Configure Docker User Permissions

Add your user to the ‘docker’ group to run Docker commands without sudo:

sudo usermod -aG docker alice

Log in as your user and verify Docker is functioning:

su - alice
docker run hello-world

Expected Output:

verify docker

Downloading OpenSearch Docker Images

Download the OpenSearch and OpenSearch Dashboards images from DockerHub:

docker pull opensearchproject/opensearch:latest
docker pull opensearchproject/opensearch-dashboards:latest

Expected Output:

download opensearch image

download opensearch dashboards

Setup Project Directory

Log in as the user ‘alice’ and create the project directory:

su - alice
mkdir -p ~/opensearch-project/certs; cd ~/opensearch-project

Create necessary configuration and YAML files:

touch docker-compose.yml opensearch.yml opensearch_dashboards.yml internal_users.yml

setup project directory

Generating SSL/TLS Certificates

This step will guide you through generating TLS certificates to secure the OpenSearch deployment:

  • Root CA certificates for signing other certificates.
  • Admin certificates for administrative rights.
  • OpenSearch Dashboards certificates for HTTPS connections.
  • Node and client certificates for OpenSearch nodes.

Create directories for storing certificates:

mkdir -p certs/{ca,os-dashboards}

Set environment variable for DN:

export MYDN="/C=CA/ST=ONTARIO/L=TORONTO/O=HWDOMAIN"

Generate Certificates

Generate CA, admin, OpenSearch Dashboards, and node certificates using OpenSSL commands as detailed in the tutorial.

setup certs

Setting Up User

Generate password hashes using the OpenSearch container:

docker run -it --rm opensearchproject/opensearch sh -c "/usr/share/opensearch/plugins/opensearch-security/tools/hash.sh"

Edit internal_users.yml to configure users:

nano internal_users.yml

Add user credentials and roles:

internal_users.yml

Setup docker-compose.yml Script

Configure Docker Compose script for deploying OpenSearch and OpenSearch Dashboards:

nano docker-compose.yml

Add services for OpenSearch and OpenSearch Dashboards as detailed in the tutorial.

Set up OpenSearch and OpenSearch Dashboards

Create configurations for opensearch.yml and opensearch_dashboards.yml. Ensure all necessary files are in the project directory.

Verify the setup structure using:

sudo apt install tree
tree .

Deploying OpenSearch Cluster and OpenSearch Dashboards

In the project directory, start your deployment:

docker compose up -d

Verify the running services:

docker compose ps

Apply new user configuration:

docker compose exec os01 bash -c "chmod +x plugins/opensearch-security/tools/securityadmin.sh && bash plugins/opensearch-security/tools/securityadmin.sh -cd config/opensearch-security -icl -nhnv -cacert config/certificates/ca/ca.pem -cert config/certificates/ca/admin.pem -key config/certificates/ca/admin.key -h localhost"

Verify connections using curl commands:

curl https://192.168.5.100:9200 -u admin:password -k
curl https://192.168.5.100:9200 -u kibanaserver:password -k

Accessing OpenSearch Dashboards

Use a browser to access OpenSearch Dashboards at:

https://192.168.5.100:5601/

opensearch login

Verify connection via Dev Tools in OpenSearch Dashboards.

Conclusion

This tutorial guided you through installing and configuring OpenSearch and OpenSearch Dashboards on an Ubuntu 22.04 server using Docker and Docker Compose. You set up a secure, multi-node OpenSearch cluster and connected OpenSearch Dashboards for visualization. The tools offer a powerful platform for analytics and search, providing a foundation for further exploration and integration.

FAQ

  • What is OpenSearch? OpenSearch is an open-source search engine and analytics suite, originally forked from Elasticsearch and Kibana by Amazon.
  • Why use Docker for OpenSearch? Docker provides containerization, making applications portable, scalable, and easier to manage while allowing quick deployment.
  • How secure is OpenSearch? OpenSearch is highly secure with enterprise-grade features like TLS encryption, authentication, and authorization.
  • Can I contribute to OpenSearch? Yes, OpenSearch is community-driven, meaning contributions are welcome from developers and users worldwide.