Installing ModSecurity 3 with Nginx on Ubuntu 22.04

Ensuring the security of websites and web applications can be challenging for system administrators. Thankfully, numerous open-source tools are available to help secure your site from DDoS attacks and other malicious activities. One such tool is ModSecurity, a free and open-source Web Application Firewall (WAF) that provides robust protection against various attacks, including cross-site scripting (XSS), SQL injection, and session hijacking.

This guide will walk you through the process of installing ModSecurity with Nginx on Ubuntu 22.04.

Prerequisites

  • An Ubuntu 22.04 server.
  • A root password configured on the server.

Getting Started

To begin, update and upgrade your software packages to the latest versions by executing the following commands:

        apt update -y
        apt upgrade -y

After updating, install the necessary packages with this command:

        apt install g++ flex bison curl apache2-dev doxygen libyajl-dev ssdeep liblua5.2-dev libgeoip-dev libtool \
                dh-autoreconf libcurl4-gnutls-dev libxml2 libpcre++-dev libxml2-dev git liblmdb-dev libpkgconf3 \
                lmdb-doc pkgconf zlib1g-dev libssl-dev -y

Proceed to the next step after installation.

Install ModSecurity on Ubuntu 22.04

ModSecurity isn’t included in Ubuntu’s default repositories, necessitating manual compilation from source.

Download the latest ModSecurity version:

        wget https://github.com/SpiderLabs/ModSecurity/releases/download/v3.0.8/modsecurity-v3.0.8.tar.gz

Extract the downloaded file:

tar -xvzf modsecurity-v3.0.8.tar.gz

Navigate to the extracted directory and configure it:

        cd modsecurity-v3.0.8
        ./build.sh
        ./configure

Build and install ModSecurity:

        make
        make install

Install Nginx with ModSecurity 3 Support

To enable ModSecurity in Nginx, download and configure the ModSecurity-nginx connector:

        cd ~
        git clone https://github.com/SpiderLabs/ModSecurity-nginx.git

Download the Nginx source:

wget https://nginx.org/download/nginx-1.20.2.tar.gz

Extract the Nginx source:

tar xzf nginx-1.20.2.tar.gz

Create a dedicated user for Nginx:

useradd -r -M -s /sbin/nologin -d /usr/local/nginx nginx

Configure Nginx with ModSecurity support:

        cd nginx-1.20.2
        ./configure --user=nginx --group=nginx --with-pcre-jit --with-debug --with-compat --with-http_ssl_module \
                    --with-http_realip_module --add-dynamic-module=/root/ModSecurity-nginx \
                    --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log

Install Nginx:

        make
        make modules
        make install

Create a symbolic link for easy access:

ln -s /usr/local/nginx/sbin/nginx /usr/local/sbin/

Verify the installation:

nginx -V

Output will display:

        nginx version: nginx/1.20.2
        built by gcc 11.2.0 (Ubuntu 11.2.0-19ubuntu1) 
        built with OpenSSL 3.0.2 15 Mar 2022
        TLS SNI support enabled
        configure arguments: --user=nginx --group=nginx --with-pcre-jit --with-debug --with-compat \
                             --with-http_ssl_module --with-http_realip_module \
                             --add-dynamic-module=/root/ModSecurity-nginx \
                             --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log

Now proceed to configure Nginx with ModSecurity support.

Configure Nginx with ModSecurity

Copy the recommended ModSecurity configuration files:

        cp ~/modsecurity-v3.0.8/modsecurity.conf-recommended /usr/local/nginx/conf/modsecurity.conf
        cp ~/modsecurity-v3.0.8/unicode.mapping /usr/local/nginx/conf/

Backup your Nginx configuration file:

cp /usr/local/nginx/conf/nginx.conf{,.bak}

Edit the Nginx configuration file:

nano /usr/local/nginx/conf/nginx.conf

Replace existing content with:

        load_module modules/ngx_http_modsecurity_module.so;
        user  nginx;
        worker_processes  1;
        pid        /run/nginx.pid;
        events {
            worker_connections  1024;
        }
        http {
            include       mime.types;
            default_type  application/octet-stream;
            sendfile        on;
            keepalive_timeout  65;
            server {
                listen       80;
                server_name  nginx.example.com;
                modsecurity  on;
                modsecurity_rules_file  /usr/local/nginx/conf/modsecurity.conf;
                access_log  /var/log/nginx/access_example.log;
                error_log  /var/log/nginx/error_example.log;
                location / {
                    root   html;
                    index  index.html index.htm;
                }
                error_page   500 502 503 504  /50x.html;
                location = /50x.html {
                    root   html;
                }
            }
        }

Save the changes and exit the editor. Enable ModSecurity with:

sed -i 's/SecRuleEngine DetectionOnly/SecRuleEngine On/' /usr/local/nginx/conf/modsecurity.conf

Continue to the next step.

Install ModSecurity Core Rule Set

The OWASP ModSecurity Core Rule Set helps detect and mitigate a wide range of attacks effectively.

Download the rule set:

        cd
        git clone https://github.com/SpiderLabs/owasp-modsecurity-crs.git /usr/local/nginx/conf/owasp-crs

Rename the CRS setup file:

cp /usr/local/nginx/conf/owasp-crs/crs-setup.conf{.example,}

Define the rules:

        echo -e "Include owasp-crs/crs-setup.conf\nInclude owasp-crs/rules/*.conf" >> /usr/local/nginx/conf/modsecurity.conf

Check Nginx configuration:

nginx -t

If successful, you will see:

        nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
        nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful

Create a Systemd Service File for Nginx

Create a systemd service file to manage the Nginx service:

nano /etc/systemd/system/nginx.service

Insert the following configuration:

        [Unit]
        Description=A high performance web server and a reverse proxy server
        Documentation=man:nginx(8)
        After=network.target nss-lookup.target

        [Service]
        Type=forking
        PIDFile=/run/nginx.pid
        ExecStartPre=/usr/local/nginx/sbin/nginx -t -q -g 'daemon on; master_process on;'
        ExecStart=/usr/local/nginx/sbin/nginx -g 'daemon on; master_process on;'
        ExecReload=/usr/local/nginx/sbin/nginx -g 'daemon on; master_process on;' -s reload
        ExecStop=-/sbin/start-stop-daemon --quiet --stop --retry QUIT/5 --pidfile /run/nginx.pid
        TimeoutStopSec=5
        KillMode=mixed

        [Install]
        WantedBy=multi-user.target

Save and close the file. Reload the systemd daemon to apply changes:

systemctl daemon-reload

Start and enable Nginx with:

        systemctl start nginx
        systemctl enable nginx

Check the status of Nginx with:

systemctl status nginx

You’ll see output like:

        ? nginx.service - A high performance web server and a reverse proxy server
             Loaded: loaded (/etc/systemd/system/nginx.service; disabled; vendor preset: enabled)
             Active: active (running) since Tue 2022-10-11 15:40:39 UTC; 6s ago
               Docs: man:nginx(8)
            Process: 68438 ExecStartPre=/usr/local/nginx/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
            Process: 68439 ExecStart=/usr/local/nginx/sbin/nginx -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
           Main PID: 68440 (nginx)
              Tasks: 2 (limit: 4579)
             Memory: 20.0M
                CPU: 293ms
             CGroup: /system.slice/nginx.service
                     ??68440 "nginx: master process /usr/local/nginx/sbin/nginx -g daemon on; master_process on;"
                     ??68441 "nginx: worker process" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" >
        
        Oct 11 15:40:38 ubuntu2204 systemd[1]: Starting A high performance web server and a reverse proxy server...
        Oct 11 15:40:39 ubuntu2204 systemd[1]: Started A high performance web server and a reverse proxy server.

Verify ModSecurity

With everything set up, it’s time to verify that ModSecurity is functioning correctly. Test ModSecurity against command injection with:

curl localhost?doc=/bin/ls

The expected output should be a “403 Forbidden” message:

        <html>
        <head><title>403 Forbidden</title></head>
        <body>
        <center><h1>403 Forbidden</h1></center>
        <hr><center>nginx/1.20.2</center>
        </body>
        </html>

Verify ModSecurity logs:

tail /var/log/modsec_audit.log

You’ll see log details similar to this:

        ModSecurity: Warning. Matched "Operator `PmFromFile' with parameter `unix-shell.data' against variable `ARGS:doc' (Value: `/bin/ls' ) [file "/usr/local/nginx/conf/owasp-crs/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf"] [line "496"] [id "932160"] [rev ""] [msg "Remote Command Execution: Unix Shell Code Found"] [data "Matched Data: bin/ls found within ARGS:doc: /bin/ls"] [severity "2"] [ver "OWASP_CRS/3.2.0"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-shell"] [tag "platform-unix"] [tag "attack-rce"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "OWASP_CRS/WEB_ATTACK/COMMAND_INJECTION"] [tag "WASCTC/WASC-31"] [tag "OWASP_TOP_10/A1"] [tag "PCI/6.5.2"] [hostname "127.0.0.1"] [uri "/"] [unique_id "166550286018.572845"] [ref "o1,6v10,7t:urlDecodeUni,t:cmdLine,t:normalizePath,t:lowercase"]
        ModSecurity: Access denied with code 403 (phase 2). Matched "Operator `Ge' with parameter `5' against variable `TX:ANOMALY_SCORE' (Value: `5' ) [file "/usr/local/nginx/conf/owasp-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "80"] [id "949110"] [rev ""] [msg "Inbound Anomaly Score Exceeded (Total Score: 5)"] [data ""] [severity "2"] [ver "OWASP_CRS/3.2.0"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-generic"] [hostname "127.0.0.1"] [uri "/"] [unique_id "166550286018.572845"] [ref ""]
        
        ---IcTYGSZl---I--
        
        ---IcTYGSZl---J--
        
        ---IcTYGSZl---Z--

Conclusion

Congratulations! You have successfully installed ModSecurity with Nginx on Ubuntu 22.04. This setup enables strong defense mechanisms in your production environment against various types of attacks. Feel free to reach out if you have any questions or need further assistance.

FAQ

Q: What is ModSecurity?

A: ModSecurity is a web application firewall (WAF) that provides security by monitoring traffic and intercepting attacks against your web applications.

Q: Can ModSecurity protect against DDoS attacks?

A: While ModSecurity can help mitigate certain types of attack vectors that could be exploited in a DDoS attack, it’s primarily designed to offer protection against known vulnerabilities like XSS, SQL injection, and more.

Q: Will installing ModSecurity affect my website’s performance?

A: Any security layer can have a performance overhead, but ModSecurity is generally efficient. You can tune its settings to balance between security and performance.

Q: Can I customize the rules in ModSecurity?

A: Yes, ModSecurity supports custom rule sets, allowing you to tailor its security measures to the specific needs of your website or application.

Q: How do I verify if ModSecurity is successfully installed and working?

A: You can perform test attacks such as command injection to see if ModSecurity intercepts them, as described in the guide, and examine the logs for evidence of blocked attacks.