coreruleset: 10GB ModSecurity File - Tons of 920100 Warnings. Need Help Understanding Rule Violation

@peppies @airween @theseion @dune73

I suspected it was an issue with one of the PCRE dependencies, so in order to reduce variables, I decided to reinstall without using the ondrej repository. Now I have everything working with Nginx 1.23.2 ModSecurity 3.0.8 ModSecurity-nginx 1.0.3

Also I didn’t have to touch my config files under /etc/nginx/modsec (including coreruleset-3.3.4 left alone).

How I did it:

1. Remove ondrej repository sudo add-apt-repository --remove ppa:ondrej/nginx-mainline sudo apt update sudo apt install ppa-purge sudo ppa-purge ppa:ondrej/nginx-mainline sudo apt clean sudo apt autoremove sudo apt update

2. Remove nginx installed from ondrej repository sudo apt remove nginx sudo apt remove nginx-common

3. At this point there were some packages which showed error removing as still-in-use dependencies. List problem packages with: sudo apt list libpcre* --installed

For example my list included … libpcre16-3/now 2:8.44-2+ubuntu20.04.1+deb.sury.org+1 amd64 [installed,local] libpcre3-dev/now 2:8.44-2+ubuntu20.04.1+deb.sury.org+1 amd64 [installed,local] libpcre32-3/now 2:8.44-2+ubuntu20.04.1+deb.sury.org+1 amd64 [installed,local] libpcre3/now 2:8.44-2+ubuntu20.04.1+deb.sury.org+1 amd64 [installed,local] libpcrecpp0v5/now 2:8.44-2+ubuntu20.04.1+deb.sury.org+1 amd64 [installed,local]

Then for each, show available version from ubuntu repository, example:

sudo apt list libpcre16-3 -a Listing… Done libpcre16-3/jammy-updates,jammy-security,now 2:8.39-13ubuntu0.22.04.1 amd64 libpcre16-3/jammy 2:8.39-13build5 amd64

Downgrade to version from jammy-updates repository

sudo apt install libpcre16-3=2:8.39-13ubuntu0.22.04.1

Repeat for all (5 packages downgraded in my case)

1. Install Nginx from official Nginx mainline repository (from http://nginx.org/en/linux_packages.html#Ubuntu)

sudo apt install curl gnupg2 ca-certificates lsb-release ubuntu-keyring

curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor
| sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg >/dev/null

echo “deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg]
http://nginx.org/packages/mainline/ubuntu lsb_release -cs nginx”
| sudo tee /etc/apt/sources.list.d/nginx.list

echo -e “Package: *\nPin: origin nginx.org\nPin: release o=nginx\nPin-Priority: 900\n”
| sudo tee /etc/apt/preferences.d/99nginx

sudo apt update sudo apt install nginx

---

1. Install ModSecurity and ModSecurity-nginx both from latest sources

apt-get install -y apt-utils autoconf automake build-essential git libcurl4-openssl-dev libgeoip-dev liblmdb-dev libpcre+±dev libtool libxml2-dev libyajl-dev pkgconf wget zlib1g-dev

sudo rm -rf /usr/local/modsecurity sudo rm -rf /usr/local/src/* cd /usr/local/src

git clone --depth 1 -b v3/master --single-branch https://github.com/SpiderLabs/ModSecurity

cd ModSecurity git submodule init git submodule update ./build.sh ./configure make sudo make install cd …

git clone --depth 1 https://github.com/SpiderLabs/ModSecurity-nginx.git

wget http://nginx.org/download/nginx-1.23.2.tar.gz tar zxvf nginx-1.23.2.tar.gz

cd nginx-1.23.2 ./configure --with-compat --add-dynamic-module=…/ModSecurity-nginx make modules sudo cp objs/ngx_http_modsecurity_module.so /etc/nginx/modules cd …

Ha! Found it!

    std::vector<Utils::SMatchCapture> captures;
    if (re->hasError()) {
        ms_dbg_a(transaction, 3, "Error with regular expression: \"" + re->pattern + "\"");
        return false;
    }

It’s a new check int ModSecurity v3.0.7, introduced in https://github.com/SpiderLabs/ModSecurity/commit/1aa7616c189cd12226c74755b35538f4937a135e.

@theseion

I completely uninstalled and reinstalled everything using the guide here: https://www.linuxcapable.com/how-to-install-nginx-with-modsecurity-3-on-ubuntu-22-04-lts/?mtm_campaign=reddit#Install_libmodsecurity3_for_ModSecurity

The only exception is that I followed your recommendation to install v3.0.6 of ModSecurity using:

sudo git clone --depth 1 -b v3.0.6 https://github.com/SpiderLabs/ModSecurity /usr/local/src/ModSecurity/

This did not resolve the issue, I am still seeing the same warnings with rule 920100.

I was thinking that it might fail to compile on one of their systems. In any case, I think the next step should be to try ModSecurity v3.0.6, before the introduction of that check and PCRE2 support.

We had two other people complain of similar, fundamental problems (and really weird problems) with ModSecurity 3 the other month: #2752. Rule 920100 was also in question, there. It all sounds very similar (although I haven’t had time to properly look at this new issue or compare them, yet.)

It really does sound like there’s a bad build or a bad release of something out there…

I took a peek at the logs. Error with regular expression looks extremely suspicious but it doesn’t seem to come from ModSecurity. Could it be comeing from PCRE?