ModSecurity: Failed to load locate the unicode map file from: unicode.mapping 20127
Describe the bug
v3 of ModSecurity - pulling from master branch.
Running mod security on Nginx inside docker container using ModSecurity nginx connector and getting the following error when the container starts up:
“modsecurity_rules_file” directive Rules error. File: /etc/nginx/modsec/modsecurity.conf. Line: 192. Column: 17. Failed to load locate the unicode map file from: unicode.mapping 20127 Looking at: ‘unicode.mapping 20127’, ‘unicode.mapping 20127’, ‘/etc/nginx/modsec/unicode.mapping 20127’, ‘/etc/nginx/modsec/unicode.mapping 20127’. in /etc/nginx/nginx.conf:17
nginx.conf (/etc/nginx/nginx.conf)
load_module modules/ngx_http_modsecurity_module.so;
error_log /dev/stdout ${LOG_LEVEL};
events {
worker_connections 1024;
}
http {
access_log /dev/stdout;
server_tokens off;
server {
listen 443 ssl;
server_name waf;
ssl_certificate certs/cert.pem;
ssl_certificate_key certs/key.pem;
modsecurity on;
modsecurity_rules_file /etc/nginx/modsec/main.conf;
location / {
proxy_pass https://${LOCATION_ADDRESS};
modsecurity on;
}
error_page 403 404 405 500 501 502 503 504 /error/error.html;
location ^~ /error/ {
internal;
root /usr/share/nginx;
modsecurity off;
}
location = /amihealthy {
return 200;
}
}
}
Dockerfile
FROM nginx:1.15.3-alpine
RUN set -xe && \
apk --no-cache update && \
apk add --no-cache --virtual .build-deps \
gcc \
libc-dev \
make \
openssl-dev \
pcre-dev \
zlib-dev \
linux-headers \
curl \
gnupg \
libxslt-dev \
gd-dev \
perl-dev \
py-pip \
py-setuptools \
&& apk add --no-cache --virtual .libmodsecurity-deps \
pcre-dev \
libxml2-dev \
automake \
autoconf \
g++ \
flex \
bison \
yajl-dev \
# dependencies that should not be removed
&& apk add --no-cache \
libtool \
doxygen \
geoip \
geoip-dev \
yajl \
libstdc++ \
git \
sed \
python
# install aws cli
RUN set -xe && \
pip --no-cache-dir install --upgrade pip && \
pip --no-cache-dir install awscli
# install modsecurity
WORKDIR /opt/ModSecurity
RUN set -xe && \
git clone -b v3/master --single-branch https://github.com/SpiderLabs/ModSecurity . && \
git submodule init && \
git submodule update && \
./build.sh && \
./configure && make && make install
# install nginx connector
WORKDIR /opt
RUN git clone --depth 1 https://github.com/SpiderLabs/ModSecurity-nginx.git && \
wget http://nginx.org/download/nginx-$NGINX_VERSION.tar.gz && \
tar zxvf nginx-$NGINX_VERSION.tar.gz
WORKDIR /opt/nginx-$NGINX_VERSION
RUN set -xe && \
./configure --with-compat --add-dynamic-module=../ModSecurity-nginx && \
make modules && \
cp objs/ngx_http_modsecurity_module.so /etc/nginx/modules
# install owasp ruleset
WORKDIR /opt
RUN set -xe && \
git clone -b v3.0/master https://github.com/SpiderLabs/owasp-modsecurity-crs && \
mv owasp-modsecurity-crs/ /usr/local/
RUN set -xe && \
mkdir /etc/nginx/modsec && \
rm -fr /etc/nginx/nginx.conf
COPY conf/nginx/nginx.conf /etc/nginx/nginx.conf.template
COPY conf/modsec/ /etc/nginx/modsec/
COPY conf/owasp/ /usr/local/owasp-modsecurity-crs/
COPY error/ /usr/share/nginx/error/
COPY certs/ /etc/nginx/certs/
# remove unnecessary stuff
RUN set -xe && \
apk del .build-deps && \
apk del .libmodsecurity-deps && \
rm -fr ModSecurity && \
rm -fr ModSecurity-nginx && \
rm -fr nginx-$NGINX_VERSION.tar.gz && \
rm -fr nginx-$NGINX_VERSION && \
rm -fr /etc/nginx/conf.d
# execute stuff
COPY ./setupEnvAndStart.sh /etc/nginx/
RUN chmod +x /etc/nginx/setupEnvAndStart.sh
CMD ["sh", "-c", ". /etc/nginx/setupEnvAndStart.sh"]
setupEnvAndStart.sh
#! /bin/bash
config=<config>
for i in $(echo $config | sed "s/,/ /g")
do
export $i
done
envsubst < /etc/nginx/nginx.conf.template > /etc/nginx/nginx.conf
exec nginx -g 'daemon off;'
modsecurity.conf
Used the modsecurity.conf-recommended
Logs and dumps
[emerg] 1#1: “modsecurity_rules_file” directive Rules error. File: /etc/nginx/modsec/modsecurity.conf. Line: 192. Column: 17. Failed to load locate the unicode map file from: unicode.mapping 20127 Looking at: ‘unicode.mapping 20127’, ‘unicode.mapping 20127’, ‘/etc/nginx/modsec/unicode.mapping 20127’, ‘/etc/nginx/modsec/unicode.mapping 20127’. in /etc/nginx/nginx.conf:17
nginx: [emerg] “modsecurity_rules_file” directive Rules error. File: /etc/nginx/modsec/modsecurity.conf. Line: 192. Column: 17. Failed to load locate the unicode map file from: unicode.mapping 20127 Looking at: ‘unicode.mapping 20127’, ‘unicode.mapping 20127’, ‘/etc/nginx/modsec/unicode.mapping 20127’, ‘/etc/nginx/modsec/unicode.mapping 20127’. in /etc/nginx/nginx.conf:17
To Reproduce
Steps to reproduce the behavior:
docker build on the Dockerfile. Add config values in setupEnvAndStart.sh for variables listed in the nginx.conf above docker run on resulting image (exposing port 443)
Expected behavior
Nginx should start up without errors, instead it errors as above
Server (please complete the following information):
- ModSecurity version (and connector): ModSecurity v3.0.2 (but pulled from v3/master at build time) with nginx-connector v1.0.0
- WebServer: nginx-1.15.3
- OS (and distro): RHEL 7.5
Rule Set (please complete the following information):
v3.0/master https://github.com/SpiderLabs/owasp-modsecurity-crs
Additional context
This has previously worked - on 23/10/18 this was built with image size of 451.11 MB…today’s (25/10/18) build is 462.04 MB - no additional config was included from our end.
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 36 (8 by maintainers)
Commits related to this issue
- Test case skeleton for #1941 — committed to owasp-modsecurity/ModSecurity by deleted user 6 years ago
- Test case skeleton for #1941 — committed to owasp-modsecurity/ModSecurity by deleted user 6 years ago
- fix modsecurity missing unicode.mapping file - https://community.centminmod.com/threads/nginx-failes-to-start-with-mod-security.15944/#post-68293 - https://github.com/SpiderLabs/ModSecurity/issues/19... — committed to centminmod/centminmod by centminmod 6 years ago
- Create unicode.mapping see [https://github.com/SpiderLabs/ModSecurity/issues/1941](https://github.com/SpiderLabs/ModSecurity/issues/1941) — committed to andrewnk/docker-alpine-nginx-modsec by andrewnk 6 years ago
@victorhora
First this Error
nginx: [emerg] “modsecurity_rules_file” directive Rules error. File: /etc/nginx/modsec/modsecurity.conf. Line: 234. Column: 17. Failed to load locate the unicode map file from: unicode.mapping 20127 Looking at: ‘unicode.mapping 20127’, ‘unicode.mapping 20127’, ‘/etc/nginx/modsec/unicode.mapping 20127’, ‘/etc/nginx/modsec/unicode.mapping 20127’. in /etc/nginx/nginx.conf:28 nginx: configuration file /etc/nginx/nginx.conf test failed
Then $ sudo cp ~/ModSecurity/unicode.mapping /etc/nginx/modsec/
$ sudo nginx -t Segmentation fault
$systemctl status nginx.service ● nginx.service - nginx - high performance web server Loaded: loaded (/etc/systemd/system/nginx.service; enabled; vendor preset: enabled) Active: failed (Result: core-dump) since Mon 2018-10-29 06:13:10 UTC; 7s ago
Commenting out #SecUnicodeMapFile unicode.mapping 20127
Makes Everything Normal.
ls /etc/nginx/conf.d/modsecurity.conf /etc/nginx/conf.d/unicode.mapping
/etc/nginx/conf.d/modsecurity.conf /etc/nginx/conf.d/unicode.mapping
This works ( I mean having modsecurity.conf and unicode.mapping in the same folder)
Started this using this URL, nginx 1.16.1 using ppa:nginx/stable
Encountered same problem, just did
cp /opt/ModSecurity/unicode.mapping /etc/nginx/modsec/and everything worked. Just need to document this now, it seems!Fixed at: 18cdffdbca75e6b9f790f6df2807a32cd805c0a0
If it’s worth anything. I fixed the problem by copying the unicode.mapping file to the nginx modsec folder.
sudo cp path/to/cloned/modsec/repo/unicode.mapping /etc/nginx/modsec/unicode.mappingNov 21 23:22:49 vpbx.voipcom.gr nginx[19879]: nginx: [emerg] “modsecurity_rules_file” directive Rules error. File: /etc/nginx/modsec/modsecurity.conf. Line: 236. Column: 17. Failed to open the unicode map file from Nov 21 23:22:49 vpbx.voipcom.gr systemd[1]: nginx.service: control process exited, code=exited status=1 Nov 21 23:22:49 vpbx.voipcom.gr systemd[1]: Failed to start nginx - high performance web server.
I copied unicode.mapping in the same folder of modsecurity.conf and I change the next line SecUnicodeMapFile unicode.mapping 20127 to ==>> SecUnicodeMapFile ./unicode.mapping 20127
nginx started
I hope this help someone
System: Debian 8
Thanks a lot to Developers/Maintainers @zimmerle @victorhora , SecUnicodeMapFile unicode.mapping file is no-longer creating errors atleast in my setup
Ubuntu cosmic minimal Nginx 1.15.5 Modsec v3/master
kudos
Added as of https://github.com/SpiderLabs/ModSecurity/commit/662fe63a47d012b89175ea7da5be1a8f7e76014c.
The error message “Failed to load locate the unicode map file” should go away as long as the file is on the same directory as the configuration file. Alternatively, the user can also change / disable the functionality on the modsecurity.conf-recommended.
Working on it 😃
This save my problem. Thanks
see above shubham-panwar said
same issue with v3/master: https://github.com/SpiderLabs/ModSecurity/commit/662fe63a47d012b89175ea7da5be1a8f7e76014c
same issue, and Even If the unicode.mapping file is placed inside the configuration folder, still the Nginx can’t be started.
Nginx does detect the file unicode.mapping , but can’t make use of it .