coreruleset: False positive when charset=utf-8\x0d\x0a in content-type header

Are you going to open an issue with ModSecurity?

I try to find the bug and send a PR to fix it. If do not have time, I just open an issue.

It definitely looks like an illegal request to me. \r\n is the separation token for headers, so if it show up in the value then there are either more than there should be or they have been badly encoded. Do you have the ability to inspect the HTTP traffic, e.g. with tcpdump or Wireshark?

Just FYI: the sent patch above for libmodsecurity3 has been merged.

Thank you everyone for finding solution to this. Really appreciate

Thank you for your patience.

Looks like libmodsecurity3 handles buggy the multipart content types (too?), because it keeps the trailing white space chars, namely \r and \n in our case.

What can you do now?

I’ve found a workaround, this patch worked for me:

diff --git a/rules/REQUEST-922-MULTIPART-ATTACK.conf b/rules/REQUEST-922-MULTIPART-ATTACK.conf
index 1bf5a03..7c2fd36 100644
--- a/rules/REQUEST-922-MULTIPART-ATTACK.conf
+++ b/rules/REQUEST-922-MULTIPART-ATTACK.conf
@@ -53,7 +53,7 @@ SecRule MULTIPART_PART_HEADERS "@rx ^content-type\s*+:\s*+(.*)$" \
     phase:2,\
     block,\
     capture,\
-    t:none,t:lowercase,\
+    t:none,t:lowercase,t:removeWhiteSpace,\
     msg:'Illegal MIME Multipart Header content-type: charset parameter',\
     logdata:'Matched Data: %{TX.1} found within Content-Type multipart form',\
     tag:'application-multi',\
@@ -67,7 +67,7 @@ SecRule MULTIPART_PART_HEADERS "@rx ^content-type\s*+:\s*+(.*)$" \
     severity:'CRITICAL',\
     chain"
     SecRule TX:1 "!@rx ^(?:(?:\*|[^\"(),\/:;<=>?![\x5c\]{}]+)\/(?:\*|[^\"(),\/:;<=>?![\x5c\]{}]+))(?:\s*+;\s*+(?:(?:charset\s*+=\s*+(?:\"?(?:iso-8859-15?|windows-1252|utf-8)\b\"?))|(?:(?:c(?:h(?:a(?:r(?:s(?:e[^t\"(),\/:;<=>?![\x5c\]{}]|[^e\"(),/:;<=>?![\x5c\]{}])|[^s\"(),/:;<=>?![\x5c\]{}])|[^r\"(),/:;<=>?![\x5c\]{}])|[^a\"(),/:;<=>?![\x5c\]{}])|[^h\"(),/:;<=>?![\x5c\]{}])|[^c\"(),/:;<=>?![\x5c\]{}])[^\"(),/:;<=>?![\x5c\]{}]*(?:)\s*+=\s*+[^(),/:;<=>?![\x5c\]{}]+)|;?))*(?:\s*+,\s*+(?:(?:\*|[^\"(),\/:;<=>?![\x5c\]{}]+)\/(?:\*|[^\"(),\/:;<=>?![\x5c\]{}]+))(?:\s*+;\s*+(?:(?:charset\s*+=\s*+(?:\"?(?:iso-8859-15?|windows-1252|utf-8)\b\"?))|(?:(?:c(?:h(?:a(?:r(?:s(?:e[^t\"(),\/:;<=>?![\x5c\]{}]|[^e\"(),/:;<=>?![\x5c\]{}])|[^s\"(),/:;<=>?![\x5c\]{}])|[^r\"(),/:;<=>?![\x5c\]{}])|[^a\"(),/:;<=>?![\x5c\]{}])|[^h\"(),/:;<=>?![\x5c\]{}])|[^c\"(),/:;<=>?![\x5c\]{}])[^\"(),/:;<=>?![\x5c\]{}]*(?:)\s*+=\s*+[^(),/:;<=>?![\x5c\]{}]+)|;?))*)*$" \
-        "t:lowercase,\
+        "t:lowercase,t:removeWhiteSpace,\
         setvar:'tx.anomaly_score_pl1=+%{tx.critical_anomaly_score}'"
 
 # Content-Transfer-Encoding was deprecated by rfc7578 in 2015 and should not be used (see: https://www.rfc-editor.org/rfc/rfc7578#section-4.7)

so added the t:removeWhiteSpace transformation solved the problem, and rule 922110 didn’t trigger.

(ping @fzipi as author of the rule: what do you think? Does it make sense to add this permanently?)

If you want to avoid the unwanted other REQUEST_BODY matches, you can add something similar:

SecRule REQUEST_HEADERS:Content-Type "@beginsWith multipart/form-data" \
    "id:1000001,\
    phase:1,\
    pass,\
    t:none,t:lowercase,\
    nolog,\
    ctl:ruleRemoveTargetByTag=OWASP_CRS;REQUEST_BODY"

@mikegoatly - thanks.

I was able to reproduce the issue on Nginx + libmodsecurty3 3.0.8 and CRS 3.3.4.

2023/01/26 22:15:12 [info] 121615#121615: *1 ModSecurity: Warning. Matched "Operator `Rx' with parameter `^(?:(?:\*|[^\"(),\/:;<=>?![\x5c\]{}]+)\/(?:\*|[^\"(),\/:;<=>?![\x5c\]{}]+))(?:\s*+;\s*+(?:(?:charset\s*+=\s*+(?:\"?(?:iso-8859-15?|windows-1252|utf-8)\b\"?))|(?:(?:c(?:h(?:a(?:r(?:s(?:e[^t\"(),\/:;<=> (714 characters omitted)' against variable `TX:1' (Value: `text/plain; charset=utf-8\x0d\x0a' ) [file "/home/airween/src/coreruleset/rules/REQUEST-922-MULTIPART-ATTACK.conf"] [line "51"] [id "922110"] [rev ""] [msg "Illegal MIME Multipart Header content-type: charset parameter"] [data "Matched Data: text/plain; charset=utf-8\x0d\x0a found within Content-Type multipart form"] [severity "2"] [ver "OWASP_CRS/3.3.4"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "OWASP_CRS"] [tag "capec/272/220"] [tag "paranoia-level/1"] [hostname "::1"] [uri "/dump.php"] [unique_id "167476771293.913423"] [ref "o0,41o14,27v356,41t:lowercaset:lowercase"], client: ::1, server: _, request: "POST /dump.php HTTP/1.1", host: "localhost:8080"

Let me investigate the reasons, why happens this (and why does not with Apache).

ingress-nginx should be a container image, right? If you can get hold of the image, or the definition, you should be able to find out.

@mikegoatly, I tried to reproduce your issue, also with your suggested way (REST client for VSCode - thanks for the tip), but no luck.

Ok, this request fails whenever I throw it against an API fronted by ModSecurity:

now I made a small Python script which sends the raw format of your request, you can see that here. First, please check that, and let me know if there is something not well.

Then could you try it against your application? (I see you use HTTPS, but actually in my dev env I do not have any HTTPS, and it’s more convenient to try the raw requests like this without HTTPS).

This is what I see when I run the script:

SEND REQUEST:
=============

POST /dump.php HTTP/1.1
Host: localhost
Accept: */*
User-Agent: PyTcpClient v0.1
Content-Type: multipart/form-data; boundary="----x"
Content-Length: 253

------x
Content-Type: text/plain; charset=utf-8
Content-Disposition: form-data; name=title

Test
------x
Content-Disposition: form-data; name=file; filename="New Text Document.txt"; filename*=utf-8''New%20Text%20Document.txt

Some sample text

------x--

====

RECEIVED RESPONSE:
==================

HTTP/1.1 400 Bad Request
Date: Wed, 25 Jan 2023 21:24:43 GMT
Server: Apache/2.4.54 (Debian)
Content-Length: 301
Connection: close
Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>400 Bad Request</title>
</head><body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.<br />
</p>
<hr>
<address>Apache/2.4.54 (Debian) Server at localhost Port 80</address>
</body></html>

and this is what I see in my log:

[client 127.0.0.1:34804] [client 127.0.0.1] ModSecurity: Multipart parsing error: Multipart: Invalid Content-Disposition header (-11): form-data; name=file; filename="New Text Document.txt"; filename*=utf-8''New%20Text%20Document.txt. [hostname "localhost"] [uri "/dump.php"] [unique_id "Y9GeG__D42q-U0HlyXxqpAAAAAA"]
[client 127.0.0.1:34804] [client 127.0.0.1] ModSecurity: Access denied with code 400 (phase 2). Match of "eq 0" against "REQBODY_ERROR" required. [file "/etc/modsecurity/modsecurity.conf"] [line "75"] [id "200002"] [msg "Failed to parse request body."] [data "Multipart parsing error: Multipart: Invalid Content-Disposition header (-11): form-data; name=file; filename=\\x22New Text Document.txt\\x22; filename*=utf-8''New%20Text%20Document.txt."] [severity "CRITICAL"] [hostname "localhost"] [uri "/dump.php"] [unique_id "Y9GeG__D42q-U0HlyXxqpAAAAAA"]

or with Nginx+libmodsecurity3:

[client 127.0.0.1] ModSecurity: Access denied with code 400 (phase 2). Matched "Operator `Eq' with parameter `0' against variable `MULTIPART_STRICT_ERROR' (Value: `1' ) [file "/etc/nginx/modsecurity.conf"] [line "65"] [id "200003"] [rev ""] [msg "Multipart request body failed strict validation: \\x0aPE 0, \\x0aBQ 1, \\x0aBW 0, \\x0aDB 0, \\x0aDA 0, \\x0aHF 0, \\x0aLF 1, \\x0aSM 0, \\x0aIQ 0, \\x0aIP 0, \\x0aIH 0, \\x0aFL "]

So as you can see there is only one rule triggered in phase 2 with id 200002 (with both implementations):

Multipart parsing error: Multipart: Invalid Content-Disposition header (-11): form-data; name=file; filename="New Text Document.txt"; filename*=utf-8''New%20Text%20Document.txt.

and

Access denied with code 400 (phase 2). Match of "eq 0" against "REQBODY_ERROR" required.

I’m using CRS with Apache 2.9.7, and Nginx 1.18 and libmodsecurity3 3.0.8.

Could you check that script with your application, and align it to reproduce the issue?

@theseion thanks a lot for a hint. I’ve been all the time using phase 1, but in this case it has to be phase:2 as it is already part of the body. Now exception works well 😃

@theseion working on it, hopefully will get data soon