coreruleset: SQLi id:942100, false positive on combination of two chars

User dune73 commented on date 2020-03-04 08:06:21:

Decision during the CRS project chat on March 2, 2020: dune73 will get in touch with the libinjection project to try and get things moving again.

https://github.com/SpiderLabs/owasp-modsecurity-crs/issues/1683#issuecomment-593584538

User AirisX commented on date 2019-05-29 14:41:12:

Some words about CP1251 mapping.

Unicode table contains a special range for Cyrillic characters. This is 0400 - 04FF (https://unicode-table.com/en/blocks/cyrillic/). They are used in many languages, the writing of which is based on them. But as for the Russian language, this range is much narrower: 0410 - 044F and two characters outside this range are 0401 (letter “Ё”) and 0451 (small letter “ё”). Thus, I think the correct filling of CP1251 mapping looks like this:

0410:c0 0411:c1 0412:c2 0413:c3 0414:c4 0415:c5 0416:c6 0417:c7 0418:c8 0419:c9 041a:ca 041b:cb 041c:cc 041d:cd 041e:ce 041f:cf 0420:d0 0421:d1 0422:d2 0423:d3 0424:d4 0425:d5 0426:d6 0427:d7 0428:d8 0429:d9 042a:da 042b:db 042c:dc 042d:dd 042e:de 042f:df 0430:e0 0431:e1 0432:e2 0433:e3 0434:e4 0435:e5 0436:e6 0437:e7 0438:e8 0439:e9 043a:ea 043b:eb 043c:ec 043d:ed 043e:ee 043f:ef 0440:f0 0441:f1 0442:f2 0443:f3 0444:f4 0445:f5 0446:f6 0447:f7 0448:f8 0449:f9 044a:fa 044b:fb 044c:fc 044d:fd 044e:fe 044f:ff 0401:a8 0451:b8

Using this mapping for payload with russian characters will not lead to a false positive in example with “Игорь”.

But code page 1251 also contains characters of the Ukrainian, Bulgarian and Belarusian alphabets. Such as ї, Ї, і, І and so on (see http://ascii-table.com/codepage.php?1251). They also need to be added to the mapping.

User lifeforms commented on date 2017-07-03 20:31:40:

Victor has some good analysis on this, if expanding the mapping would solve this then it would be a real constructive solution… I’m not familiar with that part of ModSec myself. Tagging this for 3.1.0 as the problem happen on multiple rules and we need to support our Russian friends!

User dune73 commented on date 2017-06-14 15:20:45:

landergate : This does not look like a quick fix and I think there is no way around disabling the rule for the login parameter for the time being. Of course, if you know the acceptable values that trigger false positives, then you can also disable the rule depending on the value of login.

User victorhora commented on date 2017-06-11 01:18:20:

Hey guys, it looks like we have a few problems here with this issue. First lets see if ModSecurity is handling these transformations right:

[07/Jun/2017:09:47:35 --0400] [localhost/sid#7f9c93b86418][rid#7f9c93acb0a0][/][5] Adding request argument (BODY): name "login", value "\xd0\xb4\xd0\xbe\xd1\x80"

\xd0\xb4\xd0\xbe\xd1\x80 looks correct UTF-8 encoding of дор.

[07/Jun/2017:08:46:35 --0400] [localhost/sid#7f138e17bd20][rid#7f138e0bd0a0][/][9] T (0) Utf8toUnicode: "%u0434%u043e%u0440"

0x0434 = CYRILLIC SMALL LETTER DE (aka “De”) 0x043E = “о” CYRILLIC SMALL LETTER O 0x0440 = “р” CYRILLIC SMALL LETTER ER (aka “Er”)

This looks right as well.

Moving on, there’s the SecUnicodeMapFile which t:urlDecodeUni uses to map Unicode for normalization and we have on the recommend modsecurity.conf file which by default points to Code point 20127 US-ASCII mapping.

If you simply try to change the code point to 1251 (aka Windows-1251) it still won’t work as for some reason these cyrillic characters are missing from the mapping file. But I believe they should be there according to unicode.org.

But if you add the missing Unicode characters and their corresponding ASCII codes (0434:64 043e:6f 0440:70) to the code point the rule works fine with the input:

[07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][4] Recipe: Invoking rule 7efd043ded48; [file "/etc/apache2/conf-enabled/modsecurity.conf"] [line "258"] [id "942102"].
[07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][5] Rule 7efd043ded48: SecRule "ARGS:login" "@detectSQLi " "phase:request,log,auditlog,msg:'SQL Injection Attack Detected via libinjection',id:942102,deny,t:none,t:utf8toUnicode,t:urlDecodeUni"
[07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][9] T (0) Utf8toUnicode: "%u0434%u043e%u0440"
[07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][9] T (0) urlDecodeUni: "dop"
[07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][4] Transformation completed in 12 usec.
[07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][4] Executing operator "detectSQLi" with param "" against ARGS:login.
[07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][9] Target value: "dop"
[07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][9] ISSQL: not sqli, no libinjection sqli fingerprint matched input 'dop'
[07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][4] Operator completed in 17 usec.
[07/Jun/2017:09:53:27 --0400] [localhost/sid#7efd043dad20][rid#7efd0431c0a0][/][4] Rule returned 0.

So I think the the transformations are working fine, but I think we need to do some adjustments to the mapping file or come up with a different solution altogether.

Interestingly enough looks like that other software that fails to recognize these characters does the same thing as ModSecurity. From Burp:

User dune73 commented on date 2017-06-06 11:16:14:

Yes, looks a bit simplistic. Bad implementation in ModSec?

User dune73 commented on date 2017-06-06 11:06:03:

You suspected correctly, lifeforms. Here is my reduced rule to trigger this FP:

Minimal Rule:

SecRule ARGS:login "@detectSQLi" \
  "msg:'SQL Injection Attack Detected via libinjection',\
   id:942100,\
   phase:request,\
   deny,\
   t:none,t:utf8toUnicode,t:urlDecodeUni"

Payload:
curl localhost -d "login=дор"

Debug Log:
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][5] Adding request argument (BODY): name "login", value "\xd0\xb4\xd0\xbe\xd1\x80"
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][4] Input filter: Completed receiving request body (length 12).
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][4] Starting phase REQUEST_BODY.
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][9] This phase consists of 1 rule(s).
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][4] Recipe: Invoking rule 55dcf0641518; [file "/apache/conf/httpd.conf_pod_2017-06-06_12:52"] [line "159"] [id "942100"] [rev "1"].
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][5] Rule 55dcf0641518: SecRule "ARGS:login" "@detectSQLi " "phase:request,log,msg:'SQL Injection Attack Detected via libinjection',id:942100,severity:CRITICAL,rev:1,ver:OWASP_CRS/3.0.0,maturity:1,accuracy:8,block,t:none,t:utf8toUnicode,t:urlDecodeUni,capture,logdata:'Matched Data: %{TX.0} found within %{MATCHED_VAR_NAME}: %{MATCHED_VAR}',tag:application-multi,tag:language-multi,tag:platform-multi,tag:attack-sqli,tag:OWASP_CRS/WEB_ATTACK/SQL_INJECTION,tag:WASCTC/WASC-19,tag:OWASP_TOP_10/A1,tag:OWASP_AppSensor/CIE1,tag:PCI/6.5.2,setvar:tx.anomaly_score=+%{tx.critical_anomaly_score},setvar:tx.sql_injection_score=+%{tx.critical_anomaly_score},setvar:tx.msg=%{rule.msg},setvar:tx.%{rule.id}-OWASP_CRS/WEB_ATTACK/SQL_INJECTION-%{matched_var_name}=%{matched_var}"
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][9] T (0) Utf8toUnicode: "%u0434%u043e%u0440"
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][9] T (0) urlDecodeUni: "4>@"
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][4] Transformation completed in 22 usec.
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][4] Executing operator "detectSQLi" with param "" against ARGS:login.
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][9] Target value: "4>@"
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][9] Added phrase match to TX.0: 1ov
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][9] ISSQL: libinjection fingerprint '1ov' matched input '4>@'
[06/Jun/2017:13:02:22 +0200] [localhost/sid#55dcf065da68][rid#7f0c6c038a70][/][4] Operator completed in 30 usec.

So it looks like a combination of payload and two transformation leads to an internal value that sets off libinjection.

User lifeforms commented on date 2017-06-06 10:12:01:

Wow, very good find. There are three places where this can go wrong: CRS, ModSecurity, or libinjection.

At first sight, it seems like a false positive in libinjection. But if I feed this input to libinjection itself, in encoded or decoded form, I am not getting a hit:

Payload	Result
dor	not sqli
дор	not sqli
%D0%B4%D0%BE%D1%80	not sqli

So I am not sure it’s a libinjection problem. There seems to be something interesting going on in ModSec or CRS. We should narrow the issue first by removing some transformations.

User dune73 commented on date 2017-06-06 04:53:07:

Thank you very much for your report landergate. This is an upstream problem but very glad you reported. I am going to forward this to the libinjection maintainer asap.