ModSecurity: Problems with blocking if response is using compression

Describe the bug

When blocking a request, which uses output compression (for example Content-Encoding: gzip), in phase 4 (so response headers are already created by application), blocking is NOT done properly because Content-Encoding: gzip will already be send to client so it awaits compressed response - on the client side, browsers will show error similar to this (this one is from Firefox): Content Encoding Error

To Reproduce

Problem can be easily reproduced using this PHP script and blocking rule:

<?php
@ob_start("ob_gzHandler");
header('X-test-blocking: block-me');
echo 'test';
?>

SecRule &RESPONSE_HEADERS:X-test-blocking "!@eq 0" \
    "id:99999999999,\
    phase:4,\
    deny"

Expected behavior

modsecurity should remove or properly set the Content-Encoding header if request is blocked.

About this issue

Original URL
State: open
Created 3 years ago
Comments: 25 (11 by maintainers)

Most upvoted comments

Linking this issue with #944.

zimmerle on Jan 13, 2021

That’s it! After switching to mod_php, problem is gone.

azurit on Jan 7, 2021

If headers were already sent to client (i.e. we are in phase 4 using embedded modsecurity) AND they included header Content-Encoding with value gzip, the client is expecting gzipped response. If we decide to deny this request and do NOT send original response body, we need to compress our new response body so client is not confused.

Once the request was disruptive blocked by a deny action, there is no content to be delivered to the user – a possible malicious player in that case.

This would be sooo cool! I’m really missing this feature. As you can see in PR coreruleset/coreruleset#1968, i implemented something like t:unzip using Lua script but a transformation function will probably be much faster.

Indeed it should be faster and more generic (e.g., available to other variables). One of the ideas is to support Lua is to encourage prototyping; it fits the implementation that you have made 👍

Looking at your code https://github.com/coreruleset/coreruleset/pull/1968/commits/bfd4d490837c6eccf287a401ff0a6c23c311c5cc#diff-ec25e9162441fd2a294691ac0f3fff833c80e893f436feebd1ea86df2b19ac5d

require("m")
require("zlib")
function main()
local f = zlib.inflate()
local response_body_uncompressed = f(m.getvar("RESPONSE_BODY", "none"))
m.setvar("tx.response_body_uncompressed", response_body_uncompressed)
return nil
end

The c++ variant shouldn’t be that different. The annoying part is to understand how to add the Zlib dependency on ModSecurity. Once that is done, the transformation should be easy to implement, something like the example below (not tested, just a draft) -

void GZipDecompress::execute(const Transaction *t, const ModSecString &in, ModSecString &out) noexcept {
    zlibcomplete::GZipDecompressor decompressor;
    std::string ret = decompressor.decompress(std::string(in.c_str(), in.size()));
    out.assign(ret.c_str(), ret.size());
}

Here is an example of the transformation t:phpArgsNames added by @marshal09 few weeks ago - https://github.com/SpiderLabs/ModSecurity/pull/2387/commits/bab97237db2bee7362d6e3fa2cb60bb92af491fa . In this example we don’t have an external dependency but gives you an idea of what needs to be made.

Up for the challenge? 😃 🚀 🚀

zimmerle on Jan 7, 2021