runtime: BrotliEncoder.GetMaxCompressedLength returns invalid value
Project target: netcoreapp3.1
Test code
using System;
using System.IO.Compression;
namespace Sandbox
{
class Program
{
static void Main(string[] args)
{
const int quality = 5;
const int window = 10;
byte[] source = new byte[256000];
var rng = new Random(1234);
rng.NextBytes(source);
int maxLength = BrotliEncoder.GetMaxCompressedLength(source.Length);
var resultBuffer = new byte[maxLength + 10];
if (!BrotliEncoder.TryCompress(
source, resultBuffer, out int written1, quality, window))
throw new Exception();
if (!BrotliEncoder.TryCompress(
source, resultBuffer.AsSpan(0, maxLength), out int written2, quality, window))
throw new Exception();
}
}
}
dotnet --info
.NET Core SDK (reflecting any global.json):
Version: 3.1.201
Commit: b1768b4ae7
Runtime Environment:
OS Name: Windows
OS Version: 10.0.18363
OS Platform: Windows
RID: win10-x64
Base Path: C:\Program Files\dotnet\sdk\3.1.201\
Host (useful for support):
Version: 3.1.3
Commit: 4a9f85e9f8
.NET Core SDKs installed:
2.1.700 [C:\Program Files\dotnet\sdk]
2.1.701 [C:\Program Files\dotnet\sdk]
2.1.801 [C:\Program Files\dotnet\sdk]
2.1.802 [C:\Program Files\dotnet\sdk]
2.2.300 [C:\Program Files\dotnet\sdk]
3.1.201 [C:\Program Files\dotnet\sdk]
.NET Core runtimes installed:
Microsoft.AspNetCore.All 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Reactions: 1
- Comments: 16 (8 by maintainers)
I’ve encounted same issue. brotli/encode.c has
BrotliEncoderMaxCompressedSizemethod, that returns different value of current C# impl. https://github.com/google/brotli/blob/3914999fcc1fda92e750ef9190aa6db9bf7bdb07/c/enc/encode.c#L1200 I’ve port this code to C# and compare result.My test data,
I think this is a critical issue of compression library, could you fix it?
For the poor souls who are still affected by this, I recommend switching over to zstd, which is unaffected by this bug, and generally has a better runtime performance and compression ratio.
It’s unlikely that this one will be fixed in anytime soon, it’s been open for more than 3 years.
Let’s just P/Invoke to the native implementation, since it’s the source of truth, and this size needs to remain in sync with what the encoder actually does.
.NET 9 RC1 supports new BrotliCompressionOptions in #105430 , however this bug is not cared, I’ve tested in latest VS-preview but still exists. @buyaa-n , @carlossanlop , @stephentoub (mentioned to #105430 reviewers)
Currently all PRs should go to main (10.0), based on the customer feedback and the risk of the fix management will decide if we port the fix to previous versions and which versions will get fixed