runtime: The JSON value of length n is too large and not supported.

When returning a single JSON string of about 3,5 MB from a controller’s action (return type byte[]), the mentioned System.ArgumentException exception is thrown in my asp.net Core 3.0 (Preview 8) Api project. Looking through the docs, there seems to be no option to adjust any limit regarding this issue.

[HttpPost]
public async Task<ActionResult<byte[]>> GetFile()
{
    var largeArray = new byte[3.5 * 1024 * 1024];
    return largeArray;
}

Exception:

System.ArgumentException
  HResult=0x80070057
  Message=The JSON value of length 3770846 is too large and not supported.
  Source=System.Text.Json
  StackTrace:
   at System.Text.Json.ThrowHelper.ThrowArgumentException_ValueTooLarge(Int32 tokenLength)

Call stack:

System.Text.Json.dll!System.Text.Json.ThrowHelper.ThrowArgumentException_ValueTooLarge(int tokenLength)
System.Text.Json.dll!System.Text.Json.Utf8JsonWriter.WriteBase64StringValue(System.ReadOnlySpan<byte> bytes)
System.Text.Json.dll!System.Text.Json.Serialization.Converters.JsonConverterByteArray.Write(System.Text.Json.Utf8JsonWriter writer, byte[] value, System.Text.Json.JsonSerializerOptions options)
System.Text.Json.dll!System.Text.Json.JsonPropertyInfoNotNullable<object, byte[], byte[], byte[]>.OnWrite(ref System.Text.Json.WriteStackFrame current, System.Text.Json.Utf8JsonWriter writer)
System.Text.Json.dll!System.Text.Json.JsonPropertyInfo.Write(ref System.Text.Json.WriteStack state, System.Text.Json.Utf8JsonWriter writer)
System.Text.Json.dll!System.Text.Json.JsonSerializer.Write(System.Text.Json.Utf8JsonWriter writer, int originalWriterDepth, int flushThreshold, System.Text.Json.JsonSerializerOptions options, ref System.Text.Json.WriteStack state)
System.Text.Json.dll!System.Text.Json.JsonSerializer.WriteAsyncCore(System.IO.Stream utf8Json, object value, System.Type type, System.Text.Json.JsonSerializerOptions options, System.Threading.CancellationToken cancellationToken)
System.Text.Json.dll!System.Text.Json.JsonSerializer.SerializeAsync(System.IO.Stream utf8Json, object value, System.Type type, System.Text.Json.JsonSerializerOptions options, System.Threading.CancellationToken cancellationToken)
Microsoft.AspNetCore.Mvc.Core.dll!Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonOutputFormatter.WriteResponseBodyAsync(Microsoft.AspNetCore.Mvc.Formatters.OutputFormatterWriteContext context, System.Text.Encoding selectedEncoding)
Microsoft.AspNetCore.Mvc.Core.dll!Microsoft.AspNetCore.Mvc.Formatters.TextOutputFormatter.WriteAsync(Microsoft.AspNetCore.Mvc.Formatters.OutputFormatterWriteContext context)

There seems to be a fixed max length constant JsonConstants.MaxBase46ValueTokenSize (btw, typo in the constant’s name) set to 125 KB, that JsonWriterHelper.ValidateBytes() checks against.

Of course I can change the response type to plain\text and return it as such. Is that the intention of the hard coded 125 KB limit, or what’s the reason for that?

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Reactions: 4
  • Comments: 15 (9 by maintainers)

Most upvoted comments

The limit is based on maximum 1_000_000_000 size buffer with assumed max encoding factor of 6x and max base64 encoding factor of (4/3=1.33x) for a total max factor of 8x so given a buffer of 125_000_000 characters that could expand into a buffer of 1_000_000_000.

However, there is a bug with the const here. It should resolve to 125_000_000 but instead is only 2_604_166 causing your error using 3_770_846

The const is defined as

public const int MaxBase46ValueTokenSize = (MaxEscapedTokenSize >> 2 * 3) / MaxExpansionFactorWhileEscaping; 

but should be

public const int MaxBase64ValueTokenSize = ((MaxEscapedTokenSize >> 2) * 3) / MaxExpansionFactorWhileEscaping; 

or

public const int MaxBase64ValueTokenSize = (MaxEscapedTokenSize \ 4 * 3) / MaxExpansionFactorWhileEscaping; 

Simple repro:

var buffer = new System.Buffers.ArrayBufferWriter<byte>();
var writer = new Utf8JsonWriter(buffer);
writer.WriteStartArray();
byte[] bytes = Encoding.UTF8.GetBytes(new String('a', 2_604_167));
writer.WriteBase64StringValue(bytes);

cc @ahsonkhan

I’ll get on to it.

This restriction is not implemented in Json.Net (Newtonsoft), so why should it by here?

This should probably be made configurable.

Since, we effectively allow any reasonable size to go through, I don’t know if allowing an option to configure this is necessary. I haven’t seen or heard any feedback on trying to control max token size (unlike some of the other configurations we provide).

Agreed, 125MB is more than enough for anything sane 😃. Wasn’t clear that 125KB was unintentional.

I agree that the correct size of 125 MB is much more reasonable. If you’re returning a single JSON string larger than that, you should probably consider other options anyway because of size and encoding/decoding overhead.