aspnetcore: AspNetCore 2.1-preview2 - System.Text.DecoderFallbackException
Hi,
I recently upgraded my code from asp.net core 2.0 to asp.net core 2.1-preview2. Since we upgraded we are encountering a crash in the production (for some requests) which never occurred before for the same code:
System.Text.DecoderFallbackException · Unable to translate bytes [C7] at index 829 from specified code page to Unicode.
System.Text.DecoderExceptionFallbackBuffer.Throw(byte[] bytesUnknown, int index)
System.Text.DecoderExceptionFallbackBuffer.Fallback(byte[] bytesUnknown, int index)
System.Text.DecoderFallbackBuffer.InternalFallback(byte[] bytes, Byte* pBytes, ref Char* chars)
System.Text.UTF8Encoding.GetChars(Byte* bytes, int byteCount, Char* chars, int charCount, DecoderNLS baseDecoder)
System.Text.DecoderNLS.GetChars(byte[] bytes, int byteIndex, int byteCount, char[] chars, int charIndex, bool flush)
System.Text.DecoderNLS.GetChars(byte[] bytes, int byteIndex, int byteCount, char[] chars, int charIndex)
Microsoft.AspNetCore.WebUtilities.HttpRequestStreamReader.ReadIntoBuffer()
Microsoft.AspNetCore.WebUtilities.HttpRequestStreamReader.Read(char[] buffer, int index, int count)
Newtonsoft.Json.JsonTextReader.ReadData(bool append, int charsRequired)
Newtonsoft.Json.JsonTextReader.ParseValue()
Newtonsoft.Json.JsonTextReader.Read()
Newtonsoft.Json.JsonReader.ReadForType(JsonContract contract, bool hasConverter)
Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, bool checkAdditionalContent)
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
Microsoft.AspNetCore.Mvc.Formatters.JsonInputFormatter+<ReadRequestBodyAsync>d__17.MoveNext()
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
System.Runtime.CompilerServices.TaskAwaiter<TResult>.GetResult()
Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder+<BindModelAsync>d__7.MoveNext()
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder+<BindModelAsync>d__11.MoveNext()
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
System.Runtime.CompilerServices.TaskAwaiter<TResult>.GetResult()
Microsoft.AspNetCore.Mvc.Internal.ControllerBinderDelegateProvider+<>c__DisplayClass0_0+<<CreateBinderDelegate>g__Bind|0>d.MoveNext()
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker+<InvokeInnerFilterAsync>d__13.MoveNext()
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker+<InvokeNextExceptionFilterAsync>d__24.MoveNext()
It seems request parsing is failing which wasn’t happening before. Can’t even log the request model as crash occurs before the controller action is called.
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 19 (14 by maintainers)
Based on https://github.com/aspnet/Home/issues/3105#issuecomment-385252992, https://github.com/aspnet/Home/issues/3105#issuecomment-385492323 and the callstack, the scenario is that there’s a C++ client posting json data(having invalid characters) to asp.net core app and we failed to decode these characters in the server. AFAIK, we need not fix anything at our end.
So it sounds like everything here was behaving appropriately when presented with invalid bytes from the client.
@kichalla I can’t provide a wireshark trace because it’s happening on machines of our customers. However, I can share how I fixed the issue:
Our C++ HTTP client gets hostname of machine using GetComputerNameA function.
Some machines seem to have non-ASCII characters in the hostname, so I changed it to
GetComputerNamewhich is theutf-16version of the function, and before http post I converted theutf-16hostname string toutf-8hostname string. This fixed the issue at my end.E.g. a wireshark trace