aspnetcore: .Net 6 Preview 7 - Cannot set reponse headers before returning IAsyncEnumerable in controller

Describe the bug

After updating from .Net 5 to .Net 6 Preview 7, I’m having an exception “System.InvalidOperationException: Headers are read-only, response has already started.” when trying to setup a response header before returning an IAsyncEnumerable. This error only occurs when making another async call in the process.

    [HttpGet]
    public async IAsyncEnumerable<WeatherForecastData> Get([EnumeratorCancellation] CancellationToken cancellationToken)
    {
        var results = await _dataService.GetData();
        // System.InvalidOperationException: Headers are read-only, response has already started.
        Response.Headers.Add("Test", results.Name);
        await foreach (var result in results.Data.WithCancellation(cancellationToken))
        {
            yield return result;
        }
    }

My use case is that for pagination purposes before returning the IAsyncEnumerable I perform a query to get the total amount of data in the db, then place that pagination information as a response header for the client.

To Reproduce

Use this webapi minimal repository to reproduce

https://github.com/gabynevada/.net6-iasync-enumerable-set-header-error

Exceptions (if any)

Click to expand exception message

System.InvalidOperationException: Headers are read-only, response has already started. at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpHeaders.ThrowHeadersReadOnlyException() in Microsoft.AspNetCore.Server.Kestrel.Core.dll:token 0x6000a43+0xa at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpHeaders.System.Collections.Generic.IDictionary<System.String,Microsoft.Extensions.Primitives.StringValues>.Add(String key, StringValues value) in Microsoft.AspNetCore.Server.Kestrel.Core.dll:token 0x6000a5a+0x8 at System.Collections.Generic.CollectionExtensions.TryAdd[TKey,TValue](IDictionary2 dictionary, TKey key, TValue value) in System.Collections.dll:token 0x600005b+0x17 at SetResponseHeaders.Controllers.WeatherForecastController.Get(CancellationToken cancellationToken)+MoveNext() in /Users/elvis/Downloads/SetResponseHeaders/Controllers/WeatherForecastController.cs:line 27 at SetResponseHeaders.Controllers.WeatherForecastController.Get(CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult() in SetResponseHeaders.dll:token 0x6000014+0x0 at System.Threading.Tasks.ValueTask1.ValueTaskSourceAsTask.<>c.<.cctor>b__4_0(Object state) in System.Private.CoreLib.dll:token 0x60033c6+0x23 — End of stack trace from previous location — at System.Text.Json.Serialization.Converters.IAsyncEnumerableOfTConverter2.OnWriteResume(Utf8JsonWriter writer, TAsyncEnumerable value, JsonSerializerOptions options, WriteStack& state) in System.Text.Json.dll:token 0x6000970+0x95 at System.Text.Json.Serialization.Converters.IEnumerableDefaultConverter2.OnTryWrite(Utf8JsonWriter writer, TCollection value, JsonSerializerOptions options, WriteStack& state) in System.Text.Json.dll:token 0x600099e+0x86 at System.Text.Json.Serialization.Converters.IAsyncEnumerableOfTConverter2.OnTryWrite(Utf8JsonWriter writer, TAsyncEnumerable value, JsonSerializerOptions options, WriteStack& state) in System.Text.Json.dll:token 0x600096f+0x14 at System.Text.Json.Serialization.JsonConverter1.TryWrite(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state) in System.Text.Json.dll:token 0x6000790+0x1f4 at System.Text.Json.Serialization.JsonConverter1.WriteCore(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state) in System.Text.Json.dll:token 0x600077b+0x0 at System.Text.Json.Serialization.JsonConverter1.WriteCoreAsObject(Utf8JsonWriter writer, Object value, JsonSerializerOptions options, WriteStack& state) in System.Text.Json.dll:token 0x600077a+0x49 at System.Text.Json.JsonSerializer.WriteCore[TValue](JsonConverter jsonConverter, Utf8JsonWriter writer, TValue& value, JsonSerializerOptions options, WriteStack& state) in System.Text.Json.dll:token 0x60003c6+0x18 at System.Text.Json.JsonSerializer.WriteAsyncCore[TValue](Stream utf8Json, TValue value, JsonTypeInfo jsonTypeInfo, CancellationToken cancellationToken) in System.Text.Json.dll:token 0x60003d4+0xd4 at System.Text.Json.JsonSerializer.WriteAsyncCore[TValue](Stream utf8Json, TValue value, JsonTypeInfo jsonTypeInfo, CancellationToken cancellationToken) in System.Text.Json.dll:token 0x60003d4+0x1b5 at System.Text.Json.JsonSerializer.WriteAsyncCore[TValue](Stream utf8Json, TValue value, JsonTypeInfo jsonTypeInfo, CancellationToken cancellationToken) in System.Text.Json.dll:token 0x60003d4+0x38b at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding) in Microsoft.AspNetCore.Mvc.Core.dll:token 0x6000b14+0x132 at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResultFilterAsync>g__Awaited|30_0[TFilter,TFilterAsync](ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) in Microsoft.AspNetCore.Mvc.Core.dll:token 0x6000a89+0x6a at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResultExecutedContextSealed context) in Microsoft.AspNetCore.Mvc.Core.dll:token 0x6000a7c+0x15 at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext[TFilter,TFilterAsync](State& next, Scope& scope, Object& state, Boolean& isCompleted) in Microsoft.AspNetCore.Mvc.Core.dll:token 0x6000a77+0x3dc at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeResultFilters>g__Awaited|28_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) in Microsoft.AspNetCore.Mvc.Core.dll:token 0x6000a88+0x6e at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) in Microsoft.AspNetCore.Mvc.Core.dll:token 0x6000a81+0x65 at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope) in Microsoft.AspNetCore.Mvc.Core.dll:token 0x6000a7d+0x77 at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope) in Microsoft.AspNetCore.Mvc.Core.dll:token 0x6000a7d+0xfb at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger) in Microsoft.AspNetCore.Routing.dll:token 0x60000ab+0x5e at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context) in Microsoft.AspNetCore.Authorization.Policy.dll:token 0x6000013+0x16b at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(HttpContext httpContext) in Swashbuckle.AspNetCore.SwaggerUI.dll:token 0x6000002+0x1ce at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(HttpContext httpContext, ISwaggerProvider swaggerProvider) in Swashbuckle.AspNetCore.Swagger.dll:token 0x6000009+0x8e at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context) in Microsoft.AspNetCore.Diagnostics.dll:token 0x60000aa+0x82 warn: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[2] The response has already started, the error page middleware will not be executed. fail: Microsoft.AspNetCore.Server.Kestrel[13]

Further technical details

ASP.NET Core version:

.Net 6 Preview 7

Include the output of dotnet --info

Click to view output

.NET SDK (reflecting any global.json): Version: 6.0.100-preview.7.21379.14 Commit: 22d70b47bc

Runtime Environment: OS Name: Mac OS X OS Version: 11.5 OS Platform: Darwin RID: osx.11.0-x64 Base Path: /usr/local/share/dotnet/sdk/6.0.100-preview.7.21379.14/

Host (useful for support): Version: 6.0.0-preview.7.21377.19 Commit: 91ba01788d

.NET SDKs installed: 5.0.100 [/usr/local/share/dotnet/sdk] 5.0.101 [/usr/local/share/dotnet/sdk] 5.0.102 [/usr/local/share/dotnet/sdk] 5.0.103 [/usr/local/share/dotnet/sdk] 5.0.202 [/usr/local/share/dotnet/sdk] 6.0.100-preview.7.21379.14 [/usr/local/share/dotnet/sdk]

.NET runtimes installed: Microsoft.AspNetCore.App 5.0.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 5.0.1 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 5.0.2 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 5.0.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 5.0.5 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 6.0.0-preview.7.21378.6 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.NETCore.App 5.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 5.0.1 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 5.0.2 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 5.0.3 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 5.0.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 6.0.0-preview.7.21377.19 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

The IDE (VS / VS Code/ VS4Mac) you’re running on, and its version:

VsCode 1.59.0

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 28 (11 by maintainers)

Most upvoted comments

You can also return a Task<IAsyncEnumerable<T>> would that help?

public async Task<IAsyncEnumerable<WeatherForecastData>> Get()
{
    var results = await _dataService.GetData();

    Response.Headers.Add("Test", results.Name);

    return Enumerate();

    async IAsyncEnumerable<WeatherForecastData> Enumerate()
    {
        await foreach (var result in results)
        {
            yield return result;
        }
    }
}

https://github.com/aspnet/Announcements/issues/463

campersau on Aug 19, 2021

Documentation would be great! We will do that.

davidfowl on Aug 19, 2021

Response.Headers.Add(“Test”, results.Name);

That doesn’t compile. results is out of scope there.

Even if it was in scope, it’s impossible to get a value there that needs await.

A workaround would be to await it “synchronously”. But that would be less than ideal.

arjennienhuis on Aug 18, 2021

@dustinmoris Thanks for the explanation, I understand the issue now 👍

nikolalukovic on Aug 17, 2021

The second code snippet is the workaround.

In order for this to make sense you have to understand how an AsyncEnumerable works. A bit like an Enumerable, it will yield (hence the keyword) each element when called. You won’t know how many elements there even are before calling the method, otherwise it would just be a list. The use case is to stream elements and if you think of it as a stream then it might also make more sense than thinking of it as an “enumerable” (the naming is a bit confusing in .NET).

So, if you have an async IAsyncEnumerable then the internal code of that method will not be executed/materialised before it’s actually being called, otherwise it would behave like a list and not like a stream. Does that make sense? That’s what David was trying to explain, the compiler creates a state machine and executes the code when it’s actually being requested during runtime. So if that is being called from a method which writes the elements straight to the response stream then at that point the headers have already been flushed and you won’t be able to set/edit them anymore.

In the workaround you extract your async enumerable generator into a nested method and then return that method from within the parent method which isn’t an async method and doesn’t await the IAsyncEnumerable itself. You’ll be able to do whatever you want before the return statement:

dustinmoris on Aug 17, 2021