aspnetcore: NativeAOT is much slower that JIT version

A simple REST server as below, shows that NativeAOT is much slower than JIT version of the code.

Here is the the only controller in the app :

using Microsoft.AspNetCore.Mvc;
namespace nativeAOTapi.Controllers;

[Controller]
[Route("api/[controller]")]
public class TimeAPI :Controller
{
    [HttpGet]
    [Route("time")]
    public ActionResult<long> GetTime()
    {
        return DateTimeOffset.Now.ToUnixTimeMilliseconds();
    }
}

Here is Program.cs:

var builder = WebApplication.CreateBuilder(args);
builder.Services.AddControllers();
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();
var app = builder.Build();
app.UseSwagger();
app.UseSwaggerUI();
app.MapControllers();
app.Run();

Here is Program.csproj

<Project Sdk="Microsoft.NET.Sdk.Web">
    <PropertyGroup>
        <TargetFramework>net6.0</TargetFramework>
        <Nullable>enable</Nullable>
        <ImplicitUsings>enable</ImplicitUsings>
        <IlcOptimizationPreference>Speed</IlcOptimizationPreference>
    </PropertyGroup>

    <ItemGroup>
        <PackageReference Include="Microsoft.DotNet.ILCompiler" Version="7.0.0-*" />
        <PackageReference Include="Swashbuckle.AspNetCore" Version="6.2.3" />
    </ItemGroup>
</Project>

Here is Nuget.config:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <!--To inherit the global NuGet package sources remove the <clear/> line below -->
    <clear />
      <add key="nuget" value="https://api.nuget.org/v3/index.json" />
	<add key="dotnet-experimental" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-experimental/nuget/v3/index.json" />
	<add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
  </packageSources>
</configuration>

JIT version runs on port 5247 NativeAOT version runs on port 5000

Here is some benchmark results;

for JIT:

Bombarding http://localhost:5247/api/TimeAPI/time for 10s using 200 connection(s)
[========================================================================================================================================================================================] 10s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec    175618.11   38073.63  219971.12
  Latency        1.14ms     2.07ms   192.43ms
  HTTP codes:
    1xx - 0, 2xx - 1749150, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:    41.53MB/s

for NativeAOT

Bombarding http://localhost:5000/api/TimeAPI/time for 10s using 200 connection(s)
[========================================================================================================================================================================================] 10s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec     51621.50    7439.70   60899.66
  Latency        3.89ms     2.25ms   148.00ms
  HTTP codes:
    1xx - 0, 2xx - 514153, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:    12.21MB/s

JIT version is build and run with dotnet run NativeAOT version is build with dotnet publish -r linux-x64 -c Release

dotnet : 6.0.101 OS : Linux pop-os 5.15.11-76051511-generic #202112220937~1640185481~21.10~b3a2c21 SMP Wed Dec 22 15:41:49 U x86_64 x86_64 x86_64 GNU/Linux CPU : Intel Core i7 8700

About this issue

Original URL
State: open
Created 2 years ago
Comments: 19 (18 by maintainers)

Most upvoted comments

The code generation options are all have different tradeoffs:

DynamicMethod/IL Emit - Requires JIT (can be interpreted on Mono by we don’t on CoreCLR). Very efficient, harder to maintain.
Expression trees - Big and bloated, not being evolved, works everywhere (efficient with JIT, interpreted without).
Generics - Generic constraints are viral so highly polymorphic generic parameters still aren’t type safe. Instantiations with value types need to be statically visible to the AOT compiler.
Source generators - Generated code versions with the application, no longer part of the framework. Can’t change the call site so changes the programming model might be required.

Right now I’m thinking about a combination of generics and source generation to balance versioning (how much code exists in the app vs framework), but the generic constraints problem is a hard one to solve.

davidfowl on Jul 1, 2022

@jkoritzinsky Yep we should put this under a microscope. I have some ideas where the time is spent but it would be good to get some confirmation.

davidfowl on Feb 6, 2022

This is possibly due to the code paths that ASP.NET Core’s routing goes down on NativeAOT vs a JIT environment. In a JIT environment, ASP.NET Core uses runtime-compiled expression trees or JIT-compiled IL-emit, which can be quite fast. In an AOT environment, expression trees need to be interpreted and only traditional reflection can be used, not IL-emit, both of which are slower than their JIT equivalents.

cc: @davidfowl @dotnet/interop-contrib I believe this might relate to our earlier conversations about fast-reflection for ASP.NET Core.

jkoritzinsky on Feb 5, 2022

I believe once we integrate NativeAOT into our PerfLab infra (TE benchmarks, etc) with ability to get native traces and validate changes by sending new bits we will find low-hanging fruits there

EgorBo on Feb 6, 2022

@jkoritzinsky Can we source generate those code paths at compile time itself? I wish in .NET 7, the frameworks like aspnetcore take full advantage of source generators to address these kinda issues. That way we can solve problems for both JIT and AOT worlds! Am yet to see any issue which tracks source generator work for aspnetcore, but I hope that is on priority!

ShreyasJejurkar on Feb 6, 2022

@MichalStrehovsky yes, something like that. Or possibly not even using the ObjectMethodExecutor in some cases and using method info directly. I think it also make sense to track add NativeAOT variations to some of our benchmarks so we can observe these differences when we make the fixes.

davidfowl on Jun 16, 2022

If your app or library needs fast expression trees execution, NativeAOT is not a good fit for it.

We can work on incremental perf improvements in expression tree interpreter. I do not expect it will move the needle enough. Also, major new work in expression tree interpreter is at odds with its archived status: https://github.com/dotnet/runtime/blob/main/src/libraries/System.Linq.Expressions/README.md.

jkotas on Jul 1, 2022

Sure we can keep this open. ~~But do you expect the fix to be in this repo?~~ EDIT: You just moved the issue as I was clicking comment

Looking at the ObjectMethodExecutor, the fix is basically (simplified):

    public object? Execute(object target, object?[]? parameters)
    {
        if (LINQ is compiled)
        {
            Debug.Assert(_executor != null, "Sync execution is not supported.");
            return _executor(target, parameters);
        }
        else
        {
            // New code
            return MethodInfo.Invoke(target, parameters);
        }
    }

MichalStrehovsky on Jun 16, 2022

MVC controller actions, minimal APIs, SignalR hub methods all have runtime generated thunks that use expression trees to quickly invoke methods. It might be worth having an alternative reflection invoke mode on NativeAOT. Here’s one of the shared components used to invoke some of these generated thunks:

https://github.com/dotnet/aspnetcore/blob/da6cdcbd5dc75b695cee36d47a22e1399cbea89e/src/Shared/ObjectMethodExecutor/ObjectMethodExecutor.cs

DI has a similar issue but we detect if dynamic code is supported and fallback to a reflection based strategy.

Maybe we can leave this open since we haven’t invested in this as yet?

davidfowl on Jun 16, 2022

We’re spending a lot of time in the expression interpreter. The calls are coming from HostFilteringMiddleware.Invoke and HttpProtocol.ProcessRequests:

publishaot1!System_Linq_Expressions_System_Linq_Expressions_Interpreter_LightLambda__Run
publishaot1!S_P_CoreLib_System_Func_4__InvokeObjectArrayThunk
  publishaot1!Microsoft_AspNetCore_HostFiltering_Microsoft_AspNetCore_HostFiltering_HostFilteringMiddleware__Invoke
  publishaot1!Microsoft_AspNetCore_Server_Kestrel_Core_Microsoft_AspNetCore_Server_Kestrel_Core_Internal_Http_HttpProtocol__ProcessRequests_d__223_1__MoveNext

LINQ expressions won’t have good perf with native AOT because we can’t JIT them. Not much we can do from the runtime side.

MichalStrehovsky on Jun 16, 2022