runtime: aspnetcore tests having Dynamic.Proxy fails on ppc64le architecture

Description

Around 6000+ aspnetcore tests which are using Dynamic.Proxy are failing on ppc64le with mono complier.

Reproduction Steps

build aspnetcore on ppc64le using cross built net8 rc2 and run tests.

Expected behavior

All tests should pass

Actual behavior

We get below error for 6000+ tests

Microsoft.AspNetCore.Server.Kestrel.Core.Tests.ConnectionContextTests.ParameterlessAbortCreateConnectionAbortedException
System.InvalidCastException : Specified cast is not valid.
  at Castle.DynamicProxy.Internal.AttributeUtil.ReadAttributeValue(CustomAttributeTypedArgument argument)
  at Castle.DynamicProxy.Internal.AttributeUtil.GetArguments(IList`1 constructorArguments, Type[]& constructorArgTypes, Object[]& constructorArgs)
  at Castle.DynamicProxy.Internal.AttributeUtil.CreateInfo(CustomAttributeData attribute)
  at Castle.DynamicProxy.Internal.AttributeUtil.GetNonInheritableAttributes(MemberInfo member)+MoveNext()
  at System.Linq.Enumerable.SelectEnumerableIterator`2[[Castle.DynamicProxy.CustomAttributeInfo, Castle.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=407dd0808d44fbdc],[System.Reflection.Emit.CustomAttributeBuilder, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
  at Castle.DynamicProxy.Generators.MetaProperty.BuildPropertyEmitter(ClassEmitter classEmitter)
  at Castle.DynamicProxy.Contributors.CompositeTypeContributor.ImplementProperty(ClassEmitter emitter, MetaProperty property, ProxyGenerationOptions options)
  at Castle.DynamicProxy.Contributors.CompositeTypeContributor.Generate(ClassEmitter class, ProxyGenerationOptions options)
  at Castle.DynamicProxy.Generators.ClassProxyGenerator.GenerateType(String name, Type[] interfaces, INamingScope namingScope)
  at Castle.DynamicProxy.Generators.ClassProxyGenerator.<>c__DisplayClass1_0.<GenerateCode>b__0(String n, INamingScope s)
  at Castle.DynamicProxy.Generators.BaseProxyGenerator.ObtainProxyType(CacheKey cacheKey, Func`3 factory)
  at Castle.DynamicProxy.Generators.ClassProxyGenerator.GenerateCode(Type[] interfaces, ProxyGenerationOptions options)
  at Castle.DynamicProxy.DefaultProxyBuilder.CreateClassProxyType(Type classToProxy, Type[] additionalInterfacesToProxy, ProxyGenerationOptions options)
  at Castle.DynamicProxy.ProxyGenerator.CreateClassProxyType(Type classToProxy, Type[] additionalInterfacesToProxy, ProxyGenerationOptions options)
  at Castle.DynamicProxy.ProxyGenerator.CreateClassProxy(Type classToProxy, Type[] additionalInterfacesToProxy, ProxyGenerationOptions options, Object[] constructorArguments, IInterceptor[] interceptors)
  at Moq.CastleProxyFactory.CreateProxy(Type mockType, IInterceptor interceptor, Type[] interfaces, Object[] arguments)
  at Moq.Mock`1[[Microsoft.AspNetCore.Connections.ConnectionContext, Microsoft.AspNetCore.Connections.Abstractions, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].InitializeInstancePexProtected()
  at Moq.PexProtector.Invoke(Action action)
  at Moq.Mock`1[[Microsoft.AspNetCore.Connections.ConnectionContext, Microsoft.AspNetCore.Connections.Abstractions, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].InitializeInstance()
  at Moq.Mock`1[[Microsoft.AspNetCore.Connections.ConnectionContext, Microsoft.AspNetCore.Connections.Abstractions, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].OnGetObject()
  at Moq.Mock.get_Object()
  at Moq.Mock`1[[Microsoft.AspNetCore.Connections.ConnectionContext, Microsoft.AspNetCore.Connections.Abstractions, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].get_Object()
  at Microsoft.AspNetCore.Server.Kestrel.Core.Tests.ConnectionContextTests.ParameterlessAbortCreateConnectionAbortedException() in /_/src/Servers/Kestrel/Core/test/ConnectionContextTests.cs:line 21
  at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
  at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
0.0321365s✘ Microsoft.AspNetCore.Server.Kestrel.Core.Tests.ConnectionDispatcherTests.OnConnectionCreatesLogScopeWithConnectionId
System.InvalidCastException : Specified cast is not valid.
  at Castle.DynamicProxy.Internal.AttributeUtil.ReadAttributeValue(CustomAttributeTypedArgument argument)
  at Castle.DynamicProxy.Internal.AttributeUtil.GetArguments(IList`1 constructorArguments, Type[]& constructorArgTypes, Object[]& constructorArgs)
  at Castle.DynamicProxy.Internal.AttributeUtil.CreateInfo(CustomAttributeData attribute)
  at Castle.DynamicProxy.Internal.AttributeUtil.GetNonInheritableAttributes(MemberInfo member)+MoveNext()
  at System.Linq.Enumerable.SelectEnumerableIterator`2[[Castle.DynamicProxy.CustomAttributeInfo, Castle.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=407dd0808d44fbdc],[System.Reflection.Emit.CustomAttributeBuilder, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
  at Castle.DynamicProxy.Generators.MetaProperty.BuildPropertyEmitter(ClassEmitter classEmitter)
  at Castle.DynamicProxy.Contributors.CompositeTypeContributor.ImplementProperty(ClassEmitter emitter, MetaProperty property, ProxyGenerationOptions options)
  at Castle.DynamicProxy.Contributors.CompositeTypeContributor.Generate(ClassEmitter class, ProxyGenerationOptions options)
  at Castle.DynamicProxy.Generators.ClassProxyGenerator.GenerateType(String name, Type[] interfaces, INamingScope namingScope)
  at Castle.DynamicProxy.Generators.ClassProxyGenerator.<>c__DisplayClass1_0.<GenerateCode>b__0(String n, INamingScope s)
  at Castle.DynamicProxy.Generators.BaseProxyGenerator.ObtainProxyType(CacheKey cacheKey, Func`3 factory)
  at Castle.DynamicProxy.Generators.ClassProxyGenerator.GenerateCode(Type[] interfaces, ProxyGenerationOptions options)
  at Castle.DynamicProxy.DefaultProxyBuilder.CreateClassProxyType(Type classToProxy, Type[] additionalInterfacesToProxy, ProxyGenerationOptions options)
  at Castle.DynamicProxy.ProxyGenerator.CreateClassProxyType(Type classToProxy, Type[] additionalInterfacesToProxy, ProxyGenerationOptions options)
  at Castle.DynamicProxy.ProxyGenerator.CreateClassProxy(Type classToProxy, Type[] additionalInterfacesToProxy, ProxyGenerationOptions options, Object[] constructorArguments, IInterceptor[] interceptors)
  at Moq.CastleProxyFactory.CreateProxy(Type mockType, IInterceptor interceptor, Type[] interfaces, Object[] arguments)
  at Moq.Mock`1[[Microsoft.AspNetCore.Connections.DefaultConnectionContext, Microsoft.AspNetCore.Connections.Abstractions, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].InitializeInstancePexProtected()
  at Moq.PexProtector.Invoke(Action action)
  at Moq.Mock`1[[Microsoft.AspNetCore.Connections.DefaultConnectionContext, Microsoft.AspNetCore.Connections.Abstractions, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].InitializeInstance()
  at Moq.Mock`1[[Microsoft.AspNetCore.Connections.DefaultConnectionContext, Microsoft.AspNetCore.Connections.Abstractions, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].OnGetObject()
  at Moq.Mock.get_Object()
  at Moq.Mock`1[[Microsoft.AspNetCore.Connections.DefaultConnectionContext, Microsoft.AspNetCore.Connections.Abstractions, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].get_Object()
  at Microsoft.AspNetCore.Server.Kestrel.Core.Tests.ConnectionDispatcherTests.OnConnectionCreatesLogScopeWithConnectionId() in /_/src/Servers/Kestrel/Core/test/ConnectionDispatcherTests.cs:line 33
--- End of stack trace from previous location ---
Output:
| [0.007s] TestLifetime Information: Starting test OnConnectionCreatesLogScopeWithConnectionId at 2023-10-17T17:03:42
| [0.039s] Microsoft.AspNetCore.Server.Kestrel.Core.Tests.ConnectionDispatcherTests Error: Test threw an exception.
| System.InvalidCastException: Specified cast is not valid.
|    at Castle.DynamicProxy.Internal.AttributeUtil.ReadAttributeValue(CustomAttributeTypedArgument argument)
|    at Castle.DynamicProxy.Internal.AttributeUtil.GetArguments(IList`1 constructorArguments, Type[]& constructorArgTypes, Object[]& constructorArgs)
|    at Castle.DynamicProxy.Internal.AttributeUtil.CreateInfo(CustomAttributeData attribute)
|    at Castle.DynamicProxy.Internal.AttributeUtil.GetNonInheritableAttributes(MemberInfo member)+MoveNext()
|    at System.Linq.Enumerable.SelectEnumerableIterator`2[[Castle.DynamicProxy.CustomAttributeInfo, Castle.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=407dd0808d44fbdc],[System.Reflection.Emit.CustomAttributeBuilder, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
|    at Castle.DynamicProxy.Generators.MetaProperty.BuildPropertyEmitter(ClassEmitter classEmitter)
|    at Castle.DynamicProxy.Contributors.CompositeTypeContributor.ImplementProperty(ClassEmitter emitter, MetaProperty property, ProxyGenerationOptions options)
|    at Castle.DynamicProxy.Contributors.CompositeTypeContributor.Generate(ClassEmitter class, ProxyGenerationOptions options)
|    at Castle.DynamicProxy.Generators.ClassProxyGenerator.GenerateType(String name, Type[] interfaces, INamingScope namingScope)
|    at Castle.DynamicProxy.Generators.ClassProxyGenerator.<>c__DisplayClass1_0.<GenerateCode>b__0(String n, INamingScope s)
|    at Castle.DynamicProxy.Generators.BaseProxyGenerator.ObtainProxyType(CacheKey cacheKey, Func`3 factory)
|    at Castle.DynamicProxy.Generators.ClassProxyGenerator.GenerateCode(Type[] interfaces, ProxyGenerationOptions options)
|    at Castle.DynamicProxy.DefaultProxyBuilder.CreateClassProxyType(Type classToProxy, Type[] additionalInterfacesToProxy, ProxyGenerationOptions options)
|    at Castle.DynamicProxy.ProxyGenerator.CreateClassProxyType(Type classToProxy, Type[] additionalInterfacesToProxy, ProxyGenerationOptions options)
|    at Castle.DynamicProxy.ProxyGenerator.CreateClassProxy(Type classToProxy, Type[] additionalInterfacesToProxy, ProxyGenerationOptions options, Object[] constructorArguments, IInterceptor[] interceptors)
|    at Moq.CastleProxyFactory.CreateProxy(Type mockType, IInterceptor interceptor, Type[] interfaces, Object[] arguments)
|    at Moq.Mock`1[[Microsoft.AspNetCore.Connections.DefaultConnectionContext, Microsoft.AspNetCore.Connections.Abstractions, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].InitializeInstancePexProtected()
|    at Moq.PexProtector.Invoke(Action action)
|    at Moq.Mock`1[[Microsoft.AspNetCore.Connections.DefaultConnectionContext, Microsoft.AspNetCore.Connections.Abstractions, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].InitializeInstance()
|    at Moq.Mock`1[[Microsoft.AspNetCore.Connections.DefaultConnectionContext, Microsoft.AspNetCore.Connections.Abstractions, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].OnGetObject()
|    at Moq.Mock.get_Object()
|    at Moq.Mock`1[[Microsoft.AspNetCore.Connections.DefaultConnectionContext, Microsoft.AspNetCore.Connections.Abstractions, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].get_Object()
|    at Microsoft.AspNetCore.Server.Kestrel.Core.Tests.ConnectionDispatcherTests.OnConnectionCreatesLogScopeWithConnectionId() in /_/src/Servers/Kestrel/Core/test/ConnectionDispatcherTests.cs:line 33
|    at Xunit.Sdk.TestInvoker`1.<>c__DisplayClass48_0.<<InvokeTestMethodAsync>b__1>d[[Xunit.Sdk.IXunitTestCase, xunit.core, Version=2.4.2.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c]].MoveNext() in /_/src/xunit.execution/Sdk/Frameworks/Runners/TestInvoker.cs:line 264
| --- End of stack trace from previous location ---
|    at Xunit.Sdk.ExecutionTimer.AggregateAsync(Func`1 asyncAction) in /_/src/xunit.execution/Sdk/Frameworks/ExecutionTimer.cs:line 48
|    at Xunit.Sdk.ExceptionAggregator.RunAsync(Func`1 code) in /_/src/xunit.core/Sdk/ExceptionAggregator.cs:line 90
| [0.055s] TestLifetime Information: Finished test OnConnectionCreatesLogScopeWithConnectionId in 0.0478395s

Regression?

No response

Known Workarounds

No response

Configuration

.NET 8 RC1 cross built for the linux-ppc64 target (using the Mono runtime by default).

dotnet --info
.NET SDK:
 Version:   8.0.100-rc.2.23502.2
 Commit:    0abacfc2b6

Runtime Environment:
 OS Name:     rhel
 OS Version:  8
 OS Platform: Linux
 RID:         linux-ppc64le
 Base Path:   /root/ashutosh/aspnetcore-crossbuild/aspnetcore/.dotnet/sdk/8.0.100-rc.2.23502.2/

.NET workloads installed:
There are no installed workloads to display.

Host:
  Version:      8.0.0-rc.2.23479.6
  Architecture: ppc64le
  Commit:       0b25e38ad3

.NET SDKs installed:
  8.0.100-rc.2.23502.2 [/root/ashutosh/aspnetcore-crossbuild/aspnetcore/.dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 8.0.0-ci [/root/ashutosh/aspnetcore-crossbuild/aspnetcore/.dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 8.0.0-rc.2.23480.2 [/root/ashutosh/aspnetcore-crossbuild/aspnetcore/.dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 8.0.0-rc.2.23479.6 [/root/ashutosh/aspnetcore-crossbuild/aspnetcore/.dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
  None

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

Other information

No response

cc: @Janani66 @tmds @uweigand

About this issue

  • Original URL
  • State: closed
  • Created 8 months ago
  • Comments: 26 (26 by maintainers)

Commits related to this issue

Most upvoted comments

It’s a problem in the IMT builder. When build_imt_slots iterates over all the methods of a particular interface, it manually keeps track of the current vt_slot for the methods of that interface starting from interface_offset. The logic for incrementing vt_slot was incorrect: if there is a generic non-virtual method in the interface, the vt_slot was incremented when it should not have been. As a result the IMT builder added an incorrect implementing method for the IMT slot.

https://github.com/dotnet/runtime/blob/69702c372a051580f76defc7ba899dde8fcd2723/src/mono/mono/metadata/object.c#L1599-L1600

these two lines need to be under an if (m_method_is_virtual (method))

PR shortly.

I want to think a little bit harder whether we can make a test case - I’m not sure if we really depend on an IMT slot collision to exhibit the problem or if it will show up if I can make it show up even with a single method per IMT slot

+2
lambdageek on Nov 6, 2023

I’m not sure precisely what changed over there, but there were some interesting changes between previews 6 and 7, like https://github.com/dotnet/efcore/pull/31003 for example.

This is indeed the change that introduces the regression:

5ef57c1d3e72f16d4541f813f9b10dce56699bf1 is the first bad commit
+2
tmds on Oct 29, 2023

The backports for .NET 6, 7, 8 have been merged.

+1
lambdageek on Nov 13, 2023

Many of the test failures look like:

  Error Message:
   System.ArgumentNullException : Value cannot be null. (Parameter 'source')
  Stack Trace:
     at System.Linq.ThrowHelper.ThrowArgumentNullException(ExceptionArgument argument)
   at System.Linq.Enumerable.All[IReadOnlyForeignKey](IEnumerable`1 source, Func`2 predicate)
   at Microsoft.EntityFrameworkCore.Metadata.Conventions.ValueGenerationConvention.GetValueGenerated(IReadOnlyProperty property) in /home/tmds/repos/efcore/src/EFCore/Metadata/Conventions/ValueGenerationConvention.cs:line 193
   at Microsoft.EntityFrameworkCore.Metadata.Conventions.ValueGenerationConvention.GetValueGenerated(IConventionProperty property) in /home/tmds/repos/efcore/src/EFCore/Metadata/Conventions/ValueGenerationConvention.cs:line 185
   at Microsoft.EntityFrameworkCore.Metadata.Conventions.ValueGenerationConvention.ProcessEntityTypePrimaryKeyChanged(IConventionEntityTypeBuilder entityTypeBuilder, IConventionKey newPrimaryKey, IConventionKey previousPrimaryKey, IConventionContext`1 context) in /home/tmds/repos/efcore/src/EFCore/Metadata/Conventions/ValueGenerationConvention.cs:line 150
   at Microsoft.EntityFrameworkCore.Metadata.Conventions.Internal.ConventionDispatcher.ImmediateConventionScope.OnEntityTypePrimaryKeyChanged(IConventionEntityTypeBuilder entityTypeBuilder, IConventionKey newPrimaryKey, IConventionKey previousPrimaryKey) in /home/tmds/repos/efcore/src/EFCore/Metadata/Conventions/Internal/ConventionDispatcher.ImmediateConventionScope.cs:line 312
...

The IEnumerable that is null here is the one used at line 193:

=> property.GetContainingForeignKeys().All(fk => fk.IsBaseLinking())

It should never be null since the method is implemented as:

    public virtual IEnumerable<ForeignKey> GetContainingForeignKeys()
        => ForeignKeys ?? Enumerable.Empty<ForeignKey>();

So, what method is getting called here … 🤔

To call the method, there is some casting from one interface to another (as can be seen in the stacktrace above).

public class Property : PropertyBase, IMutableProperty, IConventionProperty, IProperty
{
...
    public virtual IEnumerable<ForeignKey> GetContainingForeignKeys()
        => ForeignKeys ?? Enumerable.Empty<ForeignKey>();

    IEnumerable<IReadOnlyForeignKey> IReadOnlyProperty.GetContainingForeignKeys()
        => GetContainingForeignKeys();

    IEnumerable<IMutableForeignKey> IMutableProperty.GetContainingForeignKeys()
        => GetContainingForeignKeys();

    IEnumerable<IConventionForeignKey> IConventionProperty.GetContainingForeignKeys()
        => GetContainingForeignKeys();

    IEnumerable<IForeignKey> IProperty.GetContainingForeignKeys()
        => GetContainingForeignKeys();
+1
tmds on Oct 30, 2023

Yea, rc2 mono with preview6 efcore - works. rc2 mono with preview7 efcore - exception.

I’m not sure precisely what changed over there, but there were some interesting changes between previews 6 and 7, like https://github.com/dotnet/efcore/pull/31003 for example.

So this is some behavior difference between Mono and CoreCLR that efcore stubmled on. Unfortunately debugging isn’t super helpful yet.

+1
lambdageek on Oct 27, 2023

@tmds Maybe #87406 /cc @ivanpovazan

I can take a look. Repro steps would be much appreciated.

We have built aspnetcore rc2 tag code using .NET8 rc2 SDK with mono and ran aspnetcore tests then observed these failures. To run single aspnetcore test which fails you can use below command

cd aspnetcore/src/Servers/Kestrel/Core/test
dotnet test --filter fullyQualifiedName~Microsoft.AspNetCore.Server.Kestrel.Core.Tests.ConnectionContextTests.ParameterlessAbortCreateConnectionAbortedException
+1
Sapana-Khemkar on Oct 26, 2023

@tmds Maybe https://github.com/dotnet/runtime/pull/87406 /cc @ivanpovazan

(I’m looking through https://github.com/dotnet/runtime/compare/v8.0.0-preview.6.23329.7...v8.0.0-preview.7.23375.6 )

@tmds you said you can repro the failure on x64? Could you share the repro steps

+1
lambdageek on Oct 25, 2023
Read more comments on GitHub
← runtime: aspnetcore-runtime-alpine3.7 (no libcurl)- The type initializer for 'Http' threw an exception - Error loading shared library libSystem.Net.Http.Native: No such file or directory
runtime: Assembly.GetType("System.Net.Http.HttpClientHandler", false, true) does not find type but finds it when ignoreCase is set to false →