runtime: AssemblyLoadContext crash when collectible assembly use XmlSerializer

[EDIT] Investigation of this issue is complete and has resulted in 5 additional issues being opened. See this comment below for links to each of those issues.

We will keep this issue open and update it as those other issues are fixed. Please add any additional comments to those issues.

When creating a collectible AssemblyLoadContext and then use System.Xml.Serialization.XmlSerializer inside of a collectible assembly you will get a crash.

I have a repro project: https://github.com/uffebjorklund/AssemblyLoadContextBug

Have tried a few different things

If creating the ALC as collectible = false there is no crash
If the XmlSerializer code is removed from the collectible assembly there is no crash
The crash will occur regardless of using netstandard2.0 or netstandard2.1 on the collectible assembly

Exception

A non-collectible assembly may not reference a collectible assembly

Stacktrace

at System.Reflection.Emit.ModuleBuilder.GetTypeRef(QCallModule module, String strFullName, QCallModule refedModule, String strRefedModuleFileName, Int32 tkResolution)
at System.Reflection.Emit.ModuleBuilder.GetTypeRefNested(Type type, Module refedModule, String strRefedModuleFileName)
at System.Reflection.Emit.ModuleBuilder.GetTypeTokenWorkerNoLock(Type type, Boolean getGenericDefinition)
at System.Reflection.Emit.ModuleBuilder.GetTypeTokenInternal(Type type, Boolean getGenericDefinition)
at System.Reflection.Emit.ILGenerator.Emit(OpCode opcode, Type cls)
at System.Xml.Serialization.CodeGenerator.Castclass(Type target)
at System.Xml.Serialization.CodeGenerator.InternalConvert(Type source, Type target, Boolean isAddress)
at System.Xml.Serialization.SourceInfo.InternalLoad(Type elementType, Boolean asAddress)
at System.Xml.Serialization.XmlSerializationWriterILGen.WriteElement(SourceInfo source, ElementAccessor element, String arrayName, Boolean writeAccessor)
at System.Xml.Serialization.XmlSerializationWriterILGen.WriteElements(SourceInfo source, String enumSource, ElementAccessor[] elements, TextAccessor text, ChoiceIdentifierAccessor choice, String arrayName, Boolean writeAccessors, Boolean isNullable)
at System.Xml.Serialization.XmlSerializationWriterILGen.WriteMember(SourceInfo source, String choiceSource, ElementAccessor[] elements, TextAccessor text, ChoiceIdentifierAccessor choice, TypeDesc memberTypeDesc, Boolean writeAccessors)
at System.Xml.Serialization.XmlSerializationWriterILGen.GenerateTypeElement(XmlTypeMapping xmlTypeMapping)
at System.Xml.Serialization.XmlSerializationWriterILGen.GenerateElement(XmlMapping xmlMapping)
at System.Xml.Serialization.TempAssembly.GenerateRefEmitAssembly(XmlMapping[] xmlMappings, Type[] types, String defaultNamespace)
at System.Xml.Serialization.TempAssembly..ctor(XmlMapping[] xmlMappings, Type[] types, String defaultNamespace, String location)
at System.Xml.Serialization.XmlSerializer.GenerateTempAssembly(XmlMapping xmlMapping, Type type, String defaultNamespace, String location)
at System.Xml.Serialization.XmlSerializer..ctor(Type type, String defaultNamespace)

dotnet --info

.NET Core SDK (reflecting any global.json):
 Version:   3.0.100
 Commit:    04339c3a26

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.13
 OS Platform: Darwin
 RID:         osx.10.13-x64
 Base Path:   /usr/local/share/dotnet/sdk/3.0.100/

Host (useful for support):
  Version: 3.0.0
  Commit:  7d57652f33

.NET Core SDKs installed:
  2.1.301 [/usr/local/share/dotnet/sdk]
  2.1.302 [/usr/local/share/dotnet/sdk]
  2.1.400 [/usr/local/share/dotnet/sdk]
  2.1.401 [/usr/local/share/dotnet/sdk]
  2.1.403 [/usr/local/share/dotnet/sdk]
  2.1.500 [/usr/local/share/dotnet/sdk]
  2.2.100-preview3-009430 [/usr/local/share/dotnet/sdk]
  2.2.101 [/usr/local/share/dotnet/sdk]
  2.2.102 [/usr/local/share/dotnet/sdk]
  2.2.103 [/usr/local/share/dotnet/sdk]
  2.2.105 [/usr/local/share/dotnet/sdk]
  2.2.106 [/usr/local/share/dotnet/sdk]
  2.2.203 [/usr/local/share/dotnet/sdk]
  2.2.300 [/usr/local/share/dotnet/sdk]
  3.0.100-alpha1-009708 [/usr/local/share/dotnet/sdk]
  3.0.100-preview-010184 [/usr/local/share/dotnet/sdk]
  3.0.100-preview3-010431 [/usr/local/share/dotnet/sdk]
  3.0.100-preview8-013656 [/usr/local/share/dotnet/sdk]
  3.0.100-preview9-014004 [/usr/local/share/dotnet/sdk]
  3.0.100 [/usr/local/share/dotnet/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.1 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.2 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.5 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.6 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.0-preview3-35497 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.1 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.4 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 3.0.0-alpha1-10062 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.1 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.2 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.5 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.6 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.0-preview3-35497 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.1 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.4 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0-alpha1-10062 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0-preview-19075-0444 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0-preview3-19153-02 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0-preview8.19405.7 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0-preview9.19424.4 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.1 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.2 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.3 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.6 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.0-preview3-27014-02 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.1 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.3 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.4 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0-preview-27324-5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0-preview1-27029-03 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0-preview3-27503-5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0-preview9-19423-09 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 10
Comments: 55 (34 by maintainers)

Commits related to this issue

Initial loading and unloading of plugins. While this works, it cannot be merged in due to https://github.com/dotnet/runtime/issues/1388. Specifically, the Tiled plugin uses the `XmlSerializer` type,... — committed to vchelaru/FlatRedBall by KallDrexx 4 years ago
Initial loading and unloading of plugins. While this works, it cannot be merged in due to https://github.com/dotnet/runtime/issues/1388. Specifically, the Tiled plugin uses the `XmlSerializer` type,... — committed to vchelaru/FlatRedBall by KallDrexx 4 years ago

Most upvoted comments

Why has this bug not been fixed yet? It has been known since September 2019 … At that time the bug fix was moved to the .Net 5.0 framework. Now it is out and XML files still cannot be deserialized.

Here is a simple example that has crashed since 2019:

using System;
using System.IO;
using System.Xml.Serialization;

// This is the class that will be deserialized.
public class OrderedItem
{
    [XmlElement(Namespace = "http://www.cpandl.com")]
    public string ItemName;
    [XmlElement(Namespace = "http://www.cpandl.com")]
    public string Description;
    [XmlElement(Namespace="http://www.cohowinery.com")]
    public decimal UnitPrice;
    [XmlElement(Namespace = "http://www.cpandl.com")]
    public int Quantity;
    [XmlElement(Namespace="http://www.cohowinery.com")]
    public decimal LineTotal;
    // A custom method used to calculate price per item.
    public void Calculate()
    {
        LineTotal = UnitPrice * Quantity;
    }
}

public class Test
{
    public static void Main()
    {
        Test t = new Test();
        // Read a purchase order.
        t.DeserializeObject("simple.xml");
    }

    private void DeserializeObject(string filename)
    {
        Console.WriteLine("Reading with Stream");

        // Create an instance of the XmlSerializer.
        // NotSupportedException: A non-collectible assembly may not reference a collectible assembly.
        XmlSerializer serializer =
        new XmlSerializer(typeof(OrderedItem));

        // Declare an object variable of the type to be deserialized.
        OrderedItem i;

        using (Stream reader = new FileStream(filename, FileMode.Open))
        {
            // Call the Deserialize method to restore the object's state.
            i = (OrderedItem)serializer.Deserialize(reader);
        }

        // Write out the properties of the object.
        Console.Write(
        i.ItemName + "\t" +
        i.Description + "\t" +
        i.UnitPrice + "\t" +
        i.Quantity + "\t" +
        i.LineTotal);
    }
}

XmlSerializer.Deserialize Method

drocx on Nov 11, 2020

We have investigated and discussed this in our team and identified 4 issues. Two with XmlSerializerGenerator (one of which nobody has reported hitting but it’s in the same area of launching the tool), and two with XmlSerializer.
Due to lack of API support to generate a dynamic assembly in a specific ALC, we can’t get runtime serialization working until .NET 5 when hopefully we’ll have the API’s available. We are going to fix the pre-generated serializer scenario case as there are api’s available to control where an assembly is loaded. We’ll update this issue with links to each of the issues we’ll open so you can track the progress. The issues and initial changes will live in the new dotnet runtime repo as that’s where new changes are made, but as soon as we’ve got things working we will back-port to the servicing branch(s) in corefx and release a servicing fix.

mconnew on Dec 3, 2019

@drocx The creation of dynamic assemblies in the unloaded contexts, and therefore the work of the XML serializer, is fixed in PR #48072

Ap0k on Feb 16, 2021

@Ap0k if I understand correctly, it will be fixed in NET6; is there any chance to backport this fix to NET5?

lsoft on Feb 19, 2021

@uffebjorklund Thanks for reporting this, sorry for the slow response. It looks like all the information we need to do further investigation is here. We will look into it.

StephenBonikowsky on Nov 12, 2019

#48072 is not in .NET 6 Preview 1. It will be in Preview 2.

jkotas on Mar 5, 2021

I am looking forward to hear if this can be fixed in Net5.0. as this is keeping our migration from .Net Framework to .Net5.0 from moving forward.

Benjiinator on Feb 23, 2021

@lsoft Yes, the problem is only solved for NET6. Technically, these changes can be made in NET5 as well, but this is up to the repository owner. cc: @vitek-karas @jkotas

Ap0k on Feb 19, 2021

Thanks. I can repro now and it’s what I predicted above.

Caused by lack of /noconfig and /nostdlib in csc invocation
Repros in VS / msbuild
Does not repro with dotnet build

So I assume it’s using 3.1.100 SDK from inside VS to build.

It depends what you mean by using 3.1.100 SDK. VS will use tasks and targets from Microsoft.NET.Sdk (and friends), but it has its own copies of msbuild, roslyn, nuget (and friends) that run on .NET Framework. This particular issue repros only with the full framework csc, because it adds default references to full framework without /noconfig and /nostdlib. .NET Core csc doesn’t do that. The usual csc invocation passes /noconfig /nostdlib and is immune to this, but the custom csc invocation in sgen targets does not.

You can see the csc parameters using http://msbuildlog.com. There are no .NET Framework assemblies being passed to csc. It is missing /noconfig and /nostdlib

Another trick is to set the environment variable MSBUILDPRESERVETOOLTEMPFILES=1 to tell msbuild to keep the temporary *.rsp repsonse files in %TEMP%.

repro.rsp.txt

> csc @repro.rsp
Microsoft (R) Visual C# Compiler version 3.5.0-beta2-19577-04 (9b6c6104)
Copyright (C) Microsoft Corporation. All rights reserved.

error CS1703: Multiple assemblies with equivalent identity have been imported: 'C:\Windows\Microsoft.NET\Framework64\v4.0.30319\System.Data.DataSetExtensions.dll' and 'C:\Program Files\dotnet\packs\Microsoft.NETCore.App.Ref\3.1.0\ref\netcoreapp3.1\System.Data.DataSetExtensions.dll'. Remove one of the duplicate references.

> csc /noconfig /nostdlib @repro.rsp
Microsoft (R) Visual C# Compiler version 3.5.0-beta2-19577-04 (9b6c6104)
Copyright (C) Microsoft Corporation. All rights reserved.

nguerrera on Nov 28, 2019

I’m not trying to put blame on anybody 😄 I’m just pointing out potential issues. @nguerrera can you please comment on the presence of .NET Framework assemblies in assembly refs list for the compilers?

vitek-karas on Nov 27, 2019

@vitek-karas, the set of reference assemblies that is passed to csc when compiling the generated serializer is just copied from the list of reference assemblies passed to csc when compiling your application assembly. We’re not changing that list at all. If full framework assemblies are being added to the set of reference assemblies, something has changed in how the .NET Core SDK compiles that list. That’s a breaking change in the .NET SDK.

mconnew on Nov 27, 2019

I’m not an expert in the XmlSerializerGenerator, so hopefully others on this thread will be able to help with that aspect of the issue.

vitek-karas on Nov 26, 2019