runtime: JsonConverter constructor causes MissingMetadataException upon instantiation

Description

We’re currently working on migrating all our JSON serialization from Newtonsoft.Json to System.Text.Json in the Microsoft Store (see also #77897), and we’re hitting some issues with trimming (we’re on .NET Native). In particular, this line:

https://github.com/dotnet/runtime/blob/264d7391ec9f6e698051db0621c5e090d0ae4710/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverter.cs#L19

This is crashing when trimming is enabled, because the linker will remove support for getting the assembly info from types. We can fix this by adding some .rd.xml directives, but it’s error prone and not really a great solution. Eg. we can use:

<Type Name="System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1" Activate="Required Public" />
<Type Name="System.Text.Json.Serialization.Converters.DictionaryOfTKeyTValueConverter`3" Activate="Required Public" />
<Type Name="System.Text.Json.Serialization.Converters.ListOfTConverter`2" Activate="Required Public" />

Etc. for all converters we need. It’d be much better if this was just fixed in System.Text.Json directly. I’m aware that reflection-free mode isn’t supported (see #68093), but fixing this would also benefit other scenarios (such as our case) by still allowing the linker to just trim out more metadata and reduce the binary size further.

Note: to clarify, the ask is not to support the reflection-free mode, just to make this path friendlier to trimming.

cc. @eiriktsarpalis @MichalStrehovsky

Reproduction Steps

The repro is pretty much the same as in the linked issue:

string json = """
    {
        "SomeMapping": { "A": "B" },
        "SomeList": ["A", "B"]
    }
    """;

_ = System.Text.Json.JsonSerializer.Deserialize(rawJson, MicrosoftStoreJsonSerializerContext.Default.SomeModel);

public sealed class SomeModel
{
    public Dictionary<string, string> SomeMapping { get; set; }
    public List<string> SomeList { get; set; }
}

[JsonSerializable(typeof(SomeModel))]
[JsonSourceGenerationOptions(GenerationMode = JsonSourceGenerationMode.Metadata)]
internal sealed partial class MicrosoftStoreJsonSerializerContext : JsonSerializerContext
{
}

Expected behavior

This should just work fine.

Actual behavior

We’re getting a MissingMetadataException:

at System.Reflection.Runtime.TypeInfos.RuntimeNoMetadataNamedTypeInfo.get_Assembly() in f:\\dd\\ndp\\fxcore\\CoreRT\\src\\System.Private.Reflection.Core\\src\\System\\Reflection\\Runtime\\TypeInfos\\RuntimeNoMetadataNamedTypeInfo.cs:line 38 
at System.Reflection.Runtime.TypeInfos.RuntimeConstructedGenericTypeInfo.get_Assembly() in f:\\dd\\ndp\\fxcore\\CoreRT\\src\\System.Private.Reflection.Core\\src\\System\\Reflection\\Runtime\\TypeInfos\\RuntimeConstructedGenericTypeInfo.cs:line 136
at System.Text.Json.Serialization.JsonConverter`1..ctor(Boolean initialize) 
at System.Text.Json.Serialization.JsonConverter`1..ctor()
at System.Text.Json.Serialization.JsonResumableConverter`1..ctor() 
at System.Text.Json.Serialization.JsonObjectConverter`1..ctor()
at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1..ctor()
at System.Text.Json.Serialization.Metadata.SourceGenJsonTypeInfo`1.<>c.<GetConverter>b__3_1()
at System.Func`1.Invoke()
at System.Text.Json.Serialization.Converters.JsonMetadataServicesConverter`1.get_Converter() 
at System.Text.Json.Serialization.Converters.JsonMetadataServicesConverter`1.get_ElementType() 
at System.Text.Json.Serialization.Metadata.JsonTypeInfo..ctor(Type type, JsonConverter converter, JsonSerializerOptions options)
at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1..ctor(JsonConverter converter, JsonSerializerOptions options) 
at System.Text.Json.Serialization.Metadata.SourceGenJsonTypeInfo`1..ctor(JsonSerializerOptions options, JsonObjectInfoValues`1 objectInfo)
at System.Text.Json.Serialization.Metadata.JsonMetadataServices.CreateObjectInfo[T](JsonSerializerOptions options, JsonObjectInfoValues`1 objectInfo) 
at SomeProject.MicrosoftStoreJsonSerializerContext.Create_SomeModel(JsonSerializerOptions options)
at SomeProject.MicrosoftStoreJsonSerializerContext.get_SomeModel()  
at SomeProject.<<TryDeserializeSomeModel>g__Foo|77_0>d.MoveNext() 

Regression?

I have a possible idea on how to fix this, by making that path entirely reflection-free. Consider this:

public abstract class JsonConverter<T>
{
    protected JsonConverter()
    {
        IsInternalType = CheckIsInternalType();
    }

    private protected virtual bool CheckIsInternalType() => false;
}

Now, all converter types in System.Text.Json would just override the method accordingly:

public class SomeSealedJsonConverter<T> : JsonConverter<T>
{
    private protected override bool CheckIsInternalType() => true;
}

public class SomeUnsealedJsonConverter<T> : JsonConverter<T>r
{
    private protected override bool CheckIsInternalType() => GetType() == typeof(SomeUnsealedJsonConverter<T>);
}

This makes sure that:

  • External converters directly inheriting from JsonConverter<T> will be marked as external.
  • External converters that inherit from unsealed STJ converters will also be marked as external.

Essentially this should provide a reflection-free way of checking whether a concrete converter type is from the STJ assembly.

Configuration

  • System.Text.Json 7.0
  • .NET Native 6.2.14

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 19 (18 by maintainers)

Most upvoted comments

One thing I also want to mention, System.Text.Json apparently treats List<A<B>> as requiring converter JsonConverter<List<A<B>>>, which has its own reflection code: https://github.com/dotnet/runtime/blob/ec9fb02a2c6b606ef06acc911a0b104fd3d2a9a3/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.cs#L22 That means Sergio’s previously mentioned workaround to add possibly every converter System.Text.Json implements needs to be changed to

<Type Name="System.Text.Json.Serialization.JsonConverter`1">
    <Subtypes Activate="Required Public" />
</Type>

for this scenario to work.

Great to see this being added to the new AOT user story for .NET 8! 🎉

@eiriktsarpalis should we add the partner-impact tag here too, since we’re hitting this in the Store? This specific reflection dependency being addressed in the next release would be a pretty nice win for us there 😄

That sounds like an even better idea 😄 Just glancing at the code it’s not immediately obvious to me what this property is even for (there’s also no comments).

Out of curiosity - I know the reflection-free mode isn’t supported in NativeAOT, but would the same still apply to the “enhanced reflection free” mode that was in the works?

The compiler doesn’t generate such data structures and I deleted the representation for this from the reflection stack in https://github.com/dotnet/runtime/pull/73612.

That doesn’t seem to be enough, I tried but unfortunately I still got MissingMetadataException-s with that 🥲