runtime: Undocumented breaking change in .NET 5.0 - HashSet moved to another assembly/library

Hello everyone, I’ve stumbled upon a breaking change which as far as I’ve searched for was not documented anywhere. It seems that with that pull request https://github.com/dotnet/runtime/pull/37180 HashSet was moved from System.Collections to System.Private.CoreLib library. Let’s say we have a fair big distributed system based on microservices with an immutable events used for a synchronization between them which are being stored forever. Newtonsoft.Json is being used as a serializer/deserializer. The problem with that breaking change is like in the example below: that’s an example of class which was serialized in a netcoreapp3.1 version (with a collection in a dictionary’s value being HashSet):

public class TestClass
    {
        public IReadOnlyDictionary<string, IReadOnlyCollection<string>> Dictionary { get; }

        public TestClass(IReadOnlyDictionary<string, IReadOnlyCollection<string>> dictionary)
        {
            Dictionary = dictionary;
        }
    }

resulting in (with a collection in a dictionary’s value being HashSet)

{
  "Dictionary": {
    "DictionaryKey": {
      "$type": "System.Collections.Generic.HashSet`1[[System.String, System.Private.CoreLib]], System.Collections",
      "$values": [
        "string value"
      ]
    }
  }
}

whereas in net5.0 it is being serialized as

{
  "Dictionary": {
    "DictionaryKey": {
      "$type": "System.Collections.Generic.HashSet`1[[System.String, System.Private.CoreLib]], System.Private.CoreLib",
      "$values": [
        "string value"
      ]
    }
  }
}

The TypeNameHandling is set to Auto.

The problem of couse occurs when trying to deserialize old-netcoreapp3.1 events/messages into the application running on net5.0

My questions are basically:

  • are there any other such an undocumented breaking changes, like any other types were moved between libraries?
  • are you aware that a mentioned change could have caused some serious implications and have some kind of solution or workaround for that?
  • @JamesNK maybe you’d have some idea, how could we disable the type name handling for the dictionaries and collections from a framework, so upon deserialization the $type would be ignored for those types?

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 2
  • Comments: 22 (20 by maintainers)

Most upvoted comments

If it isn’t, I think it should be treated as such.

In public API, it stays remaining in the original assmebly. Serialization using reflection is getting implementation detail.

For example, there is no System.Private.CoreLib in public API. There is only System.Runtime.

Implementation detail can be broken in any release.

The type has moved many times already. Serializers should ideally respect the TypeForwarded**From** attribute on the type and emit that as the owning assembly:

https://github.com/dotnet/runtime/blob/0df028bfd9ce57b045af4dc13ab8df5a66423ffa/src/libraries/System.Private.CoreLib/src/System/Collections/Generic/HashSet.cs#L16

Libraries have type forwarders (using the above mentioned TypeForwardedTo) to ensure apps looking for HashSet in System.Core will find it. This way the serialized data would be compatible all the way to .NET Framework 2.0.

“Binary serialization” is the wrong technical term, but it includes the umbrella of “serializers which embed type information in the payload,” so the general concept is applicable here. That’s also what I was trying (poorly) to say earlier w.r.t. Microsoft not investing in this category of serialization tech any further.

Moving types among assemblies isn’t treated as breaking change for common cases. There’s TypeForwardedTo added.

If it isn’t, I think it should be treated as such. As per SemVer specification:

Major version X (X.y.z | X > 0) MUST be incremented if any backwards incompatible changes are introduced to the public API.

And of course assembly name for the given type is the public API.

And my question (let me quote myself) was regarding any other such a breaking changes for .NET 5.0 release:

are there any other such an undocumented breaking changes, like any other types were moved between libraries?