runtime: Deserializing duplicate JSON object keys to a dictionary

Deserializing duplicate JSON object keys to a dictionary should be thought about. They’re not common, but they’re also not disallowed by the JSON spec.

In Newtonsoft.Json key/value pairs are set into the dictionary so that the last value in an object wins. This means dictionaries align with classes when it comes to duplicate object keys: the final result is the last value.

{
    "Hello": "friends",
    "Hello": "world"
}

var d = JsonSerializer.Parse<Dictionary<string, string>(json);
var target = d["Hello"]; // world

Throwing an error that there is a duplicate key would prevent someone from using the serializer. S.T.J should either follow “last value wins”, or have a setting to opt into it or out of it. Newtonsoft.Json has a setting like this: https://www.newtonsoft.com/json/help/html/T_Newtonsoft_Json_Linq_DuplicatePropertyNameHandling.htm

This should also be considered with JsonDocument.

My recommendation is to follow “last value wins” by default in the serializer and JsonDocument, and either add a setting now or in the future to allow people to be more strict (throw an error) about duplicate JSON object keys.

// @ahsonkhan @rynowak @steveharter @JeremyKuhne

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 1
Comments: 16 (15 by maintainers)

Most upvoted comments

Determine semantics of Json.NET for duplicate properties. Does it throw or allow?

Yes it does not throw, it follows last win approach

If it allows, for collections\dictionaries what semantics does it have? Does it replace, add or ignore the items on the extra collection\dictionary?

For collections it just replaces the collections and for dictionaries it concatenates the keys. the behavior is same for System.Text.Json

please run some perf numbers for the dictionary change

I ran the performance numbers. The change improves the performance by 5-8 percent for dictionaries and for object the change doesn’t affect the performance. (Even with the linq)

The size of the object and dictionary was 30 elements. Currently the json tests are broken in performance so I just wrote new ones to test the change.

Anipik on Jun 12, 2019

Trying to validate up front for duplicate property names and disallowing it would likely be too expensive (in terms of perf)

Yes - the code should be changed to use last-value-wins via TryAdd or an indexer (which doesn’t throw, unlike Add).

We could always add an opt-in option in the future to prevent duplicates.

steveharter on Jun 12, 2019