runtime: System.Text.Json deserializing of object[bool] does not produce boolean

I don’t really like this design. You are making the most common use cases for a parser an impenetrable mess to serve the general correctness. To serve the 2% edge cases, you are making this yet another unusable .NET Json API for the 98% use cases. You have an options bucket, use it to OPT into the generally correct, but unhelpful behavior, not have it be the default.

suggest to add a new strategy ：PreferRawObject:true to use default object mapping

Yes some global option is doable.

However, it is actually fairly easy to create a custom converter to handle the cases you show above. Earlier I provided the sample for bool and here’s a sample for bool, double, string, DateTimeOffset and DateTime. I will get the sample added to https://github.com/dotnet/runtime/blob/3e4a06c0e90e65c0ad514d8e2a9f93cb584d775a/src/libraries/System.Text.Json/tests/Serialization/CustomConverterTests.Object.cs#L267

private class SystemObjectNewtonsoftCompatibleConverter : JsonConverter<object>
{
    public override object Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
    {
        if (reader.TokenType == JsonTokenType.True)
        {
            return true;
        }

        if (reader.TokenType == JsonTokenType.False)
        {
            return false;
        }

        if (reader.TokenType == JsonTokenType.Number)
        {
            if (reader.TryGetInt64(out long l))
            {
                return l;
            }

            return reader.GetDouble();
        }

        if (reader.TokenType == JsonTokenType.String)
        {
            if (reader.TryGetDateTime(out DateTime datetime))
            {
                return datetime;
            }

            return reader.GetString();
        }

        // Use JsonElement as fallback.
        // Newtonsoft uses JArray or JObject.
        using (JsonDocument document = JsonDocument.ParseValue(ref reader))
        {
            return document.RootElement.Clone();
        }
    }

    public override void Write(Utf8JsonWriter writer, object value, JsonSerializerOptions options)
    {
        throw new InvalidOperationException("Should not get here.");
    }
}

Current functionality treats any object parameter as JsonElement when deserializing. The reason is that we don’t know what CLR type to create, and decided as part of the design that the deserializer shouldn’t “guess”.

For example, a JSON string could be a DateTime but the deserializer doesn’t attempt to inspect. For a “True” or “False” in JSON that is fairly unambiguous to deserialize to a Boolean, but we don’t since we don’t want to special case String or Number, we don’t want to make an exception for True or False.

With the upcoming preview 7, it is possible to write a custom converter for object that changes that behavior. Here’s a sample converter:

public class ObjectBoolConverter : JsonConverter<object>
{
    public override object Read(ref Utf8JsonReader reader, Type type, JsonSerializerOptions options)
    {
        if (reader.TokenType == JsonTokenType.True)
        {
            return true;
        }

        if (reader.TokenType == JsonTokenType.False)
        {
            return false;
        }

        // Forward to the JsonElement converter
        var converter = options.GetConverter(typeof(JsonElement)) as JsonConverter<JsonElement>;
        if (converter != null)
        {
            return converter.Read(ref reader, type, options);
        }

        throw new JsonException();

        // or for best performance, copy-paste the code from that converter:
        //using (JsonDocument document = JsonDocument.ParseValue(ref reader))
        //{
        //    return document.RootElement.Clone();
        //}
    }

    public override void Write(Utf8JsonWriter writer, object value, JsonSerializerOptions options)
    {
        throw new InvalidOperationException("Directly writing object not supported");
    }
}

Used like

var options = new JsonSerializerOptions();
options.Converters.Add(new ObjectConverter());

object boolObj = JsonSerializer.Parse<object>("true", options);
bool b = (bool)boolObj;
Debug.Assert(b == true);

object elemObj = JsonSerializer.Parse<object>(@"{}", options);
Debug.Assert(elemObj is JsonElement);

For those looking to convert JSON object to Hashtable, feel free to use my example. It should cover all JSON types. But, I have not tested it thoroughly, only for what we are needing it for. I have not used the write either, as we are only reading.

var options = new JsonSerializerOptions();
options.Converters.Add(new JsonHashtableConverter());
var obj = JsonSerializer.Deserialize<Hashtable>(object, options);


public class JsonHashtableConverter : JsonConverterFactory
{
	private static JsonConverter<Hashtable> _valueConverter = null;

	public override bool CanConvert(Type typeToConvert)
	{
		return typeToConvert == typeof(Hashtable);
	}

	public override JsonConverter CreateConverter(Type type, JsonSerializerOptions options)
	{
		return _valueConverter ?? (_valueConverter = new HashtableConverterInner(options));
	}

	private class HashtableConverterInner : JsonConverter<Hashtable>
	{
		private JsonSerializerOptions _options;
		private JsonConverter<Hashtable> _valueConverter = null;

		JsonConverter<Hashtable> converter
		{
			get
			{
				return _valueConverter ?? (_valueConverter = (JsonConverter<Hashtable>)_options.GetConverter(typeof(Hashtable)));
			}
		}

		public HashtableConverterInner(JsonSerializerOptions options)
		{
			_options = options;
		}

		public override Hashtable Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
		{
			if (reader.TokenType != JsonTokenType.StartObject)
			{
				throw new JsonException();
			}

			Hashtable hashtable = new Hashtable();

			while (reader.Read())
			{
				if (reader.TokenType == JsonTokenType.EndObject)
				{
					return hashtable;
				}

				// Get the key.
				if (reader.TokenType != JsonTokenType.PropertyName)
				{
					throw new JsonException();
				}

				string propertyName = reader.GetString();
				reader.Read();

				hashtable[propertyName] = getValue(ref reader, options);
			}
			return hashtable;
		}

		private object getValue(ref Utf8JsonReader reader, JsonSerializerOptions options)
		{
			switch (reader.TokenType)
			{
				case JsonTokenType.String:
					return reader.GetString();
				case JsonTokenType.False:
					return false;
				case JsonTokenType.True:
					return true;
				case JsonTokenType.Null:
					return null;
				case JsonTokenType.Number:
					if (reader.TryGetInt64(out long _long))
						return _long;
					else if (reader.TryGetDecimal(out decimal _dec))
						return _dec;
					throw new JsonException($"Unhandled Number value");
				case JsonTokenType.StartObject:
					return JsonSerializer.Deserialize<Hashtable>(ref reader, options);
				case JsonTokenType.StartArray:
					List<object> array = new List<object>();
					while (reader.Read() &&
						reader.TokenType != JsonTokenType.EndArray)
					{
						array.Add(getValue(ref reader, options));
					}
					return array.ToArray();
			}
			throw new JsonException($"Unhandled TokenType {reader.TokenType}");
		}

		public override void Write(Utf8JsonWriter writer, Hashtable hashtable, JsonSerializerOptions options)
		{
			writer.WriteStartObject();

			foreach (KeyValuePair<string, object> kvp in hashtable)
			{
				writer.WritePropertyName(kvp.Key);

				if (converter != null &&
					kvp.Value is Hashtable)
				{
					converter.Write(writer, (Hashtable)kvp.Value, options);
				}
				else
				{
					JsonSerializer.Serialize(writer, kvp.Value, options);
				}
			}

			writer.WriteEndObject();
		}
	}
}

suggest to add a new strategy ：PreferRawObject:true to use default object mapping

In most cases when I’ve seen this, it has been an anti-pattern. object is rarely what people want, and have seen far too many model.Foo.ToString() from people who didn’t realize they shouldn’t be using object.

See the SystemObjectNewtonsoftCompatibleConverter sample class in https://github.com/dotnet/runtime/blob/7eea339df0dab9feb1a9b7bf6be66ddcb9924dc9/src/libraries/System.Text.Json/tests/Serialization/CustomConverterTests.Object.cs#L267 for semantics similar to Json.NET (except for objects and arrays).

The JSON->POCO auto-generator in VS creates object members when it sees null, so this behavior breaks anyone migrating from such a thing.

But in most cases, this is again people just letting the default give them a bad model when they should be fixing what it gives them.

FWIW, using an object/dynamic property for message deserialization is a security consideration and should be handled with care:

https://www.alphabot.com/security/blog/2017/net/How-to-configure-Json.NET-to-create-a-vulnerable-web-API.html

FWIW the new JsonNode may help in certain scenarios since it at least allows explicit casts to primitives:

// If you can use JsonNode\JsonArray in the signature:
JsonArray nodes = JsonSerializer.Deserialize<JsonArray>("[1,1.1,\"Hello\"]");
long l = (long)nodes[0];
double d = (double)nodes[1];
string s = (string)nodes[2];

If you must have System.Object, you can still leverage nodes:

JsonSerializerOptions options = new();
options.UnknownTypeHandling = JsonUnknownTypeHandling.JsonNode;
object[] objects = JsonSerializer.Deserialize<object[]>("[1,1.1,\"Hello\"]", options);

long l = (long)(JsonNode)objects[0];
// or alternatively:
l = ((JsonValue)objects[0]).GetValue<long>();
double d = (double)(JsonNode)objects[1];
string s = (string)(JsonNode)objects[2];

As mentioned above, the current design avoids deserializing JSON string and bool to CLR string and bool because this would not be consistent with how JSON numbers are deserialized (since we don’t know the CLR number type and don’t want to “guess”).

Imagine having a CLR object[] property and if the JSON has mixed values (strings, bools, numbers), some array elements would be are CLR string, some bool and the numbers would be JsonElement – that would be more confusing IMHO. Also, Newtonsoft deserializes JSON string sometimes as a GUID or DateTime, which can also be confusing and\or unexpected.

So if we want to add a Newtonsoft custom converter that is possible, but we can’t change the default behavior in any case since that would break backwards compat.

@steveharter Json and Javascript don’t have many type, but boolean is known one. System.Text.Json should map json known’d type to clr type directly without custom converter