gson: Enum serializes to null when nothing matches (instead of throwing IOException)

How about something like this:

@SerializedName(defaultValue = TrafficLight.RED) public enum TrafficLight { RED, ORANGE, GREEN }

// defaultValue will also allowed to be null // if this annotation is not present, enum maintains current behavior.

Comments @JakeWharton @swankjesse @joel-leitch?

You can always install your custom type adapter for your enum that enforces this. Gson isn’t going to support this, sorry.

Trying to understand. But how would this cause backward compatibility issues? The case I can think of were there might be a problem is when json data (newer version) containing a new enum field, would be read by a java implementation (old version) using gson. In that case this will throw an exception and not be compatible. But from the java point of view. I would call this not being forward compatible. The question is if that should be a problem. Because one can not be expected to be prepared for the future. So that should not be a limitation to fix this. Regarding ignoring additional field, input. I would consider this more of a change of input type then an additional field. For changed fields it would also throw exceptions If a field name changes. Also what if the type changes in the json data. From String to double that would cause an exception. But what if the type changed from enum to string that should cause an exception if it gets an new value… but it doesn’t it will simply return null for all the unknown enum fields. It also doesn’t ignore the input in the case of a new/wrong enum value. It just reads it with a different value, the value null. It can also be seen different from for example UUID reading. That would throw an IllegalArgumentException in case the UUID is incorrect. If it had the same behavior as the enum parsing it should then return null. One could argue that if UUID parsing could be a problem no UUID should be used. The same can be argued for enum field. If it might be a problem with new enum values. One should probably read strings and parse them in code, not let the gson library handle it. In general the gson library should not make the decision to interpret the input different then expected, but let the user of the library handle it and thus throw an IllegalArgumentException in this case.

Throwing an exception will cause backward compatibility issues. If you introduce a new enum value, you dont want all existing clients to break.

Even though this is relatively old stuff, in case somebody is looking for a solution for checking the missing enum values in a general way, a type adapter factory can be created to handle this:

public class CheckedEnumTypeAdapterFactory implements TypeAdapterFactory {

    @Override
    public <T> TypeAdapter<T> create(Gson gson, TypeToken<T> type) {

        if (!type.getRawType().isEnum()) {
            return null;
        }

        var delegate = gson.getDelegateAdapter(this, type);

        return new TypeAdapter<T>() {

            @Override
            public void write(JsonWriter out, T value) throws IOException {
                delegate.write(out, value);
            }

            @Override
            public T read(JsonReader in) throws IOException {
                var rawValue = in.nextString();

                var value = delegate.fromJsonTree(new JsonPrimitive(rawValue));

                if (value == null) {
                    throw new JsonParseException(String.format("Undefined enum value '%s' for enum '%s'", rawValue, type));
                }

                return value;
            }
        };
    }
}

Too bad that there are no option to decide to throw an exception in that case. If someone add an extra enum value, there won’t be an issue with the existing json which don’t use those value.

@bric3, that looks like a good workaround for specifying a default fallback value! But I think you can simplify / optimize the read function slightly:

- val rawValue = `in`.nextString()
- return delegate.fromJsonTree(JsonPrimitive(rawValue)) ?: candidates.single()
+ return delegate.read(`in`) ?: candidates.single()

Also, the following line looks wrong:

+ it.get(enumValue) == enumValue
- it.get(null) == enumValue

Your current code just happens to work because the documentation says if the field is static (which is the case for enum constants) the argument passed as obj (for your code that is enumValue) is ignored. Maybe you could also move this it.get call outside single { ... } since it does not have to be re-evaluated for every enum constant.

As small side note: Because your EnumWithUnknownValueTypeAdapterFactory is stateless, you could probably use a object declaration for it to make it a singleton.

Create this remapping type adapter factory when no value matches, but keep "null" or null as null.

Edited with suggestions from @Marcono1234.

import com.google.gson.Gson
import com.google.gson.JsonPrimitive
import com.google.gson.TypeAdapter
import com.google.gson.TypeAdapterFactory
import com.google.gson.reflect.TypeToken
import com.google.gson.stream.JsonReader
import com.google.gson.stream.JsonToken
import com.google.gson.stream.JsonWriter
import java.io.IOException

@Retention(AnnotationRetention.RUNTIME)
@Target(AnnotationTarget.FIELD)
annotation class UnknownValue

object EnumWithFallbackValueTypeAdapterFactory : TypeAdapterFactory {
    private val logger = thisLogger()

    override fun <T> create(gson: Gson, type: TypeToken<T>): TypeAdapter<T>? {
        if (!type.rawType.isEnum) {
            return null
        }

        val candidates = type.rawType.fields
            .asSequence()
            .filter { it.type == type.rawType && it.isAnnotationPresent(FallbackValue::class.java) }
            .map { field ->
                @Suppress("UNCHECKED_CAST")
                type.rawType.enumConstants.single { enumValue ->
                    field.get(null) == enumValue
                } as T
            }
            .toList()

        val delegate = gson.getDelegateAdapter(this, type)

        return when {
            candidates.isEmpty() -> delegate
            candidates.size > 1 -> throw IllegalArgumentException("Only one enum value can be annotated with @${FallbackValue::class.java.simpleName}")
            else -> {
                val fallbackValue = candidates.single()
                object : TypeAdapter<T>() {
                    @Throws(IOException::class)
                    override fun write(writer: JsonWriter, value: T) {
                        delegate.write(writer, value)
                    }

                    @Throws(IOException::class)
                    override fun read(reader: JsonReader): T {
                        // Keep null as undefined
                        if (reader.peek() == JsonToken.NULL) {
                            reader.nextNull() // consume
                            @Suppress("UNCHECKED_CAST")
                            return null as T
                        }
                        val rawString = reader.nextString()
                        val fromDelegate = delegate.fromJsonTree(JsonPrimitive(rawString))
                        return if (fromDelegate != null) {
                            fromDelegate
                        } else {
                            logger.debug { "Unknown enum value: $rawString for ${type.rawType.name}" }
                            fallbackValue
                        }
                    }
                }
            }
        }
    }
}

import com.google.gson.Gson
import com.google.gson.GsonBuilder
import org.assertj.core.api.Assertions.assertThat
import org.assertj.core.api.Assertions.assertThatCode
import org.junit.jupiter.api.Test
import org.junit.jupiter.params.ParameterizedTest
import org.junit.jupiter.params.provider.Arguments
import org.junit.jupiter.params.provider.Arguments.arguments
import org.junit.jupiter.params.provider.MethodSource
import java.util.stream.Stream
import kotlin.reflect.KClass

@Suppress("unused")
class EnumWithFallbackValueTypeAdapterFactoryTest {
    private val gson: Gson = GsonBuilder().registerTypeAdapterFactory(EnumWithFallbackValueTypeAdapterFactory).create()

    @Test
    fun `report if more than one enum value is annotated with unknown value`() {
        assertThatCode {
            gson.fromJson("\"VALUE\"", BadEnumWithMultipleUnknownValue::class.java)
        }.hasMessageContaining("Only one enum value can be annotated with @UnknownValue")
    }

    @ParameterizedTest
    @MethodSource("deserializeWithUnknownValues")
    fun deserialize(jsonStr: String, expected: Enum<*>?, enumClass: KClass<out Enum<*>>) {
        assertThat(gson.fromJson(jsonStr, enumClass.java)).isEqualTo(expected)
    }

    @Test
    fun serialize() {
        assertThat(gson.toJson(EnumWithUnknownValue.VALUE)).isEqualTo("\"VALUE\"")
    }

    enum class EnumWithUnknownValue {
        @UnknownValue
        UNKNOWN,
        VALUE
    }

    enum class RegularEnum {
        VALUE1,
        VALUE2,
    }

    enum class BadEnumWithMultipleUnknownValue {
        @UnknownValue
        UNKNOWN1,

        @UnknownValue
        UNKNOWN2,
        VALUE
    }

    companion object {
        @JvmStatic
        fun deserializeWithUnknownValues(): Stream<Arguments> = Stream.of(
            arguments("\"VALUE\"", EnumWithUnknownValue.VALUE, EnumWithUnknownValue::class),
            arguments("\"NEW_VALUE\"", EnumWithUnknownValue.UNKNOWN, EnumWithUnknownValue::class),
            arguments("null", null, EnumWithUnknownValue::class),
            arguments("", null, EnumWithUnknownValue::class),

            arguments("\"VALUE1\"", RegularEnum.VALUE1, RegularEnum::class),
            arguments("\"NEW_VALUE\"", null, RegularEnum::class),
            arguments("", null, RegularEnum::class),
        )
    }
}

Here another example how to apply a default value instead of null. (Kotlin-Code)

class AdressklassenDeserializer: JsonDeserializer<AdressklasseCode> {

    override fun deserialize(
        json: JsonElement,
        typeOfT: Type,
        context: JsonDeserializationContext
    ): AdressklasseCode {
        val str = json.asString

        return try {
            AdressklasseCode.valueOf(str)
        } catch (ex: Exception) {
            AdressklasseCode.UNKNOWN
        }
    }
}

val gson = GsonBuilder()
    .registerTypeAdapter(AdressklasseCode::class.java, AdressklassenDeserializer())
    .create()

I agree it would be great to make the behavior configurable - both globally and on enum level (via annotation).

On enum level there could be 3 options:

• throw exception
• parse as null
• set a default value from the existing enum values

The problem is that the adapter makes no difference between an input value: null and a invalid enum value (for example a typo in the enum value). Both return null. While the first is valid, the second is invalid. Thus the adapter returns the same kind of data for both valid and invalid input. It’s not possible to know if the input was valid or invalid without performing an extra additional check if the input was null in case the result is null. I don’t think that is correct behavior and should not be solved by writing a custom adapter. Shouldn’t it throw an IllegalArgumentException when an invalid enum value is given?

It is debatable what the correct behavior is. Gson in general ignores 
additional fields or input.

You can easily write a custom type adapter for your enum to assert the correct 
behavior.

Original comment by inder123 on 12 Nov 2014 at 10:16

• Changed state: WontFix