swift-corelibs-foundation: [SR-9232] Optionals with nil value do not get encoded to JSON

Previous ID SR-9232
Radar None
Original Reporter Peter de Kraker (JIRA User)
Type Bug
Additional Detail from JIRA
Votes 22
Component/s Foundation
Labels Bug, Codable
Assignee bendjones (JIRA)
Priority Medium

md5: d40e1037406253fafe704bc8c33c89d9

Issue Description:

The implementation of Codable and JSONEncoder in Swift 4 do automatically discard optional fields that have nil as value.

I can understand that this is an option, but there is no option to be set and that makes it a big problem for working with JSON API’s that expect nulls for nil fields.

I don’t really understand why this default is chosen, since removing nulls from JSON is easier afterwards, then adding them manually by needing to write the encode() method for each class.

Anyways, it would be great if an option can be provided to automatically encode nil values to null with JSONEncoder.

class Test : Codable
{
    var optional:Int?
}


let test:Test = Test()
test.optional = nil


let data = try! JSONEncoder().encode(test)
let string = String(data: data, encoding: String.Encoding.utf8)!
print(string) -> "{}"

About this issue

  • Original URL
  • State: open
  • Created 6 years ago
  • Reactions: 5
  • Comments: 17 (11 by maintainers)

Most upvoted comments

I really hope this gets added as a JSONEncoder option like encoder.nilEncodingStrategy = .includeKeyWithJsonNull but have the default as .omitKey to not introduce a breaking change. A few server API’s I interface with treat missing keys and Json null differently, so for every model (and there are a lot) I have to implement the CodingKeys enum, then override the encode property for every model property. This produces a ton of boiler plate code. Please consider adding this option. Thank you!

Comment by Steven Grosmark (JIRA)

As a work-around, I’ve been using a property wrapper, along the lines of:

@propertyWrapper
public struct NullCodable<Wrapped: Encodable>: Encodable {
  public var wrappedValue: Wrapped?

  public func encode(to encoder: Encoder) throws {
    var container = encoder.singleValueContainer()
    switch wrappedValue {
    case .some(let value): try value.encode(to: encoder)
    case .none: try container.encodeNil()
  }
}

struct Test: Encodable {
  @NullCodable var name: String? = nil
}

JSONEncoder().encode(Test()) // -> "{\"name\": null}"

Not ideal, since each optional property needs to be marked, but it is a little easier than trying to implement a custom encoder.

I created a small package for it here: NullCodable.

Comment by Marcin Iwanicki (JIRA)

There are pros and cons of both cases.

Omitting optional properties results in a smaller payload. JSON format is frequently used in the networking layer as such the size of the payload matters.

I think it would be great to preserve the current behavior as default and avoid breaking changes. Adding a new property e.g. nilEncodingStrategy with options like skip (default), and includeNulls would work well, and be consistent with other options e.g. dataEncodingStrategy, dateEncodingStrategy.
As an alternative, it could possibly leverage the existing outputFormatting property with a new flag e.g. withNulls.

Comment by Sergey Balalaev (JIRA)

I think it’s would be better, if we take a option as DateEncodingStrategy something like that: OptionalEncodingStrategy.

This actually just caused us a pretty nasty bug. In our case we had a completed_at timestamp that’s optional. The iOS app set it to null and did an API call to mark a record as not complete (think, unchecking a checkbox). We found that while the local SQLite in the app had a null completed_at, the server still had the old timestamp. Then we realized the key was being omitted.

The problem is, omitting the key conflicts with Ruby on Rails’ Strong Parameters, which take the parameters provided in the request and then filter them through a whitelist. When used in conjunction with ActiveRecord’s update method, omitted fields are treated as “no change” for an update.

Including the optional null keys in the JSON is helpful for a few reasons:

  1. If clients are only sending fields that changed (an even smaller payload, as you mentioned payload size), there’s less problems with stale reads writing back outdated values (2 people editing different fields of the same record or something). This isn’t possible if omitted fields are treated as an explicit null.
  2. As I add more fields to my API, I do not want old versions of our app to wipe out fields that it does not even know exist. If these null optional fields were included in the JSON, I would know that the app does know that this field exists, and is setting it to null. Right now we’re forced to change a lot of JSON Encoders in our iOS app, or introduce forward-compatibility problems in our API by treating omitted keys as explicit nulls.

Comment by Jon M Snyder (JIRA)

The difference between a client of an API knowing about something and setting it to null and not including the key at all can be important in many situations. Being able to still leverage all the goodness of Encodable and just tweak this behavior would be wonderful.