gapic-showcase: HttpJson UpdateUserRequest's FieldMask is unable to be Unmarshalled

Java Gapic-Generator is implementing showcase coverage for the Identity client (modeled after golang). Draft PR with the full sample code: https://github.com/googleapis/gapic-generator-java/pull/1431

Relevant Code for the test case:

CreateUserRequest createUserRequest = CreateUserRequest.newBuilder()
            .setUser(User.newBuilder().setDisplayName("Jane Doe").setEmail("janedoe@example.com").setNickname("Doe").setHeightFeet(5).build()).build();
    User user = httpjsonClient.createUser(createUserRequest);
    User expected = createUserRequest.getUser();

    UpdateUserRequest updateUserRequest = UpdateUserRequest.newBuilder()
            .setUser(User.newBuilder()
                    .setName(user.getName())
                    .setDisplayName(user.getDisplayName())
                    .setEmail("janedoe@jane.com")
                    .setHeightFeet(6.0)
                    .setEnableNotifications(true)
                    .build())
            .setUpdateMask(FieldMask.newBuilder().addAllPaths(Arrays.asList("email", "height_feet", "enable_notifications")).build())
            .build();
    User updatedUser = httpjsonClient.updateUser(updateUserRequest);

Results in this error:

Caused by: com.google.api.client.http.HttpResponseException: 400 Bad Request
POST http://localhost:7469/v1beta1/users/1?updateMask=email,heightFeet,enableNotifications
{"error":{"code":400,"message":"error reading query params: terminal field \"updateMask\" of field path \"updateMask\" is of type \"message\" with value string \"email,heightFeet,enableNotifications\", which could not be parsed: unable to unmarshal field \"update_mask\" in message \"google.showcase.v1beta1.UpdateUserRequest\": proto: syntax error (line 1:1): invalid value email","details":null,"Body":"","Header":null,"Errors":null}}

Logging the HttpRequest:

-------------- REQUEST  --------------
POST http://localhost:7469/v1beta1/users/1?updateMask=email,heightFeet,enableNotifications
Accept-Encoding: gzip
User-Agent: Google-HTTP-Java-Client/1.43.0 (gzip)
x-http-method-override: PATCH
x-goog-api-client: gl-java/11.0.16.1 gapic/0.0.1-SHAPSHOT gax/2.23.3-SNAPSHOT rest/
Content-Type: application/json; charset=utf-8
Content-Length: 115

curl -v --compressed -X POST -H 'Accept-Encoding: gzip' -H 'User-Agent: Google-HTTP-Java-Client/1.43.0 (gzip)' -H 'x-http-method-override: PATCH' -H 'x-goog-api-client: gl-java/11.0.16.1 gapic/0.0.1-SHAPSHOT gax/2.23.3-SNAPSHOT rest/' -H 'Content-Type: application/json; charset=utf-8' -d '@-' -- 'http://localhost:7469/v1beta1/users/12?updateMask=email,heightFeet,enableNotifications' << $$$
Total: 115 bytes
{"name":"users/12","displayName":"Jane Doe","email":"janedoe@jane.com","heightFeet":6.0,"enableNotifications":true}

The logs seems to show that the updateMask field is encodedproperly as a query param, but Showcase is unable to decode this query param:

2023/03/07 10:16:40 error reading query params: terminal field "updateMask" of field path "updateMask" is of type "message" with value string "email,heightFeet,enableNotifications", which could not be parsed: unable to unmarshal field "update_mask" in message "google.showcase.v1beta1.UpdateUserRequest": proto: syntax error (line 1:1): invalid value email

Thought this might be an error with Java’s HttpJson implementation for FieldMask, but tested this with Java-Scheduler using HttpJson + FieldMask and this works properly.

-------------- REQUEST  --------------
POST https://cloudscheduler.googleapis.com:443/v1/projects/lawrence-test-project-2/locations/us-central1/jobs/72824232043972625121?$alt=json;enum-encoding%3Dint&updateMask=schedule,timeZone
Accept-Encoding: gzip
Authorization: Bearer {BEARER}
User-Agent: Google-HTTP-Java-Client/1.42.3 (gzip)
x-goog-user-project: {PROJECT}
x-http-method-override: PATCH
x-goog-api-client: gl-java/11.0.16.1 gapic/2.11.0 gax/2.23.1 rest/
Content-Type: application/json
Content-Length: 330

curl -v --compressed -X POST -H 'Accept-Encoding: gzip' -H 'Authorization: Bearer {BEARER}' -H 'User-Agent: Google-HTTP-Java-Client/1.42.3 (gzip)' -H 'x-goog-user-project: lawrence-test-project-2' -H 'x-http-method-override: PATCH' -H 'x-goog-api-client: gl-java/11.0.16.1 gapic/2.11.0 gax/2.23.1 rest/' -H 'Content-Type: application/json' -d '@-' -- 'https://cloudscheduler.googleapis.com:443/v1/projects/{PROJECT}/locations/{LOCATION}/jobs/{ID}?$alt=json;enum-encoding%3Dint&updateMask=schedule,timeZone' << $$$
Total: 330 bytes
{"name":"projects/{PROJECT}/locations/{LOCATION}/jobs/{ID}","httpTarget":{"uri":"http://example.com/","httpMethod":2,"headers":{"User-Agent":"Google-Cloud-Scheduler"}},"userUpdateTime":"2023-03-07T15:57:34Z","state":1,"schedule":"1 10 * * FRI","timeZone":"America/New_York","attemptDeadline":"180s"}

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 22 (15 by maintainers)

Most upvoted comments

Apologies for not seeing this issue sooner. I’m looking into this and seeing how to resolve it. I need to investigate more, but I am inclined to believe that Go’s implementation is indeed too strict given a related issue with query strings, the parsing of the semicolons (#1255 : the semicolon is no longer a query param separator, so sending it unencoded ought to have it parsed as just another character in the key or value, but Go assumes that one meant it as a separator and thus errors). Let me look some more into this particular error.

EDIT: To be clear, the two issues are in two distinct libraries, but they both affect query params, so maybe workarounds for both should occur in the same place. TBD.

Indeed @blakeli0 that’s what I gathered from the links Lawrence provided. This seems to be yet another discrepancy in how the proto-JSON libraries are implemented across languages. The fact that a value serialized by Java proto library cannot be deserialized by Go proto library makes proto-JSON not language-agnostic like the binary format is.

Can I ask, what serialization library is the Java implementation using for proto-to-JSON? I think specifically for the code that handles query parameters.

I just took a look and I believe we have our own internal Proto-to-Json Serializer via ProtoRestSerializer: https://github.com/googleapis/gapic-generator-java/blob/main/gax-java/gax-httpjson/src/main/java/com/google/api/gax/httpjson/ProtoRestSerializer.java

In the end, we use the JsonFormat provided by Protobuf for proto-to-JSON, see here regarding how we use it.