wire: Unable to decode proto3 message from Retrofit's response error body

Problem

Hey, I am trying to decode and create a new protobuf model, from the bytes located in Retrofit’s Response.errorBody().

Reproducing

The way I am doing this, is get an HttpException with status code 402 from our backend, containing a specific proto message in the error body. After accessing the errorBody() from the response, I have the following:

val errorBodyBytes = response?.errorBody()?.bytes()

The above in hex has the following value: 2a092801300138e8074801. Specifically I get a failure to decode the value in the bool e = 5; of the following proto message. It resolves the value of 40 when reading the next byte and fails to match that to a boolean in ProtoAdapter.COMMON_BOOL.

The proto3 message

message Subscription {
  bool a = 1;
  bool b = 2;
  bool c = 3;
  bool d = 4;
  bool e = 5;
  bool f = 6;
  int32 g = 7;
  int32 h = 8;
  bool i = 9;
}

Dependencies versions

Wire-runtime: 3.0.0-alpha03 Wire Gradle plugin: 3.0.0-alpha03 Retrofit: 2.6.0 Okio: 2.3.0-SNAPSHOT OkHttp: 4.0.0

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 19 (12 by maintainers)

Most upvoted comments

The schema you’re attempting to decode with is this:

message Subscription {
  bool a = 1;
  bool b = 2;
  bool c = 3;
  bool d = 4;
  bool e = 5;
  bool f = 6;
  int32 g = 7;
  int32 h = 8;
  bool i = 9;
}

But the correct schema for this message has an enclosing object:

message Box {
  Subscription subscription = 5;
}

message Subscription {
  bool a = 1;
  bool b = 2;
  bool c = 3;
  bool d = 4;
  bool e = 5;
  bool f = 6;
  int32 g = 7;
  int32 h = 8;
  bool i = 9;
}

You’re attempting to decode as a Subscription, but you should be attempting to decode as Box.

Sweet! Glad that Wire helped you discover an issue. I’m gonna close this ticket as it seems like there’s no action for us to take.

Yeah, the fact that protoc is so lenient is probably not a good thing, but in this case the only source of truth is code, and Wire’s is different 😃 Please keep us posted on your findings!

@swankjesse and I looked at it in more detail, and it turns out there’s difference in how protoc and Wire parse the example you provided:

  • protoc finds tag 5 with the LENGTH_DELIMITED type, and since it’s different from the type of the tag in the schema it’ll just put the value into the unknownFields. Decoding succeeds, but you have false values for all your boolean fields and almost the entire message inside unknownFields.
  • Wire finds tag 5 with the LENGTH_DELIMITED type, and since it’s different from the type of the tag in the schema, it crashes.

Although protoc does indeed succeed to parse the payload, since your schema is not consistent with the structure of the payload, you end up with an instance where all fields are initialized with false values. This looks like a bug that’s hard to identify.

Re-checked with this online proto decoder: https://protogen.marcgravell.com/decode According to its output, field 5 is of type String.

Could you please recheck two things:

  • That the bytes have been hexed correctly
  • That reverting to Wire 2 fixes the issue Wanna understand if it’s a regression from Wire 2, or an old issue we’ve had before (which seems unlikely).

Okay, so I tried decoding the hex using protoc:

echo 2a092801300138e8074801 | xxd -r -p | protoc --decode_raw

which outputs the following:

5 {
  5: 1
  6: 1
  7: 1000
  9: 1
}

If I’m reading this correctly, tag 5 contains an embedded message that in turn has what seems like boolean values for tags 5, 6, 9 and an int value for tag 7 - this looks consistent with the proto message above. That explains why decoding fails: when we’re decoding bytes against the proto above we’re expecting tag 5 to resolve to a boolean value, and the data suggests that it’s something different.

Yeap it is indeed proto3 as I mentioned on the title, and the exception you get is the correct one if you are building from master. I can attest that the data is not corrupt, since I tested it on another branch of our codebase that still uses the Google protobuf Java library and it is correctly parsed as the message I posted above.

Let me know if I can help in any other way!