armeria: [gRPC client question] how to compute checksum and add it to request header

Setup

I am using Armeria version: 1.16.0 and trying to build a feature on the gRPC client to compute a checksum and send it to the server side for validation.

Requirement

Specifically, I am trying to make the client compute a checksum from the serialized request (Protocol Buffer object) bytes and then put the computed checksum in the request header. On the server side, before it deserializes received request bytes, it computes another checksum from received bytes, extracts the client-side-computed checksum from the header, and compares 2 checksums.

On the client side, I tried to use a decorator as shown below:

clientBuilder.decorator((delegate, ctx, req) => {
    val newReq = req.mapData(data => {
        val checksum = // Compute checksum
        ctx.addAdditionalRequestHeader(someKey, checksum) // Try to put the checksum in the request header
        return data
    })
    delegate.execute(ctx, newReq)
})

Problem

In the above code example, I think the mapData callback is executed after the request headers have been sent so that ctx.addAdditionalRequestHeader(...) is actually an no-op. As a result the server side receives a request header without the someKey, checksum entry in it. I verified this behavior by using my (IntelliJ) debugger.

I have also tried a few other approaches (still using the decorator). For example, one other thing I tried is to use:

ctx
 .log()
 .whenAvailable(RequestLogProperty.XXX)
 .thenAccept(/* compute the checksum here */)

The problem here is that there is no RequestLogProperty.XXX that maps to serialized/binary request content. The closest I could find is RequestLogProperty.REQUEST_CONTENT. However, a Protocol Buffer object (instead of its serialized bytes) is passed to the thenAccept callback. In order to compute checksum, I need serialized bytes instead of a Protocol Buffer object. Technically I can serialize the object inside the thenAccept callback and then compute the checksum. But it is a waste of CPU to serialize each request twice >.<

Question

  1. Are my observations and understanding above (in the Problem section) correct?
  2. What is the best way to implement this feature (mentioned in the Requirement section)?

Thanks!

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 18 (10 by maintainers)

Most upvoted comments

  1. IIUC this new approach (appending checksum bytes at the end of each HTTP request body bytes) is more efficient than the first approach which uses req.aggregate(), right? I think using req.aggregate() delays the sending of HTTP request headers to the server side because it has to wait for the HTTP request body as well.

It depends on ClientRequestContext.exchangeType(). If ExchangeType.UNARY or ExchangeType.RESPONSE_STREAMING is used or inferred, the request is aggregated always by AggregatedHttpRequestHandler. As a result, req.aggregate() in a decorator with may have a small amount of overhead with non-streaming requests. The aggregation cost just moves to the decorator. https://github.com/line/armeria/blob/70296b99b4277eef75da93ec85eeb26beec8d39e/core/src/main/java/com/linecorp/armeria/client/HttpSessionHandler.java#L197-L210

As per https://www.ietf.org/archive/id/draft-ietf-httpbis-digest-headers-10.html#name-client-and-server-provide-f, it sends a digest field in headers so it makes more sense to compute a hash of data and attach the value to the headers.

  1. Does this approach work with gRPC streaming request? I guess it should work if we append a 4-byte checksum to each streamed message in the streaming request and the server parses out the last 4 bytes of each received message as a checksum.

As you know, it is a custom protocol that could be used only with custom clients and servers that are able to interpret the format. I recommend set a digest field to the trailers for the entire data when a streaming request is used. It does not transform the original content so some arbitrary servers that do not know the format may ignore digest headers. https://www.ietf.org/archive/id/draft-ietf-httpbis-digest-headers-10.html#name-use-with-trailer-fields-and

Yeah, please let us know if you have more questions. 😄

Got it, thanks @minwoox !

If you don’t mind, I will keep this issue open for 1 more week in case I have more questions. Otherwise, I will close it in a week.