grpc-gateway: 404s using colons in the middle of the last path segment

I have the following endpoint and user_ids of the format “user:<number>”

  rpc GetUser(GetUserRequest) returns (GetUserResponse) {
    option (google.api.http) = {
      get: "/v0/users/{user_id}"
    };
  };

I’m getting 404 status code and it doesn’t hit my GetUser handler when I call this endpoint with user_ids of the format above. However, if I call the endpoint with a user_id that doesn’t have a colon it (e.g. “user123”) it does hit my GetUser handler. Url encoding the colon doesn’t work either.

What’s even more interesting is if I add another colon to the end of the path, grpc-gateway parses the user_id into the correct format that I want. (“/v0/users/user:123:” -> user_id = “user:123”). Given this data point, I believe grpc-gateway is treating colons differently in the last path segment.

For example here: https://github.com/grpc-ecosystem/grpc-gateway/blob/master/protoc-gen-grpc-gateway/httprule/parse.go#L86 it strips off the last colon in the segment. Though this looks like the rule for parsing the endpoint path definition and not what is being used to parse incoming requests.

About this issue

  • Original URL
  • State: closed
  • Created 8 years ago
  • Comments: 35 (3 by maintainers)

Commits related to this issue

Most upvoted comments

I was able to get around this by wrapping the mux itself with a hack handler that appends a : to the request path if there’s none, because all my ids are urns:

func wrapMux(h http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		if !strings.HasSuffix(r.URL.Path, "/") && !strings.HasSuffix(r.URL.Path, ":") {
			r.URL.Path += ":"
		}
		h.ServeHTTP(w, r)
	})
}

Confirmed that both paths like /users and /users/urn:user:123 work

Old problems require modern solutions. You can add allow_colon_final_segments=true to your grpc-gateway_out in the generate file. I.e. –grpc-gateway_out=logtostderr=true,allow_colon_final_segments=true:generated
This works with protoc2:2.0.0 >

No worries, @johanbrandhorst. I do require this functionality, so let’s talk about it. I actually mentioned this backcompat issue in my pull request description. I should have forced the discussion there, though. I’ve repeated the relevant part of the PR description below.

To remark quickly on #760, it’s actually the exact example from the description of my PR 😬. Re your comment on #760:

It also seems like the wrong thing to support this tbh, I’m just disappointed we didn’t discover it earlier and didn’t break users.

I tried to make a case in #708 that the fix was in fact a step towards the letter of the spec — I still think that’s right. The problem isn’t that #708 prevents custom verbs from working, it’s that the order of the rules is such that grpc-gateway prefers the verbless match. If we fixed grpc-gateway to implement the specified “last one wins” behavior (see below), then #760 would work as expected.

I think the best solution would be to put “last one wins” behavior and the fix from #708 behind a generator flag — this maintains backwards compatibility and makes available (what I think is) the correct rule-matching behavior.

PR Description

There is a backwards compatibility issue here: this can change the behavior of existing rule sets. For example:

    option(google.api.http) = {
      get: "/users/{user_id}"
    };
    option(google.api.http) = {
      get: "/users/{user_id}:averb"
    };

Before this change, only the second rule matched a request for /users/123:averb. After this change, both rules match (correctly, by my understanding of https://github.com/googleapis/googleapis/blob/master/google/api/http.proto). Note that the spec describes a deterministic way of breaking the tie: // **NOTE:** All service configuration rules follow "last one wins" order. grpc-ecosystem/grpc-gateway’s implementation looks buggy on this score, since it accepts the first matching handler.

@kassiansun as discussed in #760, we’re likely to revert the fix for this change so you can use :verb for custom verbs as you say.

I have the same issue, because of the usage of URN as User ID. Not ID with bare colon, nor url-encoded one doesn’t match the route and results in 404.

Middleware suggested by @Azuka helps a lot, but, to my mind, this case should be considered and handled in grpc-gateway internally.

I confirm that the grpc-gateway_out argument allow_colon_final_segments=true works well with the latest v2.

It’s a good enough workaround for my use case, but I think it would be nice to have this as some kind of option to the gateway runtime mux server.

Thanks @jfhamlin it’s clear us maintainers didn’t understand the full situation. You’ve made a good case for introducing this as a flag, as you say, since we can’t break backwards compatibility, even for a case where the gateway is doing the wrong thing. Another thing to add to the list of our v2 design changes.

Would you be interested in drafting a PR to get this in?