grape: Grape ~> 1.3.0 does not set Content-Length header anymore

EDITED grape < 1.3.0 sets Content-Length header grape ~> 1.3.0 does not set it anymore

the regression is related to the change in https://github.com/rack/rack/commit/8c62821f4a464858a6b6ca3c3966ec308d2bb53e

ORIGINAL POST There are some changes inroduced in 1.3.0 (and still valid for 1.3.2) that broke our production app after update.

We have app1 (Rails + Grape), app2 (Sinatra), frontend app. Frontend app communicates with app2 which in turn proxy requests to app1 using curb gem and just return responses. No specific magic here. When we used Grape 1.2.5 - everything was OK. But when we just changed version number in Gemfile, done bundle update grape and run the app with 1.3.0 - frontend app started to show infinite loader for requests.

While debugging I found that Grape API started to respond to curb gem request with a header - Transfer-Encoding chunked and without content-length so frontend app just waits while request finishes for a super-long time.

This changes that do the stuff introduced here - https://github.com/ruby-grape/grape/pull/1956

When I use Grape version before PR merged:

gem "grape", git: "https://github.com/ruby-grape/grape.git", ref: "9f786adbffa25574916e5d5504dacb4f5e79c6bb"

Everything works. Then I change it to (git commit when PR was merged):

gem "grape", git: "https://github.com/ruby-grape/grape.git", ref: "547bb71471a9273a4c2f8023332a0d5123d3eafa"

And app2 starts to receive response headers with Transfer-Encoding chunked and never finish the request. Should I create an issue for that? I don’t know if could create a reproducable example…

Reproducable example could be found here: https://gist.github.com/nbulaj/bd89ba366290033bf71c0f9b60927709

Also a possible reason of the regression posted here: https://github.com/ruby-grape/grape/issues/2030#issuecomment-624784743

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 18 (18 by maintainers)

Most upvoted comments

Thanks everyone for your comments!

dblock on May 8, 2020

@dm1try Sounds like we want a spec around Content-Length

I’m not sure if the framework should be responsible for this. Users always can include Rack::ContentLength middleware in their stack(btw, @nbulaj this is another recipe), explicitly.

So before there was a Content_Length header and our proxy client see requests finished (we have full body), but now we don’t have this one and it waits for streaming.

nope, now your proxy builds the wrong response mixing Transfer-Encoding: chunked and building the response as usual with Content-Length, it does not waits anything, the clients of the proxy wait for the chunked response, as your proxy says that the response is chunked, but the forwarded response is built as not-chunked one.

don’t sure how #1956 could break the headers

it does not break headers, it fixes the problem where grape < 1.3.0 does not work with rack => 2.1.0. But upgrading rack imply other changes that I already referenced. So if you downgrade rack to 2.0.8 you will see the old behavior; grape uses Rack::Response under the hood https://github.com/ruby-grape/grape/blob/002280415a46b1cabea565533141298781a48553/lib/grape/middleware/base.rb#L61 Rack::Response in rack 2.0.8 sets Content-Length Rack::Response in rack 2.1.0 does not set Content-Length because the interface was changed in https://github.com/rack/rack/commit/8c62821f4a464858a6b6ca3c3966ec308d2bb53e see https://github.com/ruby-grape/grape/issues/2051#issuecomment-624927490

there’s never been a specification for setting Content-Length header, so it does not look like a regression.

dm1try on May 7, 2020

I’ve updated the title and the original post to narrow down the problem. btw, there is no spec that grape should set Content-Length, so it should be discussed first)

dm1try on May 7, 2020

which version of rack had been used in the examples above?

This is gemfile.lock content:

    rack (2.2.2)
    rack-accept (0.4.5)
      rack (>= 0.4)
    rack-protection (2.0.8.1)
      rack
    rack-test (1.1.0)
      rack (>= 1.0, < 3)

it means any version of grape< 1.3 does not work with rack => 2.1. So upgrading grape to 1.3 forces you to upgrade rack to version ~> 2.1

Whooa, I thought for such cases there is gem.add_runtime_dependency 🤔

Anyway, your proxy client should support chunked transferring if it implements HTTP/1.1

The code you see in my gist is a production code. It really uses Curl::Easy to send the request to the main API and then just returns the response (body, status, headers) to the world (frontend app). The strange thing for me is that with 1.2.5 everything was OK and with the changes pointed above it doesn’t work anymore 😦

nbulaj on May 7, 2020