httpbin: Regression with chunked transfer encoding?

I’m using https://httpbin.org and it looks like chunked transfer encoding does not work any more as expected:

$ curl -X POST https://httpbin.org/post \
 -d '{"message":"BLA"}' \
 -H 'Content-Type: application/json' \
 -H 'Transfer-Encoding: chunked'
{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Connect-Time": "1", 
    "Connection": "close", 
    "Content-Type": "application/json", 
    "Host": "httpbin.org", 
    "Total-Route-Time": "0", 
    "Transfer-Encoding": "chunked", 
    "User-Agent": "curl/7.50.1", 
    "Via": "1.1 vegur", 
    "X-Request-Id": "2a06bd5b-06dc-4ced-ac89-fe83cbb6771c"
  }, 
  "json": null, 
  "origin": "<removed>", 
  "url": "https://httpbin.org/post"
}

Observe that data and json do not contain the posted data. The same request works as expected without chunked transfer encoding:

$ curl -X POST https://httpbin.org/post \
 -d '{"message":"BLA"}' \
 -H 'Content-Type: application/json'
{
  "args": {}, 
  "data": "{\"message\":\"BLA\"}", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Connect-Time": "0", 
    "Connection": "close", 
    "Content-Length": "17", 
    "Content-Type": "application/json", 
    "Host": "httpbin.org", 
    "Total-Route-Time": "0", 
    "User-Agent": "curl/7.50.1", 
    "Via": "1.1 vegur", 
    "X-Request-Id": "dfbaa440-9871-4a3c-b482-35a68459722a"
  }, 
  "json": {
    "message": "BLA"
  }, 
  "origin": "<removed>", 
  "url": "https://httpbin.org/post"
}

Am I missing something?

About this issue

Original URL
State: closed
Created 7 years ago
Comments: 58 (46 by maintainers)

Commits related to this issue

Revert "chunked encoding" This reverts commit 2e94212886e4c9531b8ba7f55f4fb0b8134e10ac. This didn't fix the problem, and caused other issues when using werkzeug to execute httpbin, so let's remove i... — committed to postmanlabs/httpbin by kevin1024 7 years ago
Allow to define mime-type when uploading a new file (#58) * Skip PUT Upload test This skips a failing test that depends on an httpbin feature that is currently not working. Ref: https://github.c... — committed to levigross/grequests by heyvito 7 years ago
Revert "chunked encoding" This reverts commit 2e94212886e4c9531b8ba7f55f4fb0b8134e10ac. This didn't fix the problem, and caused other issues when using werkzeug to execute httpbin, so let's remove i... — committed to dimok777/httpbin by kevin1024 7 years ago
Replace FIFOBuffer implementation with BufferStream/IOBuffer wrapper. Add mark/reset support to Form <: IO Dont call eof() in body(::IO, ...) if content length is known (eof() can block) Work aroun... — committed to samoconnor/HTTP.jl by samoconnor 7 years ago
Added support for chunked encoded requests when running gunicorn. Fixed #340. In this commit: - when we see a Transfer-Encoding: chunked request, and the server is gunicorn, we set environ wsgi.in... — committed to javabrett/httpbin by javabrett 7 years ago
Added support for chunked encoded requests when running gunicorn. Fixed #340. In this commit: - when we see a Transfer-Encoding: chunked request, and the server is gunicorn, we set environ wsgi.in... — committed to javabrett/httpbin by javabrett 7 years ago
Merge pull request #405 from javabrett/support-chunked-340 Added support for chunked encoded requests when running gunicorn. #340 — committed to postmanlabs/httpbin by kennethreitz 7 years ago
WIP: Factorisation of implementation into modules and layers (#135) * Replace FIFOBuffer implementation with BufferStream/IOBuffer wrapper. Add mark/reset support to Form <: IO Dont call eof() ... — committed to JuliaWeb/HTTP.jl by samoconnor 6 years ago
Revert "chunked encoding" This reverts commit 2e94212886e4c9531b8ba7f55f4fb0b8134e10ac. This didn't fix the problem, and caused other issues when using werkzeug to execute httpbin, so let's remove i... — committed to virgiliojr94/httpbin-chat2desk by kevin1024 7 years ago
Added support for chunked encoded requests when running gunicorn. Fixed #340. In this commit: - when we see a Transfer-Encoding: chunked request, and the server is gunicorn, we set environ wsgi.in... — committed to virgiliojr94/httpbin-chat2desk by javabrett 7 years ago
Merge pull request #405 from javabrett/support-chunked-340 Added support for chunked encoded requests when running gunicorn. #340 — committed to virgiliojr94/httpbin-chat2desk by kennethreitz 7 years ago

Most upvoted comments

Sorry for being grumpy — been a long day 😃

kennethreitz on Sep 19, 2017

Hi)

Currently httpbin.org responds with 411 status and “Length Required” message to any POST/PUT requests that are sent using chunked encoding…

$  curl -F filedata=@.gitignore http://httpbin.org/post --header "Transfer-Encoding: chunked" --header "Expect:"
<html><head><title>Length Required</title></head><body><p>Length Required.</p></body></html>

Are there any plans to fix this?

0x1eaf on Sep 18, 2017

The way to support this is to wrap your app in the following middleware when running with Gunicorn (or another server that supports chunked encoding).

class InputTerminated:
    def __init__(self, wsgi_app):
        self.wsgi_app = wsgi_app

    def __call__(self, environ, start_response):
        environ['wsgi.input_terminated'] = True
        return self.wsgi_app(environ, start_response)

app.wsgi_app = InputTerminated(app.wsgi_app)

davidism on Jul 21, 2017