uvicorn: Handle exception when receive request with custom method

Checklist

The bug is reproducible against the latest release or master.
There are no similar issues or pull requests to fix it yet.

Describe the bug

I’m developing fastapi application which is using starlette and run with uvicorn.

Currently I developing locally on my PC which I’ve done for a long time, so I frequently get some external random requests from the internet from random IPs.

While it is fine if I get some request that lead to some php endpoint and server log it and show nice 404 error in the logs, sometimes I get weird custom requests that break uvicorn and it throw errors, it look like this in the console:

The main issue in /uvicorn/protocols/http/httptools_impl.py: https://github.com/encode/uvicorn/blob/a9ea657339e64c2236f3fb49616e3e0a133273c1/uvicorn/protocols/http/httptools_impl.py#L131-L138

Steps to reproduce the bug

Use curl to make custom request, the custom method is -X flag where you in theory should use GET/POST/… but instead in my case I get some weird methods, that make uvicorn to throw exception

curl -X MUX -i -H "Custom-Header: Test" http://127.0.0.1/

Expected behavior

I should get minimal and useful log, this may be need to pass on underlying libraries (to handle it there) or being able to control behavior from them. Right now when you get this as it shows you don’t understand what is happening, who is sending to you (IP) and what (what method?).

Actual behavior

Logs get full of this exception spam, it essentially work as DDoS for a server. I have to patch library manually to be able to fix this.

Debugging material

I attach screenshot above, there is also example from one request:

WARNING:  Invalid HTTP request received.
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/uvicorn/protocols/http/httptools_impl.py", line 131, in data_received
    self.parser.feed_data(data)
  File "httptools/parser/parser.pyx", line 212, in httptools.parser.parser.HttpParser.feed_data
httptools.parser.errors.HttpParserInvalidMethodError: Invalid method encountered

First line is come, probably from fastapi and other are exception.

Environment

Windows, WSL, Docker
Running uvicorn 0.15.0 with CPython 3.8.6 on Windows
Running fastapi application like that: uvicorn main:app --host 0.0.0.0 --port 80

Additional context

This is similar issues:

About this issue

Original URL
State: closed
Created 3 years ago
Reactions: 5
Comments: 22 (10 by maintainers)

Most upvoted comments

yep, you’ll just see WARNING: Invalid HTTP request received. and not the full traceback anymore, there’s no functional change

euri10 on Jul 11, 2022

Got the same error calling without method

import json
import random
import socket
import time
import uuid

HOST = "localhost"  # Standard loopback interface address (localhost)
PORT = 65432  # Port to listen on (non-privileged ports are > 1023)
NUM_MESSAGES = 3

if __name__ == "__main__":
    msgs = []
    for _ in range(NUM_MESSAGES):
        msgs.append(
            {
                "timestamp": int(time.time()) + random.randint(1, 10),
                "msg": str(uuid.uuid4()),
            }
        )

    msg_str = json.dumps(msgs)

    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
        s.connect((HOST, PORT))
        s.sendall(bytes(msg_str, encoding="utf-8"))

zxcfer on Oct 11, 2022

@Kludex I feel like there is an issue, because this basically stops the server from reporting a healthy status to the kubernetes pod health checker, as it is causing it to reload and then shutdown after 5-10 invalid HTTP requests. The issue doesn’t happen on local, and those get requests are coming from the liveness probe. It’s definitely a uvicorn issue, because I have had multiple different servers running on kubernetes but uvicorn has so far been very problematic. Happy to be educated…of course!

michealroberts on Aug 9, 2022

@euri10 That would be better, but can you also put info about what was requested? Like the type of request.

XCanG on Feb 16, 2022

What at least should be done is better writing exception message, because without it, it looks confusing. After that you may decide another approach.

I did some monkey-patch by including protocol information in exception:

        try:
            self.parser.feed_data(data)
        except httptools.HttpParserError as exc:
            msg = f"Invalid HTTP request received. | {data}"
            self.logger.warning(msg, exc_info=exc)
            self.transport.close()
        except httptools.HttpParserUpgrade:
            self.handle_upgrade()

Adding that data really helps as you now see in the logs what exactly is happening.

XCanG on Feb 4, 2022