uvicorn: High CPU usage when using --reload

You should only use --reload in development, so I don’t see this as a pressing issue. I’d be very happy to see an alternative optional reload implementation based on watchdog, since that’d be far more efficient than our stat-based reloader. But the impetus for that would need to come from someone taking on a pull request.

I had this issue with my project.

With only --reload, it was eating up 25% of a 4 core CPU watching 8135 files in my venv/ folder.

By adding --reload-dir src/ where src/ contains my application, the CPU issue went away completely.

Maybe a decent solution is to have uvicorn ignore common virtualenv directory names, like venv/ .venv/, etc?

I found that there are --reload-dir settings already. By applying app folder, CPU usage significant decrease.

on 0.11.5 it should be imported if watchdog is found so either you pip install watchdog or pip install uvicorn[watchgodreloader]

tell us if this helps, what is the number of files you’re watching ? Last time I tried on @rafalp misago project which watches quite a large number of files, I was at a nice 4% cpu all along in both statsreloader and watchgod mode, and on the same @rafalp was at 20% on a macos, so it’s rather hard to tell what’s going on, I’m on debian

ATTENTION: You need to: pip install watchgod this is not a typo.

If the server starts up, you will then see:

INFO:     Uvicorn running on http://0:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [123249] using ***watchgod***
...

Note if your venv is in project root along with your .py , this will also majorly slow things down.

Passing the additional setting to ignore scanning the venv reduced uvicorn usage to ~1% down from 100% on rpi4.

--reload-exclude venv

Here’s another observation for Misago in Docker: When I disable --reload in docker container and restart it, docker’s CPU usage stays at comfortable ~10%. So even if watchgod is not using CPU on its own, its doing something that causes docker to rev up.

Here’s CPU without reloader:

Here’s with reloader:

This CPU usage makes sense for statreload because its the one that keeps poking filesystem a lot. Watchgod is supposed to listeen for filesystem events and react to those instead.

So I’ve checked what Watchgod is actually doing, and then realized it doesn’t use filesystem events for changes detection. Instead it uses same approach that statreload does: 4 times a second it walks filesystem and compares mtime of every file against dict with previous mtimes. This has no chance to be performant in docker containers used for dev.

So we actually lie to people in the docs when we say that watchgod is faster because it uses filesystem events for change detection.

I think we should implement something like Watchman to actually have performant in-dev code reload strategy. I know Django does this.

There’s also an issue of frequency at which filesystem is checked for changes. Django checks filesystem every second. Uvicorn every 250ms. Filesystem ops are relatively cheap in Docker, as long as they are infrequent. Here we are doing IO operation potentially thousands times a second.

For comparison, here’s my CPU graph after I’ve edited Uvicorn to pool every second:

CPU usage is still noticeable, but it’s not enough to get my laptop’s fans to spin.

So we still have performance issue. Its not for small projects or people running on watcher natively, but it’s still a deal-breaker for people using dev setups based on docker.

installing uvicorn[standard] fixed this problem for me: pip3 uninstall uvicorn pip3 install uvicorn[standard]

Here’s my CPU usage in 0.12 running with --reload --reload-delay 1 --reload-dir misago --reload-dir plugins combined with few accurate docker volumes instead of one volume for project’s repo:

I’ll say this is much better.

Yes I use it only in dev, I just wanted to know if it was normal to hear my fans ramp up.

This should not be closed, it’s very much still an issue, and it’s a bug that uvicorn takes up 50% of CPU just for polling.

@tomchristie How do you feel about using watchdog (https://github.com/gorakhargosh/watchdog) for file watching? It would add a dev dependency to the project, but it looks like it handles file watching very efficiently for most OS’s with a fallback to the collect all files and then stat them every X milliseconds. I’m happy to attempt a PR with it.

Let’s close this. If what we’ve got in is not enough for somebody, we’ll open new issue to track work for event-based reloader.

I’ve learned I have to be very specific in what folders get mounted via docker, and what folders are watched to not end up using things like .mypy_cache, .coverage, .nox, etc

pipenv install "uvicorn[watchgodreload]"

then use reload_dirs:

    uvicorn.run("main:app", host="0.0.0.0", port=int(config.APP_PORT), reload=config.is_local_env(),
                reload_dirs=["app", "views", "conf"])

now fan is quiet and my MBP isn’t too hot to put on my lap anymore … wow!

on 0.11.5 it should be imported if watchdog is found so either you pip install watchdog or pip install uvicorn[watchgodreloader]

tell us if this helps, what is the number of files you’re watching ? Last time I tried on @rafalp misago project which watches quite a large number of files, I was at a nice 4% cpu all along in both statsreloader and watchgod mode, and on the same @rafalp was at 20% on a macos, so it’s rather hard to tell what’s going on, I’m on debian

Should we make reload check time configurable?

No to configuring reload check time - that’s too many dials. Tho there’s more efficient file watching systems (eg gunciorn has an optional dependency, and only falls back to stat checks if needed) we should probably do the same. It could also be reasonable for us to allow controls into which files are being watched (eg perhaps it’s having to scan a very large, unchanging, virtualenv, as well as the actual source code?)