mitmproxy: Memory Leak in WireGuard Mode

Problem Description

WireGuard and Windows Proxy Mode have a memory leak somewhere.

Steps to reproduce the behavior:

python -m http.server
mitmdump --mode osproxy:curl (or WireGuard)
while true: curl http://127.0.0.1:8000/

@decathorpe suspects TcpStream.read() 😃

refs https://github.com/awestlake87/pyo3-asyncio/issues/84

About this issue

Original URL
State: closed
Created a year ago
Comments: 17 (16 by maintainers)

Most upvoted comments

Should be fixed here then: https://github.com/mitmproxy/mitmproxy_rs/blob/main/mitmproxy-rs/src/tcp_stream.rs#L47

If noone can repro, let’s close this? 😃

mhils on Nov 1, 2023

Regarding https://github.com/decathorpe/mitmproxy_wireguard/blob/e06ec83d1983447a930d89be9aa7689e67bf5c81/src/python/tcp_stream.rs#L46 I wonder if not storing the result of PyBytes::new(...).into_py(...) with let but rather return Ok in Python::with_gil might make a difference (if it’s possible), i.e.

Damn 😮 you figured it out! ✨ ✨

I suspected that something like this might be the cause of the problem - I saw similar issue reports where holding the GIL or not made the difference … but it didn’t dawn on me that the construction of the PyBytes might also be affected here.

I applied the change you suggested (making sure there’s no reference to the PyBytes on the Rust side remaining after the GIL is held), and the memory leak seems to be gone. Previously, my benchmark workload resulted in memory usage growing up to ~2.6 GB, but now it tops out at ~200 MB and drops back down to ~13 MB after the last connection is closed. Meh, that was a lot of time I wasted on debugging this issue 😆

decathorpe on Jan 12, 2023