zephyr: http_client_req() sometimes hangs when peer disconnects
Describe the bug We have a multi-node system with some inter-node communication performed over https on ethernet. In our system tests, we restart nodes and/or node software to test resilience and robustness.
Once in a blue moon when the http server node (or server software) restarts, the http client call to http_client_req() never returns. It hangs forever and blocks the thread.
To Reproduce Reproduction is difficult. We run 10-hour system tests over night with thousands of http requests and hundreds of server restarts and the problem typically occurs only once or twice for each nightly run.
I have done some attemps at isolating the scenario and force it to occur more frequently but have so far not had great success.
Expected behavior
I expect http_client_req() to always return, at most timeout ms after being called.
Impact Impact is pretty severe since all execution in the calling thread is stalled permanently.
Environment (please complete the following information):
- The http client os is Zephyr 3.1.0 on STM32.
- The http server os is Linux 5.4.84 on Freescale i.MX6.
We first hit this issue on Zephyr 2.7, updated to 3.1 to see if it was fixed, but alas, no.
Context I know this can look like a hopeless ticket. I don’t expect anyone to be able to read this and say “eureka, here’s the fix”. What I’m hoping for is input from people who know the networking code better than me and can discuss possible causes and point at where to dig deeper.
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 15 (15 by maintainers)
Indeed, that would explain it then: https://github.com/zephyrproject-rtos/zephyr/issues/54275
This one is not fixed yet, but I have it on my radar.