bitcoin: bitcoind hangs waiting for `g_requests.empty()`

Is there an existing issue for this?

• I have searched the existing issues

Current behaviour

bitcoind hangs

Expected behaviour

bitcoind shouldn’t hang

I was able to dump the running threads and it was stuck waiting for g_requests to be empty. The issue seems to be that this callback may not actually be called in case of error: https://github.com/bitcoin/bitcoin/blob/22139f6e83a2bedd2dad9f280567d2c76c54252f/src/httpserver.cpp#L222-L227

See this issue https://github.com/libevent/libevent/issues/589 for more details. I think what happens is that interrupting bitcoin-cli before it has received the reply calls the error callback and then the success callback is never called.

I think this is also a memory leak since g_requests won’t get cleaned up. Whether it’s severe or not probably depends on the caller and what type of requests they are making. This new code was introduced here https://github.com/bitcoin/bitcoin/pull/26742

Steps to reproduce

I was running a simple shell script in 4 terminal windows for fun:

#!/bin/bash

for i in {1..50000}
do
	./src/bitcoin-cli -rpcuser=admin -rpcpassword=pass getrawtransaction 217819875434324c412d5e938f7f3c1670fb60814974cf7958f1fac90ee5afb7
done

I was messing around with this script and randomly Ctrl-C’d each terminal window in rapid succession and then when I waited up to a minute and Ctrl-C’d bitcoind, it didn’t shutdown. I can reproduce it pretty reliably as well.

Compilation commands:

./autogen.sh; ./configure --without-wallet --with-gui=no --disable-zmq; make -j4

Run-time options (note that I was able to reproduce this with the default number of rpcthreads as well):

./src/bitcoind -txindex -rpcthreads=1 -debug

Output of gcc --version:

Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
Apple clang version 11.0.3 (clang-1103.0.32.62)
Target: x86_64-apple-darwin21.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

Output of g++ --version:

Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
Apple clang version 11.0.3 (clang-1103.0.32.62)
Target: x86_64-apple-darwin21.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

My libevent version is 2.1.12

Relevant log output

^C2023-05-22T20:56:10Z [http] Interrupting HTTP server
2023-05-22T20:56:10Z [rpc] Interrupting HTTP RPC server
2023-05-22T20:56:10Z [rpc] Interrupting RPC
2023-05-22T20:56:10Z tor: Thread interrupt
2023-05-22T20:56:10Z Shutdown: In progress...
2023-05-22T20:56:10Z [rpc] Stopping HTTP RPC server
2023-05-22T20:56:10Z torcontrol thread exit
2023-05-22T20:56:10Z [http] Unregistering HTTP handler for / (exactmatch 1)
2023-05-22T20:56:10Z addcon thread exit
2023-05-22T20:56:10Z [http] Unregistering HTTP handler for /wallet/ (exactmatch 0)
2023-05-22T20:56:10Z [rpc] Stopping RPC
2023-05-22T20:56:10Z net thread exit
2023-05-22T20:56:10Z [rpc] RPC stopped.
2023-05-22T20:56:10Z [http] Stopping HTTP server
2023-05-22T20:56:10Z [http] Waiting for HTTP worker threads to exit
2023-05-22T20:56:10Z [http] Exited http event loop
2023-05-22T20:56:10Z [http] Waiting for 1 requests to stop HTTP server

The important bit is the very last log line. bitcoind was still alive and periodically printed log lines about feeding dynamic environment data into the RNG.

How did you obtain Bitcoin Core

Compiled from source

What version of Bitcoin Core are you using?

17acb2782a66f033238340e4fb81009e7f79e97a

Operating system and version

macOS intel

Machine specifications

No response