guzzle: Connection or File Descriptor leak when HTTP Exception occurs in async batch
Guzzle version(s) affected: 6.3.0 and possibly older versions
Description
When using the async feature to make a batch of requests in parallel, if one of those requests responds back with an exception type response (example, one of them is a 404 and the rest are normal 200) there will be an open file descriptor somewhere in the stack – e.g. fclose is not being closed properly on something – with no way of cleaning up this open handle except by quitting the process (not valid for long running PHP scripts).
How to reproduce
Reproduce is simple using a fairly simple script, occurs on Linux based system but also macOS with PHP 7.1.
- Have PHP 7.1 installed with cURL available, on a linux or macOS system
- Download gist to an empty dir and name it
gist.php
- Run
composer require guzzlehttp/guzzle
to get latest guzzle installed intovendor/
- Run
php gist.php
Expected:
Because the inner code operates in a function with only local variables, we expect that as execution exits the function, all objects that were created would have their __destruct()
called, freeing up any open handles that were generated.
Actual: As the script runs and the URLs are hit in parallel, the number of open file descriptors (labeled as OPEN FDs in the debug output) increases for every iteration (the main loop runs 4 times total). For bigger running batch scripts you could imagine that eventually the “open file limit” will be hit due to HTTP exceptions and bad URLs occurring.
Inverse case To observe what would happen in a perfect world where file descriptors are NOT kept open, we can make sure all URLs respond with a 200:
- Comment out using
//
on line 51, save and re-run the test script - Notice that since all URLs are 200 and no exceptions occur, the file descriptors (OPEN FDs) never increase indefinitely, they’ll stay as some static number.
Possible Solution
Likely something to do with the way CurlHandler, CurlMultiHandler, or CurlFactory works. However it could be an internal PHP or CURL bug. Could also be the Guzzle Promises implementation.
Additional context
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 1
- Comments: 16 (8 by maintainers)
@gmponos Although it seems very related, I have re-tested this reproduction script on various guzzle changes since April 2019. I reproduced the issue again on the Guzzle version at the time (tested 6.3.2). Then I upgraded to GuzzleHttp 6.4.1 and the issue went away… so apparently my issue was fixed at some point since this issue was created (or in the dependent composer packages, e.g. guzzle/promises, during that time).
Closing this issue now, Thanks.
I’m guessing this is what you meant to post: https://gist.github.com/michaelbutler/14195fbf91a525c5af0cdcbf819095eb#file-mycurlfactory-php