aws: Generator passed to yield from was aborted without proper return and is unable to continue

When waiting on multiple Results over Result::wait() we randomly got following error:

Fatal error: Uncaught Error: Generator passed to yield from was aborted without proper return and is unable to continue in /var/www/html/vendor/async-aws/core/src/Stream/IterableStream.php:51
Stack trace:
#0 /var/www/html/vendor/symfony/http-client/HttpClientTrait.php(310): AsyncAws\Core\Stream\IterableStream->getIterator()
#1 [internal function]: Symfony\Component\HttpClient\CurlHttpClient::Symfony\Component\HttpClient\{closure}(Object(AsyncAws\Core\Stream\IterableStream))
#2 /var/www/html/vendor/symfony/http-client/HttpClientTrait.php(292): Generator->valid()
#3 /var/www/html/vendor/symfony/http-client/CurlHttpClient.php(462): Symfony\Component\HttpClient\CurlHttpClient::Symfony\Component\HttpClient\{closure}(65536)
#4 /var/www/html/vendor/symfony/http-client/CurlHttpClient.php(247): Symfony\Component\HttpClient\CurlHttpClient::readRequestBody(65536, Object(Closure), '', false)
#5 [internal function]: Symfony\Component\HttpClient\CurlHttpClient::Symfony\Component\HttpClient\{closure}(Resource id #8604, NULL, 65536)
#6 / in /var/www/html/vendor/async-aws/core/src/Stream/IterableStream.php on line 51

The results that are being waited on are a list of PutObjectOutput.

We aren’t able to reproduce the error reliably and don’t know what may trigger that error.

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 24 (12 by maintainers)

Most upvoted comments

I think I was able to catch this one. A TransportException is thrown, which is caught and ignored in RetryableHttpClient. The relevant code is this, I believe:

https://github.com/symfony/http-client/blob/d2464f48482223c7c6826cd8c6ed7929d1ce6093/RetryableHttpClient.php#L59-L69

I only looked at it briefly, but I think it’s because this retry mechanism doesn’t always yield or return. I’m not sure if that is a bug in symfony/http-client or if it’s just how it interacts with the AsyncAWS’ IterableStream.

Error stack:

ErrorChunk.php:53, Symfony\Component\HttpClient\Chunk\ErrorChunk->isTimeout()
RetryableHttpClient.php:62, Symfony\Component\HttpClient\RetryableHttpClient->Symfony\Component\HttpClient\{closure:/app/vendor/symfony/http-client/RetryableHttpClient.php:59-145}()
AsyncResponse.php:342, Generator->valid()
AsyncResponse.php:342, Symfony\Component\HttpClient\Response\AsyncResponse::passthruStream()
AsyncResponse.php:286, Symfony\Component\HttpClient\Response\AsyncResponse::stream()
AsyncResponse.php:66, Symfony\Component\HttpClient\Response\AsyncResponse::Symfony\Component\HttpClient\Response\{closure:/app/vendor/symfony/http-client/Response/AsyncResponse.php:60-84}()
CommonResponseTrait.php:152, Symfony\Component\HttpClient\Response\AsyncResponse::initialize()
AsyncResponse.php:93, Symfony\Component\HttpClient\Response\AsyncResponse->getStatusCode()
Response.php:127, AsyncAws\Core\Response->resolve()
Response.php:104, AsyncAws\Core\Response->__destruct()
AsyncAwsS3Adapter.php:329, AsyncAws\S3\S3Client->putObject()
AsyncAwsS3Adapter.php:329, League\Flysystem\AsyncAwsS3\AsyncAwsS3Adapter->upload()
AsyncAwsS3Adapter.php:131, League\Flysystem\AsyncAwsS3\AsyncAwsS3Adapter->write()
Filesystem.php:44, League\Flysystem\Filesystem->write()

Sarke on Jun 25, 2021

WOOAAA you made my DAY!!! Thank you so much @Sarke for all the information ❤️

I spent so much time trying to understand what append, reading code line by line. And you pointed your finger on the “reproducer”.

Issue is: When uploading a file in S3, we replace the body by a callable that split the body in chunks, sign each chunk and send them one by one.

It works well except when the upload fails, and the RetryableHttpClient decides to retry to send the file. In that case, the callable has already run and can not be executed a second time.

This should be fixed by #1033

jderusse on Jun 25, 2021

Wow thank you all!!! I never got around digging deeper into it, so awesome to see this solved 🎉

Thank you all for your awesome work ❤️

keulinho on Jun 28, 2021

Nono, thank YOU!

Sarke on Jun 26, 2021

Thank you so much for your help on this. Really appreciate ❤️

jderusse on Jun 26, 2021

So the dump revealed that $this->content is an instance of \Generator, which is quite obvious 🤦

I’m now trying to dump $this->content->current(), but that should probably return null as the generator is already closed right?

keulinho on Mar 3, 2021

seems like today we were pretty unlucky.

I think i will create a fork and put some more logging in there.

I’ll keep you posted.

keulinho on Feb 25, 2021