aws-sdk-php: Bedrock Runtime invokeModelWithResponseStream: Undefined array key "bytes"
Describe the bug
When calling the invokeModelWithResponseStream() method from the AWS PHP SDK within a Laravel application, the EventParsingIterator expected to contain a stream of events is returning empty. This issue occurs despite the expected behavior where the iterator should yield event data. Debug logs show that when casting the iterator to an array, an empty array is obtained, and iterating over it directly does not yield any events.
Expected Behavior
The EventParsingIterator should provide a non-empty collection of events when accessed, allowing a foreach loop to iterate over each event successfully.
Current Behavior
Currently, when accessing the EventParsingIterator, no events are returned. Debugging with Log::debug() shows an empty structure:
{"Aws\\Api\\Parser\\EventParsingIterator":[]}
Attempts to iterate over the iterator result in an error:
ErrorException: Undefined array key "bytes" in /var/www/html/vendor/aws/aws-sdk-php/src/Api/Parser/EventParsingIterator.php:96
Stack trace:
#0 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Bootstrap/HandleExceptions.php(254): Illuminate\Foundation\Bootstrap\HandleExceptions->handleError()
#1 /var/www/html/vendor/aws/aws-sdk-php/src/Api/Parser/EventParsingIterator.php(96): Illuminate\Foundation\Bootstrap\HandleExceptions->Illuminate\Foundation\Bootstrap\{closure}()
#2 /var/www/html/vendor/aws/aws-sdk-php/src/Api/Parser/EventParsingIterator.php(39): Aws\Api\Parser\EventParsingIterator->parseEvent()
#3 /var/www/html/app/Services/TextGeneration.php(646): Aws\Api\Parser\EventParsingIterator->current()
#4 /var/www/html/app/Services/TextGeneration.php(554): App\Services\TextGeneration->call_aws_bedrock()
#5 /var/www/html/app/Services/TextGeneration.php(1083): App\Services\TextGeneration->chat()
#6 /var/www/html/app/Services/TextGeneration.php(1009): App\Services\TextGeneration->generate_stream()
#7 /var/www/html/vendor/laravel/framework/src/Illuminate/Support/Facades/Facade.php(353): App\Services\TextGeneration->generate_chat_stream()
#8 /var/www/html/app/Jobs/StreamChatMessage.php(39): Illuminate\Support\Facades\Facade::__callStatic()
#9 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): App\Jobs\StreamChatMessage->handle()
#10 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Util.php(41): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()
#11 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\Container\Util::unwrapIfClosure()
#12 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(35): Illuminate\Container\BoundMethod::callBoundMethod()
#13 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(662): Illuminate\Container\BoundMethod::call()
#14 /var/www/html/vendor/laravel/framework/src/Illuminate/Bus/Dispatcher.php(128): Illuminate\Container\Container->call()
#15 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(141): Illuminate\Bus\Dispatcher->Illuminate\Bus\{closure}()
#16 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(116): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()
#17 /var/www/html/vendor/laravel/framework/src/Illuminate/Bus/Dispatcher.php(132): Illuminate\Pipeline\Pipeline->then()
#18 /var/www/html/vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php(123): Illuminate\Bus\Dispatcher->dispatchNow()
#19 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(141): Illuminate\Queue\CallQueuedHandler->Illuminate\Queue\{closure}()
#20 /var/www/html/vendor/laravel/framework/src/Illuminate/Queue/Middleware/WithoutOverlapping.php(78): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()
#21 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\Queue\Middleware\WithoutOverlapping->handle()
#22 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(116): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()
#23 /var/www/html/vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php(122): Illuminate\Pipeline\Pipeline->then()
#24 /var/www/html/vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php(70): Illuminate\Queue\CallQueuedHandler->dispatchThroughMiddleware()
#25 /var/www/html/vendor/laravel/framework/src/Illuminate/Queue/Jobs/Job.php(98): Illuminate\Queue\CallQueuedHandler->call()
#26 /var/www/html/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(439): Illuminate\Queue\Jobs\Job->fire()
#27 /var/www/html/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(389): Illuminate\Queue\Worker->process()
#28 /var/www/html/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(333): Illuminate\Queue\Worker->runJob()
#29 /var/www/html/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(137): Illuminate\Queue\Worker->runNextJob()
#30 /var/www/html/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(120): Illuminate\Queue\Console\WorkCommand->runWorker()
#31 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): Illuminate\Queue\Console\WorkCommand->handle()
#32 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Util.php(41): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()
#33 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\Container\Util::unwrapIfClosure()
#34 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(35): Illuminate\Container\BoundMethod::callBoundMethod()
#35 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(662): Illuminate\Container\BoundMethod::call()
#36 /var/www/html/vendor/laravel/framework/src/Illuminate/Console/Command.php(211): Illuminate\Container\Container->call()
#37 /var/www/html/vendor/symfony/console/Command/Command.php(326): Illuminate\Console\Command->execute()
#38 /var/www/html/vendor/laravel/framework/src/Illuminate/Console/Command.php(180): Symfony\Component\Console\Command\Command->run()
#39 /var/www/html/vendor/symfony/console/Application.php(1081): Illuminate\Console\Command->run()
#40 /var/www/html/vendor/symfony/console/Application.php(320): Symfony\Component\Console\Application->doRunCommand()
#41 /var/www/html/vendor/symfony/console/Application.php(174): Symfony\Component\Console\Application->doRun()
#42 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(201): Symfony\Component\Console\Application->run()
#43 /var/www/html/artisan(35): Illuminate\Foundation\Console\Kernel->handle()
#44 {main}
Reproduction Steps
- Make a call to invokeModelWithResponseStream() with the necessary parameters.
- Log the type and contents of the EventParsingIterator that is supposed to contain the response body.
- Try to iterate over the EventParsingIterator using a foreach loop.
- Observe that the iterator errors and no iterations occur.
$bedrock = AWS::createClient('bedrock-runtime');
$messages = $this->model_parameters['messages'];
$invocation_params = [
'body' => json_encode([
"prompt" => "\n\nHuman: Hi, how are you?\n\nAssistant:",
"max_tokens_to_sample" => 4096,
"temperature" => 0.75,
"top_k" => 50,
"top_p" => 0.7,
"stop_sequences" => ["\n\nHuman:"],
"anthropic_version" => "bedrock-2023-05-31",
]),
'modelId' => 'anthropic.claude-v2',
];
$invocation_params["accept"] = "*/*";
$response = $bedrock->invokeModelWithResponseStream($invocation_params);
Log::debug('Full response body:', ['response' => $response['body']]);
Log::debug('Type of response body:', ['type' => get_class($response['body'])]);
$errors = 0;
foreach ($response['body'] as $event) { // Error occurs here on the first iteration
Log::debug('Current event:', ['event' => $event]); // This log message never runs
if (isset($event['chunk'])) {
$stream([], $event['chunk']);
} else {
$errors++;
if ($errors > 10) {
break;
}
}
}
return true;
Possible Solution
No response
Additional Information/Context
invokeModel
works without streaming.
SDK version used
3.285.1
Environment details (Version of PHP (php -v
)? OS name and version, etc.)
PHP 8.1, Ubuntu, Laravel 10
About this issue
- Original URL
- State: closed
- Created 8 months ago
- Comments: 15 (7 by maintainers)
Hi @prattcmp, sorry to hear about your issues. I was able to reproduce the error that you have reported. I will add a needs-review label here to further address this issue.
Thanks!
Everything appears to be working. Thank you!
Hi @YujiroTakahashi, we have addressed the issue that was originally reported here, which was related to the event parsing iterator not being able to correctly handle the responses gotten from the service. For this new inconvenient that you are experiencing, could you please open a separated issue. Could you also provide an example for how you do this with JS as you mentioned? so we can better understand your inquire.
Thanks!
Hi @yenfryherrerafeliz , I hope your pull request gets merged quickly. I look forward to the integrated release. Thank you.