aws-sdk-php: Bedrock Runtime invokeModelWithResponseStream: Undefined array key "bytes"

Describe the bug

When calling the invokeModelWithResponseStream() method from the AWS PHP SDK within a Laravel application, the EventParsingIterator expected to contain a stream of events is returning empty. This issue occurs despite the expected behavior where the iterator should yield event data. Debug logs show that when casting the iterator to an array, an empty array is obtained, and iterating over it directly does not yield any events.

Expected Behavior

The EventParsingIterator should provide a non-empty collection of events when accessed, allowing a foreach loop to iterate over each event successfully.

Current Behavior

Currently, when accessing the EventParsingIterator, no events are returned. Debugging with Log::debug() shows an empty structure:

{"Aws\\Api\\Parser\\EventParsingIterator":[]}

Attempts to iterate over the iterator result in an error:

ErrorException: Undefined array key "bytes" in /var/www/html/vendor/aws/aws-sdk-php/src/Api/Parser/EventParsingIterator.php:96
Stack trace:
#0 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Bootstrap/HandleExceptions.php(254): Illuminate\Foundation\Bootstrap\HandleExceptions->handleError()
#1 /var/www/html/vendor/aws/aws-sdk-php/src/Api/Parser/EventParsingIterator.php(96): Illuminate\Foundation\Bootstrap\HandleExceptions->Illuminate\Foundation\Bootstrap\{closure}()
#2 /var/www/html/vendor/aws/aws-sdk-php/src/Api/Parser/EventParsingIterator.php(39): Aws\Api\Parser\EventParsingIterator->parseEvent()
#3 /var/www/html/app/Services/TextGeneration.php(646): Aws\Api\Parser\EventParsingIterator->current()
#4 /var/www/html/app/Services/TextGeneration.php(554): App\Services\TextGeneration->call_aws_bedrock()
#5 /var/www/html/app/Services/TextGeneration.php(1083): App\Services\TextGeneration->chat()
#6 /var/www/html/app/Services/TextGeneration.php(1009): App\Services\TextGeneration->generate_stream()
#7 /var/www/html/vendor/laravel/framework/src/Illuminate/Support/Facades/Facade.php(353): App\Services\TextGeneration->generate_chat_stream()
#8 /var/www/html/app/Jobs/StreamChatMessage.php(39): Illuminate\Support\Facades\Facade::__callStatic()
#9 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): App\Jobs\StreamChatMessage->handle()
#10 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Util.php(41): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()
#11 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\Container\Util::unwrapIfClosure()
#12 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(35): Illuminate\Container\BoundMethod::callBoundMethod()
#13 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(662): Illuminate\Container\BoundMethod::call()
#14 /var/www/html/vendor/laravel/framework/src/Illuminate/Bus/Dispatcher.php(128): Illuminate\Container\Container->call()
#15 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(141): Illuminate\Bus\Dispatcher->Illuminate\Bus\{closure}()
#16 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(116): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()
#17 /var/www/html/vendor/laravel/framework/src/Illuminate/Bus/Dispatcher.php(132): Illuminate\Pipeline\Pipeline->then()
#18 /var/www/html/vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php(123): Illuminate\Bus\Dispatcher->dispatchNow()
#19 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(141): Illuminate\Queue\CallQueuedHandler->Illuminate\Queue\{closure}()
#20 /var/www/html/vendor/laravel/framework/src/Illuminate/Queue/Middleware/WithoutOverlapping.php(78): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()
#21 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(180): Illuminate\Queue\Middleware\WithoutOverlapping->handle()
#22 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(116): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()
#23 /var/www/html/vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php(122): Illuminate\Pipeline\Pipeline->then()
#24 /var/www/html/vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php(70): Illuminate\Queue\CallQueuedHandler->dispatchThroughMiddleware()
#25 /var/www/html/vendor/laravel/framework/src/Illuminate/Queue/Jobs/Job.php(98): Illuminate\Queue\CallQueuedHandler->call()
#26 /var/www/html/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(439): Illuminate\Queue\Jobs\Job->fire()
#27 /var/www/html/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(389): Illuminate\Queue\Worker->process()
#28 /var/www/html/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(333): Illuminate\Queue\Worker->runJob()
#29 /var/www/html/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(137): Illuminate\Queue\Worker->runNextJob()
#30 /var/www/html/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(120): Illuminate\Queue\Console\WorkCommand->runWorker()
#31 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): Illuminate\Queue\Console\WorkCommand->handle()
#32 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Util.php(41): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()
#33 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\Container\Util::unwrapIfClosure()
#34 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(35): Illuminate\Container\BoundMethod::callBoundMethod()
#35 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(662): Illuminate\Container\BoundMethod::call()
#36 /var/www/html/vendor/laravel/framework/src/Illuminate/Console/Command.php(211): Illuminate\Container\Container->call()
#37 /var/www/html/vendor/symfony/console/Command/Command.php(326): Illuminate\Console\Command->execute()
#38 /var/www/html/vendor/laravel/framework/src/Illuminate/Console/Command.php(180): Symfony\Component\Console\Command\Command->run()
#39 /var/www/html/vendor/symfony/console/Application.php(1081): Illuminate\Console\Command->run()
#40 /var/www/html/vendor/symfony/console/Application.php(320): Symfony\Component\Console\Application->doRunCommand()
#41 /var/www/html/vendor/symfony/console/Application.php(174): Symfony\Component\Console\Application->doRun()
#42 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(201): Symfony\Component\Console\Application->run()
#43 /var/www/html/artisan(35): Illuminate\Foundation\Console\Kernel->handle()
#44 {main}

Reproduction Steps

  1. Make a call to invokeModelWithResponseStream() with the necessary parameters.
  2. Log the type and contents of the EventParsingIterator that is supposed to contain the response body.
  3. Try to iterate over the EventParsingIterator using a foreach loop.
  4. Observe that the iterator errors and no iterations occur.
$bedrock = AWS::createClient('bedrock-runtime');
        $messages = $this->model_parameters['messages'];
        $invocation_params = [
            'body' => json_encode([
                                      "prompt" => "\n\nHuman: Hi, how are you?\n\nAssistant:",
                                      "max_tokens_to_sample" => 4096,
                                      "temperature" => 0.75,
                                      "top_k" => 50,
                                      "top_p" => 0.7,
                                      "stop_sequences" => ["\n\nHuman:"],
                                      "anthropic_version" => "bedrock-2023-05-31",
                                  ]),
            'modelId' => 'anthropic.claude-v2',
        ];

        $invocation_params["accept"] = "*/*";

        $response = $bedrock->invokeModelWithResponseStream($invocation_params);
        Log::debug('Full response body:', ['response' => $response['body']]);
        Log::debug('Type of response body:', ['type' => get_class($response['body'])]);

        $errors = 0;
        foreach ($response['body'] as $event) { // Error occurs here on the first iteration
            Log::debug('Current event:', ['event' => $event]); // This log message never runs

            if (isset($event['chunk'])) {
                $stream([], $event['chunk']);
            } else {
                $errors++;
                if ($errors > 10) {
                    break;
                }
            }
        }

        return true;

Possible Solution

No response

Additional Information/Context

invokeModel works without streaming.

SDK version used

3.285.1

Environment details (Version of PHP (php -v)? OS name and version, etc.)

PHP 8.1, Ubuntu, Laravel 10

About this issue

  • Original URL
  • State: closed
  • Created 8 months ago
  • Comments: 15 (7 by maintainers)

Most upvoted comments

Hi @prattcmp, sorry to hear about your issues. I was able to reproduce the error that you have reported. I will add a needs-review label here to further address this issue.

Thanks!

Everything appears to be working. Thank you!

Hi @YujiroTakahashi, we have addressed the issue that was originally reported here, which was related to the event parsing iterator not being able to correctly handle the responses gotten from the service. For this new inconvenient that you are experiencing, could you please open a separated issue. Could you also provide an example for how you do this with JS as you mentioned? so we can better understand your inquire.

Thanks!

Hi @yenfryherrerafeliz , I hope your pull request gets merged quickly. I look forward to the integrated release. Thank you.