google-cloud-php: [spanner] : STATUS_UNAVAILABLE isn't retried in some cases in Result->rows()

Environment details

• OS:linux/gke/gke-1254-gke2100-cos-101-17162-40-34-v221207-c-cgpv1-pre
• PHP version: 8.1.12
• Package name and version: v1.54.2, v1.54.0

Steps to reproduce

it happens very rarely , once per day or week, on relatively loaded clusters and with simple random queries and I can’t reproduce it

 PHP Notice: Google\Cloud\Core\Exception\ServiceException: {
    "message": "Broken pipe",
    "code": 14,
    "status": "UNAVAILABLE",
    "details": []
} in /var/www/vendor/google/cloud-core/src/GrpcRequestWrapper.php:257
Stack trace:
#0 /var/www/vendor/google/cloud-core/src/GrpcRequestWrapper.php(194): Google\Cloud\Core\GrpcRequestWrapper->convertToGoogleException(Object(Google\ApiCore\ApiException))
#1 [internal function]: Google\Cloud\Core\GrpcRequestWrapper->handleStream(Object(Google\ApiCore\ServerStream))
#2 /var/www/vendor/google/cloud-spanner/src/Result.php(228): Generator->next()
#3 [internal function]: Google\Cloud\Spanner\Result->rows()

according to https://cloud.google.com/spanner/docs/error-codes the call should be retried

UNAVAILABLE | The server is currently unavailable | Retry using exponential backoff. Note that it is not always safe to retry non-idempotent operations.

but according to source https://github.com/googleapis/google-cloud-php/blob/fda8c6c0061184e889d039eb4bbb2e88e35be9a7/Spanner/src/Result.php#L231 there are cases when call won’t be retried

                $hasResumeToken = $this->isSetAndTrue($result, 'resumeToken');
                if ($hasResumeToken || count($bufferedResults) >= self::BUFFER_RESULT_LIMIT) {
                ...
                    $shouldRetry = $hasResumeToken;
                ...    
                $generator->next();
                $valid = $generator->valid();
            } catch (ServiceException $ex) {
                if ($shouldRetry && $ex->getCode() === Grpc\STATUS_UNAVAILABLE) {
                    // Attempt to resume using our last stored resume token. If we
                    // successfully resume, flush the buffer.
                    $generator = $backoff->execute($call, [$this->resumeToken]);
                    $bufferedResults = [];

                    continue;
                }

i.e. it retried only when there is set $resumeToken , but it never set for queries with small results

array (
  'metadata' => 
  array (
    'rowType' => 
    array (
      'fields' => 
      array (
        0 => 
        array (
          'name' => '',
          'type' => 
          array (
            'code' => 2,
            'typeAnnotation' => 0,
          ),
        ),
      ),
    ),
    'transaction' => 
    array (
      'id' => '',
    ),
  ),
  'values' => 
  array (
    0 => '1',
  ),
  'chunkedValue' => false,
  'resumeToken' => '',
)   

sample code I used

        $project = '';
        $instance = '';
        $database = '';
        $spanner = new SpannerClient([ 'projectId' => $project ]);
        $connection = $spanner->connect($instance, $database);
        $generator = $connection
            ->execute('SELECT 1')
            ->rows();

        iterator_to_array($generator);