aws-sdk-js-v3: AWSSDK V3 S3 Setup: fromInstanceMetadata does not retry on error despite maxRetries set

Checkboxes for prior research

• I’ve gone through Developer Guide and API reference
• I’ve checked AWS Forums and StackOverflow.
• I’ve searched for previous similar issues and didn’t find any solution.

Describe the bug

There seems to be an issue with the S3 Client, specifically with credentials while utilzing the aws-credential-provider package in aws-sdk v3.

This problem arised from migrating from sdk v2 to v3.

SDK version number

@aws-sdk/client-s3": “^3.379.1, @aws-sdk/credential-provider-imds”: “^3.374.0”, @aws-sdk/credential-providers": "^3.379.1

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

Error: pulling file from s3: {"tryNextLink":true,"name":"ProviderError","$metadata":{"attempts":1,"totalRetryDelay":0}}

Reproduction Steps

import { S3Client } from '@aws-sdk/client-s3'
import { fromInstanceMetadata } from '@aws-sdk/credential-providers'

const s3Client = new S3Client({
    apiVersion: 'latest',
    credentials: fromInstanceMetadata({
        timeout: 3000,
        maxRetries: 5
    }),
    region: us-east-1
})

///
public async getSitemap(
        fileName: string
    ): Promise<string> {
        const command = new GetObjectCommand({
            Bucket: this.config.getField('SITEMAP_BUCKET_NAME'),
            Key: `${this.config.getField('SITEMAP_V3_FILES_PATH') as string}/${fileName}`
        })

        try {
            const data = await this.s3.send(command) // ! potential issue here

            if(!data.Body) {
                throw new Error('Error: getSitemap - No data available.')
            } else {
                const dataAsBuf = Buffer.from(data.Body.transformToString())
                const dataStr = dataAsBuf.toString('utf8')

                return dataStr
            }
        } catch (error) {
            console.log('Error: pulling file from s3: ', JSON.stringify(error))
            throw new Error('Reader S3 Handler - sitemap not found')
        }
    }

Observed Behavior

There seems to be an issue with the S3 Client, specifically with credentials from what I know so far.

The error originates from the Reader S3 Handler, “attempting” to .send(command) which is trying to getObject() (v2 syntax). However, unable to pull anything.

• Error message in CWL: Error: pulling file from s3: {“tryNextLink”:true,“name”:“ProviderError”,“$metadata”:{“attempts”:1,“totalRetryDelay”:0}}

• DD Logs Error: However, data is undefined due to unable to retrieve object from S3 POST http://URLHERE => The “data” argument must be of type string or an instance of Buffer, TypedArray, or DataView. Received undefined

I am utilizing fromInstanceMetadata from aws-credential-provider package, which from the aws docs seems correct:@aws-sdk/credential-providers | AWS SDK for JavaScript v3

Following what others facing similar issues here, and here, I attempted to increase the maxRetries to 3 and timeout to 3000, however still recieve same error message of attempts 1, and totalRetryDelay 0.

Expected Behavior

I should receive objects back from the S3 bucket.

Possible Solution

No response

Additional Information/Context

No response