aws-sdk-go: Default credentials provider fails slowly, doesn't respect endpoint configuration

Version of AWS SDK for Go?

v1.16.13

Version of Go (go version)?

go version go1.11.2 darwin/amd64

What issue did you see?

Attempting an S3 download without proper credentials results in an HTTP request to http://169.254.169.254/latest/meta-data/iam/security-credentials/, which is unreachable outside the EC2 environment, and which can take a considerable amount of time.

Steps to reproduce

From a machine/account —

  • without network access to 169.254.169.254
  • without AWS credentials in the environment or in an AWS config file

— run the program aws_credentials_timeout.go, in this gist.

Expected

  1. session.NewSessionWithOptions returns an error, or else
  2. downloader.Download quickly returns an error, and program fails quickly.

Actual

  • program can take anywhere up to twenty seconds to fail.
$ time ./aws_credentials_timeout
2019/01/07 12:51:55 NoCredentialProviders: no valid providers in chain
caused by: EnvAccessKeyNotFound: failed to find credentials in the environment.
SharedCredsLoad: failed to load profile, .
EC2RoleRequestError: no EC2 instance role found
caused by: RequestError: send request failed
caused by: Get http://169.254.169.254/latest/meta-data/iam/security-credentials/: dial tcp 169.254.169.254:80: connect: network is unreachable

real    0m17.319s
user    0m0.011s
sys     0m0.008s

Preferred behavior

Some possibilities, more or less in order of preference:

  1. session.mergeConfigSrc could return an error if credentials cannot be retrieved.
  2. session.mergeConfigSrc could fall back to a fast-failing no-op provider instead of defaults.RemoteCredProvider.
  3. defaults.RemoteCredProvider / defaults.ec2RoleProvider could respect the configured endpoint instead of using the EndpointResolver, resulting (in this case at least) in a relatively quick failed request to 127.0.0.1

Failing (1) above, in general it would be nice to have a quick programmatic way to determine whether valid credentials have been provided, so (in this example) one could choose to error out after creating the session, instead of waiting for the download to fail. (Maybe this already exists?)

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Reactions: 2
  • Comments: 15 (3 by maintainers)

Most upvoted comments

SDKs still need a consistent, cross platform way of detecting if they are running on an EC2 instance or not.

@dmolesUC3 I think I have a better way to identify AWS machines:

$ cat /sys/devices/virtual/dmi/id/sys_vendor
Amazon EC2

WDYT ? Maybe it doesn’t work for bare metal machines, but at least, work better then uname -a.