terragrunt: Error finding AWS credentials for S3 remote state

Hi guys,

Kindly asking for your advice. I see that that the issue was previously discussed several times from different angles. I can’t make our TG code working with my creds in AWS multiaccount environments without creating dedicated IAM user in all these AWS accounts. I am successfully using assume-role consumption and aws cli is perfectly working so I can pull data from all accounts just by setting appropriate AWS_PROFILE and AWS_DEFAULT_REGION where profiles are configured in ~/.aws/config this way

[profile main]
output = json
region = eu-central-1

[profile dev]
output = json
region = eu-central-1
role_arn = arn:aws:iam::xxxxxxxxxx:role/AdminRole
source_profile = main

What I need to do is to create resources in AWS dev account where we have s3 remote state stored as well using IAM user from main account via assume-role approach.

Terragrunt code I am trying to push looks like this, for example, which has variables specifications only and is pulling TF module config from git repo

terragrunt = {
  terraform {
    source = "git::path"
  }

  # Include all settings from the root terraform.tfvars file
  include = {
    path = "${find_in_parent_folders()}"
  }
}

solution_owner                = "Devops"
vpc_cidr_block                = "10.0.1.0/20"
...

It also takes TG configs from parent folders where I have remote state config

terragrunt = {
  # Configure Terragrunt to automatically store tfstate files in an S3 bucket
  remote_state {
    backend = "s3"
    config {
      encrypt        = true
      bucket         = "dev-terraform-state"
      key            = "${path_relative_to_include()}/terraform.tfstate"
      region         = "eu-central-1"
      dynamodb_table = "terraform-locks"
      profile        = "dev"
    }
  }
  # Configure root level variables that all resources can inherit
  terraform {
    extra_arguments "bucket" {
      commands = ["${get_terraform_commands_that_need_vars()}"]
      optional_var_files = [
          "${get_tfvars_dir()}/${find_in_parent_folders("account.tfvars", "ignore")}"
      ]
    }
  }
}

During execution of terragrunt plan I am getting the error pointing to absent creds for reaching remote state s3


[terragrunt] [...] 2019/03/06 12:01:40 Running command: terraform --version
[terragrunt] 2019/03/06 12:01:40 Reading Terragrunt config file at .../core/vpc/terraform.tfvars
[terragrunt] 2019/03/06 12:01:40 WARNING: no double-slash (//) found in source URL /tf/module-aws-vpc.git. Relative paths in downloaded Terraform code may not work.
[terragrunt] 2019/03/06 12:01:40 Cleaning up existing *.tf files in .../core/vpc/.terragrunt-cache/VSGljja7WSjrKw1hwRMSENKXBAA/yAyfAK-S9z7ucSeeqfftLQhM-MA
[terragrunt] 2019/03/06 12:01:40 Downloading Terraform configurations from git::ssh://asdasd.git?ref=pr/12 into .../core/vpc/.terragrunt-cache/VSGljja7WSjrKw1hwRMSENKXBAA/yAyfAK-S9z7ucSeeqfftLQhM-MA using terraform init
[terragrunt] [.../core/vpc] 2019/03/06 12:01:40 Initializing remote state for the s3 backend
[terragrunt] 2019/03/06 12:02:00 Error finding AWS credentials (did you set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables?): NoCredentialProviders: no valid providers in chain. Deprecated.
        For verbose messaging see aws.Config.CredentialsChainVerboseErrors
[terragrunt] 2019/03/06 12:02:00 Unable to determine underlying exit code, so Terragrunt will exit with error code 1

I tried to use all 3 approaches described in section “Work with multiple AWS accounts” of main TG repo page with setting AWS_PROFILE env. variable, using sts assume-role etc, but I am stuck on the same stage of init remote s3.

As I see TG supported assume-role approach for quite long time. Is there any caveats or limitations with that? What am I doing wrong?

Please help.

Thank you very much! #

About this issue

  • Original URL
  • State: open
  • Created 5 years ago
  • Reactions: 4
  • Comments: 16 (5 by maintainers)

Most upvoted comments

Any updates on this? I am seeing the same issue on my end

I think I figured it out… I will test it deeply but for now it works! My scenario is as follow:

  • I have my IAM User in the Management AWS Account
  • this user has a Role which allows to Assume the IAM Role on Dev AWS Account
  • the IAM Role at Dev account is available after the MFA process

So I think that is something like above problem plus MFA which increases the complexity… but the solution is quite simple - I had to have a [default] profile in my ~/.aws/credentials which seems to be mandatory for Terragrunt! Of course other profiles are valid but this one is something like a must-have

My ~/.aws/credentials file was as follow:

[default]
aws_access_key_id = AKIA...
aws_secret_access_key = ry5tgFree...

[assumeHelper]
role_arn = arn:aws:iam::123456789012:role/FooRole
mfa_serial = arn:aws:iam::098765432109:mfa/foo.bar
region = eu-central-1
source_profile = default

[mfaAssume]
aws_access_key_id = ASIA...
aws_secret_access_key = wi47gPZR...
aws_session_token = FwoGZXIv...

[default] - contains the data for the IAM User at Management AWS Account [assumeHelper] - it is a “notepad” for the aws sts command, not used in any Terraform’s configuration [mfaAssume]- contains the data which was generated by the aws sts command

Then my terragrunt.hcl looked like below which the most important variable is profile = “mfaAssume”

remote_state {
  backend = "s3"
  config = {
    bucket         = "foo-terraform-state"
    dynamodb_table = "foo-terraform-state"
    encrypt        = true
    key            = "terraform.tfstate"
    profile        = "mfaAssume"
    key            = "${path_relative_to_include()}/terraform.tfstate"
    region         = "eu-central-1"
  }
}

This configuration worked properly! I did not have any messages from Terragrunt like

Error finding AWS credentials in file '~/.aws/credentials' (did you set the correct file name and/or profile?): NoCredentialProviders: no valid providers in chain. Deprecated

or

Error finding AWS credentials (did you set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables?): NoCredentialProviders: no valid providers in chain. Deprecated.

TEST

To be sure that it is all about the default profile I testes it by:

  1. renaming [default] to some other e.g. [myProfile] - DID NOT WORK
  2. leave the [default] as an empty profile and put its previous credentials to other e.g. [myProfile] - DID NOT WORK

During the tests my variable profile had the same value mfaAssume.

CONCLUSION It seems that for Terragrunt somehow the profile [default] must be present in case we want to use some other. I do not know whether it is because of some relation between [mfaAssume] and [default] or some other reason…

Is this the same error?

ERRO[0001] Error finding AWS credentials (did you set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables?): MissingEndpoint: 'Endpoint' configuration is required for this service
ERRO[0001] Unable to determine underlying exit code, so Terragrunt will exit with error code 1

If so, it was caused (for me) by a lack of a setting for AWS_REGION (or AWS_DEFAULT_REGION)

similar issue, easily being resolved by doing exactly what the msg asks to do, set glob ENVs via export like export AWS_ACCESS_KEY_ID=xxx export AWS_SECRET_ACCESS_KEY=xxx works 👍

I can confirm adding [default] profile in ~/.aws/credentials solved the problem for me, thanks @khdevel