terragrunt: AWS CLIv2 and AWS SSO auth fails

We are using new aws cli v2 SSO auth feature. This works with aws cli, but terragrunt fails to get credentials.

$ aws --version
aws-cli/2.0.6 Python/3.8.2 Darwin/18.7.0 botocore/2.0.0dev10

.aws/config


[profile 123456789012-AWSAdministratorAccess]
sso_start_url = https://d-123456789.awsapps.com/start
sso_region = eu-west-1
sso_account_id = 123456789012
sso_role_name = AWSAdministratorAccess
region = eu-west-1
output = json

Check aws cli works

$ export AWS_PROFILE=123456789012-AWSAdministratorAccess
$ aws sso login
Attempting to automatically open the SSO authorization page in your default browser.
If the browser does not open or you wish to use a different device to authorize this request, open the following URL:

https://device.sso.eu-west-1.amazonaws.com/

Then enter the code:

REDACTED
Successully logged into Start URL: https://d-123456789.awsapps.com/start
$ aws s3 ls
2020-04-08 08:09:44 tf-states.lab.REDACTED

$ aws sts get-caller-identity
{
    "UserId": "REDACTED:redacted@redacted",
    "Account": "123456789012",
    "Arn": "arn:aws:sts::123456789012:assumed-role/AWSReservedSSO_AWSAdministratorAccess_abcdef12345/redacted@redacted"
}
$ terragrunt plan --terragrunt-source ~/REDACTED/git/tf-modules//aws/bootstrap
[terragrunt] 2020/04/09 10:38:44 Terragrunt Version: v0.23.6
[terragrunt] 2020/04/09 10:38:44 Reading Terragrunt config file at /Users/REDACTED/git/cloud-config/terraform/lab/aws/bootstrap/terragrunt.hcl
[terragrunt] 2020/04/09 10:38:44 Did not find any locals block: skipping evaluation.
[terragrunt] 2020/04/09 10:38:44 Running command: /Users/REDACTED/git/cloud-config/terraform/../scripts/aws/get_admin_roles.sh
AWSReservedSSO_AWSAdministratorAccess_abcdef12345[terragrunt] 2020/04/09 10:38:46 run_cmd output: [AWSReservedSSO_AWSAdministratorAccess_abcdef12345]
[terragrunt] 2020/04/09 10:38:46 Found locals block: evaluating the expressions.
[terragrunt] 2020/04/09 10:38:46 Evaluated 5 locals (remaining 8): cloud, environment, default_empty_yaml, api_domain, aws_vars
[terragrunt] 2020/04/09 10:38:46 Evaluated 3 locals (remaining 5): terraform_module_local_path, module_settings_path, aws_root_domain
[terragrunt] 2020/04/09 10:38:46 Evaluated 2 locals (remaining 3): terraform_module_path, terraform_module_version
[terragrunt] 2020/04/09 10:38:46 Evaluated 3 locals (remaining 0): terraform_module_name, default_tags, terraform_source_default
[terragrunt] [/Users/REDACTED/git/cloud-config/terraform/lab/aws/bootstrap] 2020/04/09 10:38:51 Running command: terraform --version
[terragrunt] 2020/04/09 10:38:51 Downloading Terraform configurations from file:///Users/REDACTED/git/tf-modules into /Users/REDACTED/git/cloud-config/terraform/lab/aws/bootstrap/.terragrunt-cache/bL_WKXycpKiko0WERZJHEmdsjyA/fnop9qC_WS7CKETpjnoJj_MM_ro
[terragrunt] 2020/04/09 10:38:51 Copying files from /Users/REDACTED/git/cloud-config/terraform/lab/aws/bootstrap into /Users/REDACTED/git/cloud-config/terraform/lab/aws/bootstrap/.terragrunt-cache/bL_WKXycpKiko0WERZJHEmdsjyA/fnop9qC_WS7CKETpjnoJj_MM_ro/aws/bootstrap
[terragrunt] 2020/04/09 10:38:51 Setting working directory to /Users/REDACTED/git/cloud-config/terraform/lab/aws/bootstrap/.terragrunt-cache/bL_WKXycpKiko0WERZJHEmdsjyA/fnop9qC_WS7CKETpjnoJj_MM_ro/aws/bootstrap
[terragrunt] 2020/04/09 10:38:51 The file path /Users/REDACTED/git/cloud-config/terraform/lab/aws/bootstrap/.terragrunt-cache/bL_WKXycpKiko0WERZJHEmdsjyA/fnop9qC_WS7CKETpjnoJj_MM_ro/aws/bootstrap/provider.tf already exists and if_exists for code generation set to "overwrite". Regenerating file.
[terragrunt] 2020/04/09 10:38:51 Generated file /Users/REDACTED/git/cloud-config/terraform/lab/aws/bootstrap/.terragrunt-cache/bL_WKXycpKiko0WERZJHEmdsjyA/fnop9qC_WS7CKETpjnoJj_MM_ro/aws/bootstrap/provider.tf.
[terragrunt] 2020/04/09 10:38:51 The file path /Users/REDACTED/git/cloud-config/terraform/lab/aws/bootstrap/.terragrunt-cache/bL_WKXycpKiko0WERZJHEmdsjyA/fnop9qC_WS7CKETpjnoJj_MM_ro/aws/bootstrap/backend.tf already exists, but was a previously generated file by terragrunt. Since if_exists for code generation is set to "overwrite_terragrunt", regenerating file.
[terragrunt] 2020/04/09 10:38:51 Generated file /Users/REDACTED/git/cloud-config/terraform/lab/aws/bootstrap/.terragrunt-cache/bL_WKXycpKiko0WERZJHEmdsjyA/fnop9qC_WS7CKETpjnoJj_MM_ro/aws/bootstrap/backend.tf.
[terragrunt] [/Users/REDACTED/git/cloud-config/terraform/lab/aws/bootstrap] 2020/04/09 10:38:51 Initializing remote state for the s3 backend
[terragrunt] 2020/04/09 10:38:51 Error finding AWS credentials (did you set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables?): NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors
[terragrunt] 2020/04/09 10:38:51 Unable to determine underlying exit code, so Terragrunt will exit with error code 1

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 9
Comments: 27 (14 by maintainers)

Most upvoted comments

In case like this one or other similar cases where AWS SSO result in incompatibilities with your library and you don’t want to play with workarounds or complicated fixes, maybe you can give a try to our open-source project: https://github.com/Noovolari/leapp. It deals with AWS SSO authentication and accounts/roles retrieval then it creates short-lived temporary credentials in .aws/credentials to maximize compatibility with third party tools / sdks.

urz9999 on Jan 14, 2021

this should be fixed in the AWS Go SDK in version 1.37: https://github.com/aws/aws-sdk-go/releases/tag/v1.37.0

can we somehow get this into terragrunt? i am not that familiar with go, but for my understanding it should be only updating the version here: https://github.com/gruntwork-io/terragrunt/blob/a7c0d434970c8519f5bec377e9c5bee9df25a6e7/go.mod#L7 ?

can someone confirm? if so, i would be able to provide a PR

hereandnow on Feb 3, 2021

I’ve created: https://github.com/gruntwork-io/terragrunt/pull/1537

z0mbix on Feb 9, 2021

AWS SSO is going to be a mess across the community because it does not make credentials available in a way that works with the current credential chain. Each SDK will need to implement support for the new credential chain, and perhaps each tool also (depending on how they setup their chain).

For GO, see: https://github.com/aws/aws-sdk-go/issues/3186

lorengordon on Apr 9, 2020

We just updated the AWS Go SDK version in Terragrunt, which, in theory, should allow AWS SSO / AWS CLI v2 to work with the new release: https://github.com/gruntwork-io/terragrunt/releases/tag/v0.28.9 (binaries should show up shortly). Please give it a shot!

brikis98 on Mar 11, 2021

Wow, in the meantime I think I managed…Or I made who knows what with lot of credentials etc… I will have to do it all over again with everything 😃 Now it looks it might passed, but will have to double check. Didn’t see that it is deprecated. I will try the new one.

UrosCvijan on Feb 26, 2021

@UrosCvijan, did you install aws-sso-credential-process? Actually that tool is deprecated, the author replaced it with aws-sso-util

oscarmorasu on Feb 26, 2021

Thanks for starting this work @z0mbix

Note Terraform already supports SSO creds started with version 0.14.6 (for the backend) and in version 3.26.0 for the Provider

I had to pin Terraform 0.14.5 + Provider 3.25.0 until Terragrunt also supports SSO. The reason is I am not using AWS Key/Secret Key, but I leverage the aws-sso-credential-process to bridge the gap.

Basically my profiles in the .aws/config file look like this:

[profile my-profile]
credential_process = aws-sso-credential-process --profile my-profile
sso_start_url = https://myorg.awsapps.com/start
sso_region = us-west-2
sso_account_id = 1234567890
sso_role_name = MyRole
region = us-west-2
output = json

Terraform 0.14.6+ blows up because both the credential_process and the sso config are present. If I remove the credential_process now Terragrunt blows up because it’s unaware of the sso config and needs the AWS Key/Secret Key.

oscarmorasu on Feb 22, 2021

@z0mbix : when you have already done it, can you provide a PR for this?

Yup, will do.

z0mbix on Feb 9, 2021