terragrunt: Don't convert inputs to env vars. Instead, generate terragrunt.tfvars.json and use as -var-file

I’ve been reading documentation for terraform 0.12 in regards to handling env vars, and https://www.terraform.io/docs/configuration/variables.html states that:

Some special rules apply to the -var command line option and to environment variables. For convenience, Terraform defaults to interpreting -var and environment variable values as literal strings, which do not need to be quoted

and

However, if a root module variable uses a type constraint to require a complex value (list, set, map, object, or tuple), Terraform will instead attempt to parse its value using the same syntax used within variable definitions files, which requires careful attention to the string escaping rules in your shell

And it’s so error-prone and counter-intuitive! Just take a look at https://github.com/gruntwork-io/terragrunt/issues/740 and https://github.com/gruntwork-io/terragrunt/issues/738

Instead, terraform directly supports loading variables from json files: https://www.terraform.io/docs/configuration/variables.html#variable-definitions-tfvars-files

Terraform 0.11 supports it as well, although it’s not properly documented:

Variables files use HCL or JSON syntax to define variable values

The main point here is that types are not lost and terraform doesn’t have to “guess” them. Also, the whole logic of converting inputs to env vars can be simplified to one json.Marshal.

Benefits:

Simplified logic
Supported by terraform 0.12 and properly documented
Types are not lost in translation, no need to set constraints in variable {} blocks
Works with terraform 0.11 nicely, for compatibility sake!

I can make a PR should it be needed.

About this issue

Original URL
State: open
Created 5 years ago
Reactions: 17
Comments: 24 (17 by maintainers)

Most upvoted comments

This isn’t a secret in the file. It’s a secret read using a Terragrunt helper that would only exist in memory… Unless we start writing var files to disk.

What if we can define env vars and var files ? Something like:

env_inputs = {
  password = ${get_env("MY_SECRET_PASSWORD")}
}

inputs = {
  instance  = "t2.nano"
  region    = "eu-west-1"
}

env_inputs will be treated like Terraform env vars TF_VARS_password and inputs will be converted into json.

I suspect that will be a serious problem for many users… So we’d be trading one mild problem (variables without a type being parsed incorrectly) to a bigger problem (reuse of values becoming very difficult).

I think that reuse of values will be difficult in Terraform 0.12, because they take the decision to move to explicite variable definition. This is neither good nor bad move, it’s juste a direction they take.

So should we follow them or not ? I’ll say yes, because Terragrunt is a wrapper. It should add value/feature but it’s not meant to change Terraform behavior. This could be confusing for users.

Saying that, how can Terragrunt become explicit without loosing DRY concept ? I think get_input() or a map with all merged inputs is a good direction. This map or locals can be used to explicitly define inputs in terragrunt.hcl :

inputs = {
  aws_region = local.region
  # or
  aws_region = get_input("region")
}

barryib on Jun 21, 2019