atlantis: Unable to override `required_version` terraform version using a flag
Community Note
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
- Please do not leave “+1” or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
- If you are interested in working on this issue or have submitted a pull request, please leave a comment.
Overview of the Issue
A new atlantis feature automatically tries to manage terraform versions for you, downloads the latest version for you and runs that instead. For repos that have floating version requirements like >=1.3 that means atlantis will try to download and use 1.4.0 for you and run that even if your state file is in 1.3.9. this should have been documented as a breaking change
The issue occurs when you have a policy check after a plan. The plan can run in tf 1.3.9 but the policy_check will run in a different terraform version, in this case 1.4.0 was released so atlantis tries to be clever and use the latest version. Of course that fails because the state file is in tf 1.3.9.
Looking at the docs I have tried setting the default version with these envs but they do nothing for the policy_check part.
ATLANTIS_DEFAULT_TF_VERSION: 1.3.9
DEFAULT_TERRAFORM_VERSION: 1.3.9
What does one need to do in order to turn off this “feature” completely and let us control what terraform version is being run? If not possible what flag / option is there to override the tf version to make sure that a consistent and same version is used at all times in all places?
Reproduction Steps
- your atlantis.yaml has an open
required_versionlike>=1.3. - use not the latest version of terraform to create an initial plan.
- let altantis do the rest
Environment details
- Atlantis version: 0.22.3
- Deployment method: helm
- If not running the latest Atlantis version have you tried to reproduce this issue on the latest version: no
- Atlantis flags:
ATLANTIS_AUTOMERGE: true
ATLANTIS_CHECKOUT_STRATEGY: merge
ATLANTIS_ENABLE_POLICY_CHECKS: true
ATLANTIS_QUIET_POLICY_CHECKS: true
ATLANTIS_ENABLE_REGEXP_CMD: true
ATLANTIS_SILENCE_NO_PROJECTS: true
ATLANTIS_DEFAULT_TF_VERSION: 1.3.9
DEFAULT_TERRAFORM_VERSION: 1.3.9
Atlantis server-side config file:
repos:
- id: /.*/
apply_requirements: [mergeable]
workflow: default
allowed_overrides: []
allow_custom_workflows: false
workflows:
default:
plan:
steps:
- env:
name: TF_WORKSPACE
value: ""
- env:
name: TF_IN_AUTOMATION
value: "true"
- env:
name: TF_PLUGIN_CACHE_DIR
command: 'echo "$ATLANTIS_DATA_DIR/plugin-cache"'
- run: terragrunt init
- run: |
terraform plan -input=false -refresh -no-color -lock=false -parallelism=5 $(eval echo ${COMMENT_ARGS}) -out $PLANFILE 2>&1 > out.log
exitCode=$?
cat out.log | grep -v 'Refreshing state...' | sed 's/^ +/+/' | sed 's/^ ~/~/' | sed 's/^ -/-/'
exit $exitCode
policy_check:
steps:
- env:
name: TF_WORKSPACE
value: ""
- env:
name: ATLANTIS_POLICIES_DIR
command: 'echo "${DIR%$REPO_REL_DIR}policies/"'
- env:
name: ATLANTIS_CURRENT_DIR
command: 'echo "${DIR}"'
- show
- policy_check:
extra_args: ["--policy ${ATLANTIS_POLICIES_DIR} --all-namespaces"]
Repo atlantis.yaml file: its 16k lines. can provide relevant info if needed
About this issue
- Original URL
- State: open
- Created 4 years ago
- Reactions: 15
- Comments: 27 (15 by maintainers)
Perhaps we need a new flag such as this?
It can default to true unless specified as false. By enabling this potential flag, atlantis will stop reading the
required_versionargument and stop downloading the latest terraform that matches that argument.yes.
On Fri, Aug 25, 2023 at 9:58 PM Will Swinson @.***> wrote:
Same here today. We use atlantis since quite some time and this change was very surprising. I also tried the different environment variables to pin a specific terraform version and also used
required_version>= x.y.z like @ekhaydarov . I would say this is very common and totally agree that settingrequired_versionlike that should not influence atlantis in a way that the latest version should be applied automatically.I apologise for my lack of understanding here. I am also having a hard time seeing why atlantis needs to manage terraform versions for you? Thats outside the scope of the tool is it not?
Consider most CI workflows are all running on pretty much custom images that have all their inputs controlled. i.e. we set the atlantis version in our docker image, we set the terragrunt version, we set the terraform version. If atlantis randomly starts using differnt version of terraform why doesnt atlantis just update itself at this point and use latest for everything? I am not understanding why atlantis needs to control the tf version for you? end users need that right in normal workflows, not only in custom ones surely?
Secondly with overriding tf version through
required_version. There are cases when project specific pinning needs to happen in cases like tf version only needs to be above 1.3 since they implemented a new feature for optional variable defaults. That lets our teams testing locally know that any version above 1.3 will be fine. Why would atlantis need to look at that and try to be smart for you pretty much all Ci’s are custom images where teams tightly control all their versions of tools that go into that image. This atlantis change forces us to pin exact version all the time and all developers will at all times need to match required version.Weve clearly hit some issue here though. I have set the Default version, so the plan perhaps picks up the version
1.3.9. the policy check picks up1.4.0. If I do not set the default version, the plan tries to bump to v1.4.0 when there is no need for us to do that right now, our CI image has no clue about1.4.0, were sitting there thinking what in our dockerfile managed to put tf1.4.0in there. Meanwhile our own tf installation is entirely redundant now.So if we have a fork of terraform we cannot run it unless its a custom workflow? I apologies but i fail to see how this is not a breaking change? My atlantis.yaml needs a rerite to make our forked tf work now. thats a breaking change surely?
I am simply asking for a contingency where a user can simply use a specified tf version in all usecases. Currently its inconsistent, do you not agree? Why would
required_versionneed to be hard coupled to tf version atlantis chooses, they should not be linked at all and managing tf version is not atlantis’ responsibility IMO. Again i may be lacknig understanding and appreciate if there are other reasons for this, but I am failing to see that.So setting that would allow the policy check command to use
1.3.9? if I set tf1.2.0will atlantis go an download that for me? and use that in call instances? If that is the case I did not parse the docs correctly and this issue can be closedBut also if that is the case i dont understand what these variables are for and they dont work as their naming implies. They clearly dont default for all commands. They are confusing as there are two of those variables. Differences are confusing to end users, why is there a need for both? Can they be just be removed? Setting tf version explicitly though atlantis yaml and implicitly through
required_versionis enough. Why provide extra variables that dont actually do what you want?I hope this is constructive feedback and it doesnt feel like an atttack. Just would like to get better understanding Thanks for the OS work and trying to help
No, the old behaviour worked due to the required_version not being respected before atlantis version ~0.23.0~ 0.21.0 (docs) EDIT: wrong version
Another flag that could be added to allow downloading but give it a maximum version so there is a central place where the tf version can be controlled
Both of the above could default to empty strings to maintain the current workflow
We saw this issue also on
v0.23.2