atlantis: Atlantis v0.5.0 with apply_requirements: [mergeable] wont work

Hi,

I’m testing the latest release of Atlantis, v0.5.0 and have enabled the new check on the repo for apply/atlantis and apply/atlantis. After enabling those I’ve found out that after a successfully plan, when commenting on the PR with atlantis apply I am not able to merge branch if I have the setting apply_requirements: [mergeable] configured on atlantis.yaml If I disable everything works.

Issue New Atlantis checks incompatible with apply requirement == mergeable

Expected Behaviour Expected new checks not to impact mergeable requirement


Atlantis Plan - OK screenshot-github com-2019 03 08-10-54-11

Atlantis Apply - Fails screenshot-github com-2019 03 08-10-55-15

atlantys.yaml config

version: 2
automerge: true
projects:
  - name: poc-dev
    dir: poc-aws-roles-poc/poc-dev/terraform
    apply_requirements: [mergeable]
    autoplan:
      when_modified: ["../dev-policies/*.terraform", "*.tf", "*.tfvars"]
    workflow: landscape

About this issue

  • Original URL
  • State: closed
  • Created 8 years ago
  • Reactions: 15
  • Comments: 31 (21 by maintainers)

Most upvoted comments

This was our work around since we have an org wide status check we wanted atlantis to ignore in addition to atlantis/apply: https://github.com/lyft/atlantis/blob/release-v0.17.0-beta-lyft.1/server/events/vcs/github_client.go#L296

If this gets enough likes I can merge just the part about atlantis/apply to the upstream.

Yeah the mergeable check won’t work with the status checks unfortunately. There’s no good way to solve it either because GitHub doesn’t tell us why a pull request isn’t mergeable and we can’t do:

  • check if pull is mergeable
  • if not, check if atlantis statuses aren’t passing
  • if atlantis statuses aren’t passing, assume that’s the reason it’s not mergeable and continue

I think we’ll have to add some new apply requirements like checks_passing and codeowners_approved which are more fine-grained.

This seems to still be an issue using the latest version v0.18.1

Is there any changes on this issue? I am surprisingly starting to hit this one since Thursday. I wonder how I was able to not hit this issue before and what has been changed on Thursday, June 17, 2021?

This was our work around since we have an org wide status check we wanted atlantis to ignore in addition to atlantis/apply: https://github.com/lyft/atlantis/blob/release-v0.17.0-beta-lyft.1/server/events/vcs/github_client.go#L296

If this gets enough likes I can merge just the part about atlantis/apply to the upstream.

hey @nishkrishnan any chance you can merge this? seems like a pretty common usecase that a lot of folks are running into

Looks like this is still an issue in v0.19.3. @nishkrishnan looks like you added support for filtering out apply from the mergeability, but then it was reverted with #1968? Looks like there was a regression associated with that that allowed unapproved plans to be applied.

@pratikmallya thanks for some thoughts on work arounds in the meantime and that’s what I am about to do.

I spoke with @lkysow and we think that the best fix would be during the apply stage to send an ok/passing status to atlantis/apply prior to checking its mergability, in the case that mergability still fails it should rescue and set the status of that one to false to prevent someone from accidentally merging it.

There is a somewhat easy way around this in Github at least.

  • enable the Github branch protection option: ‘Restrict who can push to matching branches’ to only the atlantis github svc account and
  • enable the merge on apply option for atlanits
  • leave the apply/atlantis check as NOT REQUIRED

This will:

  • allow atlantis apply to run because apply/Atlantis is not a required status check
  • prevent anyone from merging branch except atlantis svc account
  • atlantis svc account can merge any changes once all plans are applied

This setup allows you to not use required check for apply/Atlantis while continuing to be able to:

  • gate merges on successful applies of all plans and
  • allow applies to run even though apply success is required

i.e. the merge only if apply succeeds condition is met by the auto-merge option, which will merge iff all the plans are applied.

@lkysow We check that:

  • Required status checks pass (excluding apply/atlantis, although we still want this to be required from GH’s perspective before actually merging a PR). IMHO, this is the most important.
  • Number of required approvals have been given (this one isn’t so important for us, because we have a custom status check to require that certain people/teams have approved the PR).
  • There are no conflicts with the base branch.
  • CODEOWNERS have given their approval (probably covered by required approvals above).

We don’t care if Atlantis is actually able to merge the PR to the base branch or not. In other words, Atlantis doesn’t need to see the big green merge button to consider the PR mergeable.

From my perspective, all the usual mergeability checks would be nice, only excluding:

  1. The apply/atlantis status check (even if GH considers it a required status check under branch protection)
  2. That the Atlantis GH user sees the big green merge button (ie. has write/push permission to the base branch).

yep sorry been swamped with work lately, will try to get this in the next week or so.

i wrote to github and support said it would forward it internally… but of course, no promises nor ETA

@majormoses @lkysow Sorry to nag, but any new on this?

we have to choose between (@pratikmallya method) not require the apply (but admins may merge by accident without applying and not changing the resources) or allow the apply without mergeability (and allow one to apply outdated info)

How about contact github, maybe they could extend the api to allow this checks in a better way?

I’m getting this error, and even though atlantis/apply is not ‘required’, and all required status checks are passing, it’s still barfing.

yet: image

[requiring approval instead hasn’t been working for me so far]

To debug, log in as the Atlantis user and look at the pull request, is the merge button clickable?

The checks for mergeable would stop the atlantis apply to be run before the mergeable condition is met.

Without the mergeable check configurations start to drift.

The example is this:

  • Create PR with conflict
  • atlantis plan -> check turns ok
  • atlantis apply -> check turns ok
  • auto merge fails
  • changes contained on the PR where apply on the target infra
  • master is different from the provisioned infra
  • PR is still not merged