atlantis: atlantis testdrive failed with 422 Validation Failed [{Resource:PullRequest Field:base Code:invalid Message:}]

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
  • Please do not leave “+1” or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Overview of the Issue

I tried atlantis testdriv with following the instruction. But I couldn’t let atlantis create a PR, even though I give a personal GitHub token full access.

Reproduction Steps

./atlantis testdrive

Logs

Logs
$ ./atlantis testdrive

Welcome to Atlantis testdrive!

This mode sets up Atlantis on a test repo so you can try it out. We will
- fork an example terraform project to your username
- install terraform (if not already in your PATH)
- install ngrok so we can expose Atlantis to GitHub
- start Atlantis

Press Ctrl-c at any time to exit


github.com username: pomcho555

To continue, we need you to create a GitHub personal access token
with "repo" scope so we can fork an example terraform project.

Follow these instructions to create a token (we don't store any tokens):
https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token#creating-a-fine-grained-personal-access-token
- use "atlantis" for the token description
- add "repo" scope
- copy the access token

GitHub access token (will be hidden):
=> forking repo
=> fork completed!
=> terraform found in $PATH at /usr/local/bin/terraform
=> ngrok not found in $PATH.
=> downloading ngrok
=> downloaded ngrok successfully!
=> creating secure tunnel
=> started tunnel!
=> starting atlantis server
â ™=> atlantis server is now securely exposed at https://80fb-113-36-202-140.ngrok.io

=> creating atlantis webhook
=> atlantis webhook created!
=> creating a new pull request
â ąError: creating new pull request for repo pomcho555/atlantis-example: POST https://api.github.com/repos/pomcho555/atlantis-example/pulls: 422 Validation Failed [{Resource:PullRequest Field:base Code:invalid Message:}]

Usage:
  atlantis testdrive [flags]

Flags:
  -h, --help   help for testdrive

Environment details

Version: atlantis 0.22.3 OS: macOS Ventura 13.0.1 (22A400)

Additional Context

  • I tried a solution shared in #2329, but it failed also.

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 6
  • Comments: 15 (7 by maintainers)

Most upvoted comments

BTW, if you re-try atlantis testdrive with already forked atlantis-example repo, it may spit out the following error.

=> creating a new pull request
â ™Error: creating new pull request for repo pomcho555/atlantis-example: POST https://api.github.com/repos/pomcho555/atlantis-example/pulls: 422 Validation Failed [{Resource:PullRequest Field:base Code:invalid Message:}]

To solve this, you need to remove the existing repo.

It works well with v0.23.5. So I think you can close this issue.

So 0.19.9 is from October 2022

Here are all the changes in the testdrive module

https://github.com/runatlantis/atlantis/commits/3212fe9cbdb831b0bf1ffd4e2851fbfed4215e73/testdrive

Most of the changes seem to be package upgrades

The pr issue seems to be related to here

https://github.com/runatlantis/atlantis/blob/3212fe9cbdb831b0bf1ffd4e2851fbfed4215e73/testdrive/testdrive.go#L255-L260

https://github.com/runatlantis/atlantis/blob/3212fe9cbdb831b0bf1ffd4e2851fbfed4215e73/testdrive/github.go#L73-L98

Upstream example

https://github.com/google/go-github/blob/31350112a17a67bdc672f2a73865a2985b15fd60/example/commitpr/main.go#L164-L175

Maybe the base should be master instead of main? We changed to main in Nov 2022 and this pr changed the base to main in testdrive https://github.com/runatlantis/atlantis/pull/2662 (v0.21.0).

Perhaps the fork doesnt have a main branch?

https://github.com/runatlantis/atlantis/blob/3212fe9cbdb831b0bf1ffd4e2851fbfed4215e73/testdrive/testdrive.go#L39-L40

https://github.com/runatlantis/atlantis/blob/3212fe9cbdb831b0bf1ffd4e2851fbfed4215e73/testdrive/testdrive.go#L117-L121

Yep looks like the example is still set to master

https://github.com/runatlantis/atlantis-example

@lkysow could you add a main branch for now to the above? I do not have permission.

We could also update testdrive to always retrieve the base branch of the fork prior to creating the pull request.


The ngrok version is interesting

https://github.com/runatlantis/atlantis/blob/3212fe9cbdb831b0bf1ffd4e2851fbfed4215e73/testdrive/utils.go#L39 https://github.com/runatlantis/atlantis/blob/3212fe9cbdb831b0bf1ffd4e2851fbfed4215e73/testdrive/testdrive.go#L154-L157

Latest stable is https://bin.equinox.io/c/4VmDzA7iaHb/ngrok-v3-stable-linux-amd64.tgz from ngrok website. The new url contains a v3 meaning the testdrive must be using the previous stable which I assume is v1/v2. Maybe we should see what changes need to be made to make v3 compatible.

Here is the current ngrok config that’s v1/v2 compatible and breaking for v3

https://github.com/runatlantis/atlantis/blob/3212fe9cbdb831b0bf1ffd4e2851fbfed4215e73/testdrive/testdrive.go#L175-L183

Seems like the config could be run through an upgrade command

ngrok config check
ngrok config upgrade

Or use the config posted above by @mdmelo

-version: 1
+version: "2"
+region: us
 web_addr: %s 
 tunnels: 
   atlantis: 
     addr: %d
-    bind_tls: true
+    schemes:
+    - https
     proto: http

you also need to downgrade to ngrok 2.x if running ngrok v3.x, i had 3.2.1 installed and ngrok didnt like the config file created by atlantis. you can manually upgrade the ngrok config in /tmp to verify this. downgrade to ngrok v2.3.41 worked around this issue.

this

web_addr: localhost:41414
tunnels:
  atlantis:
    addr: 4141
    bind_tls: true
    proto: http

needs to be this for use with ngrok v3

web_addr: localhost:41414
tunnels:
    atlantis:
        addr: 4141
        schemes:
            - https
        proto: http
version: "2"
region: us

I used an older release and ran testdrive successfully.

This release can work: https://github.com/runatlantis/atlantis/releases/download/v0.19.8/atlantis_linux_amd64.zip