terraform-provider-tfe: `tfe_workspace` `vcs_repo` error

For anyone trying to get it working, here is what worked for me:

Create an oauth client:

resource "tfe_oauth_client" "github" {
  organization     = tfe_organization.org.name
  api_url          = "https://api.github.com"
  http_url         = "https://github.com"
  service_provider = "github"
  oauth_token      = var.github_oauth_token
}

Where var.github_oauth_token is a personal access token to a machine user.

I gave the machine user’s personal access token the following scopes: admin:repo_hook, repo and an owner permissions over the organization (sadly couldn’t figure out how to narrow down access for the user itself).

Then used it in the workspace definition:

resource "tfe_workspace" "test" {
  name         = "test"
  organization = tfe_organization.org.name
  vcs_repo {
    identifier     = "<redacted>/test-tf"
    oauth_token_id = tfe_oauth_client.github.oauth_token_id
  }
}

that script that I posted in the gist helps with getting the correct token. Though, I believe it will only work if you have one VCS connection in the Org.

https://gist.github.com/djaboxx/4749e44551c8326aa39fd64da3c20e80

In order to use this script, you need to

pip install pyhcl

and make sure that your user token is setup from https://app.terraform.io/app/settings/tokens

in ~/.terraformrc

credentials "app.terraform.io" {
  token = "<user_token>"
}

This still took me a while to figure out, so just making it suuper clear how to declare the tfe_oauth_client data source:

data "tfe_oauth_client" "client" {
  # In TF Cloud under VCS Providers, you find the OAuth Token ID starting with "ot-"
  # Now go to https://app.terraform.io/api/v2/organizations/<Your org>/oauth-tokens
  # There you will find a relationship to an OAuth client with an ID starting with "oc-"
  oauth_client_id = "oc-<redacted>"
}

Thanks to @svanharmelen suggestion to just use the browser and @djaboxx explanation of the mechanism I was able to hack around the limitations, but it would be awesome if this would be easier, or at least documented in the provider docs.

Tha main confusion is regarding how to get the oauth_token_id and what it is exactly.

For anyone that needs to use a shell, try this:

curl -v -H "Authorization: Bearer <redacted>" https://app.terraform.io/api/v2/organizations/<redacted>/oauth-tokens | jq .data[].id

I tried using data "tfe_oauth_client" "client_id" on our self hosted TFE which is connected to Gitlab. On the docs here it shows an example with the token id with “oc” prefix. However, when I look at the settings on our TFE instance, all the VCS provider tokens begin with “ot”.

Hence it is within expectations that data.tfe_oauth_client.client_id fails… I’d like to understand why. What exactly does “oc” refer to? From the above comment, does it mean this data block only supports Personal Access tokens?

On a positive note, I did manage to configure a tfe workspace using the tfe_workspace resource by passing the oauth token id directly instead of using the data block to first retrieve it.

TLDR: What is the difference between “oc” and “ot” tokens? What do the prefixes mean? How should data.tfe_oauth_client be used exactly?

@pgrinstead1 at the moment this is hard as a token does not have a way to be uniquely identified. But if you only have one VCS connection, it’s a bit easier to find the correct ID.

For now I suggest you open the following URL in your browser (when logged into PTFE, so with an authenticated session): https://ptfe.company.com/api/v2/organizations/your-org-name/oauth-clients

This will return a JSON document containing a list of all your defined VCS connections with their attributes and relationships. In the relationships section there will be an oauth-tokens entry which contains the token ID.

We understand that this is far from perfect, but for now this is probably the best we can do.

Another solution would be to create a new VCS connection using the tfe_oauth_client resource. It’s fine to have multiple VCS connections to the same VCS service so you could add the new connection next to the existing connection and then reference the exported oauth_token_id in resources that need the ID (e.g. tfe_workspace).

I hope this helps solve the issue for now?

Cool… Thanks for your help @djaboxx!

And as mentioned we are working on improving this in a next release, to make it easier to get the correct token.