terrakube: Workspace Importer is not importing state
Bug description š
Iāve just noticed that whenever I use the Workspace Importer to import a workspace from Terraform Cloud to Terrakube whilst the actual operation gets reported as being successful, when I actually go to look at the workspace in Terrakube there is no state! Looking at the API log I can see:
INFO 1 --- [nio-8080-exec-8] o.t.a.p.i.t.services.WorkspaceService : VCS ID:
ERROR 1 --- [nio-8080-exec-8] o.t.a.p.i.t.services.WorkspaceService : URI is not absolute
But unfortunately no more information from the log.
The (sanitised) payload being sent to the workspaces/ endpoint in this case was:
{
"organizationId": "xxxxxxxx-8372-4fec-8500-xxxxxxxxxxxx",
"vcsId": "",
"id": "ws-49WuV6tyxxxxxxxx",
"organization": "xxxxxx",
"name": "xxxxxx-xxxxxxx-xxxxxxxxxx",
"terraformVersion": "1.5.7",
"description": null
}
Iāve tried this for a few of our workspaces and unfortunately I seem to get the same issue each time.
Steps to reproduce
- Import a workspace from Terraform Cloud using the Workspace Importer
- Check the imported workspace and look at the empty state
Expected behavior
State should be imported.
Example repository
No response
Anything else?
No response
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 1
- Comments: 21 (9 by maintainers)
When you do a
terraform pull, internally the terraform CLI is using the following API method to download the RAW state file.When you do a
terraform state push, internally the terraform CLI is using the following API method, depending of the terraform CLI version it could also use this API method (if I remember correctly this one is use in terraform above 1.6.X) to upload theterraform stateand thejson statein two different request.So I guess that when you run
terraform state pushyou are using a terraform version above 1.3 that is including the ājson-stateā that is generated internally by terraform CLI.If you tried to do a
terraform state pushwith an earlier terraform version below 1.3 you wonāt see the json state representation in the UI.Finally if you will be using the
"local execution mode"when the terraform CLI run the apply and do the request to create state using the method it wont include the property ājson-stateā when using terraform version bellow 1.3.So as an alternative for now you could try to migrate at least to terraform 1.3 so when you use the terraform cli in local execution mode and run an apply it will create the ājson stateā
By the way we are having another discussion of using the ājson state in the UIā here, maybe in the near future we will have to adjust that part of the UI and use the RAW state there insted of the json representation
Excellent. Thanks for the swift response fixing this @jcanizalez š
I have added this to 2.20.0-beta.4 if you want to test it. However, it has a known issue, #733, due to which you wonāt be able to see the Modules in the Registry section. You can either try this version or wait for the next beta release, in which we will attempt to fix this issue.
Thanks @dcopestake This information has been invaluable; finally, I succeeded in recreating the issue by using a workspace with Terraform versions earlier than 1.3. I have updated the importer to import the state, allowing its use even if the JSON state is not available. However, it will display a warning like this:
Hi @dcopestake Iāve tested using MinIO and confirmed that itās working fine. As a result, I have released version 2.20.0-beta.3. This version introduces support for importing tags and also enhances logging to better debug your issue. However, it also includes a known issue with the Registry for Redis, preventing the visibility of Modules in the UI. If youāre interested in trying out the importer, you can either upgrade to this version or wait for the next beta release, where we aim to resolve the Redis issue. Below is an example of a successful log entry that you can use for comparison in your logs.
UI
API
After Importing I can even execute a Plan and the plan is running using the imported state
@alfespa17 if you get some time please try out and let us know any findings
When you open gitpod you can use the following command in the terminal.
It will do all the setup so you can use MINIO as backend storage inside Gitpod, that is basically the same code that when you are using a S3 in AWS. I think that can help you to test
Hello @dcopestake, Iām currently developing and testing the importer from Gitpod using local storage, and itās working well. Based on the feedback provided by you and @iam-take, I suspect the issue might be related to use the importer with a cloud storage. I need to create an Azure storage account or use S3 to test it in that environment. I will try to configure environment this week to debug
We experienced the same issue when trying to migrate state into Terrakube. We had used the Azure Storage Account as backend storage for Terrakube. When looking into the API log we see the same kind of logging as @dcopestake. One difference is that I did see the state file in the Azure Storage Account. It just shows as empty in Terrakube.
It seems not to matter where your orginal state is located the import goes wrong.