terraform-provider-ibm: 504 Errors While Deploying LPARs on PowerVS

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave “+1” or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform IBM Provider Version

Terraform Version: v1.0.5
Terraform Provider Version: 1.33.1

Affected Resource(s)

ibm_pi_instance

Terraform Configuration Files

I’m utilizing the OCP on PowerVS terraform from the below URL

https://github.com/ocp-power-automation/ocp4-upi-powervs

Debug Output

Full debug here: https://gist.github.com/mmarakov/e779ce381c4a85fab57bcf8fc75f823c

Panic Output

Error: Failed to Get PVM Instance 6aa29573-28d0-46e3-b70d-d81914bd2539 :Get "https://lon.power-iaas.cloud.ibm.com/pcloud/v1/cloud-instances/98bdaa44-9166-48e6-903e-1d3b2e56f511/pvm-instances/6aa29573-28d0-46e3-b70d-d81914bd2539": context deadline exceeded

  with module.nodes.ibm_pi_instance.master[1],
  on modules/4_nodes/nodes.tf line 92, in resource "ibm_pi_instance" "master":
  92: resource "ibm_pi_instance" "master" {


Error: failed to provision Failed to Create PVM Instance :unknown error (status 504): {}

  with module.nodes.ibm_pi_instance.master[2],
  on modules/4_nodes/nodes.tf line 92, in resource "ibm_pi_instance" "master":
  92: resource "ibm_pi_instance" "master" {


Error: failed to provision Failed to Create PVM Instance :unknown error (status 504): {}

  with module.nodes.ibm_pi_instance.master[0],
  on modules/4_nodes/nodes.tf line 92, in resource "ibm_pi_instance" "master":
  92: resource "ibm_pi_instance" "master" {

Expected Behavior

Terraform should’ve continued monitoring build status and queryed the LPARs

Actual Behavior

PowerVS API returned 504 error causing Terraform to error out. This has happened in multiple PowerVS Regions. Sometimes it works. Other times it does not.

Steps to Reproduce

terraform apply

Important Factoids

Reached out to #terraform-ibmcloud-users on IBM Slack. Was requested to open an Issue. Relevant thread

About this issue

Original URL
State: closed
Created 3 years ago
Reactions: 5
Comments: 21

Most upvoted comments

We are seeing this issue too, in certain PowerVS locations, however we are calling it via the IBM cloud collection PI modules. After 5 or 6 mins, when the VSI moves from building to warning stage the module fails. It is looking for a 0 rc from Terraform, which it is not receiving.

cunlifs on Oct 22, 2021

The service broker team did many performance improvements to minimize the 504 errors.

rmoralesjr on May 25, 2023

which apiserver are we talking about here? retry logic is just a workaround, most of the client modules will a dumb wrapper to make rest calls and return.

The service broker where all the APIs are hosted.

I understand this is getting frequent but we cannot do anything from terraform other than handling the 504 errors and hope it works the next time. Have raised this with the API team but that will need investigation and permanent fixes from their side.

yussufsh on Oct 28, 2021