terraform-provider-aws: [Bug]: After switching an RDS instance from gp2 to gp3, following apply will fail if IOPS is not specified
Terraform Core Version
1.2.3
AWS Provider Version
4.46.0
Affected Resource(s)
- aws_db_instance
Expected Behavior
If specifying iops is mandatory when choosing gp3:
- You should not be able to apply the change without specifying it, a validator is missing
If IOPS is NOT mandatory when choosing gp3:
- You should not be blocked during further apply, and your plan should not describe a change about
iops
Actual Behavior
The first apply was successful, allowing me to switch an existing RDS instance from GP2 to GP3. But now my pipeline is blocked with the following plan/error
~ resource "aws_db_instance" "this" {
id = "a_db"
~ iops = 3000 -> 0
name = "a_db"
....
│ Error: updating RDS DB Instance (a_db): operation error RDS: ModifyDBInstance, https response error StatusCode: 400, RequestID: 61792e08-c93b-4834-a35c-eb029b29ba30, api error InvalidParameterCombination: You can't specify IOPS or storage throughput for engine postgres and a storage size less than 400.
I can work around the error by adding to my resource the optional iops parameter with a value of 3000, unblocking my pipeline by avoiding the change:
resource "aws_db_instance" "this" {
id = "a_db"
iops = 3000
name = "a_db"
...
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Relevant Error/Panic Output Snippet
No response
Terraform Configuration Files
resource "aws_db_instance" "default" {
...
allocated_storage = 20
storage_type = "gp2"
}
After first apply (you have an RDS instance on gp2), change storage_type, and apply two times: after the 1st apply you are on gp3, but now the 2nd (and following) applies will fail:
resource "aws_db_instance" "default" {
...
allocated_storage = 20
storage_type = "gp3"
}
Steps to Reproduce
- Create an RDS instance (postgres might be important), with a gp2 volume of 20 GB
- After first apply, switch it to gp3 without specifying the IOPS parameter
- Try to apply again, you should now see an unexpected change:
- 0 → 3000 IOPS
- Your apply will fail with the following error:
api error InvalidParameterCombination: You can't specify IOPS or storage throughput for engine postgres and a storage size less than 400
Debug Output
No response
Panic Output
No response
Important Factoids
No response
References
No response
Would you like to implement a fix?
None
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 13
- Comments: 19 (4 by maintainers)
As a conclusion, my feeling is that there is a bug with the AWS Provider module:
gp3gp3while specifyingiops/storage_throughputto the default values if your volume is smaller than400GBstorage_type = gp3in combination with the parametersiops/storage_throughputif your volume is smaller than400GBThis bug is turning to be quite blocking for the users using: https://github.com/terraform-aws-modules/terraform-aws-rds, as explained by @TomWizen, since you cannot have a valid configuration without 2x PRs, and this is because of the default values used by the module for the variable
iops/storage_throughput@etiennechabert Thanks for the very thorough write-up of your failing scenarios 👏. I think the most maintainable answer here is to add to the
aws_db_instanceresource documentation noting thatiopsandstorage_throughputcannot be specified with certain combinations ofengineandallocated_storage(and to link to relevant AWS documentation) so that practitioners are warned that they may have to remove these attributes’ values from Terraform code. Trying to add further logic to the provider to deal with configured values that should be ignored (rather than computed values returned from the RDS API) will further complicate an already large resource.We are facing the same issue. When trying to configure a migration from gp2 to gp3 with iops parameter it is failing with the above error. The only possible way to do it right now it to separate it to two terraform runs, which is not ideal.
I’ve hit the same with AWS RDS module because it has explicit
variable "iops" { default = 0 }, specifyingiops = nullexplicitly makes clean plans.Yes