aws-cdk: aws_rds: Fails to create DB due to error "DBInstance ady9tx26mcix2t not found. (Service: AmazonRDS; Status Code: 404; Error Code: DBInstanceNotFound"

Describe the bug

I am trying to create a Postgres DB using CDK. When I do this the create fails with the errorMsg: DBInstance ady9tx26mcix2t not found. (Service: AmazonRDS; Status Code: 404; Error Code: DBInstanceNotFound.

It seems CDK is not waiting for AWS to initialise the DB properly before some lookup.

Expected Behavior

I would expect DB to get created.

Current Behavior

CDK starts to provision DB and I can see that the DB is created in the AWS console. This is the CDK output log:

This is the CDK output:

ArchiveService2Stack | 0/7 | 7:14:11 AM | CREATE_IN_PROGRESS   | AWS::CloudFormation::Stack                  | ArchiveService2Stack User Initiated
ArchiveService2Stack | 0/7 | 7:14:15 AM | CREATE_IN_PROGRESS   | AWS::RDS::DBSubnetGroup                     | db-instance/SubnetGroup/Default (dbinstanceSubnetGroupE4F5C66F) 
ArchiveService2Stack | 0/7 | 7:14:15 AM | CREATE_IN_PROGRESS   | AWS::CDK::Metadata                          | CDKMetadata/Default (CDKMetadata) 
ArchiveService2Stack | 0/7 | 7:14:15 AM | CREATE_IN_PROGRESS   | AWS::EC2::SecurityGroup                     | db-instance/SecurityGroup (dbinstanceSecurityGroupF70613D6) 
ArchiveService2Stack | 0/7 | 7:14:16 AM | CREATE_IN_PROGRESS   | AWS::SecretsManager::Secret                 | db-instance/Secret (ArchiveService2StackdbinstanceSecret4305B9BC3fdaad7efa858a3daf9490cf0a702aeb) 
ArchiveService2Stack | 0/7 | 7:14:16 AM | CREATE_IN_PROGRESS   | AWS::RDS::DBSubnetGroup                     | db-instance/SubnetGroup/Default (dbinstanceSubnetGroupE4F5C66F) Resource creation Initiated
ArchiveService2Stack | 1/7 | 7:14:17 AM | CREATE_COMPLETE      | AWS::RDS::DBSubnetGroup                     | db-instance/SubnetGroup/Default (dbinstanceSubnetGroupE4F5C66F) 
ArchiveService2Stack | 1/7 | 7:14:17 AM | CREATE_IN_PROGRESS   | AWS::CDK::Metadata                          | CDKMetadata/Default (CDKMetadata) Resource creation Initiated
ArchiveService2Stack | 1/7 | 7:14:17 AM | CREATE_IN_PROGRESS   | AWS::SecretsManager::Secret                 | db-instance/Secret (ArchiveService2StackdbinstanceSecret4305B9BC3fdaad7efa858a3daf9490cf0a702aeb) Resource creation Initiated
ArchiveService2Stack | 2/7 | 7:14:17 AM | CREATE_COMPLETE      | AWS::CDK::Metadata                          | CDKMetadata/Default (CDKMetadata) 
ArchiveService2Stack | 3/7 | 7:14:17 AM | CREATE_COMPLETE      | AWS::SecretsManager::Secret                 | db-instance/Secret (ArchiveService2StackdbinstanceSecret4305B9BC3fdaad7efa858a3daf9490cf0a702aeb) 
ArchiveService2Stack | 3/7 | 7:14:20 AM | CREATE_IN_PROGRESS   | AWS::EC2::SecurityGroup                     | db-instance/SecurityGroup (dbinstanceSecurityGroupF70613D6) Resource creation Initiated
ArchiveService2Stack | 4/7 | 7:14:21 AM | CREATE_COMPLETE      | AWS::EC2::SecurityGroup                     | db-instance/SecurityGroup (dbinstanceSecurityGroupF70613D6) 
ArchiveService2Stack | 4/7 | 7:14:23 AM | CREATE_IN_PROGRESS   | AWS::RDS::DBInstance                        | db-instance (dbinstance310A317F) 
ArchiveService2Stack | 4/7 | 7:14:25 AM | CREATE_IN_PROGRESS   | AWS::RDS::DBInstance                        | db-instance (dbinstance310A317F) Resource creation Initiated
4/7 Currently in progress: ArchiveService2Stack, dbinstance310A317F
ArchiveService2Stack | 4/7 | 7:15:09 AM | CREATE_FAILED        | AWS::RDS::DBInstance                        | db-instance (dbinstance310A317F) DBInstance ady9tx26mcix2t not found. (Service: AmazonRDS; Status Code: 404; Error Code: DBInstanceNotFound; Request ID: 737ad075-c085-43aa-911a-1bff4c6b9c3c; Proxy: null)
ArchiveService2Stack | 4/7 | 7:15:10 AM | ROLLBACK_IN_PROGRESS | AWS::CloudFormation::Stack                  | ArchiveService2Stack The following resource(s) failed to create: [dbinstance310A317F]. Rollback requested by user.
Stack ArchiveService2Stack has an ongoing operation in progress and is not stable (ROLLBACK_IN_PROGRESS (The following resource(s) failed to create: [dbinstance310A317F]. Rollback requested by user.))
Stack ArchiveService2Stack has an ongoing operation in progress and is not stable (ROLLBACK_IN_PROGRESS)
ArchiveService2Stack | 4/7 | 7:15:20 AM | DELETE_IN_PROGRESS   | AWS::RDS::DBInstance                        | db-instance (dbinstance310A317F) 
ArchiveService2Stack | 4/7 | 7:15:21 AM | DELETE_FAILED        | AWS::RDS::DBInstance                        | db-instance (dbinstance310A317F) DBInstance ady9tx26mcix2t was not found during DescribeDBInstances
ArchiveService2Stack | 4/7 | 7:15:21 AM | DELETE_IN_PROGRESS   | AWS::RDS::DBSubnetGroup                     | db-instance/SubnetGroup/Default (dbinstanceSubnetGroupE4F5C66F) 

This is the log from CloudFormation console (same error as above):

DBInstance ady9tx26mcix2t not found. (Service: AmazonRDS; Status Code: 404; Error Code: DBInstanceNotFound; Request ID: 737ad075-c085-43aa-911a-1bff4c6b9c3c; Proxy: null)

Reproduction Steps

import {Stack, StackProps, aws_rds as rds, aws_ec2 as ec2, aws_iam as iam, aws_ecs as ecs, Duration, RemovalPolicy,} from 'aws-cdk-lib';
import { Construct } from 'constructs';

export class CdkStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);


    const vpc = ec2.Vpc.fromLookup(this, 'CoreVPC', { vpcName: 'SOME_VPC_NAME' });
    
    const dbInstance = new rds.DatabaseInstance(this, 'db-instance', {
      vpc,
      vpcSubnets: {
        subnetType: ec2.SubnetType.PRIVATE_ISOLATED,
      },
      engine: rds.DatabaseInstanceEngine.postgres({
        version: rds.PostgresEngineVersion.VER_14,
      }),
      instanceType: ec2.InstanceType.of(
          ec2.InstanceClass.BURSTABLE3,
          ec2.InstanceSize.SMALL,
      ),
      credentials: rds.Credentials.fromGeneratedSecret('postgres'),
      multiAz: false,
      allocatedStorage: 20,
      maxAllocatedStorage: 25,
      allowMajorVersionUpgrade: false,
      autoMinorVersionUpgrade: true,
      backupRetention: Duration.days(0),
      deleteAutomatedBackups: true,
      removalPolicy: RemovalPolicy.DESTROY,
      deletionProtection: false,
      databaseName: 'archive',
      publiclyAccessible: false,
    });

  }
}

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.20.0 (build 738ef49)

Framework Version

No response

Node.js Version

v16.4.2

OS

macOS 12.3.1 (21E258)

Language

Typescript

Language Version

3.9.10

Other information

No response

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 17 (6 by maintainers)

Most upvoted comments

I have figured out what is wrong. The DB is not encrypted and we have company rules stating that db’s has to be encrypted (different rules in different accounts apparently). Anyways there is a “CloudCustodian” that deletes the db when an unencrypted db is discovered.

Added this to the cdk construct: storageEncrypted:true

I should have spotted this one earlier. Thanks for you time and effort to solve this issue, even if the error was on my side