cluster-api-provider-aws: Deleting EC2 Cluster Hangs: The Created VPC is not Deleted

/kind bug

What steps did you take and what happened: [A clear and concise description of what the bug is.] We created an EC2 cluster utilizing the current “clusterctl generate” command. (manifest attached). The cluster was then deleted. The current status shows

$ kubectl get clusters
NAME                 PHASE      AGE   VERSION
gartner-group-mgmt   Deleting   8d    

The CAPA controller manager logs show: I1031 18:41:16.927938 1 awscluster_controller.go:209] controller/awscluster "msg"="Reconciling AWSCluster delete" "cluster"="gartner-group-mgmt" "name"="gartner-group-mgmt" "namespace"="default" "reconciler group"="infrastructure.cluster.x-k8s.io" "reconciler kind"="AWSCluster" E1031 18:41:17.758625 1 awscluster_controller.go:248] controller/awscluster "msg"="error deleting network" "error"="failed to delete vpc \"vpc-0792b7ba04b71f3da\": DependencyViolation: The vpc 'vpc-0792b7ba04b71f3da' has dependencies and cannot be deleted.\n\tstatus code: 400, request id: 64626725-ee55-45e5-ac71-575d9d03b988" "cluster"="gartner-group-mgmt" "name"="gartner-group-mgmt" "namespace"="default" "reconciler group"="infrastructure.cluster.x-k8s.io" "reconciler kind"="AWSCluster" E1031 18:41:17.974814 1 controller.go:317] controller/awscluster "msg"="Reconciler error" "error"="failed to delete vpc \"vpc-0792b7ba04b71f3da\": DependencyViolation: The vpc 'vpc-0792b7ba04b71f3da' has dependencies and cannot be deleted.\n\tstatus code: 400, request id: 64626725-ee55-45e5-ac71-575d9d03b988" "name"="gartner-group-mgmt" "namespace"="default" "reconciler group"="infrastructure.cluster.x-k8s.io" "reconciler kind"="AWSCluster" I1031 18:41:17.975511 1 awscluster_controller.go:209] controller/awscluster "msg"="Reconciling AWSCluster delete" "cluster"="gartner-group-mgmt" "name"="gartner-group-mgmt" "namespace"="default" "reconciler group"="infrastructure.cluster.x-k8s.io" "reconciler kind"="AWSCluster"

Examining the environment with the AWS console shows no other resources except the VPC.

To complete the deletion of this cluster, we will have to delete the VPC that was created. Once done, the cluster is removed from the “kubectl get clusters” listing.

What did you expect to happen: The cluster and all its resources would have been removed.

Anything else you would like to add: [Miscellaneous information that will assist in solving the issue.] This issue began to occur very recently within the last two weeks. Prior to that time, the cluster would delete as expected.

Environment: Cluster controller running in EKS in the same region as the clusters that are created. Only tested with EC2 clusters.

• Cluster-api-provider-aws version:
  capa-system infrastructure-aws 19d InfrastructureProvider aws v1.5.0

• Kubernetes version: (use kubectl version): Client Version: version.Info{Major:“1”, Minor:“25”, GitVersion:“v1.25.3”, GitCommit:“434bfd82814af038ad94d62ebe59b133fcb50506”, GitTreeState:“clean”, BuildDate:“2022-10-12T10:47:25Z”, GoVersion:“go1.19.2”, Compiler:“gc”, Platform:“darwin/amd64”} Kustomize Version: v4.5.7 Server Version: version.Info{Major:“1”, Minor:“22+”, GitVersion:“v1.22.13-eks-15b7512”, GitCommit:“94138dfbea757d7aaf3b205419578ef186dd5efb”, GitTreeState:“clean”, BuildDate:“2022-08-31T19:15:48Z”, GoVersion:“go1.16.15”, Compiler:“gc”, Platform:“linux/amd64”}

• OS (e.g. from /etc/os-release): AME="Amazon Linux" VERSION="2" ID="amzn" ID_LIKE="centos rhel fedora" VERSION_ID="2" PRETTY_NAME="Amazon Linux 2" ANSI_COLOR="0;33" CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2" HOME_URL="https://amazonlinux.com/"