aws-cdk: aws-eks: addHelmChart() fails with public.ecr.aws
Describe the bug
When updating a stack containing an EKS cluster and adding an helm chart as following:
const chart = this.cluster.addHelmChart('karpenter', {
wait: true,
timeout: Duration.minutes(15),
chart: 'karpenter',
release: 'karpenter',
repository: 'oci://public.ecr.aws/karpenter/karpenter',
namespace: 'karpenter',
version: 'v0.19.2',
createNamespace: false,
values: {
serviceAccount: {
create: false,
name: serviceAccount.serviceAccountName,
annotations: { 'eks.amazonaws.com/role-arn': serviceAccount.role.roleArn },
},
clusterName: this.cluster.clusterName,
clusterEndpoint: this.cluster.clusterEndpoint,
aws: { defaultInstanceProfile: instanceProfile.ref },
},
});
chart.node.addDependency(namespace);
The process timeout after the specified (maximum allowed) 15 minutes.
Looking at the lambda logs on CloudWatch, we get:
[INFO] 2022-11-23T02:24:24.338Z ba8e8ffd-8069-4590-a492-9a58168180d0 b'\nCould not connect to the endpoint URL: "https://api.ecr-public.ap-southeast-2.amazonaws.com/"\nLogin Succeeded\nPulled: public.ecr.aws/karpenter/karpenter:v0.19.2\nDigest: sha256:4ee72250b55d7c45d72f4c8382ed9b2f0f63ec67def84cd63300667e58ff2167\n'
2022-11-23T02:39:13.812Z ba8e8ffd-8069-4590-a492-9a58168180d0 Task timed out after 900.10 seconds
Expected Behavior
The helm chart should be pulled and installed.
Current Behavior
The process timeout.
Reproduction Steps
This is a simplified version of our code
export class EksCluster extends Stack {
cluster: Cluster;
version = eks.KubernetesVersion.of('1.24');
vpc: ec2.IVpc;
constructor(scope: Construct, id: string, props: EksClusterProps) {
super(scope, id);
this.vpc = ec2.Vpc.fromLookup(this, 'Vpc', { tags: { BaseVPC: 'true' } });
this.cluster = new eks.Cluster(this, 'EKSCluster', {
clusterName: 'testCluster',
version: this.version,
vpc: this.vpc,
defaultCapacity: 0,
kubectlLayer: new KubectlV24Layer(this, 'KubeCtlLayer'),
});
this.cluster.addHelmChart('karpenter', {
timeout: Duration.minutes(15),
wait: true,
chart: 'karpenter',
release: 'karpenter',
repository: 'oci://public.ecr.aws/karpenter/karpenter',
namespace: 'karpenter',
version: 'v0.19.2',
createNamespace: false,
});
}
}
Possible Solution
No response
Additional Information/Context
https://api.ecr-public.ap-southeast-2.amazonaws.com/ seems wrong as it looks like only us-east-1 and us-west-2 are available regions. This URL is generated by cdk using the default region of our stack. I don’t know if there is a way to specify a region for this specific helm chart repo?
CDK CLI Version
2.50.0 (build 4c11af6)
Framework Version
2.50.0
Node.js Version
v16.17.0
OS
Ubuntu 22.04.1 LTS
Language
Typescript
Language Version
No response
Other information
No response
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 1
- Comments: 17 (8 by maintainers)
Also… one thing that helm chart
valueschanged between Karpenter v0.18.x and v0.19.x.is now…
Hi @Wyfy0107, I was just going to test my change before pushing a PR!
I think it’s probably because of this line:
https://github.com/aws/aws-cdk/blob/6224b6d850ad1e019e60e905b1799baa071cf269/packages/%40aws-cdk/aws-eks/lib/kubectl-handler/helm/__init__.py#L113
When we authenticate with aws ECR public, I believe we should always use
us-east-1rather than the $AWS_REGION environ variable.I am making this a P2 and any PR would be highly appreciated.