amazon-eks-pod-identity-webhook: ECR Published Image Cannot be Fetched for Custom Cluster
What happened: When trying to run make cluster-up
deployments get stuck in an image pull backoff., this causes the latter parts of the make cluster-up
to fail in that there are no CSRs. When trying to pull the image locally, the ECR repository is not able to be pulled from publicly.
e.g.
$ docker pull 602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/pod-identity-webhook:latest
Error response from daemon: Get https://602401143452.dkr.ecr.us-west-2.amazonaws.com/v2/eks/pod-identity-webhook/manifests/latest: no basic auth credentials
Relevant k8s events from the pod:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 2m45s default-scheduler Successfully assigned default/pod-identity-webhook-56c67dff46-c6h9h to docker-desktop
Normal Pulling 78s (x4 over 2m44s) kubelet, docker-desktop Pulling image "602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/pod-identity-webhook"
Warning Failed 78s (x4 over 2m44s) kubelet, docker-desktop Failed to pull image "602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/pod-identity-webhook": rpc error: code = Unknown desc = Error response from daemon: Get https://602401143452.dkr.ecr.us-west-2.amazonaws.com/v2/eks/pod-identity-webhook/manifests/latest: no basic auth credentials
Warning Failed 78s (x4 over 2m44s) kubelet, docker-desktop Error: ErrImagePull
Warning Failed 65s (x6 over 2m43s) kubelet, docker-desktop Error: ImagePullBackOff
Normal BackOff 51s (x7 over 2m43s) kubelet, docker-desktop Back-off pulling image "602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/pod-identity-webhook"
What you expected to happen: The pod-identity-webhook
image to be publicly pullable.
How to reproduce it (as minimally and precisely as possible):
- Be on a machine with no authentication to the
602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/pod-identity-webhook
repo and attempt to run:
docker pull 602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/pod-identity-webhook:latest
Anything else we need to know?: Obviously this can be fixed by pushing an image the cluster can be reached from and passing in the relevant env vars to make
, just following the current README.md
instructions does not produce an expected result.
Environment:
(N/A here since this is unrelated to k8s/EKS specific bits)
- AWS Region: N/A
- EKS Platform version (if using EKS, run
aws eks describe-cluster --name <name> --query cluster.platformVersion
): N/A - Kubernetes version (if using EKS, run
aws eks describe-cluster --name <name> --query cluster.version
): N/A - Webhook Version: N/A
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 11
- Comments: 22 (4 by maintainers)
@tabern any word? An issue to just publish a docker image has been open for over 6 months now…
Any update on official docker images being available?
Ahhh, it seems I have misunderstood what is required to use IAM roles with pods on EKS. We don’t need to have this docker image or run this ourselves, it’s built into the EKS service. We only need to create an OIDC provider, annotate the
ServiceAccount
and create an IAM role with the correct trust policy.This will teach me for skim reading READMEs and AWS announcements 😅
What is the status? 😃
Hi @micahhausler Can this move forward?
I think @nckturner is going to fix this 🎉
Docker images are now available on https://hub.docker.com/r/amazon/amazon-eks-pod-identity-webhook
I’m hesitant to actually use this as I’m afraid that it’s abandon-ware.
Why are no public images yet available? Is there a general refusal to make them public? But why then leave links to private images in the documentation? Is the current state too unstable to be used so AWS won’t make them public? Why does making an image public take so many months after being reported?
No releases, few commits … doesn’t look good to me.Edit: I missed the releases somehow