spinnaker: Clouddriver: Intermitent DockerRegistryImageCachingAgent Could not load tags for private ECR
The latest version of clouddriver coudl’t load the ECR image tags and returns the error
WARN 1 --- [ecutionAction-3] .d.r.p.a.DockerRegistryImageCachingAgent : Could not load tags for <IMAGE_NAME> in https://<AWS_ACCOUNT_ID>.dkr.ecr.us-east-1.amazonaws.com, reason: Status: 404, Method: GET, URL: https://<AWS_ACCOUNT_ID>.dkr.ecr.us-east-1.amazonaws.com/v2/<REPOSITORY>/tags/list%3Flast=..., Message: Not Found
I’m using the Docker Registry command configuration (ignore <> format values)
...
dockerRegistry:
enabled: true
accounts:
- name: <name>
requiredGroupMembership: []
permissions: {}
address: https://<AWS_ACCOUNT_ID>.dkr.ecr.<REGION>.amazonaws.com
username: AWS
passwordCommand: aws --region us-east-1 ecr get-authorization-token --output
text --query 'authorizationData[].authorizationToken' | base64 -d | sed
's/^AWS://'
email: <FAKE_EMAIL>
cacheIntervalSeconds: 30
clientTimeoutMillis: 60000
cacheThreads: 2
paginateSize: 1000
sortTagsByDate: false
trackDigests: false
insecureRegistry: false
repositories: []
primaryAccount: <name>
cacheThreads is not relevant, I’ve tried with 1.
The AmazonEC2ContainerRegistryReadOnly policy attachment is working properly. All the registry repositories are configured equals, all of those are private and all of those contain tags.
I’ve tried start with a cleaned Redis to avoid the serialization problems.
Spinnaker Version: 1.37.3
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 18 (10 by maintainers)
Please try https://spinnaker.io/changelogs/1.37.6-changelog/
Update: I am able to reproduce the issue with an ECR account, working on the fix.
com.netflix.spinnaker.kork.retrofit.exceptions.SpinnakerHttpException: Status: 405, Method: GET, URL: https://<AWS_ACC>.dkr.ecr.ap-south-1.amazonaws.com/v2/my-alpine/tags/list?last=Nde5FCDw1zWkZzJ6A1Z62peakkDPEPvmR3NM9aL3q7yzx95U3S6iyeBRKbWaSMthLeZ6FvJmBiVg1Woj%252FNOhjepFq7kPAr%252FEw%252FYxfcJoHPrRtyiqe%252F1Qj5V9ZxkRw9GwXH%252BWwaHjHAnN0A1toOLyBZ9N52%252BloQCrQRkcIFULfkBoIdAEO2yIn88Sy4ASRhZ%252B8XhnF%252BF0DBKDr%252B6xA9qquZewsLyd7NEDx0mFv1EcUjqVHbvg7uDTKtzIDZiclAELtvuSVxxpi56CCCOqkc77YpQSdji3Xmp299otByhHR8r1BJdvMXB2R6H1YeVMd1GvzppUgB6J%252F9D9HLnOKXm%252FZ%252FKx2K7Ea3C8cBwm8KkO%252FVCdTSJf2HXrtgLgHWRSzY4gPQkqlwFE35PWO74CPp3qAQ%253D%253D&n=1, Message: Method Not Allowed
I am able to reproduce the issue. Retrofit2 doesn’t accept
?in the@Pathparameter and hence replaces it with%3Feven though it’s not supposed to encode. This is because,?is reserved for@Queryparams.@kirangodishala is there a trailing-slash (or lack of trailing slash) issue around here in DockerRegistryNamedAccountCredentials?