portieris: Error retrieving OAuth token from repositories that do not support it

What commit ID of Portieris did you experience the problem with?

939c65b4e121c778d97f5dd1384484f5fcb078f1

What went wrong?

I am unable to deploy images from Docker Hub or a self-hosted Artifactory. When deploying an image from either registry Portieris queries https://<registry>/oauth/token. As far as I am aware this endpoint only exists for IBM Container Registry. So when Docker Hub is queried a 404 and large amount of HTML (probably the Docker 404 page) is returned. For Artifactory we receive an x509 error from our self-signed cert. This causes both deployments to fail despite being signed images with correct registry credentials.

These errors happen when querying the registry, before it gets as far as communicating with our self-hosted Notary.

We can deploy images hosted in IBM Container Registry just fine. Even when we host the signing information in our self-hosted Notary.

What should have happened differently?

Portieris should not query https://<registry>/oauth/token for non-IBM registries. Portieris should be able to handle registries using self-signed TLS certificates.

How can it be reproduced?

Install Portieris and attempt to any Docker Hub pod with a valid pull secret.

Any other relevant information

We are running Portieris on an EKS cluster so we have deployed with the IBMContainerService option set to false.

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 6
Comments: 68 (51 by maintainers)

Commits related to this issue

Patch in fixes for blocking portieris issues Specifically, these two issues are blocking using portieris app with starlingx: * https://github.com/IBM/portieris/issues/51 * https://github.com/IBM/po... — committed to starlingx/portieris-armada-app by josephdrichard 4 years ago

Most upvoted comments

Looks good to me. In my opinion there isn’t any need to block this on using a fake registry for the test cases, so I hope that it can merge soon.

josephdrichard on Jul 21, 2020

we should make an unauthenticated request to the resource that we actually want to access

So, if I wanted to access, for example, https://abc.azurecr.io/v2/foo/bar, the flow would be:

I make a https://abc.azurecr.io/v2/foo/bar with no Authorization header
If the response comes back as 2xx, that’s our content and we’re done - no auth necessary. If the response comes back as a 401 with a Www-Authenticate header, we continue. If the response is any other 4xx or 5xx, we fail out.
We go and get the auth token using our credentials and the scope etc in the Www-Authenticate header.
We go back to https://abc.azurecr.io/v2/foo/bar using the auth token in the Authorization header and get our content.

molepigeon on Apr 28, 2019

glad to hear that! thanks for the quick answer @jerrinss5 😃

ralgozino on Oct 31, 2019

@jerrinss5 I think you’re going to the registry to get responses on those APIs. We should use the Notary API to be sure that we get the correct data back.

Notary’s API is a little different to Registry’s, but in short a sensible URL to call would be: GET https://<notary-domain>:<port>/v2/<image_repository>/_trust/tuf/root.json

For example: GET https://us.icr.io:4443/v2/us.icr.io/molepigeon/testimage/_trust/tuf/root.json

$ curl -vvv https://us.icr.io:4443/v2/us.icr.io/molepigeon/testimage/_trust/tuf/root.json 2>&1 | grep Www-Authenticate
< Www-Authenticate: Bearer realm="https://registry.ng.bluemix.net/oauth/token",service="notary",scope="repository:us.icr.io/molepigeon/testimage:pull"

Don’t forget that the API could return something other than a 401 - for example, if you make an unauthenticated request to IBM’s notary asking for a Docker Hub image, you’ll get a 404 even without auth:

$ curl -vvv https://us.icr.io:4443/v2/docker.io/ubuntu/_trust/tuf/root.json 2>&1 | grep HTTP
> GET /v2/docker.io/ubuntu/_trust/tuf/root.json HTTP/1.1
< HTTP/1.1 404 Not Found

molepigeon on May 7, 2019