image: error pushing image to authenticated repository

Pushing to docker.io/valentinrothberg/alpine:latest works fine the first time, but will break reproducibly for following attempts when the image already exists and fail with the following error:

Error copying image to the remote destination: Error trying to reuse blob sha256:7bff100f35cb359a368537bb07829b055fe8e0b1cb01085a3a628ae9c187c7b8 at destination: Error checking whether a blob
 sha256:7bff100f35cb359a368537bb07829b055fe8e0b1cb01085a3a628ae9c187c7b8 exists in docker.io/valentinrothberg/alpine: errors:
denied: requested access to the resource is denied
error parsing HTTP 401 response body: unexpected end of JSON input: ""

Cc @mtrmac @runcom

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 6
Comments: 21 (8 by maintainers)

Most upvoted comments

My exact setup is kinda weird. I use nginx as auth proxy and traefik as ssl terminating proxy. So request goes as follows:

Client -(SSL)> Traefik -(HTTP+BasicAuth)> Nginx -(HTTP)> registry:2

My main suspect right now is Nginx and its configuration. I’ve roughly followed this page to set it up.

Exact nginx config. Click to expand.

events {
    worker_connections  1024;
}

http {

  upstream docker-registry {
    server registry:5000;
  }

  ## Set a variable to help us decide if we need to add the
  ## 'Docker-Distribution-Api-Version' header.
  ## The registry always sets this header.
  ## In the case of nginx performing auth, the header is unset
  ## since nginx is auth-ing before proxying.
  map $upstream_http_docker_distribution_api_version $docker_distribution_api_version {
    '' 'registry/2.0';
  }

  server {
    listen 80;
    server_name _;

    # disable any limits to avoid HTTP 413 for large image uploads
    client_max_body_size 0;

    # required to avoid HTTP 411: see Issue #1486 (https://github.com/moby/moby/issues/1486)
    chunked_transfer_encoding on;

    location /v2/ {
      # Do not allow connections from docker 1.5 and earlier
      # docker pre-1.6.0 did not properly set the user agent on ping, catch "Go *" user agents
      if ($http_user_agent ~ "^(docker\/1\.(3|4|5(?!\.[0-9]-dev))|Go ).*$" ) {
        return 404;
      }

      # To add basic authentication to v2 use auth_basic setting.
      auth_basic "My registry";
      auth_basic_user_file /etc/nginx/conf.d/readonly.htpasswd;

      limit_except GET HEAD {
        auth_basic "My registry";
        auth_basic_user_file /etc/nginx/conf.d/admin.htpasswd;
      }

      ## If $docker_distribution_api_version is empty, the header is not added.
      ## See the map directive above where this variable is defined.
      add_header 'Docker-Distribution-Api-Version' $docker_distribution_api_version always;

      proxy_pass                          http://docker-registry;
      proxy_set_header  Host              $http_host;   # required for docker client's sake
      proxy_pass_request_headers          on;
      proxy_read_timeout                  900;
    }
  }
}

Since proxy could be an issue as mentioned here, I am going to thoroughly investigate that.

p-hash on Oct 11, 2021