regclient: [Issue] Getting "Not Found" while pushing to DockerHub after a sleep for backoff

Hi @sudo-bmitch – moving on from our discussion over at SO… I had to make minor changes and ran into another issue with my repo.

Current Behavior

Getting “Not Found” while pushing to DockerHub after a sleep for backoff.

Expected Behavior

Expect it to continue.

Steps To Reproduce

It starts out normal as far as I can tell:

./regctl image import s4lab/gipsy-json-u18:04-messy2-18.04 image.tar -v debug
DEBU[0000] Loading docker config                         api= blobChunk=0 blobMax=0 helper= hostname=ghcr.io mirrors="[]" name=ghcr.io pathPrefix= repoAuth=false tls=enabled user=smokhov
DEBU[0000] Loading docker config                         api= blobChunk=0 blobMax=0 helper= hostname=registry-1.docker.io mirrors="[]" name=docker.io pathPrefix= repoAuth=false tls=enabled user=mokhov
DEBU[0000] Loading docker config                         api= blobChunk=0 blobMax=0 helper= hostname=registry-1.docker.io mirrors="[]" name=docker.io pathPrefix= repoAuth=false tls=enabled user=mokhov
WARN[0000] Changing credential host for registry         host=docker.io new="https://registry-1.docker.io/" orig="https://index.docker.io/v1/"
DEBU[0000] Loading docker config                         api= blobChunk=0 blobMax=0 helper= hostname=registry-1.docker.io mirrors="[]" name=docker.io pathPrefix= repoAuth=false tls=enabled user=mokhov
WARN[0000] Changing credential host for registry         host=docker.io new="https://index.docker.io/v1/" orig="https://registry-1.docker.io/"
DEBU[0000] regclient initialized                         VCSRef=732c2a3a6312fc70dfcfcd821505ef0ad1ed230f VCSTag=v0.4.5
DEBU[0000] Image import                                  file=image.tar ref="docker.io/s4lab/gipsy-json-u18:04-messy2-18.04"
DEBU[0000] http req                                      method=POST url="https://registry-1.docker.io/v2/s4lab/gipsy-json-u18/blobs/uploads/" withAuth=false
DEBU[0000] Auth request parsed                           challenge="[{bearer map[realm:https://auth.docker.io/token scope:repository:s4lab/gipsy-json-u18:push,pull service:registry.docker.io]}]"
DEBU[0000] Auth scope added                              host=registry-1.docker.io scope="repository:s4lab/gipsy-json-u18:pull,push"
DEBU[0000] http req                                      method=POST url="https://registry-1.docker.io/v2/s4lab/gipsy-json-u18/blobs/uploads/" withAuth=true
DEBU[0000] Upload location received                      location="https://registry-1.docker.io/v2/s4lab/gipsy-json-u18/blobs/uploads/8e7b8327-1b76-4b36-8db8-679f8d9ac3ed?_state=6rqzzlN3eo35gzx6qCPh6UgPpuRtGX6KFykF__CtT4V7Ik5hbWUiOiJzNGxhYi9naXBzeS1qc29uLXUxOCIsIlVVSUQiOiI4ZTdiODMyNy0xYjc2LTRiMzYtOGRiOC02NzlmOGQ5YWMzZWQiLCJPZmZzZXQiOjAsIlN0YXJ0ZWRBdCI6IjIwMjItMTAtMDlUMjA6NTU6MTEuNDY2MDcxNTQ0WiJ9"
DEBU[0000] http req                                      method=PATCH url="https://registry-1.docker.io/v2/s4lab/gipsy-json-u18/blobs/uploads/8e7b8327-1b76-4b36-8db8-679f8d9ac3ed?_state=6rqzzlN3eo35gzx6qCPh6UgPpuRtGX6KFykF__CtT4V7Ik5hbWUiOiJzNGxhYi9naXBzeS1qc29uLXUxOCIsIlVVSUQiOiI4ZTdiODMyNy0xYjc2LTRiMzYtOGRiOC02NzlmOGQ5YWMzZWQiLCJPZmZzZXQiOjAsIlN0YXJ0ZWRBdCI6IjIwMjItMTAtMDlUMjA6NTU6MTEuNDY2MDcxNTQ0WiJ9" withAuth=true
...

then it barfs out with

...
WARN[0034] Sleeping for backoff                          Host=docker.io Seconds=1.999998466
DEBU[0036] http req                                      method=PATCH url="https://registry-1.docker.io/v2/s4lab/gipsy-json-u18/blobs/uploads/eecb727a-9b99-43f3-a77e-80a853898e4c?_state=9b5WxGCBTHRDKgCWjuXCcO72lN_fKd1nkZqRK1h-1vF7Ik5hbWUiOiJzNGxhYi9naXBzeS1qc29uLXUxOCIsIlVVSUQiOiJlZWNiNzI3YS05Yjk5LTQzZjMtYTc3ZS04MGE4NTM4OThlNGMiLCJPZmZzZXQiOjIwOTcxNTIwLCJTdGFydGVkQXQiOiIyMDIyLTEwLTA5VDIwOjU1OjMzWiJ9" withAuth=true
DEBU[0036] Request failed                                Status="Not Found" URL="https://registry-1.docker.io/v2/s4lab/gipsy-json-u18/blobs/uploads/eecb727a-9b99-43f3-a77e-80a853898e4c?_state=9b5WxGCBTHRDKgCWjuXCcO72lN_fKd1nkZqRK1h-1vF7Ik5hbWUiOiJzNGxhYi9naXBzeS1qc29uLXUxOCIsIlVVSUQiOiJlZWNiNzI3YS05Yjk5LTQzZjMtYTc3ZS04MGE4NTM4OThlNGMiLCJPZmZzZXQiOjIwOTcxNTIwLCJTdGFydGVkQXQiOiIyMDIyLTEwLTA5VDIwOjU1OjMzWiJ9"
failed to import layers from docker tar: failed to send blob (chunk), ref docker.io/s4lab/gipsy-json-u18:04-messy2-18.04: request failed: not found [http 404]: {"errors":[{"code":"BLOB_UPLOAD_INVALID","message":"blob upload invalid"},{"code":"BLOB_UPLOAD_INVALID","message":"blob upload invalid","detail":"already closed"}]}

Version

{
  "VCSRef": "732c2a3a6312fc70dfcfcd821505ef0ad1ed230f",
  "VCSTag": "v0.4.5"
}

Environment

• Running as binary or container: binary
• Host platform: Ubuntu 18.04
• Registry description: DockerHub