moby: Failed to upload schema2 manifest: manifest invalid: manifest invalid
Description
Failed to upload schema2 manifest: manifest invalid: manifest invalid
Briefly describe the problem you are having in a few paragraphs.
I am trying to push Windows Container image to Jfrog Artifactory Docker Registry it fails with an error “failed to upload schema2 manifest “
Similar thing happens when I try to push an image to Docker private Registry it fails with an error “manifest blob unknown: blob unknown to registry”
Steps to reproduce the issue:
C:\Windows\system32>docker push xx/windowservercore
The push refers to a repository [xx]
cc6b0a07c696: Skipped foreign layer
3fd27ecef6a3: Skipped foreign layer
manifest invalid: manifest invalid
Note: Using xx to hide the internal repo information
docker push xx.xx.xx.xxx:5000/windowsservercore
The push refers to a repository [xx.xx.xx.xx:5000/windowsservercore]
cc6b0a07c696: Skipped foreign layer
3fd27ecef6a3: Skipped foreign layer```
errors:
manifest blob unknown: blob unknown to registry
manifest blob unknown: blob unknown to registry
Describe the results you received:
Log Name: Application
Source: docker
Date:
10/14/2016 6:51:45 AM
Event ID: 1
Task Category: None
Level: Warning
Keywords: Classic
User: N/A
Computer: WIN-H3VK04MET11
Description:
failed to upload schema2 manifest: manifest invalid: manifest invalid
Log Name: Application
Source: docker
Date: 10/14/2016
6:51:45 AM
Event ID: 1
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: WIN-H3VK04MET11
Description:
Not continuing with push
after error: manifest invalid: manifest invalid
Log Name: Application
Source: docker
Date: 10/14/2016 6:53:48 AM
Event ID: 1
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: WIN-H3VK04MET11
Description:
Not continuing with push after error: errors:
manifest blob unknown: blob unknown to registry
manifest blob unknown: blob unknown to registry
Log Name: Application
Source: docker
Date: 10/14/2016 6:53:48 AM
Event ID: 1
Task Category: None
Level: Warning
Keywords: Classic
User: N/A
Computer: WIN-H3VK04MET11
Description:
failed to upload schema2 manifest: errors:
manifest blob unknown: blob unknown to registry
manifest blob unknown: blob unknown to registry
Describe the results you expected: Successful upload the image to repo
Additional information you deem important (e.g. issue happens only occasionally): Dockerd 1.12 & 1.12.2 also have this issue.
Output of docker version:
C:\Windows\system32>docker -v
Docker version 1.13.0-dev, build 535f52c
C:\Windows\system32>docker -v
Docker version 1.13.0-dev, build 535f52c (paste your output here)
More information on Docker Image
C:\Windows\system32>docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
microsoft/windowsservercore latest 93a9c37b36d0 3 weeks ago 8.68 GB
Output of docker info:
C:\Windows\system32> docker info
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 1
Server Version: 1.13.0-dev
Storage Driver: windowsfilter
Windows:
Logging Driver: json-file
Plugins:
Volume: local
Network: nat null overlay
Swarm: inactive
Default Isolation: process
Kernel Version: 10.0 14393 (14393.0.amd64fre.rs1_release.160715-1616)
Operating System: Windows Server 2016 Standard
OSType: windows
Architecture: x86_64
CPUs: 5
Total Memory: 3 GiB
Name: WIN-H3VK04MET11
ID: QJYG:RN3K:GOEM:XST5:QMSJ:XRJA:NBFN:OPE3:A7ZS:XPPI:QQZQ:V6KB
Docker Root Dir: C:\ProgramData\docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Insecure Registries:
xxx.xxx.xxx.xxx:5000
127.0.0.0/8
Live Restore Enabled: false
C:\Windows\system32>
Additional environment details (AWS, VirtualBox, physical, etc.): Artifactory Enterprise : 4.9.0 rev 40226 Docker Registry version: Docker-registry-0.9.1-7.el7.x86_64
I will be opening a support ticket with Jfrog Artifactory as well, but wanted to check with the newer version of Dockerd engine will the schema version will always be on version 2 or will there be a fallback to version 1 or if this is a bug in docker ?
About this issue
- Original URL
- State: closed
- Created 8 years ago
- Comments: 69 (30 by maintainers)
Can confirm Artifactory push works fine with following entry in daemon.json and 17.06 client. Is it possible to specify flag in docker CLI somehow instead? I don’t understand how this shall be integrated into CI process since modifying docker daemon is not really an option in such cases and the only option is to specify additional switches to CLI.
stevvooe’s primary concern seems to be keeping the user workflow simple. It won’t be simple if we add a new flag to docker push that allows pushing foreign layers, e.g.
docker push --push-foreign-layer-data myimage. Such a flag would be hard to remember to use and wouldn’t make much sense to users (they don’t necessarily know that their container hosts are air-gapped, they just want to push their images to the registry that their production servers use).So instead, let’s add a new configuration option to the registry daemon,
--accept-foreign-layer-data, that means that when a user pushes to that registry, even foreign layer tars are pushed. When dockerd pulls from that registry, it should also pull foreign layer data from that registry if it is present; if it’s not, then it should fall back to the normal approach of downloading from the foreign URLs. In either case, dockerd should receive the list of foreign URLs so that it can preserve them when pushing to a registry that does not accept foreign layer data (e.g. the public Docker registry).I’m sure this will require some protocol changes between dockerd and the registry, and thus forces users to upgrade both components, but it is the solution with the simplest user experience.
There are some scenarios (such as air gaps) where it may not be possible or efficient to retrieve layers from central servers. I feel we should allow users to push foreign layers in these exceptional circumstances, rather than being coercive. What’s most important with foreign layers is not to push them by default, to avoid unintentional copyright infringement. However, it should ultimately be the user’s decision, with our guidance, on how to handle them.
@friism That did work as a workaround. Thanks
Alright, here’s what you can do as a workaround:
manifest.jsonand remove theLayerSourcesstuffmanifest.json-gset to non-standard locationdocker load -i image.tar.gzinto that fresh Docker engine (eg. use-Hto point at it)docker tag <your-favorite-foreign-layer-image> <private-repo-tag>docker push <private-repo-tag>This is obviously not great, but should work as a workaround.