moby: Concurrent pulls fail
This issue seems to be triggered while pulling multiple images which map up to an identical base image. A reproducible example is below. This has been reproduced on Ubuntu 14.04 with DeviceMapper and the Amazon ECS-Optimized Amazon Linux AMI. This has also been reproduced on Docker 1.3.3 and Docker 1.4.1.
[ec2-user@ip-172-31-35-232 ~]$ docker version
Client version: 1.4.1
Client API version: 1.16
Go version (client): go1.3.3
Git commit (client): 5bc2ff8
OS/Arch (client): linux/amd64
Server version: 1.4.1
Server API version: 1.16
Go version (server): go1.3.3
Git commit (server): 5bc2ff8
[ec2-user@ip-172-31-35-232 ~]$
[ec2-user@ip-172-31-35-232 ~]$ docker -D info
Containers: 0
Images: 0
Storage Driver: devicemapper
Pool Name: docker-202:1-1835239-pool
Pool Blocksize: 65.54 kB
Data file: /var/lib/docker/devicemapper/devicemapper/data
Metadata file: /var/lib/docker/devicemapper/devicemapper/metadata
Data Space Used: 307.2 MB
Data Space Total: 107.4 GB
Metadata Space Used: 725 kB
Metadata Space Total: 2.147 GB
Library Version: 1.02.82-git (2013-10-04)
Execution Driver: native-0.2
Kernel Version: 3.14.26-24.46.amzn1.x86_64
Operating System: Amazon Linux AMI 2014.09
CPUs: 1
Total Memory: 1.957 GiB
Name: ip-172-31-35-232
ID: 2RCA:OSRS:4M6S:XHJG:MAUC:IYAA:KIGE:K2HM:VBRF:SCXN:LTRD:T3WP
Debug mode (server): false
Debug mode (client): true
Fds: 57
Goroutines: 58
EventsListeners: 0
Init Path: /usr/bin/docker
Docker Root Dir: /var/lib/docker
[ec2-user@ip-172-31-35-232 ~]$ uname -a
Linux ip-172-31-35-232 3.14.26-24.46.amzn1.x86_64 #1 SMP Wed Dec 10 10:02:43 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
[ec2-user@ip-172-31-35-232 ~]$ docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
[ec2-user@ip-172-31-35-232 ~]$ cat repro
docker run nginx &
docker run mysql &
[ec2-user@ip-172-31-35-232 ~]$ sh repro
[ec2-user@ip-172-31-35-232 ~]$ Unable to find image 'mysql:latest' locally
Unable to find image 'nginx:latest' locally
mysql:latest: The image you are pulling has been verified
f10807909bc5: Pulling fs layer
f6fab3b798be: Pulling fs layer
d832c6f40cc3: Pulling fs layer
cfab4f0d8972: Pulling fs layer
cf8221608a63: Pulling fs layer
4755e012c4cf: Pulling fs layer
e012a865bac8: Pulling fs layer
e2778dfe9ae9: Pulling fs layer
b15206a696ec: Pulling fs layer
9280b0f1af9a: Pulling fs layer
9f1c487e78cd: Pulling fs layer
10491040baf9: Pulling fs layer
d8978d323757: Pulling fs layer
f10807909bc5: Download complete
f10807909bc5: Extracting [=================================================> ] 89.69 MB/90.17 MB 0
d832c6f40cc3: Download complete
f10807909bc5: Pull complete
f10807909bc5: Download complete
f6fab3b798be: Error downloading dependent layers
d21beea329f5: Download complete
04499cf33a0e: Download complete
b15206a696ec: Downloading [====> ] 10.8 MB/118.7 MB 19s
b15206a696ec: Download complete
92360dceda1a: Download complete
9495a555677d: Download complete
911e2df03930: Download complete
88daddf047c6: Download complete
db8a64f62509: Download complete
e46b3488b010: Download complete
e46b3488b010: Error pulling image (latest) from nginx, Driver devicemapper failed to create image rootfs f10807909bc552de261ca7463effc467600c3dca68d8e7704425283a6911d2ca: device f10807909bc552de261ca7463effc467600c3dca68d8e7704425283a6911d2ca already exists 3dca68d8e7704425283a6911d2ca already exists
FATA[0032] Error pulling image (latest) from nginx, Driver devicemapper failed to create image rootfs f10807909bc552de261ca7463effc467600c3dca68d8e7704425283a6911d2ca: device f10807909bc552dea385835786ac: Error pulling image (latest) from mysql, Error mounting '/dev/mapper/docker-202:1-1835239-f6fab3b798be3174f45aa1eb731f8182705555f89c9026d8c1ef230cbf8301dd' on '/var/lib/docker/devicemapper/mnt/f6fab3b798be3174f45aa1eb731f8182705555f89c9026d8c1ef230cbf8301dd': invalid argument 9c9026d8c1ef230cbf8301dd': invalid argument
FATA[0034] Error pulling image (latest) from mysql, Error mounting '/dev/mapper/docker-202:1-1835239-f6fab3b798be3174f45aa1eb731f8182705555f89c9026d8c1ef230cbf8301dd' on '/var/lib/docker/devicemapper/mnt/f6fab3b798be3174f45aa1eb731f8182705555f89c9026d8c1ef230cbf8301dd': invalid argument
^C
[ec2-user@ip-172-31-35-232 ~]$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
[ec2-user@ip-172-31-35-232 ~]$ docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
<none> <none> f10807909bc5 5 weeks ago 85.1 MB
About this issue
- Original URL
- State: closed
- Created 10 years ago
- Comments: 80 (46 by maintainers)
Commits related to this issue
- Serialize any image pulling There are known issues with concurrent docker pulls. Keep things simple for now. https://github.com/docker/docker/issues/9718 — committed to vektra/strive by evanphx 9 years ago
- Disable color output, respect PREFETCH_IMAGES var. Prefetching of Docker images can be turned of now by setting the env var PREFETCH_IMAGES to 0. This might speedup things, however you can easily run... — committed to rhuss/fabric8-installer by rhuss 9 years ago
- ref(registry): remove simpleflock and depend on Docker 1.9 minimum Concurrent pull issue was resolved in 1.9 (https://github.com/docker/docker/issues/9718) and Kubernetes is validated against that as... — committed to helgi/controller by helgi 8 years ago
- ref(registry): remove simpleflock and depend on Docker 1.9 minimum Concurrent pull issue was resolved in 1.9 (https://github.com/docker/docker/issues/9718) and Kubernetes is validated against that as... — committed to helgi/controller by helgi 8 years ago
- ref(registry): remove simpleflock and depend on Docker 1.9 minimum (#1090) Concurrent pull issue was resolved in 1.9 (https://github.com/docker/docker/issues/9718) and Kubernetes is validated against... — committed to deis/controller by helgi 8 years ago
-subscribe-