moby: rpc error: code = 13 desc = invalid header field value "oci runtime error: exec failed: cannot exec a container that has run and stopped\n"
I see similar issues with previous versions of Docker, which have been marked Closed. I believe this issue is still present in latest Docker release 1.12.5.
I am running the latest release of Docker on a series of Vagrant VirtualBox VMs:
[vagrant@host3 ~]$ docker version
Client:
Version: 1.12.5
API version: 1.24
Go version: go1.6.4
Git commit: 7392c3b
Built: Fri Dec 16 02:23:59 2016
OS/Arch: linux/amd64
Server:
Version: 1.12.5
API version: 1.24
Go version: go1.6.4
Git commit: 7392c3b
Built: Fri Dec 16 02:23:59 2016
OS/Arch: linux/amd64
Attempting to run a docker exec on several running container, throws the same error. For example:
[vagrant@host3 ~]$ docker exec -it b17a8574bb5b /bin/bash
rpc error: code = 13 desc = invalid header field value "oci runtime error: exec failed: cannot exec a container that has run and stopped\n"
The container(s) are running:
[vagrant@host3 ~]$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
4c5cff3cc868 grafana/grafana:3.0.1 "/.r/r /run.sh" 35 minutes ago Up 35 minutes r-Prometheus-grafana-1-d0ce0f91
09fd975ee0ca infinityworks/graf-db:09 "cat" 37 minutes ago Up 37 minutes r-Prometheus-grafana-graf-db-1-7bd57453
59779e74cd78 infinityworks/prometheus-rancher-exporter:10 "/.r/r /bin/rancher_e" 40 minutes ago Up 40 minutes r-Prometheus-prometheus-rancher-exporter-1-601148d6
986670c092a3 google/cadvisor:latest "/.r/r /usr/bin/cadvi" 48 minutes ago Up 48 minutes r-Prometheus-cadvisor-3-a34c7ef7
34144c8e5f5e prom/node-exporter:latest "/.r/r /bin/node_expo" 48 minutes ago Up 48 minutes r-Prometheus-node-exporter-3-30c77bd7
82fe72052ce7 mongo:latest "/.r/r /entrypoint.sh" About an hour ago Up About an hour r-voter-mongodb-1-1271f3d0
b17a8574bb5b garystafford/voter-service:latest "/.r/r java -jar vote" About an hour ago Up About an hour r-voter-voter-1-f1971cc1
More info:
[vagrant@host3 ~]$ docker info
Containers: 22
Running: 20
Paused: 0
Stopped: 2
Images: 21
Server Version: 1.12.5
Storage Driver: devicemapper
Pool Name: docker-253:0-25952534-pool
Pool Blocksize: 65.54 kB
Base Device Size: 10.74 GB
Backing Filesystem: xfs
Data file: /dev/loop0
Metadata file: /dev/loop1
Data Space Used: 4.073 GB
Data Space Total: 107.4 GB
Data Space Available: 231 MB
Metadata Space Used: 8.413 MB
Metadata Space Total: 2.147 GB
Metadata Space Available: 231 MB
Thin Pool Minimum Free Space: 10.74 GB
Udev Sync Supported: true
Deferred Removal Enabled: false
Deferred Deletion Enabled: false
Deferred Deleted Device Count: 0
Data loop file: /var/lib/docker/devicemapper/devicemapper/data
WARNING: Usage of loopback devices is strongly discouraged for production use. Use `--storage-opt dm.thinpooldev` to specify a custom block storage device.
Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
Library Version: 1.02.135-RHEL7 (2016-09-28)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: null host bridge overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: seccomp
Kernel Version: 3.10.0-327.36.1.el7.x86_64
Operating System: Red Hat Enterprise Linux
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 993 MiB
Name: host3
ID: 2P3F:NVGZ:YDOO:I2DI:MVNB:BFZN:ANWF:XJQR:RUN5:FJUB:3SZC:R32I
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Insecure Registries:
127.0.0.0/8
About this issue
- Original URL
- State: closed
- Created 8 years ago
- Reactions: 12
- Comments: 85 (30 by maintainers)
@at-daonguyen looks like that’s not a bug; Are you running the
alpineversion of HAProxy? If so, you’re trying to startbashin a container that doesn’t have bash installed. Try usingshinstead orbashYES 👍 Can confirm that the official packages work! Just tried it at 3 severs. Somewhere at the update of the centos docker-1.12.5-14.el7.centos.x86_64 & docker-1.12.6-11.el7.centos.x86_64 things stop working.
Thank you all for the help! And very sorry for your time!
Can this issue please be closed? It looks like this was caused by a kernel module that was intentionally breaking container features. If anyone hits an issue with a similar-looking error please submit a new issue (after reading the error message, obviously).
Looks like it is related to some distribution containers. For me, it works for Alpine (latest) and Debian (latest), but not for Ubuntu (latest) and Centos (latest):
===> Ubuntu
===> Centos
===> Alpine
===> Debian
I had the same issue, the solution removed all docker containers with docker rm $(docker ps -a -q)
docker-runc, notdocker -runc.docker-runcis Docker’s packaged version of runC.Yes, Ubuntu 16.04.2 LTS Yes, docker-ce: Docker version 17.03.0-ce, build 3a232c8
Well, here even
systemctl stop dockerrm -fr /var/lib/docker/*systemctl start dockerdid not help 😦I have similar issues with
docker exec -it <image-name> bashwhen setting up the starter kit of hyperledger fabric project. The error is reproducible with a simpler setup, e.g.Run bash in a docker container “ubuntu” and want to exec this running container from another terminal.
$ docker imagesREPOSITORY TAG IMAGE ID CREATED SIZE…ubuntu latest 104bec311bcd 2 weeks ago 129 MB ...run in terminal 1
$ docker run --rm --name test1 -it ubuntu bashroot@f454c9a79ed4:/#check status with docker ps
$ docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMESf454c9a79ed4 ubuntu "bash" 21 seconds ago Up 19 seconds test1try to exec another bash command in second terminal
$ docker exec -it test1 bashrpc error: code = 13 desc = invalid header field value "oci runtime error: exec failed: container_linux.go:247: starting container process caused \"process_linux.go:83: executing setns process caused \\\"exit status 17\\\"\"\n"