podman: "podman start" not working

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

On Ubuntu 18.04 in AWS, when stopping and starting a standard container, it seems to always show an error during “start”.

Error: unable to start container "<container id>": container create failed (no logs from conmon): EOF

Steps to reproduce the issue:

Launch a small instance in AWS with Ubuntu 18.04. Connect and switch to the root user.

sudo su -

Create install.sh:

set -x
sudo apt-get install -y \
  btrfs-tools \
  git \
  golang-go \
  go-md2man \
  iptables \
  libassuan-dev \
  libc6-dev \
  libdevmapper-dev \
  libglib2.0-dev \
  libgpgme-dev \
  libgpg-error-dev \
  libprotobuf-dev \
  libprotobuf-c0-dev \
  libseccomp-dev \
  libselinux1-dev \
  libsystemd-dev \
  pkg-config \
  runc \
  uidmap

sudo apt-get update -qq
sudo apt-get install -qq -y software-properties-common uidmap
sudo add-apt-repository -y ppa:projectatomic/ppa
sudo apt-get update -qq
sudo apt-get -qq -y install podman
sudo mkdir -p /etc/containers
echo -e "[registries.search]\nregistries = ['docker.io', 'quay.io']" | sudo tee /etc/containers/registries.conf

Run the installation:

chmod 755 install.sh
./install.sh

Download and run a container:

podman pull docker.io/nginx
podman run -d docker.io/library/nginx

View that it’s running:

podman ps

Stop the container:

podman stop 571c0  (or podman stop <container id>)

The ‘stop’ is successful, and shows the id.

Start the container:

podman start 571c0 (or podman start <container id>)

Describe the results you received:

Error: unable to start container "571c0": container create failed (no logs from conmon): EOF

Describe the results you expected:

Just the id of the started container:

571c0

Additional information you deem important (e.g. issue happens only occasionally):

With those steps, the issue seems to be consistent and reproducible.

Output of podman version:

# podman version
Version:            1.6.2
RemoteAPI Version:  1
Go Version:         go1.10.4
OS/Arch:            linux/amd64

Output of podman info --debug:

# podman info --debug
debug:
  compiler: gc
  git commit: ""
  go version: go1.10.4
  podman version: 1.6.2
host:
  BuildahVersion: 1.11.3
  CgroupVersion: v1
  Conmon:
    package: 'conmon: /usr/bin/conmon'
    path: /usr/bin/conmon
    version: 'conmon version 2.0.3, commit: unknown'
  Distribution:
    distribution: ubuntu
    version: "18.04"
  MemFree: 1668878336
  MemTotal: 4135120896
  OCIRuntime:
    name: runc
    package: 'containerd.io: /usr/bin/runc'
    path: /usr/bin/runc
    version: |-
      runc version 1.0.0-rc8+dev
      commit: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
      spec: 1.0.1-dev
  SwapFree: 0
  SwapTotal: 0
  arch: amd64
  cpus: 2
  eventlogger: journald
  hostname: ip-172-31-13-241
  kernel: 4.15.0-1051-aws
  os: linux
  rootless: false
  uptime: 22m 1.82s
registries:
  blocked: null
  insecure: null
  search:
  - docker.io
  - quay.io
store:
  ConfigFile: /etc/containers/storage.conf
  ContainerStore:
    number: 1
  GraphDriverName: overlay
  GraphOptions: {}
  GraphRoot: /var/lib/containers/storage
  GraphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  ImageStore:
    number: 1
  RunRoot: /var/run/containers/storage
  VolumePath: /var/lib/containers/storage/volumes

Package info (e.g. output of rpm -q podman or apt list podman):

# apt list podman
Listing... Done
podman/bionic,now 1.6.2-1~ubuntu18.04~ppa1 amd64 [installed]

Additional environment details (AWS, VirtualBox, physical, etc.):

AWS