podman: [macos] cannot start a VM created with podman machine init after a reboot

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

On MacOS, after rebooting, the VM created with podman machine init cannot start anymore.

Steps to reproduce the issue:

  1. create a VM with podman machine init : working as expected

  2. start it with podman machine start : working as expected

  3. reboot

  4. start VM again with podman machine start : VM does not start

Describe the results you received:

Waiting for VM ... qemu-system-x86_64: -qmp unix://var/folders/78/y7tgcc410y34zywtm83dgjt80000gn/T/podman/qmp_podman-machine-default.sock,server=on,wait=off: Failed to bind socket to //var/folders/78/y7tgcc410y34zywtm83dgjt80000gn/T/podman/qmp_podman-machine-default.sock: No such file or directory Error: dial unix /var/folders/78/y7tgcc410y34zywtm83dgjt80000gn/T/podman/podman-machine-default_ready.sock: connect: no such file or directory

Describe the results you expected:

VM should start

Additional information you deem important (e.g. issue happens only occasionally):

Same behaviour creating a VM with another name

Output of podman version:

Without any VM started, podman versiondoes not work, but podman --version does

podman version 3.2.2

After initialising a new VM:

Client:
Version:      3.2.2
API Version:  3.2.2
Go Version:   go1.16.5
Built:        Fri Jun 25 20:21:29 2021
OS/Arch:      darwin/amd64

Server:
Version:      3.2.1
API Version:  3.2.1
Go Version:   go1.16.3
Built:        Mon Jun 14 21:12:29 2021
OS/Arch:      linux/amd64

Output of podman info --debug: It does not work without a started VM. After starting a new one:

host:
  arch: amd64
  buildahVersion: 1.21.0
  cgroupControllers: []
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.0.27-2.fc34.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.27, commit: '
  cpus: 1
  distribution:
    distribution: fedora
    version: "34"
  eventLogger: journald
  hostname: localhost
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.12.12-300.fc34.x86_64
  linkmode: dynamic
  memFree: 1656451072
  memTotal: 2072817664
  ociRuntime:
    name: crun
    package: crun-0.20.1-1.fc34.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 0.20.1
      commit: 0d42f1109fd73548f44b01b3e84d04a279e99d2e
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.9-1.fc34.x86_64
    version: |-
      slirp4netns version 1.1.8+dev
      commit: 6dc0186e020232ae1a6fcc1f7afbc3ea02fd3876
      libslirp: 4.4.0
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.0
  swapFree: 0
  swapTotal: 0
  uptime: 1m 6.94s
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /var/home/core/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/home/core/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 0
  runRoot: /run/user/1000/containers
  volumePath: /var/home/core/.local/share/containers/storage/volumes
version:
  APIVersion: 3.2.1
  Built: 1623697949
  BuiltTime: Mon Jun 14 19:12:29 2021
  GitCommit: ""
  GoVersion: go1.16.3
  OsArch: linux/amd64
  Version: 3.2.1

Package info (e.g. output of rpm -q podman or apt list podman): Installed on MacOS Big Sur with brew

brew info podman
podman: stable 3.2.2 (bottled)
Tool for managing OCI containers and pods
https://podman.io/
/usr/local/Cellar/podman/3.2.2 (167 files, 29.8MB) *
  Poured from bottle on 2021-06-28 at 17:14:59
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/podman.rb
License: Apache-2.0
==> Dependencies
Build: go ✔, go-md2man ✘
==> Caveats
Bash completion has been installed to:
  /usr/local/etc/bash_completion.d
==> Analytics
install: 4,694 (30 days), 11,050 (90 days), 30,721 (365 days)
install-on-request: 4,691 (30 days), 11,046 (90 days), 30,280 (365 days)
build-error: 0 (30 days)

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/master/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

macOS Big Sur 11.4

Logging out an in again, I can start the VM as expected. So it seems to be related to the reboot.

I also tried to stop the VM before rebooting but it didn’t help.

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 18 (8 by maintainers)

Commits related to this issue

Most upvoted comments

i faced same kind of issue.

Remove podman brew uninstall podman Remove containers files from the following directories: rm -rf ~/.config/containers/ rm -rf ~/.local/share/containers

rm ~/.ssh/podman* related files Reinstalled podman using brew brew install podman Init the podman machine and start it.

alright, whats going on here is that the tmpdir is being cleaned on each reboot (as it should) and podman machine start does not create the TMPDIR/podman directory if it does not exist. PR soon…

After some time debugging, I found the cause of this problem.

This problem is caused due to qemu 7.0.0 startup latency (3-5s) that occour in every first qemu execution after Mac Machine machine boots.

Podman has some bug that doesn’t expect that the creation of socks files, done by the qemu call, can be delayed some seconds, and when podman tries to access the socks files, the qemu is not created them yet, showing the error “Error: dial unix <TEMPDIR>/podman/podman-machine-default_ready.sock: connect: connection refused”.

To avoid this problem, just execute qemu, even with invalid options (just to initialize), before call “podman machine start”.

            echo "* Podman VM machine for MACOSX is stoped, starting..."
            
            # workaround - initialize qemu before machine start to avoid socket error
            /usr/local/bin/qemu-system-x86_64 -machine q35,accel=hvf:tcg -cpu host -display none INVALID_OPTION >> /dev/null 2>&1

            podman machine start podman-machine-default
            ECODE=$?;if [ $ECODE -ne 0 ];then echo "* Error starting podman linux vm machine: $ECODE";exit $ECODE;fi

I hope help. Carlos Eduardo Gorges.