podman: [podman machine] Port auto-forwarding does not work in rootless mode and macOS

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

When I run the podman machine with Podman v3.3.x and gvproxy. In rootless mode on Linux or macOS, automatic port forwarding by gvproxy does not work when I start the container and expose the port. In root mode on Linux, automatic port forwarding does work.

Steps to reproduce the issue:

(rootless mode on Linux)

  1. Initialize and start the podman machine.
$ podman machine init; podman machine start
  1. Pull the container image and start it.
$ podman -r pull docker.io/library/nginx
$ podman -r run --rm -d --name nginx -p 8888:80 nginx
  1. Checking the port number, there are no published ports.
$ podman -r ps
CONTAINER ID  IMAGE                           COMMAND               CREATED        STATUS            PORTS                 NAMES
4c9564b5d1aa  docker.io/library/nginx:latest  nginx -g daemon o...  2 minutes ago  Up 2 minutes ago  0.0.0.0:8888->80/tcp  nginx
$ curl http://localhost:8888
curl: (7) Failed to connect to localhost port 8888: Connection refused
$ ss -ltnup | grep 8888

Describe the results you received:

$ podman -r ps
CONTAINER ID  IMAGE                           COMMAND               CREATED        STATUS            PORTS                 NAMES
4c9564b5d1aa  docker.io/library/nginx:latest  nginx -g daemon o...  2 minutes ago  Up 2 minutes ago  0.0.0.0:8888->80/tcp  nginx
$ ss -ltnup | grep 8888
$ curl http://localhost:8888
curl: (7) Failed to connect to localhost port 8888: Connection refused

Describe the results you expected:

Automatic port forwarding works, allowing access to exposed ports.

Additional information you deem important (e.g. issue happens only occasionally):

Automatic port forwarding does not work on macOS as well as rootless mode on Linux. It works fine in root mode on Linux.

# podman machine init; podman machine start
# podman -r pull docker.io/library/nginx
# podman -r run --rm -d --name nginx -p 8888:80 nginx
# podman -r ps
CONTAINER ID  IMAGE                           COMMAND               CREATED         STATUS             PORTS                 NAMES
b75a5a32c09a  docker.io/library/nginx:latest  nginx -g daemon o...  30 seconds ago  Up 30 seconds ago  0.0.0.0:8888->80/tcp  nginx
# ss -ltnup | grep 8888
tcp   LISTEN 0      4096               *:8888             *:*    users:(("gvproxy",pid=8857,fd=31))
# curl http://localhost:8888
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
...
</html>

Output of podman version:

$ podman version
Version:      3.3.1
API Version:  3.3.1
Go Version:   go1.16.6
Built:        Tue Aug 31 05:46:36 2021
OS/Arch:      linux/amd64

Output of podman info --debug:

$ podman info --debug
host:
  arch: amd64
  buildahVersion: 1.22.3
  cgroupControllers: []
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.0.29-2.fc34.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.29, commit: '
  cpus: 2
  distribution:
    distribution: fedora
    version: "34"
  eventLogger: journald
  hostname: fedora34
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.13.12-200.fc34.x86_64
  linkmode: dynamic
  memFree: 137220096
  memTotal: 4103892992
  ociRuntime:
    name: crun
    package: crun-0.21-1.fc34.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 0.21
      commit: c4c3cdf2ce408ed44a9e027c618473e6485c635b
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.12-2.fc34.x86_64
    version: |-
      slirp4netns version 1.1.12
      commit: 7a104a101aa3278a2152351a082a6df71f57c9a3
      libslirp: 4.4.0
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.0
  swapFree: 4094160896
  swapTotal: 4103073792
  uptime: 25h 24m 58.8s (Approximately 1.04 days)
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /home/user/.config/containers/storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 0
    stopped: 1
  graphDriverName: overlay
  graphOptions:
    overlay.mount_program:
      Executable: /usr/bin/fuse-overlayfs
      Package: fuse-overlayfs-1.7.1-1.fc34.x86_64
      Version: |-
        fusermount3 version: 3.10.4
        fuse-overlayfs: version 1.7.1
        FUSE library version 3.10.4
        using FUSE kernel interface version 7.31
  graphRoot: /home/user/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 16
  runRoot: /run/user/1000/containers
  volumePath: /home/user/.local/share/containers/storage/volumes
version:
  APIVersion: 3.3.1
  Built: 1630356396
  BuiltTime: Tue Aug 31 05:46:36 2021
  GitCommit: ""
  GoVersion: go1.16.6
  OsArch: linux/amd64
  Version: 3.3.1

Package info (e.g. output of rpm -q podman or apt list podman):

$ rpm -q podman
podman-3.3.1-1.fc34.x86_64

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/master/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

  • Fedora release 34 (Thirty Four)
  • VM on VMware ESXi

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 2
  • Comments: 33 (21 by maintainers)

Commits related to this issue

Most upvoted comments

Just confirming that adding rootless_networking = "cni" under the [containers] section of ~/.config/containers/containers.conf does also fix this on MacOS 😃

+26
AverageMarcus on Sep 1, 2021

The next release (probably 3.3.2?) should include the changed default. I don’t have a solid date on when that’s coming out, probably in the next two weeks.

+8
mheon on Sep 2, 2021

Just confirming that adding rootless_networking = "cni" under the [containers] section of ~/.config/containers/containers.conf does also fix this on MacOS 😃

FWIW, I installed Podman via Homebrew today and I had to add the above to work with forwarding.

Followed the instructions for macOS as well.

+8
jef on Sep 2, 2021

We decided against 3.3.2, and instead are releasing 3.4.0 on Wednesday. That should have all the bugfixes we’ve accumulated. 3.4.0-RC2 is presently available, though I don’t know if we’ve built the RC for OS X.

On Sun, Sep 26, 2021 at 7:08 PM hawkeng @.***> wrote:

any updates on this?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/containers/podman/issues/11396#issuecomment-927388351, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB3AOCD53WDEDWCBY6YMDYTUD6RWHANCNFSM5DHAGSJA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

+6
mheon on Sep 26, 2021

We’re putting together a Podman 3.3.2 later this week that should have all the fixes bundled together and should Just Work.

+4
mheon on Sep 15, 2021

It should be in the release notes of the containers/common library as the fix was there. It is fixed in 3.4.0.

+3
mheon on Oct 13, 2021

This one’s probably me, we should be doing that automatically for machine VMs. Self-assigning.

+3
mheon on Sep 1, 2021

@Luap99 Wow, It works fine!

(rootless mode on Linux)

$ podman -r run --rm -d --name nginx -p 8888:80 --network bridge nginx
$ ss -ltnup |grep 8888
tcp   LISTEN 0      4096               *:8888             *:*    users:(("gvproxy",pid=9725,fd=17))

(macOS)

$ podman network create nginx
$ podman run --rm -d --name nginx -p 8888:80 --network nginx nginx
$ netstat -an |grep 8888
tcp46      0      0  *.8888                 *.*                    LISTEN

It also works when using --network bridge on macOS.

+2
tnk4on on Sep 1, 2021

Please open a fresh issue, that version should have all the issues we know about with port forwarding worked out.

+1
mheon on Nov 1, 2021

this works in 3.3.1 and later. if you have 3.3.1, it should have installed gvproxy … then simply create a new network and run the container like you did earlier.

+1
baude on Sep 1, 2021

Argh. Alright, makes sense. Maybe we should make that the default for OS X and Windows builds of c/common and containers.conf?

+1
mheon on Sep 1, 2021

@mheon The problem is that we parse this into specgen on the client, so rootless_networking = "cni" must be set on the client I think.

+1
Luap99 on Sep 1, 2021

Can you try running with --network bridge and see if that works

+1
Luap99 on Sep 1, 2021
Read more comments on GitHub
← podman: podman machine on macOS becomes unresponsive after some time
podman: podman machine start cannot find gvproxy when brew is not installed in the standard directory →