podman: Podman port mapping cannot access externally

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug Description

I have an AWS EC2 instance running RHEL 8 where I have done the basic port mapping to a container on ports 443 and 80, using root account.

The response is able to be displayed locally with curl -k https://localhost:443. However when trying to access the application using the public dns, the url timed out.

I am just using a basic AWS EC2 set up without any route53, load balancer, just pinging the instance with its public dns so it would not be an infra issue.

I have also tried the sample network port mapping laid out here: https://podman.io/getting-started/network Using: $ podman run -dt -p 8080:8080/tcp registry.fedoraproject.org/f29/httpd But the result is the same, curl returns the response on port 8080, but using the public url it cannot access.

The same basic set up was done on Docker, which works fine both locally and publicly.

Steps to reproduce the issue:

  1. podman run -dt -p 8080:8080/tcp registry.fedoraproject.org/f29/httpd

  2. curl -k http://localhost:8080 returns response

  3. Going to the web browser to browse public IP on port 8080 returns timed out issue.

Describe the results you received: Going to the public url on the browser returns connection timed out. Pinging the site locally on the instance, the response is returned.

Describe the results you expected: Going to the public url on the browser should return the pplication response.

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

Version:            1.9.3
RemoteAPI Version:  1
Go Version:         go1.13.4
OS/Arch:            linux/amd64```

**Output of `podman info --debug`:**

debug: compiler: gc gitCommit: “” goVersion: go1.13.4 podmanVersion: 1.9.3 host: arch: amd64 buildahVersion: 1.14.9 cgroupVersion: v1 conmon: package: conmon-2.0.17-1.module+el8.2.1+6771+3533eb4c.x86_64 path: /usr/bin/conmon version: ‘conmon version 2.0.17, commit: 3c703d9f178a3a53966e1d5c03d0275ea6cb36a0’ cpus: 8 distribution: distribution: ‘“rhel”’ version: “8.2” eventLogger: file hostname: ip-172-31-24-35.ap-southeast-2.compute.internal idMappings: gidmap: null uidmap: null kernel: 4.18.0-193.el8.x86_64 memFree: 19208482816 memTotal: 33554034688 ociRuntime: name: runc package: runc-1.0.0-66.rc10.module+el8.2.1+6465+1a51e8b6.x86_64 path: /usr/bin/runc version: ‘runc version spec: 1.0.1-dev’ os: linux rootless: false slirp4netns: executable: “” package: “” version: “” swapFree: 0 swapTotal: 0 uptime: 1h 42m 39.42s (Approximately 0.04 days) registries: search:

  • registry.access.redhat.com
  • registry.redhat.io
  • docker.io store: configFile: /etc/containers/storage.conf containerStore: number: 35 paused: 0 running: 32 stopped: 3 graphDriverName: overlay graphOptions: overlay.mountopt: nodev,metacopy=on graphRoot: /var/lib/containers/storage graphStatus: Backing Filesystem: xfs Native Overlay Diff: “false” Supports d_type: “true” Using metacopy: “true” imageStore: number: 49 runRoot: /var/run/containers/storage volumePath: /var/lib/containers/storage/volumes```

Package info (e.g. output of rpm -q podman or apt list podman):

podman-1.9.3-2.module+el8.2.1+6867+366c07d6.x86_64

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide?

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

cat /etc/os-release NAME=“Red Hat Enterprise Linux” VERSION=“8.2 (Ootpa)” ID=“rhel” ID_LIKE=“fedora” VERSION_ID=“8.2” PLATFORM_ID=“platform:el8” PRETTY_NAME=“Red Hat Enterprise Linux 8.2 (Ootpa)” ANSI_COLOR=“0;31” CPE_NAME=“cpe:/o:redhat:enterprise_linux:8.2:GA” HOME_URL=“https://www.redhat.com/” BUG_REPORT_URL=“https://bugzilla.redhat.com/

REDHAT_BUGZILLA_PRODUCT=“Red Hat Enterprise Linux 8” REDHAT_BUGZILLA_PRODUCT_VERSION=8.2 REDHAT_SUPPORT_PRODUCT=“Red Hat Enterprise Linux” REDHAT_SUPPORT_PRODUCT_VERSION=“8.2”

Result of netstat command:

netstat -ntlp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 10.89.0.1:53 0.0.0.0:* LISTEN 78784/dnsmasq
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1480/sshd
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 92963/conmon
tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN 101069/conmon
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 92963/conmon
tcp6 0 0 fe80::db:c5ff:fe09:e:53 ::😗 LISTEN 78784/dnsmasq
tcp6 0 0 :::22 ::😗 LISTEN 1480/sshd

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 18 (6 by maintainers)

Most upvoted comments

Had the same issue as above and tried all the recommended steps. Never even had Docker installed. I found a solution here that doesn’t require deactivating UFW or adding routes. The issue was UFW not allowing any forwarded traffic by default.

  • edit /etc/default/ufw to change DEFAULT_FORWARD_POLICY value to ACCEPT
  • sudo ufw reload

And it works ✔️

for what its worth: I just experienced this phenomenon myself on a relatively fresh Debian bullseye installation. I previously had docker and docker-compose running on it and 2 containers on top of that.

There is no firewall installed on that box, at least not that I am aware of, I did nothing beyond the most basic minimal install from the debian.iso and only installed docker and docker-compose (and then later also installed podman)

While trying to switch from docker to podman for these 2 containers I was not able to get any ports accessible from the outside. I could access them from the local machine (for example nc localhost 3306 would connect my mariadb container) but not from any other host in my network, nmap on any other machine would detect the ports but show them as “filtered” and any attempt to connect would time-out.

Initially I did not want to uninstall docker before I got podman working, so only after I found this issue here I finally apt purged docker and docker-compose and all their dependencies, rebooted and suddenly it all began working like a charm, all ports are accessible from localhost as well as external.

So there is definitely a conflict between docker and podman.

Has this issue now been resolved? As I’m experiencing the same issue on a clean installed Ubuntu20.04 machine. Even if I remove all firewalld / ufw firewalls I can’t access exposed container ports. The same with docker runs just fine.

After tons of googling I managed to get it working by:

  • deactivate ufw
  • completely purge docker (which was still installed parallel)

I think the issue is somehow caused by docker (which should be able to co-exist next to podman, but somehow isn’t).

Same here, but on NixOS, tried to flush iptables, disable firewall.service, set the backend option in the network definition, nothing helped…