podman: DNS does not work in containers if host uses local server
Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)
/kind bug
Description
Steps to reproduce the issue:
-
Be in a network that prohibits external DNS queries, disable external DNS communication or just use some only-locally available hostname in step 3.
-
Setup local DNS server/forwarder (e.g.
systemd-resolved) so that the local address is in /etc/resolv.conf -
Start any container (without
--network host) and try to resolve a hostname (e.g.podman run --rm -it fedora curl -v ifconfig.me)
Describe the results you received:
curl: (6) Could not resolve host: ifconfig.me
Describe the results you expected: No error (some IP address)
Additional information you deem important (e.g. issue happens only occasionally):
The contents of /etc/resolv.conf are:
search virt
nameserver 8.8.8.8
nameserver 8.8.4.4
nameserver 2001:4860:4860::8888
nameserver 2001:4860:4860::8844
nameserver 10.0.2.3
options edns0
Which would normally work (although I might not want to send my DNS requests somewhere else because I might have services available in a local network), but I am in a network that prohibits external DNS queries, so that doesn’t work.
If I leave just the slirp4netns nameserver there (echo nameserver 10.0.2.3 >/etc/resolv.conf) it works in a VM where I am trying to reproduce this issue. However on my original host, where I discovered this, 10.0.2.3 is still inaccessible (even though the version and the command-line of slirp4netns is identical, apart from the PID argument).
Output of podman version:
Version: 1.3.1
RemoteAPI Version: 1
Go Version: go1.12.2
OS/Arch: linux/amd64
Output of podman info --debug:
debug:
compiler: gc
git commit: ""
go version: go1.12.2
podman version: 1.3.1
host:
BuildahVersion: 1.8.2
Conmon:
package: podman-1.3.1-1.git7210727.fc30.x86_64
path: /usr/libexec/podman/conmon
version: 'conmon version 1.12.0-dev, commit: c9a4c48d1bff85033b7fc9b62d25961dd5048689'
Distribution:
distribution: fedora
version: "30"
MemFree: 2884521984
MemTotal: 4133556224
OCIRuntime:
package: runc-1.0.0-93.dev.gitb9b6cc6.fc30.x86_64
path: /usr/bin/runc
version: |-
runc version 1.0.0-rc8+dev
commit: e3b4c1108f7d1bf0d09ab612ea09927d9b59b4e3
spec: 1.0.1-dev
SwapFree: 644870144
SwapTotal: 644870144
arch: amd64
cpus: 4
hostname: fedora30.virt
kernel: 5.0.9-301.fc30.x86_64
os: linux
rootless: true
uptime: 19m 40.53s
registries:
blocked: null
insecure: null
search:
- docker.io
- registry.fedoraproject.org
- quay.io
- registry.access.redhat.com
- registry.centos.org
store:
ConfigFile: /home/nert/.config/containers/storage.conf
ContainerStore:
number: 0
GraphDriverName: overlay
GraphOptions:
- overlay.mount_program=/usr/bin/fuse-overlayfs
GraphRoot: /home/nert/.local/share/containers/storage
GraphStatus:
Backing Filesystem: xfs
Native Overlay Diff: "false"
Supports d_type: "true"
Using metacopy: "false"
ImageStore:
number: 1
RunRoot: /tmp/1000
VolumePath: /home/nert/.local/share/containers/storage/volumes
Additional environment details (AWS, VirtualBox, physical, etc.): I am trying this in a Fedora 30 VM, clean install, as that is the easiest and cleanest reproducer I can get. I cannot reproduce the issue related to my local environment in there.
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 32 (16 by maintainers)
so on fedora (as the main developer platform for podman people) and also rhel-8 systemd-resolved is the default. so using a properly configured systemd-resolved on host can’t be used as a name server for containers run by root!? and the only solution is to use --net host!
why did you close this issue???
A
/etc/resolv.confgenerated by systemd-resolved looks like this:As a result podman seems to remove the nameserver line and adds the “upstream” DNS servers directly. Bypassing systemd-resolved on the host may work in some scenarios, but it breaks others. Consider a corporate VPN connections where “upstream” is not defined. (We have at least two “upstreams” when connected)
Bypassing the host’s systemd-resolved has at least the following problems:
So whatever which DNS server I choose by setting
--dns=<ip>, I will never get the same results as talking to the systemd-resolved running on the host.How to fix that? Well, I guess it’s not possible at all.
podmanwould have to replacenameserver 127.0.0.53with something that is forwarded or hosted on the host. But systemd-resolved is listening on the loopback interface only and does not allow (AFAIK) to change / configure that.[Edit] The combination of systemd-resolved and podman is the default for Fedora users. So privileged containers are quite unusable when corporate VPNs are in the game.
@rhatdan Can this issue be reopened?
From what I read above podman is still unreliable when there are non-trivial dns setups on the host.
Ah, I see.
systemd-resolvdhas apparently decided that the rules for everyone else don’t apply to it, and is doing its own thing. So that’s fun.Docker has the same issue.
Although I think some new features have been added to allow users to share the hosts localhost network. At least in rootless mode.
@mheon @giuseppe WDYT?