podman: podman-3.3.0 - Changes to default /etc/hosts handling are breaking container workloads

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

With podman-3.2.3 and prior, podman could (optionally) write hosts entries to the container /etc/hosts file.

With the podman-3.3.0 release-candidates, this behaviour has changed and /etc/hosts within the container is now bind-mounted by default. This is causing failures with a wide-spread and unpredictable blast-radius as a variety of services are revealed to write a new temporary file and then move this over /etc/hosts, which is now failing.

There’s the --no-hosts option which prevents and /etc/host being written at all, but there doesn’t appear to be any option to revert to the pre-3.3.0 functionality.

I’ve only tested very briefly, but new failures are with containers performing software builds/deployments (which legitimately try to deploy a new /etc/hosts file) and - randomly - the spampd service which it turns out tries to replace /etc/hosts on startup and fails otherwise with a message reading sed: can't move '/etc/hostsCGNQu3' to '/etc/hosts': Device or resource busy.

In general, I’d hope any potentially-breaking change such as this would be opt-in rather than opt-out, especially when first introduced.

Output of podman version:

podman version 3.3.0-rc3

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.22.0
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - hugetlb
  - pids
  - rdma
  - misc
  cgroupManager: cgroupfs
  cgroupVersion: v2
  conmon:
    package: app-emulation/conmon-2.0.29
    path: /usr/bin/conmon
    version: 'conmon version 2.0.29, commit: 7e6de6678f6ed8a18661e1d5721b81ccee293b9b'
  cpus: 8
  distribution:
    distribution: gentoo
    version: unknown
  eventLogger: file
  hostname: dellr330
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.13.7-gentoo
  linkmode: dynamic
  memFree: 2497138688
  memTotal: 67267272704
  ociRuntime:
    name: crun
    package: app-emulation/crun-0.21
    path: /usr/bin/crun
    version: |-
      crun version 0.21
      commit: c4c3cdf2ce408ed44a9e027c618473e6485c635b
      spec: 1.0.0
      +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  os: linux
  remoteSocket:
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_AUDIT_WRITE,CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_MKNOD,CAP_NET_BIND_SERVICE,CAP_NET_RAW,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: app-emulation/slirp4netns-1.1.12
    version: |-
      slirp4netns version 1.1.12
      commit: 7a104a101aa3278a2152351a082a6df71f57c9a3
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.1
  swapFree: 25234563072
  swapTotal: 25769787392
  uptime: 195h 15m 38.63s (Approximately 8.12 days)
registries:
  localhost:5000:
    Blocked: false
    Insecure: true
    Location: localhost:5000
    MirrorByDigestOnly: false
    Mirrors: []
    Prefix: localhost:5000
  search:
  - docker.io
  - docker.pkg.github.com
  - quay.io
  - public.ecr.aws
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 24
    paused: 0
    running: 22
    stopped: 2
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev
  graphRoot: /space/podman/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "true"
  imageStore:
    number: 69
  runRoot: /var/run/podman
  volumePath: /space/podman/volumes
version:
  APIVersion: 3.3.0-rc3
  Built: 1629323346
  BuiltTime: Wed Aug 18 22:49:06 2021
  GitCommit: 88559c197da3d05c7758920bce90d07e0f066101
  GoVersion: go1.16.7
  OsArch: linux/amd64
  Version: 3.3.0-rc3

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/master/troubleshooting.md)

Yes