buildah: podman build with secret meet permission error, while podman run with secrets is ok.

podman build with secret meet permission error

I follow the example in https://www.redhat.com/sysadmin/sensitive-data-containers to build the container image using secrets and meet permission error

Steps to reproduce the issue:

1. Create Dockerfile and touch one secret.txt

$ cat Dockerfile
FROM alpine
RUN --mount=type=secret,id=mysecret,target=/mysecret cat /mysecret
RUN cat /mysecret

1. Run build command

$ podman build --secret id=mysecret,src=./secret.txt .
STEP 1/3: FROM alpine
STEP 2/3: RUN --mount=type=secret,id=mysecret,target=/mysecret cat /mysecret
error running container: error from /usr/bin/runc creating container for [/bin/sh -c cat /mysecret]: time="2022-09-11T11:05:06+02:00" level=error msg="container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: rootfs_linux.go:75: mounting \"/var/tmp/buildah1779424060/mnt/buildah-bind-target-0\" to rootfs at \"/mysecret\" caused: operation not permitted"
: exit status 1
ERRO[0000] did not get container create message from subprocess: EOF 
Error: error building at STEP "RUN --mount=type=secret,id=mysecret,target=/mysecret cat /mysecret": error while running runtime: exit status 1

Describe the results you received:

see above, it reports permission error

Noticed some extra info about my storage

$ mount 
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
/dev/mapper/rootvg-root on / type xfs (rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota)
/dev/mapper/vglocal-lvlocal on /local type xfs (rw,nosuid,relatime,attr2,inode64,logbufs=8,logbsize=32k,usrquota,grpquota)

will nosuid be the issue? but it works for podman secret/run command

$ printf "top secrets" | podman secret create mysecret -
$ podman run --secret source=mysecret,type=env alpine printenv
..
mysecret=top secrets
$ podman run --secret source=mysecret,type=mount alpine mount
overlay on / type overlay (rw,relatime,lowerdir=/local/share/containers-user-1003/storage/overlay/l/2XJ3KVS3EGPTOXANZRHOQ4Z2NK,upperdir=/local/share/containers-user-
..
/dev/mapper/vglocal-lvlocal on /run/secrets/mysecret type xfs (rw,nosuid,relatime,attr2,inode64,logbufs=8,logbsize=32k,usrquota,grpquota)
$ podman run --secret source=mysecret,type=mount alpine cat //run/secrets/mysecret
top secrets

Describe the results you expected:

In other machine, the Step 2 shall report ok

STEP 2/3: RUN --mount=type=secret,id=mysecret,target=/mysecret cat /mysecret
top secrets
--> 10e64e9cba1
STEP 3/3: RUN cat /mysecret
cat: can't open '/mysecret': No such file or directory

Output of rpm -q buildah or apt list buildah:

podman-4.1.1-2.module+el8.6.0+15917+093ca6f8.x86_64

Output of podman version if reporting a podman build issue:

$ podman version
Client:       Podman Engine
Version:      4.1.1
API Version:  4.1.1
Go Version:   go1.17.7
Built:        Mon Jul 11 16:56:53 2022
OS/Arch:      linux/amd64
$ podman info
host:
  arch: amd64
  buildahVersion: 1.26.2
  cgroupControllers: []
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.0-1.module+el8.6.0+14877+f643d2d6.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.0, commit: 87b7a9037cbd1d81288bdf2d6705dfda889f7cf9'
  cpuUtilization:
    idlePercent: 99.53
    systemPercent: 0.2
    userPercent: 0.27
  cpus: 8
  distribution:
    distribution: '"rhel"'
    version: "8.6"
  eventLogger: file
  hostname: xxx
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 64000
      size: 1
    - container_id: 1
      host_id: 3000655360
      size: 65535
    uidmap:
    - container_id: 0
      host_id: 1003
      size: 1
    - container_id: 1
      host_id: 3000655360
      size: 65535
  kernel: 4.18.0-372.13.1.el8_6.x86_64
  linkmode: dynamic
  logDriver: k8s-file
  memFree: 12041859072
  memTotal: 33506668544
  networkBackend: cni
  ociRuntime:
    name: runc
    package: runc-1.0.3-2.module+el8.6.0+14877+f643d2d6.x86_64
    path: /usr/bin/runc
    version: |-
      runc version 1.0.3
      spec: 1.0.2-dev
      go: go1.17.7
      libseccomp: 2.5.2
  os: linux
  remoteSocket:
    exists: true
    path: /run/user/1003/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_NET_RAW,CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.8-2.module+el8.6.0+14877+f643d2d6.x86_64
    version: |-
      slirp4netns version 1.1.8
      commit: d361001f495417b880f20329121e3aa431a8f90f
      libslirp: 4.4.0
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.2
  swapFree: 0
  swapTotal: 0
  uptime: 381h 55m 9.41s (Approximately 15.88 days)
plugins:
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
...
store:
  configFile: /home/bob/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /local/share/containers-user-1003/storage
  graphRootAllocated: 107317563392
  graphRootUsed: 8479199232
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 1
  runRoot: /run/user/1003/containers
  volumePath: /local/share/containers-user-1003/storage/volumes
version:
  APIVersion: 4.1.1
  Built: 1657551413
  BuiltTime: Mon Jul 11 16:56:53 2022
  GitCommit: ""
  GoVersion: go1.17.7
  Os: linux
  OsArch: linux/amd64
  Version: 4.1.1

Output of cat /etc/*release:

NAME="Red Hat Enterprise Linux"
VERSION="8.6 (Ootpa)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="8.6"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
HOME_URL="https://www.redhat.com/"
DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
Red Hat Enterprise Linux release 8.6 (Ootpa)
Red Hat Enterprise Linux release 8.6 (Ootpa)

Output of uname -a:

Linux xxxx 4.18.0-372.13.1.el8_6.x86_64 #1 SMP Mon Jun 6 15:05:22 EDT 2022 x86_64 x86_64 x86_64 GNU/Linux

Output of cat /etc/containers/storage.conf:

[storage]
# Default Storage Driver, Must be set for proper operation.
driver = "overlay"
# Temporary storage location
runroot = "/run/containers/storage"
graphroot = "/var/lib/containers/storage"

# Storage path for rootless users
#
# rootless_storage_path = "$HOME/.local/share/containers/storage"
rootless_storage_path = "/local/share/containers-user-$UID/storage"

[storage.options]
# Storage options to be passed to underlying storage drivers

# AdditionalImageStores is used to pass paths to additional Read/Only image stores
# Must be comma separated list.
additionalimagestores = [
]

[storage.options.overlay]
# ignore_chown_errors can be set to allow a non privileged user running with
# a single UID within a user namespace to run containers. The user can pull
# and use any image even those with multiple uids.  Note multiple UIDs will be
# squashed down to the default uid in the container.  These images will have no
# separation between the users in the container. Only supported for the overlay
# and vfs drivers.
#ignore_chown_errors = "false"

# Inodes is used to set a maximum inodes of the container image.
# inodes = ""

# Path to an helper program to use for mounting the file system instead of mounting it
# directly.
#mount_program = "/usr/bin/fuse-overlayfs"

# mountopt specifies comma separated list of extra mount options
mountopt = "nodev,metacopy=on"

[storage.options.thinpool]