podman: podman machine fails when ~/.ssh/config contains mac content

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

podman machine fails with 255 when ~/.ssh/config contains Mac related content:

Host github.com
  IgnoreUnknown AddKeysToAgent,UseKeychain
  AddKeysToAgent yes
  UseKeychain yes
  IdentityFile ~/.ssh/github

Steps to reproduce the issue:

  1. Create a ssh config like the above

  2. podman machine start (see error)

  3. podman machine ssh (see error complaining about config)

Describe the results you received:

➜ ~ podman machine start Starting machine “podman-machine-default” Waiting for VM … Mounting volume… /Users/gahealy:/Users/gahealy Error: exit status 255 ➜ ~ podman machine list NAME VM TYPE CREATED LAST UP CPUS MEMORY DISK SIZE podman-machine-default* qemu 7 minutes ago Currently running 1 2.147GB 107.4GB ➜ ~ podman machine ssh Connecting to vm podman-machine-default. To close connection, use ~. or exit /Users/gahealy/.ssh/config: line 4: Bad configuration option: usekeychain /Users/gahealy/.ssh/config: line 10: Bad configuration option: usekeychain /Users/gahealy/.ssh/config: line 16: Bad configuration option: usekeychain /Users/gahealy/.ssh/config: terminating, 3 bad configuration options Error: exit status 255

Describe the results you expected:

google suggests using IgnoreUnknown should fix the error - git for example, follows this. I’d expect podman to follow the same idea.

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

Client:       Podman Engine
Version:      4.1.0
API Version:  4.1.0
Go Version:   go1.18.1
Built:        Thu May  5 21:07:47 2022
OS/Arch:      darwin/arm64

Server:       Podman Engine
Version:      4.1.0
API Version:  4.1.0
Go Version:   go1.18
Built:        Fri May  6 17:16:38 2022
OS/Arch:      linux/arm64

Output of podman info --debug:

host:
  arch: arm64
  buildahVersion: 1.26.1
  cgroupControllers:
  - cpu
  - io
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.0-2.fc36.aarch64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.0, commit: '
  cpuUtilization:
    idlePercent: 97.67
    systemPercent: 1.74
    userPercent: 0.59
  cpus: 1
  distribution:
    distribution: fedora
    variant: coreos
    version: "36"
  eventLogger: journald
  hostname: localhost.localdomain
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 1000000
    uidmap:
    - container_id: 0
      host_id: 501
      size: 1
    - container_id: 1
      host_id: 100000
      size: 1000000
  kernel: 5.17.5-300.fc36.aarch64
  linkmode: dynamic
  logDriver: journald
  memFree: 1682010112
  memTotal: 2052694016
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: crun-1.4.4-1.fc36.aarch64
    path: /usr/bin/crun
    version: |-
      crun version 1.4.4
      commit: 6521fcc5806f20f6187eb933f9f45130c86da230
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/user/501/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.0-0.2.beta.0.fc36.aarch64
    version: |-
      slirp4netns version 1.2.0-beta.0
      commit: 477db14a24ff1a3de3a705e51ca2c4c1fe3dda64
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.3
  swapFree: 0
  swapTotal: 0
  uptime: 4m 50.97s
plugins:
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /var/home/core/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/home/core/.local/share/containers/storage
  graphRootAllocated: 106825756672
  graphRootUsed: 2271117312
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 0
  runRoot: /run/user/501/containers
  volumePath: /var/home/core/.local/share/containers/storage/volumes
version:
  APIVersion: 4.1.0
  Built: 1651853798
  BuiltTime: Fri May  6 17:16:38 2022
  GitCommit: ""
  GoVersion: go1.18
  Os: linux
  OsArch: linux/arm64
  Version: 4.1.0

Package info (e.g. output of rpm -q podman or apt list podman):

brew list podman
/opt/homebrew/Cellar/podman/4.1.0/bin/podman
/opt/homebrew/Cellar/podman/4.1.0/bin/podman-mac-helper
/opt/homebrew/Cellar/podman/4.1.0/bin/podman-remote
/opt/homebrew/Cellar/podman/4.1.0/etc/bash_completion.d/podman
/opt/homebrew/Cellar/podman/4.1.0/libexec/gvproxy
/opt/homebrew/Cellar/podman/4.1.0/share/fish/vendor_completions.d/podman.fish
/opt/homebrew/Cellar/podman/4.1.0/share/man/ (163 files)
/opt/homebrew/Cellar/podman/4.1.0/share/zsh/site-functions/_podman

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

No

Additional environment details (AWS, VirtualBox, physical, etc.):

MacOS M1

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 3
  • Comments: 19 (4 by maintainers)

Most upvoted comments

@benko removing UseKeyChain works but that then breaks my git setup, which is what this issue is about.

Ahh, got you. I somehow missed the start of your original message and just focused on the exit status part.

Your problem is that ssh parses the entire config file but only on the odd chance that there will be another section which is more specific than the one it’s currently set to use, and yet some directives are ignored in that context because it already has a host match - so basically it is complaining about unknown content, even if it doesn’t apply to its current config in any way.

Move the IgnoreUnknown to the top of the file, even before the first Host section. You only need it once.

@benko ; can you provide an example, I’ve tried as you suggested but still see the 255 error

$ cat ~/.ssh/config
Host localhost
  IdentitiesOnly yes

Obtain the port of your machine as I’ve shown above, then do try the following and paste the output here:

$ env SSH_AUTH_SOCK= ssh -v -p PORT -i ~/.ssh/YOURMACHINE core@localhost

Just noting that I seemed to have this problem until I restarted my Mac:

$ podman machine start --log-level debug                    
INFO[0000] podman filtering at log level debug          
DEBU[0000] [/opt/homebrew/bin/qemu-system-aarch64 -m 8192 -smp 4 -fw_cfg name=opt/com.coreos/config,file=/Users/myuser/.config/containers/podman/machine/qemu/podman-machine-default.ign -qmp unix://var/folders/42/22jv0xts5730z1h5clsrr8wh0000gn/T/podman/qmp_podman-machine-default.sock,server=on,wait=off -netdev socket,id=vlan,fd=3 -device virtio-net-pci,netdev=vlan,mac=5a:94:ef:e4:0c:ee -device virtio-serial -chardev socket,path=/var/folders/42/22jv0xts5730z1h5clsrr8wh0000gn/T/podman/podman-machine-default_ready.sock,server=on,wait=off,id=podman-machine-default_ready -device virtserialport,chardev=podman-machine-default_ready,name=org.fedoraproject.port.0 -accel hvf -accel tcg -cpu host -M virt,highmem=on -drive file=/opt/homebrew/share/qemu/edk2-aarch64-code.fd,if=pflash,format=raw,readonly=on -drive file=/Users/myuser/.local/share/containers/podman/machine/qemu/podman-machine-default_ovmf_vars.fd,if=pflash,format=raw -virtfs local,path=/Users/myuser,mount_tag=vol0,security_model=mapped-xattr -drive if=virtio,file=/Users/myuser/.local/share/containers/podman/machine/qemu/podman-machine-default_fedora-coreos-36.20220723.2.2-qemu.aarch64.qcow2] 
Starting machine "podman-machine-default"
[/opt/homebrew/opt/podman/libexec/gvproxy -listen-qemu unix:///var/folders/42/22jv0xts5730z1h5clsrr8wh0000gn/T/podman/qmp_podman-machine-default.sock -pid-file /var/folders/42/22jv0xts5730z1h5clsrr8wh0000gn/T/podman/podman-machine-default.pid -ssh-port 58610 -forward-sock /Users/myuser/.local/share/containers/podman/machine/podman-machine-default/podman.sock -forward-dest /run/user/501/podman/podman.sock -forward-user core -forward-identity /Users/myuser/.ssh/podman-machine-default --debug]
DEBU[0000] [/opt/homebrew/bin/qemu-system-aarch64 -m 8192 -smp 4 -fw_cfg name=opt/com.coreos/config,file=/Users/myuser/.config/containers/podman/machine/qemu/podman-machine-default.ign -qmp unix://var/folders/42/22jv0xts5730z1h5clsrr8wh0000gn/T/podman/qmp_podman-machine-default.sock,server=on,wait=off -netdev socket,id=vlan,fd=3 -device virtio-net-pci,netdev=vlan,mac=5a:94:ef:e4:0c:ee -device virtio-serial -chardev socket,path=/var/folders/42/22jv0xts5730z1h5clsrr8wh0000gn/T/podman/podman-machine-default_ready.sock,server=on,wait=off,id=podman-machine-default_ready -device virtserialport,chardev=podman-machine-default_ready,name=org.fedoraproject.port.0 -accel hvf -accel tcg -cpu host -M virt,highmem=on -drive file=/opt/homebrew/share/qemu/edk2-aarch64-code.fd,if=pflash,format=raw,readonly=on -drive file=/Users/myuser/.local/share/containers/podman/machine/qemu/podman-machine-default_ovmf_vars.fd,if=pflash,format=raw -virtfs local,path=/Users/myuser,mount_tag=vol0,security_model=mapped-xattr -drive if=virtio,file=/Users/myuser/.local/share/containers/podman/machine/qemu/podman-machine-default_fedora-coreos-36.20220723.2.2-qemu.aarch64.qcow2] 
Waiting for VM ...
Mounting volume... /Users/myuser:/Users/myuser
DEBU[0015] Executing: ssh [-i /Users/myuser/.ssh/podman-machine-default -p 58610 core@localhost -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -q -- sudo chattr -i / ; sudo mkdir -p /Users/myuser ; sudo chattr +i / ;] 
Error: exit status 255
$ podman machine ssh --log-level debug
INFO[0000] podman filtering at log level debug          
DEBU[0000] [/opt/homebrew/bin/qemu-system-aarch64 -m 8192 -smp 4 -fw_cfg name=opt/com.coreos/config,file=/Users/myuser/.config/containers/podman/machine/qemu/podman-machine-default.ign -qmp unix://var/folders/42/22jv0xts5730z1h5clsrr8wh0000gn/T/podman/qmp_podman-machine-default.sock,server=on,wait=off -netdev socket,id=vlan,fd=3 -device virtio-net-pci,netdev=vlan,mac=5a:94:ef:e4:0c:ee -device virtio-serial -chardev socket,path=/var/folders/42/22jv0xts5730z1h5clsrr8wh0000gn/T/podman/podman-machine-default_ready.sock,server=on,wait=off,id=podman-machine-default_ready -device virtserialport,chardev=podman-machine-default_ready,name=org.fedoraproject.port.0 -accel hvf -accel tcg -cpu host -M virt,highmem=on -drive file=/opt/homebrew/share/qemu/edk2-aarch64-code.fd,if=pflash,format=raw,readonly=on -drive file=/Users/myuser/.local/share/containers/podman/machine/qemu/podman-machine-default_ovmf_vars.fd,if=pflash,format=raw -virtfs local,path=/Users/myuser,mount_tag=vol0,security_model=mapped-xattr -drive if=virtio,file=/Users/myuser/.local/share/containers/podman/machine/qemu/podman-machine-default_fedora-coreos-36.20220723.2.2-qemu.aarch64.qcow2] 
Connecting to vm podman-machine-default. To close connection, use `~.` or `exit`
DEBU[0000] Executing: ssh [-i /Users/myuser/.ssh/podman-machine-default -p 58610 core@localhost -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no] 
Warning: Permanently added '[localhost]:58610' (ED25519) to the list of known hosts.
Received disconnect from ::1 port 58610:2: Too many authentication failures
Disconnected from ::1 port 58610
Error: exit status 255
$ podman machine inspect
[
     {
          "ConfigPath": {
               "Path": "/Users/myuser/.config/containers/podman/machine/qemu/podman-machine-default.json"
          },
          "Created": "2022-07-28T09:16:42.378795-10:00",
          "Image": {
               "IgnitionFilePath": {
                    "Path": "/Users/myuser/.config/containers/podman/machine/qemu/podman-machine-default.ign"
               },
               "ImageStream": "testing",
               "ImagePath": {
                    "Path": "/Users/myuser/.local/share/containers/podman/machine/qemu/podman-machine-default_fedora-coreos-36.20220723.2.2-qemu.aarch64.qcow2"
               }
          },
          "LastUp": "2022-07-28T09:33:31.839911-10:00",
          "Name": "podman-machine-default",
          "Resources": {
               "CPUs": 4,
               "DiskSize": 60,
               "Memory": 8192
          },
          "SSHConfig": {
               "IdentityPath": "/Users/myuser/.ssh/podman-machine-default",
               "Port": 58610,
               "RemoteUsername": "core"
          },
          "State": "running"
     }
]
$ podman machine stop
Machine "podman-machine-default" stopped successfully

Then, after restart:

$ podman machine start --log-level debug
INFO[0000] podman filtering at log level debug          
DEBU[0000] [/opt/homebrew/bin/qemu-system-aarch64 -m 8192 -smp 4 -fw_cfg name=opt/com.coreos/config,file=/Users/myuser/.config/containers/podman/machine/qemu/podman-machine-default.ign -qmp unix://var/folders/42/22jv0xts5730z1h5clsrr8wh0000gn/T/podman/qmp_podman-machine-default.sock,server=on,wait=off -netdev socket,id=vlan,fd=3 -device virtio-net-pci,netdev=vlan,mac=5a:94:ef:e4:0c:ee -device virtio-serial -chardev socket,path=/var/folders/42/22jv0xts5730z1h5clsrr8wh0000gn/T/podman/podman-machine-default_ready.sock,server=on,wait=off,id=podman-machine-default_ready -device virtserialport,chardev=podman-machine-default_ready,name=org.fedoraproject.port.0 -accel hvf -accel tcg -cpu host -M virt,highmem=on -drive file=/opt/homebrew/share/qemu/edk2-aarch64-code.fd,if=pflash,format=raw,readonly=on -drive file=/Users/myuser/.local/share/containers/podman/machine/qemu/podman-machine-default_ovmf_vars.fd,if=pflash,format=raw -virtfs local,path=/Users/myuser,mount_tag=vol0,security_model=mapped-xattr -drive if=virtio,file=/Users/myuser/.local/share/containers/podman/machine/qemu/podman-machine-default_fedora-coreos-36.20220723.2.2-qemu.aarch64.qcow2] 
Starting machine "podman-machine-default"
[/opt/homebrew/opt/podman/libexec/gvproxy -listen-qemu unix:///var/folders/42/22jv0xts5730z1h5clsrr8wh0000gn/T/podman/qmp_podman-machine-default.sock -pid-file /var/folders/42/22jv0xts5730z1h5clsrr8wh0000gn/T/podman/podman-machine-default.pid -ssh-port 58610 -forward-sock /Users/myuser/.local/share/containers/podman/machine/podman-machine-default/podman.sock -forward-dest /run/user/501/podman/podman.sock -forward-user core -forward-identity /Users/myuser/.ssh/podman-machine-default --debug]
DEBU[0000] [/opt/homebrew/bin/qemu-system-aarch64 -m 8192 -smp 4 -fw_cfg name=opt/com.coreos/config,file=/Users/myuser/.config/containers/podman/machine/qemu/podman-machine-default.ign -qmp unix://var/folders/42/22jv0xts5730z1h5clsrr8wh0000gn/T/podman/qmp_podman-machine-default.sock,server=on,wait=off -netdev socket,id=vlan,fd=3 -device virtio-net-pci,netdev=vlan,mac=5a:94:ef:e4:0c:ee -device virtio-serial -chardev socket,path=/var/folders/42/22jv0xts5730z1h5clsrr8wh0000gn/T/podman/podman-machine-default_ready.sock,server=on,wait=off,id=podman-machine-default_ready -device virtserialport,chardev=podman-machine-default_ready,name=org.fedoraproject.port.0 -accel hvf -accel tcg -cpu host -M virt,highmem=on -drive file=/opt/homebrew/share/qemu/edk2-aarch64-code.fd,if=pflash,format=raw,readonly=on -drive file=/Users/myuser/.local/share/containers/podman/machine/qemu/podman-machine-default_ovmf_vars.fd,if=pflash,format=raw -virtfs local,path=/Users/myuser,mount_tag=vol0,security_model=mapped-xattr -drive if=virtio,file=/Users/myuser/.local/share/containers/podman/machine/qemu/podman-machine-default_fedora-coreos-36.20220723.2.2-qemu.aarch64.qcow2] 
Waiting for VM ...
Mounting volume... /Users/myuser:/Users/myuser
DEBU[0014] Executing: ssh [-i /Users/myuser/.ssh/podman-machine-default -p 58610 core@localhost -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -q -- sudo chattr -i / ; sudo mkdir -p /Users/myuser ; sudo chattr +i / ;] 
DEBU[0015] Executing: ssh [-i /Users/myuser/.ssh/podman-machine-default -p 58610 core@localhost -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -q -- sudo mount -t 9p -o trans=virtio vol0 /Users/myuser -o version=9p2000.L,msize=131072] 

This machine is currently configured in rootless mode. If your containers
require root permissions (e.g. ports < 1024), or if you run into compatibility
issues with non-podman clients, you can switch using the following command: 

	podman machine set --rootful

API forwarding listening on: /var/run/docker.sock
Docker API clients default to this address. You do not need to set DOCKER_HOST.

Machine "podman-machine-default" started successfully
DEBU[0015] Called machine start.PersistentPostRunE(podman machine start --log-level debug) 

I have no idea why.