crc: CRC won't install/start when DNS can't resolve quay.io

General information

OS: Linux / macOS / Windows
Hypervisor: KVM / Hyper-V / hyperkit
Did you run crc setup before starting it (Yes/No)?
Running CRC on: Laptop / Baremetal-Server / VM

CRC version

# Put `crc version` output here
[crc@eaas-crc ~]$ crc version
CodeReady Containers version: 1.32.1+4d40571f
OpenShift version: 4.8.5 (embedded in executable)

## CRC status
```bash
# Put `crc status --log-level debug` output here
[crc@eaas-crc ~]$ crc start --cpus 12 --memory 40000 --log-level debug
DEBU HTTP-PROXY: http://10.232.233.70:8080, HTTPS-PROXY: http://10.232.233.70:8080, NO-PROXY: 127.0.0.1,localhost,127.0.0.1,localhost,.corp.amdocs.com,.apps-crc.testing,.crc.testing, proxyCAFile:
DEBU CodeReady Containers version: 1.32.1+4d40571f
DEBU OpenShift version: 4.8.5 (embedded in executable)
DEBU Running 'crc start'
DEBU Total memory of system is 47225720832 bytes
DEBU No new version available. The latest version is 1.32.1
DEBU Found binary path at /home/crc/.crc/bin/crc-driver-libvirt
DEBU Launching plugin server for driver libvirt
DEBU Plugin server listening at address 127.0.0.1:43235
DEBU () Calling .GetVersion
DEBU Using API Version 1
DEBU () Calling .SetConfigRaw
DEBU () Calling .GetMachineName
DEBU (crc) Calling .GetState
DEBU (crc) DBG | time="2021-09-20T04:51:34-04:00" level=debug msg="Getting current state..."
DEBU (crc) DBG | time="2021-09-20T04:51:34-04:00" level=debug msg="Fetching VM..."
DEBU Making call to close driver server
DEBU (crc) Calling .Close
DEBU Successfully made call to close driver server
DEBU Making call to close connection to plugin binary
DEBU (crc) DBG | time="2021-09-20T04:51:34-04:00" level=debug msg="Closing plugin on server side"
DEBU Checking if systemd-resolved.service is running
DEBU Running 'systemctl status systemd-resolved.service'
DEBU Command failed: exit status 3
DEBU stdout: * systemd-resolved.service - Network Name Resolution
   Loaded: loaded (/usr/lib/systemd/system/systemd-resolved.service; disabled; vendor preset: disabled)
   Active: inactive (dead)
     Docs: man:systemd-resolved.service(8)
           https://www.freedesktop.org/wiki/Software/systemd/resolved
           https://www.freedesktop.org/wiki/Software/systemd/writing-network-configuration-managers
           https://www.freedesktop.org/wiki/Software/systemd/writing-resolver-clients
DEBU stderr:
INFO Checking if running as non-root
INFO Checking if running inside WSL2
INFO Checking if crc-admin-helper executable is cached
DEBU Running '/home/crc/.crc/bin/crc-admin-helper-linux --version'
DEBU Found crc-admin-helper-linux version 0.0.6
DEBU crc-admin-helper executable already cached
INFO Checking for obsolete admin-helper executable
DEBU Checking if an older admin-helper executable is installed
DEBU No older admin-helper executable found
INFO Checking if running on a supported CPU architecture
INFO Checking minimum RAM requirements
DEBU Total memory of system is 47225720832 bytes
INFO Checking if crc executable symlink exists
INFO Checking if Virtualization is enabled
DEBU Checking if the vmx/svm flags are present in /proc/cpuinfo
DEBU CPU virtualization flags are good
INFO Checking if KVM is enabled
DEBU Checking if /dev/kvm exists
DEBU /dev/kvm was found
INFO Checking if libvirt is installed
DEBU Checking if 'virsh' is available
DEBU 'virsh' was found in /usr/bin/virsh
DEBU Checking 'virsh capabilities' for libvirtd/qemu availability
DEBU Running 'virsh --readonly --connect qemu:///system capabilities'
DEBU Found x86_64 hypervisor with 'hvm' capabilities
INFO Checking if user is part of libvirt group
DEBU Checking if current user is part of the libvirt group
DEBU Current user is already in the libvirt group
INFO Checking if active user/process is currently part of the libvirt group
INFO Checking if libvirt daemon is running
DEBU Checking if libvirtd service is running
DEBU Running 'systemctl status virtqemud.socket'
DEBU Command failed: exit status 3
DEBU stdout: * virtqemud.socket - Libvirt qemu local socket
   Loaded: loaded (/usr/lib/systemd/system/virtqemud.socket; disabled; vendor preset: disabled)
   Active: inactive (dead)
   Listen: /run/libvirt/virtqemud-sock (Stream)
DEBU stderr:
DEBU virtqemud.socket is neither running nor listening
DEBU Running 'systemctl status libvirtd.socket'
DEBU libvirtd.socket is running
INFO Checking if a supported libvirt version is installed
DEBU Checking if libvirt version is >=3.4.0
DEBU Running 'virsh -v'
INFO Checking if crc-driver-libvirt is installed
DEBU Checking if crc-driver-libvirt is installed
DEBU Running '/home/crc/.crc/bin/crc-driver-libvirt version'
DEBU Found crc-driver-libvirt version 0.13.1
DEBU crc-driver-libvirt is already installed
INFO Checking crc daemon systemd socket units
WARN Skipping above check...
INFO Checking if systemd-networkd is running
DEBU Checking if systemd-networkd.service is running
DEBU Running 'systemctl status systemd-networkd.service'
DEBU Command failed: exit status 4
DEBU stdout:
DEBU stderr: Unit systemd-networkd.service could not be found.
DEBU systemd-networkd.service is not running
INFO Checking if NetworkManager is installed
DEBU Checking if 'nmcli' is available
DEBU 'nmcli' was found in /usr/bin/nmcli
INFO Checking if NetworkManager service is running
DEBU Checking if NetworkManager.service is running
DEBU Running 'systemctl status NetworkManager.service'
DEBU NetworkManager.service is already running
INFO Checking if /etc/NetworkManager/conf.d/crc-nm-dnsmasq.conf exists
DEBU Checking NetworkManager configuration
DEBU NetworkManager configuration is good
INFO Checking if /etc/NetworkManager/dnsmasq.d/crc.conf exists
DEBU Checking dnsmasq configuration
DEBU dnsmasq configuration is good
INFO Checking if libvirt 'crc' network is available
DEBU Checking if libvirt 'crc' network exists
DEBU Running 'virsh --connect qemu:///system net-info crc'
DEBU Checking if libvirt 'crc' definition is up to date
DEBU Running 'virsh --connect qemu:///system net-dumpxml --inactive crc'
DEBU libvirt 'crc' network has the expected value
INFO Checking if libvirt 'crc' network is active
DEBU Checking if libvirt 'crc' network is active
DEBU Running 'virsh --connect qemu:///system net-info crc'
DEBU libvirt 'crc' network is already active
DEBU Checking file: /home/crc/.crc/machines/crc/.crc-exist
DEBU Found binary path at /home/crc/.crc/bin/crc-driver-libvirt
DEBU Launching plugin server for driver libvirt
DEBU Plugin server listening at address 127.0.0.1:44081
DEBU () Calling .GetVersion
DEBU Using API Version 1
DEBU () Calling .SetConfigRaw
DEBU () Calling .GetMachineName
DEBU (crc) Calling .GetBundleName
DEBU (crc) Calling .GetState
DEBU (crc) DBG | time="2021-09-20T04:51:34-04:00" level=debug msg="Getting current state..."
DEBU (crc) DBG | time="2021-09-20T04:51:34-04:00" level=debug msg="Fetching VM..."
INFO Starting CodeReady Containers VM for OpenShift 4.8.5...
DEBU Updating CRC VM configuration
DEBU (crc) Calling .GetConfigRaw
DEBU (crc) Calling .Start
DEBU (crc) DBG | time="2021-09-20T04:51:34-04:00" level=debug msg="Starting VM crc"
DEBU (crc) DBG | time="2021-09-20T04:51:34-04:00" level=debug msg="Validating network"
DEBU (crc) DBG | time="2021-09-20T04:51:34-04:00" level=debug msg="Validating storage pool"
DEBU (crc) DBG | time="2021-09-20T04:51:40-04:00" level=debug msg="GetIP called for crc"
DEBU (crc) DBG | time="2021-09-20T04:51:40-04:00" level=debug msg="Getting current state..."
DEBU (crc) DBG | time="2021-09-20T04:51:40-04:00" level=debug msg="IP address: 192.168.130.11"
DEBU (crc) DBG | time="2021-09-20T04:51:40-04:00" level=info msg="Found IP for machine: 192.168.130.11"
DEBU (crc) Calling .GetConfigRaw
DEBU Waiting for machine to be running, this may take a few minutes...
DEBU retry loop: attempt 0
DEBU (crc) Calling .GetState
DEBU (crc) DBG | time="2021-09-20T04:51:40-04:00" level=debug msg="Getting current state..."
DEBU Machine is up and running!
DEBU (crc) Calling .GetState
DEBU (crc) DBG | time="2021-09-20T04:51:40-04:00" level=debug msg="Getting current state..."
DEBU (crc) Calling .GetIP
DEBU (crc) DBG | time="2021-09-20T04:51:40-04:00" level=debug msg="GetIP called for crc"
DEBU (crc) DBG | time="2021-09-20T04:51:40-04:00" level=debug msg="Getting current state..."
DEBU (crc) DBG | time="2021-09-20T04:51:40-04:00" level=debug msg="IP address: 192.168.130.11"
INFO CodeReady Containers instance is running with IP 192.168.130.11
DEBU Waiting until ssh is available
DEBU retry loop: attempt 0
DEBU Running SSH command: exit 0
DEBU Using ssh private keys: [/home/crc/.crc/cache/crc_libvirt_4.8.5/id_ecdsa_crc /home/crc/.crc/machines/crc/id_ecdsa]
DEBU SSH command results: err: dial tcp 192.168.130.11:22: connect: no route to host, output:
DEBU error: Temporary error: ssh command error:
command : exit 0
err     : dial tcp 192.168.130.11:22: connect: no route to host\n - sleeping 1s
DEBU retry loop: attempt 1
DEBU Running SSH command: exit 0
DEBU Using ssh private keys: [/home/crc/.crc/cache/crc_libvirt_4.8.5/id_ecdsa_crc /home/crc/.crc/machines/crc/id_ecdsa]
DEBU SSH command results: err: dial tcp 192.168.130.11:22: connect: no route to host, output:
DEBU error: Temporary error: ssh command error:
command : exit 0
err     : dial tcp 192.168.130.11:22: connect: no route to host\n - sleeping 1s
DEBU retry loop: attempt 2
DEBU Running SSH command: exit 0
DEBU Using ssh private keys: [/home/crc/.crc/cache/crc_libvirt_4.8.5/id_ecdsa_crc /home/crc/.crc/machines/crc/id_ecdsa]
DEBU SSH command results: err: dial tcp 192.168.130.11:22: connect: no route to host, output:
DEBU error: Temporary error: ssh command error:
command : exit 0
err     : dial tcp 192.168.130.11:22: connect: no route to host\n - sleeping 1s
DEBU retry loop: attempt 3
DEBU Running SSH command: exit 0
DEBU Using ssh private keys: [/home/crc/.crc/cache/crc_libvirt_4.8.5/id_ecdsa_crc /home/crc/.crc/machines/crc/id_ecdsa]
DEBU SSH command results: err: dial tcp 192.168.130.11:22: connect: no route to host, output:
DEBU error: Temporary error: ssh command error:
command : exit 0
err     : dial tcp 192.168.130.11:22: connect: no route to host\n - sleeping 1s
DEBU retry loop: attempt 4
DEBU Running SSH command: exit 0
DEBU Using ssh private keys: [/home/crc/.crc/cache/crc_libvirt_4.8.5/id_ecdsa_crc /home/crc/.crc/machines/crc/id_ecdsa]
DEBU SSH command results: err: <nil>, output:
INFO CodeReady Containers VM is running
DEBU Running SSH command: cat /home/core/.ssh/authorized_keys
DEBU SSH command results: err: <nil>, output: ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAGzvyf6w72E7TLSCFN00F7T0fYqiAV0iyy0BcujvN0/UzzzkMfJIsaYmZ4KUElNAqcJAoWysFq1khJa/aEvRSIqLgFVeTl6a1MLMq/FkouydW/gYZT+JeTfgJjoEX20vgnSbARWlz/IbKiKxIqMO5FfcNH5VY5DXVw0xQwbUuIguTFfeA==
DEBU Running SSH command: realpath /dev/disk/by-label/root
DEBU SSH command results: err: <nil>, output: /dev/vda4
DEBU Using root access: Growing /dev/vda4 partition
DEBU Running SSH command: sudo /usr/bin/growpart /dev/vda 4
DEBU SSH command results: err: Process exited with status 1, output: NOCHANGE: partition 4 is size 63961055. it cannot be grown
DEBU No free space after /dev/vda4, nothing to do
DEBU Using root access: make root Podman socket accessible
DEBU Running SSH command: sudo chmod 777 /run/podman/ /run/podman/podman.sock
DEBU SSH command results: err: <nil>, output:
DEBU Creating /var/srv/dnsmasq.conf with permissions 0644 in the CRC VM
DEBU Running SSH command: <hidden>
DEBU SSH command succeeded
DEBU Running SSH command: sudo podman rm -f dnsmasq
DEBU SSH command results: err: Process exited with status 1, output:
DEBU Running SSH command: sudo rm -f /var/lib/cni/networks/podman/10.88.0.8
DEBU SSH command results: err: <nil>, output:
DEBU Running SSH command: sudo podman run  --ip 10.88.0.8 --name dnsmasq -v /var/srv/dnsmasq.conf:/etc/dnsmasq.conf -p 53:53/udp --privileged -d quay.io/crcont/dnsmasq:latest
DEBU SSH command results: err: Process exited with status 125, output:
DEBU Making call to close driver server
DEBU (crc) Calling .Close
DEBU Successfully made call to close driver server
DEBU (crc) DBG | time="2021-09-20T04:53:55-04:00" level=debug msg="Closing plugin on server side"
DEBU Making call to close connection to plugin binary
Error running post start: ssh command error:
command : sudo podman run  --ip 10.88.0.8 --name dnsmasq -v /var/srv/dnsmasq.conf:/etc/dnsmasq.conf -p 53:53/udp --privileged -d quay.io/crcont/dnsmasq:latest
err     : Process exited with status 125\n


## CRC config
```bash
# Put `crc config view` output here
[crc@eaas-crc ~]$ crc config view
- consent-telemetry                     : no
- http-proxy                            : http://10.232.233.70:8080
- https-proxy                           : http://10.232.233.70:8080
- no-proxy                              : 127.0.0.1,localhost,.corp.amdocs.com,.apps-crc.testing,.crc.testing
- skip-check-daemon-systemd-sockets     : true
- skip-check-daemon-systemd-unit        : true


## Host Operating System
```bash
# Put the output of `cat /etc/os-release` in case of Linux
# put the output of `sw_vers` in case of Mac
# Put the output of `systeminfo` in case of Windows
[crc@eaas-crc ~]$ cat /etc/os-release
NAME="Red Hat Enterprise Linux"
VERSION="8.4 (Ootpa)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="8.4"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
HOME_URL="https://www.redhat.com/"
DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.4"


### Steps to reproduce

  1. 
  2. 
  3. 
  4. 

### Expected


### Actual


### Logs

Before gather the logs try following if that fix your issue
```bash
$ crc delete -f
$ crc cleanup
$ crc setup
$ crc start --log-level debug

Please consider posting the output of crc start --log-level debug on http://gist.github.com/ and post the link in the issue.

[crc@eaas-crc ~]$ cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.2.100 eaas-crc.corp.amdocs.com eaas-crc

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 46 (22 by maintainers)

Most upvoted comments

Please do not close the case , till the issue get resolved.

If the issue is determined to be unrelated to crc, we’ll close it as this is not a generic support forum, but an issue tracker for crc issues.

cfergeau on Sep 21, 2021