minikube: pulling kic base image does not respect --image-repository or --image-mirror-country flags for CN registry mirrors

终于算是搞定这个问题了，成功启动minikube的docker驱动的集群，如图：

这个问题预计kicbase没同步到国内镜像源前，大陆用户都会遇到。下面说一下我的思路和方案：

方案1：根据源码重新编译minikube（连带kicbase），如sharifelgamal所述的方案。【根据源码编译带来更多不确定的问题，且minikube刚接触的都是入门用户，此方案不可靠，不推荐】

方案2：让minikube所在的机器走全局代理翻墙，假装身在国外，直接从gcr.io中pull kicbase镜像。【可行，但是kicbase镜像接近1G，翻墙下载估计会比较慢。】

方案3：想各种办法将kicbase镜像搞到本机（通过墙外机器pull下来然后save&load到本机、在国内的某些源中找到kicbase后pull到本地等），然后按照listenerri所提供的帖子中的方案，将镜像重新tag，然后恢复Digest，让minikube能识别，最后搞定。

这里我采用的是方案3，镜像是从阿里云的镜像仓库找到的（感谢上传镜像的同学！），如图：

其中1个的镜像地址：registry.cn-hangzhou.aliyuncs.com/smartliby/kicbase:v0.0.8

使用新的1.10.0版本指定–base-image registry.cn-hangzhou.aliyuncs.com/google_containers/kicbase:v0.0.10 可以正常启动了。

好像还是不行：

oc@ocserver:~$ minikube start --vm-driver docker --base-image registry.cn-hangzhou.aliyuncs.com/google_containers/kicbase:v0.0.10 
😄  minikube v1.10.1 on Ubuntu 18.04
✨  Using the docker driver based on existing profile
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
E0514 11:17:54.137973    3798 cache.go:129] Error downloading kic artifacts:  getting remote image: Get https://docker.pkg.github.com/v2/: dial tcp: lookup docker.pkg.github.com on 127.0.0.53:53: read udp 127.0.0.1:50986->127.0.0.53:53: i/o timeout
🤷  docker "minikube" container is missing, will recreate.
E0514 11:17:54.383291    3798 oci.go:79] docker daemon seems to be stuck. Please try restarting your docker. Will try to delete anyways: unknown state "minikube": docker inspect minikube --format={{.State.Status}}: exit status 1
stdout:


stderr:
Template parsing error: template: :1:8: executing "" at <.State.Status>: map has no entry for key "State"
🔥  Creating docker container (CPUs=2, Memory=3900MB) ...
🤦  StartHost failed, but will try again: recreate: creating host: create: creating: create kic node: create container: docker run -d -t --privileged --security-opt seccomp=unconfined --security-opt apparmor=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --volume minikube:/var --cpus=2 --memory=3900mb -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 docker.pkg.github.com/kubernetes/minikube/kicbase:v0.0.10: exit status 125
stdout:

stderr:
Unable to find image 'docker.pkg.github.com/kubernetes/minikube/kicbase:v0.0.10' locally
docker: Error response from daemon: Get https://docker.pkg.github.com/v2/: dial tcp: lookup docker.pkg.github.com on 127.0.0.53:53: read udp 127.0.0.1:36426->127.0.0.53:53: i/o timeout.
See 'docker run --help'.

🤷  docker "minikube" container is missing, will recreate.
E0514 11:19:45.275536    3798 oci.go:79] docker daemon seems to be stuck. Please try restarting your docker. Will try to delete anyways: unknown state "minikube": docker inspect minikube --format={{.State.Status}}: exit status 1
stdout:


stderr:
Template parsing error: template: :1:8: executing "" at <.State.Status>: map has no entry for key "State"
🔥  Creating docker container (CPUs=2, Memory=3900MB) ...
😿  Failed to start docker container. "minikube start" may fix it: recreate: creating host: create: creating: create kic node: create container: docker run -d -t --privileged --security-opt seccomp=unconfined --security-opt apparmor=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --volume minikube:/var --cpus=2 --memory=3900mb -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 docker.pkg.github.com/kubernetes/minikube/kicbase:v0.0.10: exit status 125
stdout:

stderr:
Unable to find image 'docker.pkg.github.com/kubernetes/minikube/kicbase:v0.0.10' locally
docker: Error response from daemon: Get https://docker.pkg.github.com/v2/: dial tcp: lookup docker.pkg.github.com on 127.0.0.53:53: read udp 127.0.0.1:60680->127.0.0.53:53: i/o timeout.
See 'docker run --help'.


💣  error provisioning host: Failed to start host: recreate: creating host: create: creating: create kic node: create container: docker run -d -t --privileged --security-opt seccomp=unconfined --security-opt apparmor=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --volume minikube:/var --cpus=2 --memory=3900mb -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 docker.pkg.github.com/kubernetes/minikube/kicbase:v0.0.10: exit status 125
stdout:

stderr:
Unable to find image 'docker.pkg.github.com/kubernetes/minikube/kicbase:v0.0.10' locally
docker: Error response from daemon: Get https://docker.pkg.github.com/v2/: dial tcp: lookup docker.pkg.github.com on 127.0.0.53:53: read udp 127.0.0.1:60680->127.0.0.53:53: i/o timeout.
See 'docker run --help'.


😿  minikube is exiting due to an error. If the above message is not useful, open an issue:
👉  https://github.com/kubernetes/minikube/issues/new/choose

试试这个，我正常启动了 minikube start --registry-mirror=https://registry.docker-cn.com --image-repository=registry.cn-hangzhou.aliyuncs.com/google_containers --vm-driver=docker --alsologtostderr -v=8 --base-image registry.cn-hangzhou.aliyuncs.com/google_containers/kicbase:v0.0.10

可能跟我的问题一样，我的解决方案是： https://listenerri.com/2020/04/07/minikube-启动失败/

This is still an issue.

$ minikube version
minikube version: v1.23.2
commit: 0a0ad764652082477c00d51d2475284b5d39ceed
$ minikube start --image-mirror-country=cn --image-repository=registry.cn-hangzhou.aliyuncs.com/google_containers
* minikube v1.23.2 on Centos 7.9.2009
* Using the docker driver based on existing profile
* Starting control plane node minikube in cluster minikube
* Pulling base image ...
* Restarting existing docker container for "minikube" ...
* Found network options:
  - HTTP_PROXY=http://192.168.9.9:8080/
  - HTTPS_PROXY=https://192.168.9.9:8080/
  - NO_PROXY=localhost,127.0.0.1,10.0.0.0/8,192.168.0.0/12
! This container is having trouble accessing https://k8s.gcr.io
* To pull new external images, you may need to configure a proxy: https://minikube.sigs.k8s.io/docs/reference/networking/proxy/
* Preparing Kubernetes v1.22.2 on Docker 20.10.8 ...
  - env HTTP_PROXY=http://192.168.9.9:8080/
  - env HTTPS_PROXY=https://192.168.9.9:8080/
  - env NO_PROXY=localhost,127.0.0.1,10.0.0.0/8,192.168.0.0/12
* Verifying Kubernetes components...
  - Using image kubernetesui/dashboard:v2.3.1
  - Using image kubernetesui/metrics-scraper:v1.0.7
  - Using image gcr.io/k8s-minikube/storage-provisioner:v5
* Enabled addons: storage-provisioner, default-storageclass, dashboard
* kubectl not found. If you need it, try: 'minikube kubectl -- get pods -A'
* Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

官方已推镜像,大家可以试试~~~ https://github.com/AliyunContainerService/minikube/issues/17

可能跟我的问题一样，我的解决方案是： https://listenerri.com/2020/04/07/minikube-启动失败/

你提供的解决方案很有价值。 但是我遇到的情况和你不大一样，我这里是确实连不上外网，本地docker images看不到kicbase镜像；

同时推断，当driver=docker 时，岂不是所有墙内的用户都无法完成minikube start了？