minikube: v1.5 ISO: mysqld failed while attempting to check config: command was: mysqld --verbose --help

Hi,

I am trying to run MySQL docker container in Minikube and I am getting an error. I’ve tried already with MySQL version 5.7.14, 5.7.27, 5.7.28, 8.0.18, etc, and always the same error.

2019-10-27 14:36:04+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.28-1debian9 started.
2019-10-27 14:36:12+00:00 [ERROR] [Entrypoint]: mysqld failed while attempting to check config
	command was: mysqld --verbose --help

On the other hand, if I use Kubernetes in Docker Desktop (For Mac) or simply run MySQL with docker run or docker-compose, I don’t have any problem.

I’ve tried to deploy MySQL to Minikube using the Helm Chart and also using YAML file (commands and file shown below).

The exact command to reproduce the issue:

minikube start --memory='8000mb' --vm-driver='virtualbox' p.s. I’ve tried also with hyperkit and I have the same problem.

helm init

Helm command to deploy to Minikube

helm install \
--namespace dev \
--name my-mysql \
--set imageTag=5.7.28 \
--set mysqlDatabase=exampledb \
--set mysqlRootPassword=secret \
stable/mysql

Yaml file deployment command

kubectl apply -f mysql-deployment.yaml

mysql-deployment.yaml file

apiVersion: v1
kind: Service
metadata:
  name: mysql-service
  labels:
    app: mysql
    tier: backend
spec:
  selector:
    app: mysql
  ports:
    - port: 3306
      targetPort: 3306
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-pv-claim
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysql-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mysql
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
        - name: mysql
          image: mysql:5.7.28
          env:
            - name: MYSQL_DATABASE
              value: exampledb
            - name: MYSQL_ROOT_PASSWORD
              value: secret
          ports:
            - containerPort: 3306
          volumeMounts:
            - name: mysql-persistent-storage
              mountPath: /var/lib/mysql
      volumes:
        - name: mysql-persistent-storage
          persistentVolumeClaim:
            claimName: mysql-pv-claim

The full output of the command that failed:

kubectl describe pod mysql-deployment-66f98fcdb4-dlp8x

Name:         mysql-deployment-66f98fcdb4-dlp8x
Namespace:    default
Priority:     0
Node:         minikube/192.168.99.101
Start Time:   Sun, 27 Oct 2019 15:03:10 +0100
Labels:       app=mysql
              pod-template-hash=66f98fcdb4
Annotations:  <none>
Status:       Running
IP:           172.17.0.7
IPs:
  IP:           172.17.0.7
Controlled By:  ReplicaSet/mysql-deployment-66f98fcdb4
Containers:
  mysql:
    Container ID:   docker://fab6e63fa70e470a3eeab1496a9cfa162cb2d0eb36722643a76907c252025937
    Image:          mysql:5.7.28
    Image ID:       docker-pullable://mysql@sha256:44b33224e3c406bf50b5a2ee4286ed0d7f2c5aec1f7fdb70291f7f7c570284dd
    Port:           3306/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Sun, 27 Oct 2019 15:36:04 +0100
      Finished:     Sun, 27 Oct 2019 15:36:12 +0100
    Ready:          False
    Restart Count:  11
    Environment:
      MYSQL_DATABASE:       authorbookdb
      MYSQL_ROOT_PASSWORD:  secret
    Mounts:
      /var/lib/mysql from mysql-persistent-storage (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-l6w9g (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  mysql-persistent-storage:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  mysql-pv-claim
    ReadOnly:   false
  default-token-l6w9g:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-l6w9g
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason            Age                    From               Message
  ----     ------            ----                   ----               -------
  Warning  FailedScheduling  <unknown>              default-scheduler  pod has unbound immediate PersistentVolumeClaims
  Normal   Scheduled         <unknown>              default-scheduler  Successfully assigned default/mysql-deployment-66f98fcdb4-dlp8x to minikube
  Normal   Pulled            32m (x5 over 34m)      kubelet, minikube  Container image "mysql:5.7.28" already present on machine
  Normal   Created           32m (x5 over 34m)      kubelet, minikube  Created container mysql
  Normal   Started           32m (x5 over 34m)      kubelet, minikube  Started container mysql
  Warning  BackOff           4m12s (x131 over 33m)  kubelet, minikube  Back-off restarting failed container

kubectl logs -p mysql-deployment-66f98fcdb4-dlp8x

2019-10-27 14:36:04+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.28-1debian9 started.
2019-10-27 14:36:12+00:00 [ERROR] [Entrypoint]: mysqld failed while attempting to check config
	command was: mysqld --verbose --help

The operating system version:

MacBook Pro (Retina, 13-inch, Early 2015)

Docker Desktop (for Mac) version

2.1.0.4 (39773)

Minikube version

v1.5.0

Helm version

Client: &version.Version{SemVer:"v2.15.1", GitCommit:"cf1de4f8ba70eded310918a8af3a96bfe8e7683b", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.15.1", GitCommit:"cf1de4f8ba70eded310918a8af3a96bfe8e7683b", GitTreeState:"clean"}

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Reactions: 1
  • Comments: 23 (9 by maintainers)

Commits related to this issue

Most upvoted comments

For Docker runtime users, this seems to work:

minikube start --docker-opt="default-ulimit=nofile=102400:102400"

I’m working on a solution to be released with minikube v1.5.1 later this week, along with integration tests to ensure that this isn’t broken in the future.

For some reason, setting maxOpenFiles into the kubeadm config isn’t working as expected. I’d rather find a way to set this that doesn’t involve passing hints to each container runtime.

+3
tstromberg on Oct 29, 2019

For Docker runtime users, this seems to work:

minikube start --docker-opt="default-ulimit=nofile=102400:102400"

I’m working on a solution to be released with minikube v1.5.1 later this week, along with integration tests to ensure that this isn’t broken in the future.

For some reason, setting maxOpenFiles into the kubeadm config isn’t working as expected. I’d rather find a way to set this that doesn’t involve passing hints to each container runtime.

Thank you so much for figuring this out and fixing it in 1.5.1. Everything was working fine for me on 1.4 last week and I upgraded to 1.5.0 and couldn’t figure out what was going on. Was driving me crazy. Working again now!

+1
bradowatson on Oct 30, 2019

Thanks for the updates. If someone is able to duplicate this, this will almost certainly point to the root cause:

strace -f mysqld --verbose --help

You may need to install the strace package into the container first.

I would have expected --verbose to emit something more useful, and for --help to not fail. My best guess to a root cause is a permissions change, but it really could be anything, as the v1.5.0 ISO has both a new kernel and a new userland.

I believe this will workaround the issue in the meantime:

minikube delete && minikube start --iso-url='https://storage.googleapis.com/minikube/iso/minikube-v1.4.0.iso'

+1
tstromberg on Oct 28, 2019
Read more comments on GitHub
← minikube: Using local driver gets error 'localkube-v1.9.4: no such file or directory'
minikube: verifying ingress addon pods : timed out waiting for the condition: timed out waiting for the condition →