kubeflow: ml-pipelines [mysql] packets.go:36: unexpected EOF

/kind bug

What steps did you take and what happened:

Helllo all, I think I need your help with this issue. We have installed KF1.2 on k8s 1.20 with kfctl and https://raw.githubusercontent.com/kubeflow/manifests/v1.2-branch/kfdef/kfctl_istio_dex.v1.2.0.yaml Everything works well, beside mysql and always get this error at the mysql pod. I read/studied many possible solutions but none helped (e.g. switchin to Istio TLS to DISABLE). Same Problem like this colleague: https://stackoverflow.com/questions/65121700/kubeflow-ml-pipeline-cannot-connect-to-mysql-pod kubectl -n kubeflow logs ml-pipeline-555747c844-mfqfk ml-pipeline-api-server I0119 14:59:49.576535 6 client_manager.go:140] Initializing client manager I0119 14:59:49.576735 6 config.go:56] Config DBConfig.ExtraParams not specified, skipping [mysql] 2021/01/19 14:59:49 packets.go:36: unexpected EOF [mysql] 2021/01/19 14:59:50 packets.go:36: unexpected EOF [mysql] 2021/01/19 14:59:51 packets.go:36: unexpected EOF [mysql] 2021/01/19 14:59:52 packets.go:36: unexpected EOF [mysql] 2021/01/19 14:59:53 packets.go:36: unexpected EOF [mysql] 2021/01/19 14:59:55 packets.go:36: unexpected EOF

What did you expect to happen: Acessing the Pipelines without an Error in the UI.

Anything else you would like to add: The ml-pipeline pods are crashing, but this seems like a mysql/istio connection problem.

Environment: Managed K8s

• Kubeflow version: (version number can be found at the bottom left corner of the Kubeflow dashboard):
• 1.2
• kfctl version: (use kfctl version):
• 1.2
• Kubernetes platform: (e.g. minikube)
• 1.20
• Kubernetes version: (use kubectl version):
• OS (e.g. from /etc/os-release):