openyurt: kubectl exec to edge worker nodes joined via yurtadm fails.

What happened:

root@tomoyafujita:~/istio_work/istio-1.15.0# kubectl exec --stdin --tty ubuntu22-deamonset-glcdn -- date
error: unable to upgrade connection: fail to setup the tunnel: fail to setup TLS handshake through the Tunnel: write unix @->/tmp/interceptor-proxier.sock: write: broken pipe

What you expected to happen:

root@tomoyafujita:~/istio_work/istio-1.15.0# kubectl exec --stdin --tty ubuntu22-deamonset-x8trn -- date
Wed Sep  7 23:01:52 UTC 2022

How to reproduce it (as minimally and precisely as possible):

root@tomoyafujita:~# cat ubuntu22-daemonset.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: ubuntu22-deamonset
spec:
  selector:
    matchLabels:
      app: ubuntu22
  template:
    metadata:
      labels:
        app: ubuntu22
    spec:
      containers:
      - name: ubuntu22
        tty: true
        image: ubuntu:22.04
        command: ["/bin/bash", "-c"]
        args: ["sleep 3600"]
        imagePullPolicy: IfNotPresent
      tolerations:
      - key: node-role.kubernetes.io/master
        operator: Exists
        effect: NoSchedule
      - key: node-role.kubernetes.io/control-plane
        operator: Exists
        effect: NoSchedule
      restartPolicy: Always
  • try to login each pod via kubectl exec, e.g) kubectl exec --stdin --tty ubuntu22-deamonset-glcdn -- /bin/bash

Anything else we need to know?:

Environment:

  • OpenYurt version: 0.7.0
  • Kubernetes version (use kubectl version): v1.22.13
  • OS (e.g: cat /etc/os-release):
root@tomoyafujita:~/istio_work/istio-1.15.0# cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.4 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.4 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
  • Kernel (e.g. uname -a):
Linux tomoyafujita 5.15.0-46-generic #49~20.04.1-Ubuntu SMP Thu Aug 4 19:15:44 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
  • Install tools:
  • Others:

others

/kind bug

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 17 (14 by maintainers)

Most upvoted comments

@FengGaoCSC It looks like the same problem as issue #925 , maybe you can have a try by cleaning up the certificates of yurt-tunnel-server and yurt-tunnel-agent and recreating yurt-tunnel pods.

@luc99hen @rambohe-ch @fujitatomoya

thanks very much. I try it with issue https://github.com/openyurtio/openyurt/issues/925, it works well now. So we can make sure this is the certification issue.

root@hyan-optiplex-7050:/home/hyan# kubectl get node -o wide
NAME      STATUS   ROLES                  AGE   VERSION    INTERNAL-IP     EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME
k8s-121   Ready    <none>                 31m   v1.22.13   192.168.0.121   <none>        Ubuntu 20.04.1 LTS   5.15.0-46-generic   docker://19.3.8
master    Ready    control-plane,master   60m   v1.22.13   192.168.0.140   <none>        Ubuntu 20.04.4 LTS   5.15.0-46-generic   docker://20.10.12

root@hyan-optiplex-7050:/home/hyan# kubectl get pod -o wide
NAME                                      READY   STATUS      RESTARTS      AGE     IP          NODE      NOMINATED NODE   READINESS GATES
ubuntu22-deamonset-lhgkn                  1/1     Running     2 (13m ago)   133m    10.44.0.3   k8s-121   <none>           <none>
ubuntu22-deamonset-nnjl8                  1/1     Running     3 (20s ago)   3h3m    10.32.0.4   master    <none>   

root@hyan-optiplex-7050:/home/hyan# kubectl exec --stdin --tty ubuntu22-deamonset-lhgkn -- /bin/bash
root@ubuntu22-deamonset-lhgkn:/# ls
bin  boot  dev  etc  home  lib  lib32  lib64  libx32  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
root@ubuntu22-deamonset-lhgkn:/# exit 
exit
command terminated with exit code 127

confirmed the fix, closing in favor of https://github.com/openyurtio/yurt-app-manager/pull/115. thanks for iterating.

@luc99hen thanks for the info, https://github.com/openyurtio/yurt-app-manager/pull/115 adding toleration to be able to deploy OpenYurt components on master makes sense to me.

@rambohe-ch

yurt-tunnel-agent should be installed manually.

i did not see this requirement from the manual v0.7.0.

@fujitatomoya Thanks for your reply. In the manual v0.7.0, yurt-tunnel-agent is recommended to install at here: https://openyurt.io/docs/v0.7.0/installation/manually-setup#33-setup-yurt-tunnel. Anyway , i’d like to encourage you to try version v1.0, if you have any other problems, please feel free to contact us.

@rambohe-ch

Did the v0.7 has this issue when yurtadm join ? since I saw @fujitatomoya use OpenYurt version: 0.7.0

regards