rke: rke_linux-amd64 up: Failed to set up SSH tunneling for host

RKE version: rke version v0.2.4

Docker version: (docker version,docker info preferred) ‘docker version’ Client: Version: 18.06.2 API version: 1.38 Go version: go1.10.7 Git commit: 6d37f41 Built: Wed Jun 12 23:08:07 2019 OS/Arch: linux/amd64 Experimental: false

Server: Engine: Version: 18.06.2-ce API version: 1.38 (minimum version 1.12) Go version: go1.10.7 Git commit: 6d37f41 Built: Wed Jun 12 23:09:09 2019 OS/Arch: linux/amd64 Experimental: false

‘docker info’ Containers: 0 Running: 0 Paused: 0 Stopped: 0 Images: 0 Server Version: 18.06.2-ce Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true Logging Driver: json-file Cgroup Driver: cgroupfs Plugins: Volume: local Network: bridge host macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog Swarm: inactive Runtimes: runc Default Runtime: runc Init Binary: docker-init containerd version: 468a545b9edcd5932818eb9de8e72413e616e86e runc version: a592beb5bc4c4092b1b1bac971afed27687340c5 (expected: 69663f0bd4b60df09991c08812a60108003fa340) init version: fec3683 Security Options: apparmor seccomp Profile: default Kernel Version: 4.19.52-1.ph3-esx Operating System: VMware Photon OS/Linux OSType: linux Architecture: x86_64 CPUs: 1 Total Memory: 1.952GiB Name: rke04 ID: J34G:FK4K:C3NF:IVZY:F52P:BIMT:TPEB:L7OO:GEHZ:P4DW:VCD4:6TNO Docker Root Dir: /var/lib/docker Debug Mode (client): false Debug Mode (server): false Registry: https://index.docker.io/v1/ Labels: Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false

Operating system and kernel: (cat /etc/os-release, uname -r preferred) ‘cat /etc/os-release’ NAME=“VMware Photon OS” VERSION=“3.0” ID=photon VERSION_ID=3.0 PRETTY_NAME=“VMware Photon OS/Linux” ANSI_COLOR=“1;34” HOME_URL=“https://vmware.github.io/photon/” BUG_REPORT_URL=“https://github.com/vmware/photon/issues

‘uname -r’ 4.19.52-1.ph3-esx

Type/provider of hosts: (VirtualBox/Bare-metal/AWS/GCE/DO) EXS VM

cluster.yml file: nodes:

  • address: 192.168.1.201 user: root role: [controlplane,worker,etcd]
  • address: 192.168.1.202 user: root role: [controlplane,worker,etcd]
  • address: 192.168.1.203 user: root role: [controlplane,worker,etcd]

services: etcd: snapshot: true creation: 6h retention: 24h

Steps to Reproduce: Start 3 VMs running photon-hw13_uefi-3.0-26156e2.ova and try to install RKE

Results: INFO[0000] Initiating Kubernetes cluster
INFO[0000] [dialer] Setup tunnel for host [192.168.1.201] INFO[0000] [dialer] Setup tunnel for host [192.168.1.202] INFO[0000] [dialer] Setup tunnel for host [192.168.1.203] WARN[0000] Failed to set up SSH tunneling for host [192.168.1.203]: Can’t retrieve Docker Info: error during connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.24/info: Unable to access the service on /var/run/docker.sock. The service might be still starting up. Error: ssh: rejected: connect failed (open failed) WARN[0000] Failed to set up SSH tunneling for host [192.168.1.201]: Can’t retrieve Docker Info: error during connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.24/info: Unable to access the service on /var/run/docker.sock. The service might be still starting up. Error: ssh: rejected: connect failed (open failed) WARN[0000] Failed to set up SSH tunneling for host [192.168.1.202]: Can’t retrieve Docker Info: error during connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.24/info: Unable to access the service on /var/run/docker.sock. The service might be still starting up. Error: ssh: rejected: connect failed (open failed) WARN[0000] Removing host [192.168.1.203] from node lists WARN[0000] Removing host [192.168.1.201] from node lists WARN[0000] Removing host [192.168.1.202] from node lists WARN[0000] [state] can’t fetch legacy cluster state from Kubernetes INFO[0000] [certificates] Generating CA kubernetes certificates INFO[0000] [certificates] Generating Kubernetes API server aggregation layer requestheader client CA certificates INFO[0000] [certificates] Generating admin certificates and kubeconfig INFO[0001] [certificates] Generating Kube Controller certificates INFO[0001] [certificates] Generating Kube Scheduler certificates INFO[0001] [certificates] Generating Node certificate
INFO[0001] [certificates] Generating Kubernetes API server certificates INFO[0002] [certificates] Generating Kube Proxy certificates INFO[0002] [certificates] Generating Kubernetes API server proxy client certificates INFO[0002] Successfully Deployed state file at [./rancher-clusterWORKING.rkestate] INFO[0002] Building Kubernetes cluster
FATA[0002] Cluster must have at least one etcd plane host: please specify one or more etcd in cluster config

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 21 (8 by maintainers)

Most upvoted comments

Can you try with AllowTcpForwarding yes?

following enabled PermitRootLogin yes AllowTcpForwarding yes

sudo usermod -aG docker $USER passwordless authentication to all nodes from rke machine. OS = ubuntu18.04. but no go. same error, same problem @superseb

Hi Team,

Thank you so much for all the help with that issue. Just run into the same one and your suggestions worked like a charm.

Let me conclude and aggregate the suggestions into single message.

Prerequisites

  • Google Compute Engine
  • Provisioning K8s cluster over RKE

Error message (from RKE)

  • FATA[0002] Cluster must have at least one etcd plane host: please specify one or more etcd in cluster config

Solutions (1)

  • You must edit /etc/ssh/sshd_config and enable the following 2 options:

    PermitRootLogin yes AllowTcpForwarding yes

  • And sure, you must configure RKE to connect to those nodes using root login

(2)

  • Add the user under which you are connecting to the nodes to the docker group:

    sudo usermod -aG docker $USER

  • After that you will be fine (no need to connect using root).
  • Yet not sure, whether AllowTcpForwading required in that case.

AllowTcpForwarding no is the default for VMware Photon OS. I don’t know abou other distributions.

I clean up all the distro, tried reinstalling from scratch but didn’t worked in both cases. I give a try on Rancher OS and worked in my first try I’m thinking in give up the Photon OS for now. Thank you.

Please file a new issue with all info and logs so we can take a look.