I confirm that the first step woks on Ubuntu 14.04… Though I cannot establish communication from the VM to the Kubernetes cluster .

Kubernetes => VM ✔️ VM => kubernetes ❌

Issues:

1) istio-sidecar.deb not installing correctly on ubuntu

When installing using sudo dpkg -i istio-sidecar.deb the files are not copied into systemd. I had to extract the .deb and copy the service files to /etc/systemd/system.

2) Updating scripts

In istio-node-agent.sh I updated the istio-citadel port to the nodePort port of my cluster.
In istio-start.sh I updated pilot port (to match nodePort again) and ISTIO_SVC_IP that was getting its IP from eth0 instead of getting it from eth1.

3) Launching the two scripts using systemctl

node_agent and Istio seems ok but when I do sudo iptables -L I get nothing:

$ sudo iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Its like the iptables script is not doing anything (even when launched manualy).

4) Trying to curl the cluster

Curling myservice from the VM ends in 503.

curl -v myservice.default.svc.cluster.local
* Rebuilt URL to: myservice.default.svc.cluster.local/
* Hostname was NOT found in DNS cache
*   Trying 10.233.35.233...
* Connected to myservice.default.svc.cluster.local (10.233.35.233) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.35.0
> Host: myservice.default.svc.cluster.local
> Accept: */*
>
< HTTP/1.1 503 Service Unavailable
< content-length: 91
< content-type: text/plain
< date: Mon, 17 Feb 2020 10:03:50 GMT
* Server envoy is not blacklisted
< server: envoy
<
* Connection #0 to host myservice.default.svc.cluster.local left intact
upstream connect error or disconnect/reset before headers. reset reason: connection failure

Strange…

Also in istio.err.log I do have those occasionnal stacktrace:

[2020-02-17 10:00:03.483][2652][warning][config] [external/envoy/source/common/config/grpc_mux_subscription_impl.cc:73] gRPC config for type.googleapis.com/envoy.api.v2.Listener rejected: Error adding/updating listener(s) 10.233.27.174_44134: Didn't find a registered implementation for name: 'envoy.listener.http_inspector', 10.233.2.233_6379: Didn't find a registered implementation for name: 'envoy.listener.http_inspector', 10.233.27.187_80: Didn't find a registered implementation for name: 'envoy.listener.http_inspector', 10.233.61.231_50000: Didn't find a registered implementation for name: 'envoy.listener.http_inspector', 10.233.11.160_80: Didn't find a registered implementation for name: 'envoy.listener.http_inspector', 10.233.0.3_9153: Didn't find a registered implementation for name: 'envoy.listener.http_inspector', 0.0.0.0_3000: Didn't find a registered implementation for name: 'envoy.listener.http_inspector', 10.233.27.115_8080: Didn't find a registered implementation for name: 'envoy.listener.http_inspector', 10.233.39.188_443: Didn't find a registered implementation for name: 'envoy.listener.http_inspector', 0.0.0.0_9090: Didn't find a registered implementation for name: 'envoy.listener.http_inspector', 10.233.21.142_443: Didn't find a registered implementation for name: 'envoy.listener.http_inspector', 10.233.0.10_9153: Didn't find a registered implementation for name: 'envoy.listener.http_inspector', 10.233.34.164_80: Didn't find a registered implementation for name: 'envoy.listener.http_inspector', 0.0.0.0_47323: Didn't find a registered implementation for name: 'envoy.listener.http_inspector', 0.0.0.0_9796: Didn't find a registered implementation for name: 'envoy.listener.http_inspector', 10.233.37.168_9000: Didn't find a registered implementation for name: 'envoy.listener.http_inspector', 10.233.49.225_5683: Didn't find a registered implementation for name: 'envoy.listener.http_inspector', 10.233.3.154_80: Didn't find a registered implementation for name: 'envoy.listener.http_inspector', 10.233.57.245_42422: Didn't find a registered implementation for name: 'envoy.listener.http_inspector', 10.233.52.181_3001: Didn't find a registered implementation for name: 'envoy.listener.http_inspector', 10.233.29.203_15020: Didn't find a registered implementation for name: 'envoy.listener.http_inspector'

Steps I did:

$ export SERVICE_NAMESPACE="default"
$ kubectl cluster-info dump | grep --color service-cluster-ip-range #To get ip cidr

kubectl -n $SERVICE_NAMESPACE get secret istio.default  \
    -o jsonpath='{.data.root-cert\.pem}' |base64 --decode > root-cert.pem
kubectl -n $SERVICE_NAMESPACE get secret istio.default  \
    -o jsonpath='{.data.key\.pem}' |base64 --decode > key.pem
kubectl -n $SERVICE_NAMESPACE get secret istio.default  \
      -o jsonpath='{.data.cert-chain\.pem}' |base64 --decode > cert-chain.pem

#Installation of certs on the external VM
$ sudo mkdir -p /etc/certs
$ sudo cp {root-cert.pem,cert-chain.pem,key.pem} /etc/certs

#cluster.env
ISTIO_CP_AUTH=MUTUAL_TLS
ISTIO_SERVICE_CIDR=10.233.0.0/18

$ sudo cp cluster.env /var/lib/istio/envoy
$ sudo chown -R istio-proxy /etc/certs /var/lib/istio/envoy

cat /etc/hosts

10.10.5.101 istio-citadel istio-pilot istio-pilot.istio-system

sudo node_agent --ca-address istio-citadel:<PORT_CITADEL_VIA_GATEWAY> --env onprem

---

Istio config:

• meshExpansion.enabled: true
• controlPlaneSecurityEnabled: true

ℹ️ Istio’s gateway has been set to NodePort and is accessible from the external VM.

Also, tried with istio-1.4.3/install/kubernetes\mesh-expansion.yaml

Is it possible that the istio gateway absorbs the certificates ?