istio: istioctl get authentication problem with heptio aws authenticator

I don’t have Heptio or any Kubernetes client config that requires the Exec Provider. I believe my PR works but it would be very helpful if the folks following this work item could try bringing https://github.com/istio/istio/pull/6338 into their workspace and building istioctl with make istioctl.

If you trust me and are willing to try my handmade istioctl binaries you can get them at Mac OSX: https://ibm.box.com/s/l8lmda5g4q2g81fce1cfrduyw2i7bv6f Linux: https://ibm.box.com/s/d0yg8m6ee4g17avl8g4ku2my0y96rzjj

Let me know if these work for Heptio users.

@ayj I have a theory. The documentation at https://github.com/heptio/authenticator says the Heptio authenticator requires a 1.10+ kubectl binary to work. Our Gopkg.toml specifies kubernetes-1.9.1 with a TODO to go to 1.10. I will try using new K8s libraries and see if that fixes things.

I don’t have an AWS account but I was able to reproduce. Here are instructions to reproduce without AWS on Mac:

go get -u -v github.com/heptio/authenticator/cmd/heptio-authenticator-aws
kubectl create sa alice
secret=$(kubectl get sa alice -o json | jq -r .secrets[].name)
user_token=$(kubectl get secret $secret -o json | jq -r '.data["token"]' | base64 -D)
kubectl config set-credentials alice-staging --token=$user_token
kubectl config set-context alice-staging --cluster=<YOUR REAL CLUSTER NAME> --user=alice-staging --namespace=alice
kubectl config use-context alice-staging

Edit your Kube config file removing the alice token and adding

    exec:
      apiVersion: client.authentication.k8s.io/v1alpha1
      command: heptio-authenticator-aws
      args:
        - "token"
        - "-i"
        - "<anything / doesn't matter>"

After doing that, istioctl --context alice-staging --log_output_level debug get routerules fails with 403 Forbidden but kubectl -v 9 --context alice-staging get routerules fails with 401 Unauthorized

If you do rm $GOPATH/bin/heptio-authenticator-aws then kubectl with -v 9 gives “exec: “heptio-authenticator-aws”: executable file not found in $PATH”

I’m still facing the issue @ayj. This is my version:

istioctl version
Version: 0.8.0
GitRevision: 6f9f420f0c7119ff4fa6a1966a6f6d89b1b4db84
User: root@48d5ddfd72da
Hub: docker.io/istio
GolangVersion: go1.10.1
BuildStatus: Clean

@StevenACoffman Sure. I can’t make our org repo public so I just tarred it up and put it here No guarantees it will work for you though. 😄 We use aws-adfs so I can’t say if it works with any other aws CLI auth since I’m still new to AWS and K8s and Heptio and Istio and apparently all the new fun stuff.

I’m just hoping they (istio? heptio?) can fix this so this kludgy, ugly wrapper can go away. I did test it on my Mac and an Ubuntu VM (the original istioctl commands for Mac and Linux are included in the tarball) and both work if you have heptio, aws cli (with EKS module), and your kubeconfig file setup properly and in the right context.

istioctl_wrapper.tar.gz

mtlsExcludedServices was removed on June 11 from the Istio configuration. The binary I posted and the version of istioctl that will ship with the next release of Istio will produce this error when used against Istio builds from early June.

https://github.com/istio/api/commit/fe829db48cc2d7a797f3659d6a93450bf43ff743#diff-e7424aa96af1917e2a00aea6798dc071

Yep that was it.

Sounds good @subfuzion. I threw up a quick medium post on my resolution. It would be awesome to read yours once it’s done!

@sprutner I’ve been using 0.8, but the version curl’d from the git.io/getLatestIstio link. This is indeed good news and if it works, you have both just made my day. I’ll be able to release my blog post on running Istio on EKS if all goes well.

@subfuzion Correct. 0.8.0 came out about two week ago so I am just assuming that there was an older version binary in the tarball posted by @jaredeis.

[UPDATED]

@subfuzion I did git clone git@github.com:istio/istio.git, cd istio

Then you need my PR https://github.com/istio/istio/pull/6338 . Git won’t generate CLI instructions for it, but I think it is git cherry-pick 793824964cbeb8edb27e6bd13c6ed27e7bab9b65 531194705bea78e01e575ac98eaa89d69615d64b.

Then make istioctl

I second the issue. For me, the easiest was to add the loopback in the kubecfg file. (thanks to @jpancoast-kenzan for getting that written out above)

Key file additions are:

clusters:
- <your existing first cluster statement>
- cluster:
    server: http://127.0.0.1:8001
  name: istio
contexts:
- <your original context>
- context:
    cluster: istio
  name: istio

Then, run: kubectl config use-context istio

And to switch back kubectl config use-context aws or whatever the name of your original cluster is.