spinnaker: spin-clouddriver gcloud command not found

Issue Summary:

I deploy spinnaker, but spin-clouddriver is Readiness probe failed.
I checked the log with kubectl logs pod-name-n spinnaker and an internal server error has occurred.

Cloud Provider(s):

Kubernetes

Environment:

Halyard:
on GCE(ubuntu-1604-xenial-v20180522)

$ hal -v
1.2.0-20180509113540

spinnaker: on GKE(1.10.4-gke.0)

$ hal config version edit --version 1.7.6
$ hal config provider kubernetes account add spinnaker \
    --provider-version v2 \
    --context $(kubectl config current-context)

Feature Area:

clouddriver

Description:

The status of pod is below and the status of spin-clouddriver is not normal.

$ kubectl get pod -n spinnaker
NAME                                READY     STATUS    RESTARTS   AGE
spin-clouddriver-8585c8cc88-pnsqb   0/1       Running   0          23m
spin-deck-64c77858bc-j2bjf          1/1       Running   0          23m
spin-echo-5b778876dd-5kfzx          1/1       Running   0          23m
spin-front50-65c677f697-fv5xx       1/1       Running   0          23m
spin-gate-7655445d46-9jlb7          1/1       Running   0          23m
spin-orca-b88b864d6-c479t           1/1       Running   0          22m
spin-redis-6d49c9c5b9-x4vkw         1/1       Running   0          23m
spin-rosco-7d8cc5bb6f-5gwj6         1/1       Running   0          22m

Steps to Reproduce:

  1. install halyard on GCE(ubuntu-1604-xenial-v20180522)
  2. hal config provider kubernetes enable
  3.  --provider-version v2 \
     --context $(kubectl config current-context)
    
  4. hal config features edit --artifacts true
  5. hal config deploy edit --type distributed --account-name spinnaker
  6. storage edit
     --json-path $SERVICE_ACCOUNT_DEST \
     --bucket $BUCKET_NAME
    
  7. hal config version edit --version 1.7.6
  8. hal deploy apply

Additional Details:

2018-06-13 02:03:48.418 ERROR 1 --- [0.0-7002-exec-3] c.n.s.k.w.e.GenericExceptionHandlers     : Internal Server Error

java.lang.RuntimeException: com.netflix.spinnaker.clouddriver.kubernetes.v2.op.job.KubectlJobExecutor$KubectlException: Failed to read [namespace] from : Unable to connect to the server: error executing access token command "/usr/lib/google-cloud-sdk/bin/gcloud config config-helper --format=json": err=fork/exec /usr/lib/google-cloud-sdk/bin/gcloud: no such file or directory output= stderr=

	at com.netflix.spinnaker.clouddriver.kubernetes.v2.security.KubernetesV2Credentials.getDeclaredNamespaces(KubernetesV2Credentials.java:311) ~[clouddriver-kubernetes-2.22.0-SNAPSHOT.jar:2.22.0-SNAPSHOT]
	at com.netflix.spinnaker.clouddriver.kubernetes.security.KubernetesCredentials$getDeclaredNamespaces.call(Unknown Source) ~[na:na]
	at com.netflix.spinnaker.clouddriver.kubernetes.health.KubernetesHealthIndicator.checkHealth(KubernetesHealthIndicator.groovy:65) ~[clouddriver-kubernetes-2.22.0-SNAPSHOT.jar:2.22.0-SNAPSHOT]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_151]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_151]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_151]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_151]
	at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65) ~[spring-context-4.3.11.RELEASE.jar:4.3.11.RELEASE]
	at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-4.3.11.RELEASE.jar:4.3.11.RELEASE]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_151]
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[na:1.8.0_151]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[na:1.8.0_151]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[na:1.8.0_151]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_151]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_151]
	at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_151]
Caused by: com.netflix.spinnaker.clouddriver.kubernetes.v2.op.job.KubectlJobExecutor$KubectlException: Failed to read [namespace] from : Unable to connect to the server: error executing access token command "/usr/lib/google-cloud-sdk/bin/gcloud config config-helper --format=json": err=fork/exec /usr/lib/google-cloud-sdk/bin/gcloud: no such file or directory output= stderr=

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 15 (4 by maintainers)

Most upvoted comments

gcloud is installed to a different path than /usr/lib/google-cloud-sdk/bin/gcloud on the clouddriver container. Just change the path from /usr/lib/google-cloud-sdk/bin/gcloud to gcloud in your ~/.kube/config and you’ll be good to go.

This is because gcloud configured in your kubeconfig doesn’t exist in the pod. If you either change the gcloud path, or use a service account, Spinnaker will be able to read your cluster.