helm: Cannot perform helm init behind proxy
Output of helm version
: v2.10.0-rc.1 (https://github.com/kubernetes/helm/releases/tag/v2.10.0-rc.1)
Output of kubectl version
: v1.11.0
Cloud Provider/Platform (AKS, GKE, Minikube etc.): bare metal
Description: When checking the latest RC version (https://github.com/kubernetes/helm/releases/tag/v2.10.0-rc.1) I am unable to perform “helm init” behind proxy:
$ ./helm init --node-selectors "beta.kubernetes.io/os"=linux
Creating /home/ubuntu/.helm
Creating /home/ubuntu/.helm/repository
Creating /home/ubuntu/.helm/repository/cache
Creating /home/ubuntu/.helm/repository/local
Creating /home/ubuntu/.helm/plugins
Creating /home/ubuntu/.helm/starters
Creating /home/ubuntu/.helm/cache/archive
Creating /home/ubuntu/.helm/repository/repositories.yaml
Adding stable repo with URL: https://kubernetes-charts.storage.googleapis.com
Error: Looks like "https://kubernetes-charts.storage.googleapis.com" is not a valid chart repository or cannot be reached: Get https://kubernetes-charts.storage.googleapis.com/index.yaml: dial tcp 216.58.214.80:443: connect: connection timed out
My environment is set properly for proxy:
$ export
...
declare -x HTTPS_PROXY="http://some-proxy:1234"
declare -x HTTP_PROXY="http://some-proxy:1234"
...
declare -x http_proxy="http://some-proxy:1234"
declare -x https_proxy="http://some-proxy:1234"
And downgrading to latest stable release v2.9.1 (https://github.com/kubernetes/helm/releases/tag/v2.9.1) solves the issue:
$ ./helm init --node-selectors "beta.kubernetes.io/os"=linux
Creating /home/ubuntu/.helm/repository/repositories.yaml
Adding stable repo with URL: https://kubernetes-charts.storage.googleapis.com
Adding local repo with URL: http://127.0.0.1:8879/charts
$HELM_HOME has been configured at /home/ubuntu/.helm.
Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.
Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.
For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation
Happy Helming!
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Reactions: 11
- Comments: 52 (15 by maintainers)
Commits related to this issue
- Set proxy for all connections, fixes #4326 (#4579) Signed-off-by: Christian Koeberl <christian.koeberl@gmail.com> Signed-off-by: Sebastien Plisson <sebastien.plisson@gmail.com> — committed to splisson/helm by derkoe 6 years ago
- Set proxy for all connections, fixes #4326 (#4579) Signed-off-by: Christian Koeberl <christian.koeberl@gmail.com> Signed-off-by: Sebastien Plisson <sebastien.plisson@gmail.com> — committed to splisson/helm by derkoe 6 years ago
- Set proxy for all connections, fixes #4326 (#4579) Signed-off-by: Christian Koeberl <christian.koeberl@gmail.com> — committed to splisson/helm by derkoe 6 years ago
- Set proxy for all connections, fixes #4326 (#4579) Signed-off-by: Christian Koeberl <christian.koeberl@gmail.com> — committed to splisson/helm by derkoe 6 years ago
- Update Helm packages to 2.11 This is to get past the problem described in https://github.com/helm/helm/issues/4326, which prevents people having a proxy in front of external Helm repos. — committed to fluxcd/flux by squaremo 5 years ago
Problem still exists for 2.10.0:
No issues for 2.9.1:
Create PR #4579 which should fix this.
@tardich I can tell 2.9.1 does work. I suggest you stick to that version for your tests. Are you able to curl or
wget https://kubernetes-charts.storage.googleapis.com/index.yaml
? If so, try:or even try to copy manually the index.yaml (if you wget worked)
For reference, with 2.9.1 my Dockerfile contained:
RUN helm init --client-only --wait
With 2.10.0 the workaround is:It’s a good question. Generally speaking, once we go forward with the next minor release we avoid cutting patch releases for prior releases, doubly so if there’s a workaround for the patch. That’s mostly done to reduce workload constraints on our end since we’re a small team maintaining the project. I’ll bring it up with the rest of the team and see what they think about cutting another release.
If anyone would be willing to try the v2.11.0-rc.2 release candidates for the fix, that would also be appreciated (and should not require the workaround). 😃
@marchowarth’s workaround also worked for me on MacOS. Assuming you have openssl installed, add the repo as follows: helm repo add --ca-file=/usr/local/etc/openssl/cert.pem stable https://kubernetes-charts.storage.googleapis.com
I had the same issue with v2.10.0, can confirm it isn’t on the v2.11.0-rc.2.
Perhaps a naive question, but isn’t this a v2.10 bug? Will it be included in the v2.10.1 (bugfixes) release?
I think the reason for this is 3eaa1bfd3b040cb644c7974d32047dc70037cf95. This removed the use of http.DefaultClient and created the client with a new transport - this removed the default setting for using a proxy (http.ProxyFromEnvironment).
$ helm init --skip-refresh Creating /home/xxxxxx/.helm/repository/repositories.yaml Adding stable repo with URL: https://kubernetes-charts.storage.googleapis.com Adding local repo with URL: http://127.0.0.1:8879/charts $HELM_HOME has been configured at /home/xxxxxx/.helm.
Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.
Please note: by default, Tiller is deployed with an insecure ‘allow unauthenticated users’ policy. To prevent this, run
helm init
with the --tiller-tls-verify flag. For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation Happy Helming! $ helm list Error: configmaps is forbidden: User “system:serviceaccount:kube-system:default” cannot list configmaps in the namespace “kube-system” $ helm repo add foo https://kubernetes-charts.storage.googleapis.com --debug Error: Looks like “https://kubernetes-charts.storage.googleapis.com” is not a valid chart repository or cannot be reached: Get https://kubernetes-charts.storage.googleapis.com/index.yaml: dial tcp 172.217.24.208:443: connect: connection refusedNotice that our corp policy prevent any direct http connection to the internet, and all web traffic must be directed to the proxy. However it seems that helm repo command doesn’t respect the proxy settings.
@bacongobbler it is likely, that helm simply connected directly with the internet and ignored the proxy settings. In a real corporate environment, this is not possible, and though it fails.
I am not sure on how to reproduce this without the real need for a proxy. But I am happy to try anything you like on my machine with the problematic Helm version on it. I just need directions on which logs to capture…
maybe it would also make sense to scan the diff between the two versions, maybe the error is obvious for somebody familiar with the codebase?