helm: v3.12.0 Failed deployments

Output of helm version: version.BuildInfo{Version:“v3.12.0”, GitCommit:“c9f554d75773799f72ceef38c51210f1842a1dea”, GitTreeState:“clean”, GoVersion:“go1.20.3”}

Cloud Provider/Platform (AKS, GKE, Minikube etc.): EKS.

In our pipelines , we download the latest version of helm3 and use it for deployments. Starting with three hours ago, our deployments started failing with the following error.

Error: UPGRADE FAILED: error validating "": error validating data: the server responded with the status code 426 but did not return more information

I reverted to the last working version of helm in our pipelines ( v3.11.3 ) and our pipelines are working now.

curl -L https://git.io/get_helm.sh | bash -s -- --version v3.11.3

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Reactions: 10
  • Comments: 26 (10 by maintainers)

Commits related to this issue

Most upvoted comments

I have the same issue, It doesn’t seem to have anything to do with any helm repo. I’m using it to deploy a local (on-disk) helm chart. Seems to have something to do with the connection to the cluster.

EDIT I fixed it by downgrading to previous helm version.

For other people, like me that is using Gitlab CI.

- curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
- chmod +x get_helm.sh && ./get_helm.sh --version v3.11.3

Note: make sure bash and openssl is installed first.

Kubernetes patch release scheduled for 2023-05-17

@joejulian added debug variable to my CI and upgraded to 3.12 again. Debug log is huge, and i am not really into go, but this looks intresting to me:

[000:11] h2_bundle.go:9019: http2: Transport encoding header ":authority" = "my-gitlab-kas.example.org"
[000:11] h2_bundle.go:9019: http2: Transport encoding header ":method" = "GET"
[000:11] h2_bundle.go:9019: http2: Transport encoding header ":path" = "/k8s-proxy/openapi/v3?timeout=32s"
[000:11] h2_bundle.go:9019: http2: Transport encoding header ":scheme" = "https"
[000:11] h2_bundle.go:9019: http2: Transport encoding header "authorization" = "Bearer ci:3:[MASKED]"
[000:11] h2_bundle.go:9019: http2: Transport encoding header "accept" = "application/json, */*"
[000:11] h2_bundle.go:9019: http2: Transport encoding header "user-agent" = "Helm/3.12.0"
[000:11] h2_bundle.go:9019: http2: Transport encoding header "accept-encoding" = "gzip"
[000:11] h2_bundle.go:9968: http2: Transport received HEADERS flags=END_HEADERS stream=121 len=88
[000:11] h2_bundle.go:9968: http2: Transport received DATA stream=121 len=7029 data="XXX" (6773 bytes omitted)
[000:11] h2_bundle.go:9968: http2: Transport received DATA flags=END_STREAM stream=121 len=10 data="XXX"
[000:11] h2_bundle.go:9019: http2: Transport encoding header ":authority" = "my-gitlab-kas.example.org"
[000:11] h2_bundle.go:9019: http2: Transport encoding header ":method" = "GET"
[000:11] h2_bundle.go:9019: http2: Transport encoding header ":path" = "/openapi/v3/api/v1?hash=some-hash&timeout=32s"
[000:11] h2_bundle.go:9019: http2: Transport encoding header ":scheme" = "https"
[000:11] h2_bundle.go:9019: http2: Transport encoding header "authorization" = "Bearer ci:3:[MASKED]"
[000:11] h2_bundle.go:9019: http2: Transport encoding header "accept" = "application/json"
[000:11] h2_bundle.go:9019: http2: Transport encoding header "user-agent" = "Helm/3.12.0"
[000:11] h2_bundle.go:9019: http2: Transport encoding header "accept-encoding" = "gzip"
[000:11] h2_bundle.go:9968: http2: Transport received HEADERS flags=END_HEADERS stream=123 len=41
[000:11] h2_bundle.go:9968: http2: Transport received DATA stream=123 len=76 data="WebSocket protocol violation: Connection header \"\" does not contain Upgrade\n"
[000:11] h2_bundle.go:9968: http2: Transport received DATA flags=END_STREAM stream=123 len=0 data=""
[000:11] Error: unable to build kubernetes objects from release manifest: error validating "": error validating data: the server responded with the status code 426 but did not return more information
[000:11] helm.go:84: [debug] error validating "": error validating data: the server responded with the status code 426 but did not return more information
[000:11] helm.sh/helm/v3/pkg/kube.scrubValidationError
[000:11] helm.sh/helm/v3/pkg/kube/client.go:809

In particular this one:

data="WebSocket protocol violation: Connection header \"\" does not contain Upgrade\n"

As said, there is KAS running in my case: https://docs.gitlab.com/ee/administration/clusters/kas.html

Throwing the same command directly against my cluster works fine.

I had the issue with 3.12.0, it seems to work fine with 3.12.1

@joejulian #12096 does the job for me.


    curl -Ss https://dl.google.com/go/go1.20.4.linux-amd64.tar.gz | tar -C /usr/local -xzf-
    mv /usr/local/go/bin/go /usr/local/bin/
    
    git clone https://github.com/joejulian/helm.git
    cd helm

    git checkout client-go
    make
    cp bin/helm /usr/local/bin/

    helm version
version.BuildInfo{Version:"v3.11+unreleased", GitCommit:"54ca3a82c289907a691539f8e25c2d92ae068ba3", GitTreeState:"clean", GoVersion:"go1.20.4"}

What version of Kubernetes are y’all using?

I’ve used it both 1.24 and 1.25, same issue on both.