helm: `helm upgrade --install` doesn't actually perform install if there are duplicates

If you have the same resources (ie secrets) in a chart, (or more likely, multiple subcharts create the same secret) helm upgrade --install fails with

Error: release releaseName failed: secrets "secret-name" already exists


To reproduce:

  • helm create test
  • add secrets file to test/templates
  • make a copy of the secrets file in the same path
  • helm upgrade --install releaseName test/

Work-around:

  • run the same cmd again: helm upgrade --install releaseName test/

It seems that after the namespace exists, upgrade allows for the duplicate resources. It switches from creating 3 resource(s) to checking 3 resources for changes.

This workaround defeats the whole purpose of having an --install option to upgrade. But it’s worth noting that this error also appears when running helm install test/ --name releaseName


Logs in the tiller:

[tiller] 2017/09/18 15:28:13 getting history for release t2
[storage] 2017/09/18 15:28:13 getting release history for "t2"
[tiller] 2017/09/18 15:28:13 preparing install for t2
[storage] 2017/09/18 15:28:13 getting release history for "t2"
[tiller] 2017/09/18 15:28:13 rendering test2 chart using values
2017/09/18 15:28:13 info: manifest "test2/templates/ingress.yaml" is empty. Skipping.
[tiller] 2017/09/18 15:28:13 performing install for t2
[tiller] 2017/09/18 15:28:13 executing 0 pre-install hooks for t2
[tiller] 2017/09/18 15:28:13 hooks complete for pre-install t2
[storage] 2017/09/18 15:28:13 getting release history for "t2"
[storage] 2017/09/18 15:28:13 creating release "t2.v1"
[kube] 2017/09/18 15:28:13 building resources from manifest
[kube] 2017/09/18 15:28:13 creating 3 resource(s)
[tiller] 2017/09/18 15:28:13 warning: Release "t2" failed: secrets "secret-name" already exists
[storage] 2017/09/18 15:28:13 updating release "t2.v1"
[tiller] 2017/09/18 15:28:13 failed install perform step: release t2 failed: secrets "secret-name" already exists

and if you run the same command again (ie: workaround):

[tiller] 2017/09/18 15:23:27 getting history for release t2
[storage] 2017/09/18 15:23:27 getting release history for "t2"
[tiller] 2017/09/18 15:23:28 preparing update for t2
[storage] 2017/09/18 15:23:28 getting last revision of "t2"
[storage] 2017/09/18 15:23:28 getting release history for "t2"
[tiller] 2017/09/18 15:23:28 rendering test2 chart using values
2017/09/18 15:23:28 info: manifest "test2/templates/ingress.yaml" is empty. Skipping.
[tiller] 2017/09/18 15:23:28 creating updated release for t2
[storage] 2017/09/18 15:23:28 creating release "t2.v2"
[tiller] 2017/09/18 15:23:28 performing update for t2
[tiller] 2017/09/18 15:23:28 executing 0 pre-upgrade hooks for t2
[tiller] 2017/09/18 15:23:28 hooks complete for pre-upgrade t2
[kube] 2017/09/18 15:23:28 building resources from updated manifest
[kube] 2017/09/18 15:23:28 checking 3 resources for changes
[kube] 2017/09/18 15:23:28 Looks like there are no changes for Secret "secret-name"
[kube] 2017/09/18 15:23:28 Looks like there are no changes for Secret "secret-name"
[kube] 2017/09/18 15:23:28 Looks like there are no changes for Secret "secret-name"
[tiller] 2017/09/18 15:23:28 executing 0 post-upgrade hooks for t2
[tiller] 2017/09/18 15:23:28 hooks complete for post-upgrade t2
[storage] 2017/09/18 15:23:28 updating release "t2.v1"
[tiller] 2017/09/18 15:23:28 updating status for updated release for t2
[storage] 2017/09/18 15:23:28 updating release "t2.v2"
[storage] 2017/09/18 15:23:29 getting last revision of "t2"
[storage] 2017/09/18 15:23:29 getting release history for "t2"
[kube] 2017/09/18 15:23:29 Doing get for Secret: "secret-name"
[kube] 2017/09/18 15:23:29 Doing get for Secret: "secret-name"
[kube] 2017/09/18 15:23:29 Doing get for Secret: "secret-name"

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Reactions: 24
  • Comments: 35 (5 by maintainers)

Most upvoted comments

remove –wait

I went through so many troubleshooting efforts related to UPGRADE FAILS or release already exists on Azure Pipelines, that all it took was to remove/uncheck wait

Azure Pipelines defaults to 2.9.1–works there. Also works on 2.13.0

No force, no wait, no problems.

Also didn’t need to delete --purge.

Amazing, really.

same issue exsits with version 2.9.1

Still exists and v3, bunch of secrets in my case. --force does not help

I think --force makes sense in this case, but if not, we could also introduce a new parameter such as --ignore-existing-secrets.

The problem of re-running is that it makes very less scriptable, and time wasting.

For what is worth still happens on v2.12.3 (tiller)/v2.12.2 (client).

I found this really surprising/annoying. Wasn’t kubernetes declarative? Why do I care if a configmap/resource is there or not (usually)? Kubernetes will figure things up and eventually have make the state I requested happen.

It’s not clear to me at this stage what Helm is trying to do. I thought Helm was pretty much “just” rendering the templates and run a kubectl apply of the rendered YAML files? (and of course keeping track of the releases/revisions).

If anyone understand this better I’d love to understand the issue a bit better 🤷‍♂️

same issue here with version 2.9.1

same issue here with version 2.9.1

helm upgrace -i --force xx works well with me, client version v2.12.2, server version v2.12.0

update: By the way, it needs executing helm upgrade -i --force xx twice to work well, it gets failed for the first time, and I don’t know why. 😦

Still exists, client/server v2.14.2, even with upgrade --install --force, can’t make it work. In my case it’s a configmap.

still happens on version.BuildInfo{Version:"v3.10.3", GitCommit:"835b7334cfe2e5e27870ab3ed4135f136eecc704", GitTreeState:"clean", GoVersion:"go1.19.4"}

same issue exists in v2.10.0. Only way is to helm del --purge which deletes the existing secret and it’s bad. Other way would be to use versioned names like

..
name: "secret-name-{{ .Chart.Version }}"
..

Has anyone found a solution to this yet? I’m having the same problem.

Can you try with v2.8.2 with helm upgrade --install --force?

same issue here

I found a solution for my case. I have an umbrella chart and subcharts that all create a regcred to pull docker images. The thing : use a .helmignore file. I just set charts/*/regcred.yaml in a .helmignore file in umbrella chart root, then helm ignores these duplicate files when installing or upgrading. Hope it helps !

@cs94njw run helm version will tell you what version of helm (Client) and tiller (Server) you’re using.

@bacongobbler thanks for your replies:

  1. I see what you mean about the design decision
  2. I didn’t get that the OP had a problem where the same resource was created twice.

Still not sure how to work around my problem where I’m trying to upgrade a release which has a dependency (on stable/redis in my case) without getting weird errors. Bear in mind the existing resource was created by Helm, I just upgraded the dependency and now it’s not happy.

Bear in mind that in a test environment I managed to have the upgrade work, by some combination of --force/--no-hooks/--wait/etc…but I can’t make it work again.

Any ideas on how to work around helm refusing to apply these resources? Surely the --force flag should tell him to do it anyway?

This was an intentional design decision. We chose the defensive decision: If something other than Helm creates a resource, Helm will not alter it. A name collision is treated as an error condition. In our view, it would be an error for Helm to assume that it can “adopt” an existing resource and treat it as part of a release.

/remove-lifecycle stale

I think we are having a similar problem, but with ServiceAccount. We do have a single copy of a ServiceAccount resource in our chart. When we do helm upgrade --install for the first time, we are getting the following error:

[debug] Created tunnel using local port: '36831'

[debug] SERVER: "localhost:36831"

[debug] Fetched jiff/environment to /root/.helm/cache/archive/environment-0.0.1.tgz

Release "environment" does not exist. Installing it now.
[debug] CHART PATH: /root/.helm/cache/archive/environment-0.0.1.tgz

2017/10/11 16:17:22 warning: skipped value for secrets: Not a table.
2017/10/11 16:17:22 warning: skipped value for secrets: Not a table.
Error: release environment failed: serviceaccounts "bastion" already exists
[main] 2017/10/11 16:16:00 Starting Tiller v2.6.1 (tls=false)
[main] 2017/10/11 16:16:00 GRPC listening on :44134
[main] 2017/10/11 16:16:00 Probes listening on :44135
[main] 2017/10/11 16:16:00 Storage driver is ConfigMap
[storage] 2017/10/11 16:16:09 listing all releases with filter
[storage] 2017/10/11 16:16:13 listing all releases with filter
[tiller] 2017/10/11 16:17:22 getting history for release environment
[storage] 2017/10/11 16:17:22 getting release history for "environment"
[tiller] 2017/10/11 16:17:24 preparing install for environment
[storage] 2017/10/11 16:17:24 getting release history for "environment"
2017/10/11 16:17:24 warning: destination for secrets is a table. Ignoring non-table value []
2017/10/11 16:17:24 warning: skipped value for secrets: Not a table.
2017/10/11 16:17:24 warning: skipped value for secrets: Not a table.
[tiller] 2017/10/11 16:17:24 rendering environment chart using values
[tiller] 2017/10/11 16:17:26 performing install for environment
[tiller] 2017/10/11 16:17:26 executing 0 pre-install hooks for environment
[tiller] 2017/10/11 16:17:26 hooks complete for pre-install environment
[storage] 2017/10/11 16:17:26 getting release history for "environment"
[storage] 2017/10/11 16:17:26 creating release "environment.v1"
[kube] 2017/10/11 16:17:26 building resources from manifest
[kube] 2017/10/11 16:17:28 creating 57 resource(s)
[tiller] 2017/10/11 16:17:28 warning: Release "environment" failed: serviceaccounts "bastion" already exists
[storage] 2017/10/11 16:17:28 updating release "environment.v1"
[tiller] 2017/10/11 16:17:28 failed install perform step: release environment failed: serviceaccounts "bastion" already exists
[storage] 2017/10/11 16:48:43 listing all releases with filter
[storage] 2017/10/11 16:53:53 listing all releases with filter

Subsequent runs of the same command succeed.