helm: `helm upgrade --install` doesn't actually perform install if there are duplicates
If you have the same resources (ie secrets) in a chart, (or more likely, multiple subcharts create the same secret) helm upgrade --install fails with
Error: release releaseName failed: secrets "secret-name" already exists
To reproduce:
helm create test- add secrets file to
test/templates - make a copy of the secrets file in the same path
helm upgrade --install releaseName test/
Work-around:
- run the same cmd again:
helm upgrade --install releaseName test/
It seems that after the namespace exists, upgrade allows for the duplicate resources. It switches from creating 3 resource(s) to checking 3 resources for changes.
This workaround defeats the whole purpose of having an --install option to upgrade. But it’s worth noting that this error also appears when running helm install test/ --name releaseName
Logs in the tiller:
[tiller] 2017/09/18 15:28:13 getting history for release t2
[storage] 2017/09/18 15:28:13 getting release history for "t2"
[tiller] 2017/09/18 15:28:13 preparing install for t2
[storage] 2017/09/18 15:28:13 getting release history for "t2"
[tiller] 2017/09/18 15:28:13 rendering test2 chart using values
2017/09/18 15:28:13 info: manifest "test2/templates/ingress.yaml" is empty. Skipping.
[tiller] 2017/09/18 15:28:13 performing install for t2
[tiller] 2017/09/18 15:28:13 executing 0 pre-install hooks for t2
[tiller] 2017/09/18 15:28:13 hooks complete for pre-install t2
[storage] 2017/09/18 15:28:13 getting release history for "t2"
[storage] 2017/09/18 15:28:13 creating release "t2.v1"
[kube] 2017/09/18 15:28:13 building resources from manifest
[kube] 2017/09/18 15:28:13 creating 3 resource(s)
[tiller] 2017/09/18 15:28:13 warning: Release "t2" failed: secrets "secret-name" already exists
[storage] 2017/09/18 15:28:13 updating release "t2.v1"
[tiller] 2017/09/18 15:28:13 failed install perform step: release t2 failed: secrets "secret-name" already exists
and if you run the same command again (ie: workaround):
[tiller] 2017/09/18 15:23:27 getting history for release t2
[storage] 2017/09/18 15:23:27 getting release history for "t2"
[tiller] 2017/09/18 15:23:28 preparing update for t2
[storage] 2017/09/18 15:23:28 getting last revision of "t2"
[storage] 2017/09/18 15:23:28 getting release history for "t2"
[tiller] 2017/09/18 15:23:28 rendering test2 chart using values
2017/09/18 15:23:28 info: manifest "test2/templates/ingress.yaml" is empty. Skipping.
[tiller] 2017/09/18 15:23:28 creating updated release for t2
[storage] 2017/09/18 15:23:28 creating release "t2.v2"
[tiller] 2017/09/18 15:23:28 performing update for t2
[tiller] 2017/09/18 15:23:28 executing 0 pre-upgrade hooks for t2
[tiller] 2017/09/18 15:23:28 hooks complete for pre-upgrade t2
[kube] 2017/09/18 15:23:28 building resources from updated manifest
[kube] 2017/09/18 15:23:28 checking 3 resources for changes
[kube] 2017/09/18 15:23:28 Looks like there are no changes for Secret "secret-name"
[kube] 2017/09/18 15:23:28 Looks like there are no changes for Secret "secret-name"
[kube] 2017/09/18 15:23:28 Looks like there are no changes for Secret "secret-name"
[tiller] 2017/09/18 15:23:28 executing 0 post-upgrade hooks for t2
[tiller] 2017/09/18 15:23:28 hooks complete for post-upgrade t2
[storage] 2017/09/18 15:23:28 updating release "t2.v1"
[tiller] 2017/09/18 15:23:28 updating status for updated release for t2
[storage] 2017/09/18 15:23:28 updating release "t2.v2"
[storage] 2017/09/18 15:23:29 getting last revision of "t2"
[storage] 2017/09/18 15:23:29 getting release history for "t2"
[kube] 2017/09/18 15:23:29 Doing get for Secret: "secret-name"
[kube] 2017/09/18 15:23:29 Doing get for Secret: "secret-name"
[kube] 2017/09/18 15:23:29 Doing get for Secret: "secret-name"
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Reactions: 24
- Comments: 35 (5 by maintainers)
remove –wait
I went through so many troubleshooting efforts related to UPGRADE FAILS or release already exists on Azure Pipelines, that all it took was to remove/uncheck wait
Azure Pipelines defaults to 2.9.1–works there. Also works on 2.13.0
No force, no wait, no problems.
Also didn’t need to delete --purge.
Amazing, really.
same issue exsits with version 2.9.1
Still exists and v3, bunch of secrets in my case.
--forcedoes not helpI think
--forcemakes sense in this case, but if not, we could also introduce a new parameter such as--ignore-existing-secrets.The problem of re-running is that it makes very less scriptable, and time wasting.
For what is worth still happens on
v2.12.3(tiller)/v2.12.2(client).I found this really surprising/annoying. Wasn’t kubernetes declarative? Why do I care if a configmap/resource is there or not (usually)? Kubernetes will figure things up and eventually have make the state I requested happen.
It’s not clear to me at this stage what Helm is trying to do. I thought Helm was pretty much “just” rendering the templates and run a
kubectl applyof the rendered YAML files? (and of course keeping track of the releases/revisions).If anyone understand this better I’d love to understand the issue a bit better 🤷♂️
same issue here with version 2.9.1
same issue here with version 2.9.1
helm upgrace -i --force xxworks well with me, client versionv2.12.2, server versionv2.12.0update: By the way, it needs executing
helm upgrade -i --force xxtwice to work well, it gets failed for the first time, and I don’t know why. 😦Still exists, client/server v2.14.2, even with upgrade --install --force, can’t make it work. In my case it’s a configmap.
still happens on
version.BuildInfo{Version:"v3.10.3", GitCommit:"835b7334cfe2e5e27870ab3ed4135f136eecc704", GitTreeState:"clean", GoVersion:"go1.19.4"}same issue exists in v2.10.0. Only way is to
helm del --purgewhich deletes the existing secret and it’s bad. Other way would be to use versioned names likeHas anyone found a solution to this yet? I’m having the same problem.
Can you try with v2.8.2 with
helm upgrade --install --force?same issue here
I found a solution for my case. I have an umbrella chart and subcharts that all create a regcred to pull docker images. The thing : use a .helmignore file. I just set
charts/*/regcred.yamlin a.helmignorefile in umbrella chart root, then helm ignores these duplicate files when installing or upgrading. Hope it helps !@cs94njw run
helm versionwill tell you what version of helm (Client) and tiller (Server) you’re using.@bacongobbler thanks for your replies:
Still not sure how to work around my problem where I’m trying to upgrade a release which has a dependency (on
stable/redisin my case) without getting weird errors. Bear in mind the existing resource was created by Helm, I just upgraded the dependency and now it’s not happy.Bear in mind that in a test environment I managed to have the upgrade work, by some combination of
--force/--no-hooks/--wait/etc…but I can’t make it work again.Any ideas on how to work around helm refusing to apply these resources? Surely the
--forceflag should tell him to do it anyway?This was an intentional design decision. We chose the defensive decision: If something other than Helm creates a resource, Helm will not alter it. A name collision is treated as an error condition. In our view, it would be an error for Helm to assume that it can “adopt” an existing resource and treat it as part of a release.
There is a hack : https://medium.com/@valercara/helm-stuck-in-pending-update-and-how-to-fix-it-13c4b2eaf9f7
/remove-lifecycle stale
I think we are having a similar problem, but with
ServiceAccount. We do have a single copy of a ServiceAccount resource in our chart. When we dohelm upgrade --installfor the first time, we are getting the following error:Subsequent runs of the same command succeed.