helm: Cannot upgrade a release with Job

The existing job need to be deleted, because the template section in the job is immutable or not updatable. so you have 2 following options.

1. Always create a new job with a unique name, so it leaves the old jobs and creates a new one - every time if you include the version of image would be sensible.
2. Automatic cleaning of jobs (more info check here) - The job property ttlSecondsAfterFinished allows helps to delete the job automatically after a sepecified period. for eg:

    apiVersion: batch/v1
    kind: Job
    metadata:
      name: pi-with-ttl
    spec:
      ttlSecondsAfterFinished: 100
      template:
        spec:
          containers:
          - name: pi
            image: perl
            command: ["perl",  "-Mbignum=bpi", "-wle", "print bpi(2000)"]
          restartPolicy: Never

If helm bills itself as a “package manager for Kubernetes”, shouldn’t it be a bit more helpful in cases like that? Whenever I run into this situation, I have to do uninstall of the release, then reinstall the chart anew. Seems like helm could do that for me with a flag on helm install…

Hi everyone, In addition to all of the above responses you can check in the side of helm-hooks : https://helm.sh/docs/topics/charts_hooks/ I solved my issue thanks to @distnie and its second point which suggests to use ttlSecondsAfterFinished. The purpose of my job was to fill the database with seeds and in addition I have covered all potential issues by only allow job running with post-install hook

As mentioned earlier,

However, Helm doesn’t know which values those are, because they are not exposed by the schema. So Helm sends the manifest to Kubernetes, which responds back with a list of fields that are immutable.

If you know of a way to improve the situation, please feel free to submit a patch.

The situation you are describing is orthogonal to the issue described above. I’d suggest taking a look at the issue queue for a larger write up on that issue; there are other PRs looking to alleviate that issue but it’s not the same as what’s being discussed here.

To anyone that came looking for similar use cases on upgrading immutable resources, the pre-deploy section in the docs may address it. From https://fluxcd.io/docs/use-cases/running-jobs/

Setting spec.force to true will make Flux recreate the Job when any immutable field is changed, forcing the Job to run every time the container image tag changes.

The issue is that certain things are considered by Kubernetes to be immutable. You cannot upgrade certain values on certain types of Kubernetes objects. However, Helm doesn’t know which values those are, because they are not exposed by the schema. So Helm sends the manifest to Kubernetes, which responds back with a list of fields that are immutable. The deployment fails because the objects cannot be upgraded.

The solution is for your charts to not attempt to modify immutable fields. You can find more information in the Kubernetes documentation https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/ (though you may have to go to the Kubernetes YAML API docs to find out exactly which fields are immutable)

FWIW, many of the Kubernetes controllers will not let you change certain things in the spec template metadata because Kubernetes uses that info to find things you have deployed.