helmfile: Releases from `helmfiles` are ignored when the environment is specified

I’m facing the issue when releases defined in separate .yaml files and referenced via helmfiles: are ignored when I try to install the full stack for a certain environment.

Example:

#environments.yaml
environments:
  dev:
  staging:
  production:

#helmfiles/namespaces.yaml
repositories:
  - name: incubator
    url: https://kubernetes-charts-incubator.storage.googleapis.com/

releases:
  - name: namespaces
    namespace: default
    chart: incubator/raw
    values:
      - resources:
        - apiVersion: v1
          kind: Namespace
          metadata:
            name: redis
          spec:

#helmfile.yaml
bases:
- environments.yaml
---
helmfiles:
  - helmfiles/namespaces.yaml

repositories:
  - name: stable
    url: https://kubernetes-charts.storage.googleapis.com

helmDefaults:
  verify: false
  wait: false
  timeout: 600
  recreatePods: false
  force: true

releases:
  - name: redis
    namespace: redis
    chart: stable/redis
    version: ~9.2.1
    values:
      - usePassword: false
        cluster:
          enabled: false

Then, let’s try to output the list of releases

ip113:helmfile andrewnazarov$ helmfile -e staging --log-level=debug list
processing file "helmfile.yaml" in directory "."
first-pass rendering starting for "helmfile.yaml.part.0": inherited=&{staging map[] map[]}, overrode=<nil>
first-pass uses: &{staging map[] map[]}
first-pass rendering output of "helmfile.yaml.part.0":
 0: bases:
 1: - environments.yaml

error in first-pass rendering: result of "helmfile.yaml.part.0":
 0: bases:
 1: - environments.yaml

first-pass produced: &{staging map[] map[]}
first-pass rendering result of "helmfile.yaml.part.0": {staging map[] map[]}
second-pass rendering result of "helmfile.yaml.part.0":
 0: bases:
 1: - environments.yaml

first-pass rendering starting for "environments.yaml.part.0": inherited=&{staging map[] map[]}, overrode=<nil>
first-pass uses: &{staging map[] map[]}
first-pass rendering output of "environments.yaml.part.0":
 0: environments:
 1:   dev:
 2:   staging:
 3:   production:

first-pass produced: &{staging map[] map[]}
first-pass rendering result of "environments.yaml.part.0": {staging map[] map[]}
vals:
map[]
defaultVals:[]
second-pass rendering result of "environments.yaml.part.0":
 0: environments:
 1:   dev:
 2:   staging:
 3:   production:

merged environment: &{staging map[] map[]}
merged environment: &{staging map[] map[]}
first-pass rendering starting for "helmfile.yaml.part.1": inherited=&{staging map[] map[]}, overrode=<nil>
first-pass uses: &{staging map[] map[]}
first-pass rendering output of "helmfile.yaml.part.1":
 0: helmfiles:
 1:   - helmfiles/namespaces.yaml
 2: 
 3: repositories:
 4:   - name: stable
 5:     url: https://kubernetes-charts.storage.googleapis.com
 6: 
 7: helmDefaults:
 8:   verify: false
 9:   wait: false
10:   timeout: 600
11:   recreatePods: false
12:   force: true
13: 
14: releases:
15:   - name: redis
16:     namespace: redis
17:     chart: stable/redis
18:     version: ~9.2.1
19:     values:
20:       - usePassword: false
21:         cluster:
22:           enabled: false

first-pass produced: &{staging map[] map[]}
first-pass rendering result of "helmfile.yaml.part.1": {staging map[] map[]}
vals:
map[]
defaultVals:[]
second-pass rendering result of "helmfile.yaml.part.1":
 0: helmfiles:
 1:   - helmfiles/namespaces.yaml
 2: 
 3: repositories:
 4:   - name: stable
 5:     url: https://kubernetes-charts.storage.googleapis.com
 6: 
 7: helmDefaults:
 8:   verify: false
 9:   wait: false
10:   timeout: 600
11:   recreatePods: false
12:   force: true
13: 
14: releases:
15:   - name: redis
16:     namespace: redis
17:     chart: stable/redis
18:     version: ~9.2.1
19:     values:
20:       - usePassword: false
21:         cluster:
22:           enabled: false

merged environment: &{staging map[] map[]}
processing file "namespaces.yaml" in directory "helmfiles"
changing working directory to "/Users/andrewnazarov/Repo/iris/test/helmfile/helmfiles"
first-pass rendering starting for "namespaces.yaml.part.0": inherited=&{staging map[] map[]}, overrode=<nil>
first-pass uses: &{staging map[] map[]}
first-pass rendering output of "namespaces.yaml.part.0":
 0: repositories:
 1:   - name: incubator
 2:     url: https://kubernetes-charts-incubator.storage.googleapis.com/
 3: 
 4: releases:
 5:   - name: namespaces
 6:     namespace: default
 7:     chart: incubator/raw
 8:     values:
 9:       - resources:
10:         - apiVersion: v1
11:           kind: Namespace
12:           metadata:
13:             name: redis
14:           spec:

first-pass produced: &{staging map[] map[]}
first-pass rendering result of "namespaces.yaml.part.0": {staging map[] map[]}
vals:
map[]
defaultVals:[]
second-pass rendering result of "namespaces.yaml.part.0":
 0: repositories:
 1:   - name: incubator
 2:     url: https://kubernetes-charts-incubator.storage.googleapis.com/
 3: 
 4: releases:
 5:   - name: namespaces
 6:     namespace: default
 7:     chart: incubator/raw
 8:     values:
 9:       - resources:
10:         - apiVersion: v1
11:           kind: Namespace
12:           metadata:
13:             name: redis
14:           spec:

changing working directory back to "/Users/andrewnazarov/Repo/iris/test/helmfile"
NAME 	NAMESPACE	INSTALLED	LABELS
redis	redis    	true

Without environment specified everything looks ok:

ip113:helmfile andrewnazarov$ helmfile --log-level=debug list
processing file "helmfile.yaml" in directory "."
first-pass rendering starting for "helmfile.yaml.part.0": inherited=&{default map[] map[]}, overrode=<nil>
first-pass uses: &{default map[] map[]}
first-pass rendering output of "helmfile.yaml.part.0":
 0: bases:
 1: - environments.yaml

error in first-pass rendering: result of "helmfile.yaml.part.0":
 0: bases:
 1: - environments.yaml

first-pass produced: &{default map[] map[]}
first-pass rendering result of "helmfile.yaml.part.0": {default map[] map[]}
second-pass rendering result of "helmfile.yaml.part.0":
 0: bases:
 1: - environments.yaml

first-pass rendering starting for "environments.yaml.part.0": inherited=&{default map[] map[]}, overrode=<nil>
first-pass uses: &{default map[] map[]}
first-pass rendering output of "environments.yaml.part.0":
 0: environments:
 1:   dev:
 2:   staging:
 3:   production:

first-pass produced: &{default map[] map[]}
first-pass rendering result of "environments.yaml.part.0": {default map[] map[]}
vals:
map[]
defaultVals:[]
second-pass rendering result of "environments.yaml.part.0":
 0: environments:
 1:   dev:
 2:   staging:
 3:   production:

merged environment: &{default map[] map[]}
merged environment: &{default map[] map[]}
first-pass rendering starting for "helmfile.yaml.part.1": inherited=&{default map[] map[]}, overrode=<nil>
first-pass uses: &{default map[] map[]}
first-pass rendering output of "helmfile.yaml.part.1":
 0: helmfiles:
 1:   - helmfiles/namespaces.yaml
 2: 
 3: repositories:
 4:   - name: stable
 5:     url: https://kubernetes-charts.storage.googleapis.com
 6: 
 7: helmDefaults:
 8:   verify: false
 9:   wait: false
10:   timeout: 600
11:   recreatePods: false
12:   force: true
13: 
14: releases:
15:   - name: redis
16:     namespace: redis
17:     chart: stable/redis
18:     version: ~9.2.1
19:     values:
20:       - usePassword: false
21:         cluster:
22:           enabled: false

first-pass produced: &{default map[] map[]}
first-pass rendering result of "helmfile.yaml.part.1": {default map[] map[]}
vals:
map[]
defaultVals:[]
second-pass rendering result of "helmfile.yaml.part.1":
 0: helmfiles:
 1:   - helmfiles/namespaces.yaml
 2: 
 3: repositories:
 4:   - name: stable
 5:     url: https://kubernetes-charts.storage.googleapis.com
 6: 
 7: helmDefaults:
 8:   verify: false
 9:   wait: false
10:   timeout: 600
11:   recreatePods: false
12:   force: true
13: 
14: releases:
15:   - name: redis
16:     namespace: redis
17:     chart: stable/redis
18:     version: ~9.2.1
19:     values:
20:       - usePassword: false
21:         cluster:
22:           enabled: false

merged environment: &{default map[] map[]}
processing file "namespaces.yaml" in directory "helmfiles"
changing working directory to "/Users/andrewnazarov/Repo/iris/test/helmfile/helmfiles"
first-pass rendering starting for "namespaces.yaml.part.0": inherited=&{default map[] map[]}, overrode=<nil>
first-pass uses: &{default map[] map[]}
first-pass rendering output of "namespaces.yaml.part.0":
 0: repositories:
 1:   - name: incubator
 2:     url: https://kubernetes-charts-incubator.storage.googleapis.com/
 3: 
 4: releases:
 5:   - name: namespaces
 6:     namespace: default
 7:     chart: incubator/raw
 8:     values:
 9:       - resources:
10:         - apiVersion: v1
11:           kind: Namespace
12:           metadata:
13:             name: redis
14:           spec:

first-pass produced: &{default map[] map[]}
first-pass rendering result of "namespaces.yaml.part.0": {default map[] map[]}
vals:
map[]
defaultVals:[]
second-pass rendering result of "namespaces.yaml.part.0":
 0: repositories:
 1:   - name: incubator
 2:     url: https://kubernetes-charts-incubator.storage.googleapis.com/
 3: 
 4: releases:
 5:   - name: namespaces
 6:     namespace: default
 7:     chart: incubator/raw
 8:     values:
 9:       - resources:
10:         - apiVersion: v1
11:           kind: Namespace
12:           metadata:
13:             name: redis
14:           spec:

merged environment: &{default map[] map[]}
changing working directory back to "/Users/andrewnazarov/Repo/iris/test/helmfile"
NAME      	NAMESPACE	INSTALLED	LABELS
namespaces	default  	true     	      
redis     	redis    	true

The same story for helmfile diff and helmfile apply.

helmfile version v0.94.0

Also, the v0.94.1 has been checked for helmfile list. This issue looks like #1009, but not exactly the same.

About this issue

  • Original URL
  • State: open
  • Created 5 years ago
  • Comments: 17 (9 by maintainers)

Most upvoted comments

To revive this issue:

  • I think it would be really handy to be able to define environments in a remote / shared helmfile. This would reduce the boilerplate we run into having to copy/paste the environment block around in every helmfile.
  • Maybe rethink how releases are included in environments then? Releases are included in an environment through label selector matching perhaps?
+9
albertrdixon on Jun 12, 2020

@mumoshu In my case, the releases from sub-helmfiles were silently ignored. Thus, I was confused.

Basically there could be a few use cases for sub-helmfiles:

  1. The umbrella helmfile with only references to external helmfiles
  2. Regular helmfile with releases and stuff, but at the same time referencing to some external helmfiles.

I got the point about processing the sub-helmfile before the root one, so it’s strange to take some part of the payload from the base.

On the other hand, it feels natural and DRY-compliant to define environments once and use them everywhere. If I want to ignore the release for a certain env I can do if-logic.

So, I really don’t know what is better. I can understand both approaches.

I would prefer either using “global” environments (taken from the “root” file) or at least be warned by the output somehow.

+2
andrewnazarov on Dec 22, 2019
Read more comments on GitHub
← helmfile: release templating cannot refer to other release variables
helmfile: remove "helm diff" errors when doing apply  →