helmfile: Using helmfiles, but unable to pass Environment down to helmfile.yaml

@mumoshu This sounds like a great set of changes and improvements. Thank you for investing such much time and energy and keeping everyone updated here!

To summarize, the current Layering system doesn’t work as advertised, as it relies on helmfile to template each “part” of your helmfile.yaml THEN merge them one by one.

The reality was that helmfile template all the parts of your helmfile.yaml at once, and then merge those YAML documents. So @sruon was making a GREAT point that we may need to change helmfile to render templates earlier - that is to evaluate a template per each helmfile.yaml part separated by ---. Sorry I missed my expertise to follow your great idea you raised it last year @sruon 😭

This, in combination with the wrong documentation, confused so many people. To finally overcome this situation, I’m authoring a fairly large PR that introduces the 2 enhancements:

• bases: for easier layering without gotemplates(but not perfect)
• Split before template, and then merge helmfile.yaml parts(inspired by/enhanced version of @sruon’s work)

After that, everyone’s use-cases can be addressed like follows:

(1) @nikolajbrinch https://github.com/roboll/helmfile/issues/388#issue-373930524

before

{{ readFile "environments.yaml }}}
--- 
helmfiles:
- ./*/helmfile.yaml

after: just add {{ readFile "environments.yaml" }} or bases: [environments.yaml] to each sub-helmfile. helmfile is still trying hard not to introduce global state and global variables other than envvars. So this might be the best thing we can do at the moment.

(2) @Tarick’s https://github.com/roboll/helmfile/issues/388#issuecomment-433372631 and @rajiteh’s https://github.com/roboll/helmfile/issues/388#issuecomment-467972236

The below would just work after the upcoming enhancement.

{{ readFile "common.yaml" }}
---
{{ readFile "environments.yaml" }}

(3) @joshwand https://github.com/roboll/helmfile/issues/388#issuecomment-450203933

because the context is not passed to readFile ?

I think that’s almost correct. While evaluating the helmfile.yaml as a go template, readFile is evaluated to “inject” the file into the caller helmfile.yaml, and that’s all. There’s no further go template evaluations to render expressions in your helmfile-app-base.yaml, hence it isn’t rendered.

I think it should be edited to use Release Template.

Instead of:

releases:
- name: app-{{ .Environment.Name }}
  namespace: {{ .Environment.Values.namespace }}
  chart: ../charts/myapp
  values: 
    - {{ .Environment.Values }} 

Use:

releases:
- name: app-{{` {{ .Environment.Name }} `}}
  namespace: {{` {{ .Environment.Values.namespace }} ``}
  chart: ../charts/myapp
  values: 
    - {{` {{ .Environment.Values }} ``}

This “defers” evaluations of template expressions immediately before processing each release.

(4) @royjs’s https://github.com/roboll/helmfile/issues/388#issuecomment-468670403

I think you have one of the most advanced use-cases ☺️

environments:
  dev:
  prod:

templates:
  default: &default
    missingFileHandler: Debug
    values:
      - values/common.yaml.gotmpl 
      {{- range $overrideFolder := .Environment.Values.overrideFolders }}
      - values/{{ $overrideFolder }}/{{`{{  .Release.Name }}`}}.yaml
      {{- end }}

releases:
  ....

This requires you environments to be “already parsed” before evaluating .Environment.Values within your helmfile.yaml. Externalizing environments with {{ readFile ... }} does the opposite - because {{ readFile }} is executed only after environments are loaded. That’s why your example didn’t work.

After the upcoming change, you can make it a multi-part template with ---.

{{ readFile "environments.yaml" }}
---
templates:
  default: &default
    missingFileHandler: Debug
    values:
      - values/common.yaml.gotmpl 
      {{- range $overrideFolder := .Environment.Values.overrideFolders }}
      - values/{{ $overrideFolder }}/{{`{{  .Release.Name }}`}}.yaml
      {{- end }}

releases:
  ....

I think I’ve reviewed and answered all the use-cases introduced in this thread. But please feel free to ask if I missed any!

Is it possible to use same technique to access data from previous values files, described as several files in release, or from secrets? Something like this: helmfile:

releases:
  - name: my-chart
    chart: my-chart
    namespace: my-chart
    values:
      - ./config/my-chart/secrets.yaml
    values:
      - ./config/my-chart/values1.yaml.gotmpl
      - ./config/my-chart/values2.yaml.gotmpl

./config/my-chart/secrets.yaml.:

mySecret: superSecret

./config/my-chart/values1.yaml.gotmpl:

Variable1: value1

./config/my-chart/values2.yaml.gotmpl:

Variable2: {{ .Values | get "Variable1" }}
MySecretInValues: {{ .Values | get "mySecret" }}

I was about to log a new issue with this precise problem, is there any traction on this feature? It seems like an obvious thing to support and it would really help with DRY

After debugging for a couple hours, I don’t see how Layering can work the way it’s described in the documentation.

https://github.com/roboll/helmfile/blob/master/tmpl/tmpl.go#L24 parses the file and does not know about the environment yet. If you use a value loaded from an environment, it will just choke on the unknown value and stop loading there i.e.

{{ readFile "environments.yaml"}}
---
releases:
  - name: my-app-{{ .Environment.Values.releaseName }}

I toyed around with splitting the input on --- and parsing each part as a separate fragment / reloading the environment on each loop and that appears to be almost entirely working. The only issue I’m seeing is the templated values cannot be used as part of a release name, I think the first pass renderer needs to be modified similarily.

index 399109f..00fc2de 100644
--- a/main.go
+++ b/main.go
@@ -750,20 +750,26 @@ func (r *twoPassRenderer) renderEnvironment(content []byte) environment.Environm
 
 func (r *twoPassRenderer) renderTemplate(content []byte) (*bytes.Buffer, error) {
 	// try a first pass render. This will always succeed, but can produce a limited env
-	firstPassEnv := r.renderEnvironment(content)
+	splitContent := bytes.Split(content, []byte("---"))
+	var yamlBuf bytes.Buffer
 
-	secondPassRenderer := tmpl.NewFileRenderer(r.reader, filepath.Dir(r.filename), firstPassEnv, r.namespace)
-	yamlBuf, err := secondPassRenderer.RenderTemplateContentToBuffer(content)
-	if err != nil {
+	firstPassEnv := r.renderEnvironment(content)
+	for _, subContent := range splitContent {
+		secondPassRenderer := tmpl.NewFileRenderer(r.reader, filepath.Dir(r.filename), firstPassEnv, r.namespace)
+		subBuf, err := secondPassRenderer.RenderTemplateContentToBuffer(subContent)
+		if err != nil {
+			if r.logger != nil {
+				r.logger.Debugf("second-pass rendering failed, input of \"%s\":\n%s", r.filename, prependLineNumbers(string(subContent)))
+			}
+			return nil, err
+		}
 		if r.logger != nil {
-			r.logger.Debugf("second-pass rendering failed, input of \"%s\":\n%s", r.filename, prependLineNumbers(string(content)))
+			r.logger.Debugf("second-pass rendering result of \"%s\":\n%s", r.filename, prependLineNumbers(subBuf.String()))
 		}
-		return nil, err
-	}
-	if r.logger != nil {
-		r.logger.Debugf("second-pass rendering result of \"%s\":\n%s", r.filename, prependLineNumbers(yamlBuf.String()))
-	}
-	return yamlBuf, nil
+			yamlBuf.WriteString(subBuf.String())
+			firstPassEnv = r.renderEnvironment(yamlBuf.Bytes())
+		}
+	return &yamlBuf, nil
 }
 
 func (a *app) VisitDesiredStates(fileOrDir string, converge func(*state.HelmState, helmexec.Interface) (bool, []error)) error {```

The enahcements will be included since helmfile v0.60.0

I started to think we need #96 proposed by @gtaylor for this.

With that we could reuse whatever helmfile.yaml fragment(s) like:

includes:
- common-environments.yaml

And you’ll explicitly repeat it in every sub-helmfile. Btw, I think the explicitness is a must-have, because implicitness means that we rely on “globals” #398, and I believe globals should be implemented by environment variables(not values) and the {{ env ... }} template function.

Once this comes reality, I’d deprecate the current layering system with multi-doc YAML as unnecessary.

WDYT?

I’m not sure it’s necessary, but here’s a use case that misses this feature as far as I can tell. We are managing multiple clusters via helmfile and there are certain “global” values that I would like to pass only once to helmfile. For example, a cluster ID / name that is used for all ingress hostnames.

I tried serveral things. This looks like it does not work because of this issue:

values.yaml:

clusterID: 010101

helmfile.yaml:

environments:
  - default:
    values:
      - values.yaml

helmfiles:
  - helmfiles/prometheus.yaml

helmfiles/prometheus.yaml:

releases:
  - name: prometheus
    namespace: prometheus
    chart: stable/prometheus
    values:
      - values.yaml.gotmpl

helmfiles/values.yaml.gotmpl:

server:
  ingress:
    hosts:
      - prometheus.{{ .Environment.Values.clusterID }}.clusters.lcoal

I also tried including the helmfiles via readFile, but unfortunately their configuration will not get templated and therefore this takes away a considerable amount of features (the hook example from the README won’t work with that, for example).

Passing down the configured environment via helmfiles would really be awesome!

I am also having difficulty with this-- I have a situation where I have many production environments (one per customer), and I want to put them in separate helmfiles to keep the file readable, but then I can’t use the environment vars in the base file brought in via readfile:

helmfile-dev.yaml

environments: 
  ci-master:
    values: 
    - values-ci-master.yaml 
  dev1:
    values: 
    - values-dev1.yaml

{{ readFile "../Helmfile-app-base.yaml" }}

Helmfile-prod-customer1:

environments: 
  prod-customer1:
    values: 
      - prod-customer1/prod-customer1.yaml
  prod-customer1-sandbox:
    values: 
      - prod-customer1/prod-customer1-sandbox.yaml

{{ readFile ../Helmfile-app-base.yaml }}

but the values from the evnironment aren’t rendered in Helmfile-app-base.yaml, which looks like this:

releases:
- name: app-{{ .Environment.Name }}
  namespace: {{ .Environment.Values.namespace }}
  chart: ../charts/myapp
  values: 
    - {{ .Environment.Values }} 

because the context is not passed to readFile ?

@Tarick Hey! Just to clarify, are you talking about the fact that helmfile doesn’t propagate the environment and the environment values from the parent to children? Or you’re maybe talking about a possible bug(?) that multiple yaml docs declared within a single helmfile.yaml doesn’t get merged?