helm: The template command fails to dump generated files if it cannot parse as valid yaml file

Output of helm version:

version.BuildInfo{Version:"v3.0.0", GitCommit:"e29ce2a54e96cd02ccfce88bee4f58bb6e2a28b6", GitTreeState:"clean", GoVersion:"go1.13.4"}

Output of kubectl version:

Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.6", GitCommit:"96fac5cd13a5dc064f7d9f4f23030a6aeface6cc", GitTreeState:"clean", BuildDate:"2019-08-19T11:13:49Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"windows/amd64"}
Server Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.8-gke.12", GitCommit:"188432a69210ca32cafded81b4dd1c063720cac0", GitTreeState:"clean", BuildDate:"2019-11-07T19:27:01Z", GoVersion:"go1.12.11b4", Compiler:"gc", Platform:"linux/amd64"}

Cloud Provider/Platform (AKS, GKE, Minikube etc.):

GKE

The helm template command in helm 3.0.0 tries to parse the results as yaml files. So if there is a problem with the generated yaml file, I cannot see the file itself to check the problem. With helm 2, the template command dumped the generated files without trying to parse it as yaml file, so I could check the problem. Now it is not possible.

Sample chart: templatebug.tar.gz

With helm 3:

$ helm3 template templatebug --debug
install.go:148: [debug] Original chart version: ""
install.go:165: [debug] CHART PATH: C:\templatebug

Error: YAML parse error on templatebug/templates/service.yaml: error converting YAML to JSON: yaml: line 4: found character that cannot start any token
helm.go:76: [debug] error converting YAML to JSON: yaml: line 4: found character that cannot start any token
YAML parse error on templatebug/templates/service.yaml
helm.sh/helm/v3/pkg/releaseutil.(*manifestFile).sort
        /home/circleci/helm.sh/helm/pkg/releaseutil/manifest_sorter.go:129
helm.sh/helm/v3/pkg/releaseutil.SortManifests
        /home/circleci/helm.sh/helm/pkg/releaseutil/manifest_sorter.go:98
helm.sh/helm/v3/pkg/action.(*Configuration).renderResources
        /home/circleci/helm.sh/helm/pkg/action/install.go:455
helm.sh/helm/v3/pkg/action.(*Install).Run
        /home/circleci/helm.sh/helm/pkg/action/install.go:214
main.runInstall
        /home/circleci/helm.sh/helm/cmd/helm/install.go:209
main.newTemplateCmd.func1
        /home/circleci/helm.sh/helm/cmd/helm/template.go:63
github.com/spf13/cobra.(*Command).execute
        /go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:826
github.com/spf13/cobra.(*Command).ExecuteC
        /go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:914
github.com/spf13/cobra.(*Command).Execute
        /go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:864
main.main
        /home/circleci/helm.sh/helm/cmd/helm/helm.go:75
runtime.main
        /usr/local/go/src/runtime/proc.go:203
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1357

With helm 2:

$ helm template templatebug
---
# Source: templatebug/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: %!s(<nil>)
spec:
  ports:
    - port: 80
      targetPort: http
      protocol: TCP
      name: http

It seems that heml 3 uses install internally from template with dry-run and client-only set to true. Install tries to detect hooks, and sort files accordingly. It needs to parse yaml files, but in case of error the whole command failes. Maybe in case of client only, ordering errors should be swallowed…

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 9
Comments: 24 (6 by maintainers)

Commits related to this issue

Fixes #7088 template command bug The template command (and install/upgrade) will dump the content of the problematic yaml content if it cannot be unmarshalled. Signed-off-by: Laszlo Varadi <laszlo.v... — committed to vazul/helm by vazul 5 years ago
Fixes #7088 template command bug The template command (and install/upgrade) will dump the content of the problematic yaml content if it cannot be unmarshalled. Signed-off-by: Laszlo Varadi <laszlo.v... — committed to vazul/helm by vazul 5 years ago
Fixes #7088 template command bug The template command (and install/upgrade) will dump the content of the problematic yaml content if it cannot be unmarshalled. Signed-off-by: Laszlo Varadi <laszlo.v... — committed to vazul/helm by vazul 5 years ago
Fixes #7088 template command bug The template command (and install/upgrade) will dump the content of the problematic yaml content if it cannot be unmarshalled. Signed-off-by: Laszlo Varadi <laszlo.v... — committed to vazul/helm by vazul 5 years ago
Fixes #7088 template command bug The template command (and install/upgrade) will dump the content of the problematic yaml content if it cannot be unmarshalled. Signed-off-by: Laszlo Varadi <laszlo.v... — committed to vazul/helm by vazul 5 years ago
Fixes #7088 template command bug The template command (and install/upgrade) will dump the content of the problematic yaml content if it cannot be unmarshalled. Signed-off-by: Laszlo Varadi <laszlo.v... — committed to vazul/helm by vazul 5 years ago
Fixes #7088 template command bug The template command (and install/upgrade) will dump the content of the problematic yaml content if it cannot be unmarshalled. Signed-off-by: Laszlo Varadi <laszlo.v... — committed to vazul/helm by vazul 5 years ago
Fixes #7088 template command bug The template command (and install/upgrade) will dump the content of the problematic yaml content if it cannot be unmarshalled. Signed-off-by: Laszlo Varadi <laszlo.v... — committed to vazul/helm by vazul 5 years ago
Fixes #7088 template command bug The template command (and install/upgrade) will dump the content of the problematic yaml content if it cannot be unmarshalled. Signed-off-by: Laszlo Varadi <laszlo.v... — committed to vazul/helm by vazul 5 years ago
Fixes #7088 template command bug The template command (and install/upgrade) will dump the content of the problematic yaml content if it cannot be unmarshalled. Signed-off-by: Laszlo Varadi <laszlo.v... — committed to vazul/helm by vazul 5 years ago

Most upvoted comments

+1 Trying to locate the cause of invalid YAML with Helm 3 is a nightmare.

+24

xsreality on Jan 31, 2020

Can anyone take a look at this issue? I submit another one due to this is away from my original proposal (make Helm3 also outputs YAML contents regardless it’s invaild or not), but it has been closed.

wi1dcard on Jan 16, 2020

I must be missing something because this doesn’t work for me. When it encounters the faulty file (I’m quite sure it’s an indentation issue), I just get the error and no file:

helm template -f test_4_common_config.yml --debug mail 

...
---
# Source: mail/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
...
Error: YAML parse error on mail/templates/statefulset.yaml: error converting YAML to JSON: yaml: line 84: mapping values are not allowed in this context
helm.go:84: [debug] error converting YAML to JSON: yaml: line 84: mapping values are not allowed in this context
YAML parse error on mail/templates/statefulset.yaml
elm.sh/helm/v3/pkg/releaseutil.(*manifestFile).sort
        /private/tmp/helm-20200619-73670-pk36oo/pkg/releaseutil/manifest_sorter.go:146
helm.sh/helm/v3/pkg/releaseutil.SortManifests
        /private/tmp/helm-20200619-73670-pk36oo/pkg/releaseutil/manifest_sorter.go:106
helm.sh/helm/v3/pkg/action.(*Configuration).renderResources
        /private/tmp/helm-20200619-73670-pk36oo/pkg/action/action.go:159
helm.sh/helm/v3/pkg/action.(*Install).Run
        /private/tmp/helm-20200619-73670-pk36oo/pkg/action/install.go:238
main.runInstall
        /private/tmp/helm-20200619-73670-pk36oo/cmd/helm/install.go:229
main.newTemplateCmd.func1
        /private/tmp/helm-20200619-73670-pk36oo/cmd/helm/template.go:66
github.com/spf13/cobra.(*Command).execute
        /private/tmp/helm-20200619-73670-pk36oo/.brew_home/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:842
github.com/spf13/cobra.(*Command).ExecuteC
        /private/tmp/helm-20200619-73670-pk36oo/.brew_home/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:950
github.com/spf13/cobra.(*Command).Execute
        /private/tmp/helm-20200619-73670-pk36oo/.brew_home/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:887
main.main
        /private/tmp/helm-20200619-73670-pk36oo/cmd/helm/helm.go:83
runtime.main
        /usr/local/Cellar/go/1.14.3/libexec/src/runtime/proc.go:203
runtime.goexit
        /usr/local/Cellar/go/1.14.3/libexec/src/runtime/asm_amd64.s:1373

helm version
version.BuildInfo{Version:"v3.2.4", GitCommit:"0ad800ef43d3b826f31a5ad8dfbb4fe05d143688", GitTreeState:"dirty", GoVersion:"go1.14.3"}

bokysan on Nov 3, 2020

I had a similar issue with my chart, where my stupid use of whitespace trimming led to bad Yaml outputs. The problem is, its really difficult to diagnose where the problem is, because instead of giving you the bad yaml, which you could immediately run through a yaml parser, and see your error, it just tells you what file the error is on, and gives you a line number, which could be in any number of places if you have multiple definitions inside a file.

half2me on Jan 2, 2020

Why is this closed. I also get mad to find the issue without getting valuable info, like the printed out yaml (even invalid)

{{- define "myChart.Name" -}}
{{- if .Values.canary.enabled }}
{{- printf "%s-canary" .Chart.Name -}}
{{- else }}
{{- printf "%s" .Chart.Name -}}
{{- end }}
{{- end -}}

{{- define "labels" -}}
key: {{ .Values.app.key }}
name: {{ .Values.app.name }}
version: {{ .Values.app.version | quote }}
canary: {{ .Values.canary.enabled | quote }}
app: {{ printf "%s-%s" .Chart.Name .Chart.Version }}
chart: {{ printf "%s-%s" .Chart.Name .Chart.Version }}
release: {{ include "myRelease.Name" . }}
heritage: {{ .Release.Service }}
helm.sh/chart: {{ printf "%s-%s" .Chart.Name .Chart.Version }}
app.kubernetes.io/name: {{ .Values.app.name }}
app.kubernetes.io/version: {{ .Values.app.version | quote }}
app.kubernetes.io/instance: {{ include "myRelease.Name" . }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end -}}

{{- define "annotations" -}}
meta.helm.sh/release-name: {{ include "myRelease.Name" . }}
meta.helm.sh/release-namespace: {{ .Release.Namespace }}
{{- end -}}

kind: ConfigMap
apiVersion: v1
metadata:
  name: {{ (include "myChart.Name" .) }}
  namespace: {{ .Release.Namespace }}
  labels:
    {{- include "labels" . | nindent 4 }}
  annotations:
    {{- include "annotations" . | nindent 4 }}

Error: UPGRADE FAILED: YAML parse error on myservice/templates/configmap.yaml: error converting YAML to JSON: yaml: line 6: could not find expected ':'
helm.go:81: [debug] error converting YAML to JSON: yaml: line 6: could not find expected ':'
YAML parse error on myservice/templates/configmap.yaml
helm.sh/helm/v3/pkg/releaseutil.(*manifestFile).sort
	/home/circleci/helm.sh/helm/pkg/releaseutil/manifest_sorter.go:146
helm.sh/helm/v3/pkg/releaseutil.SortManifests
	/home/circleci/helm.sh/helm/pkg/releaseutil/manifest_sorter.go:106
helm.sh/helm/v3/pkg/action.(*Configuration).renderResources
	/home/circleci/helm.sh/helm/pkg/action/action.go:165
helm.sh/helm/v3/pkg/action.(*Upgrade).prepareUpgrade
	/home/circleci/helm.sh/helm/pkg/action/upgrade.go:215
helm.sh/helm/v3/pkg/action.(*Upgrade).Run
	/home/circleci/helm.sh/helm/pkg/action/upgrade.go:124
main.newUpgradeCmd.func2
	/home/circleci/helm.sh/helm/cmd/helm/upgrade.go:155
github.com/spf13/cobra.(*Command).execute
	/go/pkg/mod/github.com/spf13/cobra@v1.1.1/command.go:850
github.com/spf13/cobra.(*Command).ExecuteC
	/go/pkg/mod/github.com/spf13/cobra@v1.1.1/command.go:958
github.com/spf13/cobra.(*Command).Execute
	/go/pkg/mod/github.com/spf13/cobra@v1.1.1/command.go:895
main.main
	/home/circleci/helm.sh/helm/cmd/helm/helm.go:80
runtime.main
	/usr/local/go/src/runtime/proc.go:204
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1374
UPGRADE FAILED
main.newUpgradeCmd.func2
	/home/circleci/helm.sh/helm/cmd/helm/upgrade.go:157
github.com/spf13/cobra.(*Command).execute
	/go/pkg/mod/github.com/spf13/cobra@v1.1.1/command.go:850
github.com/spf13/cobra.(*Command).ExecuteC
	/go/pkg/mod/github.com/spf13/cobra@v1.1.1/command.go:958
github.com/spf13/cobra.(*Command).Execute
	/go/pkg/mod/github.com/spf13/cobra@v1.1.1/command.go:895
main.main
	/home/circleci/helm.sh/helm/cmd/helm/helm.go:80
runtime.main
	/usr/local/go/src/runtime/proc.go:204
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1374

cforce on Apr 11, 2021

Turns out you have to use helm template --debug [... all the same parameters].

It wasn’t obvious.

gajus on Jul 23, 2020

fixed in #7556, which will be in Helm 3.2.0. Use the --debug flag to display the invalid YAML.

Doesn’t work.

contra-file-upload-api on  master via ⬢ v14.5.0
❯ helm version
version.BuildInfo{Version:"v3.2.4", GitCommit:"0ad800ef43d3b826f31a5ad8dfbb4fe05d143688", GitTreeState:"dirty", GoVersion:"go1.14.3"}

contra-file-upload-api on  master via ⬢ v14.5.0
❯ helm install contra-file-upload-api charts/server --debug --values .server.values.yaml --set app.datadog.env=production --set app.datadog.service=contra-file-upload-api --set app.datadog.version=foo
install.go:159: [debug] Original chart version: ""
install.go:176: [debug] CHART PATH: /Users/gajus/Documents/dev/contra/contra-file-upload-api/charts/server

Error: YAML parse error on contra-file-upload-api/templates/deployment.yaml: error converting YAML to JSON: yaml: line 49: mapping values are not allowed in this context
helm.go:84: [debug] error converting YAML to JSON: yaml: line 49: mapping values are not allowed in this context
YAML parse error on contra-file-upload-api/templates/deployment.yaml
helm.sh/helm/v3/pkg/releaseutil.(*manifestFile).sort
	/private/tmp/helm-20200619-73670-pk36oo/pkg/releaseutil/manifest_sorter.go:146
helm.sh/helm/v3/pkg/releaseutil.SortManifests
	/private/tmp/helm-20200619-73670-pk36oo/pkg/releaseutil/manifest_sorter.go:106
helm.sh/helm/v3/pkg/action.(*Configuration).renderResources
	/private/tmp/helm-20200619-73670-pk36oo/pkg/action/action.go:159
helm.sh/helm/v3/pkg/action.(*Install).Run
	/private/tmp/helm-20200619-73670-pk36oo/pkg/action/install.go:238
main.runInstall
	/private/tmp/helm-20200619-73670-pk36oo/cmd/helm/install.go:229
main.newInstallCmd.func1
	/private/tmp/helm-20200619-73670-pk36oo/cmd/helm/install.go:117
github.com/spf13/cobra.(*Command).execute
	/private/tmp/helm-20200619-73670-pk36oo/.brew_home/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:842
github.com/spf13/cobra.(*Command).ExecuteC
	/private/tmp/helm-20200619-73670-pk36oo/.brew_home/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:950
github.com/spf13/cobra.(*Command).Execute
	/private/tmp/helm-20200619-73670-pk36oo/.brew_home/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:887
main.main
	/private/tmp/helm-20200619-73670-pk36oo/cmd/helm/helm.go:83
runtime.main
	/usr/local/Cellar/go/1.14.3/libexec/src/runtime/proc.go:203
runtime.goexit
	/usr/local/Cellar/go/1.14.3/libexec/src/runtime/asm_amd64.s:1373

gajus on Jul 13, 2020

@aianus I too thought it was still broken until I did helm template --debug ... | grep <broken file>. The files are outputted in some random order, which means the broken file can appear way before the error message, it probably isn’t the last one that helm outputs.

Anyway, nobody mentioned this and it is not obvious.

galzetta on Mar 2, 2021