kustomize: `helmCharts` repo field doesn't work with OCI registry

Describe the bug

Kustomize fails to pull charts through helmCharts yaml that are hosted within an OCI registry.

** Example **

Create a kustomize.yaml file:

helmCharts:
  - name: someName
    includeCRDs: true
    valuesFile: values.yaml
    releaseName: myRelease
    version: 0.0.26
    repo: oci://myRegistry

Run the command kustomize build --enable-helm --helm-command $helmCommand ./folder

Kustomize will fail to pull the helm chart and produce the following error:

Error: Error: looks like "oci://myRegistry" is not a valid chart repository or cannot be reached: object required
: unable to run: '/usr/sbin/helm pull --untar --untardir <someDir> --repo oci://myRegistry myRelease --version 0.0.26' with env=[HELM_CONFIG_HOME=/tmp/kustomize-helm-991395685/helm HELM_CACHE_HOME=/tmp/kustomize-helm-991395685/helm/.cache HELM_DATA_HOME=/tmp/kustomize-helm-991395685/helm/.data] (is '/usr/sbin/helm' installed?)

Running a regular helm pull works fine on the registry, but pulling with --repo causes issues:

# Working
$ helm pull oci://myRegistry/myRelease --version 0.0.26
Pulled: myRegistry/myRelease:0.0.26
Digest: sha256:14e1bc1ec1d0147eb8b41f081faae2ecea63e862ad4259d3a9cd1c7d1584be63

# Not working with `--repo`
$ helm pull myRelease --version 0.0.26 --repo myRegistry
Error: could not find protocol handler for: 

$ helm pull myRelease --version 0.0.26 --repo oci://myRegistry
Error: looks like "oci://myRegistry" is not a valid chart repository or cannot be reached: object required

I also cannot work around this by just adding it as a repo:

$ helm repo add reg myRegistry
Error: could not find protocol handler for: 

$ helm repo add reg oci://myRegistry
Error: looks like "oci://myRegistry" is not a valid chart repository or cannot be reached: object required

Expected output

Expected Kustomize to be able to pull the chart and apply the values.yaml to the chart. The solution would be for oci:// urls, to not use the --repo flag when performing a helm pull.

Kustomize version

$ kustomize version {Version:kustomize/v4.3.0 GitCommit:cd17338759ef64c14307991fd25d52259697f1fb BuildDate:2021-08-24T19:24:28Z GoOs:linux GoArch:amd64}

$ helm version version.BuildInfo{Version:“v3.7.2”, GitCommit:“663a896f4a815053445eec4153677ddc24a0a361”, GitTreeState:“clean”, GoVersion:“go1.16.10”}

Platform

Ubuntu

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 47
  • Comments: 27 (5 by maintainers)

Commits related to this issue

Most upvoted comments

I think there is a concern about backwards compatibility here, but the backwards compatibility issue was a side effect of the previous implementation and not an advertised feature. My vote is to move forward with the proposed PR.

@annasong20 discussed offline - we will add support in kustomize for oci registries by having the different commands running that helm provides for oci-based charts.

Any reason we can’t reconsider one of the other pull requests that address this as a short term solution while the long term vision is still being planned?

https://github.com/kubernetes-sigs/kustomize/pull/4614/files

While it is stated under Long term support that

We will not add support for:

  • OCI registries

I would kindly ask you to reconsider this opinion, given that the fix actually is just +7-2 LOC and the krm function does not support this yet, either and is not production-ready it seems.

any update on this issue? this will be a great feature for kustomize and it will expand kustomize’s adaptations

will this problem be solved soon?

As another ugly workaround, I found the solution: Running kustomize with --helm-command ~/tmp/sanitized_helm.sh

where ~/tmp/sanitized_helm.sh is:

#!/bin/bash
if [[ $@ = pull* ]]; then
    # If the command is `helm pull (..)` skips --repo flag and chartName
    # from command line args to make helm pull run

    # For explanation:
    # https://github.com/kubernetes-sigs/kustomize/issues/4381
    arr=(${@//--repo/});  # Skipping --repo
    args="${arr[@]:0:5} ${arr[@]:6}";  # Skipping chartName
else
    args="$@"
fi
cmd="/usr/bin/helm --registry-config ~/.config/helm/registry/config.json $args"
echo "Running '$cmd' " >> /tmp/helm-debug
eval $cmd

It’d be greatly appreciated if this fix is deployed soon!

@mikebz i’ll dare you free blowjob if you close this shit, dude >_<

Based off of this comment, it appears that the helm chart inflation tool is supporting bug fixes and adding fields that are normally supported through helm template. That would make this issue and #4335 valid and would not be waiting for the new KRM functionality. Is there any priority on these two issues which already have a fix?

Edit: Never-mind… I didn’t read the “we will not support OCI or private registries”… That is sad. I would consider being incompatible with Helm 3.8.0 features (which has been released for almost a year now) a bug…

@minnie-jeong-otsk I created a patch here and also published a release for this patch on my fork of kustomize (download the patched binary here). This patch works for us to pull oci:// registries. I will be submitting a MR to kustomize once my company allows us to sign the CLA.

Note that this release is NOT officially supported by kustomize.

To be fair, this work is required because the Helm team doesn’t follow SemVer with their tooling. They made a minor release with major CLI interface changes, which kustomize should not be responsible for fixing. If you read the comments, there appears to be a larger SIG working group discussion to evaluate how to properly handle Helm longer term, which I suspect this behavior will be addressed from that working group.

If anyone takes the above patch, I think they should note the trailing comma needs fixing:

- 		"--untardir", p.absChartHome(),
+ 		"--untardir", p.absChartHome()

I am unable to make a MR because I can’t sign the CLA but can someone make this MR for me?

This line should be:

func (p *HelmChartInflationGeneratorPlugin) pullCommand() []string {
	args := []string{
		"pull",
		"--untar",
		"--untardir", p.absChartHome(),
-               "--repo", p.Repo
-		p.Name}
+             }
	if p.Version != "" {
		args = append(args, "--version", p.Version)
	}
+	if strings.HasPrefix(p.Repo, "oci://") {
+	        args = append(args, p.Repo)
+	} else {
+              args = append(args, "--repo", p.Repo)
+              args = append(args, p.Name)
+       }
	return args
}

This is untested, but --repo should not be in the pull command if oci:// is specified. If oci:// is specified, just do a straight helm pull