zarf: Zarf does not support deploying Flux HelmReleases with OCI Helm chart sources
Is your feature request related to a problem? Please describe.
Zarf does not support deploying Flux HelmReleases with OCI Helm chart sources
Describe the solution you’d like
- Given
- A
zarf.yamlwithmanifests:including a Flux HelmRepository referencing anoci://chart and a HelmRelease referencing it. - And including the OCI helm chart in
images:inzarf.yaml.
- A
- When creating the package
- Then
- The package cannot be created because
images:does not support arbitrary (non-container) OCI imagesERROR: Failed to create package: unable to pull images after 3 attempts: error when trying to save the img (ghcr.io/stefanprodan/charts/podinfo:6.4.1): mismatched fs layers (1) and diff ids (0) - Additionally, Zarf’s mutating admission webhooks do not support mutating HelmRepository and HelmRelease resources to reference the Zarf registry
- The package cannot be created because
Example Flux manifests referencing an OCI Helm chart: https://fluxcd.io/flux/cheatsheets/oci-artifacts/#helm-oci
A potential issue: Zarf uses an insecure registry and Flux source-controller may not support insecure OCI registries with Helm: https://github.com/fluxcd/source-controller/issues/807
- A little unclear on what’s affected and if this issue is still current
- OCIRepository (consumed by kustomize-controller, not helm-controller) does support
.spec.insecure - Need to check the current status of HelmRepository and HelmReleases with an insecure localhost registry (sometimes localhost is special cased and allowed)
Describe alternatives you’ve considered
Zarf’s built in Helm support does allow OCI, see demo-helm-oci-chart in this example: https://docs.zarf.dev/examples/helm-charts/
However, this does not support using Flux.
Additional context
The scenario prompting this is a vendor only provides OCI packaged Helm charts. We do not need to modify the upstream vendor charts, and would like to copy them unmodified, and deploy them using Flux.
Additionally, Big Bang now distributes Helm charts in OCI format. If Zarf supported this, the dependency on a Gitea git repository could be dropped in many cases.
Support for generic OCI artifacts may be generally useful for other non-Flux use cases too.
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 4
- Comments: 16 (12 by maintainers)
Commits related to this issue
- test: pod agent unit tests (#2526) ## Description This also changes how we the agent gets secrets from the cluster. Instead of mounting secrets in the deployment and reading from a file the agen... — committed to zarf-dev/zarf by AustinAbro321 3 years ago
- test: pod agent unit tests (#2526) ## Description This also changes how we the agent gets secrets from the cluster. Instead of mounting secrets in the deployment and reading from a file the agen... — committed to schristoff/zarf by AustinAbro321 3 years ago
- test: pod agent unit tests (#2526) ## Description This also changes how we the agent gets secrets from the cluster. Instead of mounting secrets in the deployment and reading from a file the agent wi... — committed to zarf-dev/zarf by AustinAbro321 3 years ago
The challenge with tls on the registry is their is no safe k8s-native way to tell the CRIs to use a given CA like you can with admission controllers. The most generic k8-way is to use a privileged daemonset to modify your CRI config (which is as awful as it sounds). Many platforms will modify the CRI out-of-band to accommodate this during cluster init or node image building.
Flux added a
.spec.insecuretotype: ociHelmRepositoryresources, the flux issue linked above was closed. 🎉In addition to mutating the source of an OCI HelmRepository/OCIRepository, zarf could add the insecure flag automatically if a localhost registry is used.
Unassigned as this will be likely 2 PRs, one for OCIRepo and another that needs more discussion between Zarf team. OCIRepo should be ready soon but is pending one PR.
The current PR that I am working on is addressed by this issue (Which is specific for OCIRepo)
Not quite. These are OCI artifacts (not quite, that specific term can mean something else), not actual “images”. These aren’t “Docker images” at all (the layers are not
application/vnd.oci.image.layer.v1.tar+gziptarball layers with aapplication/vnd.oci.image.index.v1+jsonconfig). This is packaging non-Docker images as into “OCI images” something still compatible enough with registries to work, but storing different types of data (Helm charts in this case). You’d use something likehelm pushto create these OCI images.@Noxsios would be familiar and probably help explain. It’s also how Zarf’s own OCI support works (Zarf uses ORAS, Helm does something slightly different, but it’s the same concept).
And yes you can assume these images already exist, they’re created outsize Zarf. Zarf does need to package and transfer them all the way to the Zarf registry somehow though.
In theory these generic OCI images can be copied around the same way as Docker images, which is why I suggested listing them in
images:, but it seems something throws an error while downloading them.If at all helpful, for #3 listed above, I had to do this internally for a corporate network with Flux HelmRepository and the only work around I could easily get working was to patch the deployment with the CA that I had into the container like mentioned here: https://github.com/fluxcd/flux2/issues/3417#issuecomment-1470961381 i think if its completely insecure (no HTTPS) then I am not sure its at all possible based on what I have tried to do with Flux before and insecure OCI.