argo-cd: Application set templating fails with arrays (Helm valueFiles)

Checklist:

[ x] I’ve searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
[ x] I’ve included steps to reproduce the bug.
[ x] I’ve pasted the output of argocd version.

Describe the bug Application set with Git file generator consuming a config JSON file that includes a parameter (valueFiles) with value being an array of strings cannot be applied using kubectl

To Reproduce config.json:

{
  "releaseName": "nf1",
  "namespace": "ns1",
  "syncWave": "6",
  "valueFiles": [ "nf1-values.yaml", "nf1-config.yaml" ]
}

nf.yaml

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: nf
  namespace: argocd
spec:
  generators:
  - git:
      repoURL: https://github.com/<org>/cd
      revision: HEAD
      files:
      - path: "*/*/config.json"
  template:
    metadata:
      name: '{{path.basename}}'
      namespace: '{{namespace}}'
      annotations:
        argocd.argoproj.io/sync-wave: '{{syncWave}}'
      finalizers:
      - resources-finalizer.argocd.argoproj.io
    spec:
      project: sysarch
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
        syncOptions:
        - CreateNamespace=true
      source:
        helm:
          releaseName: '{{releaseName}}'
          valueFiles: '{{valueFiles}}'   <---- OPTION A
          valueFiles: {{valueFiles}}     <---- OPTION B
        path: '{{path}}'
        repoURL: https://github.com/<org>/<repo>
        targetRevision: HEAD
      destination:
        namespace: '{{namespace}}'
        name: '{{path[0]}}-cluster'

OPTION A:

error: error validating "nf.yaml": error validating data: ValidationError(ApplicationSet.spec.template.spec.source.helm.valueFiles): invalid type for io.argoproj.v1alpha1.ApplicationSet.spec.template.spec.source.helm.valueFiles: got "string", expected "array"; if you choose to ignore these errors, turn validation off with --validate=false

OPTION B: error: error parsing nf.yaml: error converting YAML to JSON: yaml: invalid map key: map[interface {}]interface {}{"valueFiles":interface {}(nil)}

Expected behavior Templating to work with arrays of string to support templating of Helm valueFiles defined as array of strings in the CRD

Screenshots

Version

argocd: v2.3.4+ac8b7df
  BuildDate: 2022-05-18T13:06:25Z
  GitCommit: ac8b7df9467ffcc0920b826c62c4b603a7bfed24
  GitTreeState: clean
  GoVersion: go1.17.9
  Compiler: gc
  Platform: linux/amd64
argocd-server: v2.3.4+ac8b7df

Logs

Paste any relevant application logs here.

About this issue

Original URL
State: open
Created 2 years ago
Reactions: 14
Comments: 21 (10 by maintainers)

Most upvoted comments

Hello, is there a plan to include the array support for the applicationset as a feature in 2023?

+10

alexchernykh on Jan 9, 2023

Just curious, is this still on the table or was it abandoned for another approach?

lsoica on Apr 11, 2023

I plan to get some general fix for this issue into 2.9. I’m not exactly sure what form that fix will take yet.

crenshaw-dev on Jul 11, 2023

Hello this is currently implemented here https://github.com/argoproj/argo-cd/issues/9583 I just need to rebase and make it available for next release 😃

speedfl on Nov 22, 2023

@jessebot I wish. 😃 It’s gonna be 2.9.0-rc1. @speedfl and I need to brainstorm some more to make sure we have the right design for more extensive templating support. We’re hoping to hit a nice balance between solving the known use cases and avoiding full build-your-own-yaml type support, which is unnecessarily finicky, given that we have a known CR structure.

crenshaw-dev on Aug 3, 2023

@aarontavio , @barthy1 As the helm task provides for a stringArray used as a value into the -f parameter, I have successfully tried out the following setup:

Application set (note, how the valueFiles parameter is set):

      source:
        helm:
          releaseName: '{{releaseName}}'
          valueFiles:
          - '{{valueFiles}}'
        path: '{{path}}'
        repoURL: https://github.com/sysarch-repo/cd
        targetRevision: HEAD

The GIT file (JSON):

{
  "releaseName": "rel1",
  "namespace": "ns1",
  "syncWave": "6",
  "valueFiles": "values1.yaml,values2.yaml,values3.yaml,values4.yaml"
}

With this, you can define multiple values files to be used. The applicationSet CR, the application CR and Argo CD seem to deal with this setup correctly, but just try and let me know.

@crenshaw-dev Could you pls confirm that this is a valid approach rather than exploiting a gap in ArgoCD?

sysarch-repo on Nov 22, 2022

With same number of value files you can use the simple string templating:

spec:
  source:
        repoURL: https://argoproj.github.io/argo-helm
        chart: argo-cd
        targetRevision: 5.51.2
        helm:
          releaseName: argo-cd
          valueFiles:
            - values.{{ ternary "default" "custom" .useCustom }}.yaml # Renders values.custom.yaml if generator returns .useCustom=true
            - {{ ternary "values.default.yaml" .overrideValueFile  .overrideValueFile }} # Renders values.mycustomvaluefile.yaml if generators returns .overrideValueFile=values.mycustomvaluefile.yaml

However if you need to add extra value files based on a variable then you have to wait for my MR on the patchTemplate.

This discussion looks like a support question. I invite you to contact us via the CNCF slack because I am sure a lot of person could be interested by our discussion 😃

speedfl on Nov 26, 2023

@jessebot you can also use multi-sources

speedfl on Nov 26, 2023

Thanks for the prompt reply. I am closing the ticket and will proceed with the static provisioning of the valueFiles.:

      source:
        helm:
          releaseName: '{{releaseName}}'
          valueFiles:
          - '{{path.basename}}-values.yaml'
          - '{{path.basename}}-config.yaml'

sysarch-repo on Jun 6, 2022