argo-cd: Cannot deploy multiple helm dependencies from same private repo server : repository name already exists

If you are trying to resolve an environment-specific issue or have a one-off question about the edge case that does not require a feature then please consider asking a question in argocd slack channel.

Checklist:

  • I’ve searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
  • I’ve included steps to reproduce the bug.
  • I’ve pasted the output of argocd version.

Describe the bug

Argocd 2.1.0 fails at deploying helm chart with dependencies fetched from same private help repository server. It triggers the error :

ComparisonError: rpc error: code = Unknown desc = Manifest generation error (cached): `helm repo add --username ****** --password ****** registry.xxx.io https://registry.xxx.io/chartrepo/cloud/` failed exit status 1: Error: repository name (registry.xxx.io) already exists, please specify a different name

It appears from code that the each new repo using repo-creds will be named as the host. So, if several repositories use the same host, the code tries to add a helm repo with an existing name.

This is a regression from 2.0.5 as this chart was deployed correctly with repository.credentials from argocd-cm

To Reproduce

I have a chart defined as following :

apiVersion: v2
name: my-portal
appVersion: 0.1.0
version: 0.1.0

dependencies:
  - name: admin-portal-web
    version: 1.0.0
    repository: https://registry.xxx.io/chartrepo/enterprise/
    condition: admin-portal-web.enabled

  - name: app-global-cert-manager
    version: 0.1.0
    repository: https://registry.xxx.io/chartrepo/cloud/
    condition: app-global-cert-manager.enabled

Both dependencies comes from the same private server, here harbor, ie chartmuseum. I have configured the repo-creds secret.

apiVersion: v1
kind: Secret
metadata:
  name: harbor-xxx-creds
  namespace: argocd
  labels:
    argocd.argoproj.io/secret-type: repo-creds
data:
  password: xxx=
  username: xxx==
stringData:
  url: https://registry.xxx.io
  type: helm

By parsing the Chart.yaml, the code will add a helm repo named registry.xxx.io for registry.xxx.io/chartrepo/enterprise. Then. for the second depedency, it will try to add the repo registry.xxx.io/chartrepo/cloud which will also be named registry.xxx.io https://github.com/argoproj/argo-cd/blob/a894d4b128c724129752bac9971c903ab6c650ba/reposerver/repository/repository.go#L493

Helm therefore will fire the error Error: repository name (registry.xxx.io) already exists

Expected behavior

This helm chart was correctly deployed with 2.0.5. It should still be deployed with 2.1.0.

Screenshots

n/a

Version

{
    "Version": "v2.1.0+d0b2d55",
    "BuildDate": "2021-08-20T05:30:54Z",
    "GitCommit": "d0b2d55e3fb7fe8b17385d6687886de41651f31b",
    "GitTreeState": "clean",
    "GoVersion": "go1.16.5",
    "Compiler": "gc",
    "Platform": "linux/amd64",
    "KsonnetVersion": "v0.13.1",
    "KustomizeVersion": "v4.2.0 2021-06-30T22:49:26Z",
    "HelmVersion": "v3.6.0+g7f2df64",
    "KubectlVersion": "v0.21.0",
    "JsonnetVersion": "v0.17.0"
}

Logs

argocd-repo-server-67f7f6fd9-796zd argocd-repo-server time="2021-08-25T06:42:21Z" level=error msg="finished unary call with code Unknown" error="Manifest generation error (cached): `helm repo add --username ****** --password ****** registry.xxx.io https://registry.xxx.io/chartrepo/cloud/` failed exit status 1: Error: repository name (registry.xxx.io) already exists, please specify a different name" grpc.code=Unknown grpc.method=GenerateManifest grpc.request.deadline="2021-08-25T06:43:21Z" grpc.service=repository.RepoServerService grpc.start_time="2021-08-25T06:42:21Z" grpc.time_ms=20.81 span.kind=server system=grpc

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Reactions: 8
  • Comments: 23

Most upvoted comments

@stieglma could you elaborate on this workaround? I’m seeing this too when deploying any app that uses one of our charts as a subchart. We do have that repo added as a repo to argo and have several projects using it, here’s some specifics:

Example Chart.yaml:

apiVersion: v2
name: mw-example-app
type: application
version: 2.0.0
maintainers:
dependencies:
    - name: common
      version: ">=0.0.2"
      repository: https://our.internal.artifactory.instance/artifactory/helm

Here’s the repository’s secret (managedFields and similar removed):

apiVersion: v1
data:
  name: artifactory
  password: redacted
  type: helm
  url: https://our.internal.artifactory.instance/artifactory/helm
  username: redacted
kind: Secret
metadata:
  annotations:
    managed-by: argocd.argoproj.io
  labels:
    argocd.argoproj.io/secret-type: repository
  name: repo-872087197
  namespace: argocd
type: Opaque

We also have a set of repositories configured in argocd-cm, but these are all public:

- name: bitnami
  type: helm
  url: https://charts.bitnami.com/bitnami
- name: jupyterhub
  type: helm
  url: https://jupyterhub.github.io/helm-chart/
- name: influxdata
  type: helm
  url: https://helm.influxdata.com/
- name: grafana
  type: helm
  url: https://grafana.github.io/helm-charts

repository.credentials includes our GitHub repo template only. We have a few projects but I’m only trying to use one with this helm subchart right now (some fields omitted for brevity):

apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: mw-devtest
  namespace: argocd
spec:
  clusterResourceWhitelist:
    ...
  description: Middleware (Development/Test)
  destinations:
    ...
  orphanedResources:
    warn: false
  roles:
  - jwtTokens:
    ...
  sourceRepos:
  - https://our.internal.github/*
  - git@our.internal.github/*
  -  '*'

I’m experiencing two problems. If, on my application, I do not enable passCredentials, then our artifactory repo is added with no credentials, and we can’t find the common chart. If I do enable passCredentials, then everything works directly after a bounce of argocd-repo-server. Right after it starts the /helm-working-dir directory in the pod is totally empty, even after syncing our impacted application.

Sometime after that, this directory fills up with repositories (including the fun https: one) and then I start getting the error in this issue.

I’m on ArgoCD 2.3.3, what am I missing from the workaround?

Edit: Is it possible that there’s one app with passCredentials: false that causes a helm repo add with no credentials which then later ruins it for all other apps that have passCredentials: true?

Assigning Alex because you has a pretty good idea what was going on here, but I’ll also keep this issue in mind. Can’t prioritize reproducing it just yet but hopefully soon.