argo-cd: Deployment is still progressing since Ingress resource is not considered as synced

Deployment is still progressing since Ingress resource is not considered as synced

Describe the bug We deploy an application in an “On Premise” Kubernetes using nginx Ingress. When deploying an application through Argo, the ingress object is still syncing although the application is available behind the Ingress. Consequently, our deployment is still progressing in Argo.

We think that this is related to that PR https://github.com/argoproj/argo-cd/pull/1053/files#diff-d5a0105b0157a44898f0cd002d7d827dR157 and that issue https://github.com/argoproj/argo-cd/issues/997 .

Our ingress controller is nginx and each ingress has the following status (which is a healthy status according to us):

...
status:
  loadBalancer: {}

To Reproduce

  1. Deploy an application with an ingress on a Kubernetes cluster using an internal ingress controller nginx (an on premise cluster for example)
  2. yaml of the ingress:
  kind: Ingress
  metadata:
    name: my-ingress
    namespace: myns
  spec:
    rules:
    - host: whatever.example.com
      http:
        paths:
        - backend:
            serviceName: my-service
            servicePort: 80
    tls:
    - hosts:
      - whatever.example.com
  1. See error

Expected behavior The ingress should be deployed in the Kubernetes cluster, and still be syncing in Argo. The Application deployment should still be progressing.

Screenshots If applicable, add screenshots to help explain your problem.

Version

argocd: v1.0.1+5fe1447.dirty
  BuildDate: 2019-05-28T17:26:35Z
  GitCommit: 5fe1447b722716649143c63f9fc054886d5b111c
  GitTreeState: dirty
  GoVersion: go1.11.4
  Compiler: gc
  Platform: linux/amd64
argocd-server: v1.0.1+5fe1447.dirty
  BuildDate: 2019-05-28T17:27:38Z
  GitCommit: 5fe1447b722716649143c63f9fc054886d5b111c
  GitTreeState: dirty
  GoVersion: go1.11.4
  Compiler: gc
  Platform: linux/amd64
  Ksonnet Version: 0.13.1

Logs

/tmp/argocd-linux-amd64 app list
NAME               CLUSTER                         NAMESPACE             PROJECT  STATUS  HEALTH       SYNCPOLICY  CONDITIONS
white-application  https://kubernetes.default.svc  whiteapp-development  default  Synced  Progressing  <none>      <none>


/tmp/argocd-linux-amd64 app get white-application
Name:               white-application
Project:            default
Server:             https://kubernetes.default.svc
Namespace:          whiteapp-development
URL:                https://cd.devops.caas.cagip.group.gca/applications/white-application
Repo:               https://scm.saas.cagip.group.gca/cagip/devops/tools-dashboard
Target:             HEAD
Path:               deploy
Sync Policy:        <none>
Sync Status:        Synced to HEAD (2b1892f)
Health Status:      Progressing

GROUP       KIND        NAMESPACE             NAME             STATUS  HEALTH
apps        Deployment  whiteapp-development  tools-dashboard  Synced  Healthy
extensions  Ingress     whiteapp-development  tools-dashboard  Synced  Progressing
            Service     whiteapp-development  tools-dashboard  Synced  Healthy

Have you thought about contributing a fix yourself?

Yes 😃

Open Source software thrives with your contribution. It not only gives skills you might not be able to get in your day job, it also looks amazing on your resume.

If you want to get involved, check out the contributing guide, then reach out to us on Slack so we can see how to get you started.

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Reactions: 5
  • Comments: 22 (4 by maintainers)

Commits related to this issue

Most upvoted comments

Hi. You need to make sure to apply the resource customization to the correct resource type.

I think recent versions of Kubernetes moved the Ingress resource type from the extensions API to the networking.k8s.io API, so what was

 resource.customizations: |
    extensions/Ingress:

before should now be

 resource.customizations: |
    networking.k8s.io/Ingress:

Please see. https://argoproj.github.io/argo-cd/faq/

You can use this for the Ingress (thank you @stevesea )

 resource.customizations: |
    extensions/Ingress:
        health.lua: |
          hs = {}
          hs.status = "Healthy"
          return hs

@enys Thanks for the great tip, I’ve ended up at this issue several times, but for those of us traefik users yours is the correct answer (as it actually fixes the issue rather than ignoring it). The documentation you linked is absolutely correct but when I read it I was still a bit confused so here are a few details for others that might stumble upon this issue.

The traefik publishedService config is applied to the kubernetesingress “controller”. This flag tells traefik what IP address it is using for its ingress so that it can correctly update standard kubernetes Ingress objects with the IP address used for ingress. This can be seen in the source here

I was thinking to myself when first reading the documentation that I’d need to set that flag to the service that each downstream ingress object targeted, but the reality is that it just needs to be set to the namespace/service of traefik itself. As you noted it’s very easy via the helm chart:

providers:
  kubernetesIngress:
    publishedService:
      enabled: true

This ends up just adding the following flag (assuming traefik is installed in the ingress-ns namespace and the traefik service is named traefik-svc) to traefik.

--providers.kubernetesingress.ingressendpoint.publishedservice=ingress-ns/traefik-svc

For Traefik users : https://doc.traefik.io/traefik/providers/kubernetes-ingress/#publishedservice And this a boolean option in the helm chart.

For those who could still find this thread in 2023: In my values.yaml file, i added:

configs:
  cm:
    resource.customizations: |
      networking.k8s.io/Ingress:
        health.lua: |
          hs = {}
          hs.status = "Healthy"
          return hs

It works now!

I using Helm Chart to deploy ArgoCD and I have nginx Ingress contoroller, I used this value.yaml file to fix this issue:

# Fixing issue with Stuck Processing for Ingress resource
server:
  config:
    resource.customizations: |
      networking.k8s.io/Ingress:
        health.lua: |
          hs = {}
          hs.status = "Healthy"
          return hs        

After update, just Resync your applications Ingress resource.

Shouldn’t this be the default behavior? I dont think it makes sense that everyone has to adjust config maps.

Huh, i don’t seem to be able to get this to work, i see the snippet i added in the argocd-cm configmap correctly, i rebooted the argocd-server pods and im still seeing my ingress as progressing. What am i missing? This is with 1.5.1.

resource.customizations:

configs:
  cm:
    resource.customizations: |
      networking.k8s.io/Ingress:
        health.lua: |
          hs = {}
          hs.status = "Healthy"
          return hs

It’s 2023 and I still have this issue.

Is this put in the argocd values.yaml? Where should I put this entry in the values.yaml file?

Can you provide a step by step guide for all future sufferers…

直接修改这个configmap:argocd-cm

apiVersion: v1
data:
  resource.customizations: |
    networking.k8s.io/Ingress:
      health.lua: |
        hs = {}
        hs.status = "Healthy"
        return hs
 
kind: ConfigMap
metadata:
  name: argocd-cm

改完后argocd会自动生效

For helm user, add this into traefik override.yaml:

additionalArguments:
  - "--providers.kubernetesingress.ingressendpoint.publishedservice=traefik/traefik"

And upgrade traefik use helm.

For K3S Users that should come across this that are confused about how to actually implement the fix above, edit /var/lib/rancher/k3s/server/manifests/traefik.yaml and add the above-provided lines, so that it looks roughly like this:

apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: traefik
  namespace: kube-system
spec:
  chart: https://%{KUBERNETES_API}%/static/charts/traefik-1.81.0.tgz
  valuesContent: |-
    rbac:
      enabled: true
.....
    kubernetes:
      ingressEndpoint:
        useDefaultPublishedService: true
    providers:
      kubernetesIngress:
        publishedService:
          enabled: true
.....

Then restart K3S. You should then look at the config map (‘kubectl -n kube-system get cm/traefik -o yaml’) and you’ll see these lines added:

    [kubernetes]
      [kubernetes.ingressEndpoint]
      publishedService = "kube-system/traefik"

Thanks @enys and @FernFerret for the solution!

yes, it’s in the argocd-cm configmap. Here’s the relevant section of the documentation: https://argoproj.github.io/argo-cd/operator-manual/health/#way-1-define-a-custom-health-check-in-argocd-cm-configmap