argo-workflows: BASE_HREF is not working

For those who are experiencing this issue with Treafik Ingress Controller, you will need to create a “StripPrefix” middleware that matches the BASE_HREF value. Make sure to attach that to your ingress (if you are using ingress instead of IngressRoute):

In my case, I am using “/workflows/” as the BASE_HREF value:

apiVersion: traefik.containo.us/v1alpha1 kind: Middleware metadata: name: workflow-prefix spec: stripPrefix: prefixes: - /workflows/

and my Ingress includes the following annotation:

traefik.ingress.kubernetes.io/router.middlewares: argo-workflows-workflow-prefix@kubernetescrd

Hopefully, this will help someone!

@alexec I got it working with this setup

1. BASE_HREF set to subpath with trailing slash (trailing slash is an important one because of how <base href> tag works, I guess we can just update docs here https://github.com/argoproj/argo/blob/676868f31da1bce361e89bebfa1eea81471784ac/docs/argo-server.md#base-href from /argo to /argo/ - that’s what got me confused at first), or maybe just add a check to always append a slash if it’s not present.
2. No updates to argo Service, just put it behind Ingress with a similar config:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    external-dns.alpha.kubernetes.io/alias: "true"
    external-dns.alpha.kubernetes.io/target: target.example.com
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/backend-protocol: HTTP
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
    nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
  rules:
  - host: server.example.com
    http:
      paths:
      - backend:
          serviceName: argo-server
          servicePort: 2746
        path: /argo/(.*)
      - backend:
          serviceName: argo-server
          servicePort: 2746
        path: /argo
  tls:
  - hosts:
    - server.example.com
    secretName: some-cert-manager-seceret

Does anybody has a similar one for AWS LoadBalancer Controller, instead of nginx controller?

After a day of trying various permutations, here is a config that works for me. I am using k3s, version v1.24.4+k3s1, with the default traefik deployment.

Reproducible example -

1. Create the following files:

Argo patch file:

riaz@server:~/bin/k8s$ cat deployments/argo-patch.yaml 
spec:
  template:
    spec:
      containers:
      - name: argo-server
        args:
          - server
          - "--auth-mode=server"  # not relevant to reprex
          - "--secure=false"           # not relevant to reprex
        env:
          - name: BASE_HREF
            value: /argo/
        readinessProbe:
          httpGet:
            scheme: HTTP

Traefik ingress:

riaz@server:~/bin/k8s$ cat deployments/argo-dashboard-ingress.yaml 
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: argo-ingressroute
  namespace: argo
spec:
  entryPoints:
    - web
  routes:
    - match: PathPrefix(`/argo`)
      kind: Rule
      services:
        - name: argo-server
          port: 2746
      middlewares:
        - name: argo-stripprefix
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: argo-stripprefix
  namespace: argo
spec:
  stripPrefix:
    prefixes:
      - /argo
    forceSlash: true

1. Execute the following commands:

kubectl create namespace argo
kubectl apply -n argo -f https://github.com/argoproj/argo-workflows/releases/download/v3.3.9/install.yaml
kubectl patch deployment argo-server --namespace argo --patch-file deployments/argo-patch.yaml
kubectl apply -f deployments/argo-dashboard-ingress.yaml

1. Wait for pods to get into a running state

2. Visit http://[IP_ADDRESS]/argo/

Hope this helps someone with a similar issue.

an updated version that works with k8s 1.27 (at least where I tested):

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: argo-server
  namespace: argo
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$2
    ingress.kubernetes.io/protocol: https # Traefik
    nginx.ingress.kubernetes.io/backend-protocol: https # ingress-nginx
spec:
  ingressClassName: nginx
  rules:
    - http:
        paths:
          - backend:
              service:
                name: argo-server
                port:
                  number: 2746
            path: /argo(/|$)(.*)
            pathType: ImplementationSpecific

Rough Istio equivalent of the traefik manifests above in case someone needs it. Istio’s rewrite here is doing the job of traefik’s stripPrefix.

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: argo-gw
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway
  servers:
    - hosts:
        - yoursite.com
      port:
        name: https
        number: 443
        protocol: https
      tls:
        credentialName: argo-tls
        mode: SIMPLE
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: argo-https-vs
  namespace: argo
spec:
  gateways:
    - istio-system/argo-gw
  hosts:
    - yoursite.com
  http:
  - match:
    - uri:
        prefix: /argo/
    rewrite:
      uri: /
    route:
    - destination:
        host: argo-server
        port:
          number: 2746

For anyone who finds their way to this thread - or runs into this problem - here’s a working implementation for a local K3D cluster. It’s all laid out flexibly enough that it should be portable/obvious to understand + obvious to modify or adapt to your own workflow/implementation.

Overview

K3D is a wrapper and set of conveniences for running a fully-dockerized local cluster on Rancher K3s.

- LoadBalancer: Klipper (built-in default LB for K3D/K3s) - Ingress: Traefik - ArgoCD Install Method: Bitnami Helm Chart (I like it, I know it’s not perfect…or from “the source,” but it suits our workflows and needs, so that’s what I chose).

Notes

In this configuration, there are a few things to be aware of.

• ArgoCD is configured to listen and serve on http://kubernetes.docker.internal/argocd/ (be sure to add this to your hosts file on the system where you deploy!)
• SSL/TLS are disabled
• The ingress targets Traefik’s web service, not web-secure (because HTTP)
• The included Helm example also includes a fully-working setup for a configurationManagementPlugin sidecar which you can use for building your charts & doing “last-mile kustomization” in case you don’t want ArgoCD just treating/building your apps as vanilla Kustomize apps

AND ONE MORE THING

YOU CAN have ArgoCD just treat the included Kustomize example as an Argo App of type Kustomize. In order to do so - and not NEED a configurationManagementPlugin sidecar - please see this configMapGenerator…these are required configMap entries for ArgoCD to be able to properly/correctly build Kustomizations that include Helm charts in just one step (through kustomize build --enable-helm .)

See here for documentation:

• https://cloud.google.com/anthos-config-management/docs/how-to/use-repo-kustomize-helm
• https://github.com/kubernetes-sigs/kustomize/blob/master/examples/chart.md
• https://argo-cd.readthedocs.io/en/stable/user-guide/kustomize/

See here for the configMap: https://github.com/armenr/5thK8s/blob/main/dependencies/argo-cd/generators/configmap-argocd-cm.yaml

The goodies

Example Repo: https://github.com/armenr/5thK8s

K3D config: https://github.com/armenr/5thK8s/blob/main/assets/k3d_local.yaml

Traefik Ingress Example: https://github.com/armenr/5thK8s/blob/main/dependencies/argo-cd/patches/traefik-middleware.yaml —> BIG THANK YOU TO @erkerb4

Helm Values for ArgoCD: https://github.com/armenr/5thK8s/blob/main/dependencies/argo-cd/values.yaml

How to Build/Deploy

# go to the directory
cd dependencies/argo-cd

# build via kustomize + built-in helm generator
kustomize build --enable-helm . | kubectl apply -n argocd -f -

@alexec Looking for AWS Loadbalancer Controller. Any one has an example?

@alexec In ideal case it should load the UI on /argo we shouldn’t put extra routes.

@alexec I got it working with this setup

1. BASE_HREF set to subpath with trailing slash (trailing slash is an important one because of how <base href> tag works, I guess we can just update docs here https://github.com/argoproj/argo/blob/676868f31da1bce361e89bebfa1eea81471784ac/docs/argo-server.md#base-href from /argo to /argo/ - that’s what got me confused at first), or maybe just add a check to always append a slash if it’s not present.
2. No updates to argo Service, just put it behind Ingress with a similar config:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    external-dns.alpha.kubernetes.io/alias: "true"
    external-dns.alpha.kubernetes.io/target: target.example.com
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/backend-protocol: HTTP
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
    nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
  rules:
  - host: server.example.com
    http:
      paths:
      - backend:
          serviceName: argo-server
          servicePort: 2746
        path: /argo/(.*)
      - backend:
          serviceName: argo-server
          servicePort: 2746
        path: /argo
  tls:
  - hosts:
    - server.example.com
    secretName: some-cert-manager-seceret

you config saved the day. Thanks a lot. I just had to turn off ssl redirect.