ingress-nginx: Breaking change in helm chart 4.4.1 / controller 1.5.2

As soon as we automatically upgraded to 4.4.1, many of our services were inaccessible. Looked for common denominator, and discovered that Ingress definitions that use multiple paths/prefixes broke in strange and bizarre ways.

They would serve fake cert instead of cert-manager brokered cert, and if we bypassed the fake cert with e.g. curl --insecure, they would nginx 404 on all paths/prefixes.

Here is a sample broken Ingress (identifying information redacted):

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test-api
  namespace: test-production
  annotations:
    kubernetes.io/ingress.class: nginx
    cert-manager.io/cluster-issuer: letsencrypt-prod
    nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
  tls:
  - hosts:
    - test.com
    secretName: test-cert
  rules:
  - host: test.com
    http:
      paths:
      - path: /?(.*)
        pathType: Prefix
        backend:
          service:
            name: mirror
            port:
              number: 80
      - path: /v1/?(.*)
        pathType: Prefix
        backend:
          service:
            name: test-api
            port:
              number: 80

This is on DOKS v1.24.4.

This caused major outages for us across the board, and was extremely insidious to debug.

There were no errors in the ingress controller pods. No errors in the ingress descriptions (k describe ingress). No issue modifying the ingress definitions (k apply -f). No issues in the compiled nginx configs in the ingress controller pods. Because it was so insidious to debug this issue, we had a long downtime and huge impact.

Reverting to 4.4.0 immediately solved the issues.