ingress-nginx: Large working ingress working with Modsecurity set to DetectionOnly, but ignored when set On

What happened:

We have several Azure AKS clusters installed using Nginx as the ingress controller and using the built-in Modsecurity for WAF functionallity. All clusters have Modsecurity set to “SecRuleEngine On” and all are fully functional except our largest production environment. The major difference is the large ingress definition in the production cluster. It has many host definitions, each having a certificate secret assigned. The large ingress definition works fine with Modsecurity set to “SecRuleEngine DetectionOnly”, but once we set Modsecuriy to "Modsecurity set to “SecRuleEngine On”, the ingress definition is ignored and all sites are down. While Modsecurity is set “SecRuleEngine On” in this production environment, If I cut the number of hosts down to just a few, it works again. When original large ingress is in place, I can only get sites up with Modsecutiy set to “SecRuleEngine DetectionOnly”

What you expected to happen:

I expect large ingress to work with all sites responding when Modsecurity is set “SecRuleEngine On”

NGINX Ingress controller version (exec into the pod and run nginx-ingress-controller --version.):

Release:       v1.6.4
 Build:         69e8833858fb6bda12a44990f1d5eaa7b13f4b75
 Repository:    https://github.com/kubernetes/ingress-nginx
 nginx version: nginx/1.21.6

Kubernetes version (use kubectl version):

v1.25.6

Environment:

  • Cloud provider or hardware configuration: Azure
  • OS (e.g. from /etc/os-release):
  • Kernel (e.g. uname -a):
  • Install tools:
    • Please mention how/where was the cluster created like kubeadm/kops/minikube/kind etc. AKS cluster
  • Basic cluster related info:
    • kubectl version v1.25.6
    • kubectl get nodes -o wide
NAME                                STATUS   ROLES   AGE     VERSION   INTERNAL-IP    EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME
aks-consent-40674510-vmss000008     Ready    agent   50d     v1.25.6   10.0.114.159   <none>        Ubuntu 22.04.2 LTS   5.15.0-1035-azure   containerd://1.6.18+azure-1
aks-consent-40674510-vmss000009     Ready    agent   50d     v1.25.6   10.0.114.190   <none>        Ubuntu 22.04.2 LTS   5.15.0-1035-azure   containerd://1.6.18+azure-1
aks-pool1-40674510-vmss0001km       Ready    agent   50d     v1.25.6   10.0.114.148   <none>        Ubuntu 22.04.2 LTS   5.15.0-1035-azure   containerd://1.6.18+azure-1
aks-pool1-40674510-vmss0001ry       Ready    agent   50d     v1.25.6   10.0.112.115   <none>        Ubuntu 22.04.2 LTS   5.15.0-1035-azure   containerd://1.6.18+azure-1
aks-pool1-40674510-vmss0002e0       Ready    agent   27d     v1.25.6   10.0.113.77    <none>        Ubuntu 22.04.2 LTS   5.15.0-1035-azure   containerd://1.6.18+azure-1
aks-pool1-40674510-vmss0002vx       Ready    agent   7d19h   v1.25.6   10.0.113.82    <none>        Ubuntu 22.04.2 LTS   5.15.0>1035-azure   containerd://1.6.18+azure-1
aks-pool1-40674510-vmss0003ao       Ready    agent   59m     v1.25.6   10.0.112.5     <none>        Ubuntu 22.04.2 LTS   5.15.0-1035-azure   containerd://1.6.18+azure-1
aks-pool1-40674510-vmss0003ap       Ready    agent   58m     v1.25.6   10.0.113.193   <none>        Ubuntu 22.04.2 LTS   5.15.0-1035-azure   containerd://1.6.18+azure-1
aks-pool1-40674510-vmss0003aq       Ready    agent   58m     v1.25.6   10.0.115.146   <none>        Ubuntu 22.04.2 LTS   5.15.0-1035-azure   containerd://1.6.18+azure-1
aks-truyodata-28269258-vmss000000   Ready    agent   50d     v1.25.6   10.0.115.38    <none>        Ubuntu 22.04.2 LTS   5.15.0-1035-azure   containerd://1.6.18+azure-1
aks-truyodata-28269258-vmss000001   Ready    agent   50d     v1.25.6   10.0.115.67    <none>        Ubuntu 22.04.2 LTS   5.15.0-1035-azure   containerd://1.6.18+azure-1
aks-truyodata-28269258-vmss000002   Ready    agent   50d     v1.25.6   10.0.115.96    <none>        Ubuntu 22.04.2 LTS   5.15.0-1035-azure   containerd://1.6.18+azure-1
  • How was the ingress-nginx-controller installed:
    • If helm was used then please show output of helm ls -A | grep -i ingress ngress-nginx-4.5.2 1.6.4
    • If helm was used then please show output of helm -n <ingresscontrollernamepspace> get values <helmreleasename>
    USER-SUPPLIED VALUES:
controller:
  config:
    enable-modsecurity: true
    enable-owasp-modsecurity-crs: true
    http-snippet: |
      map $geoip2_country_code $allowed_country {
       default yes;
       CN no;
       SG no;
       PH no;
      }
    location-snippet: |
      # location block
      if ($allowed_country = no) {
       return 403;
      }
    proxy-body-size: 100m
    use-geoip: false
    use-geoip2: true
  extraArgs:
    default-ssl-certificate: default/truyo-com-certificate
    maxmind-edition-ids: GeoLite2-City,GeoLite2-ASN,GeoLite2-Country
  extraVolumeMounts:
  - mountPath: /etc/nginx/modsecurity/modsecurity.conf
    name: modsecurity-config
    subPath: modsecurity.conf
  - mountPath: /etc/nginx/owasp-modsecurity-crs/crs-setup.conf
    name: crs-setup-config
    subPath: crs-setup.conf
  - mountPath: /etc/nginx/owasp-modsecurity-crs/rules/RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf
    name: response-999-exclusion-rules-after-crs-config
    subPath: RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf
  extraVolumes:
  - configMap:
      name: modsecurity-config
    name: modsecurity-config
  - configMap:
      name: crs-setup-config
    name: crs-setup-config
  - configMap:
      name: response-999-exclusion-rules-after-crs-config
    name: response-999-exclusion-rules-after-crs-config
  kind: DaemonSet
  maxmindLicenseKey: **********************
  service:
    externalTrafficPolicy: Local

  • If helm was not used, then copy/paste the complete precise command used to install the controller, along with the flags and options used

  • if you have more than one instance of the ingress-nginx-controller installed in the same cluster, please provide details for all the instances

  • Current State of the controller:

    • kubectl describe ingressclasses
    Name:         nginx
    Labels:       app.kubernetes.io/component=controller
              app.kubernetes.io/instance=nginx
              app.kubernetes.io/managed-by=Helm
              app.kubernetes.io/name=ingress-nginx
              app.kubernetes.io/part-of=ingress-nginx
              app.kubernetes.io/version=1.6.4
              helm.sh/chart=ingress-nginx-4.5.2
              helm.toolkit.fluxcd.io/name=nginx
              helm.toolkit.fluxcd.io/namespace=nginx
    Annotations:  meta.helm.sh/release-name: nginx
              meta.helm.sh/release-namespace: nginx
   Controller:   k8s.io/ingress-nginx
   Events:       <none>
  • kubectl -n <ingresscontrollernamespace> get all -A -o wide
NAME                                       READY   STATUS    RESTARTS   AGE   IP             NODE                            NOMINATED NODE   READINESS GATES
pod/nginx-ingress-nginx-controller-45txv   1/1     Running   0          18d   10.0.114.250   aks-pool1-40674510-vmss0001km   <none>           <none>
pod/nginx-ingress-nginx-controller-5jtdf   1/1     Running   0          31m   10.0.112.37    aks-pool1-40674510-vmss0003bx   <none>           <none>
pod/nginx-ingress-nginx-controller-697cd   1/1     Running   0          18d   10.0.112.216   aks-pool1-40674510-vmss0001ry   <none>           <none>
pod/nginx-ingress-nginx-controller-78ksx   1/1     Running   0          18d   10.0.114.60    aks-pool1-40674510-vmss0002e0   <none>           <none>
pod/nginx-ingress-nginx-controller-fvnwg   1/1     Running   0          31m   10.0.112.237   aks-pool1-40674510-vmss0003by   <none>           <none>
pod/nginx-ingress-nginx-controller-mj569   1/1     Running   0          8d    10.0.113.126   aks-pool1-40674510-vmss0002vx   <none>           <none>
pod/nginx-ingress-nginx-controller-qzhq2   1/1     Running   0          31m   10.0.116.153   aks-pool1-40674510-vmss0003c0   <none>           <none>
pod/nginx-ingress-nginx-controller-vl979   1/1     Running   0          31m   10.0.114.28    aks-pool1-40674510-vmss0003bz   <none>           <none>

NAME                                               TYPE           CLUSTER-IP       EXTERNAL-IP     PORT(S)                      AGE    SELECTOR
service/nginx-ingress-nginx-controller             LoadBalancer   10.100.193.182   *.*.*.*   80:30497/TCP,443:32132/TCP   585d   app.kubernetes.io/component=controller,app.kubernetes.io/instance=nginx,app.kubernetes.io/name=ingress-nginx
service/nginx-ingress-nginx-controller-admission   ClusterIP      10.100.212.203   <none>          443/TCP                      585d   app.kubernetes.io/component=controller,app.kubernetes.io/instance=nginx,app.kubernetes.io/name=ingress-nginx

NAME                                            DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE    CONTAINERS   IMAGES                                                                                                                    SELECTOR
daemonset.apps/nginx-ingress-nginx-controller   8         8         8       8            8           kubernetes.io/os=linux   534d   controller   registry.k8s.io/ingress-nginx/controller:v1.6.4@sha256:15be4666c53052484dd2992efacf2f50ea77a78ae8aa21ccd91af6baaa7ea22f   app.kubernetes.io/component=controller,app.kubernetes.io/instance=nginx,app.kubernetes.io/name=ingress-nginx
  • kubectl -n <ingresscontrollernamespace> describe po <ingresscontrollerpodname>
Name:         nginx-ingress-nginx-controller-45txv
Namespace:    nginx
Priority:     0
Node:         aks-pool1-40674510-vmss0001km/10.0.114.148
Start Time:   Sun, 28 May 2023 22:38:07 -0700
Labels:       app.kubernetes.io/component=controller
              app.kubernetes.io/instance=nginx
              app.kubernetes.io/name=ingress-nginx
              controller-revision-hash=7fd584fcdb
              pod-template-generation=10
Annotations:  kubectl.kubernetes.io/restartedAt: 2023-05-28T22:34:37-07:00
Status:       Running
IP:           10.0.114.250
IPs:
  IP:           10.0.114.250
Controlled By:  DaemonSet/nginx-ingress-nginx-controller
Containers:
  controller:
    Container ID:  containerd://c0b151ca145f365f36e7c22454ddba076877cd31e27ac9a7d784391e9a207639
    Image:         registry.k8s.io/ingress-nginx/controller:v1.6.4@sha256:15be4666c53052484dd2992efacf2f50ea77a78ae8aa21ccd91af6baaa7ea22f
    Image ID:      registry.k8s.io/ingress-nginx/controller@sha256:15be4666c53052484dd2992efacf2f50ea77a78ae8aa21ccd91af6baaa7ea22f
    Ports:         80/TCP, 443/TCP, 8443/TCP
    Host Ports:    0/TCP, 0/TCP, 0/TCP
    Args:
      /nginx-ingress-controller
      --publish-service=$(POD_NAMESPACE)/nginx-ingress-nginx-controller
      --election-id=nginx-ingress-nginx-leader
      --controller-class=k8s.io/ingress-nginx
      --ingress-class=nginx
      --configmap=$(POD_NAMESPACE)/nginx-ingress-nginx-controller
      --validating-webhook=:8443
      --validating-webhook-certificate=/usr/local/certificates/cert
      --validating-webhook-key=/usr/local/certificates/key
      --maxmind-license-key=********************
      --default-ssl-certificate=default/truyo-com-certificate
      --maxmind-edition-ids=GeoLite2-City,GeoLite2-ASN,GeoLite2-Country
    State:          Running
      Started:      Sun, 28 May 2023 22:38:08 -0700
    Ready:          True
    Restart Count:  0
    Requests:
      cpu:      100m
      memory:   90Mi
    Liveness:   http-get http://:10254/healthz delay=10s timeout=1s period=10s #success=1 #failure=5
    Readiness:  http-get http://:10254/healthz delay=10s timeout=1s period=10s #success=1 #failure=3
    Environment:
      POD_NAME:       nginx-ingress-nginx-controller-45txv (v1:metadata.name)
      POD_NAMESPACE:  nginx (v1:metadata.namespace)
      LD_PRELOAD:     /usr/local/lib/libmimalloc.so
    Mounts:
      /etc/nginx/modsecurity/modsecurity.conf from modsecurity-config (rw,path="modsecurity.conf")
      /etc/nginx/owasp-modsecurity-crs/crs-setup.conf from crs-setup-config (rw,path="crs-setup.conf")
      /etc/nginx/owasp-modsecurity-crs/rules/RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf from response-999-exclusion-rules-after-crs-config (rw,path="RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf")
      /usr/local/certificates/ from webhook-cert (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-2586k (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  webhook-cert:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  nginx-ingress-nginx-admission
    Optional:    false
  modsecurity-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      modsecurity-config
    Optional:  false
  crs-setup-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      crs-setup-config
    Optional:  false
  response-999-exclusion-rules-after-crs-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      response-999-exclusion-rules-after-crs-config
    Optional:  false
  kube-api-access-2586k:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              kubernetes.io/os=linux
Tolerations:                 node.kubernetes.io/disk-pressure:NoSchedule op=Exists
                             node.kubernetes.io/memory-pressure:NoSchedule op=Exists
                             node.kubernetes.io/not-ready:NoExecute op=Exists
                             node.kubernetes.io/pid-pressure:NoSchedule op=Exists
                             node.kubernetes.io/unreachable:NoExecute op=Exists
                             node.kubernetes.io/unschedulable:NoSchedule op=Exists
Events:                      <none>
  • kubectl -n <ingresscontrollernamespace> describe svc <ingresscontrollerservicename>
Name:                     nginx-ingress-nginx-controller
Namespace:                nginx
Labels:                   app.kubernetes.io/component=controller
                          app.kubernetes.io/instance=nginx
                          app.kubernetes.io/managed-by=Helm
                          app.kubernetes.io/name=ingress-nginx
                          app.kubernetes.io/part-of=ingress-nginx
                          app.kubernetes.io/version=1.6.4
                          helm.sh/chart=ingress-nginx-4.5.2
                          helm.toolkit.fluxcd.io/name=nginx
                          helm.toolkit.fluxcd.io/namespace=nginx
Annotations:              meta.helm.sh/release-name: nginx
                          meta.helm.sh/release-namespace: nginx
Selector:                 app.kubernetes.io/component=controller,app.kubernetes.io/instance=nginx,app.kubernetes.io/name=ingress-nginx
Type:                     LoadBalancer
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.100.193.182
IPs:                      10.100.193.182
LoadBalancer Ingress:     *.*.*.*
Port:                     http  80/TCP
TargetPort:               http/TCP
NodePort:                 http  30497/TCP
Endpoints:                10.0.112.216:80,10.0.112.237:80,10.0.112.37:80 + 5 more...
Port:                     https  443/TCP
TargetPort:               https/TCP
NodePort:                 https  32132/TCP
Endpoints:                10.0.112.216:443,10.0.112.237:443,10.0.112.37:443 + 5 more...
Session Affinity:         None
External Traffic Policy:  Local
HealthCheck NodePort:     30639
Events:
  Type     Reason                        Age                   From                       Message
  ----     ------                        ----                  ----                       -------
  Warning  FailedToUpdateEndpointSlices  54m                   endpoint-slice-controller  Error updating Endpoint Slices for Service nginx/nginx-ingress-nginx-controller: [skipping Pod nginx-ingress-nginx-controller-zgh4c for Service nginx/nginx-ingress-nginx-controller: Node aks-pool1-40674510-vmss0003bp Not Found, skipping Pod nginx-ingress-nginx-controller-tm4c9 for Service nginx/nginx-ingress-nginx-controller: Node aks-pool1-40674510-vmss0003bt Not Found]
  Warning  FailedToUpdateEndpointSlices  53m (x4 over 54m)     endpoint-slice-controller  Error updating Endpoint Slices for Service nginx/nginx-ingress-nginx-controller: skipping Pod nginx-ingress-nginx-controller-zgh4c for Service nginx/nginx-ingress-nginx-controller: Node aks-pool1-40674510-vmss0003bp Not Found
  Normal   UpdatedLoadBalancer           17m (x29 over 4h29m)  service-controller         Updated load balancer with new hosts
  Warning  FailedToUpdateEndpointSlices  6m44s                 endpoint-slice-controller  Error updating Endpoint Slices for Service nginx/nginx-ingress-nginx-controller: skipping Pod nginx-ingress-nginx-controller-65b46 for Service nginx/nginx-ingress-nginx-controller: Node aks-pool1-40674510-vmss0003bv Not Found
  • Current state of ingress object, if applicable:
    • kubectl -n <appnnamespace> get all,ing -o wide
    • kubectl -n <appnamespace> describe ing <ingressname>
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    kubernetes.io/ingress.class: "nginx"
    ingress.kubernetes.io/force-ssl-redirect: "true"
  name: gdpr-gateway
  labels:
    app: kong
spec:
  tls:
  - hosts:
      - "*.wildcard.com"
    secretName: wildcard-com-certificate
  - hosts:
      - "customer1.com"
    secretName: customer1-certificate
  - hosts:
      - "customer2.com"
    secretName: customer2-tls-certs-secret
  - hosts:
      - "customer3.com"
    secretName: customer3-tls-certs-secret
  - hosts:
     - "customer4.com"
    secretName: customer4-tls-certs-secret
  - hosts:
     - "customer5.com" 
    secretName: customer5-certificate
  - hosts:
     - "customer6.com" 
    secretName: customer6-tls-certs-secret
  - hosts:
     - "customer7.com" 
    secretName: customer7-tls-certs-secret
  - hosts:
     - "customer8.com" 
    secretName: customer8-certificate
  - hosts:
     - "customer9.com" 
    secretName: customer9-certificate
  - hosts:
     - "customer10.com"
    secretName: customer10-certificate
  - hosts:
     - "customer11.com"
    secretName: customer11-tls-certs-secret
  - hosts:
     - "customer12.com"
    secretName: customer12-certificate
  - hosts:
     - "customer13.com"
    secretName: customer13-certificate
  - hosts:
     - "customer14.com"
    secretName: wildcard-com-certificate
  - hosts:
     - "customer15.com"
    secretName: customer15-certificate
  - hosts:
     - "customer16.com"
    secretName: customer16-certificate
  - hosts:
     - "customer17.com"
    secretName: customer17-certificate
  - hosts:
     - "customer18.com"
    secretName: customer18-certificate
  - hosts:
     - "customer19.com"
    secretName: customer19-certificate
  - hosts:
     - "customer20.com"
    secretName: customer20-tls-certs-secret
  - hosts:
     - "customer21.com"
    secretName: customer21-tls-certs-secret
  - hosts:
     - "customer22.com"
    secretName: customer22-certificate
  - hosts:
     - "customer23.com"
    secretName: customer23-tls-certs-secret
  - hosts:
     - "customer24.com"
    secretName: customer24-tls-certs-secret
  - hosts:
     - "customer25.com"
    secretName: customer25-tls-certs-secret
  - hosts:
     - "customer26.com"
    secretName: customer26-certificate
  - hosts:
     - "customer27.com"
    secretName: customer27-certificate
  - hosts:
     - "customer28.carvana.com"
    secretName: customer28-certificate
  - hosts:
     - "customer29.com"
    secretName: customer29-certificate 
  - hosts:
     - "customer30.com"
    secretName: customer30-certificate    
  rules:
    - host: "*.wildcard.com"
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer1.com"
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer2.com"
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer3.com"
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer4.com"
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer5.com"
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer6.com"
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer7.com"
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer8.com"
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer9.com"
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer10.com"                                                                                                                                              
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer11.com"                                                                                                                                              
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer12.com"                                                                                                                                              
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer13.com"                                                                                                                                              
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer14.com"                                                                                                                                              
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer15.com"                                                                                                                                              
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer16.com"
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer17.com"
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80                         
    - host: "customer18.com"
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80               
    - host: "customer19.com"
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer20.com"
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer21.com"
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer22.com"
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer23.com"
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer24.com"
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer25.com"
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer26.com"
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer27.com"
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer28.com"
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80
    - host: "customer29.com"
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80                
    - host: "customer30.com"
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gdpr-gateway-kong-proxy
              port:
                number: 80             
  • If applicable, then, your complete and exact curl/grpcurl command (redacted if required) and the reponse to the curl/grpcurl command with the -v flag

  • Others:

    • Any other related information like ;
      • copy/paste of the snippet (if applicable)
      • kubectl describe ... of any custom configmap(s) created and in use

See attached configmap yaml files

- Any other related information that may help

How to reproduce this issue:

Anything else we need to know:

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 31 (31 by maintainers)

Most upvoted comments

Success - with Modsecurity turned off in nginx.tmpl “server” section, we were able to set Modsecurity to “SecRuleEngine On” and all ingresses responded appropriately. I tested a mock WAF violation against site and received expected 403 from Modsecurity. Thanks for all the help on this. If you want me to submit this change as a PR, let me know and I will go through documented process.

We discussed this on the community call and believe we found the issue.

Mod security is working properly but it is blocking the controllers internal request to update the nginx.conf

POST /configuration/servers HTTP/1.1", host: "127.0.0.1:10246 is the port and uri for the controller to update the conf when there are changes. So enabling modsecurity is blocking that request for being too large. We are going to put in a fix in 1.8.2 to disable modsec for that server block in the nginx.tmpl template.

2023/06/26 05:24:50 [error] 1912#1912: *32231 [client 127.0.0.1] ModSecurity: Access denied with code 400 (phase 2). Matched "Operator `Eq' with parameter `0' against variable `REQBODY_ERROR' (Value: `1' ) [file "/etc/nginx/modsecurity/modsecurity.conf"] [line "76"] [id "200002"] [rev ""] [msg "Failed to parse request body."] [data "Request body excluding files is bigger than the maximum expected."] [severity "2"] [ver ""] [maturity "0"] [accuracy "0"] [hostname "127.0.0.1"] [uri "/configuration/servers"] [unique_id "168775709078.157738"] [ref "v0,1"], client: 127.0.0.1, server: , request: "POST /configuration/servers HTTP/1.1", host: "127.0.0.1:10246"
W0626 05:24:50.571596       7 controller.go:236] Dynamic reconfiguration failed (retrying; 6 retries left): unexpected error code: 400