kubernetes-ingress: Nginx Ingress Controller gives 404 not found

### Cluster information:
Kubernetes version:1.24.9
Bare-Metal being used:  (One master and three workers)
Installation method: Installed using Ansible.
Host OS: Ubuntu 18.04.6 LTS
CNI and version: Flannel ( rancher/mirrored-flannelcni-flannel - v0.19.2 )
CRI and version:  containerd://1.4.6

I have installed Nginx Ingress Controller by using this link https://docs.nginx.com/nginx-ingress-controller/installation/installation-with-manifests

From the above link. i have executed the below commands.

$ git clone https://github.com/nginxinc/kubernetes-ingress.git --branch v3.0.0
$ cd kubernetes-ingress/deployments
$ kubectl apply -f common/ns-and-sa.yaml
$ kubectl apply -f rbac/rbac.yaml

$ kubectl apply -f common/default-server-secret.yaml

$ kubectl apply -f common/nginx-config.yaml
$ kubectl apply -f common/ingress-class.yaml

$ kubectl apply -f common/crds/k8s.nginx.org_virtualservers.yaml
$ kubectl apply -f common/crds/k8s.nginx.org_virtualserverroutes.yaml
$ kubectl apply -f common/crds/k8s.nginx.org_transportservers.yaml
$ kubectl apply -f common/crds/k8s.nginx.org_policies.yaml

$ kubectl apply -f common/crds/k8s.nginx.org_globalconfigurations.yaml

$ kubectl apply -f daemon-set/nginx-ingress.yaml

It is runs as a daemon set,

$ kubectl get all -n nginx-ingress
NAME                      READY   STATUS    RESTARTS   AGE
pod/nginx-ingress-69z69   1/1     Running   0          26m
pod/nginx-ingress-l7z86   1/1     Running   0          26m
pod/nginx-ingress-wsjfv   1/1     Running   0          26m

NAME                           DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
daemonset.apps/nginx-ingress   3         3         3       3            3           <none>          26m
$ kubectl get svc  --all-namespaces
NAMESPACE     NAME          TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                  AGE
default       kubernetes    ClusterIP   10.96.0.1       <none>        443/TCP                  3d1h
jenkins       jenkins-svc   ClusterIP   10.103.54.142   <none>        80/TCP                   3d1h
kube-system   kube-dns      ClusterIP   10.96.0.10      <none>        53/UDP,53/TCP,9153/TCP   3d1h
$ kubectl get ingress --all-namespaces
NAMESPACE   NAME              CLASS    HOSTS                ADDRESS   PORTS   AGE
jenkins     jenkins-ingress   <none>   jenkins.company.com             80      19h
$ kubectl describe ingress jenkins-ingress -n jenkins
Name:             jenkins-ingress
Labels:           <none>
Namespace:        jenkins
Address:
Ingress Class:    nginx
Default backend:  <default>
Rules:
  Host             Path  Backends
  ----             ----  --------
  jenkins.company.com
                   /jenkins   jenkins-svc:80 (10.244.2.19:8080)
Annotations:       nginx.ingress.kubernetes.io/rewrite-target: /$2
Events:
  Type    Reason          Age   From                      Message
  ----    ------          ----  ----                      -------
  Normal  AddedOrUpdated  9m4s  nginx-ingress-controller  Configuration for jenkins/jenkins-ingress was added or updated
  Normal  AddedOrUpdated  9m4s  nginx-ingress-controller  Configuration for jenkins/jenkins-ingress was added or updated
  Normal  AddedOrUpdated  9m4s  nginx-ingress-controller  Configuration for jenkins/jenkins-ingress was added or updated
$ kubectl exec -it -n nginx-ingress nginx-ingress-l7z86 -- curl -v -H "Host: nginx" localhost:80
*   Trying ::1:80...
* Connected to localhost (::1) port 80 (#0)
> GET / HTTP/1.1
> Host: nginx
> User-Agent: curl/7.74.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 404 Not Found
< Server: nginx/1.23.3
< Date: Fri, 13 Jan 2023 11:06:53 GMT
< Content-Type: text/html
< Content-Length: 153
< Connection: keep-alive
<
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.23.3</center>
</body>
</html>
* Connection #0 to host localhost left intact

My Jenkins service yaml file as follows,

apiVersion: v1
kind: Service
metadata:
  name: jenkins-svc
  namespace: jenkins
  labels:
    app: jenkins
spec:
  selector: 
    app: jenkins
  type: ClusterIP
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080

My ingress-resource.yaml file as follows

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$2
  name: jenkins-ingress
  namespace: jenkins
spec:
  ingressClassName: nginx
  rules:
  - host: jenkins.company.com
    http:
      paths:
      - path: /jenkins
        pathType: Prefix
        backend:
          service:
            name: jenkins-svc
            port: 
              number: 80

When i tried to access our jenkins URL http://jenkins.company.com/jenkins getting 404 Not Found error on browser.

What could the problem? Also let me know Am i missing any additional installation commands?

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 22 (11 by maintainers)

Most upvoted comments

Thanks for the information. I shall check and come back soon as possible. Thank you!

Hi Shaun, Yes it was resolved and thanks a lot for your kind support šŸ˜€. Appreciate your help on this

Hi Shaun,

Requested output as follows.

$ kubectl exec -it -n nginx-ingress nginx-ingress-l7z86 -- nginx -T
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
# configuration file /etc/nginx/nginx.conf:

worker_processes  auto;
daemon off;

error_log  stderr notice;
pid        /var/lib/nginx/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    map $upstream_trailer_grpc_status $grpc_status {
        default $upstream_trailer_grpc_status;
        '' $sent_http_grpc_status;
    }


    access_log  /dev/stdout  main;




    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout 65s;
    keepalive_requests 100;

    #gzip  on;

    server_names_hash_max_size 1024;
    server_names_hash_bucket_size 256;

    variables_hash_bucket_size 256;
    variables_hash_max_size 1024;

    map $request_uri $request_uri_no_args {
        "~^(?P<path>[^?]*)(\?.*)?$" $path;
    }

    map $http_upgrade $connection_upgrade {
        default upgrade;
        ''      close;
    }
    map $http_upgrade $vs_connection_header {
        default upgrade;
        ''      $default_connection_header;
    }








    server {
        # required to support the Websocket protocol in VirtualServer/VirtualServerRoutes
        set $default_connection_header "";
        set $resource_type "";
        set $resource_name "";
        set $resource_namespace "";
        set $service "";

        listen 80 default_server;
        listen [::]:80 default_server;


        listen 443 ssl default_server;
        listen [::]:443 ssl default_server;



        ssl_certificate /etc/nginx/secrets/default;
        ssl_certificate_key /etc/nginx/secrets/default;






        server_name _;
        server_tokens "on";






        location / {
            return 404;
        }
    }
    # stub_status
    server {
        listen 8080;
        listen [::]:8080;


        allow 127.0.0.1;
        allow ::1;

        deny all;

        location /stub_status {
            stub_status;
        }
    }

    include /etc/nginx/config-version.conf;
    include /etc/nginx/conf.d/*.conf;

    server {
        listen unix:/var/lib/nginx/nginx-502-server.sock;
        access_log off;



        return 502;
    }

    server {
        listen unix:/var/lib/nginx/nginx-418-server.sock;
        access_log off;



        return 418;
    }
}

stream {
    log_format  stream-main  '$remote_addr [$time_local] '
                      '$protocol $status $bytes_sent $bytes_received '
                      '$session_time "$ssl_preread_server_name"';

    access_log  /dev/stdout  stream-main;



    map_hash_max_size 2048;
    map_hash_bucket_size 256;



    include /etc/nginx/stream-conf.d/*.conf;
}

# configuration file /etc/nginx/mime.types:

types {
    text/html                                        html htm shtml;
    text/css                                         css;
    text/xml                                         xml;
    image/gif                                        gif;
    image/jpeg                                       jpeg jpg;
    application/javascript                           js;
    application/atom+xml                             atom;
    application/rss+xml                              rss;

    text/mathml                                      mml;
    text/plain                                       txt;
    text/vnd.sun.j2me.app-descriptor                 jad;
    text/vnd.wap.wml                                 wml;
    text/x-component                                 htc;

    image/avif                                       avif;
    image/png                                        png;
    image/svg+xml                                    svg svgz;
    image/tiff                                       tif tiff;
    image/vnd.wap.wbmp                               wbmp;
    image/webp                                       webp;
    image/x-icon                                     ico;
    image/x-jng                                      jng;
    image/x-ms-bmp                                   bmp;

    font/woff                                        woff;
    font/woff2                                       woff2;

    application/java-archive                         jar war ear;
    application/json                                 json;
    application/mac-binhex40                         hqx;
    application/msword                               doc;
    application/pdf                                  pdf;
    application/postscript                           ps eps ai;
    application/rtf                                  rtf;
    application/vnd.apple.mpegurl                    m3u8;
    application/vnd.google-earth.kml+xml             kml;
    application/vnd.google-earth.kmz                 kmz;
    application/vnd.ms-excel                         xls;
    application/vnd.ms-fontobject                    eot;
    application/vnd.ms-powerpoint                    ppt;
    application/vnd.oasis.opendocument.graphics      odg;
    application/vnd.oasis.opendocument.presentation  odp;
    application/vnd.oasis.opendocument.spreadsheet   ods;
    application/vnd.oasis.opendocument.text          odt;
    application/vnd.openxmlformats-officedocument.presentationml.presentation
                                                     pptx;
    application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
                                                     xlsx;
    application/vnd.openxmlformats-officedocument.wordprocessingml.document
                                                     docx;
    application/vnd.wap.wmlc                         wmlc;
    application/wasm                                 wasm;
    application/x-7z-compressed                      7z;
    application/x-cocoa                              cco;
    application/x-java-archive-diff                  jardiff;
    application/x-java-jnlp-file                     jnlp;
    application/x-makeself                           run;
    application/x-perl                               pl pm;
    application/x-pilot                              prc pdb;
    application/x-rar-compressed                     rar;
    application/x-redhat-package-manager             rpm;
    application/x-sea                                sea;
    application/x-shockwave-flash                    swf;
    application/x-stuffit                            sit;
    application/x-tcl                                tcl tk;
    application/x-x509-ca-cert                       der pem crt;
    application/x-xpinstall                          xpi;
    application/xhtml+xml                            xhtml;
    application/xspf+xml                             xspf;
    application/zip                                  zip;

    application/octet-stream                         bin exe dll;
    application/octet-stream                         deb;
    application/octet-stream                         dmg;
    application/octet-stream                         iso img;
    application/octet-stream                         msi msp msm;

    audio/midi                                       mid midi kar;
    audio/mpeg                                       mp3;
    audio/ogg                                        ogg;
    audio/x-m4a                                      m4a;
    audio/x-realaudio                                ra;

    video/3gpp                                       3gpp 3gp;
    video/mp2t                                       ts;
    video/mp4                                        mp4;
    video/mpeg                                       mpeg mpg;
    video/quicktime                                  mov;
    video/webm                                       webm;
    video/x-flv                                      flv;
    video/x-m4v                                      m4v;
    video/x-mng                                      mng;
    video/x-ms-asf                                   asx asf;
    video/x-ms-wmv                                   wmv;
    video/x-msvideo                                  avi;
}

# configuration file /etc/nginx/config-version.conf:
server {
    listen unix:/var/lib/nginx/nginx-config-version.sock;
        access_log off;



    location /configVersion {
        return 200 2;
    }
}
map $http_x_expected_config_version $config_version_mismatch {
        "2" "";
        default "mismatch";
}
# configuration file /etc/nginx/conf.d/jenkins-jenkins-ingress.conf:
# configuration for jenkins/jenkins-ingress

upstream jenkins-jenkins-ingress-jenkins.company.com-jenkins-svc-80 {
        zone jenkins-jenkins-ingress-jenkins.company.com-jenkins-svc-80 256k;
        random two least_conn;

        server 10.244.2.2:8080 max_fails=1 fail_timeout=10s max_conns=0;

}


server {


        listen 80;
        listen [::]:80;








        server_tokens on;

        server_name jenkins.company.com;

        set $resource_type "ingress";
        set $resource_name "jenkins-ingress";
        set $resource_namespace "jenkins";







        location /jenkins {
                set $service "jenkins-svc";


                proxy_http_version 1.1;


                proxy_connect_timeout 60s;
                proxy_read_timeout 60s;
                proxy_send_timeout 60s;
                client_max_body_size 1m;
                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Host $host;
                proxy_set_header X-Forwarded-Port $server_port;
                proxy_set_header X-Forwarded-Proto $scheme;
                proxy_buffering on;

                proxy_pass http://jenkins-jenkins-ingress-jenkins.company.com-jenkins-svc-80;


        }


}

Thanks Shaun for assisting me. My deployment.yml file as follows.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: jenkins
  namespace: jenkins
  labels:
    app: jenkins
spec:
  replicas: 1
  selector:
    matchLabels:
      app: jenkins
  template:
    metadata:
      labels:
        app: jenkins
    spec:
      securityContext:
        fsGroup: 0 
        runAsUser: 0
      serviceAccountName: admin
      containers:
      - name: jenkins
        image: jenkins/jenkins:2.375.2-lts
        imagePullPolicy: IfNotPresent
        securityContext:
          privileged: true
          runAsUser: 0
        ports:
        - containerPort: 8080
        - containerPort: 50000
        env:
        - name: JENKINS_OPTS
          value: "--prefix=/jenkins --httpListenAddress=0.0.0.0"   
        - name: PROVISIONER_NAME
          value: app-nfs  
        - name: NFS_SERVER
          value: 165.13.45.56
        - name: NFS_PATH
          value: /app-nfs/kdata/jenkins
        volumeMounts:
          - name: jenkins-data
            mountPath: /var/jenkins_home         
      volumes:
        - name: jenkins-data
          persistentVolumeClaim:
            claimName: jenkins-pvc-nfs

Thanks for all the details Mohan. I’m looking into this now and will get back to you asap.