oauth2-proxy: Errors Middleware cannot be used without an insecure workaround

There are numerous issues made regarding issues with Traefik and oauth2-proxy, (see: #1639 #1538, #1297, #1255, #1023 etc) this is in part due to the fact that the errors middleware is being used in a way that goes against Traefik’s own documentation (See: https://github.com/traefik/traefik/issues/7872#issuecomment-778202397 ). Part of what the errors middleware does is that it creates it’s own page that is shown to the client, however this then overwrites the headers that were passed from the original request to the errors middleware, leaving the redirection url to not be correct. The errors middleware creates numerous issues and needs to either be removed as a configuration option, or oauth2-proxy needs to be refactored in order to take in the header that Traefik passes directly to it on it’s forward auth request. Adding insult to injury, one of the workarounds that doesn’t involve messing around with templates requires that one trusts all X-Forwarded headers globally, which is a SIGNIFICANT security risk, and could compromise the protection Traefik is meant to provide.

See the below traefik log, showing the request being made:

traefik               | time="2022-08-22T14:53:46-04:00" level=debug msg="vulcand/oxy/roundrobin/rr: completed ServeHttp on request" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/oauth2/auth/\",\"RawPath\":\"\",\"OmitHost\":false,\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\",\"RawFragment\":\"\"},\"Proto\":\"HTTP/2.0\",\"ProtoMajor\":2,\"ProtoMinor\":0,\"Header\":{\"Accept\":[\"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9\"],\"Accept-Encoding\":[\"gzip\"],\"Accept-Language\":[\"en-US,en;q=0.9\"],\"Cdn-Loop\":[\"cloudflare; loops=1\"],\"Cf-Connecting-Ip\":[\"<REDACTED>\"],\"Cf-Ipcountry\":[\"US\"],\"Cf-Ray\":[\"<REDACTED>"],\"Cf-Visitor\":[\"{\\\"scheme\\\":\\\"https\\\"}\"],\"Cookie\":[\"<REDACTED>"],\"Sec-Ch-Ua\":[\"\\\"Chromium\\\";v=\\\"104\\\", \\\" Not A;Brand\\\";v=\\\"99\\\", \\\"Microsoft Edge\\\";v=\\\"104\\\"\"],\"Sec-Ch-Ua-Mobile\":[\"?0\"],\"Sec-Ch-Ua-Platform\":[\"\\\"Windows\\\"\"],\"Sec-Fetch-Dest\":[\"document\"],\"Sec-Fetch-Mode\":[\"navigate\"],\"Sec-Fetch-Site\":[\"none\"],\"Sec-Fetch-User\":[\"?1\"],\"Upgrade-Insecure-Requests\":[\"1\"],\"User-Agent\":[\"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.81 Safari/537.36 Edg/104.0.1293.47\"],\"X-Forwarded-Host\":[\"oauth.<REDACTED>.org\"],\"X-Forwarded-Port\":[\"443\"],\"X-Forwarded-Proto\":[\"https\"],\"X-Forwarded-Server\":[\"<REDACTED>"],\"X-Real-Ip\":[\"<REDACTED>"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"oauth.<REDACTED>.org\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"<REDACTED>:62252\",\"RequestURI\":\"/oauth2/auth/\",\"TLS\":null}"
traefik               | time="2022-08-22T14:53:46-04:00" level=debug msg="Remote error https://oauth.<REDACTED>.org/oauth2/auth/. StatusCode: 403" middlewareName=oauth-auth@file middlewareType=ForwardedAuthType
traefik               | time="2022-08-22T14:53:48-04:00" level=debug msg="vulcand/oxy/roundrobin/rr: Forwarding this request to URL" ForwardURL="http://172.18.0.4:4180" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/oauth2/start\",\"RawPath\":\"\",\"OmitHost\":false,\"ForceQuery\":false,\"RawQuery\":\"rd=https%3A%2F%2Foauth.<REDACTED>.org%2F\",\"Fragment\":\"\",\"RawFragment\":\"\"},\"Proto\":\"HTTP/2.0\",\"ProtoMajor\":2,\"ProtoMinor\":0,\"Header\":{\"Accept\":[\"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9\"],\"Accept-Encoding\":[\"gzip\"],\"Accept-Language\":[\"en-US,en;q=0.9\"],\"Cdn-Loop\":[\"cloudflare\"],\"Cf-Connecting-Ip\":[\"<REDACTED>\"],\"Cf-Ipcountry\":[\"US\"],\"Cf-Ray\":[\"7<REDACTED>\"],\"Cf-Visitor\":[\"{\\\"scheme\\\":\\\"https\\\"}\"],\"Cookie\":[\"_oauth2_proxy_csrf=<REDACTED>\"],\"Referer\":[\"https://whoami.<REDACTED>.org/\"],\"Sec-Ch-Ua\":[\"\\\"Chromium\\\";v=\\\"104\\\", \\\" Not A;Brand\\\";v=\\\"99\\\", \\\"Microsoft Edge\\\";v=\\\"104\\\"\"],\"Sec-Ch-Ua-Mobile\":[\"?0\"],\"Sec-Ch-Ua-Platform\":[\"\\\"Windows\\\"\"],\"Sec-Fetch-Dest\":[\"document\"],\"Sec-Fetch-Mode\":[\"navigate\"],\"Sec-Fetch-Site\":[\"same-origin\"],\"Sec-Fetch-User\":[\"?1\"],\"Upgrade-Insecure-Requests\":[\"1\"],\"User-Agent\":[\"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.81 Safari/537.36 Edg/104.0.1293.47\"],\"X-Forwarded-Host\":[\"**whoami.<REDACTED>.org**\"],\"X-Forwarded-Port\":[\"443\"],\"X-Forwarded-Proto\":[\"https\"],\"X-Forwarded-Server\":[\"<REDACTED>\"],\"X-Real-Ip\":[\"<REDACTED>\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"whoami.<REDACTED>.org\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"<REDACTED>:53834\",\"RequestURI\":\"/oauth2/start?rd=https%3A%2F%2Foauth.<REDACTED>.org%2F\",\"TLS\":null}"
oauth-forwardauth     | <REDACTED> - d2107b65-d30b-4b5b-86f6-30262e75800e - - [2022/08/22 18:53:48] whoami.<REDACTED>.org GET - "/oauth2/start?rd=https%3A%2F%2Foauth.<REDACTED>.org%2F" HTTP/1.1 "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.81 Safari/537.36 Edg/104.0.1293.47" 302 493 0.000

Expected Behavior

oauth2-proxy should be getting the redirect url from the request traefik makes to it over port 4180, not on a separate request made before the forward authentication request. Unless that change is made, dynamic configuration using the errors middleware is not only not possible, but potentially could expose unknowing users to significant security risks. oauth2-proxy should be able to dynamically assign the rd value by parsing the request made by traefik. (NOTE: This solution would break if the provider button is skipped. As we don’t stay on the original url before the forward auth request is made)

As it stands, the errors middleware is providing a significant amount of confusion and needless troubleshooting for those on traefik. The errors middleware configuration should be removed from documentation, and the example configuration should be update to include a similar setup to https://github.com/oauth2-proxy/oauth2-proxy/issues/1639#issuecomment-1224763241

Current Behavior

oauth2-proxy only redirects to its own host address, unless insecure headers are enabled globally when using the errors middleware, which is presented as the primary configuration option for Traefik

Possible Solution

Dynamically parse and then fill the rd value by reading the X-Forwarded-Host header from the request sent by Traefik to oauth2-proxy

Remove all mention of the errors middleware, and replace it with a similar configuration to the one in https://github.com/oauth2-proxy/oauth2-proxy/issues/1639#issuecomment-1224763241

Steps to Reproduce (for bugs)

See: #1538 , #1297 , #1255 , and #1639

My setup is currently using the following for Static config, dynamic config, and config.cfg (for oauth2-proxy)

Dynamic Config

http:
  middlewares:
    oauth-chain:
      chain:
        middlewares:
          - oauth-auth
          - oauth-errors
          - less-secure-headers
    oauth-lessersec:
      chain:
        middlewares:
          - oauth-auth
          - oauth-errors
          - lesser-secure-headers
    test-secure:
      chain:
        middlewares:
          - traefik-auth
          - secure-headers
          - default-compress
    secure-link:
      chain:
        middlewares:
          - less-secure-headers
          - default-compress
    default-compress:
      compress: {}
    traefik-auth:
      basicauth:
        users:
          - <REDACTED>
    secure-headers:
      headers:
        sslforcehost: true
        browserxssfilter: true
        frameDeny: true
        stsIncludeSubdomains: true
        stsPreload: true
        stsSeconds: 63072000
        contentTypeNosniff: true
        accessControlAllowMethods:
          - GET
          - POST
        accessControlAllowOriginList:
          - https://<REDACTED>.org
          - https://www.<REDACTED>.org
        customresponseheaders:
          X-Robots-Tag:
            - noindex
            - nofollow
            - nosnippet
            - noarchive
            - notranslate
            - noimageindex
        accessControlMaxAge: 100
        forcestsheader: true
        addVaryheader: true
        contentSecurityPolicy: script-src 'self'
        referrerPolicy: origin-when-cross-origin
    less-secure-headers:
      headers:

        browserxssfilter: true
        #frameDeny: true
        stsIncludeSubdomains: true
        stsPreload: true
        stsSeconds: 63072000
        contentTypeNosniff: true
        accessControlAllowMethods:
          - GET
          - POST
        accessControlAllowOriginList:
          - https://<REDACTED>.org
          - https://www.<REDACTED>.org
        customresponseheaders:
          X-Robots-Tag:
            - noindex
            - nofollow
            - nosnippet
            - noarchive
            - notranslate
            - noimageindex
        accessControlMaxAge: 100
        addVaryheader: true
        referrerPolicy: origin-when-cross-origin

    lesser-secure-headers:
      headers:

        browserxssfilter: true
        stsIncludeSubdomains: true
        stsPreload: true
        stsSeconds: 63072000
        contentTypeNosniff: true
        customresponseheaders:
          X-Robots-Tag:
            - noindex
            - nofollow
            - nosnippet
            - noarchive
            - notranslate
            - noimageindex
        accessControlMaxAge: 100
        addVaryheader: true
      
    oauth-auth:
      forwardAuth:
        address: https://oauth.<REDACTED>.org/oauth2/auth/
        trustForwardHeader: true
        AuthResponseHeaders:
          - X-Auth-Request-Access-Token
          - Authorization
          - X-Auth-Request-User
          - X-Auth-Request-Email
          - Set-Cookie
          - X-Auth-User
          - X-Secret
          - X-Forwarded-User 
          - X-WebAuth-User
    oauth-errors:
      errors:
        status:
          - "401-403"
        service: oauth2-proxy@docker
        query: "/oauth2/sign_in"
tls:
  options:
    default:
      minVersion: VersionTLS12
      cipherSuites:
       - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
       - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
       - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
       - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
       - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
       - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
      sniStrict: true

Static config

api:
  dashboard: true
  debug: true
entryPoints:
  http:
    address: ":80"
    http:
      redirections:
        entryPoint:
          to: https
          scheme: https
  https:
    address: ":443"
    http:
      tls:
        certResolver: cloudflare
        domains:
          - main: "<REDACTED>.org"
            sans: ["*.<REDACTED>.org"]
serversTransport:
  insecureSkipVerify: true
log:
  level: 'DEBUG'
providers:
  docker:
    endpoint: "tcp://socket-proxy:<REDACTED>"
    exposedByDefault: false
    network: services
    constraints: "Label(`ai.ix.expose`, `true`)"
  file:
    directory: /config/
    watch: true
certificatesResolvers:
  cloudflare:
    acme:
      email:<REDACTED>
      storage: acme.json
      dnsChallenge:
        provider: cloudflare
        delayBeforeCheck: "60"
        resolvers:
          - "1.1.1.1:53"
          - "1.0.0.1:53"

Oauth2-proxy Config

http_address="0.0.0.0:4180"
reverse_proxy="true"
cookie_secret="<REDACTED>"
email_domains="*"
cookie_secure="true"
cookie_domains=[".<REDACTED>"]
whitelist_domains=[".<REDACTED>"]
provider="keycloak-oidc"
client_secret="<REDACTED>"
client_id="oauth2"
redirect_url="https://oauth.<REDACTED>.org/oauth2/callback"
oidc_issuer_url="https://keycloak.<REDACTED>.org/realms/master"
upstreams="static://202"
pass_access_token=true
pass_authorization_header=true
set_authorization_header=true
set_xauthrequest=true
pass_user_headers=true
cookie_httponly=false

Docker Compose Configuration for Traefik, OAuth2-proxy, Whoami, keycloak

traefik:
      image: traefik:vacherin
      container_name: traefik
      restart: unless-stopped
      security_opt:
        - no-new-privileges:true
      networks:
        - internal
        - services
        - auth
      ports:
        - 80:80
        - 443:443

      environment:
        - CF_API_EMAIL=<REDACTED>
        #- CF_API_KEY=<REDACTED>
        - CF_DNS_API_TOKEN=<REDACTED>
      volumes:
        - /etc/localtime:/etc/localtime:ro
        - /var/run/docker.sock:/var/run/docker.sock:ro
        - ./Traefik/data/traefik.yml:/traefik.yml:ro
        - ./Traefik/acme/acme.json:/acme.json
        - ./Traefik/data/:/config/:ro
        - ./Traefik/data/log:/var/log/traefik
      labels:
      - "traefik.enable=true"
      - "ai.ix.expose=true"
      - "traefik.http.middlewares.sslheader.headers.customrequestheaders.X-Forwarded-Proto=https"
      - "traefik.http.routers.traefik-secure.entrypoints=https"
      - "traefik.http.routers.traefik-secure.rule=Host(`traefik.<REDACTED>.org`)"
      - "traefik.http.routers.traefik-secure.middlewares=oauth-chain@file"
      - "traefik.http.routers.traefik-secure.tls=true"
      - "traefik.http.routers.traefik-secure.tls.certresolver=cloudflare"
      - "traefik.http.routers.traefik-secure.tls.domains[0].main=<REDACTED>.org"
      - "traefik.http.routers.traefik-secure.tls.domains[0].sans=*.<REDACTED>.org"
      - "traefik.http.routers.traefik-secure.service=api@internal"
oauth2-proxy:
    image: bitnami/oauth2-proxy:7
    container_name: oauth-forwardauth
    restart: unless-stopped
    networks:
      services:
        aliases:
          - oauth.local.<REDACTED>.org
      auth:

    volumes:
      - ./oauth2/config.cfg:/config.cfg:ro
    command: 
      - "--config=/config.cfg"
      - "--code-challenge-method=S256"
    labels:
      - "traefik.enable=true"
      - "ai.ix.expose=true"
      - "traefik.http.routers.oauth2-proxy.middlewares=less-secure-headers@file"
      - "traefik.http.routers.oauth2-proxy.entrypoints=https"
      - "traefik.http.routers.oauth2-proxy.rule=Host(`oauth.<REDACTED>.org`) || PathPrefix(`/oauth2`)"
      - "traefik.http.routers.oauth2-proxy.tls=true"
      - "traefik.http.routers.oauth2-proxy.tls.certresolver=cloudflare"
      - "traefik.http.routers.oauth2-proxy.tls.domains[0].main=<REDACTED>.org"
      - "traefik.http.routers.oauth2-proxy.tls.domains[0].sans=*.<REDACTED>.org"
      - "traefik.http.routers.oauth2-proxy.service=oauth2-proxy@docker"
      - "traefik.http.services.oauth2-proxy.loadbalancer.server.port=4180"
      #- "traefik.http.services.oauth2-proxy.loadbalancer.passHostHeader=false"
    depends_on:
      keycloak:
        condition: service_healthy
  keycloak:
    image: bitnami/keycloak:18
    container_name: keycloak
    restart: unless-stopped
    volumes: 
      - /etc/localtime:/etc/localtime:ro
    networks:
      services:
      auth:
        aliases:
          - keycloak.local.<REDACTED>.org
      keycloakDB:
    depends_on:
        KCPostgres:
          condition: service_healthy
    env_file: ./Keycloak/config.env
    labels:
      - "traefik.enable=true"
      - "ai.ix.expose=true"
      - "traefik.http.routers.keycloak.rule=Host(`keycloak.<REDACTED>.org`)"
      - "traefik.http.routers.keycloak.entrypoints=https"
      - "traefik.http.routers.keycloak.middlewares=secure-link@file"
      - "traefik.http.routers.keycloak.tls=true"
      - "traefik.http.routers.keycloak.tls.certresolver=cloudflare"
      - "traefik.http.routers.keycloak.tls.domains[0].main=<REDACTED>.org"
      - "traefik.http.routers.keycloak.tls.domains[0].sans=*.<REDACTED>.org"
      - "traefik.http.services.keycloak.loadbalancer.server.port=8080"
      - "traefik.http.routers.keycloak.service=keycloak@docker"
    healthcheck:
      test: ["CMD-SHELL", "curl --fail http://localhost:8080/realms/master || exit 1"]
      interval: 30s
      timeout: 10s
      retries: 10
      start_period: 1m

  whoami:
    image: "traefik/whoami"
    networks:
      - services
    labels:
      - "traefik.enable=true"
      - "ai.ix.expose=true"
      - "traefik.http.routers.whoami.rule=Host(`whoami.<REDACTED>.org`)"
      - "traefik.http.routers.whoami.entrypoints=https"
      - "traefik.http.routers.whoami.tls=true"
      - "traefik.http.routers.whoami.middlewares=oauth-lessersec@file"
      - "traefik.http.services.whoami.loadbalancer.server.port=80"
      - "traefik.http.routers.whoami.tls.certresolver=cloudflare"
      - "traefik.http.routers.whoami.tls.domains[0].main=<REDACTED>.org"
      - "traefik.http.routers.whoami.tls.domains[0].sans=*.<REDACTED>.org"
      - "traefik.http.routers.whoami.service=whoami@docker"
    depends_on:
      oauth2-proxy:
        condition: service_started

Context

I am attempting to use the oauth2-proxy as a forward auth provider for my reverse proxy. I want to gatekeep some of my services behind sso, while still being able to provide granular protection to various services.

Your Environment

Host OS: Proxmox VE 7.2-7 Guest OS: Almalinux 8.6 (LXC: FUSE, KEYCTL, and NESTING are enabled)

• Version used: I am using the Bitnami container of oauth2-proxy, pinned to the 7 tag, which puts it on 7.3.0