istio: Unable to get two CUSTOM AuthorizationPolicies for the same workload

Bug description I’m trying to setup two CUSTOM AuthorizationPolicies both on the ingressgateway

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: external-authorization
  namespace: {{ISTIO_NAMESPACE}}
spec:
  selector:
    matchLabels:
      app: istio-ingressgateway
  action: CUSTOM
  provider:
    name: oauth2-proxy
  rules:
    - to:
        - operation:
            ports: [ "{{MONITORING_TOOLS_PORT}}" ]
            paths: [ "/*" ]
---
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: my-api-gateway-auth
  namespace: {{ISTIO_NAMESPACE}}
spec:
  selector:
    matchLabels:
      app: istio-ingressgateway
  action: CUSTOM
  provider:
    name: my-api-gateway
  rules:
    - to:
        - operation:
            methods: ["POST"]
            paths: [ "/integration*" ]

I’m getting the following error in istiod

2021-07-14T15:19:38.232512Z	error	authorization	Processed authorization policy for istio-ingressgateway-565687598-rvxgv.istio-system, 2 errors occurred:
	* failed to parse CUSTOM action, will generate a deny all config: only 1 provider can be used per workload, found multiple providers: [my-api-gateway oauth2-proxy]
	* failed to process CUSTOM action: only 1 provider can be used per workload, found multiple providers: [my-api-gateway oauth2-proxy]

[ ] Docs [ ] Installation [ ] Networking [ ] Performance and Scalability [ ] Extensions and Telemetry [X] Security [ ] Test and Release [ ] User Experience [ ] Developer Infrastructure [ ] Upgrade

Expected behavior I should be able to run multiple AuthorizationPolicies with different rules

Steps to reproduce the bug

Version (include the output of istioctl version --remote and kubectl version --short and helm version --short if you used Helm) client version: 1.9.1 control plane version: 1.9.1 data plane version: 1.9.1 (38 proxies)

How was Istio installed? using IstioOperator

Environment where the bug was observed (cloud vendor, OS, etc) AKS 1.18.19

Additionally, please consider running istioctl bug-report and attach the generated cluster-state tarball to this issue. Refer cluster state archive for more details.

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Reactions: 11
  • Comments: 17 (4 by maintainers)

Most upvoted comments

FYI, others have mentioned a workaround, and I was finally able to get it working for those interested. You can simply add a couple EnvoyFilters that are the equivalent of what would be generated by a second custom auth provider and policy. I’ve added an example below:


apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: second-ext-auth-rbac-filter
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      istio: ingressgateway
  configPatches:
    - applyTo: HTTP_FILTER
      match:
        context: GATEWAY
        listener:
          filterChain:
            filter:
              name: "envoy.filters.network.http_connection_manager"
              subFilter:
                name: "envoy.filters.http.ext_authz"  # Add after the filter added by the first custom auth policy
      patch:
        operation: INSERT_AFTER
        value:
          name: second_ext_auth_rbac
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.http.rbac.v3.RBAC
            shadow_rules:
                action: DENY
                policies:
                  second-ext-authz:
                    permissions:
                    - and_rules:
                        rules:
                        - or_rules:
                            rules:
                            - header:
                                name: ":authority"
                                string_match:
                                  exact: <<SECURE DOMAIN>> # populate matching rule here
                                  ignore_case: true
                        - or_rules:
                            rules:
                            - destination_port: 443
                    principals:
                    - and_ids:
                        ids:
                        - any: true
            shadow_rules_stat_prefix: second_ext_authz_


---

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: second-ext-authz-filter
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      istio: ingressgateway
  configPatches:
    - applyTo: HTTP_FILTER
      match:
        context: GATEWAY
        listener:
          filterChain:
            filter:
              name: "envoy.filters.network.http_connection_manager"
              subFilter:
                name: "envoy.filters.http.ext_authz"
      patch:
        operation: INSERT_AFTER
        value:
          name: second_ext_authz
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthz
            http_service:
              server_uri:
                uri: "http://oauthproxy.auth.svc.cluster.local"
                cluster: "outbound|4180||oauthproxy.auth.svc.cluster.local"
                timeout: "600s"
              authorization_request:
                allowed_headers:
                  patterns:
                  - exact: authorization
                    ignore_case: true
                  - exact: cookie
                    ignore_case: true
                  - exact: origin
                    ignore_case: true
              authorization_response:
                allowed_upstream_headers:
                  patterns:
                  - exact: accept
                    ignore_case: true
                  - exact: origin
                    ignore_case: true
                  - exact: path
                    ignore_case: true
                  - exact: authorization
                    ignore_case: true
                  - exact: x-auth-request-user
                    ignore_case: true
                  - exact: x-auth-request-email
                    ignore_case: true
                  - exact: x-auth-request-access-token
                    ignore_case: true
            transport_api_version: V3
            filter_enabled_metadata:
              filter: envoy.filters.http.rbac
              path:
              - key: second_ext_authz_shadow_effective_policy_id
              value:
                string_match:
                  prefix: second-ext-authz

I’ve run into this issue as well. Will there be any resolution? Seems like a very arbitrary and unnecessary limitation

Our team has just ran into this limitation as well and now we’re having to seek alternative solutions. I would also like know when this will be put the roadmap.

This is still an issue / feature we would like to see. What is the plan to put this on the roadmap? Thank you

Is there any news about such a feature?

We ran into the same situation. Is there any progress on this?

not stale

🚧 This issue or pull request has been closed due to not having had activity from an Istio team member since 2023-06-04. If you feel this issue or pull request deserves attention, please reopen the issue. Please see this wiki page for more information. Thank you for your contributions.

Created by the issue and PR lifecycle manager.

We are running into this limitation. We want to apply multiple providers for 1 workload. Has anyone found a workaround? Perhaps anyone has tried modifying underlying Envoy Filters?

@rbisecke This is not related to the EnvoyExtAuthzHTTP struct, but related to lower-level Envoy filters ordering.

@yangminzhu what about no guaranteed ordering to start off? and maybe a config of runAllOnFail to have try all even if one fails?