istio: JWT Token always fails with 403 in Istio 1.21.0
Is this the right place to submit this?
- This is not a security vulnerability or a crashing bug
- This is not a question about how to use Istio
Bug Description
When I upgrade Istio using Istioctl from version 1.20.3 to 1.21.0 all requests to our api fail with http status code 403.
The following configuration files work correctly in version 1.20.3:
AuthorizationPolicy
apiVersion: "security.istio.io/v1"
kind: "RequestAuthentication"
metadata:
name: "foo-jwt"
namespace: foo
spec:
selector:
matchLabels:
app: foo
jwtRules:
- issuer: "https://example.keycloak.com/realms/realm-name"
jwksUri: "https://example.keycloak.com/realms/realm-name/protocol/openid-connect/certs"
---
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
name: require-jwt-foo
namespace: foo
spec:
selector:
matchLabels:
app: foo
action: ALLOW
rules:
- from:
- source:
requestPrincipals: [ "https://example.keycloak.com/realms/realm-name/*" ]
to:
- operation:
methods: [ "GET", "POST", "PUT", "OPTIONS", "DELETE", "PATCH" ]
paths: [ "/api/v1/some-path/*" ]
when:
- key: request.auth.claims[emailVerified]
values: [ "true" ]
- key: request.auth.claims[roles]
values: [ "admin", "user" ]
The previous configuration works flawlessly with Istio version 1.20.3.
As a test we apply the following AuthorizationPolicy without deleting the previous configuration and all the requests work again but it is because in the following configuration it overrides any validation of the token
apiVersion: "security.istio.io/v1"
kind: "AuthorizationPolicy"
metadata:
name: "http-foo"
namespace: foo
spec:
selector:
matchLabels:
app: foo
rules:
- to:
- operation:
methods: ["GET", "POST", "PUT", "OPTIONS", "DELETE"]
Version
istioctl version
client version: 1.21.0
control plane version: 1.21.0
data plane version: 1.21.0 (71 proxies)
Client Version: v1.29.0
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.29.0-eks-c417bb3
Additional Information
No response
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 1
- Comments: 22 (11 by maintainers)
Please supply a config dump from Envoy to further triage (specifically jwt_authn filter and RBAC filter).
OK, I’ll send a fix to make the “aud” claim match as a list or a value. Thank you for the bug report!
@zirain This should be a release blocker, I’ll cherry pick to the release candidate.
Same issue here with
AuthorizationPolicyspec defined as:envoy config dump for v1.20.X
{ "name": "envoy.filters.http.rbac", "typed_config": { "@type": "type.googleapis.com/envoy.extensions.filters.http.rbac.v3.RBAC", "rules": { "action": "DENY", "policies": { "ns[<redacted_ns>]-policy[<redacted_name>]-rule[0]": { "permissions": [ { "and_rules": { "rules": [ { "any": true } ] } } ], "principals": [ { "and_ids": { "ids": [ { "not_id": { "or_ids": { "ids": [ { "metadata": { "filter": "istio_authn", "path": [ { "key": "request.auth.audiences" } ], "value": { "string_match": { "exact": "aud1" } } } }, { "metadata": { "filter": "istio_authn", "path": [ { "key": "request.auth.audiences" } ], "value": { "string_match": { "exact": "aud2" } } } } ] } } } ] } } ] } } }, "shadow_rules_stat_prefix": "istio_dry_run_allow_" } }envoy config dump for v1.21.1
{ "name": "envoy.filters.http.rbac", "typed_config": { "@type": "type.googleapis.com/envoy.extensions.filters.http.rbac.v3.RBAC", "rules": { "action": "DENY", "policies": { "ns[<redacted_ns>]-policy[<redacted_name>]-rule[0]": { "permissions": [ { "and_rules": { "rules": [ { "any": true } ] } } ], "principals": [ { "and_ids": { "ids": [ { "not_id": { "metadata": { "filter": "envoy.filters.http.jwt_authn", "path": [ { "key": "payload" }, { "key": "aud" } ], "value": { "or_match": { "value_matchers": [ { "string_match": { "exact": "aud1" } }, { "string_match": { "exact": "aud2" } } ] } } } } } ] } } ] } } }, "shadow_rules_stat_prefix": "istio_dry_run_allow_" } }@kyessenov do you see any issue why this throws “RBAC: access denied” in v1.21.1
The issue for repeated values and not values is the difference in how JWT is matched between Envoy and Istio. I’ll send a patch for that. Re: iss difference, I’ll look further. Sorry about this, we don’t have a good coverage for this that would have caught it earlier.