istio: End user JWT authentication is not working in istio-1.0 release build

Setting up end user JWT auth following the Istio 1.0 docs always results in Envoy returning 503 Service Unavailable

I’m aware of the documentation bug and created the Gateway and VirtualService resource to setup ingress routing. After exposing “httpbin” service I am able to reach it via ingressgateway without any token as expected:

curl $INGRESS_HOST/headers -s -o /dev/null -w "%{http_code}\n"
200

Creating the authentication policy with the SVC_ACCOUNT & JWKS set like this:

apiVersion: "authentication.istio.io/v1alpha1"
kind: "Policy"
metadata:
  name: "example-3"
spec:
  targets:
  - name: httpbin
  peers:
  - mtls:
  origins:
  - jwt:
      issuer: $SVC_ACCOUNT
      jwksUri: $JWKS
  principalBinding: USE_ORIGIN
EOF

always returns 503 even after adding the bearer token:

curl --header "Authorization: Bearer $TOKEN" $INGRESS_HOST/headers -s -o /dev/null -w "%{http_code}\n"
503

If I update the policy spec and remove the peers mtls section like this:

apiVersion: "authentication.istio.io/v1alpha1"
kind: "Policy"
metadata:
  name: "example-3"
spec:
  targets:
  - name: httpbin
  origins:
  - jwt:
      issuer: $SVC_ACCOUNT
      jwksUri: $JWKS
  principalBinding: USE_ORIGIN
EOF

I always get 401 unauthorized even with a valid token:

curl --header "Authorization: Bearer $TOKEN" $INGRESS_HOST/headers -s -o /dev/null -w "%{http_code}\n"
401

Kubernetes version: “1.9.6” Istio version: https://gcsweb.istio.io/gcs/istio-prerelease/daily-build/release-1.0-20180720-09-15/ Output of istioctl version:

Version: release-1.0-20180720-09-15
GitRevision: 86dc1496ef02591ce2f6b2490b161841733e1ffc

mTLS is globally turned off as mentioned in the task docs

Might be related to the issue: https://github.com/istio/istio/issues/6668

About this issue

Original URL
State: closed
Created 6 years ago
Comments: 18 (12 by maintainers)

Most upvoted comments

I am using the 1.0.0 release from dockerhub, and am having the same issue. There is somehow no mention of jwt in the envoy config dump. Is there a way to debug this at all? I have a very basic Policy set

---
apiVersion: "authentication.istio.io/v1alpha1"
kind: "Policy"
metadata:
  name: policy
  namespace: dev
spec:
  targets:
  - name: service-private
  peers:
  - mtls:
      mode: STRICT
  origins:
  - jwt:
      issuer: "gitlab"
      jwksUri: "gitlab/oauth/discovery/keys"
  principalBinding: USE_ORIGIN

Whats even weirder is that if I set a Policy on the ingress-gateway the filter are correctly set and everything asks for auth:

---
apiVersion: authentication.istio.io/v1alpha1
kind: Policy
metadata:
  name: ingressgateway-jwt-policy
  namespace: istio-system
spec:
  targets:
  - name: istio-ingressgateway
  origins:
  - jwt:
      issuer: "gitlab"
      jwksUri: "gitlab/oauth/discovery/keys"
  principalBinding: USE_ORIGIN

Arttii on Aug 30, 2018

This error means pilot cannot fetch the public key set from the given JwksUri. If you use https, your server certificate must be signed by a CA that is verifiable. We use CA bundle that come with standard linux distro, so it should work all common PKI. In other words, self-signing certificate will not work.

Hope this helps.

Thanks.

Diem

On Wed, Sep 12, 2018 at 5:36 AM Farasath Ahamed notifications@github.com wrote:

Did you get this working? I am facing the same issue. I could get the sample https://istio.io/help/ops/security/end-user-auth/ working only if I have the jwks endpoint in a public url (like https://gist.githubusercontent.com/mefarazath/0757201b14de6f2a3c01b62d39648367/raw/1a37b704825f64f0763dfc02dfabc1af44fb344b/test-jwt.jwks )

apiVersion: “authentication.istio.io/v1alpha1” kind: “Policy” metadata: name: “jwt-example” spec: targets:

name: httpbin origins:

jwt: issuer: “https://localhost:9443/oauth2/token” jwksUri: “https://gist.githubusercontent.com/mefarazath/0757201b14de6f2a3c01b62d39648367/raw/1a37b704825f64f0763dfc02dfabc1af44fb344b/test-jwt.jwks” principalBinding: USE_ORIGIN

I tried running an oauth2 server within the k8s cluster and setting the jwks uri using, service name, NodePort etc. but could not get it to work… In the error scenarios I see the below in the logs

[2018-09-12 12:26:12.571][17][warning][config] bazel-out/k8-opt/bin/external/envoy/source/common/config/_virtual_includes/grpc_mux_subscription_lib/common/config/grpc_mux_subscription_impl.h:70] gRPC config for type.googleapis.com/envoy.api.v2.Listener rejected: Error adding/updating listener 172.17.0.13_8000: Proto constraint validation failed (JwtAuthenticationValidationError.Rules[i]: [“embedded message failed validation”] | caused by JwtRuleValidationError.LocalJwks: [“embedded message failed validation”] | caused by DataSourceValidationError.InlineString: [“value length must be at least " ‘\x01’ " bytes”]): rules { issuer: “https://localhost:9443/oauth2/token” local_jwks { inline_string: “” } forward: true forward_payload_header: “istio-sec-948f5aaa926a8a4d63e3ad5bf058ce381a69561e” } allow_missing_or_failed: true

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/istio/istio/issues/7290#issuecomment-420632018, or mute the thread https://github.com/notifications/unsubscribe-auth/AX99cf2ZdjL_8wBO3sNiguifAmu01dNXks5uaP_jgaJpZM4VZSjV .

–

Diem Vu | Software Engineer | diemvu@google.com | +1 408-215-8127

diemtvu on Sep 12, 2018