pyjwt: RS256 Token Validation & Decoding using Public Key Not Working "ValueError: Could not deserialize key data."

I’m trying to validate Google’s ID Tokens for user authentication on a web app. The id token can be decoded fine if I disable verification, but won’t verify when I pass it the RSA256 Public Key. The Public Key in question is Base64urlUInt-Encoded (RFC 7518 Specification).

The Entire Public Key Response

  {
   "kty": "RSA",
   "alg": "RS256",
   "use": "sig",
   "kid": "8c9eb968f73744eaed421e48010142bce51a067f",
   "n": "uweJ3hFY9wqZ6ZG-iSNhQwHtKCGl8G_jcQgGPjOrS-Rum3dyDjicqkAyfS8XDn480KD_TZ5m-lqBjqfimePu2_cH4URDPIwsqSzJI2_piEhaqnXRptIe5YB5imAL6iETKaOPjw284Fc7EdHK-ekHMn3AXjsy9AIErwAVw4-4ZXXwHbyQXJy1DyUB4ZzxiEvw_qkQmLdltmrNkLOw-Xh-C9UkTZ9NA58bYPBnxLwnAu_ggw_g_-hCAs6OvXZbAfFHhIGBLyjtdDLVrfXo1112QREB9d5sEds0bKZtJcD9afl4E7Ht6G-g3jNP2clAu6-6B-cIe4-j8Ph1uJDPkAmDfw",
   "e": "AQAB"
  }

According to the Specification I linked above, the “n” parameter is the Modulus, and the “e” parameter is the exponent. I’ve tried absolutely every combination of decoding these to common Base64 format, but no matter what I do, pyJWT doesn’t like it.

Expected Result

A Verified, decoded JSON data packet.

Actual Result

Traceback (most recent call last):
  File "/anaconda3/lib/python3.6/site-packages/jwt/algorithms.py", line 205, in prepare_key
    key = load_pem_private_key(key, password=None, backend=default_backend())
  File "/anaconda3/lib/python3.6/site-packages/cryptography/hazmat/primitives/serialization.py", line 20, in load_pem_private_key
    return backend.load_pem_private_key(data, password)
  File "/anaconda3/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1015, in load_pem_private_key
    password,
  File "/anaconda3/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1234, in _load_key
    self._handle_key_loading_error()
  File "/anaconda3/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1292, in _handle_key_loading_error
    raise ValueError("Could not deserialize key data.")
ValueError: Could not deserialize key data.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/anaconda3/lib/python3.6/site-packages/jwt/api_jwt.py", line 93, in decode
    jwt, key=key, algorithms=algorithms, options=options, **kwargs
  File "/anaconda3/lib/python3.6/site-packages/jwt/api_jws.py", line 157, in decode
    key, algorithms)
  File "/anaconda3/lib/python3.6/site-packages/jwt/api_jws.py", line 221, in _verify_signature
    key = alg_obj.prepare_key(key)
  File "/anaconda3/lib/python3.6/site-packages/jwt/algorithms.py", line 207, in prepare_key
    key = load_pem_public_key(key, backend=default_backend())
  File "/anaconda3/lib/python3.6/site-packages/cryptography/hazmat/primitives/serialization.py", line 24, in load_pem_public_key
    return backend.load_pem_public_key(data)
  File "/anaconda3/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1041, in load_pem_public_key
    self._handle_key_loading_error()
  File "/anaconda3/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1292, in _handle_key_loading_error
    raise ValueError("Could not deserialize key data.")
ValueError: Could not deserialize key data.

Reproduction Steps

import jwt

IDjwt = <my id_token here>  # Decoding this with verify=False works correctly, so the problem isn't with the ID Token

GoogPubKey = b'uweJ3hFY9wqZ6ZG-iSNhQwHtKCGl8G_jcQgGPjOrS-Rum3dyDjicqkAyfS8XDn480KD_TZ5m-lqBjqfimePu2_cH4URDPIwsqSzJI2_piEhaqnXRptIe5YB5imAL6iETKaOPjw284Fc7EdHK-ekHMn3AXjsy9AIErwAVw4-4ZXXwHbyQXJy1DyUB4ZzxiEvw_qkQmLdltmrNkLOw-Xh-C9UkTZ9NA58bYPBnxLwnAu_ggw_g_-hCAs6OvXZbAfFHhIGBLyjtdDLVrfXo1112QREB9d5sEds0bKZtJcD9afl4E7Ht6G-g3jNP2clAu6-6B-cIe4-j8Ph1uJDPkAmDfw'

#Convert to Base64 (replace '-', '_' with '+', '/', respectively, and pad with '=' to make multiple of 4)
GoogPubKey = GoogPubKey.replace(b'-', b'+')
GoogPubKey = GoogPubKey.replace(b'_', b'/')
GoogPubKey += b'=='
len(GoogPubKey) % 4  # 0
GoogPubKey = b'-----BEGIN PUBLIC KEY-----\n' + GoogPubKey + b'\n-----END PUBLIC KEY-----'

decoded = jwt.decode(IDjwt, GoogPubKey, algorithms='RS256')

Again, I’ve tried over a dozen different ways of using this RSA Public Key Google supplies, but nothing works (Using Base64url like they provide, using or not using the ‘BEGIN PUBLIC KEY’ prefix/suffix, type bytes or str, adding the “AQAB” prefix in multiple places, nothing works)

Any help would be greatly appreciated!! Thank you

System Information

$ python -m jwt.help
{
  "cryptography": {
    "version": "2.1.4"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.4"
  },
  "platform": {
    "release": "17.4.0",
    "system": "Darwin"
  },
  "pyjwt": {
    "version": "1.6.4"
  }
}

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Reactions: 13
  • Comments: 19 (2 by maintainers)

Most upvoted comments

Try something like this:

import jwt
from jwt.algorithms import RSAAlgorithm

IDjwt = <my id_token here>  # Decoding this with verify=False works correctly, so the problem isn't with the ID Token

key_json = '{"kty": "RSA","alg": "RS256","use": "sig","kid": "4129db2ea1860d2e871ee48506287fb05b04ca3f","n": "sxorUSxfZZjQL1mDr1rtbNGJE9lbVMiBmNZFqLhnQaefTfqMO3YgSlb_cptw5wS2Dn4phGNzjBaO1Hg5572mEqsmPl5z9MmybIOuqWXxYyIiCGWH3hoR2VPJ-1bN-SdszHb4ZWadXCCYqnHS216nrvHZK8vJyQ7XCchw43O00LC5Iwi2eKspQEj8YDQSZFsd7Mp2ULhKXVPyKeLH06aenBZZFwgjw8bow7MXS4uUkg4NOeH2iHNxclOYycg6Z87QrTVzHGBo9r-6s1XRTFh-rqcZC8RnR62wkPqB2AEHctOof_ZtaaDTZ1Xw7db8dRhhCnFkpiK_1d8c9N2Vm7Frxw","e": "AQAB"}'

public_key = RSAAlgorithm.from_jwk(key_json)

decoded = jwt.decode(IDjwt, public_key, algorithms='RS256')
import jwt
from jwt.algorithms import RSAAlgorithm

IDjwt = <my id_token here>  # Decoding this with verify=False works correctly, so the problem isn't with the ID Token

key_json = '{"kty": "RSA","alg": "RS256","use": "sig","kid": "4129db2ea1860d2e871ee48506287fb05b04ca3f","n": "sxorUSxfZZjQL1mDr1rtbNGJE9lbVMiBmNZFqLhnQaefTfqMO3YgSlb_cptw5wS2Dn4phGNzjBaO1Hg5572mEqsmPl5z9MmybIOuqWXxYyIiCGWH3hoR2VPJ-1bN-SdszHb4ZWadXCCYqnHS216nrvHZK8vJyQ7XCchw43O00LC5Iwi2eKspQEj8YDQSZFsd7Mp2ULhKXVPyKeLH06aenBZZFwgjw8bow7MXS4uUkg4NOeH2iHNxclOYycg6Z87QrTVzHGBo9r-6s1XRTFh-rqcZC8RnR62wkPqB2AEHctOof_ZtaaDTZ1Xw7db8dRhhCnFkpiK_1d8c9N2Vm7Frxw","e": "AQAB"}'

public_key = RSAAlgorithm.from_jwk(key_json)

decoded = jwt.decode(IDjwt, public_key, algorithms='RS256')

just fyi, a simpler approach also worked for me;

import jwt
IDjwt = <my id_token here>
decoded = jwt.decode(IDjwt, '', verify=False)

@jpadilla Out of 1000’s of messages i’ve read today, your hint WORKED.

@tyrelkostyk give up on passing public key yourself. Instead realy on OIDC well known config to discover exactly how you can construct RSA from JWK. It is there you don’t have to guess it, at least it worked for me. Here is how I retrieved it.

Find your well-known config endpoint that lists all other endpoints. In case of keycloak its here (obviously realm will be different)

screen shot 2018-09-11 at 01 41 05

…then load “certs” endpoint

screen shot 2018-09-11 at 01 43 53

and then… use @jpadilla example above but with new token and detected jwk construction set

import jwt
from jwt.algorithms import RSAAlgorithm
IDjwt = 'eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJoSjZYWnF2cWVuVjVIay1qWnV5TUN5bGNPT3FnTzNpWlYtMnVBRTVhMDYwIn0.eyJqdGkiOiI3M2Y0NDk2NC03M2UxLTRmZTgtYmRlZi1mNGJiY2FkNDQ5NjUiLCJleHAiOjE1MzY2MjY2NzksIm5iZiI6MCwiaWF0IjoxNTM2NjI2MDc5LCJpc3MiOiJodHRwczovL3NpZ25pbi5kZW1vLm1hZGNvcmUuY2xvdWQvYXV0aC9yZWFsbXMvbWFzdGVyIiwiYXVkIjoibG9jYWxob3N0Iiwic3ViIjoiMjUzNDk1ZmUtZGY2MC00NDE5LWExMzQtZWZhZTFiMGE5OWQwIiwidHlwIjoiQmVhcmVyIiwiYXpwIjoibG9jYWxob3N0IiwiYXV0aF90aW1lIjowLCJzZXNzaW9uX3N0YXRlIjoiMWZmYTgzZGItZDg3MS00YjQzLTk5YjgtOTMzNDQxNTg1OTExIiwiYWNyIjoiMSIsImFsbG93ZWQtb3JpZ2lucyI6WyIqIl0sInJlYWxtX2FjY2VzcyI6eyJyb2xlcyI6WyJvZmZsaW5lX2FjY2VzcyIsInVtYV9hdXRob3JpemF0aW9uIl19LCJyZXNvdXJjZV9hY2Nlc3MiOnsiYWNjb3VudCI6eyJyb2xlcyI6WyJtYW5hZ2UtYWNjb3VudCIsIm1hbmFnZS1hY2NvdW50LWxpbmtzIiwidmlldy1wcm9maWxlIl19fSwic2NvcGUiOiJwcm9maWxlIGVtYWlsIiwiZW1haWxfdmVyaWZpZWQiOnRydWUsIm5hbWUiOiJQZXRlciBTdHlrIiwicHJlZmVycmVkX3VzZXJuYW1lIjoicG9sZmlsbSIsImdpdmVuX25hbWUiOiJQZXRlciIsImZhbWlseV9uYW1lIjoiU3R5ayIsImVtYWlsIjoicG9sZmlsbUBnbWFpbC5jb20ifQ.gaYPTNBLVYQJTRU5UHJ3GpVNKOrpekBIHmso6ZWLvaUJu4lgXf5wY1fdCqtsubTl2IlCy-zL81LABqua3He1M8qdAFvhKiTqyjm5SvzT40sjIDEnzCfUjYTvJIor8gcVscyNlqAuph0LNJb-aGu_tkaXjjLBl0DXqbyVyUtQ80ai9-ReCsmLobVLWeoyfi7hu9-elP6pOxaueIh5kT-MIux63xvvhSXBwQTxN9Dv7mNfSd1MCUUGtUJE1Fsb8dnIzpGjQh23Mw3p7SLq_ox_IhFP_mzRh3H8ye0gwVjILOOFUrCvFcylBS6TGhCZtkZP6luqRFbkCM9flQjQkON6lw'
key_json = '{"kid":"hJ6XZqvqenV5Hk-jZuyMCylcOOqgO3iZV-2uAE5a060","kty":"RSA","alg":"RS256","use":"sig","n":"q33wpseciIME_pakwslqucEAC6f_T9lN1OYaNhFN3cs_50KhWuPu8918JZFECvtby835CIyIEngKWLFr-VPbe5GW94dujvlaZJOj0eGst3t2gd6TOeu5FwzsAJWHNP725fu5SwGlN2J81fmYSYAWG1QNK3Bu5Fn5KD0gCN1MRD8gjC-hXHte904fdwRxZdLfQinaEyW1xwlItsJ1U9_Ve6hZbE4HMZeyeGPrJna__xWbNi9xCize32L-pepyeXWGmcTgq7--p9bXu6xtm_8Pmt5KkuLS-sE1Lrj19sZffjeJoy5q6tTXr8CAJT5qU-P9km4WAdKkb-2IlWMmGtHyQw","e":"AQAB"}'
public_key = RSAAlgorithm.from_jwk(key_json)
decoded = jwt.decode(IDjwt, public_key, algorithms='RS256')

TADA…

Thank you!!!

Just a note that I faced a similar issue and found a solution. In my case, I was using a signing certificate downloaded from Auth0 in PEM format, which I had incorrectly assumed was a public key. According to https://cryptography.io/en/latest/hazmat/primitives/asymmetric/serialization/#pem:

A PEM block which starts with -----BEGIN CERTIFICATE----- is not a public or private key, it’s an X.509 Certificate. You can load it using load_pem_x509_certificate() and extract the public key with Certificate.public_key.

My PEM file did start with -----BEGIN CERTIFICATE-----, so I had to implement what is suggested by the cryptograpy library. Here’s some sample code for doing this:

import jwt
from cryptography import x509
from cryptography.hazmat.backends import default_backend
from django.conf import settings


def get_public_key_from_certificate(certificate_content: bytes) -> bytes:
    """
    Given a public signing certificate (e.g. downloaded from Auth0), load the certificate and then return the public key for it.

    :param certificate_content: The content of a public signing certificate in PEM format.
    :return: The public key.
    """
    certificate = x509.load_pem_x509_certificate(
        certificate_content, backend=default_backend()
    )

    return certificate.public_key()


def decode_access_token(access_token: str) -> dict:
    """
    Verify a JWT access token generated by Auth0, and return its decoded content.
    """
    public_key = get_public_key_from_certificate(settings.AUTH0_PUBLIC_CERTIFICATE)

    return jwt.decode(
        access_token,
        public_key,
        audience=settings.AUTH0_API_IDENTIFIER,
        algorithms=["RS256"],
        verify=True,
    )

Note that in this code, settings.AUTH0_PUBLIC_CERTIFICATE is a bytes value containing the content of the PEM key.

Now that I’m actually passing a public key into jwt.decode, everything works correctly.

See also: https://community.auth0.com/t/token-validation-with-python/21589/2.

The same issue.

FYI:

Openidc well.known endpoint for Azure: doc: https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-protocols-oidc https://login.microsoftonline.com/{tenant}/v2.0/.well-known/openid-configuration

@tyrelkostyk and for Google: doc: https://developers.google.com/identity/protocols/OpenIDConnect#discovery https://accounts.google.com/.well-known/openid-configuration

@scottwn At least, you should use public instead of key.

decoded = jwt.decode(token, public, algorithms=key[‘alg’])

@jpadilla Out of 1000’s of messages i’ve read today, your hint WORKED.

@tyrelkostyk give up on passing public key yourself. Instead realy on OIDC well known config to discover exactly how you can construct RSA from JWK. It is there you don’t have to guess it, at least it worked for me. Here is how I retrieved it.

Find your well-known config endpoint that lists all other endpoints. In case of keycloak its here (obviously realm will be different)

screen shot 2018-09-11 at 01 41 05

…then load “certs” endpoint

screen shot 2018-09-11 at 01 43 53

and then… use @jpadilla example above but with new token and detected jwk construction set

import jwt
from jwt.algorithms import RSAAlgorithm
IDjwt = 'eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJoSjZYWnF2cWVuVjVIay1qWnV5TUN5bGNPT3FnTzNpWlYtMnVBRTVhMDYwIn0.eyJqdGkiOiI3M2Y0NDk2NC03M2UxLTRmZTgtYmRlZi1mNGJiY2FkNDQ5NjUiLCJleHAiOjE1MzY2MjY2NzksIm5iZiI6MCwiaWF0IjoxNTM2NjI2MDc5LCJpc3MiOiJodHRwczovL3NpZ25pbi5kZW1vLm1hZGNvcmUuY2xvdWQvYXV0aC9yZWFsbXMvbWFzdGVyIiwiYXVkIjoibG9jYWxob3N0Iiwic3ViIjoiMjUzNDk1ZmUtZGY2MC00NDE5LWExMzQtZWZhZTFiMGE5OWQwIiwidHlwIjoiQmVhcmVyIiwiYXpwIjoibG9jYWxob3N0IiwiYXV0aF90aW1lIjowLCJzZXNzaW9uX3N0YXRlIjoiMWZmYTgzZGItZDg3MS00YjQzLTk5YjgtOTMzNDQxNTg1OTExIiwiYWNyIjoiMSIsImFsbG93ZWQtb3JpZ2lucyI6WyIqIl0sInJlYWxtX2FjY2VzcyI6eyJyb2xlcyI6WyJvZmZsaW5lX2FjY2VzcyIsInVtYV9hdXRob3JpemF0aW9uIl19LCJyZXNvdXJjZV9hY2Nlc3MiOnsiYWNjb3VudCI6eyJyb2xlcyI6WyJtYW5hZ2UtYWNjb3VudCIsIm1hbmFnZS1hY2NvdW50LWxpbmtzIiwidmlldy1wcm9maWxlIl19fSwic2NvcGUiOiJwcm9maWxlIGVtYWlsIiwiZW1haWxfdmVyaWZpZWQiOnRydWUsIm5hbWUiOiJQZXRlciBTdHlrIiwicHJlZmVycmVkX3VzZXJuYW1lIjoicG9sZmlsbSIsImdpdmVuX25hbWUiOiJQZXRlciIsImZhbWlseV9uYW1lIjoiU3R5ayIsImVtYWlsIjoicG9sZmlsbUBnbWFpbC5jb20ifQ.gaYPTNBLVYQJTRU5UHJ3GpVNKOrpekBIHmso6ZWLvaUJu4lgXf5wY1fdCqtsubTl2IlCy-zL81LABqua3He1M8qdAFvhKiTqyjm5SvzT40sjIDEnzCfUjYTvJIor8gcVscyNlqAuph0LNJb-aGu_tkaXjjLBl0DXqbyVyUtQ80ai9-ReCsmLobVLWeoyfi7hu9-elP6pOxaueIh5kT-MIux63xvvhSXBwQTxN9Dv7mNfSd1MCUUGtUJE1Fsb8dnIzpGjQh23Mw3p7SLq_ox_IhFP_mzRh3H8ye0gwVjILOOFUrCvFcylBS6TGhCZtkZP6luqRFbkCM9flQjQkON6lw'
key_json = '{"kid":"hJ6XZqvqenV5Hk-jZuyMCylcOOqgO3iZV-2uAE5a060","kty":"RSA","alg":"RS256","use":"sig","n":"q33wpseciIME_pakwslqucEAC6f_T9lN1OYaNhFN3cs_50KhWuPu8918JZFECvtby835CIyIEngKWLFr-VPbe5GW94dujvlaZJOj0eGst3t2gd6TOeu5FwzsAJWHNP725fu5SwGlN2J81fmYSYAWG1QNK3Bu5Fn5KD0gCN1MRD8gjC-hXHte904fdwRxZdLfQinaEyW1xwlItsJ1U9_Ve6hZbE4HMZeyeGPrJna__xWbNi9xCize32L-pepyeXWGmcTgq7--p9bXu6xtm_8Pmt5KkuLS-sE1Lrj19sZffjeJoy5q6tTXr8CAJT5qU-P9km4WAdKkb-2IlWMmGtHyQw","e":"AQAB"}'
public_key = RSAAlgorithm.from_jwk(key_json)
decoded = jwt.decode(IDjwt, public_key, algorithms='RS256')

TADA…

Thank you!!!

This was exactly what i needed. i also struggled with keycloak. i had to make a small change to the jwt.decode though to make it working on my side:
decoded = jwt.decode(token, public_key, algorithms="RS256", audience="account")

Thanks alot for this solution!

@vjsimha1 Indeed, this is the solution I came up with for decoding AWS Cognito tokens:

import json
import jwt
import requests
from jwt.algorithms import RSAAlgorithm

def validate_cognito_token(id_token, cognito_region, cognito_user_pool_id, cognito_app_client_id):
    jwks = requests.get('https://cognito-idp.{aws_region}.amazonaws.com/{user_pool_id}/.well-known/jwks.json'.format(aws_region=cognito_region, user_pool_id=cognito_user_pool_id)).json()
    keys = {k['kid']: RSAAlgorithm.from_jwk(json.dumps(k)) for k in jwks['keys']}
    header = jwt.get_unverified_header(id_token)
    key_id = header['kid']
    algorithm = header['alg']
    pub_key = keys[key_id]
    # Next line raises errors if the audience isn't right or if the token is expired or has other errors.
    valid_token_data = jwt.decode(id_token, pub_key, audience=cognito_app_client_id, algorithm=algorithm)
    return valid_token_data

Does this look similar to your solution?

I could get this working without a public/private key using options.

decoded_details = jwt.decode(jwt_id_token, options={"verify_signature": False})
python -m jwt.help
{
  "cryptography": {
    "version": "3.2.1"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.5"
  },
  "platform": {
    "release": "19.6.0",
    "system": "Darwin"
  },
  "pyjwt": {
    "version": "2.0.0a1"
  }
}

Note: “verify_signature”: False is not recommended for production use.

same problem when trying to validate firebase tokens with the public key https://www.googleapis.com/robot/v1/metadata/x509/securetoken@system.gserviceaccount.com

works only with verify=False

` @app.post(“/auth_firebase”) async def route_firebase_login_access_token(form_data: OAuth2FirebaseTokenRequestForm = Depends()): token = form_data.token

header = jwt.get_unverified_header(token)

kid = header['kid']
alg = header['alg']
public_key = ""

with urllib.request.urlopen("https://www.googleapis.com/robot/v1/metadata/x509/securetoken@system.gserviceaccount.com") as url:
    data = json.loads(url.read().decode())
    public_key = data[kid]

payload = jwt.decode(
    token, 
    algorithms=alg, 
    verify=False
)

# buggy:
#payload = jwt.decode(
#    token, 
#    public_key, 
#    algorithms=alg
#)

return {
    "payload": payload
}

` the token test: http://jwt.io

Just for reference, I’m also using keycloak and for me the issue was line breaks (windows vs linux) and adding linebreaks (after 76 characters) in the keystring (besides adding the header and footer)