mozilla-django-oidc: OIDC callback state not found in session `oidc_states`!

After a lot of research I was able to identify the source of the problem. I will try to briefly explain why the error occurs and how to solve it.

By default, the first endpoint called is /oidc/authenticate/ which redirects to a page that will be responsible for logging in and then redirects again to /oidc/callback/ sending some information.

We can divide this into 2 steps: 1 - Call the /oidc/authenticate/ endpoint by sending a payload containing a randomly generated state; 2 - Receive in /oidc/callback/ via redirect the authentication code and the state generated in the previous step.

In the first step, the library generates a very important value, the state and save it in the session. In the second step, the state value received after the redirect is checked, and if not found in the session, raises this error OIDC callback state not found in session oidc_states!. Reference: https://github.com/mozilla/mozilla-django-oidc/blob/master/mozilla_django_oidc/views.py#L88

This whole process happens very quickly. The problem is there, when there are too many simultaneous requests to /oidc/authenticate/, depending on which session engine you are using, the session cannot be saved in time before the second step.

I thought of some ways to solve this problem, and I will quote the simplest ones: Warning: These codes are mere examples, implement in a way that meets your needs.

1. The less elegant way: Inherit the OIDCCallbackClass class and override the get method by adding a time.sleep(1) and configure the project to use your class instead of the default one. example/views.py

   import time
   
   from mozilla_django_oidc.views import OIDCCallbackClass
   
   
   class MyOIDCCallbackClass(OIDCCallbackClass):
       def get(self, request):
           time.sleep(1) 
           super().get(request)
   

   settings.py

   ...
   OIDC_CALLBACK_CLASS = "example.views.MyOIDCCallbackClass"
   ...
   

2. Change the SESSION_ENGINE: By default django uses the database to save sessions, there are several positives to using it, but in our case, we need the session to be saved very quickly. I recommend that you read about it before making any changes: https://docs.djangoproject.com/en/4.0/topics/http/sessions/

   settings.py

   ...
   SESSION_ENGINE = "django.contrib.sessions.backends.cache"
   ...
   

   According to django’s own documentation, this is the most performant engine, especially if you use Redis. This is the way I consider the most elegant and it solved my problems.

This is the summary of my studies, I may be wrong or incomplete, I would love to receive opinions. Hope this helps.

Any update on this issue? We see this exception quite regularly (a few times a day). We’re somewhat reluctant to change our Session Backend as described in https://github.com/mozilla/mozilla-django-oidc/issues/435#issuecomment-1036372844; especially to a purely in-memory one.

After reading up on Django Session Management, the comment from @MrJizzler (https://github.com/mozilla/mozilla-django-oidc/issues/435#issuecomment-1025231256) seems totally reasonable. The very same problem is listed in the Django documentation (https://docs.djangoproject.com/en/3.2/topics/http/sessions/#when-sessions-are-saved): The implementation of add_state_and_nonce_to_session alters the oidc_states dict in the session if it is already present (e.g. to a previous/parallel login flow) and thus the session may not be stored.

I believe to have found one way this can happen:

When a user sends two requests requiring authentication roughly at the same time, e.g. because they reopened their browser with old tabs, and the session refresh middleware is being used, both requests try to modify the session to add their state, however only as there is no locking, the changes of one of the two request are overwritten by the other request.

We see this issue on our site and added some more logging to track it down, and we see patterns like this from clients:

1. GET /object/foo Adds state state_a and returns 302 redirecting to identity provider because oidc login is too old
2. GET /object/bar Adds state state_b and returns 302 redirecting to identity provider because oidc login is too old
3. GET /oidc/callback/?code=bla&state=state_a triggers “OIDC callback state not found”

According to our debug log, request 1 and request 2 each add their state to the session.

However, in the log output for request 3, we see that state_b is present in oidc_states, but state_a is missing.

Looking closer at request 2, we can see that state_a is missing in its oidc_states.

So I would assume that something like the following is happening:

1. request 1 loads the session with oidc_states = {}
2. request 2 loads the session with oidc_states = {}
3. request 1 modifies and saves its local copy of the session, so oidc_states = { 'state_a': {...} } is in the backend
4. request 2 modifies and saves its local copy of the session, so oidc_states = { 'state_b': {...} } is in the backend

While actually oidc_states = { 'state_a': {...}, 'state_b': {...} } should be in the backend.

I am not sure how this should best be solved, and I believe that the specifics might also depend on the Session Backend that is being used.

Not sure if this fixes the issue, but the def add_state_and_nonce_to_session function in utils.py does not call request.session.modified = True as described here https://docs.djangoproject.com/en/3.2/topics/http/sessions/#when-sessions-are-saved. This allows for multiple session states to be saved.

You needs add class of authentication backend and orverride create_user method to add is_staff or is_superuser to True. Add AUTHENTICATION_BACKENDS in settings.py refering to you custom class backent Example:

# auth_backends.py

from mozilla_django_oidc.auth import OIDCAuthenticationBackend


class KeycloakOIDCAuthenticationBackend(OIDCAuthenticationBackend):

    def create_user(self, claims):
        """ Overrides Authentication Backend so that Django users are
            created with the keycloak preferred_username.
            If nothing found matching the email, then try the username.
        """
        user = super(KeycloakOIDCAuthenticationBackend, self).create_user(claims)
        user.first_name = claims.get('given_name', '')
        user.last_name = claims.get('family_name', '')
        user.email = claims.get('email')
        user.is_staff = True #Here fix that error
        user.username = claims.get('preferred_username')
        user.save()
        return user

    def filter_users_by_claims(self, claims):
        """ Return all users matching the specified email.
            If nothing found matching the email, then try the username
        """
        email = claims.get('email')

        if not email:
            return self.UserModel.objects.none()
        users = self.UserModel.objects.filter(email__iexact=email)
        return users

    def update_user(self, user, claims):
        user.first_name = claims.get('given_name', '')
        user.last_name = claims.get('family_name', '')
        user.email = claims.get('email')
        user.save()
        return user

In settings.py

#settings.py
AUTHENTICATION_BACKENDS = (
    'my_app.auth_backends.KeycloakOIDCAuthenticationBackend',
)