argo-workflows: "token not valid for running mode" after SSO authentication

Summary

Argo UI comes back to login page with error {"code":16,"message":"token not valid for running mode”} after completing SSO user authentication.

Diagnostics

argo version: argo: v2.12.3 BuildDate: 2021-01-05T00:49:48Z GitCommit: 93ee530126cc1fc154ada84d5656ca82d491dc7f GitTreeState: clean GitTag: v2.12.3 GoVersion: go1.13 Compiler: gc Platform: darwin/amd64

I have added sso configuration in the config map “workflow-controller-configmap”

sso: |
      issuer: https://cloudsso-test.xxx.com
      clientId:
        name: argo-server-sso
        key: clientID
      clientSecret:
        name: argo-server-sso
        key: clientSecret
      redirectUrl: http://argo-code.xxx.com/workflows/argo
      scopes:
       - groups
       - email
      rbac:
        enabled: false

I have added the required secrets for SSO and also starting argo-server in “–auth-mode sso”

argo-server logs :

time="2021-02-02T07:34:01.198Z" level=info msg="finished unary call with code Unauthenticated" error="rpc error: code = Unauthenticated desc = token not valid for running mode" grpc.code=Unauthenticated grpc.method=GetVersion grpc.service=info.InfoService grpc.start_time="2021-02-02T07:34:01Z" grpc.time_ms=0.34 span.kind=server system=grpc
time="2021-02-02T07:34:01.218Z" level=info msg="finished unary call with code Unauthenticated" error="rpc error: code = Unauthenticated desc = token not valid for running mode" grpc.code=Unauthenticated grpc.method=ListWorkflowTemplates grpc.service=workflowtemplate.WorkflowTemplateService grpc.start_time="2021-02-02T07:34:01Z" grpc.time_ms=0.065 span.kind=server system=grpc

Please note we use JWT token for SSO and not JWE.

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.

About this issue

Original URL
State: closed
Created 3 years ago
Reactions: 28
Comments: 24 (9 by maintainers)

Commits related to this issue

fix(ui): strip inner quotes from argoToken closes #4991 Signed-off-by: Michael Ruoss <michael.ruoss@ufirstgroup.com> — committed to ufirstgroup/argo-workflows by deleted user 3 years ago

Most upvoted comments

I’m also having problems getting service account bearier tokens working with --auth-mode sso

Behaviour I would like is real users using SSO to login, but also supporting ServiceAccounts with Bearier tokens for service-to-service API calls.

+17

paveq on Feb 23, 2021

Just FYI, the solution for us was to ensure we were specifying both SSO and Client with the server configuration: https://github.com/argoproj/argo-workflows/issues/7490

Hopefully this helps someone else 😃

mrsiesta on Jan 24, 2022

Alright - I think I just got lucky and found the problem. I assume you copied the credentials from the “Using Your Login With The CLI” dialogue in the UI. If I do that, for the token it does this:

export ARGO_TOKEN='"Bearer ..."'

if I remove the inner quotes it works:

export ARGO_TOKEN='Bearer ...'

mruoss on Apr 12, 2021

missing `Bearer` in our case

Ran into this when switching from --auth-mode=server to a combination of --auth-mode=sso for UI usage and --auth-mode=client for programmatic API access.

In our case, --auth-mode=server was allowing an incorrect SA token access to the API, and we did not realize this until we removed --auth-mode=server (previously we had --auth-mode=server for UI access with an NGINX OAuth redirect + --auth-mode=client for programmatic API access).

The problem was we were missing Bearer in front of the token entirely. Even when logging in the UI with a token, you have to input Bearer insertmytokenhere and it will not work without that. Bearer is also case-sensitive.

If you run into this, I strongly recommend testing the token in the UI – that was a very helpful validation step for us to make sure we were on the right track.

DX improvements to error messages etc

This was a misconfiguration on our side, but I think the error message that Argo gives back could be a lot more detailed. Many people (including various teams I work with) were confused by token not valid for running mode and thought that --auth-mode=client was incompatible with --auth-mode=sso or something, rather than the token just being incorrect.

--auth-mode=server is also, just, well, incredibly permissive even when used in combination with other modes, which can cause some confusion. Perhaps disallowing other modes when server is used might make sense? Not sure of the proper fix there, but I think some DX improvements could be made there as well.

agilgur5 on Mar 13, 2023

@mruoss would you like to sumbit a PR to fix this mistake?

alexec on Apr 14, 2021