dex: Google connector with ADC: unexpected end of JSON input

Preflight Checklist

  • I agree to follow the Code of Conduct that this project adheres to.
  • I have searched the issue tracker for an issue that matches the one I want to file, without success.
  • I am not looking for support or already pursued the available support channels without success.

Version

ghcr.io/dexidp/dex:v2.34.0

Storage Type

Kubernetes

Installation Type

Other (specify below)

Expected Behavior

google connector successfully use Application Default Credential via Compute Engine metadata server

Actual Behavior

failed to initialize server: server: Failed to open connector google: failed to open connector: failed to create connector google: could not create directory service: unable to parse credentials to config: unexpected end of JSON input

Steps To Reproduce

  1. deploy dex on GCE / GKE. Use google connector without specifying serviceAccountFilePath
  2. check the logs

Additional Information

I installed dex via ArgoCD’s install.yaml, and replaced its version from v2.30.2 to v2.34.0. My setup worked OK before, with a key file in serviceAccountFilePath.

I tried running the following code on the same GKE pod by kubectl exec, and it went all good.

func main() {
        cred, err := google.FindDefaultCredentials(context.Background())
        if err != nil {
                fmt.Errorf("FindDefaultCredentials: %w", err)
        }
        fmt.Printf("Json: %v\n", cred.JSON)
        cfg, err := google.JWTConfigFromJSON(cred.JSON, admin.AdminDirectoryGroupReadonlyScope)
        if err != nil {
                fmt.Errorf("JWTConfigFromJSON: %w\n", err)
        }
        fmt.Printf("cfg: %v\n", cfg)
}

The above code works OK with following output, which is something I expected.

cred: &{my-project-name 0xc0000aed00 []}
Json:
cfg: <nil>

The code does basically the same as dex v2.34.0, so I am not sure why it doesn’t work in dex.

cc @ichbinfrog

Configuration

connectors:
    - type: google
      id: google
      name: Google
      config:
        issuer: https://accounts.google.com
        clientID: $GOOGLE_CLIENT_ID
        clientSecret: $GOOGLE_CLIENT_SECRET
        redirectURI: https://my.domain/api/dex/callback
        hostedDomains: ['my.domain']
        serviceAccountFilePath: /tmp/keyfile.json
        adminEmail: test@my.domain

Logs

Dex Version: v2.34.0-dirty, Go Version: go1.19.1, Go OS/ARCH: linux amd64
config issuer: https://***
config connector: google
config skipping approval screen
config refresh tokens rotation enabled: true
the application default credential is used since the service account file path is not used
failed to initialize server: server: Failed to open connector google: failed to open connector: failed to create connector google: could not create directory service: unable to parse credentials to config: unexpected end of JSON input

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 18 (10 by maintainers)

Most upvoted comments

I have submitted a pull request addressing the mentioned issue: https://github.com/dexidp/dex/pull/2989. It would be greatly appreciated if someone could test it.

It is possible to achieve what you want by constructing your desired JWT and passing it to the IAM Service Account Credentials API (see the signJwt method).

Nowadays you can grant service accounts workspace admin roles without domain wide delegation. That’s probably a more comfortable route.

Pitching in here that we managed to get dex running with groups on Anthos Bare Metal (no metadata server) with workload identity using these changes.

The issue is that google.JWTConfigFromJSON will fail for Anthos setups because they use federated tokens and their credential setup for ADC is a little different than on GKE:

{
  "type": "external_account",
  "audience": "identitynamespace:WORKLOAD_IDENTITY_POOL:IDENTITY_PROVIDER",
  "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/GSA_NAME@GSA_PROJECT_ID.iam.gserviceaccount.com:generateAccessToken",
  "subject_token_type": "urn:ietf:params:oauth:token-type:jwt",
  "token_url": "https://sts.googleapis.com/v1/token",
  "credential_source": {
    "file": "/var/run/secrets/tokens/gcp-ksa/token"
  }
}

See the docs for reference.

@milesarmstrong we are still trying to determine the correct solution for the problem.

Chances are the latest release does NOT work with workload identity. The current workaround is manually passing an IAM service account to the Dex workload. It should work with the latest release.

I know it’s not ideal, but that’s the currently available workaround.

Although we wanted to fix the problem in the next release (which is due this week), it may not just be part of it.