gspread: Cannot use default service account in Compute Engine

I’m trying to figure out how I can use the service account associated with a Compute Engine without the need of a JSON file. I cant seem to be able to do that in gspread.

Google has an example here: https://cloud.google.com/docs/authentication/production

` from google.cloud import storage

# If you don't specify credentials when constructing the client, the
# client library will look for credentials in the environment.
storage_client = storage.Client()

# Make an authenticated API request
buckets = list(storage_client.list_buckets())
print(buckets)`

For them it just works. No JSON necessary. Can this be replicated in any way in gspread?

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 8
Comments: 30

Most upvoted comments

@niczky12 Maybe try the following:

Check the type of creds: type(creds) it must return google.oauth2.service_account.Credentials
Check the service account e-mail: creds.service_account_email it must be the service account email you expect
Check if the spreadsheet is shared with that service account email (I assume that’s fine since you mentioned it works with JSON)

If the two first steps fail, make sure your Compute Engine / Google Kubernetes Engine / App Engine / Cloud Run / Cloud Function have the right service account set in its configurations. If you don’t do that you’ll have the default service account which might not be the same you’re trying to use.

If you want to test that on your local computer, try setting the environment variable GOOGLE_APPLICATION_CREDENTIALS to the path of the service account. That’s not exactly what happens inside the google engines but google.auth.default should treat the same.

Finally, you can also try using from googleapiclient._auth import default_credentials instead of google.auth.default since that one is a bit more generic as it includes oauth2client.

gpkc on Dec 10, 2020

Hi,

I know this is quite an old thread but I was also looking for a solution for a long time.

In my case it helped to add additional scopes for VM :

gcloud compute instances set-service-account INSTANCE_NAME --scopes=cloud-platform,https://www.googleapis.com/auth/drive --service-account=SERVICE_ACCOUNT_EMAIL --zone=ZONE

Of course, in scopes you also need to specify others that you want to have. After this operation, the problem of insufficient permissions disappeared. I hope it can help someone.

Best

PCichowczyc on Dec 21, 2022

I’m trying to do the same thing with no luck. I was trying to use google.auth like so:

import google.auth
import gspread

DEFAULT_SCOPES =['https://www.googleapis.com/auth/spreadsheets','https://www.googleapis.com/auth/drive']
creds, _ = google.auth.default(scopes=DEFAULT_SCOPES)

gc = gspread.authorize(creds)

gc.create("hello world")

But this gives: gspread.exceptions.APIError: {'errors': [{'domain': 'global', 'reason': 'insufficientPermissions', 'message': 'Insufficient Permission: Request had insufficient authentication scopes.'}], 'code': 403, 'message': 'Insufficient Permission: Request had insufficient authentication scopes.'}

Works fine with a json key, but I’m trying to avoid that.

niczky12 on Dec 4, 2020

for anyone stumbling on this like I did, it’s covered here: https://github.com/googleapis/google-auth-library-python/issues/1211 as apparently by design.

My workaround:

import google.auth
import google.auth.transport.requests
SCOPES = []
credentials, project = google.auth.default(scopes=SCOPES)
google_request = google.auth.transport.requests.Request()
credentials.refresh(google_request)
logger.debug(f"CREDENTIAL EMAIL: {credentials.service_account_email}")

jeffbryner on Dec 13, 2023

@niczky12 Oops. I spoke too soon it seems. I’ve updated my comment. The code ran fine on my local machine so I assumed it’d work on a compute engine as well. I should’ve tested it. My bad.

When running the same code in a Compute Engine Instance, I get the following error.

APIError: {'code': 403, 'message': 'Request had insufficient authentication scopes.', 'status': 'PERMISSION_DENIED'}

The instance service account does have access to the google sheet I’m using.

ghost on Jan 8, 2021