dvc: DVC+Google Cloud: Could not automatically determine credentials (GOOGLE_APPLICATION_CREDENTIALS not set)

Trying to run dvc pull with a Google Cloud remote, got this error message:

ERROR: unexpected error - Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create credentials and re-run the application. For more information, please see https://cloud.google.com/docs/authentication/getting-started

gsutil ls gs://my_bucket worked fine.

Running gcloud auth login didn’t help, but gcloud beta auth application-default login solved the problem.

$ echo ${GOOGLE_APPLICATION_CREDENTIALS}

$ dvc --version
0.77.3

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.3 LTS
Release:        18.04
Codename:       bionic

DVC was installed using sudo pip install dvc[all].

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 6
Comments: 24 (12 by maintainers)

Most upvoted comments

I had the error below when I tried to push to a newly created GS remote, while regular gsutil commands worked fine. gcloud auth login didn’t fix the problem, but gcloud beta auth application-default login did.

Huge thanks to @pmrowla for helping me figure this out!

fedorov on Oct 28, 2020

I think this resolved it for me

export GOOGLE_APPLICATION_CREDENTIALS="/Path/to/my/keyfile.json" where this is the JSON key file generated for the google service account.

Hopefully this monologue is helpful to someone!

vade on Mar 15, 2020

And an additional question from the same user:

succeeded with a service account approach, but only after I set the role for it to “Owner”. Feels like a sledgehammer approach. What are the minimum required role(s) for a GS bucket service account?

(Should this be a separate question issue?) Cc @mvshmakov maybe remembers 🙂

Feels like @Suor can help with that. Sorry for such a late response, I’ve missed the notification.

mvshmakov on Oct 28, 2020

My 2cts: For someone used to S3 remotes, the GCP track is much more painful. It seems like using a GCP bucket is very different than one on S3.

drorata on Sep 21, 2020

@vade I tried to “play” with this a little bit more … could you please, try to use

dvc remote modify storage credentialpath /Path/to/my/keyfile.json for the service account

where keyfile.json is credentials file for the service account that has proper access.

that alone worked for me.

Though env variable should be fine also.

I’ll review the docs PR and probably simplify a bit (to put links to the Google official auth docs mostly instead of replicating auth flow on our end).

shcheklein on Mar 15, 2020