google-cloud-ruby: Caller does not have storage.buckets.get access

gcloud auth activate-service-account --key-file "service-account.json"
gsutil ls gs://some-bucket ### this works fine!

via ruby it doesn’t work using the same service-account.json! why?!

storage = Google::Cloud::Storage.new(
      project: 'someproject', keyfile: 'service-account.json')
storage.bucket('some-bucket').files ...

… forbidden: Caller does not have storage.buckets.get access to bucket ‘some-bucket’

About this issue

Original URL
State: closed
Created 7 years ago
Reactions: 5
Comments: 15 (12 by maintainers)

Links to this issue

java - Running Apache Beam pipeline in Spring Boot project on Google Data Flow - Stack Overflow

Commits related to this issue

use skip_lookup as documented in: https://github.com/GoogleCloudPlatform/google-cloud-ruby/issues/1588#issuecomment-319741972 — committed to carrierwaveuploader/carrierwave-google-storage by jasdeepsingh 6 years ago
Fix Travis CI (#23) * remove the script from before_install * try manual encryption as specified in Travis docs * use skip_lookup as documented in: https://github.com/GoogleCloudPlatform/google... — committed to carrierwaveuploader/carrierwave-google-storage by jasdeepsingh 6 years ago

Most upvoted comments

We have added the ability to create Bucket and File objects without first accessing the Storage API by using skip_lookup in the 1.4.0 release.

This means that you should be able to accomplish this using the following code:

require "google/cloud/storage"

storage = Google::Cloud::Storage.new

bucket = storage.bucket "some-bucket", skip_lookup: true
files = bucket.files

+14

blowmage on Aug 2, 2017

Does OOP say everything have to be instantiated and self-inspected? I don’t think so. It is about how we write code, not how it is being executed.
It is lazyness and Ruby is fine with it.

One of the benefits is that my code raises an exception when I call bucket.storage "bucket-that-doesn't exit" rather than raising an exception later on, which may be more challenging to debug.

I won’t say it’s challenging. Just put it in documentation that this stuff is lazy and if you really want to fail immediately do some intermediate check. I would love skip_lookup to be true by default and stating false would be that check.

Nakilon on Feb 9, 2018

I can think of two possible solutions for this issue:

Expose Objects.list as a top-level method (Project#files) that accepts the bucket name.
Add a mode (option) to Project#bucket to return a stub containing the given bucket name without retrieving the bucket metadata. This stub could then be used to call Bucket#files without permissions to the bucket.

quartzmo on Jul 14, 2017

Unfortunately, it is not possible to list files without access to the bucket that contains them. This is due to the current design of the library, which requires that the bucket is loaded before listing its files.

why?!

I believe one of the founding goals of this project was a “clean, OOP-inspired design”, which comes at some cost to flexibility. For more flexibility, the google-api-client/Google/Apis/StorageV1/StorageService offers a “flatter” API, although authentication is a bit more involved.

quartzmo on Jul 13, 2017