nodejs-storage: "Cannot sign data without `client_email`" for getSignedUrl
I’m trying to get a signed url from a storage bucket, but am getting a signing error. I have permission to read from the bucket, and have been able to stream its contents fine.
Code:
...
const expires = new Date()
expires.setSeconds(expires.getSeconds() + 30)
const storage = new Storage({ projectId });
storage
.bucket(bucketName)
.file(`${directory}/${packageName}`)
.getSignedUrl({
expires,
action: 'read',
})
.then(url => {
res.send(url)
console.log(url)
})
.catch(console.log)
Error:
{ SigningError: Cannot sign data without `client_email`.
at /Users/pikelnys/Desktop/code/gcs-project/api/node_modules/@google-cloud/storage/src/file.js:1784:16
at Auth._signWithApi (/Users/pikelnys/Desktop/code/gcs-project/api/node_modules/google-auto-auth/index.js:331:7)
at getCredentials (/Users/pikelnys/Desktop/code/gcs-project/api/node_modules/google-auto-auth/index.js:316:14)
at googleAuthClient.getCredentials (/Users/pikelnys/Desktop/code/gcs-project/api/node_modules/google-auto-auth/index.js:194:9)
at /Users/pikelnys/Desktop/code/gcs-project/api/node_modules/google-auth-library/build/src/auth/googleauth.js:602:67
at <anonymous>
at process._tickCallback (internal/process/next_tick.js:188:7) message: 'Cannot sign data without `client_email`.' }
Environment details
- OS: macOS 10.13.4
- Node.js version: 8.11.2
- npm version: 6.4.1
@google-cloud/storageversion: 1.7.0
Edit: I’m getting this in both a local environment (where I authenticate with gcloud auth application-default login) and while the code is deployed to cloud functions.
About this issue
- Original URL
- State: closed
- Created 8 years ago
- Reactions: 20
- Comments: 48 (12 by maintainers)
Is this possible to authenticate without a keyfile?
Service account isn’t good for multi-stage environments and security. This should be reopened. You should be using the exposed environment that has the same values.
Hi @pikelnys.
You will have to specify keyFilename while creating the Storage object if you are deploying locally.
keyFilename is not required if you have deployed your code under the same project your Google Cloud Bucket was created. You just have to ensure you have the right access by having a look at your service-account-key.
This is well explained here
The error says cannot sign without client_email. client_email is available inside the service-account-key.json but was not provided while creating the object hence the error. This error would not have arrived had you deployed your code inside the same project as your Cloud Bucket was.
Here is a sample service-account-key.json file. Look for client_email in the file.
Steps to Follow.
Download the service account key in JSON format and specify its path inside keyFilename. Default service-account-key provided by Google has read-only access. So make sure if you want to write to your bucket you would have to change the permissions inside the service-account.
Here is the Google Codelab link for reference
Hope this resolves your issue.
For anyone else that comes across this issue, here is a bit more info for posterity.
The GCP client libraries docs use the Storage libary as the example when leveraging ADC locally. The same docs also call out the fact that using service accounts locally is bad practice (see the Note in this section).
However, for some reason Google seems to contradict itself with the implementation of the
getSignedUrlmethod and its corresponding endpoint. After a couple hours of debugging and re-reading the disparate and disconnected docs scattered all over the place, I found the source of the issue as documented here. V4 (and the legacy V2) signing requires usage of a service account.It would be nice if Google would have bothered to call this out in the ADC docs where they use Storage as an example, or better yet, return the relevant instructions in the SigningError message, especially because they seemingly contradict themselves within the different docs on related topics.
Hope this helps someone!
I was getting “Cannot sign data without
client_email” error when runningfirebase emulators:start.I fixed it by setting up a service account and downloading the service account .json credentials file as
service_account.json, then runningGOOGLE_APPLICATION_CREDENTIALS="service_account.json" firebase emulators:startBUMP as there isn’t seem to be a way to deal with it via
emulatorwithout exposingdev/prodconfigs.Got the issue and solved it thanks to this solution firebase-adminsdk-ov3eh@staging-louve.iam.gserviceaccount.com thanks @christiangenco
Jeez. Enabling OIDC has been a massive challenge with the garbage documentation out there.
Anyhow, after hours of fighting, here’s how I could sign the URL using service account impersonation:
1st. Create the service account:
2nd. Create the
local_dev_rolethat will be attached to the service account:roles-local.gcp.yml:
3rd. Attach the service account at the project level
Note: There’s a similar command called
gcloud iam service-accounts add-iam-policy-binding… Don’t! This will not work.4th. Impersonate the service account:
If that didn’t work, and you’re getting errors about the account not getting impersonated:
Attach the
roles/iam.serviceAccountTokenCreatorto YOUR email address:And just for the sake of completeness: Here’s the full Storage class wrapper I’ve created in Node:
Note: See that I only pass the
projectId(YOUR_PROJECT_ID) and the bucket name.Additional Notes:
Helpful Commands For Debugging:
Just realized the previously linked issue was merged in last year and the
client_emailissue was resolved (no longer required) for impersonated service accounts. I just updated my library version and can confirm no workaround is necessary anymore. Adding my version info for posterity:bump. this should be re-opened.
I can’t believe it’s been six years since I ran into this issue and it’s still not resolved. Since then, I found a work around for the issue, the project died, I quit that job, and the company went out of business.
For me it is while using firebase emulator, I just want to be able to sign local storage URLs. Nothing cloud side. Everything just local!
Bump. Firebase emulator should emulate the environment, this appears to be a failure state
@dinkel-katilyst I found this public issue that was filed against
gcloud cli. For yourself and anyone else, I would suggest making additional noise on that issue to try and get it visible again. In the meantime I found the same issue internally and I will reach out to the team and link them to this issue. Can’t promise anything, but I’ll try what I can to get this either fixed or rationale why it was never done.I see a few bumps on this old issue so wanted to leave a comment. It seems that the issue most are bumping about is trying to sign using ADC which does not include the
client_emailfield. Unfortunately, this library does not own the JSON file produced during login nor the signing process. Signing with impersonation should now work and might be one alternative to explore.If there is a different issue, I would ask that you open a new issue so that I can investigate / resolve as appropriate.
I don’t think you should switch to a json keyfile. I just added an npm command to add the
client_emailto the application default credentials after login using the json lib.I haven’t deployed yet to Google Cloud, but I think you should be able to run this login command locally only, and it should just work when deployed to cloud?
Edit: added
--impersonate-service-accountas a required argumentI recently wrote a Medium article describing a workaround I created for local development. There is also a git repo with my solution. Article | Repo I use IAM Credentials API coupled with a cloud function. Hope this helps someone.
Circling back. I spoke with the teams who own the login / writing of the ADC JSON file. This isn’t just a matter of adding a
client_emailto the JSON as this alone would not allow for signing. It would require involvement of HMAC keys and from what I was told this is beyond the scope of ADC.The best course of action is to utilize SA impersonation for signing in these situations.
@dinkel-katilyst if you just add
client_emailof the impersonated service account after signing in (which will clear the client_email), this is the quickest workaround. I automated it as a single npm command in my previous solutionAny workaround?
Double bump. There’s not a great firebase-admin solution for this when running in the emulator without using a service account json. It’s messy, insecure, and inconvenient for testing…
It’s really easy on the frontend with Firebase Auth. Ex:
@howlettga I have tried using the solution you suggest by adding in a
client_emailfield to the existing ADC, but that doesn’t seem to work now. Were you able to get it to work?The only way I can get it working currently is by passing in a service account key when using locally to generate a Signed URL.
Thanks @tritone, appreciate the response!
Hey @timbuckiii and @ollyde , just wanted to note a couple things:
Also, in the future, if you could open a new issue for any problems rather than commenting on a closed one, that would help us respond faster. Thanks!