datasets: All attempts to get a Google authentication bearer token failed
What I need help with / What I was wondering Due to network problems, I can’t log in to GCs, so there will be this problem
All attempts to get a Google authentication bearer token failed, returning an empty token. Retrieving token from files failed with "Not found: Could not locate the credentials file.". Retrieving token from GCE failed with "Failed precondition: Error executing an HTTP request: libcurl code 6 meaning 'Couldn't resolve host name', error details: Couldn't resolve host 'metadata'".
So I wonder if there’s a way to block this.
At present, my requirement is to load my own dataset and use instructions:
tfds new my_dataset
tfds build my_dataset
Error in TFDs build step。
What I’ve tried so far My version and try:
tensorflow-gpu:2.0 2.1 and 2.3
tensorflow-dataset:4.1.0,3.0.0 and 2.1.0
tfds:tfds-nightly
According to the error log, trace to / core/ builder.py
tensorflow_datasets/core/dataset_builder.py", line 180, in __init__
self.info.initialize_from_bucket()
I have tried to modify the code, but it seems that there are many changes, so I hope to have a switch like variable to control this
It would be nice if… Could we have done anything to make things better (documentation, etc.)?
Environment information (if applicable)
- Operating System: Ubuntu 18.04
- Python version: 3.7
tensorflow-datasets/tfds-nightlyversion: 3.0.0 and tfds night-lytensorflow/tensorflow-gpu/tf-nightly/tf-nightly-gpuversion: 2.3
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 24 (7 by maintainers)
Commits related to this issue
- Disable GCE token when accessing public bucket (Fix #2761) PiperOrigin-RevId: 350331529 — committed to tensorflow/datasets by Conchylicultor 3 years ago
Running these two together gets rid of all of the error messaging:
tfds.core.utils.gcs_utils._is_gcs_disabled = Trueos.environ['NO_GCE_CHECK'] = 'true'But the dataset is loaded correctly right ? This is only a warning message which can be ignored. You can set
If you want to silence those messages
I have met the same problem. And I replace the tensorflow-datasets of version 4.1.0 with 2.1.0. And my tensorflow is v2.3.0 also. It works.
You can disable this GCS nonsense as follows:
tensorflow_datasets.core.utils.gcs_utils._is_gcs_disabled = TrueAfter search for a few hours, I find this works for me! Thank you very much!
As far as I understand, the code is working properly. This is simply a warning message indicating that GCS is used as guest. As our bucket is public, this can be safely ignored.