datasets: All attempts to get a Google authentication bearer token failed

What I need help with / What I was wondering Due to network problems, I can’t log in to GCs, so there will be this problem

 All attempts to get a Google authentication bearer token failed, returning an empty token. Retrieving token from files failed with "Not found: Could not locate the credentials file.". Retrieving token from GCE failed with "Failed precondition: Error executing an HTTP request: libcurl code 6 meaning 'Couldn't resolve host name', error details: Couldn't resolve host 'metadata'".

So I wonder if there’s a way to block this.

At present, my requirement is to load my own dataset and use instructions:

tfds new my_dataset
tfds build my_dataset

Error in TFDs build step。

What I’ve tried so far My version and try:

tensorflow-gpu:2.0 2.1 and 2.3
tensorflow-dataset:4.1.0,3.0.0 and 2.1.0
tfds:tfds-nightly

According to the error log, trace to / core/ builder.py

tensorflow_datasets/core/dataset_builder.py", line 180, in __init__
    self.info.initialize_from_bucket()

I have tried to modify the code, but it seems that there are many changes, so I hope to have a switch like variable to control this

It would be nice if… Could we have done anything to make things better (documentation, etc.)?

Environment information (if applicable)

  • Operating System: Ubuntu 18.04
  • Python version: 3.7
  • tensorflow-datasets/tfds-nightly version: 3.0.0 and tfds night-ly
  • tensorflow/tensorflow-gpu/tf-nightly/tf-nightly-gpu version: 2.3

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 24 (7 by maintainers)

Commits related to this issue

Most upvoted comments

Running these two together gets rid of all of the error messaging:

tfds.core.utils.gcs_utils._is_gcs_disabled = True os.environ['NO_GCE_CHECK'] = 'true'

But the dataset is loaded correctly right ? This is only a warning message which can be ignored. You can set

os.environ['NO_GCE_CHECK'] = 'true'

If you want to silence those messages

I have met the same problem. And I replace the tensorflow-datasets of version 4.1.0 with 2.1.0. And my tensorflow is v2.3.0 also. It works.

You can disable this GCS nonsense as follows:

tensorflow_datasets.core.utils.gcs_utils._is_gcs_disabled = True

Running these two together gets rid of all of the error messaging:

tfds.core.utils.gcs_utils._is_gcs_disabled = True os.environ['NO_GCE_CHECK'] = 'true'

After search for a few hours, I find this works for me! Thank you very much!

As far as I understand, the code is working properly. This is simply a warning message indicating that GCS is used as guest. As our bucket is public, this can be safely ignored.