google-cloud-python: Bigquery: KeyError: 'configuration'
We are using google-cloud-bigquery==0.32.0, and have an hourly script that monitors the status of async queries. The code basically looks like this:
from google.cloud import bigquery
bq_client = bigquery.Client(project=GCLOUD_PROJECT_ID_BIGQUERY)
all_jobs = bq_client.list_jobs(all_users=True)
for job in all_jobs:
analyse_job(job)
Excerpt of installed google cloud packages:
google-api-core==1.1.1
google-api-python-client==1.6.6
google-auth==1.4.1
google-auth-httplib2==0.0.3
google-cloud-bigquery==0.32.0
google-cloud-core==0.28.1
google-cloud-monitoring==0.28.1
google-cloud-storage==1.8.0
google-resumable-media==0.3.1
googleapis-common-protos==1.5.3
Starting from 20/04/2018 08:00 AM UTC the script started to fail with the following stacktrace:
Traceback (most recent call last):
14:26:19 File "/home/script_repos/bigquery/monitor_jobs.py", line 214, in <module>
14:26:19 main()
14:26:19 File "/home/script_repos/bigquery/monitor_jobs.py", line 175, in main
14:26:19 for idx, job in enumerate(all_jobs):
14:26:19 File "/home/script_repos/bigquery/lib/python2.7/site-packages/google/api_core/page_iterator.py", line 200, in _items_iter
14:26:19 for item in page:
14:26:19 File "/home/script_repos/bigquery/lib/python2.7/site-packages/google/api_core/page_iterator.py", line 125, in next
14:26:19 result = self._item_to_value(self._parent, item)
14:26:19 File "/home/script_repos/bigquery/lib/python2.7/site-packages/google/cloud/bigquery/client.py", line 1396, in _item_to_job
14:26:19 return iterator.client.job_from_resource(resource)
14:26:19 File "/home/script_repos/bigquery/lib/python2.7/site-packages/google/cloud/bigquery/client.py", line 554, in job_from_resource
14:26:19 config = resource['configuration']
14:26:19 KeyError: 'configuration'
Strange thing is that with the exact same virtual environment on my local computer, I don’t get this error. My local dev computer is a macOS High Sierra 10.13.3, while the server is running CentOS Linux 7. Both use Python 2.7.12. Do you guys have any idea what can possibly cause the error and why I am not seeing it locally?
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 16 (10 by maintainers)
Yes, it happens when calling
list_jobswithall_users=Truewhile not having thebigquery.jobs.listAllpermission.