google-cloud-python: Bigquery: import error with v.1.23.0

Steps to reproduce

Using google.cloud.bigquery v.1.23.0

https://pypi.org/project/google-cloud-bigquery/1.23.0/

Insead of v.1.22.0 works fine.

Code example

from concurrent.futures import ThreadPoolExecutor
from logging import StreamHandler, Formatter, INFO, getLogger
from datetime import datetime, timedelta, timezone
 
from google.cloud import bigquery
from google.cloud.bigquery import LoadJobConfig
from google.cloud.bigquery import SchemaField

Stack trace

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<command-778378> in <module>
      3 from datetime import datetime, timedelta, timezone
      4 
----> 5 from google.cloud import bigquery
      6 from google.cloud.bigquery import LoadJobConfig
      7 from google.cloud.bigquery import SchemaField

/databricks/python/lib/python3.7/site-packages/google/cloud/bigquery/__init__.py in <module>
     33 __version__ = get_distribution("google-cloud-bigquery").version
     34 
---> 35 from google.cloud.bigquery.client import Client
     36 from google.cloud.bigquery.dataset import AccessEntry
     37 from google.cloud.bigquery.dataset import Dataset

/databricks/python/lib/python3.7/site-packages/google/cloud/bigquery/client.py in <module>
     56 from google.cloud.bigquery._helpers import _verify_job_config_type
     57 from google.cloud.bigquery._http import Connection
---> 58 from google.cloud.bigquery import _pandas_helpers
     59 from google.cloud.bigquery.dataset import Dataset
     60 from google.cloud.bigquery.dataset import DatasetListItem

/databricks/python/lib/python3.7/site-packages/google/cloud/bigquery/_pandas_helpers.py in <module>
     38     pyarrow = None
     39 
---> 40 from google.cloud.bigquery import schema
     41 
     42 

/databricks/python/lib/python3.7/site-packages/google/cloud/bigquery/schema.py in <module>
     15 """Schemas for BigQuery tables / queries."""
     16 
---> 17 from six.moves import collections_abc
     18 
     19 from google.cloud.bigquery_v2 import types

ImportError: cannot import name 'collections_abc' from 'six.moves' (unknown location)

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 27
Comments: 36 (9 by maintainers)

Links to this issue

Dataproc job has failed with an import error | Google Cloud

Commits related to this issue

Add 'six' as bigquery dependency Workaround for https://github.com/googleapis/google-cloud-python/issues/9965 — committed to allegro/bigflow by anjensan 4 years ago

Most upvoted comments

I am reproducing this issue with these versions: google-cloud-bigquery==1.24.0 six==1.14.0

+10

mkalpar on Mar 26, 2020

Adding six==1.13.0 to the requirements file of the main project worked for me

+10

winsonhys on Dec 13, 2019

Hi,

Would it be possible for either google-cloud-bigquery to explicitly list that it requires six >= 1.13.0, or for google-cloud-core to increase its lower bound and google-cloud-bigquery require the new version?

The problem is that pip install --upgrade on projects that depend upon google-cloud-bigquery now fail, because google-cloud-core incorrectly asserts that it only needs six >= 1.10.0 (see here).

I agree that this problem can be fixed by manually upgrading six, but the point of listing dependencies and their versions is that this happens automatically.

beckjake on Dec 13, 2019

@HemangChothani Shouldn’t it work out of the box after installing google.cloud.bigquerywithout any need to install further packages?

SaschaHeyer on Dec 12, 2019

Still facing this issue. My jenkins deployment pipeline broke with the same message.

Packages on my server:

google-cloud-bigquery==2.0.0
six==1.15.0

Please advice.

anistark on Oct 6, 2020

Sorry that you have encountered this issue. I should have checked the minimum version of six for the collections.abc fix for Python 3.8. I do see it was added in 1.13.0. https://github.com/benjaminp/six/blob/203b81c2a719466ed13681f0062a4426c07c7481/CHANGES#L11

Indeed, the correct fix is for bigquery to add six >=1.13.0,<2.0.0dev as a dependency in setup.py.

tswast on Dec 13, 2019

My pipeline broke today. I install google-cloud-bigquery using bootstrap script while creating dataproc cluster and i’m getting the following error after submitting my spark job :

    from google.cloud.bigquery.client import Client
  File "/usr/local/lib/python2.7/dist-packages/google/cloud/bigquery/client.py", line 58, in <module>
    from google.cloud.bigquery import _pandas_helpers
  File "/usr/local/lib/python2.7/dist-packages/google/cloud/bigquery/_pandas_helpers.py", line 40, in <module>
    from google.cloud.bigquery import schema
  File "/usr/local/lib/python2.7/dist-packages/google/cloud/bigquery/schema.py", line 17, in <module>
    from six.moves import collections_abc
ImportError: cannot import name collections_abc

So i’ve migrated to previous version(1.22.0) and it’s working fine.

shihabuddinbuet on Dec 12, 2019

@jordangonzales Latest version of google.cloud.bigquery package is 1.23.1. Try downgrading it to 1.23.0 as it’s working with six==1.14

sohamsil on Jan 28, 2020

@smdmts @shihabuddinbuet @SaschaHeyer Please update or install the latest version of six which is 1.13.0. It’s due to the older version six like 1.12.0.

HemangChothani on Dec 12, 2019

google-cloud-bigquery==1.24.0 six==1.14 works with python 2.7 but not python 3.7

pblin on Oct 23, 2020