beam: [Bug]: Could not find a version that matches protobuf

What happened?

Cannot install latest apache-beam[gcp]==2.42.0 because there is a conflict of protobuf version dependencies.

Could not find a version that matches protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<4,<4.0.0dev,<5.0.0dev,>=3.12.0,>=3.12.2,>=3.15.0,>=3.19.0,>=3.19.5,>=4.21.6 (from proto-plus==1.22.1->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
Tried: 2.0.3, 2.3.0, 2.4.1, 2.5.0, 2.6.0, 2.6.1, 3.0.0, 3.0.0, 3.1.0, 3.1.0.post1, 3.1.0.post1, 3.2.0, 3.2.0, 3.3.0, 3.4.0, 3.4.0, 3.5.0.post1, 3.5.0.post1, 3.5.1, 3.5.1, 3.5.2, 3.5.2, 3.5.2.post1, 3.5.2.post1, 3.6.0, 3.6.0, 3.6.1, 3.6.1, 3.7.0, 3.7.0, 3.7.1, 3.7.1, 3.8.0, 3.8.0, 3.9.0, 3.9.0, 3.9.1, 3.9.1, 3.9.2, 3.9.2, 3.10.0, 3.10.0, 3.11.0, 3.11.0, 3.11.1, 3.11.1, 3.11.2, 3.11.2, 3.11.3, 3.11.3, 3.12.2, 3.12.2, 3.12.4, 3.12.4, 3.13.0, 3.13.0, 3.14.0, 3.14.0, 3.15.0, 3.15.0, 3.15.1, 3.15.1, 3.15.2, 3.15.2, 3.15.3, 3.15.3, 3.15.4, 3.15.4, 3.15.5, 3.15.5, 3.15.6, 3.15.6, 3.15.7, 3.15.7, 3.15.8, 3.15.8, 3.16.0, 3.16.0, 3.17.0, 3.17.0, 3.17.0, 3.17.1, 3.17.1, 3.17.1, 3.17.2, 3.17.2, 3.17.2, 3.17.3, 3.17.3, 3.17.3, 3.18.0, 3.18.0, 3.18.0, 3.18.1, 3.18.1, 3.18.1, 3.18.3, 3.18.3, 3.18.3, 3.19.0, 3.19.0, 3.19.0, 3.19.1, 3.19.1, 3.19.1, 3.19.2, 3.19.2, 3.19.2, 3.19.3, 3.19.3, 3.19.3, 3.19.4, 3.19.4, 3.19.4, 3.19.5, 3.19.5, 3.19.5, 3.19.6, 3.19.6, 3.19.6, 3.20.0, 3.20.0, 3.20.0, 3.20.1, 3.20.1, 3.20.1, 3.20.2, 3.20.2, 3.20.2, 3.20.3, 3.20.3, 3.20.3, 4.21.0, 4.21.0, 4.21.0, 4.21.0, 4.21.1, 4.21.1, 4.21.1, 4.21.1, 4.21.2, 4.21.2, 4.21.2, 4.21.2, 4.21.3, 4.21.3, 4.21.3, 4.21.3, 4.21.4, 4.21.4, 4.21.4, 4.21.4, 4.21.5, 4.21.5, 4.21.5, 4.21.5, 4.21.6, 4.21.6, 4.21.6, 4.21.6, 4.21.7, 4.21.7, 4.21.7, 4.21.7, 4.21.8, 4.21.8, 4.21.8, 4.21.8, 4.21.9, 4.21.9, 4.21.9, 4.21.9
Skipped pre-versions: 2.0.0b0, 3.0.0a2, 3.0.0a3, 3.0.0b1, 3.0.0b1.post1, 3.0.0b1.post2, 3.0.0b2, 3.0.0b2, 3.0.0b2.post1, 3.0.0b2.post1, 3.0.0b2.post2, 3.0.0b2.post2, 3.0.0b3, 3.0.0b4, 3.0.0b4, 3.2.0rc1, 3.2.0rc1, 3.2.0rc1.post1, 3.2.0rc1.post1, 3.2.0rc2, 3.2.0rc2, 3.7.0rc2, 3.7.0rc2, 3.7.0rc3, 3.7.0rc3, 3.8.0rc1, 3.8.0rc1, 3.9.0rc1, 3.9.0rc1, 3.10.0rc1, 3.10.0rc1, 3.11.0rc1, 3.11.0rc1, 3.11.0rc2, 3.11.0rc2, 3.13.0rc3, 3.13.0rc3, 3.14.0rc1, 3.14.0rc1, 3.14.0rc2, 3.14.0rc2, 3.14.0rc3, 3.14.0rc3, 3.15.0rc1, 3.15.0rc1, 3.15.0rc2, 3.15.0rc2, 3.16.0rc1, 3.16.0rc1, 3.16.0rc2, 3.16.0rc2, 3.17.0rc1, 3.17.0rc1, 3.17.0rc1, 3.17.0rc2, 3.17.0rc2, 3.17.0rc2, 3.18.0rc1, 3.18.0rc1, 3.18.0rc1, 3.18.0rc2, 3.18.0rc2, 3.18.0rc2, 3.19.0rc1, 3.19.0rc1, 3.19.0rc1, 3.19.0rc2, 3.19.0rc2, 3.19.0rc2, 3.20.0rc1, 3.20.0rc1, 3.20.0rc1, 3.20.0rc2, 3.20.0rc2, 3.20.0rc2, 3.20.1rc1, 3.20.1rc1, 3.20.1rc1, 4.0.0rc1, 4.0.0rc1, 4.0.0rc2, 4.0.0rc2, 4.21.0rc1, 4.21.0rc1, 4.21.0rc1, 4.21.0rc1, 4.21.0rc2, 4.21.0rc2, 4.21.0rc2, 4.21.0rc2
There are incompatible versions in the resolved dependencies:
  protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5 (from google-api-core[grpc,grpcgcp]==2.10.2->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
  protobuf<5.0.0dev,>=3.19.0 (from google-cloud-recommendations-ai==0.7.1->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
  protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5 (from google-cloud-dlp==3.9.2->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
  protobuf>=4.21.6 (from grpcio-status==1.50.0->google-cloud-pubsub==2.13.10->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
  protobuf<4.0.0dev (from google-cloud-bigtable==1.7.2->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
  protobuf<4.0.0dev,>=3.12.0 (from google-cloud-bigquery==2.34.4->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
  protobuf<5.0.0dev,>=3.15.0 (from googleapis-common-protos==1.56.4->google-api-core==2.10.2->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
  protobuf<4.0.0dev (from google-cloud-datastore==1.15.5->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
  protobuf<4.0.0dev,>=3.19.0 (from google-cloud-bigquery-storage==2.13.2->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
  protobuf<4.0.0dev (from google-cloud-language==1.3.2->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
  protobuf<4.0.0dev (from google-cloud-videointelligence==1.16.3->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
  protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5 (from google-cloud-pubsub==2.13.10->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
  protobuf<4.0.0dev (from google-cloud-spanner==1.19.3->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
  protobuf<5.0.0dev,>=3.19.0 (from proto-plus==1.22.1->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
  protobuf<4.0.0dev (from google-cloud-vision==1.0.2->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
  protobuf<4,>=3.12.2 (from apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))

Issue Priority

Priority: 1

Issue Component

Component: io-py-gcp

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 17 (10 by maintainers)

Most upvoted comments

intresting. is poetry using : the metadata from https://packaging.python.org/en/latest/specifications/core-metadata/#requires-dist-multiple-use / https://pypi.org/pypi/apache-beam/2.43.0/json as SOT?

it appears to be generated incorrectly. @AnandInguva I think we could fix it by adding a platform constraint (see https://peps.python.org/pep-0508/) instead of evaluating the platform dynamically in https://github.com/apache/beam/blob/f40057b7fb68ca556be7fbd0cc420311937dcf80/sdks/python/setup.py#L183

echo -e "apache-beam[gcp]\ngoogle-cloud-secret-manager" > requirements.in
pip-compile

It is due to pip-compile resolver.

You can try using

pip-compile --resolver backtracking with latest version of pip-tools. You can find more details at https://github.com/jazzband/pip-tools/issues/1190

It seems like a bug in dependency resolution in piptools. But it may be also time to try to upgrade our protobuf depdendency. Opened https://github.com/apache/beam/issues/24569