poetry: Tensorflow 2.13.0 cannot be imported after install

Poetry version: Poetry (version 1.5.1)
Python version: Python 3.11.4
OS version and name: macOS Ventura 13.3 M2 Chip macOS-13.3-arm64-arm-64bit
pyproject.toml: pyproject.toml

I am on the latest stable Poetry version, installed using a recommended method.
I have searched the issues of this repo and believe that this is not a duplicate.
I have consulted the FAQ and blog for any relevant entries or release notes.
If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

After installing tensorflow 2.13.0 with above pyproject.toml on macOs Apple Sillicon , importing fails with

ModuleNotFoundError: No module named 'tensorflow

Same method works under linux. HINT: in tensorflow 2.13 the authors introduce platform specific install for pip. Under macOs arm it is supposed to install tensorflow-macos . under linux there is another dependency tree

About this issue

Original URL
State: open
Created a year ago
Reactions: 9
Comments: 27 (7 by maintainers)

Most upvoted comments

For people still looking for a full combo workaround, this should work for tensorflow 2.11 in all situations (in reality I haven’t tested everything). Don’t hesitate to comment with your problems so that we improve on this. I’ve tested :

new apple silicon
docker in new apple silicon
standard linux

This should work with windows, new mac, old mac, docker in new mac, amd and non-amd linux machines … EDIT : Tested :

windows by @Vuizur

Currently, I advise to copy the full code below in your pyproject.toml in the poetry’s dependencies. Using a poetry group, [tool.poetry.group.tensorflow.dependencies] section could be a good thing to be able to isolate it :

# Issue between poetry and tensorflow metadata since >=2.11
# This is a temporary workaround
# related to https://github.com/python-poetry/poetry/issues/8271
# Inspired from https://github.com/tensorflow/tensorflow/blob/adb39b04e9cb116df4659a7e2de9eea27e62f25c/tensorflow/tools/pip_package/setup.py#L148-L162
tensorflow = {version = "^2.13.0" }
tensorflow-macos = { version = "^2.13.0", platform = "darwin", markers = "platform_machine=='arm64'" }
tensorflow-intel = { version = "^2.13.0", platform = "win32" }
tensorflow-cpu = [
    { version = "^2.13.0", platform = "linux", markers = "platform_machine!='arm64' and platform_machine!='aarch64'" },
    { version = "^2.13.0", platform = "darwin", markers = "platform_machine!='arm64' and platform_machine!='aarch64'" },
]
tensorflow-cpu-aws = { version = "^2.13.0", platform = "linux", markers = "platform_machine=='arm64' or platform_machine=='aarch64'" }
# https://github.com/tensorflow/tensorflow/blob/adb39b04e9cb116df4659a7e2de9eea27e62f25c/tensorflow/tools/pip_package/setup.py#L107-L108
# https://github.com/python-poetry/poetry/issues/8271#issuecomment-1697740447
tensorflow-io-gcs-filesystem = [
    { version = ">= 0.23.1", markers = "platform_machine!='arm64' or platform_system!='Darwin'" },
    { version = "< 0.32.0", markers = "platform_system == 'Windows'" }
]

EDIT : I fixed above the tensorflow-io-gcs-filesystem issue from @RRiva EDIT 2 : adding @radoering answer EDIT 3 : fixing windows from @Vuizur

+16

Wirg on Sep 9, 2023

I found the solution: Wirg’s solution actually specified the wrong platform for pytorch-intel, one doesn’t have to write windows, but win32. Here is a corrected version:

[tool.poetry.dependencies]
python = "^3.10,<3.12"
# Issue between poetry and tensorflow metadata since >=2.11
# This is a temporary workaround
# related to https://github.com/python-poetry/poetry/issues/8271
# Inspired from https://github.com/tensorflow/tensorflow/blob/adb39b04e9cb116df4659a7e2de9eea27e62f25c/tensorflow/tools/pip_package/setup.py#L148-L162
tensorflow = {version = "^2.13.0" }
tensorflow-macos = { version = "^2.13.0", platform = "darwin", markers = "platform_machine=='arm64'" }
tensorflow-intel = { version = "^2.13.0", platform = "win32" }
tensorflow-cpu = [
    { version = "^2.13.0", platform = "linux", markers = "platform_machine!='arm64' and platform_machine!='aarch64'" },
    { version = "^2.13.0", platform = "darwin", markers = "platform_machine!='arm64' and platform_machine!='aarch64'" },]
tensorflow-cpu-aws = { version = "^2.13.0", platform = "linux", markers = "platform_machine=='arm64' or platform_machine=='aarch64'" }
# https://github.com/tensorflow/tensorflow/blob/adb39b04e9cb116df4659a7e2de9eea27e62f25c/tensorflow/tools/pip_package/setup.py#L107-L108
# https://github.com/python-poetry/poetry/issues/8271#issuecomment-1697740447
tensorflow-io-gcs-filesystem = [
    { version = ">= 0.23.1", markers = "platform_machine!='arm64' or platform_system!='Darwin'" },
    { version = "< 0.32.0", markers = "platform_system == 'Windows'" }
]

+11

Vuizur on Sep 8, 2023

@dre-hh Have using environment markers to get around the issue?

If we take a look at tensorflow/tools/pip_package/setup.py, we see that they require the tensorflow-macos package where the environment marker has been satisfied.

It’s generated, but would look something like this. tensorflow-macos == 2.15.0 ;platform_system=="Darwin" and platform_machine=="arm64"

Following the Poetry documentation here, your dependency specification should look this.

[tool.poetry.dependencies]
python = ">=3.11,<3.12"
tensorflow-macos = { version = "^2.15.0", markers = "platform_system=='Darwin' and platform_machine=='arm64'" }
tensorflow = "^2.13.0"

I’d tend to agree with @dimbleby, the devs over at @tensorflow should improve the compatibility of their project with package managers, but that’s another battle entirely.

oldfielj-ansto on Aug 17, 2023

The issue has been resolved on Windows: please refer https://github.com/tensorflow/tensorflow/issues/62899#issuecomment-1935052936

mraunak on Feb 8, 2024

https://github.com/python-poetry/poetry/issues/8271#issuecomment-1928038659

Telling users that they should manually add the dependencies that tensorflow forgot to declare is at best a workaround, not a resolution.

dimbleby on Feb 9, 2024

#1616 is about different path dependencies with the same version (and resolved), thus, not related.

It’s related to #2012 and #4046. If there is no wheel for Windows (and no sdist), you’ll want to exclude that version for Windows:

tensorflow-io-gcs-filesystem = [
    { version = ">= 0.23.1", markers = "platform_machine!='arm64' or platform_system!='Darwin'" },
    { version = "< 0.32.0", markers = "platform_system == 'Windows'" }
]

radoering on Aug 29, 2023

Data point: Vuizur 's solution worked for me on M2 max.

Geometrein on Nov 13, 2023

tensorflow’s metadata does not express that dependency:

$ curl -s https://pypi.org/pypi/tensorflow/2.13.0/json | jq '.info.requires_dist'
[
  "absl-py (>=1.0.0)",
  "astunparse (>=1.6.0)",
  "flatbuffers (>=23.1.21)",
  "gast (<=0.4.0,>=0.2.1)",
  "google-pasta (>=0.1.1)",
  "h5py (>=2.9.0)",
  "libclang (>=13.0.0)",
  "numpy (<=1.24.3,>=1.22)",
  "opt-einsum (>=2.3.2)",
  "packaging",
  "protobuf (!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3)",
  "setuptools",
  "six (>=1.12.0)",
  "termcolor (>=1.1.0)",
  "typing-extensions (<4.6.0,>=3.6.6)",
  "wrapt (>=1.11.0)",
  "grpcio (<2.0,>=1.24.3)",
  "tensorboard (<2.14,>=2.13)",
  "tensorflow-estimator (<2.14,>=2.13.0)",
  "keras (<2.14,>=2.13.1)",
  "tensorflow-io-gcs-filesystem (>=0.23.1) ; platform_machine != \"arm64\" or platform_system != \"Darwin\""
]

No mention here of tensorflow-macos.

I guess this is another variation on https://github.com/tensorflow/tensorflow/issues/58674

please encourage the tensorflow folk to publish consistent metadata in all of their wheels, with platform-specific variations described by markers

dimbleby on Aug 4, 2023