pyvista: `ModuleNotFoundError` with `vtk-osmesa` installation

Describe the bug, what’s wrong, and what you expected.

I would like to use PyVista with vtk-osmesa in a Docker environment. Following the instructions from here: https://discourse.vtk.org/t/status-update-vtk-python-wheels/11212, I created the Dockerfile below. After building a container with this Dockerfile and using python within that container, I am unable to import pyvista due to the vtk module not being found. I have confirmed that vtk-osmesa 9.3.0 is installed in that environment, and the original vtk is not installed, as the above link suggests. Is there something missing from this configuration?

Steps to reproduce the bug.

FROM ghcr.io/pyvista/pyvista:latest

RUN pip install pyvista
RUN pip uninstall vtk -y
RUN pip install --extra-index-url https://wheels.vtk.org vtk-osmesa

Run the following commands in the directory with the Dockerfile:

docker build -t test .
docker run -it test python
(in container test python interpreter) >>> import pyvista

The following error is displayed:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/conda/lib/python3.10/site-packages/pyvista/__init__.py", line 9, in <module>
    from pyvista.core import *
  File "/opt/conda/lib/python3.10/site-packages/pyvista/core/__init__.py", line 4, in <module>
    from . import _vtk_core
  File "/opt/conda/lib/python3.10/site-packages/pyvista/core/_vtk_core.py", line 12, in <module>
    from vtkmodules.vtkCommonCore import vtkVersion
ModuleNotFoundError: No module named 'vtkmodules'

System Information

Below is the output of `pip list` in the Docker container:

Package                       Version
----------------------------- -----------
aiohttp                       3.9.1
aiosignal                     1.3.1
alembic                       1.11.1
anyio                         3.6.2
argon2-cffi                   21.3.0
argon2-cffi-bindings          21.2.0
arrow                         1.3.0
asttokens                     2.2.1
async-generator               1.10
async-lru                     2.0.2
async-timeout                 4.0.3
attrs                         23.1.0
Babel                         2.12.1
backcall                      0.2.0
backports.functools-lru-cache 1.6.4
beautifulsoup4                4.12.2
bleach                        6.0.0
blinker                       1.6.2
boltons                       23.0.0
certifi                       2023.5.7
certipy                       0.1.3
cffi                          1.15.1
charset-normalizer            3.1.0
cmocean                       3.0.3
colorama                      0.4.6
colorcet                      3.0.1
comm                          0.1.3
conda                         23.3.1
conda-package-handling        2.0.2
conda_package_streaming       0.8.0
contourpy                     1.2.0
cryptography                  40.0.2
cycler                        0.12.1
debugpy                       1.6.7
decorator                     5.1.1
defusedxml                    0.7.1
entrypoints                   0.4
executing                     1.2.0
fastjsonschema                2.17.1
flit_core                     3.9.0
fonttools                     4.46.0
fqdn                          1.5.1
frozenlist                    1.4.0
greenlet                      2.0.2
idna                          3.4
imageio                       2.33.0
imageio-ffmpeg                0.4.9
importlib-metadata            6.6.0
importlib-resources           5.12.0
ipykernel                     6.23.1
ipython                       8.13.2
ipython-genutils              0.2.0
ipywidgets                    8.1.1
isoduration                   20.11.0
jedi                          0.18.2
Jinja2                        3.1.2
json5                         0.9.5
jsonpatch                     1.32
jsonpointer                   2.0
jsonschema                    4.17.3
jupyter_client                8.2.0
jupyter_core                  5.3.0
jupyter-events                0.6.3
jupyter-lsp                   2.1.0
jupyter_server                2.6.0
jupyter_server_proxy          4.1.0
jupyter_server_terminals      0.4.4
jupyter-telemetry             0.1.0
jupyterhub                    4.0.0
jupyterlab                    4.0.1
jupyterlab-pygments           0.2.2
jupyterlab_server             2.22.1
jupyterlab-widgets            3.0.9
kiwisolver                    1.4.5
libmambapy                    1.4.2
Mako                          1.2.4
mamba                         1.4.2
markdown-it-py                3.0.0
MarkupSafe                    2.1.2
matplotlib                    3.8.2
matplotlib-inline             0.1.6
mdurl                         0.1.2
meshio                        5.3.4
mistune                       2.0.5
multidict                     6.0.4
nbclassic                     1.0.0
nbclient                      0.8.0
nbconvert                     7.4.0
nbformat                      5.8.0
nest-asyncio                  1.5.6
notebook                      6.5.4
notebook_shim                 0.2.3
numpy                         1.26.2
oauthlib                      3.2.2
overrides                     7.3.1
packaging                     23.1
pamela                        1.0.0
pandocfilters                 1.5.0
param                         2.0.1
parso                         0.8.3
pexpect                       4.8.0
pickleshare                   0.7.5
Pillow                        10.1.0
pip                           23.1.2
pkgutil_resolve_name          1.3.10
platformdirs                  3.5.1
pluggy                        1.0.0
pooch                         1.8.0
prometheus-client             0.17.0
prompt-toolkit                3.0.38
psutil                        5.9.5
ptyprocess                    0.7.0
pure-eval                     0.2.2
pycosat                       0.6.4
pycparser                     2.21
pyct                          0.5.0
pycurl                        7.45.1
Pygments                      2.15.1
PyJWT                         2.7.0
pyOpenSSL                     23.1.1
pyparsing                     3.1.1
pyrsistent                    0.19.3
PySocks                       1.7.1
python-dateutil               2.8.2
python-json-logger            2.0.7
pytz                          2023.3
pyvista                       0.43.0
PyYAML                        6.0
pyzmq                         25.0.2
requests                      2.31.0
rfc3339-validator             0.1.4
rfc3986-validator             0.1.1
rich                          13.7.0
ruamel.yaml                   0.17.29
ruamel.yaml.clib              0.2.7
scooby                        0.9.2
Send2Trash                    1.8.2
setuptools                    67.7.2
simpervisor                   1.0.0
six                           1.16.0
sniffio                       1.3.0
soupsieve                     2.3.2.post1
SQLAlchemy                    2.0.15
stack-data                    0.6.2
terminado                     0.17.1
tinycss2                      1.2.1
tomli                         2.0.1
toolz                         0.12.0
tornado                       6.3.2
tqdm                          4.65.0
traitlets                     5.9.0
trame                         3.3.0
trame-client                  2.13.0
trame-server                  2.12.1
trame-vtk                     2.6.2
trame-vuetify                 2.3.1
trimesh                       4.0.5
types-python-dateutil         2.8.19.14
typing_extensions             4.6.2
typing-utils                  0.1.0
uri-template                  1.3.0
urllib3                       2.0.2
vtk-osmesa                    9.3.0
wcwidth                       0.2.6
webcolors                     1.13
webencodings                  0.5.1
websocket-client              1.5.2
wheel                         0.40.0
widgetsnbextension            4.0.9
wslink                        1.12.4
yarl                          1.9.4
zipp                          3.15.0
zstandard                     0.19.0

Screenshots

No response

About this issue

Original URL
State: closed
Created 7 months ago
Comments: 16 (16 by maintainers)

Most upvoted comments

@jourdain and I found the problem with these examples I provided. Each time I repeated the installation instructions, some version of vtk-osmesa had been installed prior. This installation is broken when vtk is uninstalled, but pip cannot tell the difference.

In the docker environment, using the base image ghcr.io/pyvista/pyvista:latest, vtk-osmesa appears to already be installed. So when executing the recommended steps, the following is happening:

pip install pyvista: install vtk and override vtk-osmesa
pip uninstall vtk: remove vtk and leave vtk-osmesa empty
pip install --extra-index-url https://wheels.vtk.org vtk-osmesa: pip thinks vtk-osmesa is already installed, performs a no-op. But this installation has been made unusable after the removal of the vtk directory.

We figured this out because of the experiments I had been attempting in a native venv. I had been trying different versions of vtk-osmesa, and each time repeating the above steps. But without fully removing vtk-osmesa prior to repeating these steps, the installation had been a no-op each time, and the vtkmodules directory was always missing due to the removal of vtk.

Fix: using a python base image (not the pyvista base image) and performing the three above commands works as expected.

TLDR: Ensure no version of vtk-osmesa exists in the environment before running the recommended installation steps.

annehaley on Dec 8, 2023

yes that was the reason of the issue. We were running the install twice (base image + ours) which was confusing pip.

jourdain on Dec 8, 2023

@banesullivan was there a reason libosmesa6 was removed in #4686? Should it be re-added?

No, this is now bundled in the upstream vtk-osmesa. The selling point of this new wheel is that it bundle’s everything needed directly in the wheel.

banesullivan on Dec 8, 2023

PyVista works with vtk-osmesa at least in certain instances, see devcontainer example I gave. I tested with 9.3.0 and it worked for me. We should drill down to the root cause of the issue. I’m suspecting it is a quirk of the Docker container environment and vtk-osmesa. This would be important to figure out as it would be good to give a better error message in PyVista if that is the case.

I’m wondering if you install the osmesa libraries whether it will import correctly. And if so, whether we should put that into our docker image/container.

MatthewFlamm on Dec 8, 2023

I think something might be up with the latest version of vtk-osmesa.

jourdain on Dec 8, 2023