scipy: BUG: Unable to use scipy development environment from IDE

Describe your issue.

After following the steps to build scipy from source (https://scipy.github.io/devdocs/building/index.html#building-from-source):

mamba env create -f environment.yml
mamba activate scipy-dev
python dev.py build

I was unable to use the environment in the IDE (e.g. in the Jupyter Interactive Window in VSCode), as the scipy package cannot be imported: ModuleNotFoundError: No module named 'scipy'

scipy is not present in the list of packages in the environment: conda list | grep scipy

Presumably, this might be fixed once support for editable installs is added (https://github.com/scipy/scipy/issues/16293).

Reproducing Code Example

import scipy

Error message

ModuleNotFoundError: No module named 'scipy'

SciPy/NumPy/Python version and system information

1.12.0.dev0+1341.4172aee 1.25.1 sys.version_info(major=3, minor=10, micro=12, releaselevel='final', serial=0)
Build Dependencies:
  blas:
    detection method: pkgconfig
    found: true
    include directory: /Users/artem/micromamba/envs/scipy-dev/include
    lib directory: /Users/artem/micromamba/envs/scipy-dev/lib
    name: openblas
    openblas configuration: USE_64BITINT=0 DYNAMIC_ARCH=1 DYNAMIC_OLDER= NO_CBLAS=
      NO_LAPACK=0 NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP=1 CORE2 MAX_THREADS=128
    pc file directory: /Users/artem/micromamba/envs/scipy-dev/lib/pkgconfig
    version: 0.3.23
  lapack:
    detection method: pkgconfig
    found: true
    include directory: /Users/artem/micromamba/envs/scipy-dev/include
    lib directory: /Users/artem/micromamba/envs/scipy-dev/lib
    name: openblas
    openblas configuration: USE_64BITINT=0 DYNAMIC_ARCH=1 DYNAMIC_OLDER= NO_CBLAS=
      NO_LAPACK=0 NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP=1 CORE2 MAX_THREADS=128
    pc file directory: /Users/artem/micromamba/envs/scipy-dev/lib/pkgconfig
    version: 0.3.23
  pybind11:
    detection method: pkgconfig
    include directory: /Users/artem/micromamba/envs/scipy-dev/include
    name: pybind11
    version: 2.11.0
Compilers:
  c:
    args: -march=core2, -mtune=haswell, -mssse3, -ftree-vectorize, -fPIC, -fPIE, -fstack-protector-strong,
      -O2, -pipe, -isystem, /Users/artem/micromamba/envs/scipy-dev/include, -D_FORTIFY_SOURCE=2,
      -isystem, /Users/artem/micromamba/envs/scipy-dev/include
    commands: x86_64-apple-darwin13.4.0-clang
    linker: ld64
    linker args: -Wl,-pie, -Wl,-headerpad_max_install_names, -Wl,-dead_strip_dylibs,
      -Wl,-rpath,/Users/artem/micromamba/envs/scipy-dev/lib, -L/Users/artem/micromamba/envs/scipy-dev/lib,
      -march=core2, -mtune=haswell, -mssse3, -ftree-vectorize, -fPIC, -fPIE, -fstack-protector-strong,
      -O2, -pipe, -isystem, /Users/artem/micromamba/envs/scipy-dev/include, -D_FORTIFY_SOURCE=2,
      -isystem, /Users/artem/micromamba/envs/scipy-dev/include
    name: clang
    version: 15.0.7
  c++:
    args: -march=core2, -mtune=haswell, -mssse3, -ftree-vectorize, -fPIC, -fPIE, -fstack-protector-strong,
      -O2, -pipe, -stdlib=libc++, -fvisibility-inlines-hidden, -fmessage-length=0,
      -isystem, /Users/artem/micromamba/envs/scipy-dev/include, -D_FORTIFY_SOURCE=2,
      -isystem, /Users/artem/micromamba/envs/scipy-dev/include
    commands: x86_64-apple-darwin13.4.0-clang++
    linker: ld64
    linker args: -Wl,-pie, -Wl,-headerpad_max_install_names, -Wl,-dead_strip_dylibs,
      -Wl,-rpath,/Users/artem/micromamba/envs/scipy-dev/lib, -L/Users/artem/micromamba/envs/scipy-dev/lib,
      -march=core2, -mtune=haswell, -mssse3, -ftree-vectorize, -fPIC, -fPIE, -fstack-protector-strong,
      -O2, -pipe, -stdlib=libc++, -fvisibility-inlines-hidden, -fmessage-length=0,
      -isystem, /Users/artem/micromamba/envs/scipy-dev/include, -D_FORTIFY_SOURCE=2,
      -isystem, /Users/artem/micromamba/envs/scipy-dev/include
    name: clang
    version: 15.0.7
  cython:
    commands: cython
    linker: cython
    name: cython
    version: 0.29.36
  fortran:
    args: -march=core2, -mtune=haswell, -ftree-vectorize, -fPIC, -fstack-protector,
      -O2, -pipe, -isystem, /Users/artem/micromamba/envs/scipy-dev/include
    commands: /Users/artem/micromamba/envs/scipy-dev/bin/x86_64-apple-darwin13.4.0-gfortran
    linker: ld64
    linker args: -Wl,-pie, -Wl,-headerpad_max_install_names, -Wl,-dead_strip_dylibs,
      -Wl,-rpath,/Users/artem/micromamba/envs/scipy-dev/lib, -L/Users/artem/micromamba/envs/scipy-dev/lib,
      -march=core2, -mtune=haswell, -ftree-vectorize, -fPIC, -fstack-protector, -O2,
      -pipe, -isystem, /Users/artem/micromamba/envs/scipy-dev/include
    name: gcc
    version: 12.2.0
  pythran:
    include directory: ../../../../micromamba/envs/scipy-dev/lib/python3.10/site-packages/pythran
    version: 0.13.1
Machine Information:
  build:
    cpu: x86_64
    endian: little
    family: x86_64
    system: darwin
  cross-compiled: false
  host:
    cpu: x86_64
    endian: little
    family: x86_64
    system: darwin
Python Information:
  path: /Users/artem/micromamba/envs/scipy-dev/bin/python3.10
  version: '3.10'

About this issue

Original URL
State: closed
Created a year ago
Comments: 30 (30 by maintainers)

Commits related to this issue

DOC: add notes on editable installs under the Building from source This has been quite stable for some months, so it's time to add this as an option to the docs. Closes gh-18900 [skip cirrus] [skip... — committed to rgommers/scipy by rgommers 7 months ago

Most upvoted comments

Last time we discussed editable installed, you mentioned it was not ready for prime time. Has this changed or are we still waiting for some changes?

I think it’s ready for early adopters, but as Matt found it is still a little rough unless you know exactly what to do. I have a plan to fix the double pip install -e . --no-build-isolation thing in Pythran. I’ve verified that for NumPy that actually works fine; the problem in SciPy is a recursion error where scipy builds with pythran, and pythran tries to import scipy during the build.

Should we start adding the capability to dev.py?

Maybe / perhaps / not sure. We’ve always had a split between in-place builds and dev.py/runtests.py. In a way it’s quite attractive to unify them, as long as it’s equally robust. I think once the above pythran-related issue is resolved, we can try it.

rgommers on Jul 24, 2023

I just thought I’d report that I’ve been using editable install + PyCharm on an M1 pretty successfully for a while now. I was able to do some development today using editable install + PyCharm on a Windows machine, too. I can run unit tests individually using the features of the IDE, I can run scripts using the “play” button, and I can debug tests and scripts, too. The only hiccup is https://github.com/scipy/scipy/issues/18900#issuecomment-1657258983, but I think that might just be on the Mac, and either way it might just be a PyCharm oddity.

mdhaber on Oct 10, 2023

Presumably, this might be fixed once support for editable installs is added (#16293).

This works already, you can use pip install -e . --no-build-isolation. See https://meson-python.readthedocs.io/en/latest/how-to-guides/editable-installs.html. Note that it’s important to then only use that, and never dev.py; it’s one or the other.

rgommers on Jul 22, 2023