setuptools: [BUG] setuptools 62.1.0 breaks project that requires fortran sources (Nonexistent include directory)

setuptools version

62.1.0

Python version

3.10

OS

Linux, macOS

Additional environment information

I am using the version supplied by conda-forge

Description

We have a project that requires compiling fortran source files, and everything has worked fine with setuptools until version 62.1.0 (62.0.0, and all other lower versions, work!). The problem exists using macos and linux with github actions, as well as on my own macos machine. I re-emphasize, that this is a problem ONLY with 62.1.0.

I will admit that I have no idea how to debug this problem: in the build process, all of the source and module files compile, but at the very end when we try to compile the python wrapper, it complains about a non-existent include directory, which prevents the source file from finding a pre-compiled module.

INFO: gfortran:f90: src/PythonWrapper.f95
      f951: Warning: Nonexistent include directory ‘build/temp.linux-x86_64-3.8’ [-Wmissing-include-dirs]
      src/PythonWrapper.f95:2:12:
      
          2 |         use shtools, only: PlmBar
            |            1
      Fatal Error: Cannot open module file ‘shtools.mod’ for reading at (1): No such file or directory
      compilation terminated.

Expected behavior

compile with no errors, as it has been doing for the past several years…

How to Reproduce

Try to install the pyshtools project: https://github.com/SHTOOLS/SHTOOLS using pip install -v -e .

Output

Please see the github actions output for this run:

https://github.com/SHTOOLS/SHTOOLS/runs/5987783398?check_suite_focus=true

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 24 (14 by maintainers)

Commits related to this issue

pin setuptools to <=62.0.0 (62.1.0 is broken) Pin setuptools to version 62.0.0 because 62.1.0 is broken: https://github.com/pypa/setuptools/issues/3263 — committed to conda-forge/pyshtools-feedstock by MarkWieczorek 2 years ago

Most upvoted comments

distutils should find the fftw library, but we always had problems with this working properly. The easiest way would be to install fftw with conda.

MarkWieczorek on Apr 27, 2022

This probably comes from this StackOverflow question:

https://stackoverflow.com/questions/14320220/testing-python-c-libraries-get-build-path

There’s a big warning there “This is an internal detail of setuptools (based on the bundled version of distutils), and so is not guaranteed to be stable.” even before I edited it to include the current Setuptools. The internal build directory is a private detail and building the name yourself is going to be prone to breakage.

Look at the edited response to see how you can build this (and there’s also an answer that avoids that using build_lib from build, if that’s accessible).

henryiii on Apr 21, 2022

This hardcoded assumption about the build directory is now wrong:

https://github.com/SHTOOLS/SHTOOLS/blob/8d83c42d1313d5624c4db8c2e57300c5d819834e/setup.py#L177

henryiii on Apr 12, 2022

Build scripts may need to be updated to the new path names if they are using the internal pathname. build/temp.linux-x86_64-3.10 clashes when you are using an alternative interpreter, like pypy, so the actual build tag is now used, temp.linux-x86_64-cpython-310; otherwise rebuilding for pypy can cause hard errors or two sets of binaries in the wheels. Hard coded build paths (like in https://github.com/scikit-hep/awkward-1.0/pull/1407) have to be loosened. In that case, it was a hard coded grep for the build directory in the setup.py.

henryiii on Apr 12, 2022