scikit-learn: BUG: Build from source fails for scikit-learn v1.6.1 on Windows 11 with Visual Studio Build Tools 2022, Ninja subprocess error
Describe the bug
First of all, thank you guys for the fantastic job with Sklearn. I’m trying to build from source to start contributing to the project, but it ended with me bringing more issues to you. After struggling for some days with this problem, I’m seeking help. Maybe if you have some clue or workaround, I could open a Pull Request with the solution for this.
I am following the guidelines for Contributing with Scikit-learn, and for that, it is necessary to Build from source on Windows, which recomends install Build Tools for Visual Studio 2019, but nowadays is not possible to download the 2019 version just the Build Tools for Visual Studio 2022 installer.
The installation of Build Tools for Visual Studio 2022 runs smoothly(and also the initialization of its Environment), as well as the creation of the Python virtual environment and the installation of the packages wheel, numpy, scipy, cython, meson-python, ninja.
But in the step of building from source using the pip install --editable, the build breaks when Compiling C objects after some C4090 warnings, throwing an metadata-generation-failed error from a subprocess of ninja build.
This seems related/similar to issue #31123. Despite not being the same problem, if we find a solution, it may work for both issues.
Steps/Code to Reproduce
I have tried the steps using pip install and also conda-forge in different versions of Python: pip :{3.10.11, 3.12.7} conda:{ 3.13.2} to check if it was a problem with Python/pip itself.
- Environment Setup:
- OS: Windows 11 Pro, Version 24H2, OS build 26100.3476
- System type: 64-bit operating system, x64-based processor
- Installed Visual Studio Build 2022, version 17.13.5 (It is not possible to download from Microsoft the 2019 version as recommended)
- I also restarted the computer after installing (because it is Windows, so you never know)
- Project directory
C:\Users\vitor.pohlenz\Documents\Python\vpz\scikit-learn
- Installing compiler Installed Microsoft build Tools 2022
Initialized the enviroment:
SET DISTUTILS_USE_SDK=1
"C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\Build\vcvarsall.bat" x64
Which gives the output:
**********************************************************************
** Visual Studio 2022 Developer Command Prompt v17.13.5
** Copyright (c) 2022 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x64'
- Steps Using pip, the same error occurs using Python 3.10 or 3.12. I will provide the steps and logs just for 3.10 for the sake of simplicity.
Navigated to the project directory: C:\Users\vitor.pohlenz\Documents\Python\vpz\scikit-learn
Created the Python virtual environment, upgraded pip, and installed the dependencies
py -3.10 -m venv envsklearn
envsklearn\Scripts\python.exe -m pip install --upgrade pip
envsklearn\Scripts\activate
pip install wheel numpy scipy cython meson-python ninja
Until here, everything works as expected. But when building from source, in the command below, the error occurs:
pip install --editable . --verbose --no-build-isolation --config-settings editable-verbose=true
- Using conda forge:
To check if it wasn’t a problem with pip I have also used
conda-forge, as described in the Building from Source tutorial:
conda create -n sklearn-env -c conda-forge python numpy scipy cython meson-python ninja
conda activate sklearn-env
pip install --editable . --verbose --no-build-isolation --config-settings editable-verbose=true
Note: I used 2 different environment names to avoid confusion:
envsklearnis thevenvthat usespip, andsklearn-envis theconda env
Expected Results
The build of scikit-learn should finish and install without error, or at least with a few warnings.
Actual Results
Using pip install
It seems that the Meson build system finds the compiler for C, C+++, and also the ninja.exe in the environment. But ninja subprocess breaks when compiling C object sklearn/metrics/_dist_metrics.cp310-win_amd64.pyd.p/meson-generated_sklearn_metrics__dist_metrics.pyx.c.obj
Error message:
[...]
[46/164] Compiling C object sklearn/metrics/_dist_metrics.cp310-win_amd64.pyd.p/meson-generated_sklearn_metrics__dist_metrics.pyx.c.obj
sklearn/metrics/_dist_metrics.cp310-win_amd64.pyd.p/sklearn/metrics/_dist_metrics.pyx.c(29605): warning C4090: '=': different 'const' qualifiers
sklearn/metrics/_dist_metrics.cp310-win_amd64.pyd.p/sklearn/metrics/_dist_metrics.pyx.c(30390): warning C4090: '=': different 'const' qualifiers
sklearn/metrics/_dist_metrics.cp310-win_amd64.pyd.p/sklearn/metrics/_dist_metrics.pyx.c(49229): warning C4090: '=': different 'const' qualifiers
sklearn/metrics/_dist_metrics.cp310-win_amd64.pyd.p/sklearn/metrics/_dist_metrics.pyx.c(50014): warning C4090: '=': different 'const' qualifiers
ninja: build stopped: subcommand failed.
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: C:\Users\vitor.pohlenz\Documents\Python\vpz\scikit-learn\envsklearn\Scripts\ninja.EXE
error: subprocess-exited-with-error
× Preparing editable metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
full command: 'C:\Users\vitor.pohlenz\Documents\Python\vpz\scikit-learn\envsklearn\Scripts\python.exe' 'C:\Users\vitor.pohlenz\Documents\Python\vpz\scikit-learn\envsklearn\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py' prepare_metadata_for_build_editable 'C:\Users\VITOR~1.POH\AppData\Local\Temp\tmpp45ks547'
cwd: C:\Users\vitor.pohlenz\Documents\Python\vpz\scikit-learn
Preparing editable metadata (pyproject.toml) ... error
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
Full Log
Using pip 25.0.1 from C:\Users\vitor.pohlenz\Documents\Python\vpz\scikit-learn\envsklearn\lib\site-packages\pip (python 3.10)
Obtaining file:///C:/Users/vitor.pohlenz/Documents/Python/vpz/scikit-learn
Running command Checking if build backend supports build_editable
Checking if build backend supports build_editable ... done
Running command Preparing editable metadata (pyproject.toml)
+ meson setup --reconfigure C:\Users\vitor.pohlenz\Documents\Python\vpz\scikit-learn C:\Users\vitor.pohlenz\Documents\Python\vpz\scikit-learn\build\cp310 -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --native-file=C:\Users\vitor.pohlenz\Documents\Python\vpz\scikit-learn\build\cp310\meson-python-native-file.ini
Cleaning... 0 files.
The Meson build system
Version: 1.7.1
Source dir: C:\Users\vitor.pohlenz\Documents\Python\vpz\scikit-learn
Build dir: C:\Users\vitor.pohlenz\Documents\Python\vpz\scikit-learn\build\cp310
Build type: native build
Project name: scikit-learn
Project version: 1.7.dev0
C compiler for the host machine: cl (msvc 19.43.34809 "Microsoft (R) C/C++ Optimizing Compiler Version 19.43.34809 for x64")
C linker for the host machine: link link 14.43.34809.0
C++ compiler for the host machine: cl (msvc 19.43.34809 "Microsoft (R) C/C++ Optimizing Compiler Version 19.43.34809 for x64")
C++ linker for the host machine: link link 14.43.34809.0
Cython compiler for the host machine: cython (cython 3.0.12)
Host machine cpu family: x86_64
Host machine cpu: x86_64
Compiler for C supports arguments -Wno-unused-but-set-variable: NO (cached)
Compiler for C supports arguments -Wno-unused-function: NO (cached)
Compiler for C supports arguments -Wno-conversion: NO (cached)
Compiler for C supports arguments -Wno-misleading-indentation: NO (cached)
Library m found: NO
Program sklearn/_build_utils/tempita.py found: YES (C:\Users\vitor.pohlenz\Documents\Python\vpz\scikit-learn\envsklearn\Scripts\python.exe C:\Users\vitor.pohlenz\Documents\Python\vpz\scikit-learn\sklearn/_build_utils/tempita.py)
Program python found: YES (C:\Users\vitor.pohlenz\Documents\Python\vpz\scikit-learn\envsklearn\Scripts\python.exe)
Dependency OpenMP found: YES 2.0 (cached)
Build targets in project: 111
scikit-learn 1.7.dev0
User defined options
Native files: C:\Users\vitor.pohlenz\Documents\Python\vpz\scikit-learn\build\cp310\meson-python-native-file.ini
b_ndebug : if-release
b_vscrt : md
buildtype : release
Found ninja.EXE-1.11.1.git.kitware.jobserver-1 at C:\Users\vitor.pohlenz\Documents\Python\vpz\scikit-learn\envsklearn\Scripts\ninja.EXE
+ meson compile
[1/164] Generating sklearn/write_built_with_meson_file with a custom command
[2/164] Compiling Cython source C:/Users/vitor.pohlenz/Documents/Python/vpz/scikit-learn/sklearn/metrics/cluster/_expected_mutual_info_fast.pyx
[3/164] Compiling Cython source sklearn/utils/_seq_dataset.pyx
[4/164] Compiling Cython source C:/Users/vitor.pohlenz/Documents/Python/vpz/scikit-learn/sklearn/cluster/_dbscan_inner.pyx
[5/164] Compiling Cython source C:/Users/vitor.pohlenz/Documents/Python/vpz/scikit-learn/sklearn/utils/_vector_sentinel.pyx
[6/164] Compiling Cython source C:/Users/vitor.pohlenz/Documents/Python/vpz/scikit-learn/sklearn/metrics/_pairwise_fast.pyx
[7/164] Compiling Cython source sklearn/utils/_weight_vector.pyx
[8/164] Compiling Cython source C:/Users/vitor.pohlenz/Documents/Python/vpz/scikit-learn/sklearn/utils/_isfinite.pyx
[9/164] Compiling Cython source C:/Users/vitor.pohlenz/Documents/Python/vpz/scikit-learn/sklearn/cluster/_hierarchical_fast.pyx
[10/164] Compiling Cython source sklearn/metrics/_pairwise_distances_reduction/_base.pyx
[11/164] Compiling Cython source C:/Users/vitor.pohlenz/Documents/Python/vpz/scikit-learn/sklearn/utils/_fast_dict.pyx
[12/164] Compiling Cython source C:/Users/vitor.pohlenz/Documents/Python/vpz/scikit-learn/sklearn/utils/arrayfuncs.pyx
[13/164] Compiling Cython source C:/Users/vitor.pohlenz/Documents/Python/vpz/scikit-learn/sklearn/cluster/_k_means_common.pyx
[14/164] Compiling Cython source sklearn/metrics/_pairwise_distances_reduction/_datasets_pair.pyx
[15/164] Compiling Cython source sklearn/metrics/_pairwise_distances_reduction/_argkmin_classmode.pyx
[16/164] Compiling Cython source sklearn/metrics/_pairwise_distances_reduction/_argkmin.pyx
[17/164] Compiling Cython source sklearn/metrics/_pairwise_distances_reduction/_radius_neighbors_classmode.pyx
[18/164] Compiling Cython source C:/Users/vitor.pohlenz/Documents/Python/vpz/scikit-learn/sklearn/utils/_cython_blas.pyx
[19/164] Compiling Cython source sklearn/metrics/_pairwise_distances_reduction/_middle_term_computer.pyx
[20/164] Compiling C object sklearn/utils/_weight_vector.cp310-win_amd64.pyd.p/meson-generated_sklearn_utils__weight_vector.pyx.c.obj
[21/164] Compiling Cython source C:/Users/vitor.pohlenz/Documents/Python/vpz/scikit-learn/sklearn/utils/sparsefuncs_fast.pyx
[22/164] Compiling Cython source sklearn/metrics/_pairwise_distances_reduction/_radius_neighbors.pyx
[23/164] Compiling Cython source sklearn/metrics/_dist_metrics.pyx
warning: sklearn\metrics\_dist_metrics.pyx:855:44: Assigning to 'float64_t *' from 'const float64_t *' discards const qualifier
warning: sklearn\metrics\_dist_metrics.pyx:918:40: Assigning to 'float64_t *' from 'const float64_t *' discards const qualifier
warning: sklearn\metrics\_dist_metrics.pyx:3435:44: Assigning to 'float32_t *' from 'const float32_t *' discards const qualifier
warning: sklearn\metrics\_dist_metrics.pyx:3498:40: Assigning to 'float32_t *' from 'const float32_t *' discards const qualifier
[24/164] Compiling C object sklearn/utils/_seq_dataset.cp310-win_amd64.pyd.p/meson-generated_sklearn_utils__seq_dataset.pyx.c.obj
[25/164] Compiling C++ object sklearn/metrics/_pairwise_distances_reduction/_radius_neighbors_classmode.cp310-win_amd64.pyd.p/meson-generated_sklearn_metrics__pairwise_distances_reduction__radius_neighbors_classmode.pyx.cpp.obj
FAILED: sklearn/metrics/_pairwise_distances_reduction/_radius_neighbors_classmode.cp310-win_amd64.pyd.p/meson-generated_sklearn_metrics__pairwise_distances_reduction__radius_neighbors_classmode.pyx.cpp.obj
"cl" "-Isklearn\metrics\_pairwise_distances_reduction\_radius_neighbors_classmode.cp310-win_amd64.pyd.p" "-Isklearn\metrics\_pairwise_distances_reduction" "-I..\..\sklearn\metrics\_pairwise_distances_reduction" "-I..\..\envsklearn\lib\site-packages\numpy\_core\include" "-IC:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\Include" "-DNDEBUG" "/MD" "/nologo" "/showIncludes" "/utf-8" "/Zc:__cplusplus" "/W2" "/EHsc" "/std:c++14" "/permissive-" "/O2" "/Gw" "-DMS_WIN64=" "/openmp" "-DNPY_NO_DEPRECATED_API=NPY_1_9_API_VERSION" "/Fdsklearn\metrics\_pairwise_distances_reduction\_radius_neighbors_classmode.cp310-win_amd64.pyd.p\meson-generated_sklearn_metrics__pairwise_distances_reduction__radius_neighbors_classmode.pyx.cpp.pdb" /Fosklearn/metrics/_pairwise_distances_reduction/_radius_neighbors_classmode.cp310-win_amd64.pyd.p/meson-generated_sklearn_metrics__pairwise_distances_reduction__radius_neighbors_classmode.pyx.cpp.obj "/c" sklearn/metrics/_pairwise_distances_reduction/_radius_neighbors_classmode.cp310-win_amd64.pyd.p/sklearn/metrics/_pairwise_distances_reduction/_radius_neighbors_classmode.pyx.cpp
sklearn/metrics/_pairwise_distances_reduction/_radius_neighbors_classmode.cp310-win_amd64.pyd.p/sklearn/metrics/_pairwise_distances_reduction/_radius_neighbors_classmode.pyx.cpp(32619): warning C4551: function call missing argument list
C:\Users\vitor.pohlenz\Documents\Python\vpz\scikit-learn\build\cp310\sklearn\metrics\_pairwise_distances_reduction\_radius_neighbors_classmode.cp310-win_amd64.pyd.p\sklearn\metrics\_pairwise_distances_reduction\_radius_neighbors_classmode.pyx.cpp : fatal error C1083: Cannot open compiler generated file: '': Invalid argument
[26/164] Linking target sklearn/utils/_weight_vector.cp310-win_amd64.pyd
Creating library sklearn\utils\_weight_vector.cp310-win_amd64.lib and object sklearn\utils\_weight_vector.cp310-win_amd64.exp
[27/164] Linking target sklearn/utils/_seq_dataset.cp310-win_amd64.pyd
Creating library sklearn\utils\_seq_dataset.cp310-win_amd64.lib and object sklearn\utils\_seq_dataset.cp310-win_amd64.exp
[28/164] Compiling C++ object sklearn/utils/_vector_sentinel.cp310-win_amd64.pyd.p/meson-generated_sklearn_utils__vector_sentinel.pyx.cpp.obj
sklearn/utils/_vector_sentinel.cp310-win_amd64.pyd.p/sklearn/utils/_vector_sentinel.pyx.cpp(15416): warning C4551: function call missing argument list
[29/164] Compiling C object sklearn/metrics/_pairwise_fast.cp310-win_amd64.pyd.p/meson-generated_sklearn_metrics__pairwise_fast.pyx.c.obj
[30/164] Compiling C++ object sklearn/cluster/_dbscan_inner.cp310-win_amd64.pyd.p/meson-generated_sklearn_cluster__dbscan_inner.pyx.cpp.obj
sklearn/cluster/_dbscan_inner.cp310-win_amd64.pyd.p/sklearn/cluster/_dbscan_inner.pyx.cpp(23107): warning C4551: function call missing argument list
[31/164] Compiling C object sklearn/metrics/cluster/_expected_mutual_info_fast.cp310-win_amd64.pyd.p/meson-generated_sklearn_metrics_cluster__expected_mutual_info_fast.pyx.c.obj
sklearn/metrics/cluster/_expected_mutual_info_fast.cp310-win_amd64.pyd.p/sklearn/metrics/cluster/_expected_mutual_info_fast.pyx.c(18720): warning C4244: 'function': conversion from 'Py_ssize_t' to 'double', possible loss of data
sklearn/metrics/cluster/_expected_mutual_info_fast.cp310-win_amd64.pyd.p/sklearn/metrics/cluster/_expected_mutual_info_fast.pyx.c(18720): warning C4244: 'function': conversion from 'Py_ssize_t' to 'double', possible loss of data
sklearn/metrics/cluster/_expected_mutual_info_fast.cp310-win_amd64.pyd.p/sklearn/metrics/cluster/_expected_mutual_info_fast.pyx.c(18720): warning C4244: 'function': conversion from 'Py_ssize_t' to 'double', possible loss of data
[32/164] Compiling C object sklearn/utils/arrayfuncs.cp310-win_amd64.pyd.p/meson-generated_sklearn_utils_arrayfuncs.pyx.c.obj
[33/164] Compiling C++ object sklearn/utils/_fast_dict.cp310-win_amd64.pyd.p/meson-generated_sklearn_utils__fast_dict.pyx.cpp.obj
sklearn/utils/_fast_dict.cp310-win_amd64.pyd.p/sklearn/utils/_fast_dict.pyx.cpp(26950): warning C4551: function call missing argument list
[34/164] Compiling C object sklearn/utils/_isfinite.cp310-win_amd64.pyd.p/meson-generated_sklearn_utils__isfinite.pyx.c.obj
[35/164] Compiling C++ object sklearn/metrics/_pairwise_distances_reduction/_argkmin_classmode.cp310-win_amd64.pyd.p/meson-generated_sklearn_metrics__pairwise_distances_reduction__argkmin_classmode.pyx.cpp.obj
cl : Command line warning D9002 : ignoring unknown option '-fno-sized-deallocation'
sklearn/metrics/_pairwise_distances_reduction/_argkmin_classmode.cp310-win_amd64.pyd.p/sklearn/metrics/_pairwise_distances_reduction/_argkmin_classmode.pyx.cpp(29323): warning C4551: function call missing argument list
[36/164] Compiling C++ object sklearn/metrics/_pairwise_distances_reduction/_base.cp310-win_amd64.pyd.p/meson-generated_sklearn_metrics__pairwise_distances_reduction__base.pyx.cpp.obj
sklearn/metrics/_pairwise_distances_reduction/_base.cp310-win_amd64.pyd.p/sklearn/metrics/_pairwise_distances_reduction/_base.pyx.cpp(32606): warning C4551: function call missing argument list
[37/164] Compiling C++ object sklearn/metrics/_pairwise_distances_reduction/_argkmin.cp310-win_amd64.pyd.p/meson-generated_sklearn_metrics__pairwise_distances_reduction__argkmin.pyx.cpp.obj
sklearn/metrics/_pairwise_distances_reduction/_argkmin.cp310-win_amd64.pyd.p/sklearn/metrics/_pairwise_distances_reduction/_argkmin.pyx.cpp(35870): warning C4551: function call missing argument list
[38/164] Compiling C++ object sklearn/metrics/_pairwise_distances_reduction/_middle_term_computer.cp310-win_amd64.pyd.p/meson-generated_sklearn_metrics__pairwise_distances_reduction__middle_term_computer.pyx.cpp.obj
sklearn/metrics/_pairwise_distances_reduction/_middle_term_computer.cp310-win_amd64.pyd.p/sklearn/metrics/_pairwise_distances_reduction/_middle_term_computer.pyx.cpp(42045): warning C4551: function call missing argument list
[39/164] Compiling C++ object sklearn/cluster/_hierarchical_fast.cp310-win_amd64.pyd.p/meson-generated_sklearn_cluster__hierarchical_fast.pyx.cpp.obj
sklearn/cluster/_hierarchical_fast.cp310-win_amd64.pyd.p/sklearn/cluster/_hierarchical_fast.pyx.cpp(23677): warning C4244: '=': conversion from '__pyx_t_7sklearn_5utils_9_typedefs_intp_t' to '__pyx_t_7sklearn_5utils_9_typedefs_float64_t', possible loss of data
sklearn/cluster/_hierarchical_fast.cp310-win_amd64.pyd.p/sklearn/cluster/_hierarchical_fast.pyx.cpp(23688): warning C4244: '=': conversion from '__pyx_t_7sklearn_5utils_9_typedefs_intp_t' to '__pyx_t_7sklearn_5utils_9_typedefs_float64_t', possible loss of data
sklearn/cluster/_hierarchical_fast.cp310-win_amd64.pyd.p/sklearn/cluster/_hierarchical_fast.pyx.cpp(23712): warning C4244: '=': conversion from '__pyx_t_7sklearn_5utils_9_typedefs_intp_t' to '__pyx_t_7sklearn_5utils_9_typedefs_float64_t', possible loss of data
sklearn/cluster/_hierarchical_fast.cp310-win_amd64.pyd.p/sklearn/cluster/_hierarchical_fast.pyx.cpp(24838): warning C4244: '=': conversion from '__pyx_t_7sklearn_5utils_9_typedefs_intp_t' to '__pyx_t_7sklearn_5utils_9_typedefs_float64_t', possible loss of data
sklearn/cluster/_hierarchical_fast.cp310-win_amd64.pyd.p/sklearn/cluster/_hierarchical_fast.pyx.cpp(24849): warning C4244: '=': conversion from '__pyx_t_7sklearn_5utils_9_typedefs_intp_t' to '__pyx_t_7sklearn_5utils_9_typedefs_float64_t', possible loss of data
sklearn/cluster/_hierarchical_fast.cp310-win_amd64.pyd.p/sklearn/cluster/_hierarchical_fast.pyx.cpp(33802): warning C4551: function call missing argument list
[40/164] Compiling C++ object sklearn/metrics/_pairwise_distances_reduction/_datasets_pair.cp310-win_amd64.pyd.p/meson-generated_sklearn_metrics__pairwise_distances_reduction__datasets_pair.pyx.cpp.obj
sklearn/metrics/_pairwise_distances_reduction/_datasets_pair.cp310-win_amd64.pyd.p/sklearn/metrics/_pairwise_distances_reduction/_datasets_pair.pyx.cpp(43425): warning C4551: function call missing argument list
[41/164] Compiling C++ object sklearn/metrics/_pairwise_distances_reduction/_radius_neighbors.cp310-win_amd64.pyd.p/meson-generated_sklearn_metrics__pairwise_distances_reduction__radius_neighbors.pyx.cpp.obj
sklearn/metrics/_pairwise_distances_reduction/_radius_neighbors.cp310-win_amd64.pyd.p/sklearn/metrics/_pairwise_distances_reduction/_radius_neighbors.pyx.cpp(38873): warning C4551: function call missing argument list
[42/164] Compiling C object sklearn/cluster/_k_means_common.cp310-win_amd64.pyd.p/meson-generated_sklearn_cluster__k_means_common.pyx.c.obj
[43/164] Compiling Cython source sklearn/_loss/_loss.pyx
[44/164] Compiling C object sklearn/utils/_cython_blas.cp310-win_amd64.pyd.p/meson-generated_sklearn_utils__cython_blas.pyx.c.obj
[45/164] Compiling C object sklearn/utils/sparsefuncs_fast.cp310-win_amd64.pyd.p/meson-generated_sklearn_utils_sparsefuncs_fast.pyx.c.obj
[46/164] Compiling C object sklearn/metrics/_dist_metrics.cp310-win_amd64.pyd.p/meson-generated_sklearn_metrics__dist_metrics.pyx.c.obj
sklearn/metrics/_dist_metrics.cp310-win_amd64.pyd.p/sklearn/metrics/_dist_metrics.pyx.c(29605): warning C4090: '=': different 'const' qualifiers
sklearn/metrics/_dist_metrics.cp310-win_amd64.pyd.p/sklearn/metrics/_dist_metrics.pyx.c(30390): warning C4090: '=': different 'const' qualifiers
sklearn/metrics/_dist_metrics.cp310-win_amd64.pyd.p/sklearn/metrics/_dist_metrics.pyx.c(49229): warning C4090: '=': different 'const' qualifiers
sklearn/metrics/_dist_metrics.cp310-win_amd64.pyd.p/sklearn/metrics/_dist_metrics.pyx.c(50014): warning C4090: '=': different 'const' qualifiers
ninja: build stopped: subcommand failed.
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: C:\Users\vitor.pohlenz\Documents\Python\vpz\scikit-learn\envsklearn\Scripts\ninja.EXE
error: subprocess-exited-with-error
× Preparing editable metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
full command: 'C:\Users\vitor.pohlenz\Documents\Python\vpz\scikit-learn\envsklearn\Scripts\python.exe' 'C:\Users\vitor.pohlenz\Documents\Python\vpz\scikit-learn\envsklearn\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py' prepare_metadata_for_build_editable 'C:\Users\VITOR~1.POH\AppData\Local\Temp\tmpp45ks547'
cwd: C:\Users\vitor.pohlenz\Documents\Python\vpz\scikit-learn
Preparing editable metadata (pyproject.toml) ... error
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
Using Conda
For conda env, the problem seems to be associated with undeclared identifiers and syntax errors
[22/136] Compiling C++ object sklearn/metrics/_pairwise_distances_reduction/_middle_term_computer.cp313t-win_amd64.pyd.p/meson-generated_sklearn_metrics__pairwise_distances_reduction__middle_term_computer.pyx.cpp.obj
FAILED: sklearn/metrics/_pairwise_distances_reduction/_middle_term_computer.cp313t-win_amd64.pyd.p/meson-generated_sklearn_metrics__pairwise_distances_reduction__middle_term_computer.pyx.cpp.obj
"cl" "-Isklearn\metrics\_pairwise_distances_reduction\_middle_term_computer.cp313t-win_amd64.pyd.p" "-Isklearn\metrics\_pairwise_distances_reduction" "-I..\..\sklearn\metrics\_pairwise_distances_reduction" "-I..\..\..\..\..\..\AppData\Local\anaconda3\envs\sklearn-env\Lib\site-packages\numpy\_core\include" "-IC:\Users\vitor.pohlenz\AppData\Local\anaconda3\envs\sklearn-env\Include" "-DNDEBUG" "/MD" "/nologo" "/showIncludes" "/utf-8" "/Zc:__cplusplus" "/W2" "/EHsc" "/std:c++14" "/permissive-" "/O2" "/Gw" "-DPy_GIL_DISABLED" "-DNPY_NO_DEPRECATED_API=NPY_1_9_API_VERSION" "/Fdsklearn\metrics\_pairwise_distances_reduction\_middle_term_computer.cp313t-win_amd64.pyd.p\meson-generated_sklearn_metrics__pairwise_distances_reduction__middle_term_computer.pyx.cpp.pdb" /Fosklearn/metrics/_pairwise_distances_reduction/_middle_term_computer.cp313t-win_amd64.pyd.p/meson-generated_sklearn_metrics__pairwise_distances_reduction__middle_term_computer.pyx.cpp.obj "/c" sklearn/metrics/_pairwise_distances_reduction/_middle_term_computer.cp313t-win_amd64.pyd.p/sklearn/metrics/_pairwise_distances_reduction/_middle_term_computer.pyx.cpp
[lots of lines similar to the line below ...]
sklearn/metrics/_pairwise_distances_reduction/_middle_term_computer.cp313t-win_amd64.pyd.p/sklearn/metrics/_pairwise_distances_reduction/_middle_term_computer.pyx.cpp(43156): error C2065: 'vc': undeclared identifier
ninja: build stopped: subcommand failed.
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: C:\Users\vitor.pohlenz\AppData\Local\anaconda3\envs\sklearn-env\Library\bin\ninja.EXE
error: subprocess-exited-with-error
× Preparing editable metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
full command: 'C:\Users\vitor.pohlenz\AppData\Local\anaconda3\envs\sklearn-env\python.exe' 'C:\Users\vitor.pohlenz\AppData\Local\anaconda3\envs\sklearn-env\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py' prepare_metadata_for_build_editable 'C:\Users\VITOR~1.POH\AppData\Local\Temp\tmp25gfqq4d'
cwd: C:\Users\vitor.pohlenz\Documents\Python\vpz\scikit-learn
Preparing editable metadata (pyproject.toml) ... error
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
Versions
## General and OS
- OS: Windows 11 Pro, Version 24H2, OS build 26100.3476
- System type: 64-bit operating system, x64-based processor
- Installed Visual Studio Build 2022, version 17.13.5
Neither using `pip` in the `venv`, nor using `conda env`, I could use the `import sklearn; sklearn.show_versions()`.
### venv using pip
- Python: 3.10.11
- pip 25.0.1
- Libs inside env:
- Cython==3.0.12
- meson==1.7.2
- meson-python==0.17.1
- ninja==1.11.1.4
- numpy==2.2.4
- packaging==24.2
- pyproject-metadata==0.9.1
- scipy==1.15.2
- tomli==2.2.1
### conda env
- Python 3.13.2
- conda 24.11.3
Libs inside conda env:
# Name Version Build Channel
bzip2 1.0.8 h2466b09_7 conda-forge
ca-certificates 2025.1.31 h56e8100_0 conda-forge
cython 3.0.12 pyh2c78169_100 conda-forge
intel-openmp 2024.2.1 h57928b3_1083 conda-forge
libblas 3.9.0 31_h641d27c_mkl conda-forge
libcblas 3.9.0 31_h5e41251_mkl conda-forge
libexpat 2.7.0 he0c23c2_0 conda-forge
libffi 3.4.6 h537db12_1 conda-forge
libhwloc 2.11.2 default_ha69328c_1001 conda-forge
libiconv 1.18 h135ad9c_1 conda-forge
liblapack 3.9.0 31_h1aa476e_mkl conda-forge
liblzma 5.8.1 h2466b09_0 conda-forge
libmpdec 4.0.0 h2466b09_0 conda-forge
libsqlite 3.49.1 h67fdade_2 conda-forge
libwinpthread 12.0.0.r4.gg4f2fc60ca h57928b3_9 conda-forge
libxml2 2.14.0 had0eb51_0 conda-forge
libzlib 1.3.1 h2466b09_2 conda-forge
meson 1.7.1 pyhd8ed1ab_0 conda-forge
meson-python 0.17.1 pyh70fd9c4_1 conda-forge
mkl 2024.2.2 h66d3029_15 conda-forge
ninja 1.12.1 hc790b64_0 conda-forge
numpy 2.2.4 py313hd96daed_0 conda-forge
openssl 3.4.1 ha4e3fda_0 conda-forge
packaging 24.2 pyhd8ed1ab_2 conda-forge
pip 25.0.1 pyh145f28c_0 conda-forge
pyproject-metadata 0.9.1 pyhd8ed1ab_0 conda-forge
python 3.13.2 hd7c436d_1_cp313t conda-forge
python_abi 3.13 5_cp313t conda-forge
scipy 1.15.2 py313h8a36d7e_0 conda-forge
setuptools 78.1.0 pyhff2d567_0 conda-forge
tbb 2021.13.0 h62715c5_1 conda-forge
tk 8.6.13 h5226925_1 conda-forge
tomli 2.2.1 pyhd8ed1ab_1 conda-forge
tzdata 2025b h78e105d_0 conda-forge
ucrt 10.0.22621.0 h57928b3_1 conda-forge
vc 14.3 h2b53caa_26 conda-forge
vc14_runtime 14.42.34438 hfd919c2_26 conda-forge
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 19 (19 by maintainers)
For later reference, I have found reports that seem to indicate that cl.exe doesn’t support long paths even when allowed in the registry, not sure how relevant they still are … https://stackoverflow.com/q/69082543, https://old.reddit.com/r/cpp/comments/zdndig/msvc_compiler_cant_find_source_file_in_path_that/.
There is also this comment that seems to say that you need to enable long paths with a manifest https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107974#c3. The manifest thingy is also mentioned in the documentation: https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry#application-manifest-updates-to-declare-long-path-capability.
I actually managed to reproduce on my Windows VM and opened https://github.com/conda-forge/python-feedstock/issues/792 about the free-threading Python by default on Windows when install scipy. Thanks @vitorpohlenz for the report!
@vitorpohlenz another thing you could do if you want: it is very surprising than you got a free-threading python when installing your conda environment i.e. https://github.com/scikit-learn/scikit-learn/issues/31149#issuecomment-2783921334.
If you manage to reproduce with the last version of
condathis may be worth reporting it somewhere (maybe conda-forge or conda, not sure?) …Sorry I didn’t see your newer messages before posting my comment about your conda env which you can forget about.
Great if you managed to fix your issue! The error message is not helpful at all, not sure if there is a way to improve it under our control …
That would be great if you can open a PR indeed! No need to create a separate issue, you can link to this issue in your PR.
On the conda env, for some unknown reason you are using free-threaded in your conda env, note cp313t (rather than cp313) in your build for python:
Can you try in a new conda env where you explicitly mention
python=3.12[^1][^1]: this seems like the easiest work-around off the top of my head to avoid getting a free-threaded Python. I seem to remember issues at one point where
python-freethreadingwas picked rather than vanilla CPython but I thought they had been fixed. Searching for them I was only able to find https://github.com/conda-forge/python-feedstock/issues/738 or https://github.com/conda-incubator/setup-miniconda/issues/395.@lesteve, your “wild guess” was quite accurate! Even though I do not know why, because my path length limit is disabled, I can build from source in the
venvif I clone the project and create thevenvin a folder with short path, like directly in"C:\scikit-learn"or in"C:\Users\vitor.pohlenz\scikit-learn".The build runs smoothly in those folders, an I can run the command
python -c "import sklearn; sklearn.show_versions()"getting the following response:sklearn.show_versions() in C:/ venv
But in conda env I still get an error when building:
Conda env error in C:/
Using pip 25.0.1 from C:\Users\vitor.pohlenz\AppData\Local\anaconda3\envs\sklearn-env-new\Lib\site-packages\pip (python 3.13) Obtaining file:///C:/scikit-learn Running command Checking if build backend supports build_editable Checking if build backend supports build_editable … done Running command Preparing editable metadata (pyproject.toml)
scikit-learn 1.7.dev0
Found ninja.EXE-1.12.1 at C:\Users\vitor.pohlenz\AppData\Local\anaconda3\envs\sklearn-env-new\Library\bin\ninja.EXE
× Preparing editable metadata (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip. full command: ‘C:\Users\vitor.pohlenz\AppData\Local\anaconda3\envs\sklearn-env-new\python.exe’ ‘C:\Users\vitor.pohlenz\AppData\Local\anaconda3\envs\sklearn-env-new\Lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py’ prepare_metadata_for_build_editable ‘C:\Users\VITOR~1.POH\AppData\Local\Temp\tmprtek77ht’ cwd: C:\scikit-learn Preparing editable metadata (pyproject.toml) … error error: metadata-generation-failed
× Encountered error while generating package metadata. ╰─> See above for output.
note: This is an issue with the package mentioned above, not pip. hint: See above for details.
From my side, I do not need to use conda env, I can work completely fine just with the
venv, so this issue could be closed. But if you want me to investigate it a bit more, I can do, but I’m running out of ideas…Can you include the output of the build command (pip install in editable mode) when you build within the conda forge env? The error you get when importing sklearn implies that scikit-learn wasn’t built in successfully.
Just a wild guess, maybe this is due to windows path length limitation, see https://scikit-learn.org/dev/install.html#error-caused-by-file-path-length-limit-on-windows for a possible work-around.
Here is the part of your detailed log where the error seems to happen, notice the
fatal error C1083: Cannot open compiler generated file: '': Invalid argument. This does not make much sense since there is a.cpppath as the last argument, but the.cpppath length seems to be close to the path length limit so maybe this is related:The ninja error message is not very explicit but it’s definitely different from https://github.com/scikit-learn/scikit-learn/issues/31123 which is about running mkdir in a non existing folder. Your issue does not seem related to non-existing folders.
Thanks for the report.
For the conda-forge method, you still need to build scikit-learn from source. That is:
After you do:
you need to:
and then only check that you can import “sklearn” with: