PyPardiso: [Windows] The Pardiso solver failed with error code -3

I was also bitten by this issue, under Linux. The issue appeared on an install that was created with:

conda install -c conda-forge -c cmutel brightway25
# and then, I forgot to keep the conda-forge channel as the main source by doing:
conda update conda

The snippet above (the second conda execution in particular) pulled intel-openmp dependency and trashed my original environment.

TIP: keep conda-forge as the main source to get packages

a) Don’t forget to use it as the first channel in your conda commands (update, install, etc.) or b) [nuke option] update your $HOME/.condarc to look like follows:

channel_priority: strict
channels:
  - conda-forge
  - defaults

ensure that all the dependencies come from the conda-forge channel unless they exist only on defaults

ref here from conda-forge

and never forget you did this !

Indeed

After some more tests, I’m pretty certain that the issue is caused because mkl version 2021.4.0 is not compatible with intel-openmp 2022.0.0. The mkl 2021.4.0 package from both the conda defaults channel and pypi have pinned their dependency on intel-openmp to version 2021.*.

• conda defaults channel:

conda search -c defaults --subdir=win-64 mkl=2021.4 --info
Loading channels: done
mkl 2021.4.0 haa95532_640
-------------------------
file name   : mkl-2021.4.0-haa95532_640.conda
name        : mkl
version     : 2021.4.0
build       : haa95532_640
build number: 640
size        : 114.9 MB
license     : LicenseRef-ProprietaryIntel
subdir      : win-64
url         : https://repo.anaconda.com/pkgs/main/win-64/mkl-2021.4.0-haa95532_640.conda
md5         : f7802e70a203f51ac4898309d4ed4115
timestamp   : 2021-10-18 08:53:01 UTC
dependencies: 
  - intel-openmp 2021.*

• pypi:

curl -s https://pypi.org/pypi/mkl/2021.4.0/json | jq .info.requires_dist
[
  "intel-openmp (==2021.*)",
  "tbb (==2021.*)"
]

However on conda-forge this pinning of intel-openmp to 2021.* is missing for mkl 2021.4.0:

conda search --override-channels -c conda-forge --subdir=win-64 mkl=2021.4 --info
Loading channels: done
mkl 2021.4.0 h0e2418a_729
-------------------------
file name   : mkl-2021.4.0-h0e2418a_729.tar.bz2
name        : mkl
version     : 2021.4.0
build       : h0e2418a_729
build number: 729
size        : 181.7 MB
license     : LicenseRef-ProprietaryIntel
subdir      : win-64
url         : https://conda.anaconda.org/conda-forge/win-64/mkl-2021.4.0-h0e2418a_729.tar.bz2
md5         : 42fcb45077a716cb8d967117b8b88f28
timestamp   : 2021-10-13 14:39:23 UTC
dependencies: 
  - intel-openmp
  - tbb 2021.*

This leads to the current situation where conda install -c conda-forge pypardiso installs mkl 2021.4.0 alongside intel-openmp 2022.0.0 which are apparently not fully compatible. And while mkl 2022.0.0 is already available on conda-forge, the latest mkl-build of libblas has an explicit dependency on the older mkl 2021.4.0:

conda search -c conda-forge --subdir='win-64' libblas=3.9.0=12_win64_mkl --info
Loading channels: done
libblas 3.9.0 12_win64_mkl
--------------------------
file name   : libblas-3.9.0-12_win64_mkl.tar.bz2
name        : libblas
version     : 3.9.0
build       : 12_win64_mkl
build number: 12
size        : 4.5 MB
license     : BSD-3-Clause
subdir      : win-64
url         : https://conda.anaconda.org/conda-forge/win-64/libblas-3.9.0-12_win64_mkl.tar.bz2
md5         : 96b3e24fd626a2a964c81045da31c7b1
timestamp   : 2021-10-15 08:34:53 UTC
constraints : 
  - liblapack 3.9.0 12_win64_mkl
  - liblapacke 3.9.0 12_win64_mkl
  - libcblas 3.9.0 12_win64_mkl
  - blas * mkl
dependencies: 
  - mkl 2021.4.0 h0e2418a_729

thanks Adrian! with a fresh installation it does indeed work. I’ll follow your advice

Thanks for this summary, @haasad. Just to clarify: there should not be a problem with mkl 2021.4 and intel-omp 2021.4 on Windows, right?

That’s correct, according to my tests the problem only occurs with the combination of mkl 2021.4 and intel-openmp 2022.0.

Summary of the current situation:

• mkl 2021.4 is not fully compatible with intel-openmp 2022.0, this only affects Windows
• If you experience problems with pypardiso installed from conda-forge, downgrading intel-openmp is a temporary solution to fix it:

conda install -c conda-forge intel-openmp=2021.4

• Because intel-openmp is only a Windows dependency (mkl conda-forge packages for linux and osx both use llvm-openmp) and pypardiso is a noarch package, there is no straightforward way to fix this with pypardiso’s own dependency pins

Possible solutions:

1. new release of libblas with a mkl 2022.0 build, as the 2022.0 versions of mkl and intel-openmp appear to be compatible -> PR: https://github.com/conda-forge/blas-feedstock/pull/81
2. correct pins for intel-openmp in the conda-forge mkl package -> issue: https://github.com/conda-forge/intel_repack-feedstock/issues/33
3. switch pypardiso from a noarch build to separate builds for each OS with a pinning for compatible mkl and intel-openmp on windows -> this is a lot of effort with the potential to break other things on linux and osx, last resort
4. remove the requirement for a mkl-build of libblas and pin mkl to >=2022.0 -> this is probably the way to go short-term if there is no progress on 1. and 2., but it could lead to changes in performance on all operating systems for non-pardiso linalg workloads, because of the diferent libblas-build