poetry: Poetry does not install from wheels on macOS Big Sur

I’ve fixed this locally for the time being by setting an alias for Poetry… alias poetry='env SYSTEM_VERSION_COMPAT=1 poetry'

Setting SYSTEM_VERSION_COMPAT=1 instructs macOS to return 10.16 for any system version lookups, which fixes the issues with older versions of https://github.com/pypa/packaging.

There should be a new release of packaging somewhat soon, and it will fix a bug that causes correctly named universal2 wheels to not be loaded on Apple Silicon, but it looks like it will also drop Python 2.7, 3.4, and 3.5 support, so it might be something to watch for, as I think Poetry partially supports 2.7 and fully supports 3.5, at least that was my impression. Not that I’ve tried that 😃. Going up to 20.8 would at least fix the issue for all macOS 11 users on Intel that can’t get things like NumPy wheels with Poetry currently! See https://github.com/pypa/packaging/issues/385 and https://github.com/pypa/packaging/pull/376 .

For a sketchy workarount in the meantime

pip=$(which pip)
pushd $PWD
cd "$(dirname $(which poetry))/../lib/poetry/_vendor/"
cd "py$( python -V | grep -o '[23]\.[0-9]\+')"
cd poetry/core/_vendor
sed -i '' 's/packaging==20.4/packaging==20.5/' vendor.txt
$pip install -t $PWD --upgrade -r vendor.txt
unset pip
popd

It’s the same issue if you use pip 20.2 (which tends to be default some places, like the CLT venv), it’s due to old versions of packaging. Pip updated the vendored packaging in 20.3, and poetry needs to, too.

Same issue on Intel silicon (16" 2019 MBP). Numpy and others don’t download MacOS wheels and the attempted compile fails. My workaround is installing the failed packages at the specified revision with pip, after which running Poetry install/update succeeds. Make sure you also update pip if you see a message to that effect. I’ve had problems if I don’t, although not sure if related.

The old version of packaging also means poetry can’t install NumPy on Big Sur (Intel) with the last released version (the build fails since it’s not getting a wheel, and there’s some setup required to add a BLAS library since Apple Accelerate is buggy).

For Big Sur Apple Silicon, you’ll want https://github.com/pypa/packaging/pull/380 which has just landed and is not yet in a released version of packaging (or pip, obviously), as otherwise it still will not be able to pick up normally-named binary wheels.

This solution worked better than most but fails with xgboost even with the new poetry core. I am sticking with converting to requirements.txt for now it seems.

@sinoroc Excellent point! I forgot to mention: I evaluated PR #3530 and it did not appear to resolve the issue for me. I posted details here: https://github.com/python-poetry/poetry/pull/3530#issuecomment-756393563.

For a sketchy workarount in the meantime

Feels weird but works (changed mine to install packaging==20.7 since that’s latest). Also calling out that virtualenv still has wheels for pip 20.2.2 packaged with it, I needed to run poetry run pip install -U pip so that pip would install macOS wheels without erroring.

But with those two changes, seems like things are working… so one upstream dependency and then figuring out what all’s in Poetry that needs changing.