conda-forge.github.io: DISCUSS: Raise MacOS minimum target from 10.9 to 10.13?
current todo list
- make an announcement (https://github.com/conda-forge/conda-forge.github.io/pull/1993)
- decide how we want to enforce the __osx constraint
- ~going to use recipe_append.yaml generated by
conda-forge-ci-setup(https://github.com/conda-forge/conda-forge-ci-setup-feedstock/issues/260)~ - The append doesn’t work for recipes with outputs, so we’ll have to use a run export on the compilers for osx.
- ~going to use recipe_append.yaml generated by
- Do https://github.com/conda-forge/conda-forge.github.io/issues/2102
- Decide if we will allow folks to set a min version less the global minimum.
- ~If yes, do nothing.~ stdlib jinja mechanism will allow this
- ~If no, then we need a minimigrator to adjust / remove any custom minimum pins below 10.13~
- clean up recipes with outdated pins in CBC (that don’t explicitly want/need it)
- figure out if there are any finicky builds on osx that need adjusting? I recall some but maybe that is out of date by now.
- figure out remaining(?) places in the infrastructure that involve assumptions about the sysroot and fix them up
- patch old builds with 10.9 __osx constraint?
- bump
c_stdlib_versionto 10.13 forosx-64in global pinnings
issue text from start
When azure deprecated the macOS-10.15 images, it turned out macOS-11 still supports targets all the way back to our baseline 10.9, and so it was decided to separate the discussion of the image upgrade from the default MACOSX_DEPLOYMENT_TARGET.
However our baseline target 10.9 is now EOL since almost 6(!) years, so I guess that discussion should be had at some point. I don’t think there’s a really urgent need (what with shipping our own libcxx, plus the _LIBCPP_DISABLE_AVAILABILITY mechanism[^1]), but this topic recently came up in a numpy discussion, so I though I’d open an issue.
Originally, I thought our hand would be forced once macOS-11 images are deprecated, but it turns out that even in the macOS-12 images, there’s still an SDK with Xcode 13.1, which in turn still supports targets back to 10.9. Only once we’re forced to use Xcode 14+ would we have to bump the minimum target to 10.13.
[^1]: though there are quite a few feedstocks that just bump the MACOSX_DEPLOYMENT_TARGET because it happens to unbreak CI - not least because the compiler errors point in this direction (“X is unavailable: introduced in macOS Y.Z”) - but often not adding __osx >={{ MACOSX_DEPLOYMENT_TARGET|default("10.9") }} as a dependence; such packages might already be broken on old OSX anyway…
July 2023 update
Out of curiosity, I wanted to check when the last bump of the MACOSX_DEPLOYMENT_TARGET happened, and, as it turns out, it’s been at 10.9 since the initial commit of https://github.com/conda-forge/conda-forge-pinning-feedstock 🤯
With a bit of digging (and luck), I found the bump from 10.7 to 10.9 though: https://github.com/conda-forge/toolchain-feedstock/commit/7a470c5ec71ad250bbfe6565016e793f3cc8f339 - 7 years ago. At the time, 10.9 was just before its EOL. If we applied the same standard today we should jump directly to 11.0.
Given that most users these days are on much newer versions, and want to use newer features (relevant for our packaging, like support for metal or the new LAPACK implementation), I think it might make sense to stop dragging our feet so much on this. We’re (slowly but steadily) moving with the time on linux and windows as well, why should osx need to fall so far behind?
To substantiate this a bit more, I wanted to look at the usage numbers for different MacOS versions. The broadest measure is “everyone who uses a browser”, but there are apparently no good usage numbers, because Apple keeps misreporting its OS version in HTTP headers, for some complicated reasons. I did find however that 92% are on 10.15+, which is the version every version after that pretends to be. Notably, all (distinguishable) versions that’re EOL have max 1-1.5% usage numbers (<7% cumulated) – and again, this is all MacOS users, not just those of conda-forge.
About this issue
- Original URL
- State: open
- Created 2 years ago
- Reactions: 4
- Comments: 73 (64 by maintainers)
libcxx 17 has recently dropped support for macOS<10.13 when building the shared libcxx library (I noted this in https://github.com/conda-forge/libcxx-feedstock/issues/110 when the RFC opened).
As far as I can tell, this is more fundamental than the
availabilityworkarounds we do (for building projects on top of libcxx), as it will really become impossible to build (shared) libcxx itself for targets <10.13.Like for other fundamental toolchain dependencies (like upstream requiring vs2019 vs. vs2017 recently), I don’t think it’s feasible to try to keep patching back in support for older macos versions, as there will be zero coverage upstream and therefore things will bitrot fast.
Seeing that LLVM is our default toolchain on osx (and assuming I’m not overlooking something), I think we’ll have to increase the minimum version to 10.13 at the latest when we switch our default compilers on osx to clang 17 (earliest that could happen would be late 2023; of course we could drag it out for quite a while longer by staying on clang 16 for another year or so afterwards).
PS. It’s worth noting that libcxx had been waiting on chrome to bump their required minimum version to 10.13, which was apparently the last major “holdout” (from the POV of LLVM). So I think end of 2023 would actually a very reasonable timeline to bump to 10.13; at that point 10.12 will have been EOL for 4 years.
FYI: We are not restricted by the xcode version and our builds are completely independent of the macos build image. See https://github.com/conda-forge/conda-forge-ci-setup-feedstock/blob/main/recipe/download_osx_sdk.sh
Newest abseil requires MacOS 10.13 (and fails with 10.9). I’d like to migrate this together with the new libgrpc & libprotobuf, but seeing how widely abseil is used, that’s effectively a conda-forge-wide decision (at least as far as C++ deps are concerned).
I saw that @chenghlee had put up the idea of dropping osx-64 support entirely (in the meeting notes of the last core call, though apparently it wasn’t discussed yet). Given all of the above datapoints (libcxx, clang, abseil, protobuf, C11’s
aligned_alloc, how long ago the EOL was, the comparison to the last bump, etc.), is there any serious opposition to just bumping to 10.13?Please don’t. Ask them to give all of the information asked in https://github.com/conda-forge/google-re2-feedstock/issues/6 and redirect there.
An example feedstock with selectors on compilers and things like
compiler('m2w64_c')would be good to check.AP would likely provide arm64 agents like GHA is doing now which we can use to cross compile for x64 and run tests with rosetta 2 emulation.
I was thinking all migrations.
OK, this issue was closed now, meaning we should have a green light to move ahead with the current implementation, which has already been released in conda-build 3.28. 🥳
The main question is now how to introduce
{{ stdlib("c") }}across a very large amount of recipes.PS. Cryptography 42.0 will now (by way of the rust standard library) require 10.12+.
Noticed today that even with a 10.13 deployment target, “10.9” is still encoded in some of the build infra (I presume it’s irrelevant; just mentioning it for completeness):
docs here: https://docs.conda.io/projects/conda-build/en/stable/resources/variants.html#appending-to-recipes
See here: https://github.com/conda-forge/conda-forge-ci-setup-feedstock/issues/260
Thanks for the kind words! Bit too high a praise for using
path:<glob>, but you inspired me to try if multiline regexes work with the new search, and they do! 🤩So here’s a search for only those feedstocks pinning
MACOSX_DEPLOYMENT_TARGETto something <10.13.OK! So it looks like 10.13 is the minimum. I think we had settled on 10.12 or so a few years ago so that seems fine.
My understanding of what we need to do to make this happen.
What else did I miss?
I only meant to say as old as possible. If 10.13 is that number, great!
I didn’t realize you had numbers for all internet users above. Sorry about that.
If we were to drop osx-64 (not my proposal), then obviously this discussion becomes moot, hence the pretty direct relation. In any case, let’s shelve that one, as @chenghlee is clearly not planning that in the short term.
Obviously numbers from our package downloads would be interesting. I already provided some numbers above. 95% of all internet users are on >=10.13, which includes an even longer tail than conda-forge. The fact that chrome dropped support for <10.13 should be a good indicator that even google with all its resources does not see this as worth supporting anymore.
Apple aggressively forces people to upgrade, and we’re talking about moving from something that’s been EOL almost 7 years to something that’s “only” been EOL 3 years. The last upgrade in conda-forge was 7 years ago[^1], which is also staggering.
[^1]: to a version that wasn’t EOL at the time; the equivalent of that today would be 11.0
There’s ample reasons in this thread already. We’re on a 10 year old SDK that is not just unsupported but now broken on many projects. We’ve managed to extend it way past its shelf life with hooking up our own C++ stdlib, but we’ve reached the end of the road.
Continuing to support <10.13 means we never upgrade abseil anymore, never upgrade LLVM anymore, etc. I have trouble finding neutral words for how bizarre I find this desire to support museum OS versions over current users and packages.
I’d push back on dropping
osx-64. It seems premature, but I don’t know why it is getting brought up here as it is tangential to the topic of macOS 10.9 support, which is starting to get pretty oldAgree with Matt we should collect data on OS version usage and see what we find
This is exactly what I am going to ask my users for when raising similar issues.