setuptools: [BUG] setuptools 75.7.0 and CFLAGS

setuptools version

75.7.0

Python version

3.12.8

OS

ALTLinux

Additional environment information

No response

Description

With https://github.com/pypa/distutils/issues/299 (merged to setuptools 75.7.0) the CFLAGS behavior changed. Docs are not updated to that change and describe the previous state.

https://setuptools.pypa.io/en/latest/userguide/ext_modules.html#compiler-and-linker-options

The compiler options appear in the command line in the following order:

  • first, the options provided by the sysconfig variable CFLAGS,
  • then, the options provided by the environment variables CFLAGS and CPPFLAGS,

This is the behavior used for RPM build for long time. CFLAGS was/is used to append additional distro-specific options to the CPython’s ones.

After the aforementioned change CFLAGS env variable overrides sysconfigs CFLAGS and it’s no longer possible to supply additional flags.

So it’s a behavior change.

Expected behavior

There should be a way to pass additional flags for compilers according to documentation.

How to Reproduce

Try to build an extension with CFLAGS and without to see the difference in compiler flags.

Output


About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 17 (11 by maintainers)

Commits related to this issue

Most upvoted comments

I’ll note that this change from “append” to “overwrite” is indeed a significant backwards compatibility break.

The “append” behavior is nicer there, because one wants flags from compiler activation and from CPython (for backwards compatibility

I still think it’s worth it for the reasons I mentioned in https://github.com/pypa/setuptools/issues/4836#issuecomment-2674578747. But not only that, the flags that CPython was built with aren’t necessarily the ones used – it depends on which variables were used (NODIST or not), and the results of CPython’s configure tests. I just can’t see a situation where you could actually rely on any side-effects from that.

To put it another way: I can’t see a realistic scenario where this affects someone adversely:

  • If you needed a flag set system-wide, you already had to take more care for other (including Python) build systems.
  • If we’re talking about flags now missing, you again would’ve needed to build with it until now in CFLAGS for other build systems, and it’d be an odd way of going about it to inject only into CPython.
  • For NDEBUG, the issue is usually “we turned off asserts, and it turned out we were relying on side-effects in an assert”, but the reverse I’ve never seen happen (though possible) – and there it depended on your Python configuration too.

I understand it’s a material change, I can’t see how it’s something you should’ve relied on ever because even with the old behaviour, you couldn’t predict it in many cases. It also prevented enabling assertions if you wanted them (i.e. I see it as actively harmful given it might make one think they’re testing something they’re not).

It causes problems to append for several cases, I can’t think of a case where it actually helps.

(For OP’s case, from a packaging perspective, it seems very strange to have been setting bare CFLAGS="-Wall" with nothing else, not just appending -Wall to CFLAGS.)

so switching this default has probably changed behavior of a lot of package builds

We do the same and it was part of the motivation for the change, even.

All of that said: of course, if folks want, I can take the approach you did for numpy.

This answer seems to be incorrect? To go back to the backwards compatible situation, it looks like one has do something like this:

I think we’re in agreement? If CFLAGS is completely unset, you get the sysconfig value. If you set CFLAGS at all yourself, you have full control over what you set .

(I was making the point that “it’s no longer possible” isn’t accurate, of course it doesn’t append for you anymore to the sysconfig flags unless you handle that. I couldn’t understand what the actual complaint was so was guessing.)

But it has been changed without any reason.

No, this isn’t true. Please assume good faith. There are several reasons:

  • consistency with CXXFLAGS, as noted in the commit message
  • consistency with other build systems (both inside of the Python ecosystem and outside)
  • allowing Python packages using setuptools to be built with another compiler than was used to build CPython (if you use a flag that is incompatible when building CPython itself…)
  • avoiding unexpected behaviour (e.g. building CPython with -DNDEBUG, then suddenly it appears in all package builds)

There is no standard that regulates usage of CFLAGS (correct me if I’m wrong), it can be used to either override project’s options passed to compiler or append to them. From my personal subjective experience it’s mostly the latter.

I have extensive experience packaging and it’s considered a bug when something hardcodes/calcifies CFLAGS from the build of another package. It causes all sorts of problems. Note that we are not overriding the CFLAGS used by a specific project anyway, but rather just whatever CPython happens to use – but you couldn’t rely on that anyway, as other (even Python) build systems didn’t do that.