scipy: BUG: `@_transition_to_rng` leads to inconsistent function signatures

For example in the stats.qmc engines, the seed parameter can still be used in place of rng (and is not deprecated yet). For this SPEC 7 transition a convenient _lib._util._transition_to_rng decorator is used, effectively mapping calls with seed={} to rng={}. And even though rng and seed are not allowed to be used together, this “added” seed parameter should also be present in the function signature:

>>> import inspect
>>> import scipy
>>> list(inspect.signature(scipy.stats.qmc.Sobol).parameters)
['d', 'scramble', 'bits', 'rng', 'optimization']
>>> scipy.__version__
'1.16.0'

Mypy’s stubtest relies on these signatures, and uses that to validate if scipy-stubs matches the runtime implementation. The result is that I have removed all seed kwargs from methods and functions that are decorated with @_transition_to_rng, because I assumed that stubtest was right (as it almost always is). I just discovered that this was not the case, leading to false rejections (i.e. CI failures) in libraries such as optuna that use scipy-stubs (scipy/scipy-stubs/issues/644).

If I now restore the seed parameter, stubtest gets angry. This forces me to mark all of these methods as ignored by stubtest as a whole, which results in blind spots.

The @_transition_to_rng decorator looks pretty complicated, and I couldn’t immediately find an obvious solution to this issue. I’m also not very familiar with the low-level machinery behindinspect.signature, so I’m afraid that there’s no solution I can propose here.

I have listed the 27 functions and methods that are missing a seed kwarg as a consequence of this issue in scipy/scipy-stubs#648 .

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Reactions: 1
  • Comments: 18 (18 by maintainers)

Commits related to this issue

Most upvoted comments

Actually, I brought this up at a SPEC planning meeting. It sounds like the team would be more interested in seeing how this plays out in SciPy before changing the SPEC. So yes, @jorenham, since SciPy does not need to follow the SPEC exactly, feel free to propose a PR here. (Personally, I am probably fine with whatever happens, but I’d rather not be involved further. I personally prefer that the old keyword continues to be accepted only for backward compatibility purposes; it is preferable for the old keyword not to be listed explicitly in the function signature. If that means temporarily skipping a test about typing of a few functions for a few functions, I am comfortable with that.)