mpmath: ENH: guard against incorrect use of `mp.dps`?

I am a SciPy maintainer, and we frequently use mpmath to compute reference values. (Thank you for your work! It’s very helpful.)

A common mistake is for contributors to do, e.g.:

import numpy as np
import mpmath as mp  # should be from mpmath import mp
mp.dps = 50
a, b = mp.mpf(1e-11), mp.mpf(1.001e-11)
print(np.float64(mp.ncdf(b) - mp.ncdf(a)))  # 3.885780586188048e-15

This suffers from catastrophic cancellation just as:

from scipy.special import ndtr
a, b = 1e-11, 1.001e-11
print(ndtr(b) - ndtr(a))  # 3.885780586188048e-15

does, but the fact that mpmath is not working with 50 digits is obscured by the conversion to float. (I’ve run into other, subtler reasons for not noticing the problem, too.)

Another example in the wild: https://github.com/scipy/scipy/issues/18088#issuecomment-1712538038

I know that this is user error and not a bug in mpmath, but I wonder if it is possible to guard against this. For instance, mpmath doesn’t currently have a dps attribute. Perhaps it could be added as a property, and a warning (or error) could be raised if modification is attempted, directing the user to mpmath.mp.dps?

About this issue

Original URL
State: closed
Created a year ago
Comments: 26 (11 by maintainers)

Commits related to this issue

prevent erroneous read of dps from mpmath module What we really want for Issue #657 is an exception on write access. Instead here is a partial implementation for read access. Not even close to a sol... — committed to cbm755/mpmath by cbm755 a year ago
Use sys.modules hacks to prevent setting dps/prec Fixes Issue #657. Co-authored-by: Warren Weckesser <warren.weckesser@gmail.com> — committed to cbm755/mpmath by cbm755 a year ago
prevent erroneous setting of dps/prec on mpmath module (#678) * Use sys.modules hacks to prevent setting dps/prec Fixes Issue #657 --------- Co-authored-by: Sergey B Kirpichev <skirpichev@gm... — committed to mpmath/mpmath by cbm755 6 months ago

Most upvoted comments

I agree with @cbm755 and @oscarbenjamin that option 2 is not really a good solution. Setting an attribute of mpmath to change to the behavior of mpmath.mp is not a good API, and is not something that you would want to maintain in the long run.

… can you send a MR? I’d like to try that out.

It would be easy enough to a create a pull request, but I probably won’t have time for all the likely follow-up (unit tests, check benchmarks, release note (maybe), fix the unexpected breakage that the change causes (those darn unknown unknowns), etc.), so I don’t plan on creating a pull request at the moment. Anyone is welcome to grab that code and modify it as they see fit to create a pull request. If you just want to try it out, it is really as easy as copying that block of code into the beginning of mpsci/__init__.py.

WarrenWeckesser on Apr 1, 2023

You could make setting mpmath.dps raise a warning or error. So that way we don’t commit to the API but it does help people avoid the gotcha.

asmeurer on Apr 2, 2023

FWIW: this idea is derived from a code snippet discussed in the rejected PEP 549 that is referred to in PEP 562.

I added the following to mpmath/__init__.py:

import sys
import types


class SetMPMathPrecisionError(RuntimeError):

    def __init__(self, attribute):
        self._attribute = attribute
    
    def __str__(self):
        name = self._attribute
        return (f"cannot set '{name}' on 'mpmath'. Did you mean to "
                f"set '{name}' on 'mpmath.mp'?")


class _MPMathModuleType(types.ModuleType):

    def _set_dps(self, value):
        raise SetMPMathPrecisionError('dps')

    dps = property(fset=_set_dps)

    def _set_prec(self, value):
        raise SetMPMathPrecisionError('prec')

    prec = property(fset=_set_prec)


sys.modules[__name__].__class__ = _MPMathModuleType

Then attempting to set mpmath.dps or mpmath.prec raises an error:

In [1]: import mpmath

In [2]: mpmath.dps = 50
---------------------------------------------------------------------------
SetMPMathPrecisionError                   Traceback (most recent call last)
Cell In [2], line 1
----> 1 mpmath.dps = 50

File ~/py3.10.8/lib/python3.10/site-packages/mpmath/__init__.py:19, in _MPMathModuleType._set_dps(self, value)
     18 def _set_dps(self, value):
---> 19     raise SetMPMathPrecisionError('dps')

SetMPMathPrecisionError: cannot set 'dps' on 'mpmath'. Did you mean to set 'dps' on 'mpmath.mp'?

In [3]: mpmath.prec = 96
---------------------------------------------------------------------------
SetMPMathPrecisionError                   Traceback (most recent call last)
Cell In [3], line 1
----> 1 mpmath.prec = 96

File ~/py3.10.8/lib/python3.10/site-packages/mpmath/__init__.py:24, in _MPMathModuleType._set_prec(self, value)
     23 def _set_prec(self, value):
---> 24     raise SetMPMathPrecisionError('prec')

SetMPMathPrecisionError: cannot set 'prec' on 'mpmath'. Did you mean to set 'prec' on 'mpmath.mp'?

WarrenWeckesser on Apr 1, 2023