scipy: BUG: beta.pdf is broken on main (1.11.0.dev0)

Describe your issue.

beta.pdf has changes from 1.10 to the 1.11.0.dev0 build and now returns incorrect results.

Reproducing Code Example

import numpy as np
from scipy import stats

xx = np.array([0.        , 0.11111111, 0.22222222, 0.33333333, 0.44444444, 0.55555556, 0.66666667, 0.77777778, 0.88888889, 1.        ])
k = np.array([0., 1., 2., 3., 4.])
n=4.0

pdf = stats.beta.pdf(xx[..., None], k + 1, n - k + 1)


np.testing.assert_allclose(pdf[-1,-1],5.0)

Passes on 1.10.0, fails on 1.11.0.dev0.

Error message

AssertionError: 
Not equal to tolerance rtol=1e-07, atol=0
Mismatched elements: 1 / 1 (100%)
Max absolute difference: 5.
Max relative difference: 1.
 x: array(0.)
 y: array(5.)

SciPy/NumPy/Python version and system information

1.11.0.dev0+1817.98c51a6 1.23.5 sys.version_info(major=3, minor=11, micro=2, releaselevel='final', serial=0)
{
  "Compilers": {
    "c": {
      "name": "gcc",
      "linker": "ld.bfd",
      "version": "10.3.0",
      "commands": "cc"
    },
    "cython": {
      "name": "cython",
      "linker": "cython",
      "version": "0.29.34",
      "commands": "cython"
    },
    "c++": {
      "name": "gcc",
      "linker": "ld.bfd",
      "version": "10.3.0",
      "commands": "c++"
    },
    "fortran": {
      "name": "gcc",
      "linker": "ld.bfd",
      "version": "10.3.0",
      "commands": "gfortran"
    },
    "pythran": {
      "version": "0.12.1",
      "include directory": "C:\\Users\\runneradmin\\AppData\\Local\\Temp\\pip-build-env-024pp_ot\\overlay\\Lib\\site-packages/pythran"
    }
  },
  "Machine Information": {
    "host": {
      "cpu": "x86_64",
      "family": "x86_64",
      "endian": "little",
      "system": "windows"
    },
    "build": {
      "cpu": "x86_64",
      "family": "x86_64",
      "endian": "little",
      "system": "windows"
    },
    "cross-compiled": false
  },
  "Build Dependencies": {
    "blas": {
      "name": "openblas",
      "found": true,
      "version": "0.3.21.dev",
      "detection method": "pkgconfig",
      "include directory": "/c/opt/64/include",
      "lib directory": "/c/opt/64/lib",
      "openblas configuration": "USE_64BITINT= DYNAMIC_ARCH=1 DYNAMIC_OLDER= NO_CBLAS= NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= SKYLAKEX MAX_THREADS=2",
      "pc file directory": "c:/opt/64/lib/pkgconfig"
    },
    "lapack": {
      "name": "openblas",
      "found": true,
      "version": "0.3.21.dev",
      "detection method": "pkgconfig",
      "include directory": "/c/opt/64/include",
      "lib directory": "/c/opt/64/lib",
      "openblas configuration": "USE_64BITINT= DYNAMIC_ARCH=1 DYNAMIC_OLDER= NO_CBLAS= NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= SKYLAKEX MAX_THREADS=2",
      "pc file directory": "c:/opt/64/lib/pkgconfig"
    }
  },
  "Python Information": {
    "path": "C:\\Users\\runneradmin\\AppData\\Local\\Temp\\cibw-run-cgdr6yw7\\cp311-win_amd64\\build\\venv\\Scripts\\python.exe",
    "version": "3.11"
  }
}

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 17 (16 by maintainers)

Commits related to this issue

Most upvoted comments

I confirm itโ€™s fixed ๐Ÿš€ I am making a PR to update the submodule now. Thanks again everyone!

The linked PR for boost.math has been merged. Let me know if that solves your issues.

Thanks for the discussion everyone, much appreciated.

I think we have a few issues here:

  1. if alpha < 1 and x is zero, or beta < 1 and x is 1, then we are just plain wrong, and should be returning infinity (overflow_error in our case). @mborland the offending code is here: https://github.com/boostorg/math/blob/109a814e89f77ff8a3fc8f0391f6b35a12640669/include/boost/math/distributions/beta.hpp#L395
  2. If alpha = 1 and x is 0, or beta = 1 and x = 1, then we end up with 0^0 in the definition of the PDF which is undefined. So weโ€™re not wrong as such, but I agree that returning the limit (1/beta(alpha, beta)) would be preferable.

Hopefully I have that right ๐Ÿ˜‰

@mborland are you OK to take care of this?

Thanks @mborland!

Itโ€™s tracking 7203fa2def6347b0d5f8fe1e8522d5b0a618db9d but I could reproduce on 109a814e89f77ff8a3fc8f0391f6b35a12640669 (develop). If I did this correctly, I updated and rebuilt from scratch ๐Ÿ˜