panel: Memory leak in panel

I think I am still experiencing this issue: https://discourse.holoviz.org/t/panel-holoviews-bokeh-app-memory-leaks-looking-for-general-best-practices/2379

Module <module 'bokeh_app_2cc5b245240b4a44b4c714144b1685d1' from 'my_apps.py'> has extra unexpected referrers! This could indicate a serious memory leak. Extra referrers: [<cell at 0x7fd66ebf6310: module object at 0x7fd67015c2f0>]

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Comments: 38 (21 by maintainers)

Most upvoted comments

As I mentioned in #2302 I can recreate this problem. I don’t know if this is the cause of the problem but I can also get a memory leak message with the following code:

import panel as pn
import numba

@numba.njit
def test():
    n = 0
    for i in range(10000):
        n += i
    return n


pn.panel(f"# Test {test()}").servable()

Which gives the following output, note that it has more information than “normally”:

2021-08-18 15:57:28,395 Module <module 'bokeh_app_72fae5a1c97b49b39718157bdc4d25a3' from 'C:\\Users\\WDAGUtilityAccount\\Desktop\\tmp.py'> has extra unexpected referrers! This could indicate a serious memory leak. Extra referrers: [{'func': <function test at 0x0000022D303B55E0>, 'func_qualname': 'test', 'func_name': 'test', 'code': <code object test at 0x0000022D2DD86920, file "C:\Users\WDAGUtilityAccount\Desktop\tmp.py", line 4>, 'module': <module 'bokeh_app_72fae5a1c97b49b39718157bdc4d25a3' from 'C:\\Users\\WDAGUtilityAccount\\Desktop\\tmp.py'>, 'modname': 'bokeh_app_72fae5a1c97b49b39718157bdc4d25a3', 'is_generator': False, 'pysig': <Signature ()>, 'filename': 'C:\\Users\\WDAGUtilityAccount\\Desktop\\tmp.py', 'firstlineno': 4, 'arg_count': 0, 'arg_names': [], 'unique_name': 'test$7'}]

https://user-images.githubusercontent.com/19758978/129912100-d6304b2a-23f0-4c74-b05e-4d88553493db.mp4


Environment information:

conda info
     active environment : base
    active env location : C:\Users\WDAGUtilityAccount\Miniconda3
            shell level : 1
       user config file : C:\Users\WDAGUtilityAccount\.condarc
 populated config files :
          conda version : 4.10.3
    conda-build version : not installed
         python version : 3.9.5.final.0
       virtual packages : __win=0=0
                          __archspec=1=x86_64
       base environment : C:\Users\WDAGUtilityAccount\Miniconda3  (writable)
      conda av data dir : C:\Users\WDAGUtilityAccount\Miniconda3\etc\conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/win-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/win-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/msys2/win-64
                          https://repo.anaconda.com/pkgs/msys2/noarch
          package cache : C:\Users\WDAGUtilityAccount\Miniconda3\pkgs
                          C:\Users\WDAGUtilityAccount\.conda\pkgs
                          C:\Users\WDAGUtilityAccount\AppData\Local\conda\conda\pkgs
       envs directories : C:\Users\WDAGUtilityAccount\Miniconda3\envs
                          C:\Users\WDAGUtilityAccount\.conda\envs
                          C:\Users\WDAGUtilityAccount\AppData\Local\conda\conda\envs
               platform : win-64
             user-agent : conda/4.10.3 requests/2.26.0 CPython/3.9.5 Windows/10 Windows/10.0.19041
          administrator : True
             netrc file : None
           offline mode : False
conda list

# packages in environment at C:\Users\WDAGUtilityAccount\Miniconda3:
#
# Name                    Version                   Build  Channel
argon2-cffi               20.1.0           py39hb82d6ee_2    conda-forge
async_generator           1.10                       py_0    conda-forge
attrs                     21.2.0             pyhd8ed1ab_0    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                        py_2    conda-forge
backports.functools_lru_cache 1.6.4              pyhd8ed1ab_0    conda-forge
bleach                    4.0.0              pyhd8ed1ab_0    conda-forge
bokeh                     2.3.3            py39hcbf5309_0    conda-forge
brotlipy                  0.7.0           py39hb82d6ee_1001    conda-forge
bzip2                     1.0.8                h8ffe710_4    conda-forge
ca-certificates           2021.5.30            h5b45459_0    conda-forge
certifi                   2021.5.30        py39hcbf5309_0    conda-forge
cffi                      1.14.6           py39h0878f49_0    conda-forge
chardet                   4.0.0            py39hcbf5309_1    conda-forge
charset-normalizer        2.0.0              pyhd8ed1ab_0    conda-forge
click                     8.0.1            py39hcbf5309_0    conda-forge
cloudpickle               1.6.0                      py_0    conda-forge
colorama                  0.4.4              pyh9f0ad1d_0    conda-forge
colorcet                  2.0.6                      py_0    pyviz
conda                     4.10.3           py39hcbf5309_0    conda-forge
conda-package-handling    1.7.3            py39hb3671d1_0    conda-forge
console_shortcut          0.1.1                         4
cryptography              3.4.7            py39hd8d06c1_0    conda-forge
cycler                    0.10.0                     py_2    conda-forge
cytoolz                   0.11.0           py39hb82d6ee_3    conda-forge
dask                      2021.8.0           pyhd8ed1ab_0    conda-forge
dask-core                 2021.8.0           pyhd8ed1ab_0    conda-forge
datashader                0.13.0                     py_0    pyviz
datashape                 0.5.4                      py_1    conda-forge
debugpy                   1.4.1            py39h415ef7b_0    conda-forge
decorator                 5.0.9              pyhd8ed1ab_0    conda-forge
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
distributed               2021.8.0         py39hcbf5309_0    conda-forge
entrypoints               0.3             py39hde42818_1002    conda-forge
freetype                  2.10.4               h546665d_1    conda-forge
fsspec                    2021.7.0           pyhd8ed1ab_0    conda-forge
heapdict                  1.0.1                      py_0    conda-forge
holoviews                 1.14.5                     py_0    pyviz
icu                       68.1                 h0e60522_0    conda-forge
idna                      3.1                pyhd3deb0d_0    conda-forge
importlib-metadata        4.6.4            py39hcbf5309_0    conda-forge
intel-openmp              2021.3.0          h57928b3_3372    conda-forge
ipykernel                 6.2.0            py39h832f523_0    conda-forge
ipython                   7.26.0           py39h832f523_0    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
jbig                      2.1               h8d14728_2003    conda-forge
jedi                      0.18.0           py39hcbf5309_2    conda-forge
jinja2                    3.0.1              pyhd8ed1ab_0    conda-forge
jpeg                      9d                   h8ffe710_0    conda-forge
jsonschema                3.2.0              pyhd8ed1ab_3    conda-forge
jupyter_client            6.1.12             pyhd8ed1ab_0    conda-forge
jupyter_core              4.7.1            py39hcbf5309_0    conda-forge
jupyterlab_pygments       0.1.2              pyh9f0ad1d_0    conda-forge
kiwisolver                1.3.1            py39h2e07f2f_1    conda-forge
krb5                      1.19.2               hbae68bd_0    conda-forge
lcms2                     2.12                 h2a16943_0    conda-forge
lerc                      2.2.1                h0e60522_0    conda-forge
libarchive                3.5.1                hb45042f_2    conda-forge
libblas                   3.9.0                     8_mkl    conda-forge
libcblas                  3.9.0                     8_mkl    conda-forge
libclang                  11.1.0          default_h5c34c98_1    conda-forge
libcurl                   7.78.0               h789b8ee_0    conda-forge
libdeflate                1.7                  h8ffe710_5    conda-forge
libiconv                  1.16                 he774522_0    conda-forge
liblapack                 3.9.0                     8_mkl    conda-forge
libpng                    1.6.37               h1d00b33_2    conda-forge
libsodium                 1.0.18               h8d14728_1    conda-forge
libsolv                   0.7.19               h7755175_5    conda-forge
libssh2                   1.9.0                h680486a_6    conda-forge
libtiff                   4.3.0                h0c97f57_1    conda-forge
libxml2                   2.9.12               hf5bbc77_0    conda-forge
llvmlite                  0.36.0           py39ha0cd8c8_0    conda-forge
locket                    0.2.0                      py_2    conda-forge
lz4-c                     1.9.3                h8ffe710_1    conda-forge
lzo                       2.10              he774522_1000    conda-forge
m2w64-gcc-libgfortran     5.3.0                         6    conda-forge
m2w64-gcc-libs            5.3.0                         7    conda-forge
m2w64-gcc-libs-core       5.3.0                         7    conda-forge
m2w64-gmp                 6.1.0                         2    conda-forge
m2w64-libwinpthread-git   5.0.0.4634.697f757               2    conda-forge
mamba                     0.15.2           py39h006a82b_0    conda-forge
markdown                  3.3.4              pyhd8ed1ab_0    conda-forge
markupsafe                2.0.1            py39hb82d6ee_0    conda-forge
matplotlib                3.4.3            py39hcbf5309_0    conda-forge
matplotlib-base           3.4.3            py39h581301d_0    conda-forge
matplotlib-inline         0.1.2              pyhd8ed1ab_2    conda-forge
menuinst                  1.4.17           py39hcbf5309_1    conda-forge
mistune                   0.8.4           py39hb82d6ee_1004    conda-forge
mkl                       2020.4             hb70f87d_311    conda-forge
msgpack-python            1.0.2            py39h2e07f2f_1    conda-forge
msys2-conda-epoch         20160418                      1    conda-forge
multipledispatch          0.6.0                      py_0    conda-forge
nbclient                  0.5.4              pyhd8ed1ab_0    conda-forge
nbconvert                 6.1.0            py39hcbf5309_0    conda-forge
nbformat                  5.1.3              pyhd8ed1ab_0    conda-forge
nest-asyncio              1.5.1              pyhd8ed1ab_0    conda-forge
notebook                  6.4.3              pyha770c72_0    conda-forge
numba                     0.53.1           py39hb8cd55e_1    conda-forge
numpy                     1.21.2           py39h6635163_0    conda-forge
olefile                   0.46               pyh9f0ad1d_1    conda-forge
openjpeg                  2.4.0                hb211442_1    conda-forge
openssl                   1.1.1k               h8ffe710_1    conda-forge
packaging                 21.0               pyhd8ed1ab_0    conda-forge
pandas                    1.3.2            py39h2e25243_0    conda-forge
pandoc                    2.14.1               h8ffe710_0    conda-forge
pandocfilters             1.4.2                      py_1    conda-forge
panel                     0.12.1                     py_0    pyviz
param                     1.11.1                     py_0    pyviz
parso                     0.8.2              pyhd8ed1ab_0    conda-forge
partd                     1.2.0              pyhd8ed1ab_0    conda-forge
pickleshare               0.7.5           py39hde42818_1002    conda-forge
pillow                    8.3.1            py39h916092e_0    conda-forge
pip                       21.2.4             pyhd8ed1ab_0    conda-forge
powershell_shortcut       0.0.1                         3
prometheus_client         0.11.0             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.19             pyha770c72_0    conda-forge
psutil                    5.8.0            py39hb82d6ee_1    conda-forge
pycosat                   0.6.3           py39hb82d6ee_1006    conda-forge
pycparser                 2.20                       py_2
pyct                      0.4.8                      py_0    pyviz
pyct-core                 0.4.8                      py_0    pyviz
pygments                  2.10.0             pyhd8ed1ab_0    conda-forge
pyopenssl                 20.0.1             pyhd3eb1b0_1
pyparsing                 2.4.7              pyh9f0ad1d_0    conda-forge
pyqt                      5.12.3           py39hcbf5309_7    conda-forge
pyqt-impl                 5.12.3           py39h415ef7b_7    conda-forge
pyqt5-sip                 4.19.18          py39h415ef7b_7    conda-forge
pyqtchart                 5.12             py39h415ef7b_7    conda-forge
pyqtwebengine             5.12.1           py39h415ef7b_7    conda-forge
pyrsistent                0.17.3           py39hb82d6ee_2    conda-forge
pysocks                   1.7.1            py39hcbf5309_3    conda-forge
python                    3.9.5                h6244533_3
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python_abi                3.9                      2_cp39    conda-forge
pytz                      2021.1             pyhd8ed1ab_0    conda-forge
pyviz_comms               2.1.0                      py_0    pyviz
pywin32                   301              py39hb82d6ee_0    conda-forge
pywinpty                  1.1.3            py39h99910a6_0    conda-forge
pyyaml                    5.4.1            py39hb82d6ee_1    conda-forge
pyzmq                     22.2.1           py39he46f08e_0    conda-forge
qt                        5.12.9               h5909a2a_4    conda-forge
reproc                    14.2.1               h8ffe710_0    conda-forge
reproc-cpp                14.2.1               h0e60522_0    conda-forge
requests                  2.26.0             pyhd8ed1ab_0    conda-forge
ruamel_yaml               0.15.100         py39h2bbff1b_0
scipy                     1.7.1            py39hc0c34ad_0    conda-forge
send2trash                1.8.0              pyhd8ed1ab_0    conda-forge
setuptools                57.4.0           py39hcbf5309_0    conda-forge
six                       1.16.0             pyhd3eb1b0_0
sortedcontainers          2.4.0              pyhd8ed1ab_0    conda-forge
sqlite                    3.36.0               h8ffe710_0    conda-forge
tblib                     1.7.0              pyhd8ed1ab_0    conda-forge
terminado                 0.10.1           py39hcbf5309_0    conda-forge
testpath                  0.5.0              pyhd8ed1ab_0    conda-forge
tk                        8.6.10               h8ffe710_1    conda-forge
toolz                     0.11.1                     py_0    conda-forge
tornado                   6.1              py39hb82d6ee_1    conda-forge
tqdm                      4.62.1             pyhd8ed1ab_0    conda-forge
traitlets                 5.0.5                      py_0    conda-forge
typing_extensions         3.10.0.0           pyha770c72_0    conda-forge
tzdata                    2021a                he74cb21_1    conda-forge
ucrt                      10.0.20348.0         h57928b3_0    conda-forge
urllib3                   1.26.6             pyhd3eb1b0_1
vc                        14.2                 h21ff451_1
vs2015_runtime            14.29.30037          h902a5da_5    conda-forge
wcwidth                   0.2.5              pyh9f0ad1d_2    conda-forge
webencodings              0.5.1                      py_1    conda-forge
wheel                     0.37.0             pyhd8ed1ab_1    conda-forge
win_inet_pton             1.1.0            py39hcbf5309_2    conda-forge
wincertstore              0.2             py39hcbf5309_1006    conda-forge
winpty                    0.4.3                         4    conda-forge
xarray                    0.19.0             pyhd8ed1ab_1    conda-forge
xz                        5.2.5                h62dcd97_1    conda-forge
yaml                      0.2.5                he774522_0    conda-forge
zeromq                    4.3.4                h0e60522_0    conda-forge
zict                      2.0.0                      py_0    conda-forge
zipp                      3.5.0              pyhd8ed1ab_0    conda-forge
zlib                      1.2.11            h62dcd97_1010    conda-forge
zstd                      1.5.0                h6255e5f_0    conda-forge

Seemingly have tracked down the culprit, if you add import numba.cuda to your app you will get the extra referrer warning. Will ask the numba folks if they have an idea.

The culprit is here: https://github.com/numba/numba/blob/master/numba/cuda/cudadrv/driver.py#L224

The error on import gets held on the numba.cuda.cudadrv.driver.driver singleton object and therefore keeps a reference to the module, which doesn’t get cleaned up. Numba team is aware and will hopefully fix this asap.

I have had this “extra unexpected referrers” warning for months with my panel app that uses datashader. Over the various combinations of versions of bokeh/panel/datashader I have used, I cannot remember a time when this issue did not occur at some point. However I did not bother to report it as I thought it might be a false positive or, if not, at least it did not create noticeable memory problems from a user perspective in my case.

If there’s indeed a real memory leak involved, I’m looking forward to finding a way to prevent that. It’s not clear to me if the root cause may in part lie in the application code itself or not though.

@andhuang-CLGX I am currently not convinced that these plots are entirely accurate in a useful sense. See. e.g

https://distributed.dask.org/en/latest/worker.html#memory-not-released-back-to-the-os

In many cases, high unmanaged memory usage or “memory leak” warnings on workers can be misleading: a worker may not actually be using its memory for anything, but simply hasn’t returned that unused memory back to the operating system, and is hoarding it just in case it needs the memory capacity again. This is not a bug in your code, nor in Dask — it’s actually normal behavior for all processes on Linux and MacOS, and is a consequence of how the low-level memory allocator works (see below for details).

In the work I have done so far I can confirm that after a session cleanup there are not any Bokeh Sessions, Documents or document modules, Bokeh models (modulo one cycle I am still finishing cleaning up), or excess DataFrames anwhere in the Python runtime. This is absolutely certain from direct inspection of gc.get_objects() — those types of objects were present in gc.objects() before cleanup, and are 100% gone afterwards.

Yet, reported RSS does not really shrink after cleanup. Except until it does. If you look in the “part 5” PR you can see that I contrived an example to add 1GB of memory every session. If there is actually a leak then I would expect to eventually OOM in short order. But what happens, opening one session after other is that memory is eventually reclaimed according to RSS reported, but only after ~2GB is exceeded total. This pattern of growing and shrinking repeats indefinitely, and there is never any OOM. It seems undeniable to me that this reported number is being modulated by something at a lower level than we cannot control.

What I would actually want to see demonstrated at this point is a real, actual OOM. That would definitively prove that memory is being leaked. But I would be surprised if that is possible with pure Bokeh. It might possible that there are leaks in Panel, but that will require it’s own investigation.

thanks for your hard work! I really appreciate your quick updates; I will try it out tomorrow and let you know.

I also tried installing bokeh/bokeh#11482 But still encounter memory leaks

@andhuang-CLGX There is a lot of low level things to clean up, that PR was just Part 1 of several (as the title implied). Since you seem capable to test out things from source, here is the latest installment, that is about 95% done, at least from the perspective of anything reproduced by the OP in bokeh/bokeh#11477

Latest PR: https://github.com/bokeh/bokeh/pull/11523

It would certainly be helpful to know if it helps the situation here as well, though as @philippjfr notes, the problem here may lie on the Panel side (e.g. holding references too aggressively)

To offer more context, I am using an app with datashader + a lot of callbacks + a database

But I’ll try to find a MCVE on my personal time