pyodide: memory leak when accessing a.b. a is a Python instance & PyProxy in JS, b is either np.ndarray/bytes and is PyProxyBuffer in JS

Test environment:

  • OS: macOS 11.6
  • Chrome: 93.0.4577.82
  • Hardware: MacBook Air (M1, 2020)
  • Pyodide: 0.18.0

User scenario: I’m trying to make a medical image viewer which needs frequent (10~30 fps) rerendering in some cases. I define a Python class and plan to reuse it on JS side, this idea is from #1426 and a part of the code of the flow is like the part in https://github.com/pyodide/pyodide/discussions/1833#discussioncomment-1329114.

Python class (in an external Python file):

@dataclass
Class Dicom:
    ndarray_data: Optional[np.ndarray] = None # PyProxyBuffer in JS
    pixel_bytes: Optional[bytes] = None. #  # PyProxyBuffer in JS

When the mouse button is pressed and a mouse move event is triggered, JS will

  • execute dicomObj.rerender(). dicomObj is a PyProxy.
    • some ndarray/bytes internally are re-calculated and renewed to ndarray_data/pixel_bytes attributes on Python side.
  • trying to access the resulted ndarray_data or pixel_bytes.

The accessing result is

  • ndarray_data
    • as long as dicomObj.rgba_1d_ndarray is accessed on JS. The memory leak is over 100 MB/s (fps is over 30fps), regardless I tried rgba_1d_ndarray.destroy() and commenting the following getBuffer usage.
    • I tried @property to access Python internal ndarray_data but got the same memory leak.
    • Use a method to access internal ndarray data and the memory will not leak. Use dicomObj.get_ndarray_data() to access. On python:
    ## workaround solution
    def get_ndarray_data(self):
        return self.ndarray_data
    
  • pixel_bytes:
    • it will have memory leak but very slow (in similar flow), 1 MB/s (fps is over 30fps). And direct accessing the attribute, using property, and even using a method will all leak. Comparing to ndarray_data, this part is not very harmful.

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 20 (20 by maintainers)

Most upvoted comments

In that case I think #1854 fixes your problem. Once we merge it, you can try again using the updated development build and hopefully it will work. Of course if you are willing to build the #1854 branch yourself, you can try it out now.

Thanks again for your helpful feedback.

It is not accurate since we need to read it with human eyes.

Haha, very true.

allocate means?

Oops I started a sentence and forgot to finish it, should have read over my comment before posting.

Also, thanks for the report @grimmer0125! This is very useful.