pyodide: memory leak when accessing a.b. a is a Python instance & PyProxy in JS, b is either np.ndarray/bytes and is PyProxyBuffer in JS
Test environment:
- OS: macOS 11.6
- Chrome: 93.0.4577.82
- Hardware: MacBook Air (M1, 2020)
- Pyodide: 0.18.0
User scenario: I’m trying to make a medical image viewer which needs frequent (10~30 fps) rerendering in some cases. I define a Python class and plan to reuse it on JS side, this idea is from #1426 and a part of the code of the flow is like the part in https://github.com/pyodide/pyodide/discussions/1833#discussioncomment-1329114.
Python class (in an external Python file):
@dataclass
Class Dicom:
ndarray_data: Optional[np.ndarray] = None # PyProxyBuffer in JS
pixel_bytes: Optional[bytes] = None. # # PyProxyBuffer in JS
When the mouse button is pressed and a mouse move event is triggered, JS will
- execute
dicomObj.rerender()
. dicomObj is a PyProxy.- some
ndarray
/bytes
internally are re-calculated and renewed tondarray_data/pixel_bytes
attributes on Python side.
- some
- trying to access the resulted ndarray_data or pixel_bytes.
The accessing result is
- ndarray_data
- as long as
dicomObj.rgba_1d_ndarray
is accessed on JS. The memory leak is over 100 MB/s (fps is over 30fps), regardless I triedrgba_1d_ndarray.destroy()
and commenting the followinggetBuffer
usage. - I tried
@property
to access Python internal ndarray_data but got the same memory leak. - Use a method to access internal
ndarray
data and the memory will not leak. UsedicomObj.get_ndarray_data()
to access. On python:
## workaround solution def get_ndarray_data(self): return self.ndarray_data
- as long as
- pixel_bytes:
- it will have memory leak but very slow (in similar flow), 1 MB/s (fps is over 30fps). And direct accessing the attribute, using property, and even using a method will all leak. Comparing to
ndarray_data
, this part is not very harmful.
- it will have memory leak but very slow (in similar flow), 1 MB/s (fps is over 30fps). And direct accessing the attribute, using property, and even using a method will all leak. Comparing to
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 20 (20 by maintainers)
In that case I think #1854 fixes your problem. Once we merge it, you can try again using the updated development build and hopefully it will work. Of course if you are willing to build the #1854 branch yourself, you can try it out now.
Thanks again for your helpful feedback.
Haha, very true.
Oops I started a sentence and forgot to finish it, should have read over my comment before posting.
Also, thanks for the report @grimmer0125! This is very useful.