cython: Typed memoryview doesn't work with read-only buffers
Typed memoryviews don’t work with read-only buffers. A banal example:
cpdef getmax(double[:] x):
cdef double max = -float('inf')
for val in x:
if val > max:
max = val
return max
Example session:
>>> import numpy as np
>>> values = np.random.random(8)
>>> values
array([ 0.4574, 0.8468, 0.744 , 0.3764, 0.7581, 0.8123, 0.8783, 0.9341])
>>> import getmax
>>> getmax.getmax(values)
0.9341213061235054
>>> values.setflags(write=False)
>>> assert values.flags.writeable == False
>>> getmax.getmax(values)
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-8-669165de7389> in <module>()
----> 1 getmax.getmax(values)
/tmp/getmax.pyx in getmax.getmax (getmax.c:1617)()
----> 1 cpdef getmax(double[:] x):
2 cdef:
3 double max = -float('inf')
4
5 for val in x:
/tmp/getmax.so in View.MemoryView.memoryview_cwrapper (getmax.c:7403)()
/tmp/getmax.so in View.MemoryView.memoryview.__cinit__ (getmax.c:3678)()
ValueError: buffer source array is read-only
There is no evident reason why above simple function wouldn’t work with a read-only buffer. About two years ago, this issue was raised on the mailing list along with a RFC patch:
https://mail.python.org/pipermail/cython-devel/2015-February/004316.html
Some discussion about a possible use for a const keyword:
https://groups.google.com/forum/#!topic/cython-users/32CMgaLrNhc
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Reactions: 4
- Comments: 15 (9 by maintainers)
Commits related to this issue
- workaround cython not supporting read-only memory view. https://github.com/cython/cython/issues/1605 — committed to rainwoodman/pmesh by rainwoodman 7 years ago
- Initial attempt at implementing read-only memoryviews. (Github issue #1605) — committed to scoder/cython by scoder 7 years ago
- Initial attempt at implementing read-only memoryviews. (Github issue #1605) — committed to scoder/cython by scoder 7 years ago
- Use numpy array instead of memoryview There is a bug with the memoryview where it is unable to handle read-only buffers https://github.com/cython/cython/issues/1605 Because of this I have reverted ba... — committed to malramsay64/MD-Molecules-Hoomd by malramsay64 7 years ago
- Support grayscale inputs/outputs. Copy grayscale images due to https://github.com/cython/cython/issues/1605. — committed to rossumai/whitening by bzamecnik 7 years ago
I have an initial implementation up at https://github.com/scoder/cython/tree/readonly_buffers that automatically determines the need for a writable buffer, and otherwise sticks to a read-only one. It has some bugs and fails to detect some “read-only” vs. “needs writable” cases at compile time, but it mostly works.
It also implements explicit
constmemory views, e.g.cdef const int[:] a, which should always request a read-only view.Feedback welcome.
It appears this bug is keeping people away from the memoryview to use the older ndarray interface. for example, see this workaround in pandas https://github.com/pandas-dev/pandas/pull/12013
I have been surprised that this has not been addressed since I knew of the RFC patch two years ago. But perhaps the Cython team isn’t aware that it should be a priority because the non-support deters users from Cython best practices, particularly where read only memory maps are a key feature of parallelism as in @joblib