symusic: Inconsistency between the interface of vector bindings and python list & potential wild pointer problems.
Following #28 , I’m considering adding inplace argument for other functions. But I find that inplace operation does bring some inconsistency between the interface of vector bindings and python list. Also, this might lead to some wild pointer.
An Experiment
Here is an experiment I conducted:
Theoretical Analysis
Essentially, a python list holds an array of pointers (pointing to the real data), while the std::vector holds an array of the real data. Consider the following code:
notes = [Note(i, 0, 0, 0) for i in range(10)]
note = notes[-1]
notes.insert(0, Note(0, 0, 0, 0))
assert note == Note(9, 0, 0, 0)
For python list, we always expect the note (a reference to the last element of that list) to be Note(9, 0, 0, 0) no matter how we insert or remove elements in the list. Since in such operations, we only move some pointers, and we don’t change the position of any real data.
notes = NoteTickList([Note(i, 0, 0, 0) for i in range(10)])
note = notes[-1]
notes.insert(0, Note(0, 0, 0, 0))
assert note == Note(8, 0, 0, 0)
For the vector bindings, e.g. NoteTickList, we do move the real data when insert the new note, while the pointer in note remains the same. Normally (If we don’t reach the vector’s capacity), the note will still point to the 10th element in notes, which is Note(8, 0, 0, 0). So this is an inconsistency between the bindings and python list.
And if we reach the capacity, vector will automatically copy all the data into a new, larger block of memory and free the original. Then, the note will become a “wild pointer”, pointing to an invalid block of memory. If it’s unfortunate enough, this will cause your program to trigger a segmentation fault.
In conclusion, even if such inconsistency is acceptable any operations that modify the position of the real data might lead to some potential “wild pointer”.
Possible Solution
@Natooz I’m quite concerned about this issue and no perfect solution has been found.
- One possible solution is to disable all operations that change the location of the data, and replace it with a copy, like numpy’s append. And as we all know,
numpy.appendis slow and something that should be avoided. - Another is to Replace
vector<Note>withvector<shared_ptr<Note>>. This makes c++ vectors perform like python lists and introduces a lot of memory fragmentation and overheads.
Both of these schemes introduce a significant overhead, and neither looks elegant or pythonic.
Of course, we could write in the documentation and the README to tell the user not to hold a reference to an event for a long time, but to copy it manually.
But I think there are still a lot of people who don’t read these instructions, and then run into the problems I’ve analyzed.
Improving performance always comes at a price, which can take various forms.
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 55 (45 by maintainers)
I finally finished writing a container, pyvec, that will be used to solve the lifetime problem.
Luckily, after testing, the overhead introduced by this container is much smaller compared to shared_ptr, in many cases an order of magnitude smaller.
This means that after considering the overhead associated with python bindings, this overhead is largely negligible.
Even when using only c++, I think this overhead is acceptable. Here are the test results of midi parsing from vtune, where the
to_sharedfunction is all the overhead introduced, which is small compared to parsing midi.I finally make github actions work on the
windows-2022image, see here: https://github.com/wjakob/nanobind/issues/613#issuecomment-2222729794And the
v0.5.0b1is finally available on pypi now.Wonderful! About the tests, I think they are not passing because they weren’t finished 😃 Is it urgent? I can take the time to finish them and make sure it’s working as it should, however I have a limited amount of time right now so I must prioritize, but I can make this effort if necessary!
🥳 I have almost finished the new version. Now, the
sharedbranch could be installed.But since I have totally rewritten the binding codes, I’m note sure did I missed some functions that existed in old version. Further tests are needed.
Edit:
Well, firstly, I need to make it compiled on all platforms 😅.
As I was nearing the end of writing the container I had envisioned, I realized that this solution still didn’t guarantee semantic alignment with Python lists. I ended up choosing to use a large number of shared ptrs to solve this problem.
Of course, this will have a relatively large impact on performance, and I will test this further once the changes are complete.
@leleogere Thanks. I agree with you.
I’m now discussing with the author of nanobind. https://github.com/wjakob/nanobind/discussions/432
I’ve come up with a possible solution that has a smaller overhead. I’ll be refining this idea these days.
I did them manually in Inkscape (an open source vector drawing app). It is quite powerful, but the learning curve might be a bit steep for beginners.
About the topic of this issue, I don’t know a lot of C++, but personally I would expect from a python library to stick as much as possible to the python behavior. Asking people “not to hold a reference to an event for a long time” is kinda vague and might confuse them even more. I’m not sure what is the best solution to solve this, but I thing that I would rather a (small?) performance hit than a mysterious segmentation fault, eventually happening dozens of lines after the real origin of the problem.
I’ll add the notice as soon as possible.
@leleogere, Sorry to interrupt, but I wanted to ask what tool you used to create your previous diagram. It would be easier to understand if I could draw a schematic of the memory layout