godot_voxel: Spamming edits in a large area every frame / loading too much data at once eventually causes memory to run out
As experienced in https://github.com/Zylann/godot_voxel/issues/72#issuecomment-572217527, if you edit a large amount of blocks every frame, memory usage eventually increases to a point the engine crashes due to being out of memory.
Even though that situation is an edge case, it could be avoidable. It happens because when the terrain changes, the module currently copies voxel data (block and neighbor faces) before sending it to the meshing thread. But if the meshing thread is given more blocks to update than it can consume over a period of time, the queue will continue growing forever.
Edit: this also happens if you setup a world with a combination of high view distance and high LOD accuracy, as this will trigger too much updates at once.
A different approach may be taken to make this edge case much less likely to cause trouble.
Copy on write
This sounds like a good candidate, but the voxel data needed for the meshing thread is not just one PoolVector. VoxelBuffers don’t use PoolVectors. In fact, they sometimes don’t even have stored data. So if CoW is implemented it may need to be done with more than just PoolVectors, or differently.
If VoxelBuffer gets changed to use PoolVector, any voxel access or modification will require locking, like it’s the case with Image.
Area locking
The VoxelMap from where voxels are going to be read could be sent, along with the area to polygonize. Then this area will be copied within the thread, just before being used to produce the mesh. This means the area needs to be locked for reading, during the time of copy. During this short time, the main thread will not be allowed to modify voxels in this area, but should be allowed to read them.
I like this idea, however VoxelMap uses a HashMap. If we need to read a few blocks from the thread, we need to access this hash map, so no block can be added or removed from it, anywhere, during this time. I’m aware of concurrent maps, but I don’t know if it’s the best choice here.
One way would be to get those blocks, send them to the thread, and lock them there. This must be done while preserving a consistent iteration order (which is important to avoid deadlocking if two threads access those blocks at the same time).
In any case, in the future accessing voxels will need locking of some form.
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Comments: 15 (8 by maintainers)
On the left, what we have right now. On the right, what I’ve been considering so far:
The type of lock to use would be a read-write lock, so multiple readers would be allowed, but only one writer.
It sure looks cool when it does work 😉