mediasoup: Memory leak in mediasoup c++ worker

This is probably related to the leak I’m seeing in v3. So I’m going to reuse this issue.

STR:

• start mediasoup-demo locally
• open the browser and join a specific meet with many tabs
• monitor the mediasoup-worker memory usage
• wait some time (few seconds, or a minute)
• close tabs

What happens?

mediasoup-worker memory is not released

What should happen?

memory should be released

Notes

After some investigations I came to the fact that initialising buffer here with a much larger size (262144 instead of 65536) makes the memory to be released.

I’m still investigating this issue. I’ve verified that each cloned RTP packet is deleted, so the leak is not that.

@ibc, any idea?

In RtpStreamSend there is the following problem when creating many Consumers at the same time:

The creation of the RtpStreamSend instance initialises a vector of 65536 positions and another of 600, for video steams. Whenever the corresponding Router ends up, the memory is seen by the OS as in use, but it is not (I’ve verified this using malloc_zone_pressure_relief in MacOS).

The problem in this case is that for future Routers, the memory will keep growing. This is, the vector requires adjacent memory and it seems that it does not find it in the already released one, so it requests the OS for more, which could end up in an out of memory condition.

If I allocate four times more positions in the vector (262144) the memory is seen by the OS as free as soon as the RtpStreamSend is destructed. This is because malloc will do it (release the freed memory to the system) for big memory chunks.

On option is, thus, releasing the free memory upon Router destruction.

I commeted code from after this line, and ran heap profile again. https://github.com/versatica/mediasoup/blob/ff17d135957513aff86d45d99e82ffce45187318/worker/src/handles/UdpSocketHandler.cpp#L189

It shows RtpStreamSend objects cost most of memory now.

Total: 461.8 MB 387.3 83.9% 83.9% 387.3 83.9% RTC::RtpStreamSend::RtpStreamSend 30.3 6.6% 90.4% 30.3 6.6% std::_Rb_tree::_M_emplace_unique 23.2 5.0% 95.5% 42.2 9.1% RTC::RtpPacket::Clone 19.0 4.1% 99.6% 19.0 4.1% std::_Rb_tree::_M_emplace_hint_unique [clone .isra.0] 0.5 0.1% 99.7% 0.5 0.1% RTC::RateCalculator::GetRate 0.4 0.1% 99.8% 0.4 0.1% RTC::RateCalculator::Update 0.4 0.1% 99.8% 0.4 0.1% std::vector::reserve

When we malloc a memory < 128K, glibc will call brk()/sbrk() to alloc memory from sytem, when we malloc a memory >128K, glibc will call mmap() to alloc memory. Memory alloced by mmap() will be release back to the system once we free it. Memory alloced by brk() will only be release back to the system when there is a contiguous 128K memory on the top of heap.

See M_TRIM_THRESHOLD and M_MMAP_THRESHOLD in https://man7.org/linux/man-pages/man3/mallopt.3.html That’s maybe why the memory of a four times size vector will be release as soon as it destructed.

So i think we should make the size of UvDataSend a fixed value to avoid holes in heap, and use memory pool or circular queue, so that glibc can use mmap to alloc memory, and limit the max number of UvDataSend objects to avoid out of memory in heavy load.