uvgRTP: Memory leak using asynchronous receiving from multiple streams
Hello guys,
I am working with your library and try to push a lot of data from different processes to different ports, which works fine. But as soon as I am trying to receive the data in one process I get a memory leak. I think it has something to do with the size of the data I am pushing but the sender doesn’t seem to have problems sending the information, only the receiver process results in adding up a lot of memory until the ram is full. Below I attached the code which creates the problem, I compiled using “g++ -o <filePrefix> <filePrefix>_main.cpp -luvgrtp -lpthread -lcryptopp”. Most probably I am just using your library incorrectly…
Best regards
receiver_main.cpp:
#include <uvgrtp/lib.hh>
#include <thread>
#include <iostream>
#include <vector>
void receive_hook(void *arg, uvgrtp::frame::rtp_frame *frame)
{
/* Now we own the frame. Here you could give the frame to the application
* if f.ex "arg" was some application-specific pointer
*
* arg->copy_frame(frame) or whatever
*
* When we're done with the frame, it must be deallocated manually */
memcpy(arg, frame->payload, frame->payload_len);
std::cout << "payload: " << frame->payload << std::endl;
(void)uvgrtp::frame::dealloc_frame(frame);
}
int main(void)
{
/* See sending.cc for more details */
uvgrtp::context ctx;
/* See sending.cc for more details */
uvgrtp::session *sess = ctx.create_session("127.0.0.1");
/* See sending.cc for more details */
// these implementations didn't work properly either...
// std::vector<uvgrtp::media_stream*> media_streams;
// uvgrtp::media_stream* media_streams[4];
std::vector<char*> buffer;
for (int i = 0; i < 4; ++i) {
// these implementations didn't work properly either...
// media_streams.push_back(sess->create_stream(51000+i+1, 9, RTP_FORMAT_H265, 0));
// media_streams[i] = sess->create_stream(51000+i+1, 9, RTP_FORMAT_H265, 0);
buffer.push_back(new char[200000]);
}
uvgrtp::media_stream* media_stream1 = sess->create_stream(51001, 9, RTP_FORMAT_H265, 0);
uvgrtp::media_stream* media_stream2 = sess->create_stream(51002, 9, RTP_FORMAT_H265, 0);
// some dummy streams which don't receive anything
uvgrtp::media_stream* media_stream3 = sess->create_stream(51003, 9, RTP_FORMAT_H265, 0);
uvgrtp::media_stream* media_stream4 = sess->create_stream(51004, 9, RTP_FORMAT_H265, 0);
/* Receive hook can be installed and uvgRTP will call this hook when an RTP frame is received
*
* This is a non-blocking operation
*
* If necessary, receive hook can be given an argument and this argument is supplied to
* the receive hook every time the hook is called. This argument could a pointer to application-
* specfic object if the application needs to be called inside the hook
*
* If it's not needed, it should be set to nullptr */
// these implementations didn't work properly either...
// for (int i = 0; i < 4; ++i) {
// media_streams[i]->install_receive_hook((void*)buffer.at(i), receive_hook);
// }
media_stream1->install_receive_hook((void*)buffer.at(0), receive_hook);
media_stream2->install_receive_hook((void*)buffer.at(1), receive_hook);
media_stream3->install_receive_hook((void*)buffer.at(2), receive_hook);
media_stream4->install_receive_hook((void*)buffer.at(3), receive_hook);
while(true){}
/* Session must be destroyed manually */
ctx.destroy_session(sess);
return 0;
}
sender1_main.cpp:
#include <uvgrtp/lib.hh>
#define PAYLOAD_MAXLEN 200000
int main(void)
{
/* To use the library, one must create a global RTP context object */
uvgrtp::context ctx;
/* Each new IP address requires a separate RTP session */
uvgrtp::session *sess = ctx.create_session("127.0.0.1");
/* Each RTP session has one or more media streams. These media streams are bidirectional
* and they require both source and destination ports for the connection. One must also
* specify the media format for the stream and any configuration flags if needed.
*
* See configuration.cc for more details about configuration.
*
* First port is source port aka the port that we listen to and second port is the port
* that remote listens to
*
* This same object is used for both sending and receiving media
*
* In this example, we have one media stream with the remote participant: H265 */
uvgrtp::media_stream *hevc = sess->create_stream(51000, 51001, RTP_FORMAT_H265, RTP_NO_FLAGS);
char* buffer = new char[PAYLOAD_MAXLEN];
std::string blabla;
for (int i = 0; i < 40000; ++i) {
blabla.append("Brrr1");
}
memcpy(buffer, &blabla.front(), PAYLOAD_MAXLEN);
while(true) {
for (int i = 0; i < 10; ++i) {
if (hevc->push_frame((uint8_t*)buffer, PAYLOAD_MAXLEN, RTP_NO_FLAGS) != RTP_OK)
fprintf(stderr, "Failed to send RTP frame!");
}
}
/* Session must be destroyed manually */
delete[] buffer;
ctx.destroy_session(sess);
return 0;
}
sender2_main.cpp:
#include <uvgrtp/lib.hh>
#define PAYLOAD_MAXLEN 200000
int main(void)
{
/* To use the library, one must create a global RTP context object */
uvgrtp::context ctx;
/* Each new IP address requires a separate RTP session */
uvgrtp::session *sess = ctx.create_session("127.0.0.1");
/* Each RTP session has one or more media streams. These media streams are bidirectional
* and they require both source and destination ports for the connection. One must also
* specify the media format for the stream and any configuration flags if needed.
*
* See configuration.cc for more details about configuration.
*
* First port is source port aka the port that we listen to and second port is the port
* that remote listens to
*
* This same object is used for both sending and receiving media
*
* In this example, we have one media stream with the remote participant: H265 */
uvgrtp::media_stream *hevc = sess->create_stream(51009, 51002, RTP_FORMAT_H265, RTP_NO_FLAGS);
char* buffer = new char[PAYLOAD_MAXLEN];
std::string blabla;
for (int i = 0; i < 40000; ++i) {
blabla.append("Brrr2");
}
memcpy(buffer, &blabla.front(), PAYLOAD_MAXLEN);
while(true) {
for (int i = 0; i < 10; ++i) {
if (hevc->push_frame((uint8_t*)buffer, PAYLOAD_MAXLEN, RTP_NO_FLAGS) != RTP_OK)
fprintf(stderr, "Failed to send RTP frame!");
}
}
/* Session must be destroyed manually */
delete[] buffer;
ctx.destroy_session(sess);
return 0;
}
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 19 (9 by maintainers)
Hi,
I can’t dedicate any more time for this issue but I found was the following:
When a new participant joins a session (a media stream is created and a packet is sent), the participant is put on probation and 3 packets with consecutive sequence numbers must be received from the participant before it is fully accepted to the session. Each time a packet is received, this check along with other RTCP-related stuff [1] is done where the
update_participant_seqremoves the probation flag from a participant. I guess in theory this would mean that the packets should be buffered instead of dispatched to the next handler. If I’m reading the code correctly, because of a programming mistake, the first 3 packets are rejected for each stream becauseupdate_participant_seqreturnsRTP_GENERIC_ERROR[2] if the participant is on a probation, which is obviously incorrect. What I don’t understand is how this bug was discovered only now given that it seems to have been lurking there for quite some time.Now, for your case it would mean that the first three packets of your first 200KB frame are rejected but because all other packets of that frame are received, the frame is left incomplete and because uvgRTP doesn’t have garbage collection capabilities (it is/was on the roadmap), the memory is leaked. That is, the frame is put on a frame queue and uvgRTP waits to receive those three packets which are obviously never received because the sender doesn’t know to resend them. This is my explanation as to why memory is leaked, but there may be other issues also. I want to emphasize that I may be mistaken given that I don’t have the time to double-check.
Fixing this is a little trickier. Firstly, RTCP receive handler would have to return
RTP_OKto indicate to the packet dispatcher that is has taken ownership of the packet and that it should not be forwarded to any other handler. Then, after three valid packets have been received, it should give all the buffered packets back to the packet dispatcher which would then forward them to H.256 handler. Currently, uvgRTP doesn’t have capabilities of forwarding multiple packets between different handlers so this would require some architectural changes. Although this change would also allow the implementation of receive-side system call clustering which would provide performance improvements. I don’t know the current status of that feature or whether it’s even on the roadmap.If you want a quick fix to the problem, just convert the
RTP_GENERIC_ERRORtoRTP_OKon line 721 insrc/rtcp.cc. It should fix this issue but if there are other memory leaks in uvgRTP, it won’t fix those. If I’m correct about this RTCP issue, the first frame should be received without issues. Now I see “Received a corrupted frame” in the logs.[1] https://github.com/ultravideo/uvgRTP/blob/master/src/rtcp.cc#L720 [2] https://github.com/ultravideo/uvgRTP/blob/master/src/rtcp.cc#L598
@fador The static packet_handler function seemed like it only affects the local variables which it gets as a parameter, so it may not actually be a problem in multi-stream applications.
@lubertRoft I created an issue which might help in your use case (#76). I will try to document it better when I have time, but in essence, there are configuration parameters that should be adjusted in these kinds of use cases and the configuration example demonstrates this.
I’m currently working on refactoring the code in formats and I will try to include the code snippet (or make the code, so it is easy to include) hopefully during this week.
You’re welcome @lubertRoft! One note about the snippet, since static variable is used, it’s not safe for multi-threaded environment. In this case it might randomly clean up memory in different threads so not really a big issue either 😅 But it’s a reason we cannot just commit this to master right now, have to do some changes…
Valgrind doesn’t help that much because it only tells where the memory is allocated. It seems that the memory is not actually lost, it’s just the number of frames waiting for more data is increasing.
It might help to add this kind of simple garbage collection around here at the end of packet_handler(): https://github.com/ultravideo/uvgRTP/blob/master/src/formats/h265.cc#L451
When I was looking at the code earlier, the error conditions don’t always have a proper memory management so there might be some small memory leaking. However, I’m not aware of any continuous memory leaks.
The plan is to change uvgRTP internals to use smart pointers for memory management where possible. At the same time other memory issues should be taken care of. I would estimate this would happen in release some 2.x release later this year. This is a high priority.