mediasoup: Worker crash: 'std::out_of_range' in `absl::container_internal::raw_hash_map<>::at undefined`

Bug Report

Your environment

  • Operating system: Linux Debian
  • gcc/clang version: gcc 12.2.0 (Debian 12.2.0-14)
  • mediasoup version: 3.16.1

Issue description

mediasoup-demo crashes with this error:

2025-06-26T14:35:13.386Z mediasoup:ERROR:Worker (stderr) terminate called after throwing an instance of 'std::out_of_range' undefined
2025-06-26T14:35:13.388Z mediasoup:ERROR:Worker (stderr)   what():  absl::container_internal::raw_hash_map<>::at undefined
2025-06-26T14:35:13.419Z mediasoup:ERROR:Channel Producer Channel error: Error: read ECONNRESET undefined
2025-06-26T14:35:13.439Z mediasoup:ERROR:Worker worker process died unexpectedly [pid:3452214, code:null, signal:SIGABRT] undefined

Also reported in the forum: https://mediasoup.discourse.group/t/mediasoup-worker-unexpectedly-died-in-v3-16-1/6741

Core dump

(gdb) bt full
#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
        tid = <optimized out>
        ret = 0
        pd = <optimized out>
        old_mask = {__val = {140720918794352}}
        ret = <optimized out>
#1  0x00007f2462dcad9f in __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78
No locals.
#2  0x00007f2462d7bf32 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
        ret = <optimized out>
#3  0x00007f2462d66472 in __GI_abort () at ./stdlib/abort.c:79
        save_stage = 1
        act = {__sigaction_handler = {sa_handler = 0x20, sa_sigaction = 0x20}, sa_mask = {__val = {18446744073709551615, 13835058058503389184, 13835058058503389184, 18446744073709551615, 
              18446603336221196287, 13835058058503389184, 13835058058503389184, 93978900237968, 1152518615, 25353, 139794254143209, 139794255525504, 139794254144243, 139794255525504, 10, 
              139794253604032}}, sa_flags = 1658574970, sa_restorer = 0x55792b348e78}
#4  0x00007f246309d919 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
No symbol table info available.
#5  0x00007f24630a8e1a in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
No symbol table info available.
#6  0x00007f24630a8e85 in std::terminate() () from /lib/x86_64-linux-gnu/libstdc++.so.6
No symbol table info available.
#7  0x00007f24630a90d8 in __cxa_throw () from /lib/x86_64-linux-gnu/libstdc++.so.6
No symbol table info available.
#8  0x0000557928cccc39 in absl::lts_20240722::base_internal::ThrowStdOutOfRange(char const*) ()
No symbol table info available.
#9  0x0000557928df22fb in RTC::SimulcastConsumer::SendRtpPacket(RTC::RtpPacket*, std::shared_ptr<RTC::RtpPacket>&) ()
No symbol table info available.
#10 0x0000557928df2924 in RTC::SimulcastConsumer::SendRtpPacket(RTC::RtpPacket*, std::shared_ptr<RTC::RtpPacket>&) ()
No symbol table info available.
#11 0x0000557928da0362 in RTC::Router::OnTransportProducerRtpPacketReceived(RTC::Transport*, RTC::Producer*, RTC::RtpPacket*) ()
No symbol table info available.
#12 0x0000557928d8b2db in RTC::Producer::ReceiveRtpPacket(RTC::RtpPacket*) ()
No symbol table info available.
#13 0x0000557928e05651 in RTC::Transport::ReceiveRtpPacket(RTC::RtpPacket*) ()
No symbol table info available.
#14 0x0000557928e2fefe in RTC::WebRtcTransport::OnRtpDataReceived(RTC::TransportTuple*, unsigned char const*, unsigned long) ()
No symbol table info available.
#15 0x0000557928e25a02 in RTC::WebRtcServer::OnUdpSocketPacketReceived(RTC::UdpSocket*, unsigned char const*, unsigned long, sockaddr const*) ()
No symbol table info available.
#16 0x000055792914ac4d in uv.udp_recvmmsg ()

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 17 (16 by maintainers)

Commits related to this issue

Most upvoted comments

I’m reverting/disabled the target layer retransmission buffer until this issue is fixed:

https://github.com/versatica/mediasoup/commit/94ec472738efcf1670d80f59638b11d72e58fccc

mediasoup 3.16.2 released with it disabled.

The root cause is indeed a shared packet SSRC mismatch when packets from different spatial layers (different SSRCs) reuse the same sequence number space.

Note: That’s not the issue. It’s not related to simulcast or spatial layers. The root cause is explained in the draft PR that fixes it.

@miroslavpejic85, cause of the issue and proper solution is exposed in draft PR https://github.com/versatica/mediasoup/pull/1558.

Regarding your proposal:

  1. The safety check above is not ok. We don’t want to hide the bug. It should not happen.
  2. We have to restore values in sharedPacket only if it was filled by this consumer. Checking if the payload is different is very weak since they could perfectly be the same.