cmssw: SIGSEGV in LogErrorEventFilter::globalEndLuminosityBlock

Fri Mar 15 07:11:08 CET 2024 Thread 8 (Thread 0x149120fff640 (LWP 4137871) “cmsRun”): #0 0x00001491b353639a in __futex_abstimed_wait_common () from /lib64/libc.so.6 #1 0x00001491b3538ba0 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libc.so.6 #2 0x0000149179a12cbe in Eigen::ThreadPoolTempltsl::thread::EigenEnvironment::WaitForWork(Eigen::EventCount::Waiter*, tsl:🧵:EigenEnvironment::Task*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/external/el9_amd64_gcc12/lib/libtensorflow_cc.so.2 #3 0x0000149179a13223 in Eigen::ThreadPoolTempltsl::thread::EigenEnvironment::WorkerLoop(int) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/external/el9_amd64_gcc12/lib/libtensorflow_cc.so.2 #4 0x0000149179a10a38 in std::_Function_handler<void (), tsl:🧵:EigenEnvironment::CreateThread(std::function<void ()>)::{lambda()#1}>::_M_invoke(std::_Any_data const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/external/el9_amd64_gcc12/lib/libtensorflow_cc.so.2 #5 0x0000149168e4f422 in tsl::(anonymous namespace)::PThread::ThreadFn(void*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/external/el9_amd64_gcc12/lib/libtensorflow_framework.so.2 #6 0x00001491b3539802 in start_thread () from /lib64/libc.so.6 #7 0x00001491b34d9314 in clone () from /lib64/libc.so.6 Thread 7 (Thread 0x1491217ff640 (LWP 4137870) “cmsRun”): #0 0x00001491b353639a in __futex_abstimed_wait_common () from /lib64/libc.so.6 #1 0x00001491b3538ba0 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libc.so.6 #2 0x0000149179a12cbe in Eigen::ThreadPoolTempltsl::thread::EigenEnvironment::WaitForWork(Eigen::EventCount::Waiter*, tsl:🧵:EigenEnvironment::Task*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/external/el9_amd64_gcc12/lib/libtensorflow_cc.so.2 #3 0x0000149179a13223 in Eigen::ThreadPoolTempltsl::thread::EigenEnvironment::WorkerLoop(int) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/external/el9_amd64_gcc12/lib/libtensorflow_cc.so.2 #4 0x0000149179a10a38 in std::_Function_handler<void (), tsl:🧵:EigenEnvironment::CreateThread(std::function<void ()>)::{lambda()#1}>::_M_invoke(std::_Any_data const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/external/el9_amd64_gcc12/lib/libtensorflow_cc.so.2 #5 0x0000149168e4f422 in tsl::(anonymous namespace)::PThread::ThreadFn(void*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/external/el9_amd64_gcc12/lib/libtensorflow_framework.so.2 #6 0x00001491b3539802 in start_thread () from /lib64/libc.so.6 #7 0x00001491b34d9314 in clone () from /lib64/libc.so.6 Thread 6 (Thread 0x149121ed9640 (LWP 4137869) “cmsRun”): #0 0x00001491b353639a in __futex_abstimed_wait_common () from /lib64/libc.so.6 #1 0x00001491b3538ba0 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libc.so.6 #2 0x0000149179a12cbe in Eigen::ThreadPoolTempltsl::thread::EigenEnvironment::WaitForWork(Eigen::EventCount::Waiter*, tsl:🧵:EigenEnvironment::Task*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/external/el9_amd64_gcc12/lib/libtensorflow_cc.so.2 #3 0x0000149179a13223 in Eigen::ThreadPoolTempltsl::thread::EigenEnvironment::WorkerLoop(int) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/external/el9_amd64_gcc12/lib/libtensorflow_cc.so.2 #4 0x0000149179a10a38 in std::_Function_handler<void (), tsl:🧵:EigenEnvironment::CreateThread(std::function<void ()>)::{lambda()#1}>::_M_invoke(std::_Any_data const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/external/el9_amd64_gcc12/lib/libtensorflow_cc.so.2 #5 0x0000149168e4f422 in tsl::(anonymous namespace)::PThread::ThreadFn(void*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/external/el9_amd64_gcc12/lib/libtensorflow_framework.so.2 #6 0x00001491b3539802 in start_thread () from /lib64/libc.so.6 #7 0x00001491b34d9314 in clone () from /lib64/libc.so.6 Thread 5 (Thread 0x14915e7ff640 (LWP 4137852) “cmsRun”): #0 0x00001491b35ad975 in clock_nanosleep@GLIBC_2.2.5 () from /lib64/libc.so.6 #1 0x00001491b35b2527 in nanosleep () from /lib64/libc.so.6 #2 0x00001491b35b245e in sleep () from /lib64/libc.so.6 #3 0x00001491af7d3be0 in sig_pause_for_stacktrace () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/lib/el9_amd64_gcc12/pluginFWCoreServicesPlugins.so #4 <signal handler called> #5 0x00001491b34d8e5d in syscall () from /lib64/libc.so.6 #6 0x00001491b3aacdb2 in tbb::detail::r1::futex_wait (comparand=2, futex=0x1491b04c4024) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el9_amd64_gcc12/external/tbb/v2021.9.0-31639470b6dca6dc015e76f14c6a5a7d/tbb-v2021.9.0/src/tbb/semaphore.h:100 #7 tbb::detail::r1::binary_semaphore:😛 (this=0x1491b04c4024) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el9_amd64_gcc12/external/tbb/v2021.9.0-31639470b6dca6dc015e76f14c6a5a7d/tbb-v2021.9.0/src/tbb/semaphore.h:253 #8 tbb::detail::r1::rml::internal::thread_monitor::wait (this=0x1491b04c4020) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el9_amd64_gcc12/external/tbb/v2021.9.0-31639470b6dca6dc015e76f14c6a5a7d/tbb-v2021.9.0/src/tbb/rml_thread_monitor.h:235 #9 tbb::detail::r1::rml::private_worker::run (this=0x1491b04c4000) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el9_amd64_gcc12/external/tbb/v2021.9.0-31639470b6dca6dc015e76f14c6a5a7d/tbb-v2021.9.0/src/tbb/private_server.cpp:273 #10 tbb::detail::r1::rml::private_worker::thread_routine (arg=0x1491b04c4000) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el9_amd64_gcc12/external/tbb/v2021.9.0-31639470b6dca6dc015e76f14c6a5a7d/tbb-v2021.9.0/src/tbb/private_server.cpp:221 #11 0x00001491b3539802 in start_thread () from /lib64/libc.so.6 #12 0x00001491b34d9314 in clone () from /lib64/libc.so.6 Thread 4 (Thread 0x14915f708640 (LWP 4137851) “cmsRun”): #0 0x00001491b35ad975 in clock_nanosleep@GLIBC_2.2.5 () from /lib64/libc.so.6 #1 0x00001491b35b2527 in nanosleep () from /lib64/libc.so.6 #2 0x00001491b35b245e in sleep () from /lib64/libc.so.6 #3 0x00001491af7d3be0 in sig_pause_for_stacktrace () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/lib/el9_amd64_gcc12/pluginFWCoreServicesPlugins.so #4 <signal handler called> #5 0x00007ffc61d9cbba in clock_gettime () #6 0x00001491b35ad84d in clock_gettime@GLIBC_2.2.5 () from /lib64/libc.so.6 #7 0x00001491b014f2c1 in boost::chrono::thread_clock::now() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/external/el9_amd64_gcc12/lib/libboost_chrono.so.1.80.0 #8 0x00001491ae9f8783 in FastTimerService::Measurement::measure_and_accumulate(FastTimerService::AtomicResources&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/lib/el9_amd64_gcc12/pluginHLTriggerTimerPlugins.so #9 0x00001491b54a8297 in std::__exception_ptr::exception_ptr edm::Worker::runModuleAfterAsyncPrefetch<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3> >(std::__exception_ptr::exception_ptr, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3>::TransitionInfoType const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3>::Context const*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/lib/el9_amd64_gcc12/libFWCoreFramework.so #10 0x00001491b54950cb in edm::Worker::RunModuleTask<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3> >::execute() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/lib/el9_amd64_gcc12/libFWCoreFramework.so #11 0x00001491b545b7de in tbb::detail::d1::function_taskedm::WaitingTaskHolder::doneWaiting(std::__exception_ptr::exception_ptr):{lambda()#1}::execute(tbb::detail::d1::execution_data&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/lib/el9_amd64_gcc12/libFWCoreFramework.so #12 0x00001491b3aaa91b in tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::outermost_worker_waiter> (t=0x1490b72e1b00, waiter=…, this=0x1491b2589480) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el9_amd64_gcc12/external/tbb/v2021.9.0-31639470b6dca6dc015e76f14c6a5a7d/tbb-v2021.9.0/src/tbb/task_dispatcher.h:322 #13 tbb::detail::r1::task_dispatcher::local_wait_for_alltbb::detail::r1::outermost_worker_waiter (t=0x0, waiter=…, this=0x1491b2589480) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el9_amd64_gcc12/external/tbb/v2021.9.0-31639470b6dca6dc015e76f14c6a5a7d/tbb-v2021.9.0/src/tbb/task_dispatcher.h:458 #14 tbb::detail::r1::arena::process (tls=…, this=<optimized out>) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el9_amd64_gcc12/external/tbb/v2021.9.0-31639470b6dca6dc015e76f14c6a5a7d/tbb-v2021.9.0/src/tbb/arena.cpp:137 #15 tbb::detail::r1::market::process (this=<optimized out>, j=…) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el9_amd64_gcc12/external/tbb/v2021.9.0-31639470b6dca6dc015e76f14c6a5a7d/tbb-v2021.9.0/src/tbb/market.cpp:599 #16 0x00001491b3aacace in tbb::detail::r1::rml::private_worker::run (this=0x1491b04c4100) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el9_amd64_gcc12/external/tbb/v2021.9.0-31639470b6dca6dc015e76f14c6a5a7d/tbb-v2021.9.0/src/tbb/private_server.cpp:271 #17 tbb::detail::r1::rml::private_worker::thread_routine (arg=0x1491b04c4100) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el9_amd64_gcc12/external/tbb/v2021.9.0-31639470b6dca6dc015e76f14c6a5a7d/tbb-v2021.9.0/src/tbb/private_server.cpp:221 #18 0x00001491b3539802 in start_thread () from /lib64/libc.so.6 #19 0x00001491b34d9314 in clone () from /lib64/libc.so.6 Thread 3 (Thread 0x149160109640 (LWP 4137850) “cmsRun”): #0 0x00001491b35dc6ff in poll () from /lib64/libc.so.6 #1 0x00001491af821a9f in full_read.constprop () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/lib/el9_amd64_gcc12/pluginFWCoreServicesPlugins.so #2 0x00001491af7d60ac in edm::service::InitRootHandlers::stacktraceFromThread() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/lib/el9_amd64_gcc12/pluginFWCoreServicesPlugins.so #3 0x00001491af7d6230 in sig_dostack_then_abort () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/lib/el9_amd64_gcc12/pluginFWCoreServicesPlugins.so #4 <signal handler called> #5 std::local_Rb_tree_rotate_left (__root=@0x14915d749f78: 0x1490958006c0, __x=0x149092f17d00) at …/…/…/…/…/libstdc+±v3/src/c++98/tree.cc:138 #6 std::_Rb_tree_insert_and_rebalance (__insert_left=<optimized out>, __x=0x14908d1d80c0, __p=<optimized out>, _header=…) at …/…/…/…/…/libstdc+±v3/src/c++98/tree.cc:278 #7 0x00001491574ca785 in LogErrorEventFilter::globalEndLuminosityBlock(edm::LuminosityBlock const&, edm::EventSetup const&) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/lib/el9_amd64_gcc12/pluginDPGAnalysisSkims.so #8 0x00001491574ca9c2 in virtual thunk to edm::global::impl::LuminosityBlockCacheHolder<edm::global::EDFilterBase, leef::LumiErrors>::doEndLuminosityBlock(edm::LuminosityBlock const&, edm::EventSetup const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/lib/el9_amd64_gcc12/pluginDPGAnalysisSkims.so #9 0x00001491b555dbfb in edm::global::EDFilterBase::doEndLuminosityBlock(edm::LumiTransitionInfo const&, edm::ModuleCallingContext const*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/lib/el9_amd64_gcc12/libFWCoreFramework.so #10 0x00001491b5555270 in edm::WorkerTedm::global::EDFilterBase::implDoEnd(edm::LumiTransitionInfo const&, edm::ModuleCallingContext const*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/lib/el9_amd64_gcc12/libFWCoreFramework.so #11 0x00001491b54a825f in std::__exception_ptr::exception_ptr edm::Worker::runModuleAfterAsyncPrefetch<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3> >(std::__exception_ptr::exception_ptr, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3>::TransitionInfoType const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3>::Context const*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/lib/el9_amd64_gcc12/libFWCoreFramework.so #12 0x00001491b54950cb in edm::Worker::RunModuleTask<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3> >::execute() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/lib/el9_amd64_gcc12/libFWCoreFramework.so #13 0x00001491b545b7de in tbb::detail::d1::function_taskedm::WaitingTaskHolder::doneWaiting(std::__exception_ptr::exception_ptr):{lambda()#1}::execute(tbb::detail::d1::execution_data&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/lib/el9_amd64_gcc12/libFWCoreFramework.so #14 0x00001491b3aaa91b in tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::outermost_worker_waiter> (t=0x1490b7516400, waiter=…, this=0x1491b2589400) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el9_amd64_gcc12/external/tbb/v2021.9.0-31639470b6dca6dc015e76f14c6a5a7d/tbb-v2021.9.0/src/tbb/task_dispatcher.h:322 #15 tbb::detail::r1::task_dispatcher::local_wait_for_alltbb::detail::r1::outermost_worker_waiter (t=0x0, waiter=…, this=0x1491b2589400) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el9_amd64_gcc12/external/tbb/v2021.9.0-31639470b6dca6dc015e76f14c6a5a7d/tbb-v2021.9.0/src/tbb/task_dispatcher.h:458 #16 tbb::detail::r1::arena::process (tls=…, this=<optimized out>) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el9_amd64_gcc12/external/tbb/v2021.9.0-31639470b6dca6dc015e76f14c6a5a7d/tbb-v2021.9.0/src/tbb/arena.cpp:137 #17 tbb::detail::r1::market::process (this=<optimized out>, j=…) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el9_amd64_gcc12/external/tbb/v2021.9.0-31639470b6dca6dc015e76f14c6a5a7d/tbb-v2021.9.0/src/tbb/market.cpp:599 #18 0x00001491b3aacace in tbb::detail::r1::rml::private_worker::run (this=0x1491b04c4080) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el9_amd64_gcc12/external/tbb/v2021.9.0-31639470b6dca6dc015e76f14c6a5a7d/tbb-v2021.9.0/src/tbb/private_server.cpp:271 #19 tbb::detail::r1::rml::private_worker::thread_routine (arg=0x1491b04c4080) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el9_amd64_gcc12/external/tbb/v2021.9.0-31639470b6dca6dc015e76f14c6a5a7d/tbb-v2021.9.0/src/tbb/private_server.cpp:221 #20 0x00001491b3539802 in start_thread () from /lib64/libc.so.6 #21 0x00001491b34d9314 in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x149189dbe640 (LWP 4137784) “cmsRun”): #0 0x00001491b35b230f in wait4 () from /lib64/libc.so.6 #1 0x00001491af7d3d37 in edm::service::cmssw_stacktrace_fork() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/lib/el9_amd64_gcc12/pluginFWCoreServicesPlugins.so #2 0x00001491af7d5fda in edm::service::InitRootHandlers::stacktraceHelperThread() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/lib/el9_amd64_gcc12/pluginFWCoreServicesPlugins.so #3 0x00001491b38784d3 in std::execute_native_thread_routine (__p=0x1491a7b09590) at …/…/…/…/…/libstdc+±v3/src/c++11/thread.cc:82 #4 0x00001491b3539802 in start_thread () from /lib64/libc.so.6 #5 0x00001491b34d9314 in clone () from /lib64/libc.so.6 Thread 1 (Thread 0x1491b2e9d640 (LWP 4134141) “cmsRun”): #0 0x00001491b35ad975 in clock_nanosleep@GLIBC_2.2.5 () from /lib64/libc.so.6 #1 0x00001491b35b2527 in nanosleep () from /lib64/libc.so.6 #2 0x00001491b35b245e in sleep () from /lib64/libc.so.6 #3 0x00001491af7d3be0 in sig_pause_for_stacktrace () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/lib/el9_amd64_gcc12/pluginFWCoreServicesPlugins.so #4 <signal handler called> #5 0x00001491b34d86bb in sched_yield () from /lib64/libc.so.6 #6 0x00001491b3ab1516 in __gthread_yield () at /data/cmsbld/jenkins/workspace/build-any-ib/w/el9_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/x86_64-redhat-linux-gnu/bits/gthr-default.h:693 #7 std::this_thread::yield () at /data/cmsbld/jenkins/workspace/build-any-ib/w/el9_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/include/c++/12.3.1/bits/std_thread.h:353 #8 tbb::detail::r1::stealing_loop_backoff::pause (this=0x7ffc61d72038) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el9_amd64_gcc12/external/tbb/v2021.9.0-31639470b6dca6dc015e76f14c6a5a7d/tbb-v2021.9.0/src/tbb/scheduler_common.h:266 #9 tbb::detail::r1::waiter_base::pause (this=0x7ffc61d72030) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el9_amd64_gcc12/external/tbb/v2021.9.0-31639470b6dca6dc015e76f14c6a5a7d/tbb-v2021.9.0/src/tbb/waiters.h:35 #10 tbb::detail::r1::external_waiter::pause (this=0x7ffc61d72030) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el9_amd64_gcc12/external/tbb/v2021.9.0-31639470b6dca6dc015e76f14c6a5a7d/tbb-v2021.9.0/src/tbb/waiters.h:138 #11 tbb::detail::r1::task_dispatcher::receive_or_steal_task<true, tbb::detail::r1::external_waiter> (this=<optimized out>, tls=…, ed=…, waiter=…, isolation=<optimized out>, fifo_allowed=<optimized out>, critical_allowed=<optimized out>) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el9_amd64_gcc12/external/tbb/v2021.9.0-31639470b6dca6dc015e76f14c6a5a7d/tbb-v2021.9.0/src/tbb/task_dispatcher.h:231 #12 0x00001491b3ab3342 in tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::external_waiter> (waiter=…, t=0x0, this=0x1491b2589380) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el9_amd64_gcc12/external/tbb/v2021.9.0-31639470b6dca6dc015e76f14c6a5a7d/tbb-v2021.9.0/src/tbb/task_dispatcher.h:350 #13 tbb::detail::r1::task_dispatcher::local_wait_for_alltbb::detail::r1::external_waiter (waiter=…, t=<optimized out>, this=0x1491b2589380) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el9_amd64_gcc12/external/tbb/v2021.9.0-31639470b6dca6dc015e76f14c6a5a7d/tbb-v2021.9.0/src/tbb/task_dispatcher.h:458 #14 tbb::detail::r1::task_dispatcher::execute_and_wait (t=<optimized out>, wait_ctx=…, w_ctx=…) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el9_amd64_gcc12/external/tbb/v2021.9.0-31639470b6dca6dc015e76f14c6a5a7d/tbb-v2021.9.0/src/tbb/task_dispatcher.cpp:168 #15 0x00001491b546ba0b in edm::FinalWaitingTask::wait() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/lib/el9_amd64_gcc12/libFWCoreFramework.so #16 0x00001491b547518a in edm::EventProcessor::processRuns() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/lib/el9_amd64_gcc12/libFWCoreFramework.so #17 0x00001491b54756e1 in edm::EventProcessor::runToCompletion() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el9_amd64_gcc12/cms/cmssw/CMSSW_14_1_X_2024-03-14-2300/lib/el9_amd64_gcc12/libFWCoreFramework.so #18 0x00000000004074f5 in tbb::detail::d1::task_arena_function<main::{lambda()#1}::operator()() const::{lambda()#1}, void>::operator()() const () #19 0x00001491b3a9f96d in tbb::detail::r1::task_arena_impl::execute (ta=…, d=…) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el9_amd64_gcc12/external/tbb/v2021.9.0-31639470b6dca6dc015e76f14c6a5a7d/tbb-v2021.9.0/src/tbb/arena.cpp:688 #20 0x0000000000408ee2 in main::{lambda()#1}::operator()() const () #21 0x000000000040517c in main ()

Current Modules:

Module: LogErrorEventFilter:logErrorTooManyClusters (crashed)%MSG-w BeamFitter: AlcaBeamMonitor:AlcaBeamMonitor@endLumi 15-Mar-2024 07:11:48 CET Run: 353015 Lumi: 78 No event read! No Fitting! %MSG %MSG-w BeamFitter: AlcaBeamMonitor:AlcaBeamMonitor@endLumi 15-Mar-2024 07:11:48 CET Run: 353015 Lumi: 77 No event read! No Fitting! %MSG

Module: L1TStage2MuonShowerComp:l1tStage2uGMTMuonShowerVsuGMTMuonShowerCopy1 Module: none Module: none

</details>

I didn't manage to reproduce the issue when running locally.