legion: Non-deterministic hangs while running HTR

I am running a 45-node simulation on Lassen using HTR and the 11/24 Legion commit f4f80752 of control_replication. The runtime hangs non-deterministically, usually after ~10-100 time steps. Attached are two backtraces taken during one of these hangs, taken several minutes apart. Legion is compiled with CXXFLAGS='-g' and the GasNet version is 2021.9.0. HTR is run with DEBUG=1.

bt-first.tar.gz bt-second.tar.gz