cargo: Cargo is entering an infinite loop
Cargo is randomly entering an infinite loop in some rare cases. Noticed in CI:
- https://github.com/rust-lang/cargo/pull/7836#issuecomment-578529146 — azure
- https://github.com/rust-lang/rust/pull/68580#issuecomment-578996407 — azure
I’ve been able to reproduce it locally by putting Cargo’s own test suite into a loop (while cargo test; do :; done after about 30 minutes on my machine). It seems to randomly affect different tests. Attaching with a debugger, I see it caught in this loop, where dropping an mpsc receiver seems to be confused.
I strongly suspect this is caused by some change in #7731. Perhaps there is some odd interaction between crossbeam’s scope() and mpsc? The only relevant change that I see is that DrainState is now moved into the crossbeam scope, and thus dropped inside the scope, whereas previously the mpsc channels residing inside JobQueue were dropped much later.
@Mark-Simulacrum Are you very familiar with mpsc or crossbeam’s scope? mpsc seems wildly complex, and I don’t know where to start trying to figure out what’s wrong.
I might try removing FinishOnDrop and see if I can repro again, as that seems a little fishy.
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 16 (14 by maintainers)
Commits related to this issue
- Swap std::sync::mpsc channel with crossbeam_channel Switching this hoping it closes #7840 — committed to spastorino/cargo by spastorino 4 years ago
- Replace `std::sync::mpsc` with a much simpler queue We don't need the complexity of most channels since this is not a performance sensitive part of Cargo, nor is it likely to be so any time soon. Cou... — committed to alexcrichton/cargo by alexcrichton 4 years ago
- Auto merge of #7844 - spastorino:fix-infinite-loop, r=alexcrichton Swap std::sync::mpsc channel with crossbeam_channel Hoping it closes #7840 r? @Mark-Simulacrum — committed to rust-lang/cargo by bors 4 years ago
- Auto merge of #7848 - ehuss:beta-revert-jobserver, r=alexcrichton [beta] Revert scalable jobserver. This reverts #7731, #7829, and #7836. This should prevent #7840 on beta. I feel more comfortable ... — committed to rust-lang/cargo by bors 4 years ago
- Replace `std::sync::mpsc` with a much simpler queue We don't need the complexity of most channels since this is not a performance sensitive part of Cargo, nor is it likely to be so any time soon. Cou... — committed to ehuss/cargo by alexcrichton 4 years ago
It affects current nightly but the fix will be included in next nightly.
I can reproduce this in 100% of the cases with current nightly. Once it hangs it quickly consumes all the memory and gets killed by OOM killer. I can push up the crate with modifications if y’all still need to reproduce it.