accumulo: External Compaction stuck

In testing 2.1.0-RC1, I found that an external compaction became “stuck”.

From the tserver log:

2022-10-25T14:57:33,815 [tablet.files] DEBUG: Compacting 1;1f5c28f5c28f5c58;1eb851eb851eb88 on e.q1 for SYSTEM from [C0000zjn.rf, C0001hht.rf, F0001hjq.rf, F0001hk2.rf, C000188q.rf, C0001hjd.rf, C0001c2z.rf, F0001hkd.rf, C0001eis.rf] size 497 MB
2022-10-25T14:57:33,815 [compactions.CompactionManager] DEBUG: Reserved external compaction ECID:b6c0707e-d39b-4be8-a3d2-8a12485061d5
2022-10-25T14:57:34,923 [compactions.CompactionManager] DEBUG: Attempting to reserve external compaction, queue:q1 priority:-32749 compactor:X.Y.Z.Z:9133

From the compactor log:

2022-10-25T14:57:33,816 [compactor.Compactor] DEBUG: Received next compaction job: TExternalCompactionJob(externalCompactionId:ECID:b6c0707e-d39b-4be8-a3d2-8a12485061d5, extent:TKeyExtent(table:31, endRow:31 66 35 63 32 38 66 35 63 32 38 66 35 63 35 38, prevEndRow:31 65 62 38 35 31 65 62 38 35 31 65 62 38 38),...
2022-10-25T14:57:33,816 [compactor.Compactor] INFO : Starting up compaction runnable for job: TExternalCompactionJob(externalCompactionId:ECID:b6c0707e-d39b-4be8-a3d2-8a12485061d5, extent:TKeyExtent(table:31, endRow:31 66 35 63 32 38 66 35 63 32 38 66 35 63 35 38, prevEndRow:31 65 62 38 35 31 65 62 38 35 31 65 62 38 38)...

and then the compactor was killed by the agitator at 14:58.

From the coordinator log:

2022-10-25T14:57:33,816 [coordinator.CompactionCoordinator] DEBUG: Returning external job ECID:b6c0707e-d39b-4be8-a3d2-8a12485061d5 to X.Y.Z.Z:9133
2022-10-25T14:57:33,817 [coordinator.CompactionCoordinator] DEBUG: Compaction status update, id: ECID:b6c0707e-d39b-4be8-a3d2-8a12485061d5, timestamp: 1666709853817, update: TCompactionStatusUpdate(state:STARTED, message:Compaction started, entriesToBeCompacted:-1, entriesRead:-1, entriesWritten:-1)
2022-10-25T15:00:35,363 [coordinator.DeadCompactionDetector] DEBUG: Possible dead compaction detected ECID:b6c0707e-d39b-4be8-a3d2-8a12485061d5 1;1f5c28f5c28f5c58;1eb851eb851eb88
2022-10-25T15:05:35,380 [coordinator.DeadCompactionDetector] DEBUG: Possible dead compaction detected ECID:b6c0707e-d39b-4be8-a3d2-8a12485061d5 1;1f5c28f5c28f5c58;1eb851eb851eb88

The message 'Possible dead compaction detected…` is emitted here and then the compaction is killed here if it happens more than twice. But it doesn’t for some reason.