cmssw: Unrunnable schedule exception on a small subset of HLT menu

While investigating something else, I found that a certain small subset of the HLT menu leads to an unrunnable schedule. Recipe to reproduce is e.g.

cmsrel CMSSW_14_0_X_2023-12-06-1100
cd CMSSW_14_0_X_2023-12-06-1100/src
cmsenv
addOnTests.py
cd addOnTests/hlt_data_GRun
cat <<EOF >>RelVal_HLT_RAW2DIGI_L1Reco_RECO.py
process.schedule = cms.Schedule(
    process.DST_HLTMuon_Run3_PFScoutingPixelTracking_v22,
    process.Dataset_Muon0,
    process.Dataset_Muon1,
    process.HLT_DoubleMu43NoFiltersNoVtx_v10
)
EOF
cmsRun RelVal_HLT_RAW2DIGI_L1Reco_RECO.py

that results in

----- Begin Fatal Exception 06-Dec-2023 21:51:12 CET-----------------------
An exception of category 'ScheduleExecutionFailure' occurred while
   [0] Calling beginJob
Exception Message:
Unrunnable schedule
The Path/EndPath configuration could cause the job to deadlock
  module 'hltMuonTriggerResultsFilter' is on path 'DST_HLTMuon_Run3_PFScoutingPixelTracking_v22' and depends on module 'Dataset_Muon0'
  module 'Dataset_Muon0' is on path 'Dataset_Muon0' and follows module 'hltDatasetMuon' on the path
  module 'hltDatasetMuon' is on path 'Dataset_Muon0' and depends on module 'HLT_DoubleMu43NoFiltersNoVtx_v10'
  module 'HLT_DoubleMu43NoFiltersNoVtx_v10' is on path 'HLT_DoubleMu43NoFiltersNoVtx_v10' and follows module 'hltL3fDimuonL1f0L2NVf16L3NoFiltersNoVtxFiltered43' on the path
  module 'hltL3fDimuonL1f0L2NVf16L3NoFiltersNoVtxFiltered43' is on path 'HLT_DoubleMu43NoFiltersNoVtx_v10' and depends on module 'hltL3MuonsIterL3Links'
  module 'hltL3MuonsIterL3Links' is on path 'DST_HLTMuon_Run3_PFScoutingPixelTracking_v22' and follows module 'hltMuonTriggerResultsFilter' on the path
----- End Fatal Exception -------------------------------------------------

Is this expected?

(I’m still looking into what exactly in the full(er) menu allows the schedule to become runnable, in particular because placing hltBoolFalse in a specific place in one Sequence leads the job get stuck because of becoming starved from tasks, which is something we’d much want the framework to catch early)


In addition, the following piece in the configuration of hltL3fDimuonL1f0L2NVf16L3NoFiltersNoVtxFiltered43 caught my eye

process.hltL3fDimuonL1f0L2NVf16L3NoFiltersNoVtxFiltered43 = cms.EDFilter("HLTMuonL3PreFilter",
...
    InputLinks = cms.InputTag("hltL3MuonsIterL3Links"),
    L1CandTag = cms.InputTag(""),
...
    inputMuonCollection = cms.InputTag(""),
...

The hltL3fDimuonL1f0L2NVf16L3NoFiltersNoVtxFiltered43 modules in the HLT_DoubleMu43NoFiltersNoVtx_v10 Path, but the hltL3MuonsIterL3Links is not in the same Path, but in DST_HLTMuon_Run3_PFScoutingPixelTracking_v22. Is this expected? (I recall HLT policy would be to have each Path self contained)

The L1CandTag and inputMuonCollection, that have empty InputTags, are consumed unconditionally https://github.com/cms-sw/cmssw/blob/754d4fbc8bf3e05df6de16937fe6f4d30f5d8699/HLTrigger/Muon/plugins/HLTMuonL3PreFilter.cc#L45-L48 (which is technically ok as far as the framework is concerned)

Getting the L1CandTag collection from the Event is protected with https://github.com/cms-sw/cmssw/blob/754d4fbc8bf3e05df6de16937fe6f4d30f5d8699/HLTrigger/Muon/plugins/HLTMuonL3PreFilter.cc#L232-L236 (it would be better to not call consumes() if it is known at the module construction time that the data product is not consumed)

The inputMuonCollection is, however, gotten from the Event unconditionally, but presumably the MuonToL3s container is always empty in that module so that the de-referencing of the Handle does not throw ProductNotFound exception https://github.com/cms-sw/cmssw/blob/754d4fbc8bf3e05df6de16937fe6f4d30f5d8699/HLTrigger/Muon/plugins/HLTMuonL3PreFilter.cc#L270-L274

I’m wondering if this behavior is expected for this module?

About this issue

  • Original URL
  • State: closed
  • Created 7 months ago
  • Comments: 16 (16 by maintainers)

Most upvoted comments

+hlt

This can be verified using the following script:

#!/bin/bash
cmsrel CMSSW_14_0_X_2024-01-30-2300 
cd CMSSW_14_0_X_2024-01-30-2300/src
cmsenv
addOnTests.py --tests=hlt_data_GRun 
cd addOnTests/hlt_data_GRun
cat <<EOF >>RelVal_HLT_RAW2DIGI_L1Reco_RECO.py
process.schedule = cms.Schedule(
    process.DST_HLTMuon_Run3_PFScoutingPixelTracking_v22,
    process.Dataset_Muon0,
    process.Dataset_Muon1,
    process.HLT_DoubleMu43NoFiltersNoVtx_v10
)
process.hltDatasetMuon.throw = False
EOF
cmsRun RelVal_HLT_RAW2DIGI_L1Reco_RECO.py

For the bug in the configuration of HLT_DoubleMu43NoFiltersNoVtx_v10, I created the CMSHLT Jira ticket https://its.cern.ch/jira/browse/CMSHLT-2975 .

assign hlt