cmssw: [GPU] Workflow failures when running the alpaka customization in presence of a `Fake` menu

Several workflows {12434,12450}.{402,403,404,412} fail in GPU IB tests in CMSSW_14_1_GPU_X_2024-02-26-2300 along:

DIGI:pdigi_valid,L1,DIGI2RAW,HLT:@relval2023,ENDJOB
We have determined that this is simulation (if not, rerun cmsDriver.py with --data)
with DB:
entry filelist:step1_dasquery.log
found files:  ['/store/relval/CMSSW_13_0_10/RelValTTbar_14TeV/GEN-SIM/130X_mcRun3_2023_realistic_withEarly2023BS_v1_2023-v1/2590000/4a9c4099-1812-4afd-9c94-6f9409595929.root', '/store/relval/CMSSW_13_0_10/RelValTTbar_14TeV/GEN-SIM/130X_mcRun3_2023_realistic_withEarly2023BS_v1_2023-v1/2590000/99db1b20-ec34-4bff-84df-dfffcbdfb184.root', '/store/relval/CMSSW_13_0_10/RelValTTbar_14TeV/GEN-SIM/130X_mcRun3_2023_realistic_withEarly2023BS_v1_2023-v1/2590000/c388e800-ddaa-408d-a2ec-b40a9b8c7a08.root']
Step: DIGI Spec: ['pdigi_valid']
Step: L1 Spec: 
Step: DIGI2RAW Spec: 
Step: HLT Spec: ['@relval2023']
Traceback (most recent call last):
  File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02826/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_GPU_X_2024-02-26-2300/bin/el8_amd64_gcc12/cmsDriver.py", line 40, in <module>
    run()
  File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02826/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_GPU_X_2024-02-26-2300/bin/el8_amd64_gcc12/cmsDriver.py", line 16, in run
    configBuilder.prepare()
  File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02826/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_GPU_X_2024-02-26-2300/src/Configuration/Applications/python/ConfigBuilder.py", line 2310, in prepare
    self.addStandardSequences()
  File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02826/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_GPU_X_2024-02-26-2300/src/Configuration/Applications/python/ConfigBuilder.py", line 850, in addStandardSequences
    getattr(self,"prepare_"+stepName)(stepSpec = '+'.join(stepSpec))
  File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02826/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_GPU_X_2024-02-26-2300/src/Configuration/Applications/python/ConfigBuilder.py", line 1670, in prepare_HLT
    self.loadAndRemember('HLTrigger/Configuration/HLT_%s_cff' % stepSpec)
  File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02826/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_GPU_X_2024-02-26-2300/src/Configuration/Applications/python/ConfigBuilder.py", line 376, in loadAndRemember
    self.process.load(includeFile)
  File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02826/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_GPU_X_2024-02-26-2300/src/FWCore/ParameterSet/python/Config.py", line 761, in load
    module = __import__(moduleName)
  File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02826/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_GPU_X_2024-02-26-2300/src/HLTrigger/Configuration/python/HLT_Fake2_cff.py", line 237, in <module>
    fragment = customizeHLTforCMSSW(fragment,"Fake2")
  File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02826/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_GPU_X_2024-02-26-2300/src/HLTrigger/Configuration/python/customizeHLTforCMSSW.py", line 262, in customizeHLTforCMSSW
    (alpaka & run3_common).makeProcessModifier(customizeHLTforAlpaka).apply(process)
  File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02826/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_GPU_X_2024-02-26-2300/src/FWCore/ParameterSet/python/Config.py", line 1980, in apply
    self.__func(process)
  File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02826/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_GPU_X_2024-02-26-2300/src/HLTrigger/Configuration/python/customizeHLTforAlpaka.py", line 917, in customizeHLTforAlpaka
    process = customizeHLTforAlpakaEcalLocalReco(process)
  File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02826/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_GPU_X_2024-02-26-2300/src/HLTrigger/Configuration/python/customizeHLTforAlpaka.py", line 908, in customizeHLTforAlpakaEcalLocalReco
    process.HLTDoFullUnpackingEgammaEcalTask = cms.ConditionalTask(process.HLTDoFullUnpackingEgammaEcalWithoutPreshowerTask, process.HLTPreshowerTask)
  File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02826/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_GPU_X_2024-02-26-2300/src/FWCore/ParameterSet/python/Config.py", line 1656, in __getattribute__
    return getattr(self.__process, name)
AttributeError: 'Process' object has no attribute 'HLTDoFullUnpackingEgammaEcalWithoutPreshowerTask'

this likely comes from the integration of https://github.com/cms-sw/cmssw/pull/44026 that moved @relval2023 to @Fake2.

About this issue

  • Original URL
  • State: closed
  • Created 4 months ago
  • Comments: 30 (30 by maintainers)

Most upvoted comments

one could inhibit the *FakeHLT steps for the Alpaka wfs.

this assumes that we are (correctly) running the FakeHLT RECO+DQM sequence in the workflows that run a Fake HLT menu, but this is not in general guaranteed nor enforced (even though we’ve been trying to be diligent with it). On the other hand since all the customization thing will get reabsorbed soon, I guess it’s an academic discussion. I would open a PR now with https://github.com/cms-sw/cmssw/issues/44119#issuecomment-1966489369 to get rid of failures for the next few weeks and be done with it.

+hlt

  • for the record

+heterogeneous

This error, yes, I think so.

assign hlt, heterogeneous