rocprofiler: [HSA_STATUS_ERROR] A generic error has occurred
Hi,
I am trying to profile some CEED benchmarks. I’m using a gfx906 card with rocm 2.10.
I’m using hipcc and the compilation/run seem fine, but I can’t get any output from the profiler.
I tried different options, but I keep getting this generic error:
rcprof -C ./bp3 -o 2 -l 8 -d hip
Options used:
--mesh-dimension 3
--refinement-level 8
--order 2
--device hip
Radeon Compute Profiler V5.6.7262 is enabled
No counter file specified. Only counters that will fit into a single pass will be enabled.
Device configuration: hip,cpu
Processor partitioning: 1 1 1
Mesh dimensions: 8 8 4
Total number of elements: 256
Number of finite element unknowns: 2601
aqlprofile API table load failed: HSA_STATUS_ERROR: A generic error has occurred.
[corona90:mpi_rank_0][error_sighandler] Caught error: Aborted (signal 6)
Failed to generate profile result /g/g91/camier1/Session1.csv.
Have you seen this kind of error?
Thank you for your help,
Jean-Sylvain
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Comments: 27
Commits related to this issue
- Merging From Github Squashed commit of the following: commit 7ab6644fd04db189801f6cee70a09bb621070b60 Author: Ammar ELWazir <aelwazir@amd.com> Date: Sat Jul 15 00:47:30 2023 -0500 Gerrit amd ... — committed to ROCm/rocprofiler by ammarwa 3 years ago
- Merge From Github 08-21-2023 Squashed commit of the following: commit 5aa25263e7000c323dc9efbedb11fe9d6bd3a97c Author: Ammar ELWazir <Ammar.ELWazir@amd.com> Date: Thu Aug 17 18:25:18 2023 +0000 ... — committed to ROCm/rocprofiler by ammarwa 3 years ago
- Adding Public Sync GH Action (#10) Adding Public Sync to ROCProfiler Co-authored-by: Ammar ELWazir <aelwazir@amd.com> — committed to ROCm/rocprofiler by ammarwa 2 years ago
In case anyone else is bumping against this error
aqlprofile API table load failed, installinghsa-amd-aqlprofilepackage seemed to solve it. This issue pops up when you search foraqlprofile API table load failed. I only realised you can install that package because calling https://github.com/ROCm-Developer-Tools/rocprofiler/blob/207458f251f223803dbbce64821dde15107a1781/test/util/hsa_rsrc_factory.cpp#L131 in debug spits out the library name isn’t found.Mine was failing because ctrl was falling back onto trying to load HSA_EXTENSION_AMD_AQLPROFILE and then failing on that. This is where the
HSA_STATUS_ERRORwas coming from which is being invoked from https://github.com/ROCm-Developer-Tools/rocprofiler/blob/207458f251f223803dbbce64821dde15107a1781/test/util/hsa_rsrc_factory.cpp#L133@camierjs:
roctracerfor RHEL + ROCm 2.10, this can be fixed via settingexport LD_LIBRARY_PATH=$(LD_LIBRARY_PATH):/opt/rocm/roctracer/libin your environment. This may resolve your tool lib error.hsa-amd-aqlprofile.x86_64package must be installed viasudo yum install hsa-amd-aqlprofile.x86_64, as this library isn’t open-sourced yet. On my system this lives in/opt/rocm/hsa-amd-aqlprofile/lib/libhsa-amd-aqlprofile64.so.1.0.0.Otherwise, if you don’t have sudo access and need to build from source, perhaps try building the
rocm-2.10.xbranch of roctracer since you’re using ROCm 2.10. The master branch has already pulled in many changes for ROCm 3.0. The software stack is moving fast.