tt-mlir: TTRT Perf Fails - bias_ntiles == weight_matrix_width_ntiles
TTRT perf fails on YOLOv10 performnce benchmark tests. Below I give job, error message and TTIR file.
TT-Forge-Fe job: https://github.com/tenstorrent/tt-forge-fe/actions/runs/15991192708/job/45105553059
Error message:
toolchain/venv/lib/python3.10/site-packages/ttrt/runtime/generated/profiler/reports/ops_perf_results.csv
2025-07-01 07:34:10,691 - ERROR - ERROR: test=out.ttnn experienced an error with exception=TT_FATAL @ /__w/tt-mlir/tt-mlir/third_party/tt-metal/src/tt-metal/ttnn/cpp/ttnn/operations/conv/conv2d/device/conv2d_op_sharded_program_factory.cpp:527: bias_ntiles == weight_matrix_width_ntiles
info:
Error
backtrace:
--- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/ttrt/runtime/libTTMLIRRuntime.so(+0x1ea098) [0x7fc61323d098]
--- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/ttrt/runtime/_ttnncpp.so(_ZN4ttnn10operations4conv6conv2d41multi_core_optimized_conv_sharded_v2_implERN2tt8tt_metal7ProgramERKNS4_6TensorES9_RKNS4_5ShapeESt8optionalIS8_ERKNS0_14sliding_window19SlidingWindowConfigERKNSF_14ParallelConfigERKSt6vectorIjSaIjEERKSM_INSF_13ShardBoundaryESaISR_EEjjbbRKSD_INS0_5unary14UnaryWithParamEERKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigEbRS7_St7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEbbbb+0x7986) [0x7fc6118f6546]
--- /opt/ttforge-toolchain/venv/lib/python3.10/site-packages/ttrt/runtime/_ttnncpp.so(_ZN4ttnn10operations4conv6conv2d40multi_core_optimized_conv_sharded_v2_newERKN2tt8tt_metal6TensorES7_RKSt8optionalIS6_ERKNS0_14sliding_window19SlidingWindowConfigEjjbRKS8_INS0_5unary14UnaryWithParamEERKNS2_34OptimizedConvParallelizationConfigERKNS2_24OptimizedConvBlockConfigENS4_8DataTypeESt5arrayIjLm4EES8_IKSt7variantIJNS_28GrayskullComputeKernelConfigENS_27WormholeComputeKernelConfigEEEERS5_bbbb+0x31a) [0x7fc6118ee50a]
...
Note: TTIR file is given as txt, because github doesn’t support .mlir file as attachment. Just remove the extension .txt, and it will be fine.
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 19 (12 by maintainers)
@jgrimTT CI perf machines, n150, bare-metal.