tt-metal: [Bug Report] `fp32_dest_acc_en=true` makes FPU operation slower
Describe the bug
Addition of bfloat16 tiles using FPU become slower when fp32_dest_acc_en=true.
To Reproduce The reader and writer kernels are empty and the compute kernel is as follows:
namespace NAMESPACE {
void MAIN {
constexpr uint32_t num_tiles = get_compile_time_arg_val(0);
constexpr uint32_t cb_input = tt::CB::c_in0;
constexpr uint32_t cb_other = tt::CB::c_in1;
constexpr uint32_t cb_output = tt::CB::c_out0;
binary_op_init_common(cb_input, cb_other, cb_output);
// push dummy tile in each CB
cb_reserve_back(cb_input, onetile);
cb_reserve_back(cb_other, onetile);
cb_push_back(cb_input, onetile);
cb_push_back(cb_other, onetile);
cb_wait_front(cb_input, onetile);
cb_wait_front(cb_other, onetile);
cb_reserve_back(cb_output, onetile);
for (uint32_t i = 0; i < num_tiles; i++) {
tile_regs_acquire();
binary_op_specific_init<false, EltwiseBinaryType::ELWSADD>();
add_tiles(cb_input, cb_other, 0, 0, 0);
tile_regs_commit();
tile_regs_wait();
pack_tile(0, cb_output);
tile_regs_release();
}
cb_push_back(cb_output, onetile);
cb_pop_front(cb_input, onetile);
cb_pop_front(cb_other, onetile);
}
} // namespace NAMESPACE
The reason I used empty reader and writer and pushed dummy tiles in CBs is to remove data transfer time in NOC.
Under this configuration and release build with tracy enabled, the number of cycles required to add 100 tiles (num_tiles=100 in the compute kernel above) is:
fp32_dest_acc_en |
false | true |
|---|---|---|
| device_fw_duration | 10273 | 14903 |
| device_kernel_duration | 9684 | 14397 |
| device_brisc_kernel_duration | 20 | 20 |
| device_ncrisc_kernel_duration | 62 | 62 |
| device_trisc0_kernel_duration | 9103 | 13401 |
| device_trisc1_kernel_duration | 9591 | 14039 |
| device_trisc2_kernel_duration | 9278 | 13981 |
| device_compute_cb_wait_front | 0 | 0 |
| device_compute_cb_reserve_back | 0 | 0 |
As seen, fp32_dest_acc_en makes significant difference on the number of cycles.
Expected behavior
fp32_dest_acc_en does not make big difference to speed.
Please complete the following environment information:
- commit dee1670
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 15 (11 by maintainers)
Commits related to this issue
- #13532: Add test case — committed to tenstorrent/tt-metal by deleted user 2 years ago
- #13532 : Remove inlining for packer functions — committed to tenstorrent/tt-metal by amahmudTT 2 years ago
- #13532: Remove inlining for more functions — committed to tenstorrent/tt-metal by deleted user 2 years ago
- #13532: Add FPU speed test — committed to tenstorrent/tt-metal by deleted user 2 years ago
Great 👍 Thanks for checking.
Unfortunately, this issue is not reproducible in the latest tt-metal (both in the branch here and my test branch in my private fork) so I will be closing this issue. If it happens again, I will reopen with new test branch.
I published my test branch
jeongu-moreh/fpu-speed-test. You can try the test operation with the following code:I will clean up my old test branch (in my private fork) and publish it after rebasing on main here.
It’s wormhole_b0.