tt-metal: [Bug Report] `fp32_dest_acc_en=true` makes FPU operation slower

Describe the bug Addition of bfloat16 tiles using FPU become slower when fp32_dest_acc_en=true.

To Reproduce The reader and writer kernels are empty and the compute kernel is as follows:

namespace NAMESPACE {
void MAIN {
    constexpr uint32_t num_tiles = get_compile_time_arg_val(0);

    constexpr uint32_t cb_input = tt::CB::c_in0;
    constexpr uint32_t cb_other = tt::CB::c_in1;
    constexpr uint32_t cb_output = tt::CB::c_out0;

    binary_op_init_common(cb_input, cb_other, cb_output);

    // push dummy tile in each CB
    cb_reserve_back(cb_input, onetile);
    cb_reserve_back(cb_other, onetile);
    cb_push_back(cb_input, onetile);
    cb_push_back(cb_other, onetile);
    cb_wait_front(cb_input, onetile);
    cb_wait_front(cb_other, onetile);
    cb_reserve_back(cb_output, onetile);

    for (uint32_t i = 0; i < num_tiles; i++) {
        tile_regs_acquire();
        binary_op_specific_init<false, EltwiseBinaryType::ELWSADD>();
        add_tiles(cb_input, cb_other, 0, 0, 0);
        tile_regs_commit();

        tile_regs_wait();
        pack_tile(0, cb_output);
        tile_regs_release();
    }

    cb_push_back(cb_output, onetile);
    cb_pop_front(cb_input, onetile);
    cb_pop_front(cb_other, onetile);
}
}  // namespace NAMESPACE

The reason I used empty reader and writer and pushed dummy tiles in CBs is to remove data transfer time in NOC.

Under this configuration and release build with tracy enabled, the number of cycles required to add 100 tiles (num_tiles=100 in the compute kernel above) is:

fp32_dest_acc_en false true
device_fw_duration 10273 14903
device_kernel_duration 9684 14397
device_brisc_kernel_duration 20 20
device_ncrisc_kernel_duration 62 62
device_trisc0_kernel_duration 9103 13401
device_trisc1_kernel_duration 9591 14039
device_trisc2_kernel_duration 9278 13981
device_compute_cb_wait_front 0 0
device_compute_cb_reserve_back 0 0

As seen, fp32_dest_acc_en makes significant difference on the number of cycles.

Expected behavior fp32_dest_acc_en does not make big difference to speed.

Please complete the following environment information:

  • commit dee1670

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 15 (11 by maintainers)

Commits related to this issue

Most upvoted comments

Great 👍 Thanks for checking.

Unfortunately, this issue is not reproducible in the latest tt-metal (both in the branch here and my test branch in my private fork) so I will be closing this issue. If it happens again, I will reopen with new test branch.

I published my test branch jeongu-moreh/fpu-speed-test. You can try the test operation with the following code:

import torch
import ttnn

device_id = 0
device = ttnn.CreateDevice(device_id=device_id)
ttnn.SetDefaultDevice(device)

dummy_torch = torch.rand(32, 32, dtype=torch.bfloat16)
dummy = ttnn.from_torch(dummy_torch, dtype=ttnn.bfloat16, layout=ttnn.TILE_LAYOUT, device=device)

ttnn._ttnn.operations.fpu_speed_test.fpu_speed_test(100, False, dummy=dummy)

ttnn.DumpDeviceProfiler(device)
ttnn.synchronize_device(device)
ttnn.close_device(device)

I will clean up my old test branch (in my private fork) and publish it after rebasing on main here.

It’s wormhole_b0.