tt-metal: [Bug Report] CPU and NPU produce different BFP8_B values from the same BF16 tensor.

Describe the bug The CPU and NPU produce different BFP8_B values from the same BF16 tensor.

To Reproduce Steps to reproduce the behavior:

  1. go to branch namhyeong/.... Link to branch
  2. run tests/ttnn/unit_tests/test_bfp8_bf16_conversion.py Link to test file
    1. test_typecast_bf16_to_bfp8_b fails, while test_typecast_bfp8_b_to_bf16 passes.

Expected behavior The CPU and NPU should produce the same BFP8_B values.

Screenshots 스크린샷 2024-10-21 오후 6 28 49

Please complete the following environment information:

  • OS: [Ubuntu 20.04]

Additional context No additional context

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 24 (21 by maintainers)

Commits related to this issue

Most upvoted comments

Yes, the CPU is producing the more accurate values in this case. Ok I will look into how that can be done.

@rdjogoTT Please advise when this is fully complete so we can close out this issue, thanks.

I understood the limitation. I will check before conversion to bfp8_b as your recommendation if nan should be handled. Lastly please double check the tests that I provided have enough coverage.

I pushed the tests to my branch that is rebased on your branch. Every test in test_typecast_bf16_to_bfp8_b_various_input passes. But when insert_nan is true in test_typecast_bf16_to_bfp8_b_with_inf_nan, the test fails. cpu makes nan but npu makes -inf.

idx=(tensor(0), tensor(1)) cpu=nan npu=-inf
idx=(tensor(0), tensor(4)) cpu=nan npu=-inf
idx=(tensor(0), tensor(18)) cpu=nan npu=-inf
idx=(tensor(0), tensor(29)) cpu=nan npu=-inf
idx=(tensor(0), tensor(35)) cpu=nan npu=-inf

Could you check these failed tests?

Sorry I should have clarified - typecast works by unpacking the fp16b to Dest and using the packer to convert to bfp8b. Similar with fp32 to bfp8b typecast. This bug was caused by that packer conversion, so this feature should enable accurate conversion now.

I’ve managed to trace the data through the HW path it takes, and it looks like the issue occurs in the Packer. I will be investigating this further

I have started investigating and should have an update by tomorrow.