tt-metal: [Bug Report] CPU and NPU produce different BFP8_B values from the same BF16 tensor.
Describe the bug The CPU and NPU produce different BFP8_B values from the same BF16 tensor.
To Reproduce Steps to reproduce the behavior:
- go to branch
namhyeong/....Link to branch - run
tests/ttnn/unit_tests/test_bfp8_bf16_conversion.pyLink to test filetest_typecast_bf16_to_bfp8_bfails, whiletest_typecast_bfp8_b_to_bf16passes.
Expected behavior The CPU and NPU should produce the same BFP8_B values.
Screenshots
Please complete the following environment information:
- OS: [Ubuntu 20.04]
Additional context No additional context
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 24 (21 by maintainers)
Commits related to this issue
- #14032: change bfp8 pack src format to fp16 — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #14032: change bfp8 pack src format to fp16 — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #14032: add bfp_pack_precise flag to compute config — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #14032: change bfp8 pack src format to fp16 — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #14032: add bfp_pack_precise flag to compute config and tests — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #14032: move new tests with typecast test — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #14032: change bfp8 pack src format to fp16 — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #14032: add bfp_pack_precise flag to compute config and tests — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #14032: move new tests with typecast test — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #14032: set bfp8_pack_precise based on op and dtype — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #14032: change bfp8 pack src format to fp16 — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #14032: add bfp_pack_precise flag to compute config and tests — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #14032: move new tests with typecast test — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #14032: set bfp8_pack_precise based on op and dtype — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #14032: init bfp8_pack_precise to false — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #14032: change bfp8 pack src format to fp16 — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #14032: add bfp_pack_precise flag to compute config and tests — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #14032: move new tests with typecast test — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #14032: set bfp8_pack_precise based on op and dtype — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #14032: init bfp8_pack_precise to false — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
Yes, the CPU is producing the more accurate values in this case. Ok I will look into how that can be done.
@rdjogoTT Please advise when this is fully complete so we can close out this issue, thanks.
I understood the limitation. I will check before conversion to bfp8_b as your recommendation if
nanshould be handled. Lastly please double check the tests that I provided have enough coverage.I pushed the tests to my branch that is rebased on your branch. Every test in
test_typecast_bf16_to_bfp8_b_various_inputpasses. But wheninsert_nanis true intest_typecast_bf16_to_bfp8_b_with_inf_nan, the test fails. cpu makesnanbut npu makes-inf.Could you check these failed tests?
Sorry I should have clarified - typecast works by unpacking the fp16b to Dest and using the packer to convert to bfp8b. Similar with fp32 to bfp8b typecast. This bug was caused by that packer conversion, so this feature should enable accurate conversion now.
I’ve managed to trace the data through the HW path it takes, and it looks like the issue occurs in the Packer. I will be investigating this further
I have started investigating and should have an update by tomorrow.