tt-metal: preserve_fp32_precision flag is accepted as true for OPs for which it is not valid
Background:
The preserve_fp32_precision flag was introduced to enable full fp32 precision for SFPU-only OPs, meaning in cases that the data is unpacked directly into Dest using copy_tile(). It does so by setting the unpack_dst_format for CBs to fp32 instead of tf32, so that the unpacker puts fp32 datums from the CB directly into Dest unchanged (skipping SrcA/B regs which are limited to 19bits).
Issue:
The preserve_fp32_precision flag is not valid for all OPs, but there is no warning/error produced when it is used incorrectly. For example reduce_tile utilizes the FPU, which works on the srcA and srcB registers, and setting preserve_fp32_precision=True will set the data type of srcA and srcB to fp32 instead of tf32 and lead to wrong results.
Also, the question was raised whether or not the flag is allowed in cases where a compute kernel may use one LLK for which the flag is supported but a second one for which it is not. One such example could be reloading fp32 partials into the Dest (flag supported) in order to perform further accumulation using matmul (flag not supported). In these cases, since the flag works by changing the unpack_dst_format for the buffer, if the buffer is only used for unpacking to Dest then the flag should be allowed. If the buffer is used to unpack to Dest but also unpack into SrcA/B, then the flag is not allowed.
To Reproduce: This was detected in this bug https://github.com/tenstorrent/tt-metal/issues/10337 and this example can be used.
Expected behavior: We need a mechanism to disallow this flag to be set to true for OPs for which it is not valid, and better documentation surrounding this flag.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 18 (18 by maintainers)
Commits related to this issue
- #11756: Change preserve_fp32_precision bool flag to vector — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #11756: update new preserve_fp32_precision flags to vectors — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #11756: Change preserve_fp32_precision bool flag to vector — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #11756: update new preserve_fp32_precision flags to vectors — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #11756: Change preserve_fp32_precision bool flag to vector — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #11756: Additional work — committed to tenstorrent/tt-metal by dongjin-na 2 years ago
- #11756: Apply preserve_fp32_precision vector to moreh_matmul — committed to tenstorrent/tt-metal by dongjin-na 2 years ago
- #11756: Change preserve_fp32_precision bool flag to vector — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #11756: update new preserve_fp32_precision flags to vectors — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #11756: update preserve_fp32_precision vector for moreh matmul — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #11756: update condition for fp32 precision flag — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #11756: Change preserve_fp32_precision bool flag to vector — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #11756: update new preserve_fp32_precision flags to vectors — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #11756: update preserve_fp32_precision vector for moreh matmul — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #11756: update condition for fp32 precision flag — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #11756: rename PreserveFP32Target to UnpackToDestMode for clarity — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #11756: Change preserve_fp32_precision bool flag to vector — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #11756: update new preserve_fp32_precision flags to vectors — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #11756: update preserve_fp32_precision vector for moreh matmul — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
- #11756: update condition for fp32 precision flag — committed to tenstorrent/tt-metal by rdjogoTT 2 years ago
@rdjogoTT Seems like all work as intended. Salute to the nice work! I think you can close this issue after the PR is merged to the main branch.
@razorback3 @dongjin-na can I get a review for the PR: https://github.com/tenstorrent/tt-metal/pull/12794
Nice work! @dongjin-na will check the functionality on the given branch and give feedback soon 😀.
add_tiles also uses FPU and is not compatible with the flag, but if you had some other function for which preserve_fp32_precision is valid, that one should work fine, at least that is what we aim for.
What does the above sentence mean among the two options below?
preserve_fp32feature.add_tilesandreduce_tileare present,preserve_fp32is disabled only forreduce_tile, which doesn’t support it, whileadd_tilesremains unaffected.