tt-metal: ttnn.reshape fails for float32 tensors
Necessary conditions for repro:
- Tensor is not in bfloat16 or uint32 data format
- Tensor is in
TILE_LAYOUT - Reshape of the tensor goes through
convert_tensor_to_rm_reshape_convert_back_to_orig_layout()
In this case, the following TT_FATAL gets triggered:
TT_FATAL @ /localdev/pilkic/tt-metal/ttnn/cpp/ttnn/operations/data_movement/tilize_with_val_padding/device/tilize_with_val_padding_op.cpp:19: input_tensor_a.get_dtype() == DataType::BFLOAT16 or input_tensor_a.get_dtype() == DataType::UINT32
info:
Can only tilize bfloat16 or uint32 tensors
The weird thing is that the tensors of the same shape (and data format) can be created via ttnn.from_torch() in tile layout.
We’ve hit this on tt-forge-fe on newer version of tt-metal, when the above assertion was changed from TT_ASSERT to TT_FATAL. Previously, the assertion was not being check (since we run tests with non-debug builds of tt-metal) and the tests were passing.
Related to issue #14570
Repro:
with ttnn.manage_device(device_id=0) as device:
# Creating tensors from torch tensors of the same shape
# (as input to reshape and output of the reshape) works as expected
in_a = torch.randn(1, 1, 756, 128)
in_a = ttnn.from_torch(in_a, dtype=ttnn.float32, layout=ttnn.TILE_LAYOUT, device=device, memory_config=ttnn.DRAM_MEMORY_CONFIG)
print(in_a.shape)
desired_shape = [1, 27, 28, 128]
in_b = torch.randn(*desired_shape)
in_b = ttnn.from_torch(in_b, dtype=ttnn.float32, layout=ttnn.TILE_LAYOUT, device=device, memory_config=ttnn.DRAM_MEMORY_CONFIG)
print(in_b.shape)
# Reshape the original input tensor to the desired shape
# - fails with `Can only tilize bfloat16 or uint32 tensors`
output_device = ttnn.reshape(in_a, desired_shape)
output_tensor = ttnn.to_torch(output_device)
print(output_tensor.shape)
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 15 (10 by maintainers)
Commits related to this issue
- float32 tilize support (#14963) ### Ticket [#14885](https://github.com/tenstorrent/tt-metal/issues/14885) [#14570](https://github.com/tenstorrent/tt-metal/issues/14570) ### Problem description ... — committed to tenstorrent/tt-metal by yugi957 2 years ago
- #14885: tilize_op float32 access — committed to tenstorrent/tt-metal by yugi957 2 years ago
- tilize_op float32 access (#15115) ### Ticket [Link to Github Issue](https://github.com/tenstorrent/tt-metal/issues/14885#issuecomment-2479299202) ### Problem description I missed one entry poin... — committed to tenstorrent/tt-metal by yugi957 2 years ago
- tilize_op float32 access (#15115) ### Ticket [Link to Github Issue](https://github.com/tenstorrent/tt-metal/issues/14885#issuecomment-2479299202) ### Problem description I missed one entry point for... — committed to tenstorrent/tt-metal by yugi957 2 years ago
- tilize_op float32 access (#15115) ### Ticket [Link to Github Issue](https://github.com/tenstorrent/tt-metal/issues/14885#issuecomment-2479299202) ### Problem description I missed one entry poin... — committed to tenstorrent/tt-metal by yugi957 2 years ago
- float32 tilize support (#14963) ### Ticket [#14885](https://github.com/tenstorrent/tt-metal/issues/14885) [#14570](https://github.com/tenstorrent/tt-metal/issues/14570) ### Problem description ... — committed to tenstorrent/tt-metal by yugi957 2 years ago
Yup, I’ll let you know when it’s done!