tt-metal: ttnn.reshape fails for float32 tensors

Necessary conditions for repro:

  1. Tensor is not in bfloat16 or uint32 data format
  2. Tensor is in TILE_LAYOUT
  3. Reshape of the tensor goes through convert_tensor_to_rm_reshape_convert_back_to_orig_layout()

In this case, the following TT_FATAL gets triggered:

TT_FATAL @ /localdev/pilkic/tt-metal/ttnn/cpp/ttnn/operations/data_movement/tilize_with_val_padding/device/tilize_with_val_padding_op.cpp:19: input_tensor_a.get_dtype() == DataType::BFLOAT16 or input_tensor_a.get_dtype() == DataType::UINT32
info:
Can only tilize bfloat16 or uint32 tensors

The weird thing is that the tensors of the same shape (and data format) can be created via ttnn.from_torch() in tile layout.

We’ve hit this on tt-forge-fe on newer version of tt-metal, when the above assertion was changed from TT_ASSERT to TT_FATAL. Previously, the assertion was not being check (since we run tests with non-debug builds of tt-metal) and the tests were passing.

Related to issue #14570

Repro:

with ttnn.manage_device(device_id=0) as device:
    # Creating tensors from torch tensors of the same shape
    # (as input to reshape and output of the reshape) works as expected
    in_a = torch.randn(1, 1, 756, 128)
    in_a = ttnn.from_torch(in_a, dtype=ttnn.float32, layout=ttnn.TILE_LAYOUT, device=device, memory_config=ttnn.DRAM_MEMORY_CONFIG)
    print(in_a.shape)

    desired_shape = [1, 27, 28, 128]
    in_b = torch.randn(*desired_shape)
    in_b = ttnn.from_torch(in_b, dtype=ttnn.float32, layout=ttnn.TILE_LAYOUT, device=device, memory_config=ttnn.DRAM_MEMORY_CONFIG)
    print(in_b.shape)

    # Reshape the original input tensor to the desired shape
    # - fails with `Can only tilize bfloat16 or uint32 tensors`
    output_device = ttnn.reshape(in_a, desired_shape)
    output_tensor = ttnn.to_torch(output_device)
    print(output_tensor.shape)

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 15 (10 by maintainers)

Commits related to this issue

Most upvoted comments

Yup, I’ll let you know when it’s done!