tt-metal: Reshape -> Transpose gives bad PCC

Describe the bug

    tt_input = tt_input.reshape(1, 2048, 4, 128)
    tt_output= ttnn.transpose(tt_input, 1, 2)

gives 0.0 PCC compared to Torch:

    torch_ref = torch_input.view(1, 2048, 4, 128)
    torch_ref = torch_ref.transpose(1, 2)

To Reproduce

import torch
import ttnn
from tests.ttnn.utils_for_testing import assert_with_pcc

def test_transpose_with_reshape(device):
    # Create input tensor
    torch_input = torch.rand((1, 1, 2048, 512), dtype=torch.bfloat16)
    
    # TT operations
    tt_input = ttnn.from_torch(
        torch_input,
        dtype=ttnn.DataType.BFLOAT16,
        layout=ttnn.ROW_MAJOR_LAYOUT,
        device=device,
        memory_config=ttnn.L1_MEMORY_CONFIG,
    )
    tt_input = tt_input.reshape(1, 2048, 4, 128)
    tt_output= ttnn.transpose(tt_input, 1, 2)
    
    # Convert back to PyTorch for comparison
    tt_result = ttnn.to_torch(tt_output)
    
    # PyTorch reference operations
    torch_ref = torch_input.view(1, 2048, 4, 128)
    torch_ref = torch_ref.transpose(1, 2)
    
    # Compare results
    assert_with_pcc(torch_ref, tt_result, 0.9999)

Expected behavior Expected this to match torch behaviour.

Please complete the following environment information: Internal IRD-supplied N150

Additional context Seen bringing up Llama 3.2

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 23 (22 by maintainers)

Commits related to this issue

Most upvoted comments

@nardoTT is on it! She is going to make sure the <tensor>.reshape(new_shape) and ttnn.reshape(<tensor>, new_shape) behave the same way, we will also introduce an unsafe view for cases that need it (off the top of my head I know slice needs the unsafe version but there could be more that show up).

She is bright but she is new so I would give her a couple weeks to ramp up and get this in as this is part of her ramp up task.

It sounds like tensor.reshape is doing a view without validating its inputs. I think a solution would be to change its name to something that indicates that it’s an internal unsafe view op, or add input validation and call it view. What do you think?

Also someone can check this, but in Pytorch doesn’t tensor.op(...) and torch.op(tensor, ...) usually have the same behavior?

this might be a more involved change. A lot of API uses this reshape because it abuses the data format underneath the hood to incorporate padding, which is inappropriately used by developers everywhere. We can see what needs to be done, but it might be a lot of work for little reward.

I tested on my branch sjameel/transpose_pad and it worked with ttnn.reshape I will aim to get that into main soon.