tt-metal: Pack Untilize gives bad PCC when block_ct_dim < full_ct_dim

Describe the bug for Conv unit test, block_ct_dim=1, full_ct_dim = 2, pack_untilize gives garbage PCC, when set block_ct_dim= full_ct_dim=2, PCC is good.

To Reproduce Steps to reproduce the behavior:

  1. Go to yugao/pack_untilize
  2. Run pytest tests/ttnn/unit_tests/operations/test_new_conv2d.py::test_unet_conv_wh
  3. To get rid of the PCC error, comment out the hard-coded out_subblock_h_ntiles = 1; out_subblock_w_ntiles = 1; in ttnn/cpp/ttnn/operations/conv/conv2d/conv2d.cpp

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 20 (18 by maintainers)

Commits related to this issue

Most upvoted comments

Resolved it with @avoraTT offline, the issue was improper usage of pack_untilize_block - no LLK changes necessary.

Hey @yugaoTT, I created a PR for the LLK changes on WH that enable the block_ct_dim != full_ct_dim case to work. Could you please take a look at my branch on tt_metal to see the necessary op changes for the algorithm to work? Here is the test that I used for it:

import pytest
import torch

import ttnn

from loguru import logger
from tests.ttnn.utils_for_testing import assert_with_pcc

@pytest.mark.parametrize("n", [1])
@pytest.mark.parametrize("c", [1])
@pytest.mark.parametrize("h", [32])
@pytest.mark.parametrize("w", [512])
def test_untilize(device, n, c, h, w):
    torch.manual_seed(2005)

    # Generate a range of numbers from 1 to n*c*h*w with higher precision
    total_elements = n * c * h * w
    torch_input_tensor = torch.arange(1, total_elements + 1, dtype=torch.int32).reshape((n, c, h, w))
    
    # Convert to TTNN tensor
    tt_input_tensor = ttnn.from_torch(
        torch_input_tensor,
        dtype=ttnn.DataType.INT32,
        layout=ttnn.TILE_LAYOUT,
        device=device,
        memory_config=ttnn.L1_MEMORY_CONFIG,
    )

    # Change layout and convert back to Torch tensor
    tt_output_tensor = ttnn.to_layout(tt_input_tensor, ttnn.ROW_MAJOR_LAYOUT)
    tt_output_tensor = ttnn.to_torch(tt_output_tensor)

    assert_with_pcc(torch_input_tensor, tt_output_tensor, 0.9999)

Hi @yugaoTT, there is an open bug that full_ct_dim != block_ct_dim does not work currently in the LLKs, there is a mention of it here: https://github.com/tenstorrent/tt-metal/issues/11716

If this issue becomes higher priority then the LLK team can work on fixing this issue for pack_untilize, please let us know.