tt-metal: [Bug Report] conv2d failed with `bias_ntiles == weight_matrix_width_ntiles`

Describe the bug conv2d failed with bias_ntiles == weight_matrix_width_ntiles info for certain shapes

To Reproduce

Run the example below

import torch
import ttnn

from tests.ttnn.utils_for_testing import assert_with_pcc


def main(device):
    input_shape, weight_shape, stride, padding, dilation, transposed, groups = (
        [1, 1024, 45, 80],
        [2048, 1024, 1, 1],
        [2, 2],
        [0, 0],
        [1, 1],
        False,
        1,
    )
    n, in_c, in_h, in_w = input_shape
    out_c, _, k_h, k_w = weight_shape

    n, in_c, in_h, in_w = input_shape
    out_c0, out_c1, k_h, k_w = weight_shape
    out_c = out_c1 if transposed else out_c0
    torch_input = torch.rand(input_shape, dtype=torch.bfloat16)
    torch_weight = torch.rand(weight_shape, dtype=torch.bfloat16)
    torch_bias = torch.rand((out_c,), dtype=torch.bfloat16)
    input_tensor = ttnn.from_torch(torch_input.permute((0, 2, 3, 1)), layout=ttnn.TILE_LAYOUT, device=device)
    weight_tensor = ttnn.from_torch(torch_weight, layout=ttnn.ROW_MAJOR_LAYOUT)
    bias_tensor = ttnn.from_torch(torch_bias.reshape(1, 1, 1, -1), layout=ttnn.ROW_MAJOR_LAYOUT)

    expected_output = torch.convolution(
        torch_input, torch_weight, torch_bias, stride, padding, dilation, transposed, (0, 0), groups
    )
    expected_output = expected_output.permute((0, 2, 3, 1)).reshape(1, 1, -1, out_c)

    output_tensor = ttnn.conv2d(
        input_tensor=input_tensor,
        weight_tensor=weight_tensor,
        bias_tensor=bias_tensor,
        batch_size=n,
        in_channels=in_c,
        out_channels=out_c,
        input_height=in_h,
        input_width=in_w,
        kernel_size=(k_h, k_w),
        stride=stride,
        padding=padding,
        dilation=dilation,
        groups=groups,
        device=device,
    )

    assert_with_pcc(expected_output, ttnn.to_torch(output_tensor), 0.99)


if __name__ == "__main__":
    try:
        device = ttnn.open_device(device_id=0, l1_small_size=16384)
        main(device)
    finally:
        ttnn.close_device(device)

Please complete the following environment information:

  • OS: Ubuntu 20.04
  • Version of software a8a812beba9230df1196fe9ac11bb7769f5f1d90
  • Card: N300

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 15 (15 by maintainers)

Most upvoted comments

@nvukobratTT @mstojkovicTT Ok in short if you pass in weights tensor on device conv assumes that it’s prepared, so this is not the correct usage. Try some of the options above, from my point of view we should close this one.

@sankarmanoj-tt so basically you are saying that they are calling conv with unprepared weights, but with weights tensor on device right? So they have options:

  1. Call prepare weights
  2. Pass in tensors on host
  3. Pass in always_preprocess_weights in Conv2dConfig

Is my assessment correct?

The conv op applies some preprocessing on the weights tensor before using it. It determines if the weights have been preprocessed by looking at the storage type. By manually placing the weights/bias on device, the op assumes the weights have already been preprocessed.

This code can be fixed by keeping the weights / bias tensors on host.

    weights_ttnn = ttnn.from_torch(
        weights_torch
    )
    
    bias_tensor = input_torch = torch.randn((1, 1, 1, 720), dtype=torch.float32)
    bias_ttnn = ttnn.from_torch(
        bias_tensor
    )

@nvukobratTT I’m counting on you guys to provide the repro

@sankarmanoj-tt pls try to check what is going on here

@nvukobratTT pls add repros.