tt-metal: [Bug] Unsupported mode 'bilinear' for upsample2d op

Describe the bug During the runtime of the upsample2d ttnn ir with the mode bilinear faced the unsupported mode issue.

To Reproduce

  1. Create the mlir module
module @Interpolate attributes {tt.system_desc = #tt.system_desc<[{role = host, target_triple = "x86_64-pc-linux-gnu"}], [{arch = <wormhole_b0>, grid = 8x8, l1_size = 1499136, num_dram_channels = 12, dram_channel_size = 1073741824, noc_l1_address_align_bytes = 16, pcie_address_align_bytes = 32, noc_dram_address_align_bytes = 32, l1_unreserved_base = 1024, erisc_l1_unreserved_base = 1024, dram_unreserved_base = 1024, dram_unreserved_end = 1073741824, physical_cores = {worker = [ 0x0,  0x1,  0x2,  0x3,  0x4,  0x5,  0x6,  0x7,  1x0,  1x1,  1x2,  1x3,  1x4,  1x5,  1x6,  1x7,  2x0,  2x1,  2x2,  2x3,  2x4,  2x5,  2x6,  2x7,  3x0,  3x1,  3x2,  3x3,  3x4,  3x5,  3x6,  3x7,  4x0,  4x1,  4x2,  4x3,  4x4,  4x5,  4x6,  4x7,  5x0,  5x1,  5x2,  5x3,  5x4,  5x5,  5x6,  5x7,  6x0,  6x1,  6x2,  6x3,  6x4,  6x5,  6x6,  6x7,  7x0,  7x1,  7x2,  7x3,  7x4,  7x5,  7x6,  7x7] dram = [ 8x0,  9x0,  10x0,  8x1,  9x1,  10x1,  8x2,  9x2,  10x2,  8x3,  9x3,  10x3]}, supported_data_types = [<f32>, <f16>, <bf16>, <bfp_f8>, <bfp_bf8>, <bfp_f4>, <bfp_bf4>, <bfp_f2>, <bfp_bf2>, <u32>, <u16>, <u8>], supported_tile_sizes = [ 4x16,  16x16,  32x16,  4x32,  16x32,  32x32], num_cbs = 32}], [0], [3 : i32], [ 0x0x0x0]>} {
  func.func @forward(%arg0: tensor<1x2048x7x7xf32> {ttir.name = "input"}) -> (tensor<1x2048x14x14xf32> {ttir.name = "Interpolate.output_resize2d_0"}) {
    %0 = tensor.empty() : tensor<1x7x2048x7xf32>
    %1 = "ttir.transpose"(%arg0, %0) <{dim0 = -3 : si32, dim1 = -2 : si32}> : (tensor<1x2048x7x7xf32>, tensor<1x7x2048x7xf32>) -> tensor<1x7x2048x7xf32>
    %2 = tensor.empty() : tensor<1x7x7x2048xf32>
    %3 = "ttir.transpose"(%1, %2) <{dim0 = -2 : si32, dim1 = -1 : si32}> : (tensor<1x7x2048x7xf32>, tensor<1x7x7x2048xf32>) -> tensor<1x7x7x2048xf32>
    %4 = tensor.empty() : tensor<1x14x14x2048xf32>
    %5 = "ttir.upsample2d"(%3, %4) <{mode = "bilinear", scale_factor = 2 : si32}> {channel_last = 0 : si32} : (tensor<1x7x7x2048xf32>, tensor<1x14x14x2048xf32>) -> tensor<1x14x14x2048xf32>
    %6 = tensor.empty() : tensor<1x14x2048x14xf32>
    %7 = "ttir.transpose"(%5, %6) <{dim0 = -2 : si32, dim1 = -1 : si32}> : (tensor<1x14x14x2048xf32>, tensor<1x14x2048x14xf32>) -> tensor<1x14x2048x14xf32>
    %8 = tensor.empty() : tensor<1x2048x14x14xf32>
    %9 = "ttir.transpose"(%7, %8) <{dim0 = -3 : si32, dim1 = -2 : si32}> : (tensor<1x14x2048x14xf32>, tensor<1x2048x14x14xf32>) -> tensor<1x2048x14x14xf32>
    return %9 : tensor<1x2048x14x14xf32>
  }
}
  1. Follow the below commands
ttrt query --save-artifacts
./build/bin/ttmlir-opt --ttir-load-system-desc="path=./ttrt-artifacts/system_desc.ttsys" --ttir-to-ttnn-backend-pipeline ./upsample2d_mode_issue.mlir -o ttnn.mlir
./build/bin/ttmlir-translate --ttnn-to-flatbuffer ttnn.mlir -o out.ttnn
ttrt run out.ttnn

Model Log

upsample2d_mode_issue.log

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 17 (11 by maintainers)

Commits related to this issue

Most upvoted comments

Thank you, we will try to confirm that upsample is not the cause of the low PCC of the whole model on Forge-FE.

Synced offline with @azecevicTT and seems that the mentioned 0.96 PCC is on the whole model, not the upsample op. @mstaletovicTT can you check the second example with the sharding scheme provided?

First one is the same as the one that @mstaletovicTT wrote, for the 1x28x28x256xbf16; scaling_factor=2 one, it’s 28 cores 28x256 shard shape, height shraded.

TTmetal repro steps:

import torch
import torch.nn as nn
import ttnn
from tests.ttnn.utils_for_testing import assert_with_pcc

@pytest.mark.parametrize(
    "input_shapes",
    [
        [2, 1280, 4, 4],
    ],
)
@pytest.mark.parametrize("scale_h", [2])
@pytest.mark.parametrize("scale_w", [3])
def test_upsample_single_core_unsupported_mode(device, input_shapes, scale_h, scale_w):
    batch_size, height, width, num_channels = input_shapes

    torch.manual_seed(0)
    input = torch.rand(input_shapes, dtype=torch.bfloat16)
    tt_input = input.permute(0, 3, 1, 2)

    scale_factor = (scale_h, scale_w)
    m = nn.Upsample(scale_factor=scale_factor, mode="bilinear")
    torch_result = m(tt_input)
    torch_result = torch_result.permute(0, 2, 3, 1)

    ## ttnn uses NHWC, so need to set scale_factor_c = 1
    scale_factor = (scale_h, scale_w)
    input_tensor = ttnn.from_torch(input, device=device)
    output_tensor = ttnn.upsample(input_tensor, scale_factor,mode="bilinear")
    output_tensor = ttnn.to_torch(output_tensor)

    assert_with_pcc(torch_result, output_tensor)

 

Log:

upsample_singlecore_mode_issue (1).log

cc:@nvukobratTT