tt-metal: ttnn.maximum unsupported broadcast

Maximum op doesn’t work in cases where the second input needs to be broadcast. The following exception is thrown:

RuntimeError: TT_THROW @ /localdev/pilkic/tt-metal/ttnn/cpp/ttnn/operations/eltwise/binary/device/binary_device_operation.cpp:55: tt::exception
info:
ttnn::operations::binary::BinaryDeviceOperation: unsupported broadcast

It seems to me that other binary eltwise ops are suffering from this as well.

The problem seems to be that the broadcasting mechanism relies that the input that needs to be broadcast is the second input; and in cases when the operation is commutative it will swap the inputs so that the one that needs broadcasting is the second one (this is done during preprocess_inputs(). However, this will not work for non-commutative ops.

The broadcasting logic should work regardless of whether an operation is commutative or not.

Snippet from preprocess_inputs():

    return [](const auto &input_tensor_a, const auto &input_tensor_b) {
        if constexpr (detail::is_associative(binary_op_type)) {
            // Swap tensors if input_tensor_a needs to be broadcasted to input_tensor_b
            if (input_tensor_a.get_logical_volume() < input_tensor_b.get_logical_volume()) {
                return std::make_tuple(input_tensor_b, input_tensor_a);
            }
        }
        return std::make_tuple(input_tensor_a, input_tensor_b);
    }(input_tensor_a, input_tensor_b);

Additionally, the is_associative() (I would assume this should be is_commutative) check seems incomplete.

Related to existing issue: #13566

Repro:

import torch
import ttnn

with ttnn.manage_device(0) as device:
    input0 = torch.randn(1, 12, 256, 256)
    input1 = torch.randn(1)

    input0 = ttnn.from_torch(input0, device=device, dtype=ttnn.float32, layout=ttnn.TILE_LAYOUT, memory_config=ttnn.DRAM_MEMORY_CONFIG)
    input1 = ttnn.from_torch(input1, device=device, dtype=ttnn.float32, layout=ttnn.TILE_LAYOUT, memory_config=ttnn.DRAM_MEMORY_CONFIG)

    # Add works as expected
    output = ttnn.add(input0, input1)
    output = ttnn.add(input1, input0)
    print("done add")

    # This one passes:
    output = ttnn.subtract(input0, input1)
    # This one fails:
    output = ttnn.subtract(input1, input0)
    print("done subtract")

    # This one fails:
    output = ttnn.maximum(input0, input1)
    # This one passes:
    output = ttnn.maximum(input1, input0)
    print("done maximum")

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 20 (6 by maintainers)

Commits related to this issue

Most upvoted comments

https://github.com/tenstorrent/tt-metal/pull/20390 - Merged Issue fixed with the use of flag use_legacy=False

Hi @cmaryanTT @pilkicTT @staylorTT Broadcasting support for maximum is now provided in PR https://github.com/tenstorrent/tt-metal/pull/20390 With the use of flag use_legacy=False, test now passes

Image

This will be addressed by #16147 - follow that issue for updates.

This should be on-by-default within the next 48 hrs

Solving the broadcasting issue generally is top priority for us. @yan-zaretskiy is actively working this. Related issue is also #13643.

The eta on correctly implementing this feature is November 22. Downgrading to P1. Please let me know if a temporary solution is needed before then.