tt-metal: ttnn.maximum unsupported broadcast
Maximum op doesn’t work in cases where the second input needs to be broadcast. The following exception is thrown:
RuntimeError: TT_THROW @ /localdev/pilkic/tt-metal/ttnn/cpp/ttnn/operations/eltwise/binary/device/binary_device_operation.cpp:55: tt::exception
info:
ttnn::operations::binary::BinaryDeviceOperation: unsupported broadcast
It seems to me that other binary eltwise ops are suffering from this as well.
The problem seems to be that the broadcasting mechanism relies that the input that needs to be broadcast is the second input; and in cases when the operation is commutative it will swap the inputs so that the one that needs broadcasting is the second one (this is done during preprocess_inputs(). However, this will not work for non-commutative ops.
The broadcasting logic should work regardless of whether an operation is commutative or not.
Snippet from preprocess_inputs():
return [](const auto &input_tensor_a, const auto &input_tensor_b) {
if constexpr (detail::is_associative(binary_op_type)) {
// Swap tensors if input_tensor_a needs to be broadcasted to input_tensor_b
if (input_tensor_a.get_logical_volume() < input_tensor_b.get_logical_volume()) {
return std::make_tuple(input_tensor_b, input_tensor_a);
}
}
return std::make_tuple(input_tensor_a, input_tensor_b);
}(input_tensor_a, input_tensor_b);
Additionally, the is_associative() (I would assume this should be is_commutative) check seems incomplete.
Related to existing issue: #13566
Repro:
import torch
import ttnn
with ttnn.manage_device(0) as device:
input0 = torch.randn(1, 12, 256, 256)
input1 = torch.randn(1)
input0 = ttnn.from_torch(input0, device=device, dtype=ttnn.float32, layout=ttnn.TILE_LAYOUT, memory_config=ttnn.DRAM_MEMORY_CONFIG)
input1 = ttnn.from_torch(input1, device=device, dtype=ttnn.float32, layout=ttnn.TILE_LAYOUT, memory_config=ttnn.DRAM_MEMORY_CONFIG)
# Add works as expected
output = ttnn.add(input0, input1)
output = ttnn.add(input1, input0)
print("done add")
# This one passes:
output = ttnn.subtract(input0, input1)
# This one fails:
output = ttnn.subtract(input1, input0)
print("done subtract")
# This one fails:
output = ttnn.maximum(input0, input1)
# This one passes:
output = ttnn.maximum(input1, input0)
print("done maximum")
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 20 (6 by maintainers)
Commits related to this issue
- #20241: Implement binary max (#20390) ### Ticket #20241, #18556, #16969, #14852 ### Problem description Implement Tensor-Tensor version of maximum ### What's changed Implemented Tensor-Tensor versi... — committed to tenstorrent/tt-metal by VirdhatchaniKN 2 years ago
- #20241: Implement binary max (#20390) ### Ticket #20241, #18556, #16969, #14852 ### Problem description Implement Tensor-Tensor version of maximum ### What's changed Implemented Tensor-Tensor versi... — committed to tenstorrent/tt-metal by VirdhatchaniKN 2 years ago
- #20241: Implement binary max (#20390) ### Ticket #20241, #18556, #16969, #14852 ### Problem description Implement Tensor-Tensor version of maximum ### What's changed Implemented Tensor-Tensor versi... — committed to tenstorrent/tt-metal by VirdhatchaniKN 2 years ago
https://github.com/tenstorrent/tt-metal/pull/20390 - Merged Issue fixed with the use of flag use_legacy=False
Hi @cmaryanTT @pilkicTT @staylorTT Broadcasting support for maximum is now provided in PR https://github.com/tenstorrent/tt-metal/pull/20390 With the use of flag
use_legacy=False, test now passesThis will be addressed by #16147 - follow that issue for updates.
This should be on-by-default within the next 48 hrs
Solving the broadcasting issue generally is top priority for us. @yan-zaretskiy is actively working this. Related issue is also #13643.
The eta on correctly implementing this feature is November 22. Downgrading to P1. Please let me know if a temporary solution is needed before then.