tt-metal: [Matmul] Does not respect provided mem config
Describe the bug Matmul with interleaved inputs and height sharded output does not respect the number of cores provided with shard spec.
To Reproduce Did not mange to repro this in python, so he is cpp:
ttnn::SimpleShape v1s = ttnn::SimpleShape(tt::tt_metal::LegacyShape({1, 32, 50, 100}));
ttnn::Tensor v1 = ttnn::ones(v1s, ttnn::DataType::BFLOAT16, ttnn::Layout::ROW_MAJOR, std::nullopt, std::nullopt);
ttnn::SimpleShape v2s = ttnn::SimpleShape(tt::tt_metal::LegacyShape({1, 32, 100, 12}));
ttnn::Tensor v2 = ttnn::ones(v2s, ttnn::DataType::BFLOAT16, ttnn::Layout::ROW_MAJOR, std::nullopt, std::nullopt);
ttnn::IDevice* v3 = &ttnn::open_device(0);
ttnn::MemoryConfig v4 = ttnn::MemoryConfig(ttnn::TensorMemoryLayout::INTERLEAVED, ttnn::BufferType::DRAM);
ttnn::Tensor v5 = ttnn::to_device(v1, v3, v4);
ttnn::Tensor v6 = ttnn::to_layout(v5, ttnn::Layout::TILE, std::nullopt, std::nullopt, static_cast<::ttnn::IDevice *>(nullptr));
ttnn::MemoryConfig v7 = ttnn::MemoryConfig(ttnn::TensorMemoryLayout::INTERLEAVED, ttnn::BufferType::DRAM);
ttnn::Tensor v8 = ttnn::to_device(v2, v3, v7);
ttnn::Tensor v9 = ttnn::to_layout(v8, ttnn::Layout::TILE, std::nullopt, std::nullopt, static_cast<::ttnn::IDevice *>(nullptr));
ttnn::SimpleShape v10 = ttnn::SimpleShape(tt::tt_metal::LegacyShape({32, 32, }));
ttnn::MemoryConfig v11 = ttnn::MemoryConfig(ttnn::TensorMemoryLayout::INTERLEAVED, ttnn::BufferType::DRAM);
ttnn::CoreRangeSet grid = tt::tt_metal::num_cores_to_corerangeset(50, {8, 8});
tt::tt_metal::ShardSpec shard_spec = tt::tt_metal::ShardSpec(grid, {32, 32});
ttnn::MemoryConfig mem_config = ttnn::MemoryConfig(ttnn::TensorMemoryLayout::HEIGHT_SHARDED, ttnn::BufferType::L1, shard_spec);
std::cout << "Desired shard spec: " << shard_spec << std::endl;
// Desired shard spec: ShardSpec(grid={[(x=0,y=0) - (x=5,y=7)], [(x=6,y=0) - (x=6,y=1)]},shape={32, 32},orientation=ShardOrientation::ROW_MAJOR,mode=ShardMode::PHYSICAL,physical_shard_shape=std::nullopt)
ttnn::Tensor v13 = ttnn::matmul(v6, v9, false, false, mem_config);
std::cout << "Actual shard spec: " << v13.shard_spec().value() << std::endl;
// Actual shard spec: ShardSpec(grid={[(x=0,y=0) - (x=1,y=0)]},shape={32, 32},orientation=ShardOrientation::ROW_MAJOR,mode=ShardMode::PHYSICAL,physical_shard_shape=std::nullopt)
ttnn::close_device(*v3);
Expected behavior Assertion that the provided memory config is invalid or successful execution.
Please complete the following environment information:
- commit:
78f2c0b774ab48bf243093d3111717a8c880dc78
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 19 (13 by maintainers)
Commits related to this issue
- #17536: add matmul output mem config validation — committed to tenstorrent/tt-metal by bbradelTT 2 years ago
- #17536: add matmul output mem config validation — committed to tenstorrent/tt-metal by bbradelTT 2 years ago
- #17536: update matmul test to have out shard info — committed to tenstorrent/tt-metal by bbradelTT 2 years ago
- #17536: add matmul output mem config validation — committed to tenstorrent/tt-metal by bbradelTT 2 years ago
- #17536: update matmul test to have out shard info — committed to tenstorrent/tt-metal by bbradelTT 2 years ago
- #17536: add extra matmul output mem config check, doc, and fix test — committed to tenstorrent/tt-metal by bbradelTT 2 years ago
- #17536: add matmul output mem config validation — committed to tenstorrent/tt-metal by bbradelTT 2 years ago
- #17536: update matmul test to have out shard info — committed to tenstorrent/tt-metal by bbradelTT 2 years ago
- #17536: add extra matmul output mem config check, doc, and fix test — committed to tenstorrent/tt-metal by bbradelTT 2 years ago
- #17536 update matmul interleaved test — committed to tenstorrent/tt-metal by bbradelTT 2 years ago
- #17536: change fatals to asserts for some matmul mem config checks — committed to tenstorrent/tt-metal by bbradelTT 2 years ago
- #17536: make matmul full output mem config check be assert — committed to tenstorrent/tt-metal by bbradelTT 2 years ago
- #17536: add matmul output mem config validation — committed to tenstorrent/tt-metal by bbradelTT 2 years ago
- #17536: update matmul test to have out shard info — committed to tenstorrent/tt-metal by bbradelTT 2 years ago
- #17536: add extra matmul output mem config check, doc, and fix test — committed to tenstorrent/tt-metal by bbradelTT 2 years ago
- #17536 update matmul interleaved test — committed to tenstorrent/tt-metal by bbradelTT 2 years ago
- #17536: change fatals to asserts for some matmul mem config checks — committed to tenstorrent/tt-metal by bbradelTT 2 years ago
- #17536: make matmul full output mem config check be assert — committed to tenstorrent/tt-metal by bbradelTT 2 years ago
- #17536: update matmul mem config check and test — committed to tenstorrent/tt-metal by bbradelTT 2 years ago
- #17536: clear shard_spec of more matmul test calls — committed to tenstorrent/tt-metal by bbradelTT 2 years ago
PR has been merged in. Closing.
Great. Thanks @odjuricicTT I’ll change the PR I worked on to still fail for anything other than shard spec. For shard spec, it’ll be a warning.
@skhatriTT @bbradelTT Yes, we have an alternative solution for this problem in the works. For now this is on hold, and i’ll update it once we test the other solution fully. Most probably this will not be needed.
@odjuricicTT Asserting should be something we can do. We’ll need to figure if it’ll break other existing code.
We should also update the doc string in the pybind.