tt-metal: [Bug Report] Small performance regression for binary_ng
Describe the bug
Fine-grained profiling comparing ttnn.add with ttnn.experimental.add shows 456μs vs 526μs. There seems to be a greater delay in between BINARY_OP zones when profiling various elementwise binary_ng operations such as add/mul.
To Reproduce
Run on-device profiling of ttnn.experimental.add and ttnn.add on a [1024, 8192] random int32 or float32 tensor with layout=ttnn.TILE_LAYOUT.
Expected behavior
Timings for binary_ng should be equal to or better than what exists currently.
Please complete the following environment information:
- Version: 907fffd392cd4c8101313decfc4fea99a3fea2f8
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 15 (9 by maintainers)
I’m going to close this case as committing the fixes is tracked elsewhere.
This issue is fixed. @dchenTT moved most/all of the binary_ng cases to read with the reader kernel, rather than the original split to read A with the reader and B with the writer - that was the cause of the ~15% degradation. This is fixed in binary_ng already.
The other ~4x degradation when doing row broadcast was cause by loop ordering resulting in discarding and having to re-create the subtile-broadcast result. That’s also fixed by @dchenTT , not by changing the loop ordering but by using the subtile broadcast LLK to speed things up. The overall result is parity or slightly better between all broadcast cases and the non-broadcast case. This is fixed in a PR that’s still in development and will go in after binary_ng is on-by-default.
Two tensors of identical shape, e.g.