tt-metal: Runtime SW assumes too many threads
Running time build/test/tt_metal/perf_microbenchmark/dispatch/test_pgm_dispatch -w 5000 -s 8192 -x 7 -y 6 -kg 7 on BH machine (yyzo-bh-26) consistently takes around 30s Tracy difference between WH and BH:
build_firmware on BH was around ~10s
The above config is ran with taskflow having 32 available threads
Experimented with setting TT_METAL_THREADCOUNT to 4 / 8 / 16:
build_firmwarecan take closer to ~500mstime build/test/tt_metal/perf_microbenchmark/dispatch/test_pgm_dispatch -w 5000 -s 8192 -x 7 -y 6 -kg 7takes anywhere from ~9s to ~27s
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 20 (20 by maintainers)
Commits related to this issue
- #11717: aligning jit build and collection of srcs used by multiple threads during risc compile — committed to tenstorrent/tt-metal by abhullar-tt 2 years ago
- #11717: aligning jit build and collection of srcs used by multiple threads during risc compile — committed to tenstorrent/tt-metal by abhullar-tt 2 years ago
- #11717: aligning jit build and collection of srcs used by multiple threads during risc compile — committed to tenstorrent/tt-metal by abhullar-tt 2 years ago
- #11717: aligning jit build and collection of srcs used by multiple threads during risc compile — committed to tenstorrent/tt-metal by abhullar-tt 2 years ago
- #11717: aligning jit build and collection of srcs used by multiple threads during risc compile — committed to tenstorrent/tt-metal by abhullar-tt 2 years ago
- #11717: aligning jit build and collection of srcs used by multiple threads during risc compile — committed to tenstorrent/tt-metal by abhullar-tt 2 years ago
- #11717: aligning jit build and collection of srcs used by multiple threads during risc compile — committed to tenstorrent/tt-metal by abhullar-tt 2 years ago
root cause is BH systems were running on 1G eth while WH systems had 100G eth. Fix is to use /localdev to build/run locally