tt-metal: Some TTNN operations fail with low PCC error [Grayskull and Wormhole]

Describe the bug Some TTNN operation break with low PCC error in some test cases, for both, Grayskull and Wormhole chips.

TTNN operations which failed are:

  • ttnn.sin (passed for Wormhole)
  • ttnn.sqrt
  • ttnn.rsqrt

To Reproduce Steps to reproduce the behavior for both chips, GS and WH:

  1. Checkout barsic/ttnn-ops branch (to be merged into main soon)
  2. Run unit test test_eltwise_sin.py using this command: pytest tests/ttnn/python_api_testing/non_working_unit_tests/grayskull/test_eltwise_sin.py

Expected behavior There are test cases presented in the unit tests pytest tests/ttnn/python_api_testing/non_working_unit_tests/grayskull/test_eltwise_sin.py and they all are expected to fail with low PCC error.

Other unit tests (for Grayskull, and Wormhole):

tests/ttnn/python_api_testing/non_working_unit_tests/grayskull/test_eltwise_sqrt.py
tests/ttnn/python_api_testing/non_working_unit_tests/grayskull/test_eltwise_rsqrt.py

Getting Additional info for the operation under test and its behavior To get additional information and results for different combinations of input shapes, types, layouts and memory configs for which this operation was tested you can also run locally sweeps for tt_lib.tensor.complex_abs and check the results. To do this you should:

  1. Follow the Getting Started page to setup the repo, environment variables and python-env
  2. Activate source build/python_env/bin/activate
  3. Run sweeps by using python tests/tt_eager/python_api_testing/sweep_tests/run_pytorch_test.py -i tests/ttnn/python_api_testing/sweep_tests/test_configs/ci_sweep_tests_broken/grayskull/ttnn_eltwise_sin_test.yaml -o /home/ubuntu/tt-metal/ttnn-test-sweeps/sin
  4. After the run is completed all test sweeps results should be available inside specified output directory (in this case ./result-sweeps). There you will find eltwise_sin_sweep.csv which holds all executed sweeps, among which you can also find the ones that failed and were recreated by the unit test, which you can get by searching unique data_seed field.

Other sweep tests:

tests/ttnn/python_api_testing/sweep_tests/test_configs/ci_sweep_tests_broken/grayskull/ttnn_eltwise_sqrt_test.yaml
tests/ttnn/python_api_testing/sweep_tests/test_configs/ci_sweep_tests_broken/grayskull/ttnn_eltwise_rsqrt_test.yaml

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 15 (15 by maintainers)

Commits related to this issue

Most upvoted comments

Hi @nemanjagrujic

ttnn.sqrt, ttnn.rsqrt and sin now pass, and tan and reciprocal are handled in other tickets so closing this.