tt-metal: [Bug Report] Stable Diffusion interactive demo is hanging on N150

Describe the bug Stable Diffusion interactive demo : pytest models/demos/wormhole/stable_diffusion/demo/demo.py::test_interactive_demo is hanging.

I have tested with both FWs: 80.13.2.0 and 80.10.0.0.

It best it runs up to 98%. Then crashes: see attached.

To Reproduce Steps to reproduce the behavior:

  1. Go to main branch
  2. Follow the build instructions from main and build metal.
  3. run: pytest models/demos/wormhole/stable_diffusion/demo/demo.py::test_interactive_demo
  4. the demo hangs and I lose connection to the VM. Expected behavior the interactive demo is used for the web demo as well. We need this issue resolved to enable the SD web demo.

Screenshots Image

Please complete the following environment information:

  • OS: [e.g. Ubuntu 20.04]

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 20 (19 by maintainers)

Most upvoted comments

@mbahnasTT Let’s close this issue in favour of this follow-on issue in #16057.

Addressed in 917d168aa339f37547414b9d09ae850f16eca9bf - re-open if you are still seeing the same issue.

Yes, it should be size per core. Right now it only asserts in debug mode, and will log a warning in release. Will be changed to assert in release once existing tests are fixed.

@esmalTT re-assigning to you 😃