diffusers: Performance degradation in `mps` after `einsum` replacement

Before #445 was merged I was getting ~31s inference time in mps. After the change, time goes up to 42s. I verified again in main @ b2b3b1a, and time is again 31s.

I haven’t checked other platforms yet.

Any ideas, @patil-suraj?

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 15 (13 by maintainers)

Commits related to this issue

[WEB] Cache model parameters (#452) This commit cache some of the model parameters to reduce the response time of shark web. Signed-Off-by: Gaurav Shukla <gaurav@nod-labs.com> Signed-off-by: G... — committed to nod-ai/diffusers by Shukla-Gaurav 2 years ago

Most upvoted comments

Addressed in #926.

pcuenca on Oct 29, 2022

also, a different picture is generated. despite same seed each time.

how are you using the seeds ? diffusers pipelines uses the torch.Generator objects for seeds. To get reproducible results we need to reinit the torch.Generator with the same seed as using the same generator multiple times advances the rng state.

The correct way to check this would be running this same block multiple times.

with autocast("cuda"):
    images = pipe(prompt, generator=torch.Generator(device="cuda").manual_seed(1024)).images
images[0]

patil-suraj on Sep 12, 2022