InvokeAI: [bug]: InvokeAI won't use GPU (CUDA/NVIDIA). Defaults to CPU

Is there an existing issue for this?

I have searched the existing issues

OS

Windows

GPU

cuda

VRAM

24GB

What happened?

Starting invoke.bat is always saying that it uses the CPU instead of my GPU (And it indeed tries to generate with the CPU). Here’s the cmd output with the relevant line indicated by an arrow. The line should read “>> Using device_type cuda” not “CPU”

1. command-line
2. browser-based UI
3. open the developer console
Please enter 1, 2 or 3: 2
Starting the InvokeAI browser-based UI..
* Initializing, be patient...
>> Initialization file C:\Users\Falxie II/.invokeai found. Loading...
>> InvokeAI runtime directory is "J:\AI\Art\StableDiffusion\invokeAI"
>> Patchmatch initialized
>> GFPGAN Initialized
>> CodeFormer Initialized
>> ESRGAN Initialized
>> Using device_type CPU     <-----------------------------
>> Current VRAM usage:  0.00G
>> Scanning Model: stable-diffusion-1.5
>> Model Scanned. OK!!
>> Loading stable-diffusion-1.5 from J:\AI\Art\StableDiffusion\invokeAI\models\ldm\stable-diffusion-v1\v1-5-pruned-emaonly.ckpt
   | LatentDiffusion: Running in eps-prediction mode
   | DiffusionWrapper has 859.52 M params.
   | Making attention of type 'vanilla' with 512 in_channels
   | Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
   | Making attention of type 'vanilla' with 512 in_channels
   | Using more accurate float32 precision
   | Loading VAE weights from: J:\AI\Art\StableDiffusion\invokeAI\models\ldm\stable-diffusion-v1\vae-ft-mse-840000-ema-pruned.ckpt
>> Model loaded in 7.44s
>> Current embedding manager terms: *
>> Setting Sampler to k_euler_a

* --web was specified, starting web server...
>> Initialization file C:\Users\Falxie II/.invokeai found. Loading...
>> Started Invoke AI Web Server!
>> Default host address now 127.0.0.1 (localhost). Use --host 0.0.0.0 to bind any address.
>> Point your browser at http://127.0.0.1:9090

Screenshots

Here’s a screenshot of a prompt being executed. Notice the time to completion in the bottom right corner, grafik And this is my GPU during the generation process. Notice the utilization which also indicates that the process uses the CPU. grafik

Additional context

I reinstalled InvokeAI several times and it’s always the same. Automatic1111 WebUI uses the GPU just fine.

Contact Details

No response

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 26 (1 by maintainers)

Most upvoted comments

I fixed it with:

pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 -f https://download.pytorch.org/whl/torch_stable.html

(while in the environment)

You can check if cuda is working with:

$ python
Python 3.9.16 (main, Jan 11 2023, 16:16:36) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.cuda.is_available()
True

After that, I directly ran invoke.py.

Yup this worked for me!!. i used invoke.bat

veetance on Feb 1, 2023

I fixed it with:

pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 -f https://download.pytorch.org/whl/torch_stable.html

(while in the environment)

You can check if cuda is working with:

$ python
Python 3.9.16 (main, Jan 11 2023, 16:16:36) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.cuda.is_available()
True

After that, I directly ran invoke.py.

rafalou38 on Jan 27, 2023

Still no luck here then 😕 Damn… I think this should be one of the main issues to look at. How many want to sun sirapy slow on a CPU when they can gp lightning fast on a GPU?!

It is 100% the main issue. I wish I’d never come across this app tbh it’s been nothing but a tease and a headache.

DivinAria on Dec 4, 2022

Yeah I’m not going to even bother trying at this point.

DivinAria on Dec 4, 2022

I am having the same issue. RTX 3080 Ti. Its using CPU instead of GPU. This is from a fresh automated install.

Xerophayze on Dec 4, 2022