vllm: [Installation]: Error when importing LLM from vllm

My current environment

i am using linux system, rtx 4080, cuda version: 12.1, python version - 3.9

from vllm import LLM
ImportError: cannot import name 'LLM' from 'vllm' (unknown location)

How you are installing vllm

pip install vllm

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 1
  • Comments: 18 (6 by maintainers)

Most upvoted comments

I have met the same problem. I am using ubuntu. I installed vllm via builid from source. I am trying to run the quantization tutorial.

Met the same error

    from vllm import LLM                                                                                                
ImportError: cannot import name 'LLM' from 'vllm' (unknown location)

Thank you for your prompt reply. I have fixed this issue. Sorry this is due to my stupid mistake. I just clone the vllm repo, and run the python script in the same dir that contains vllm repo.

Fixed this issue by upgrading python to 3.12

I have met the same problem. I am using ubuntu. I installed vllm via builid from source. I am trying to run the quantization tutorial.

Met the same error

    from vllm import LLM                                                                                                
ImportError: cannot import name 'LLM' from 'vllm' (unknown location)