vllm: [Installation]: no version of pip install vllm works - Failed to initialize NumPy: No Module named 'numpy'

Your current environment

Traceback (most recent call last):
  File "/mnt/MSAI/home/cephdon/sources/vllm/collect_env.py", line 15, in <module>
    from vllm.envs import environment_variables
  File "/mnt/MSAI/home/cephdon/sources/vllm/vllm/__init__.py", line 3, in <module>
    from vllm.engine.arg_utils import AsyncEngineArgs, EngineArgs
  File "/mnt/MSAI/home/cephdon/sources/vllm/vllm/engine/arg_utils.py", line 11, in <module>
    from vllm.config import (CacheConfig, CompilationConfig, ConfigFormat,
  File "/mnt/MSAI/home/cephdon/sources/vllm/vllm/config.py", line 21, in <module>
    from vllm.model_executor.layers.quantization import (QUANTIZATION_METHODS,
  File "/mnt/MSAI/home/cephdon/sources/vllm/vllm/model_executor/__init__.py", line 1, in <module>
    from vllm.model_executor.parameter import (BasevLLMParameter,
  File "/mnt/MSAI/home/cephdon/sources/vllm/vllm/model_executor/parameter.py", line 7, in <module>
    from vllm.distributed import get_tensor_model_parallel_rank
  File "/mnt/MSAI/home/cephdon/sources/vllm/vllm/distributed/__init__.py", line 1, in <module>
    from .communication_op import *
  File "/mnt/MSAI/home/cephdon/sources/vllm/vllm/distributed/communication_op.py", line 6, in <module>
    from .parallel_state import get_tp_group
  File "/mnt/MSAI/home/cephdon/sources/vllm/vllm/distributed/parallel_state.py", line 38, in <module>
    import vllm.distributed.kv_transfer.kv_transfer_agent as kv_transfer
  File "/mnt/MSAI/home/cephdon/sources/vllm/vllm/distributed/kv_transfer/kv_transfer_agent.py", line 15, in <module>
    from vllm.distributed.kv_transfer.kv_connector.factory import (
  File "/mnt/MSAI/home/cephdon/sources/vllm/vllm/distributed/kv_transfer/kv_connector/factory.py", line 3, in <module>
    from .base import KVConnectorBase
  File "/mnt/MSAI/home/cephdon/sources/vllm/vllm/distributed/kv_transfer/kv_connector/base.py", line 14, in <module>
    from vllm.sequence import IntermediateTensors
  File "/mnt/MSAI/home/cephdon/sources/vllm/vllm/sequence.py", line 16, in <module>
    from vllm.inputs import SingletonInputs, SingletonInputsAdapter
  File "/mnt/MSAI/home/cephdon/sources/vllm/vllm/inputs/__init__.py", line 7, in <module>
    from .registry import (DummyData, InputContext, InputProcessingContext,
  File "/mnt/MSAI/home/cephdon/sources/vllm/vllm/inputs/registry.py", line 13, in <module>
    from vllm.transformers_utils.tokenizer import AnyTokenizer
  File "/mnt/MSAI/home/cephdon/sources/vllm/vllm/transformers_utils/tokenizer.py", line 16, in <module>
    from vllm.utils import make_async
  File "/mnt/MSAI/home/cephdon/sources/vllm/vllm/utils.py", line 44, in <module>
    from vllm.platforms import current_platform
  File "/mnt/MSAI/home/cephdon/sources/vllm/vllm/platforms/__init__.py", line 100, in <module>
    from .cuda import CudaPlatform
  File "/mnt/MSAI/home/cephdon/sources/vllm/vllm/platforms/cuda.py", line 14, in <module>
    import vllm._C  # noqa
ModuleNotFoundError: No module named 'vllm._C'

How you are installing vllm

pip install -vvv vllm
pip install vllm==0.6.4
pip install -e .

Before submitting a new issue…

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 18 (8 by maintainers)

Most upvoted comments

I should note that this is on an NVIDIA AGX Orin 64gb machine, so aarch64. I have cuda 12.6, but building from source has this same issue that I cannot seem to get past. Installing numpy (any version) does not resolve the issue. I am using miniconda to manage virtual environments. pip list and conda list both show numpy.

Note that the error message above is trying to use the CPU device, even if I have the cuda version of torch installed. Using the CPU version of torch does not resolve the issue.

@youkaichao

I don’t get it. Building in a container just makes things more isolated, you can of course use it without a container.

That’s not how supercomputing centers work. They tend to compile everything to their platform, and try to extract every bit of performance they can. Running a linux kernel and a whole toolchain on top of a a linux kernel and a whole toolchain (repeat as needed) is not acceptable in many places. The overhead is not worth the convenience.

Plus, the security nightmare that docker makes.

I have read your documentation multiple times. I have managed to compile it, but never managed to make it work.

It is got so bad that the market has been offering gh200 under cost because literally no one can run vllm on them properly because you can’t bother to offer a precompiled wheel.

Do you need links?

because you can’t bother to offer a precompiled wheel

we cannot provide wheels, because pytorch does not have official release wheels for gh200 (arm gpus). we have to wait for pytorch 2.6 release.

technically, you can get wheels by copying from the docker container, but the wheel does not specify pytorch as dependency, and you need to install pytorch before installing vllm using that wheel.

I have managed to compile it, but never managed to make it work.

can you elaborate?

@cephdon

… UserWarning: Failed to initialize NumPy: No module named ‘numpy’ …

this is a warning, you can ignore.

File “”, line 504, in get_vllm_version RuntimeError: Unknown runtime environment

this is the real error.

see the great note from @dtrifiro

@youkaichao @dtrifiro any idea about this?