vllm: [Installation]: `ModuleNotFoundError: No module named 'numpy.lib.function_base'` due to NumPy 2.0 release

Anything you want to discuss about vllm.

Users may see the following error when trying to run vllm:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/waldo/venv/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 26, in <module>
    from vllm.entrypoints.openai.serving_chat import OpenAIServingChat
  File "/home/waldo/venv/lib/python3.11/site-packages/vllm/entrypoints/openai/serving_chat.py", line 27, in <module>
    from vllm.model_executor.guided_decoding import (
  File "/home/waldo/venv/lib/python3.11/site-packages/vllm/model_executor/guided_decoding/__init__.py", line 6, in <module>
    from vllm.model_executor.guided_decoding.lm_format_enforcer_decoding import (
  File "/home/waldo/venv/lib/python3.11/site-packages/vllm/model_executor/guided_decoding/lm_format_enforcer_decoding.py", line 15, in <module>
    from vllm.model_executor.guided_decoding.outlines_decoding import (
  File "/home/waldo/venv/lib/python3.11/site-packages/vllm/model_executor/guided_decoding/outlines_decoding.py", line 13, in <module>
    from vllm.model_executor.guided_decoding.outlines_logits_processors import (
  File "/home/waldo/venv/lib/python3.11/site-packages/vllm/model_executor/guided_decoding/outlines_logits_processors.py", line 24, in <module>
    from outlines.fsm.guide import CFGGuide, Generate, Guide, RegexGuide, Write
  File "/home/waldo/venv/lib/python3.11/site-packages/outlines/__init__.py", line 2, in <module>
    import outlines.generate
  File "/home/waldo/venv/lib/python3.11/site-packages/outlines/generate/__init__.py", line 2, in <module>
    from .cfg import cfg
  File "/home/waldo/venv/lib/python3.11/site-packages/outlines/generate/cfg.py", line 5, in <module>
    from outlines.models import OpenAI
  File "/home/waldo/venv/lib/python3.11/site-packages/outlines/models/__init__.py", line 14, in <module>
    from .openai import OpenAI, azure_openai, openai
  File "/home/waldo/venv/lib/python3.11/site-packages/outlines/models/openai.py", line 9, in <module>
    from outlines.base import vectorize
  File "/home/waldo/venv/lib/python3.11/site-packages/outlines/base.py", line 8, in <module>
    from numpy.lib.function_base import (
ModuleNotFoundError: No module named 'numpy.lib.function_base'

This is because numpy recently released a breaking change (2.0.0), and for some reason, vllm’s dependencies don’t restrict the version based on the major number (which should have been frozen at 1).

#5582 works around this, but this hasn’t reached me – I suspect it’s because I’m using pip install instead of building from source. A workaround for users like me is to run pip install numpy==1.26.4.

The purpose of this issue is to show up when users search for the above output. Please do not close it until the fix is available to users of pip install.

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 11
  • Comments: 18 (8 by maintainers)

Most upvoted comments

@glibg10b Thanks for reporting the bug! We are aware of the bug and are currently waiting for either outlines or tensorizer to update its Numpy dependency (e.g., by adding numpy < 2.0.0 in its dependency). If they don’t, we will make a new release with the fixed Numpy dependency.

I’ll let @mgoin also take a look at this before deciding.

This is a Parallels Desktop (macos virtual machine) ubuntu install, with the following done to it:

sudo apt update
sudo apt upgrade

then I created the virtual env vllm-env, followed by:

source ~/vllm-env/bin/activate
pip install numpy==1.26.4
pip install vllm

(note, this is preliminary testing to host a llm, if there are other linux distributions or ways to get started, that is acceptable for me, I do not require this to be possible with the current installation, so if I started wrong, please enlighten me)

I think that this issue is related to MacOS installation rather than numpy. Can you open a new issue for this?