vllm: [Bug]: torch.ops._C.silu_and_mul does not exist
Your current environment
macOS cpu torch.ops._C.silu_and_mul does not exist

stack trace:
[rank0]: Traceback (most recent call last):
[rank0]: File “/Users/xuji/programs/vllm_projects_2/example.py”, line 10, in <module>
[rank0]: llm = LLM(model=“deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B”,trust_remote_code=True)
[rank0]: File “/Users/xuji/programs_2/vllm/vllm/utils.py”, line 1096, in inner
[rank0]: return fn(*args, **kwargs)
[rank0]: File “/Users/xuji/programs_2/vllm/vllm/entrypoints/llm.py”, line 243, in init
[rank0]: self.llm_engine = LLMEngine.from_engine_args(
[rank0]: File “/Users/xuji/programs_2/vllm/vllm/engine/llm_engine.py”, line 521, in from_engine_args
[rank0]: return engine_cls.from_vllm_config(
[rank0]: File “/Users/xuji/programs_2/vllm/vllm/engine/llm_engine.py”, line 497, in from_vllm_config
[rank0]: return cls(
[rank0]: File “/Users/xuji/programs_2/vllm/vllm/engine/llm_engine.py”, line 281, in init
[rank0]: self.model_executor = executor_class(vllm_config=vllm_config, )
[rank0]: File “/Users/xuji/programs_2/vllm/vllm/executor/executor_base.py”, line 286, in init
[rank0]: super().init(*args, **kwargs)
[rank0]: File “/Users/xuji/programs_2/vllm/vllm/executor/executor_base.py”, line 52, in init
[rank0]: self._init_executor()
[rank0]: File “/Users/xuji/programs_2/vllm/vllm/executor/mp_distributed_executor.py”, line 125, in _init_executor
[rank0]: self._run_workers(“load_model”,
[rank0]: File “/Users/xuji/programs_2/vllm/vllm/executor/mp_distributed_executor.py”, line 185, in _run_workers
[rank0]: driver_worker_output = run_method(self.driver_worker, sent_method,
[rank0]: File “/Users/xuji/programs_2/vllm/vllm/utils.py”, line 2347, in run_method
[rank0]: return func(*args, **kwargs)
[rank0]: File “/Users/xuji/programs_2/vllm/vllm/worker/cpu_worker.py”, line 226, in load_model
[rank0]: self.model_runner.load_model()
[rank0]: File “/Users/xuji/programs_2/vllm/vllm/worker/cpu_model_runner.py”, line 491, in load_model
[rank0]: self.model = get_model(vllm_config=self.vllm_config)
[rank0]: File “/Users/xuji/programs_2/vllm/vllm/model_executor/model_loader/init.py”, line 14, in get_model
[rank0]: return loader.load_model(vllm_config=vllm_config)
[rank0]: File “/Users/xuji/programs_2/vllm/vllm/model_executor/model_loader/loader.py”, line 441, in load_model
[rank0]: model = _initialize_model(vllm_config=vllm_config)
[rank0]: File “/Users/xuji/programs_2/vllm/vllm/model_executor/model_loader/loader.py”, line 127, in _initialize_model
[rank0]: return model_class(vllm_config=vllm_config, prefix=prefix)
[rank0]: File “/Users/xuji/programs_2/vllm/vllm/model_executor/models/qwen2.py”, line 431, in init
[rank0]: self.model = Qwen2Model(vllm_config=vllm_config,
[rank0]: File “/Users/xuji/programs_2/vllm/vllm/compilation/decorators.py”, line 151, in init
[rank0]: old_init(self, vllm_config=vllm_config, prefix=prefix, **kwargs)
[rank0]: File “/Users/xuji/programs_2/vllm/vllm/model_executor/models/qwen2.py”, line 300, in init
[rank0]: self.start_layer, self.end_layer, self.layers = make_layers(
[rank0]: File “/Users/xuji/programs_2/vllm/vllm/model_executor/models/utils.py”, line 609, in make_layers
[rank0]: [PPMissingLayer() for _ in range(start_layer)] + [
[rank0]: File “/Users/xuji/programs_2/vllm/vllm/model_executor/models/utils.py”, line 610, in <listcomp>
[rank0]: maybe_offload_to_cpu(layer_fn(prefix=f"{prefix}.{idx}"))
[rank0]: File “/Users/xuji/programs_2/vllm/vllm/model_executor/models/qwen2.py”, line 302, in <lambda>
[rank0]: lambda prefix: Qwen2DecoderLayer(config=config,
[rank0]: File “/Users/xuji/programs_2/vllm/vllm/model_executor/models/qwen2.py”, line 218, in init
[rank0]: self.mlp = Qwen2MLP(
[rank0]: File “/Users/xuji/programs_2/vllm/vllm/model_executor/models/qwen2.py”, line 92, in init
[rank0]: self.act_fn = SiluAndMul()
[rank0]: File “/Users/xuji/programs_2/vllm/vllm/model_executor/layers/activation.py”, line 68, in init
[rank0]: self.op = torch.ops._C.silu_and_mul
[rank0]: File “/Users/xuji/.pyenv/versions/3.9.6/lib/python3.9/site-packages/torch/_ops.py”, line 1232, in getattr
[rank0]: raise AttributeError(
[rank0]: AttributeError: ‘_OpNamespace’ ‘_C’ object has no attribute ‘silu_and_mul’
Your output of `python collect_env.py` here
INFO 04-02 17:38:05 [init.py:239] Automatically detected platform cpu.
Collecting environment information…
PyTorch version: 2.6.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A
OS: macOS 15.3.2 (arm64)
GCC version: Could not collect
Clang version: 17.0.0 (clang-1700.0.13.3)
CMake version: version 4.0.0
Libc version: N/A
Python version: 3.9.6 (default, Apr 1 2025, 20:27:55) [Clang 17.0.0 (clang-1700.0.13.3)] (64-bit runtime)
Python platform: macOS-15.3.2-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
CPU:
Apple M1 Pro
Versions of relevant libraries:
[pip3] numpy==2.0.2
[pip3] optree==0.14.1
[pip3] pyzmq==26.3.0
[pip3] torch==2.6.0
[pip3] torchaudio==2.6.0
[pip3] torchvision==0.21.0
[pip3] transformers==4.50.3
[conda] Could not collect
ROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: 0.8.3.dev205+g25293780
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled
GPU Topology:
Could not collect
VLLM_USE_MODELSCOPE=True
VLLM_TARGET_DEVICE=cpu
LD_LIBRARY_PATH=/Users/xuji/.pyenv/versions/3.9.6/lib/python3.9/site-packages/cv2/…/…/lib:
NCCL_CUMEM_ENABLE=0
TORCHINDUCTOR_COMPILE_THREADS=1
Your output of `python collect_env.py` here
🐛 Describe the bug
Before submitting a new issue…
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 1
- Comments: 30 (15 by maintainers)
confirm the problem, its clang problem, for macOS it needs to rool back clang version to 16 or 15
17 will cause the problem,
the main reason is from here ImportError(“dlopen(/Users/aiserver/AI/vllm/vllm/_C.abi3.so, 0x0002): symbol not found in flat namespace ‘___kmpc_dispatch_deinit’”)
and for clang17 in mac there can not find
___kmpc_dispatch_deinitsymbolit had been fixed in llvm and other version
more can check:
issue: https://trac.macports.org/ticket/70877 https://github.com/yugabyte/yugabyte-db-thirdparty/pull/298 https://github.com/llvm/llvm-project/issues/98046
will open a doc patch to info it
and for the solutin
setps
cc @hmellor @youkaichao