mamba: Mamba-2 Error: `'NoneType' object has no attribute 'causal_conv1d_fwd'`
When I’m running
import torch
from mamba_ssm import Mamba, Mamba2
batch, length, dim = 2, 64, 512
x1 = torch.randn(batch, length, dim).to("cuda")
model = Mamba(
# This module uses roughly 3 * expand * d_model^2 parameters
d_model=dim, # Model dimension d_model
d_state=16, # SSM state expansion factor
d_conv=4, # Local convolution width
expand=2, # Block expansion factor
).to("cuda")
y1 = model(x1)
x2 = torch.randn(batch, length, dim).to("cuda")
model = Mamba2(
# This module uses roughly 3 * expand * d_model^2 parameters
d_model=dim, # Model dimension d_model
d_state=16, # SSM state expansion factor
d_conv=4, # Local convolution width
expand=2, # Block expansion factor
).to("cuda")
y2 = model(x2)
the mamba-1 model works fine, but the mamba-2 model encounters the following error:
Traceback (most recent call last):
File "/snap/dataspell/92/plugins/python-ce/helpers/pydev/pydevconsole.py", line 364, in runcode
coro = func()
^^^^^^
File "<input>", line 9, in <module>
File "/home/admin/.conda/envs/myenv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/admin/.conda/envs/myenv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/admin/.conda/envs/myenv/lib/python3.11/site-packages/mamba_ssm/modules/mamba2.py", line 183, in forward
out = mamba_split_conv1d_scan_combined(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/admin/.conda/envs/myenv/lib/python3.11/site-packages/mamba_ssm/ops/triton/ssd_combined.py", line 930, in mamba_split_conv1d_scan_combined
return MambaSplitConv1dScanCombinedFn.apply(zxbcdt, conv1d_weight, conv1d_bias, dt_bias, A, D, chunk_size, initial_states, seq_idx, dt_limit, return_final_states, activation, rmsnorm_weight, rmsnorm_eps, outproj_weight, outproj_bias, headdim, ngroups, norm_before_gate)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/admin/.conda/envs/myenv/lib/python3.11/site-packages/torch/autograd/function.py", line 553, in apply
return super().apply(*args, **kwargs) # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/admin/.conda/envs/myenv/lib/python3.11/site-packages/torch/cuda/amp/autocast_mode.py", line 115, in decorate_fwd
return fwd(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^
File "/home/admin/.conda/envs/myenv/lib/python3.11/site-packages/mamba_ssm/ops/triton/ssd_combined.py", line 779, in forward
causal_conv1d_cuda.causal_conv1d_fwd(rearrange(xBC, "b s d -> b d s"),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'causal_conv1d_fwd'
Following are my packages:
(myenv) admin@admin-Laptop:~$ conda list
# packages in environment at /home/admin/.conda/envs/myenv:
#
# Name Version Build Channel
_libgcc_mutex 0.1 main
_openmp_mutex 5.1 1_gnu
blas 1.0 mkl
brotli-python 1.0.9 py311h6a678d5_8
bzip2 1.0.8 h5eee18b_6
ca-certificates 2024.7.2 h06a4308_0
causal-conv1d 1.4.0 pypi_0 pypi
certifi 2024.7.4 py311h06a4308_0
charset-normalizer 2.0.4 pyhd3eb1b0_0
cuda-cudart 12.1.105 0 nvidia
cuda-cupti 12.1.105 0 nvidia
cuda-libraries 12.1.0 0 nvidia
cuda-nvrtc 12.1.105 0 nvidia
cuda-nvtx 12.1.105 0 nvidia
cuda-opencl 12.5.39 0 nvidia
cuda-runtime 12.1.0 0 nvidia
cuda-version 12.5 3 nvidia
einops 0.8.0 pypi_0 pypi
ffmpeg 4.3 hf484d3e_0 pytorch
filelock 3.13.1 py311h06a4308_0
freetype 2.12.1 h4a9f257_0
fsspec 2024.6.1 pypi_0 pypi
gmp 6.2.1 h295c915_3
gmpy2 2.1.2 py311hc9b5ff0_0
gnutls 3.6.15 he1e5248_0
huggingface-hub 0.23.4 pypi_0 pypi
idna 3.7 py311h06a4308_0
intel-openmp 2023.1.0 hdb19cb5_46306
jinja2 3.1.4 py311h06a4308_0
jpeg 9e h5eee18b_1
lame 3.100 h7b6447c_0
lcms2 2.12 h3be6417_0
ld_impl_linux-64 2.38 h1181459_1
lerc 3.0 h295c915_0
libcublas 12.1.0.26 0 nvidia
libcufft 11.0.2.4 0 nvidia
libcufile 1.10.1.7 0 nvidia
libcurand 10.3.6.82 0 nvidia
libcusolver 11.4.4.55 0 nvidia
libcusparse 12.0.2.55 0 nvidia
libdeflate 1.17 h5eee18b_1
libffi 3.4.4 h6a678d5_1
libgcc-ng 11.2.0 h1234567_1
libgomp 11.2.0 h1234567_1
libiconv 1.16 h5eee18b_3
libidn2 2.3.4 h5eee18b_0
libjpeg-turbo 2.0.0 h9bf148f_0 pytorch
libnpp 12.0.2.50 0 nvidia
libnvjitlink 12.1.105 0 nvidia
libnvjpeg 12.1.1.14 0 nvidia
libpng 1.6.39 h5eee18b_0
libstdcxx-ng 11.2.0 h1234567_1
libtasn1 4.19.0 h5eee18b_0
libtiff 4.5.1 h6a678d5_0
libunistring 0.9.10 h27cfd23_0
libuuid 1.41.5 h5eee18b_0
libwebp-base 1.3.2 h5eee18b_0
llvm-openmp 14.0.6 h9e868ea_0
lz4-c 1.9.4 h6a678d5_1
mamba-ssm 2.2.2 pypi_0 pypi
markupsafe 2.1.3 py311h5eee18b_0
mkl 2023.1.0 h213fc3f_46344
mkl-service 2.4.0 py311h5eee18b_1
mkl_fft 1.3.8 py311h5eee18b_0
mkl_random 1.2.4 py311hdb19cb5_0
mpc 1.1.0 h10f8cd9_1
mpfr 4.0.2 hb69a4c5_1
mpmath 1.3.0 py311h06a4308_0
ncurses 6.4 h6a678d5_0
nettle 3.7.3 hbbd107a_1
networkx 3.3 py311h06a4308_0
ninja 1.11.1.1 pypi_0 pypi
numpy 1.26.4 py311h08b1b3b_0
numpy-base 1.26.4 py311hf175353_0
openh264 2.1.1 h4ff587b_0
openjpeg 2.4.0 h9ca470c_2
openssl 3.0.14 h5eee18b_0
packaging 24.1 pypi_0 pypi
pillow 10.3.0 py311h5eee18b_0
pip 24.0 py311h06a4308_0
pysocks 1.7.1 py311h06a4308_0
python 3.11.9 h955ad1f_0
pytorch 2.2.2 py3.11_cuda12.1_cudnn8.9.2_0 pytorch
pytorch-cuda 12.1 ha16c6d3_5 pytorch
pytorch-mutex 1.0 cuda pytorch
pyyaml 6.0.1 py311h5eee18b_0
readline 8.2 h5eee18b_0
regex 2024.5.15 pypi_0 pypi
requests 2.32.2 py311h06a4308_0
safetensors 0.4.3 pypi_0 pypi
setuptools 69.5.1 py311h06a4308_0
sqlite 3.45.3 h5eee18b_0
sympy 1.12 py311h06a4308_0
tbb 2021.8.0 hdb19cb5_0
tk 8.6.14 h39e8969_0
tokenizers 0.19.1 pypi_0 pypi
torchaudio 2.2.2 py311_cu121 pytorch
torchtriton 2.2.0 py311 pytorch
torchvision 0.17.2 py311_cu121 pytorch
tqdm 4.66.4 pypi_0 pypi
transformers 4.42.4 pypi_0 pypi
typing_extensions 4.11.0 py311h06a4308_0
tzdata 2024a h04d1e81_0
urllib3 2.2.2 py311h06a4308_0
wheel 0.43.0 py311h06a4308_0
xz 5.4.6 h5eee18b_1
yaml 0.2.5 h7b6447c_0
zlib 1.2.13 h5eee18b_1
zstd 1.5.5 hc292b87_2
About this issue
- Original URL
- State: open
- Created 2 years ago
- Comments: 16
I had the same issue after installing mamba_ssm with
pip install mamba-ssm[causal-conv1d]command.And after uninstalling
pip uninstall mamba-ssm causal-conv1dand re-installing causal-conv1d and then mamba-ssm core versionpip install causal-conv1d && pip install mamba-ssm. Issue solved.I have already solved this problem, you can try my method: https://blog.csdn.net/lihaiyuan_0324/article/details/140553709?spm=1001.2014.3001.5502
The issue was resolved by downloading the appropriate
.whlfile from the [releases](https://github.com/Dao-AILab/causal-conv1d/releases) page.Since my PyTorch version is 2.1.1, I downloaded the file:
causal_conv1d-1.5.0.post8+cu11torch2.1cxx11abiFALSE-cp310-cp310-linux_x86_64.whl.I recommend looking for a file where
cxx11abiis set to FALSE, as this may be necessary depending on your environment.Also, the test code needs a slight adjustment:
I tried changing
mamba_ssm/ops/triton/ssd_combined.py:to
And another import error occurred: