flash-attention: Flash Attention 2.0 doesn't work: undefined symbol: _ZNK3c1010TensorImpl27throw_data_ptr_access_errorEv

Hi there,

cloing the repo and running pytest tests/test_flash_attn.py gives

ImportError while importing test module '/root/flash-attention/tests/test_flash_attn.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_flash_attn.py:10: in <module>
    from flash_attn import flash_attn_func, flash_attn_kvpacked_func, flash_attn_qkvpacked_func
/usr/local/lib/python3.10/dist-packages/flash_attn/__init__.py:3: in <module>
    from flash_attn.flash_attn_interface import flash_attn_func
/usr/local/lib/python3.10/dist-packages/flash_attn/flash_attn_interface.py:4: in <module>
    import flash_attn_2_cuda as flash_attn_cuda
E   ImportError: /usr/local/lib/python3.10/dist-packages/flash_attn_2_cuda.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZNK3c1010TensorImpl27throw_data_ptr_access_errorEv

maybe somebody else have encountered this

Thanks a lot,

Fra

About this issue

Original URL
State: open
Created a year ago
Reactions: 15
Comments: 22 (4 by maintainers)

Most upvoted comments

Try a cxx11abiFALSE version

+12

janelu9 on Aug 17, 2023

You can compile from source with FLASH_ATTENTION_FORCE_BUILD=TRUE: FLASH_ATTENTION_FORCE_BUILD=TRUE pip install flash-attn.

tridao on Aug 25, 2023

Thanks for the report. I saw just this error on nvcr 23.06 as well. nvcr 23.07 should work, can you try? The error is due to pytorch interface changing between the version used in 23.06 and 23.07.

tridao on Aug 15, 2023

Try a cxx11abiFALSE version 试用版本 cxx11abiFALSE You can use this to solve the problem. Use pip to install the version package that matches yours directly. I use python 10, cuda11.8 torch 2.0, cxx11abiFALSE

https://github.com/Dao-AILab/flash-attention/releases/download/v2.5.2/flash_attn-2.5.2+cu118torch2.0cxx11abiFALSE-cp310-cp310-linux_x86_64.whl

sleeper1023 on Feb 4, 2024

Also facing this error, in Databricks. Torch Version: 2.1.0+cu121 torch.cuda.get_device_capability(): (7,0) Compiled from source but that did not work for me.

ImportError: /databricks/python/lib/python3.10/site-packages/flash_attn_2_cuda.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZNK3c106SymIntltEl

achangtv on Feb 1, 2024

Thank you It seems that it is working now

junjun3518 on Aug 25, 2023

to

Thanks a lot, will try as soon as I go back to my pc, out of curiosity could to give me more details about due to pytorch interface changing between the version used in 23.06 and 23.07?

FrancescoSaverioZuppichini on Aug 15, 2023