AutoAWQ: [BUG] Import AWQ error
When I try to import awq, I meet error :
/opt/conda/lib/python3.10/site-packages/awq_inference_engine.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN3c106detail23torchInternalAssertFailEPKcS2_jS2_RKSs
My env is: awq: https://github.com/casper-hansen/AutoAWQ/releases/download/v0.1.8/autoawq-0.1.8+cu118-cp310-cp310-linux_x86_64.whl Torch: 2.0.1 + cu11.8 transformers: 4.36.0
About this issue
- Original URL
- State: closed
- Created 6 months ago
- Reactions: 2
- Comments: 17 (4 by maintainers)
installing from source seems to fix the issue
Closing this as not planned. This is an environment issue with torch. You must solve environment issue to install torch extensions.
It works with git clone https://github.com/casper-hansen/AutoAWQ cd AutoAWQ pip install -e .
getting the same issue…
This is probably a pytorch extensions error, it’s caused by the fact that when wheels (and kernels) are built with a specific pytorch version, any other version will probably not work. I noticed this when the new AutoAWQ kernels got released (0.0.3), my docker images built with previous pytorch 2.1.0 raised a similar error. I checked the kernels release date and relaized that it’s right after pytorch 2.2.0 release, updated my pytorch and then everything worked fine. @casper-hansen maybe pinning the pytorch version in AutoAWQ kernels can solve this ? but again only for cuda 12 🥲