unsloth: LLVM ERROR: Cannot select: intrinsic %llvm.nvvm.shfl.sync.bfly.i32
Getting this error after conda installation
Full output
🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
WARNING[XFORMERS]: xFormers can't load C++/CUDA extensions. xFormers was built for:
PyTorch 2.3.0+cu121 with CUDA 1201 (you have 2.3.0)
Python 3.10.14 (you have 3.10.14)
Please reinstall xformers (see https://github.com/facebookresearch/xformers#installing-xformers)
Memory-efficient attention, SwiGLU, sparse and more won't be available.
Set XFORMERS_MORE_DETAILS=1 for more details
==((====))== Unsloth: Fast Llama patching release 2024.5
\\ /| GPU: NVIDIA GeForce GTX 1080. Max memory: 7.915 GB. Platform = Linux.
O^O/ \_/ \ Pytorch: 2.3.0. CUDA = 6.1. CUDA Toolkit = 11.8.
\ / Bfloat16 = FALSE. Xformers = 0.0.26.post1. FA = False.
"-____-" Free Apache license: http://github.com/unslothai/unsloth
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Unsloth 2024.5 patched 32 layers with 32 QKV layers, 32 O layers and 32 MLP layers.
Detected kernel version 5.4.0, which is below the recommended minimum of 5.5.0; this can cause the process to hang. It is recommended to upgrade the kernel to the minimum version or higher.
max_steps is given, it will override any value given in num_train_epochs
==((====))== Unsloth - 2x faster free finetuning | Num GPUs = 1
\\ /| Num examples = 210,289 | Num Epochs = 1
O^O/ \_/ \ Batch size per device = 2 | Gradient Accumulation steps = 4
\ / Total batch size = 8 | Total steps = 60
"-____-" Number of trainable parameters = 41,943,040
0%| | 0/60 [00:00<?, ?it/s]
LLVM ERROR: Cannot select: intrinsic %llvm.nvvm.shfl.sync.bfly.i32
Aborted
About this issue
- Original URL
- State: open
- Created 2 years ago
- Comments: 21 (8 by maintainers)
I wanted to share a working solution for a Docker setup that’s compatible with NVIDIA Pascal cards. I’ve tested this configuration on an NVIDIA P40 GPU, and it works well. Thx to comment by @davidngrc.
Below you’ll find the contents of the necessary files to set up this environment. You can copy these and create the files locally to use this setup.
1. Dockerfile
2. unsloth_env_file.yml
3. docker-compose.yml
Setup Instructions
To use this setup:
Create three files (
Dockerfile,unsloth_env_file.yml, anddocker-compose.yml) with the contents provided above.Ensure you have Docker and Docker Compose installed on your system.
Install the NVIDIA Container Toolkit for GPU support if you haven’t already.
Place all three files in the same directory.
Open a terminal and navigate to the directory containing these files.
Run the following command to build and start the container:
Once the container is running, access Jupyter Lab by opening a web browser and navigating to
http://localhost:8888.Key Features
Oh ok loll well itll take some time - we might take Triton out and make it an optional dependecy