ctransformers: GLIBC_2.29' not found

I am using the Linux system with GPU and installed ctransformers-0.2.14 using pip. It installed all fine. But now when I try to run the GGML model quantized by @TheBloke (TheBloke/upstage-llama-30b-instruct-2048-GGML), I am getting the following error.

In fact, I run any model, I get the following error. I have CUDA 12.2.

from ctransformers import AutoModelForCausalLM
llm = AutoModelForCausalLM.from_pretrained('TheBloke/upstage-llama-30b-instruct-2048-GGML', model_type='llama')
print(llm('AI is going to'))

ERROR: OSError: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by /home/ec2-user/.conda/envs/summr/lib/python3.10/site-packages/ctransformers/lib/cuda/libctransformers.so)

About this issue

Original URL
State: open
Created a year ago
Reactions: 12
Comments: 22 (1 by maintainers)

Most upvoted comments

I did actually, just rn

` # glibc error

        pip uninstall ctransformers

        git clone https://github.com/marella/ctransformers

        cd ctransformers

        CMAKE_ARGS="-DCT_CUBLAS=ON -DCT_INSTRUCTIONS=avx" pip install .

thistleknot on Sep 30, 2023

Can you please share the OS version. Also please run the following command and share its output:

ldd --version

If you are running it on an EC2 instance, please share the instance type and AMI name. It looks related to https://repost.aws/questions/QUrXOioL46RcCnFGyELJWKLw/glibc-2-27-on-amazon-linux-2 where Amazon Linux 2 instances use older glibc versions. Based on the answers in above post, using Amazon Linux 2023 should solve this issue. If you would like to use it on your current system, you can try building it from source:

pip uninstall ctransformers # uninstall if already installed
CT_CUBLAS=1 pip install ctransformers --no-binary ctransformers

marella on Jul 29, 2023

@thistleknot I am getting the same error - were you able to resolve this issue?

cmosguy on Sep 29, 2023

@vishnua2j Try with this command “pip install ctransformers --no-binary ctransformers --no-cache-dir” it may solve your problem.

khanjandharaiya on Jan 12, 2024

I’m not the maintainer, I was merely trying to help.

On Mon, Oct 2, 2023 at 5:55 PM Adam Klein @.***> wrote:

Thanks for the feedback, gosh that feels really hacky. Is there any other solution to fix the actual ctransformers package?

— Reply to this email directly, view it on GitHub https://github.com/marella/ctransformers/issues/58#issuecomment-1743995009, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHKKOSO5VHNJ2HN3ZUEGBTX5NPBPAVCNFSM6AAAAAA2TE6PCWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBTHE4TKMBQHE . You are receiving this because you were mentioned.Message ID: @.***>

thistleknot on Oct 3, 2023

I am facing the same error

rohithq on Jul 24, 2023