gpt4all: libllama.so: cannot open shared object file: No such file or directory
Hello, First, I used the python example of gpt4all inside an anaconda env on windows, and it worked very well. Then, I try to do the same on a raspberry pi 3B+ and then, it doesn’t work.
The issue is:
Traceback (most recent call last):
File "/home/pi/Desktop/venv/tst.py", line 1, in <module>
import gpt4all
File "/home/pi/Desktop/venv/lib/python3.9/site-packages/gpt4all/__init__.py", line 1, in <module>
from . import gpt4all # noqa
File "/home/pi/Desktop/venv/lib/python3.9/site-packages/gpt4all/gpt4all.py", line 6, in <module>
from . import pyllmodel
File "/home/pi/Desktop/venv/lib/python3.9/site-packages/gpt4all/pyllmodel.py", line 39, in <module>
llmodel, llama = load_llmodel_library()
File "/home/pi/Desktop/venv/lib/python3.9/site-packages/gpt4all/pyllmodel.py", line 32, in load_llmodel_library
llama_lib = ctypes.CDLL(llama_dir, mode=ctypes.RTLD_GLOBAL)
File "/usr/local/lib/python3.9/ctypes/__init__.py", line 374, in __init__
self._handle = _dlopen(self._name, mode)
OSError: /home/pi/Desktop/venv/lib/python3.9/site-packages/gpt4all/llmodel_DO_NOT_MODIFY/build/libllama.so: cannot open shared object file: No such file or directory
To be more specific, I created a venv and installed gpt4all inside, all went well according to pip.
a sudo find / libllama.so command seems not to find anything.
If you have a hint it would be nice.
Don’t hesitate to ask for more info if you need.
About this issue
- Original URL
- State: closed
- Created a year ago
- Reactions: 2
- Comments: 18
Its a bit weird for me. It works on my M1 MacBook native in python. But when I try to build a Docker Image, the image will build, but the container wont run. Error is the same as all the others here.
Sry for the delay, I was trying to setup a new computer to make my app run on it.
According to what I’ve read on the internet, it seems that the libllama.so need some dependencies and some specifications for ARM architectures. So, we need to recompile the library with the methode proposed in the wiki: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/README.md --> Local Build Instructions
but when doing so, it seems that some dependencies are wrong for ARM architectures.
To fix that we will change some files after cloning.
The first dependency that is wrong is immintrin.h that don’t work with ARM: In gpt4all\gpt4all-backend\llama.cpp-mainline\ggml.c at line 171 replace
by
This method was discovers thanks to https://github.com/coin-or/Clp/issues/127
Then, there is some other compiling things that to be honest I don’t understand but it seems that it’s more a syntax problem with the pi compiler. It seems it have been fixed in the last version, but if you are on older versions:
in gpt4all\gpt4all-backend\gptj.cpp around line 527 replace
by
also, in mtp.cpp around line 486 it the same, replace
by
Thanks to https://github.com/ggerganov/whisper.cpp/issues/11
Then mkdir build cd build cmake … cmake --build . --parallel as said in the docs.
It worked for me on a pi 3b+. If it don"t for you, maybe you need to define the good architecture in ggml.cpp.
Hope this helps!
I get “libllama.so: cannot open shared object file: No such file or directory” when trying to build a docker container running ubuntu:jammy.
I was able to run this by building gpt4all within the container.
Dockerfile
I tried many things, but unfortunately, it seems that it’s inside the llama.cpp code that there is some specific stuf that don’t work with ARM. When I try to buid on local, I have the error on the cmake command:
/home/pi/gpt4all/gpt4all-backend/llama.cpp/ggml.c:191:23: fatal error: immintrin.h: Aucun fichier ou dossier de ce type #include <immintrin.h>
(in french but saying no file found)
It seems that immintrin.h is quite specific to intel. I foud that there might be some replacement for that library https://github.com/coin-or/Clp/issues/127 for example but it would mean dive into the source code… I might try that but to be honest, I’m not sure if i will suceed.
edit on 18/05/2023: I managed to make the library compile for ARM architecture. I will provide my modifications at the end of the week-end. Now my python script starts correctly, it downloads the model at first launch, but unfortunately, my rasp doesn’t have enough ram to read the model (i think). I tried increase the swap file a lot (like 2Gb) but no mater the size of it, I allways have the save error:
gptj_model_load: n_vocab = 50400 gptj_model_load: n_ctx = 2048 gptj_model_load: n_embd = 4096 gptj_model_load: n_head = 16 gptj_model_load: n_layer = 28 gptj_model_load: n_rot = 64 gptj_model_load: f16 = 2 gptj_model_load: ggml ctx size = 0.07 MB ggml_new_tensor_impl: not enough space in the context’s memory pool (needed 129024156, available 72960) ggml_new_tensor_impl: not enough space in the context’s memory pool (needed 129057236, available 72960) ggml_new_tensor_impl: not enough space in the context’s memory pool (needed 234836, available 72960) ggml_new_tensor_impl: not enough space in the context’s memory pool (needed 10552076, available 72960) ggml_new_tensor_impl: not enough space in the context’s memory pool (needed 10552076, available 72960) ggml_new_tensor_impl: not enough space in the context’s memory pool (needed 10552076, available 72960) ggml_new_tensor_impl: not enough space in the context’s memory pool (needed 10552076, available 72960) ggml_new_tensor_impl: not enough space in the context’s memory pool (needed 42009356, available 72960) ggml_new_tensor_impl: not enough space in the context’s memory pool (needed 131852, available 72960) ggml_new_tensor_impl: not enough space in the context’s memory pool (needed 42009356, available 72960) ggml_new_tensor_impl: not enough space in the context’s memory pool (needed 82700, available 72960) ggml_new_tensor_impl: not enough space in the context’s memory pool (needed 82700, available 72960) ggml_new_tensor_impl: not enough space in the context’s memory pool (needed 82700, available 72960) and so on…
I guess it’s simply not possible to run this on a rasp 3. I will find an other computer.