llama_index: [Bug]: AttributeError: 'HuggingFaceEmbedding' object has no attribute '_model'
Bug Description
After upgrading LlamaIndex to verison 0.10.36, my ingestion pipeline stopped working.
Version
llama-index-core 0.10.36 & llama-index-embeddings-huggingface 0.2.0
Steps to Reproduce
from llama_index.core import VectorStoreIndex
from llama_index.vector_stores.postgres import PGVectorStore
from llama_index.core import (
SimpleDirectoryReader,
StorageContext,
)
from llama_index.readers.file import PyMuPDFReader
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.core import Settings
from llama_index.llms.openai_like import OpenAILike
from llama_index.llms.vllm import VllmServer
from llama_index.core.node_parser import SentenceSplitter
from llama_index.core.ingestion import IngestionPipeline
# LlamaIndex embedding model
EMB_MODEL="BAAI/bge-base-en-v1.5" # For better results you can use the "large" variant
DEVICE="cuda:0" # If running out of GPU RAM, switch to "cpu" (although slower)
Settings.embed_model = HuggingFaceEmbedding(
model_name=EMB_MODEL,
device=DEVICE
)
# Create the pipeline with transformations
pipeline = IngestionPipeline(
transformations=[
SentenceSplitter(
chunk_size=1024,
chunk_overlap=20
),
Settings.embed_model,
]
)
Relevant Logs/Tracbacks
RemoteTraceback Traceback (most recent call last)
RemoteTraceback:
"""
Traceback (most recent call last):
File "/home/vmuser/miniconda3/envs/impv_rag/lib/python3.11/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
^^^^^^^^^^^^^^^^^^^
File "/home/vmuser/miniconda3/envs/impv_rag/lib/python3.11/multiprocessing/pool.py", line 51, in starmapstar
return list(itertools.starmap(args[0], args[1]))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vmuser/miniconda3/envs/impv_rag/lib/python3.11/site-packages/llama_index/core/ingestion/pipeline.py", line 124, in run_transformations
nodes = transform(nodes, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vmuser/miniconda3/envs/impv_rag/lib/python3.11/site-packages/llama_index/core/base/embeddings/base.py", line 450, in __call__
embeddings = self.get_text_embedding_batch(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vmuser/miniconda3/envs/impv_rag/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py", line 274, in wrapper
result = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/vmuser/miniconda3/envs/impv_rag/lib/python3.11/site-packages/llama_index/core/base/embeddings/base.py", line 341, in get_text_embedding_batch
embeddings = self._get_text_embeddings(cur_batch)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vmuser/miniconda3/envs/impv_rag/lib/python3.11/site-packages/llama_index/embeddings/huggingface/base.py", line 149, in _get_text_embeddings
return self._embed(texts, prompt_name="text")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vmuser/miniconda3/envs/impv_rag/lib/python3.11/site-packages/llama_index/embeddings/huggingface/base.py", line 124, in _embed
return self._model.encode(
^^^^^^^^^^^
AttributeError: 'HuggingFaceEmbedding' object has no attribute '_model'
"""
The above exception was the direct cause of the following exception:
AttributeError Traceback (most recent call last)
File <timed exec>:2
File ~/miniconda3/envs/impv_rag/lib/python3.11/site-packages/llama_index/core/ingestion/pipeline.py:722, in IngestionPipeline.run(self, show_progress, documents, nodes, cache_collection, in_place, store_doc_text, num_workers, **kwargs)
718 with multiprocessing.get_context("spawn").Pool(num_workers) as p:
719 node_batches = self._node_batcher(
720 num_batches=num_workers, nodes=nodes_to_run
721 )
--> 722 nodes_parallel = p.starmap(
723 run_transformations,
724 zip(
725 node_batches,
726 repeat(self.transformations),
727 repeat(in_place),
728 repeat(self.cache if not self.disable_cache else None),
729 repeat(cache_collection),
730 ),
731 )
732 nodes = reduce(lambda x, y: x + y, nodes_parallel, [])
733 else:
File ~/miniconda3/envs/impv_rag/lib/python3.11/multiprocessing/pool.py:375, in Pool.starmap(self, func, iterable, chunksize)
369 def starmap(self, func, iterable, chunksize=None):
370 '''
371 Like `map()` method but the elements of the `iterable` are expected to
372 be iterables as well and will be unpacked as arguments. Hence
373 `func` and (a, b) becomes func(a, b).
374 '''
--> 375 return self._map_async(func, iterable, starmapstar, chunksize).get()
File ~/miniconda3/envs/impv_rag/lib/python3.11/multiprocessing/pool.py:774, in ApplyResult.get(self, timeout)
772 return self._value
773 else:
--> 774 raise self._value
AttributeError: 'HuggingFaceEmbedding' object has no attribute '_model'
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 15
I think this issue should be reopened. Setting num_workers to 1 is not always desirable. In previous versions, it worked without any changes.
Multi processing requires pickling classes, and as you can imagine, pickling a local model isn’t really possible
@dosu Have you got an example about how the HuggingFaceEmbedding object need to be initialized in the current version of LlamaIndex? As I mentioned, my code was working before the upgrade.