llama_index: [Bug]: AttributeError: 'HuggingFaceEmbedding' object has no attribute '_model'

Bug Description

After upgrading LlamaIndex to verison 0.10.36, my ingestion pipeline stopped working.

Version

llama-index-core 0.10.36 & llama-index-embeddings-huggingface 0.2.0

Steps to Reproduce

from llama_index.core import VectorStoreIndex
from llama_index.vector_stores.postgres import PGVectorStore
from llama_index.core import (
    SimpleDirectoryReader,
    StorageContext,
)
from llama_index.readers.file import PyMuPDFReader
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.core import Settings
from llama_index.llms.openai_like import OpenAILike
from llama_index.llms.vllm import VllmServer
from llama_index.core.node_parser import SentenceSplitter
from llama_index.core.ingestion import IngestionPipeline


# LlamaIndex embedding model
EMB_MODEL="BAAI/bge-base-en-v1.5" # For better results you can use the "large" variant
DEVICE="cuda:0" # If running out of GPU RAM, switch to "cpu" (although slower)
Settings.embed_model = HuggingFaceEmbedding(
    model_name=EMB_MODEL,
    device=DEVICE
)



# Create the pipeline with transformations
pipeline = IngestionPipeline(
    transformations=[
        SentenceSplitter(
            chunk_size=1024,
            chunk_overlap=20
        ),
        Settings.embed_model,
    ]
)

Relevant Logs/Tracbacks

RemoteTraceback                           Traceback (most recent call last)
RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/vmuser/miniconda3/envs/impv_rag/lib/python3.11/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
                    ^^^^^^^^^^^^^^^^^^^
  File "/home/vmuser/miniconda3/envs/impv_rag/lib/python3.11/multiprocessing/pool.py", line 51, in starmapstar
    return list(itertools.starmap(args[0], args[1]))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vmuser/miniconda3/envs/impv_rag/lib/python3.11/site-packages/llama_index/core/ingestion/pipeline.py", line 124, in run_transformations
    nodes = transform(nodes, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vmuser/miniconda3/envs/impv_rag/lib/python3.11/site-packages/llama_index/core/base/embeddings/base.py", line 450, in __call__
    embeddings = self.get_text_embedding_batch(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vmuser/miniconda3/envs/impv_rag/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py", line 274, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/vmuser/miniconda3/envs/impv_rag/lib/python3.11/site-packages/llama_index/core/base/embeddings/base.py", line 341, in get_text_embedding_batch
    embeddings = self._get_text_embeddings(cur_batch)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vmuser/miniconda3/envs/impv_rag/lib/python3.11/site-packages/llama_index/embeddings/huggingface/base.py", line 149, in _get_text_embeddings
    return self._embed(texts, prompt_name="text")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vmuser/miniconda3/envs/impv_rag/lib/python3.11/site-packages/llama_index/embeddings/huggingface/base.py", line 124, in _embed
    return self._model.encode(
           ^^^^^^^^^^^
AttributeError: 'HuggingFaceEmbedding' object has no attribute '_model'
"""

The above exception was the direct cause of the following exception:

AttributeError                            Traceback (most recent call last)
File <timed exec>:2

File ~/miniconda3/envs/impv_rag/lib/python3.11/site-packages/llama_index/core/ingestion/pipeline.py:722, in IngestionPipeline.run(self, show_progress, documents, nodes, cache_collection, in_place, store_doc_text, num_workers, **kwargs)
    718     with multiprocessing.get_context("spawn").Pool(num_workers) as p:
    719         node_batches = self._node_batcher(
    720             num_batches=num_workers, nodes=nodes_to_run
    721         )
--> 722         nodes_parallel = p.starmap(
    723             run_transformations,
    724             zip(
    725                 node_batches,
    726                 repeat(self.transformations),
    727                 repeat(in_place),
    728                 repeat(self.cache if not self.disable_cache else None),
    729                 repeat(cache_collection),
    730             ),
    731         )
    732         nodes = reduce(lambda x, y: x + y, nodes_parallel, [])
    733 else:

File ~/miniconda3/envs/impv_rag/lib/python3.11/multiprocessing/pool.py:375, in Pool.starmap(self, func, iterable, chunksize)
    369 def starmap(self, func, iterable, chunksize=None):
    370     '''
    371     Like `map()` method but the elements of the `iterable` are expected to
    372     be iterables as well and will be unpacked as arguments. Hence
    373     `func` and (a, b) becomes func(a, b).
    374     '''
--> 375     return self._map_async(func, iterable, starmapstar, chunksize).get()

File ~/miniconda3/envs/impv_rag/lib/python3.11/multiprocessing/pool.py:774, in ApplyResult.get(self, timeout)
    772     return self._value
    773 else:
--> 774     raise self._value

AttributeError: 'HuggingFaceEmbedding' object has no attribute '_model'

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 15

Most upvoted comments

I think this issue should be reopened. Setting num_workers to 1 is not always desirable. In previous versions, it worked without any changes.

Multi processing requires pickling classes, and as you can imagine, pickling a local model isn’t really possible

@dosu Have you got an example about how the HuggingFaceEmbedding object need to be initialized in the current version of LlamaIndex? As I mentioned, my code was working before the upgrade.