crewAI: [BUG] Memory Rag Storage Issue

Description

Started getting this error regarding rag with the latest version 0.83.0

Steps to Reproduce

Just using the default crew configuration with memory=True.

Expected behavior

no errors

Screenshots/Code snippets

    return Crew(
        agents=agents,
        tasks=tasks,
        manager_agent=create_manager_agent(),
        process=Process.sequential,
        memory=True,
        verbose=crewai_verbose,
        **kwargs
    )

Operating System

Ubuntu 20.04

Python Version

3.10

crewAI Version

0.83.0

crewAI Tools Version

0.14.0

Virtual Environment

Conda

Evidence

2024-11-27 10:42:36,152 - 13765931008 - rag_storage.py-rag_storage:118 - ERROR: Error during short_term search: APIStatusError.__init__() missing 2 required keyword-only arguments: 'response' and 'body'
2024-11-27 10:42:36,457 - 13765931008 - rag_storage.py-rag_storage:118 - ERROR: Error during entities search: APIStatusError.__init__() missing 2 required keyword-only arguments: 'response' and 'body'

Possible Solution

Not sure currently

Additional context

Nothing else

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 7
  • Comments: 35

Most upvoted comments

the same with memory based on Azure OpenAI:

embedder_config = {
    "provider": "azure",
    "config": {
        "api_key": os.getenv("AZURE_OPENAI_API_KEY"),
        "api_base": os.getenv("AZURE_OPENAI_ENDPOINT"),
        "api_version": os.getenv("AZURE_API_VERSION"),
        "model_name": os.getenv("AZURE_OPENAI_EMBEDDED_MODEL"),
        "deployment_id": os.getenv("AZURE_OPENAI_EMBEDDED_DEPLOYMENT"),
    }
}
crew = Crew(
...
embedder=embedder_config
...

error:

2024-11-27 23:46:04,920 - 17808 - rag_storage.py-rag_storage:89 - ERROR: Error during short_term save: APIStatusError.__init__() missing 2 required keyword-only arguments: 'response' and 'body'
2024-11-27 23:46:26,410 - 17808 - rag_storage.py-rag_storage:89 - ERROR: Error during entities save: APIStatusError.__init__() missing 2 required keyword-only arguments: 'response' and 'body'

in my case i had to change embedder config to:

embedder_config = {
    "provider": "azure",
    "config": {
        "api_key": os.getenv("AZURE_OPENAI_API_KEY"),
        "api_base": os.getenv("AZURE_OPENAI_ENDPOINT"),
        "api_version": os.getenv("AZURE_API_VERSION"),
        "model_name": 'text-embedding-ada-002',
        "model": os.getenv("AZURE_OPENAI_EMBEDDED_DEPLOYMENT"),
    }
}

So deployment_id was changed to model - name of deployment made on Azure OpenAI.

``> @joaomdmoura @bhancockio I solved this annoying BUG and it took me around 12+ hrs to debug. You can close this issue

Note: this works for Ollama embedding model, so one can use ollama pull nomic-embed-text to pull the model or any other model to solve (RECOMMENDED)

The error was happening due to incorrect configuration of the embedder attribute in the Crew class, just change the model name as per your need just make sure not include any tag with it.

crew = Crew( agents=[sales_rep_agent,lead_sales_rep_agent], tasks=[lead_profiling_task,personalized_outreach_task], verbose=True, process=Process.sequential, memory=True, embedder={ “provider”: “ollama”, “config”: { “model”: “nomic-embed-text” } } ) For Google users, add the api_key attribute and change the model to models/text-embedding-004, note in between you still might face error like:

2025-01-19 00:10:58,441 - 20228 - rag_storage.py-rag_storage:109 - ERROR: Error during short_term save: 400 400 Request payload size exceeds the limit: 10000 bytes. in add.
2025-01-19 00:11:05,404 - 20228 - rag_storage.py-rag_storage:138 - ERROR: Error during short_term search: 400 400 Request payload size exceeds the limit: 10000 bytes. in query.
2025-01-19 00:11:06,654 - 20228 - rag_storage.py-rag_storage:138 - ERROR: Error during entities search: 400 400 Request payload size exceeds the limit: 10000 bytes. in query.

So it is recommended to use Ollama as an embedding provider(or openai). I highly RECOMMEND to use Ollama and ditch google’s text-embedding-004 for embedding. I hope this issue will be closed and my comment will be pinned for beginners 😃

PROOF of working:

Image

In my case I had to specify the “url”, I am using a docker instance and needed to point the url to “host.docker.internal:11434”

ollama_host = "http://" + os.environ.get("OLLAMA_HOST")

embedder={
        "provider": "ollama",
        "config": {
            "model": "mxbai-embed-large",
            "url": ollama_host + "/api/embeddings",
        }
    }

I have the same error with memory=True. crewai version 0.86.0 OS: Windows 11 pyhton: 3.12.6

@crew
  def crew(self) -> Crew:
      return Crew(
          agents=self.agents,
          tasks=self.tasks,
          #manager_llm=ChatOpenAI(temperature=0, model="gpt-4o-mini"),
          #process=Process.hierarchical,
          memory=True,
          verbose=True
      )
ERROR:root:Error during short_term save: APIStatusError.__init__() missing 2 required keyword-only arguments: 'response' and 'body'
ERROR:root:Error during entities save: APIStatusError.__init__() missing 2 required keyword-only arguments: 'response' and 'body'
ERROR:root:Error during entities save: APIStatusError.__init__() missing 2 required keyword-only arguments: 'response' and 'body'
ERROR:root:Error during entities save: APIStatusError.__init__() missing 2 required keyword-only arguments: 'response' and 'body'
ERROR:root:Error during short_term search: APIStatusError.__init__() missing 2 required keyword-only arguments: 'response' and 'body'
ERROR:root:Error during entities search: APIStatusError.__init__() missing 2 required keyword-only arguments: 'response' and 'body'

@messeiry Right now the only solution is to use OpenAI embeddings or the one I did above

@joaomdmoura @bhancockio I solved this annoying BUG and it took me around 12+ hrs to debug. You can close this issue

Note: this works for Ollama embedding model, so one can use ollama pull nomic-embed-text to pull the model or any other model to solve (RECOMMENDED)

The error was happening due to incorrect configuration of the embedder attribute in the Crew class, just change the model name as per your need just make sure not include any tag with it.

crew = Crew( agents=[sales_rep_agent,lead_sales_rep_agent], tasks=[lead_profiling_task,personalized_outreach_task], verbose=True, process=Process.sequential, memory=True, embedder={ “provider”: “ollama”, “config”: { “model”: “nomic-embed-text” } } )

This worked beautifully. Thank you.

the same with memory based on Azure OpenAI:

embedder_config = {
    "provider": "azure",
    "config": {
        "api_key": os.getenv("AZURE_OPENAI_API_KEY"),
        "api_base": os.getenv("AZURE_OPENAI_ENDPOINT"),
        "api_version": os.getenv("AZURE_API_VERSION"),
        "model_name": os.getenv("AZURE_OPENAI_EMBEDDED_MODEL"),
        "deployment_id": os.getenv("AZURE_OPENAI_EMBEDDED_DEPLOYMENT"),
    }
}
crew = Crew(
...
embedder=embedder_config
...

error:

2024-11-27 23:46:04,920 - 17808 - rag_storage.py-rag_storage:89 - ERROR: Error during short_term save: APIStatusError.__init__() missing 2 required keyword-only arguments: 'response' and 'body'
2024-11-27 23:46:26,410 - 17808 - rag_storage.py-rag_storage:89 - ERROR: Error during entities save: APIStatusError.__init__() missing 2 required keyword-only arguments: 'response' and 'body'

hey, could you please tell me if it’s resolved for you?

change deployment_id for model as i wrote here: https://github.com/crewAIInc/crewAI/issues/1669#issuecomment-2519951743

the same with memory based on Azure OpenAI:

embedder_config = {
    "provider": "azure",
    "config": {
        "api_key": os.getenv("AZURE_OPENAI_API_KEY"),
        "api_base": os.getenv("AZURE_OPENAI_ENDPOINT"),
        "api_version": os.getenv("AZURE_API_VERSION"),
        "model_name": os.getenv("AZURE_OPENAI_EMBEDDED_MODEL"),
        "deployment_id": os.getenv("AZURE_OPENAI_EMBEDDED_DEPLOYMENT"),
    }
}
crew = Crew(
...
embedder=embedder_config
...

error:

2024-11-27 23:46:04,920 - 17808 - rag_storage.py-rag_storage:89 - ERROR: Error during short_term save: APIStatusError.__init__() missing 2 required keyword-only arguments: 'response' and 'body'
2024-11-27 23:46:26,410 - 17808 - rag_storage.py-rag_storage:89 - ERROR: Error during entities save: APIStatusError.__init__() missing 2 required keyword-only arguments: 'response' and 'body'

hey, could you please tell me if it’s resolved for you?

I have tried the latest release 0.95.0, but have the same error as with previous versions 0.86.0

ERROR:root:Error during short_term search: APIStatusError.__init__() missing 2 required keyword-only arguments: 'response' and 'body'
ERROR:root:Error during entities search: APIStatusError.__init__() missing 2 required keyword-only arguments: 'response' and 'body'