langchain: vectorstores error: "search_phase_execution_exceptionm" after using elastic search

Hi

I’m using elasticsearch as Vectorstores, just a simple call, but it’s reporting an error, I’ve called add_documents beforehand and it’s working. But calling similarity_search is giving me an error. Thanks for checking

Related Environment

docker >> image elasticsearch:7.17.0
python >> elasticsearch==7.17.0

Test code

from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.vectorstores import ElasticVectorSearch

if __name__ == "__main__":
    embeddings = OpenAIEmbeddings()
    elastic_vector_search = ElasticVectorSearch(
        elasticsearch_url="http://192.168.1.2:9200",
        index_name="test20222",
        embedding=embeddings
    )
    searchResult = elastic_vector_search.similarity_search("What are the characteristics of sharks")

Error

(.venv) apple@xMacBook-Pro ai-chain % python test.py
Traceback (most recent call last):
  File "/Users/apple/work/x/ai-chain/test.py", line 14, in <module>
    result = elastic_vector_search.client.search(index="test20222",query={
  File "/Users/apple/work/x/ai-chain/.venv/lib/python3.9/site-packages/elasticsearch/_sync/client/utils.py", line 414, in wrapped
    return api(*args, **kwargs)
  File "/Users/apple/work/x/ai-chain/.venv/lib/python3.9/site-packages/elasticsearch/_sync/client/__init__.py", line 3798, in search
    return self.perform_request(  # type: ignore[return-value]
  File "/Users/apple/work/x/ai-chain/.venv/lib/python3.9/site-packages/elasticsearch/_sync/client/_base.py", line 320, in perform_request
    raise HTTP_EXCEPTIONS.get(meta.status, ApiError)(
elasticsearch.BadRequestError: BadRequestError(400, 'search_phase_execution_exception', 'runtime error')

About this issue

Original URL
State: closed
Created a year ago
Comments: 21 (17 by maintainers)

Commits related to this issue

fix: elasticsearch (#2402) - Create a new docker-compose file to start an Elasticsearch instance for integration tests. - Add new tests to `test_elasticsearch.py` to verify Elasticsearch functiona... — committed to langchain-ai/langchain by sergerdn a year ago
feat: add pytest-vcr for recording HTTP interactions in integration tests (#2445) Using `pytest-vcr` in integration tests has several benefits. Firstly, it removes the need to mock external services... — committed to langchain-ai/langchain by sergerdn a year ago

Most upvoted comments

No, it was executed after adding the data and confirming that kibana saw the data, so I guess some key data initialization was missing that caused the inconsistency between the two structures

I believe you are correct. An index was created, but with incorrect mappings.

Right mappings:

{
  "mappings": {
    "properties": {
      "metadata": {
        "properties": {
          "source": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      },
      "text": {
        "type": "text"
      },
      "vector": {
        "type": "dense_vector",
        "dims": 1536
      }
    }
  }
}

Wrong mappings:

{
  "mappings": {
    "properties": {
      "metadata": {
        "properties": {
          "source": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      },
      "text": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "vector": {
        "type": "float"
      }
    }
  }
}

sergerdn on Apr 6, 2023

Okay, it seems that it is a bug that I have described above, which can be found at https://github.com/hwchase17/langchain/issues/2386#issuecomment-1495781790.

I am having difficulty updating the tests properly, as it will be more challenging for me than fixing the bug. Please be patient, I will work on fixing it.

sergerdn on Apr 4, 2023

Hi, I am currently using version 0.0.248 and I still encounter this issue. Is there any way that I can address this? Thank you

Baro1502 on Aug 2, 2023

@longgui0318

Thank you very much for your help. I have fixed it!

When it will be merged, please test it on your end to ensure that the new changes is working properly. https://github.com/hwchase17/langchain/pull/2445

sergerdn on Apr 6, 2023