llama_index: [Bug]: Opensearch Serverless throwing compile error when trying to filter using MetadataFilter object
Bug Description
Hello I am running into an odd situation where when I try to connect up to opensearch serverless in AWS and attempt to filter on some metadata, in my case a field called index_type, it returns the following error RequestError: RequestError(400, 'search_phase_execution_exception', 'compile error')
I spent some time pulling out the OpensearchVectorClient and OpensearchVectorStore classes since I didnt see a good way to extract the query it was producing, but once I did and logged the query it generated this is what it attempts to build which is breaking. I also ran this query directly against opensearch-py library and got the same error as above so I am guessing the query being generated is incorrect some how. If I just do a basic “scan” query and filter on the metadata.index_type.keyword field it returns fine but the llamaindex generated query seems off.
Here is the query it generates before it attempts to query Opensearch serverless under the hood. I removed the values in the embedding field to save on screen real estate. the following query is the one that creates that compile error.
{'size': 6, 'query': {'script_score': {'query': {'bool': {'filter': [{'bool': {'must': [{'term': {'metadata.index_type.keyword': 'smart_chunk'}}]}}]}}, 'script': {'source': "1/(1.0 + l2Squared(params.query_value, doc['embedding']))", 'lang': 'painless', 'params': {'field': 'embedding', 'query_value': []}}}}}
But this gets weirder when I modify the Metadatafilter key values to be metadata.index_type instead of just index_type it produces the following query which then runs fine but doesnt return any results, due to it not finding anything to match that filter I assume.
{'size': 6, 'query': {'script_score': {'query': {'bool': {'filter': [{'bool': {'must': [{'term': {' metadata.metadata.index_type.keyword': 'smart_chunk'}}]}}]}}, 'script': {'source': "1/(1.0 + l2Squared(params.query_value, doc['embedding']))", 'lang': 'painless', 'params': {'field': 'embedding', 'query_value': []}}}}}
I am at a loss as to best sort out where in the query it is wrong, need to get better at understanding OS query language, or if its something about the index? we used llamaindex to create the index so I would have assumed that was taken care of correctly under the hood there. Any thoughts or ideas? Also why is the OpensearchClient Async only? I cant seem to be able to run the client under a fastapi service layer due to event loop issues, but I can deal with that later.
Version
0.10.68
Steps to Reproduce
Here is the code that tries to call the vector store and retrieve results from opensearch
vector_store = OpensearchVectorStore(client=self.vector_client)
index = VectorStoreIndex.from_vector_store(vector_store)
smart_chunk_filters = MetadataFilters(
filters=[
MetadataFilter(key="index_type", value="smart_chunk"),
]
)
query_engine = index.as_retriever(similarity_top_k=k, filters=smart_chunk_filters)
node_postprocessors = MetadataReplacementPostProcessor(
target_metadata_key="window"
)
result = query_engine.retrieve(query)
Relevant Logs/Tracbacks
backend-1 | opensearchpy.exceptions.RequestError: RequestError(400, 'search_phase_execution_exception', 'compile error')
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 15 (8 by maintainers)
To resolve the compile error in your OpenSearch query when using the
MetadataFilterobject and thel2Squaredfunction, ensure that thevalueattribute of theMetadataFilterobject is correctly referenced in the_parse_filtermethod. The current implementation does not correctly referencefilter.valuewhen determining theequality_postfix.Here is the corrected code snippet for the
_parse_filtermethod:Additionally, ensure that your test setup and execution environment are correctly configured. For example, the
test_opensearch_client.pyfile from the LlamaIndex repository provides a comprehensive setup for testing OpenSearch integration, including starting OpenSearch locally, defining sample vectors, and running various tests. This setup can help you verify that your_parse_filtermethod works correctly in different scenarios [1][2].To continue talking to Dosu, mention @dosu.
The compile error you’re encountering is due to the
l2Squaredfunction not being supported in OpenSearch’s Painless scripting language. This is a known issue that can cause compile errors when used in your script.To resolve this, you need to use an alternative method for calculating the distance between vectors. One common approach is to manually implement the L2 distance calculation in the Painless script. Here is how you can modify your query to avoid using the
l2Squaredfunction:This script manually calculates the L2 distance by iterating over the elements of the query vector and the document vector, summing the squared differences, and then applying the same transformation as before.
Make sure to update your query generation code to use this new script. This should resolve the compile error and allow your query to execute correctly in AWS OpenSearch.
For reference, the issue with
l2Squarednot being supported in Painless scripting is documented in the LlamaIndex repository [1].To continue talking to Dosu, mention @dosu.
The compile error in your OpenSearch query is likely due to the incorrect handling of the
valueattribute in the_parse_filtermethod. Specifically, thevalueattribute is accessed without being defined, which can cause a compile error.Here is the problematic part of the code:
The
valueattribute is accessed in the line:To fix this, ensure that the
valueattribute is correctly accessed from thefilterobject. The corrected code should look like this:This ensures that the
valueattribute is correctly accessed and used in the filter parsing logic, which should resolve the compile error [1].Additionally, ensure that the
script_scorefunction is correctly implemented and integrated with the OpenSearch client. Thellama-index-legacy/llama_index/legacy/vector_stores/opensearch.pyfile from LlamaIndex provides a comprehensive set of functions and classes that can help verify the functionality of your OpenSearch integration. Make sure to review and utilize these functions, such as_default_painless_scripting_queryand_knn_search_query, to confirm that your implementation works as expected [2].To continue talking to Dosu, mention @dosu.
The compile error in your AWS OpenSearch query is due to the incorrect usage of the
l2Squaredfunction in the script. The correct syntax for using thel2Squaredfunction in a Painless script should be:Here is the corrected query:
This should resolve the compile error [1].
To continue talking to Dosu, mention @dosu.
The error
opensearchpy.exceptions.RequestError: RequestError(400, 'search_phase_execution_exception', 'compile error')is likely due to an issue in the query construction, specifically in theMetadataFilterparsing logic. Even though you have added the operator field to theMetadataFilter, the error suggests that there might be a problem with how the filter is being translated into an OpenSearch query.In the provided code, the
_parse_filtermethod is responsible for converting aMetadataFilterinto an OpenSearch query expression. Here is the relevant part of the code:Ensure that the
operatorfield in yourMetadataFilteris correctly set and that the value is one of the supportedFilterOperatorvalues. If the operator is not recognized, it will raise aValueError, which might be causing the compile error in the query.Additionally, verify that the
valuefield in theMetadataFilteris correctly formatted and appropriate for the specified operator. For example, if you are using a range operator (e.g.,GT,GTE), ensure that the value is a number.If the issue persists, you might want to add logging or print statements to debug the exact query being generated and sent to OpenSearch. This can help identify any discrepancies or formatting issues in the query.
For further reference, you can look at how the LlamaIndex framework handles similar filter transformations for Couchbase. The
_transform_couchbase_filter_operatorfunction converts standard metadata filter operators to Couchbase-specific filter operations, ensuring that the operators and values are correctly formatted for the target database:This approach ensures that the filter operators and values are correctly interpreted by the target database, reducing the likelihood of compile errors [1][2].
To continue talking to Dosu, mention @dosu.
The compile error you’re encountering when using the
MetadataFilterobject in Opensearch Serverless is likely due to incorrect usage or missing imports. Here is the correct way to use theMetadataFilterandMetadataFiltersobjects:Ensure you have the necessary imports and that the
MetadataFilterandMetadataFiltersobjects are correctly instantiated and used in your query. Additionally, make sure you are using the latest version of thellama-index-vector-stores-opensearchpackage, as recent updates have added support for fullMetadataFilters(all operators and nested filters) and removed the necessity to prefix filter keys with “metadata” [1][2].If you follow the above example and still encounter issues, please provide more specific details about the compile error message. Additionally, ensure that your
OpensearchVectorClientandOpensearchVectorStoreare correctly set up as shown in the documentation:This setup ensures that your Opensearch index is correctly configured and ready to handle vector queries with metadata filters [1][3].
For a more detailed example, you can refer to the Alibaba Cloud OpenSearch Vector Store documentation, which includes steps for setting up and querying with metadata filters [4].
To continue talking to Dosu, mention @dosu.