llama_index: [Bug]: Odata filter in azure ai search vector store is not filtering the chunks
Bug Description
i have more than 1200 unique relative paths among total 50k relative paths needs to perform filter so while indexing in azure ai search vector store i passed metadatafields as relative_path and i used the existing index which can able to filter based on relative_path now as per the documentation i used this code below for performing folder filter which is doing great but as the relative_path or condition is getting increased iam getting this error1 Then to solve Error 1 i was trying odata filter string which allows me to pass of all relative paths with or condition so for that i just passed this parameter in index.as_retriever which is #vector_store_kwargs={“odata_filters”: odata_filters}, which is throwing the below error2
#Error 1 { “detail”: “() Invalid expression: Recursion depth exceeded allowed limit.\r\nParameter name: $filter\nCode: \nMessage: Invalid expression: Recursion depth exceeded allowed limit.\r\nParameter name: $filter” }
Logs of cmd of vscode
length of all paths59071 The length of cached_all_paths are 59071 The cached dynamic all possible folder are {‘Sample1’, ‘green’, ‘Support’, ‘Data’, ‘Base’, ‘paper’, ‘sample2’, ‘test_folder’} Folder: ‘Sample1’ contains 33 paths. Folder: ‘green’ contains 530 paths. Folder: ‘Support’ contains 603 paths. Folder: ‘Data’ contains 15 paths. Folder: ‘Base’ contains 6 paths. Folder: ‘paper’ contains 5 paths. Folder: ‘sample2’ contains 2 paths. Folder: ‘test_folder’ contains 4 paths. INFO: Application startup complete. INFO: 127.0.0.1:61238 - “GET /docs HTTP/1.1” 200 OK INFO: 127.0.0.1:61238 - “GET /openapi.json HTTP/1.1” 200 OK INFO: 127.0.0.1:61240 - “GET /folders HTTP/1.1” 200 OK these are all dynamic-all_possible_folders {‘Sample1’, ‘green’, ‘Support’, ‘Data’, ‘Base’, ‘paper’, ‘sample2’, ‘test_folder’} these are user selected_folders {‘Sample1’, ‘green’, ‘Support’, ‘Data’, ‘Base’, ‘paper’, ‘sample2’, ‘test_folder’}
if part used the if part matching _paths are non fall back search 59071 INFO: {“interactionid”: “0dc9ec68-6b50-4e4b-9bb2-d1f69bbc5cd9_1752040373”, “user_name”: “string”, “user_query”: “explain me about health score formula”, “execution_time_seconds”: 0.0, “response”: “An error occurred: () Invalid expression: Recursion depth exceeded allowed limit.\r\nParameter name: $filter\nCode: \nMessage: Invalid expression: Recursion depth exceeded allowed limit.\r\nParameter name: $filter” }
##Code
#####fetching relative_paths all
def fetch_all_relative_paths(): results = search_client.search(search_text=“*”, select=[“relative_path”]) all_paths = [doc[“relative_path”] for doc in results if “relative_path” in doc] print(f"length of all paths{len(all_paths)}") return all_paths
###vector_store
def get_vector_store():
metadata_fields = {
"file_name": ("file_name", MetadataIndexFieldType.STRING),
"relative_path": ("relative_path", MetadataIndexFieldType.STRING),
}
vector_store = AzureAISearchVectorStore(
search_or_index_client=search_client,
filterable_metadata_field_keys=metadata_fields,
index_management=IndexManagement.VALIDATE_INDEX,
id_field_key="id",
chunk_field_key="chunk",
embedding_field_key="embedding",
embedding_dimensionality=1536,
metadata_string_field_key="metadata",
doc_id_field_key="doc_id",
language_analyzer="en.lucene",
vector_algorithm_type="hnsw",
)
return vector_store
main.py Piece of code throwing error
=== Folder-specific search (if folders are selected) ===
all_paths = cached_all_paths
dynamic_all_possible_folders = cached_dynamic_all_possible_folders
print("these are all dynamic-all_possible_folders",dynamic_all_possible_folders)
user_selected_folders = set([f for f in request.folders if f])
print("these are user selected_folders",user_selected_folders)
if user_selected_folders == dynamic_all_possible_folders:
print("if part used")
matching_paths = all_paths
print("the if part matching _paths are non fall back search ",len(matching_paths))
folder_filter = MetadataFilters(
filters=[MetadataFilter(key="relative_path", value=path, operator=FilterOperator.EQ)
for path in matching_paths],
condition="or"
)
else:
print("else part used")
matching_paths = []
for folder in user_selected_folders:
# Use cached all_paths here
matching_paths.extend(cached_folder_paths_map.get(folder, []))
print("the else part matching _paths are non fall back search ",len(matching_paths))
odata_filters = " or ".join([
f"relative_path eq '{path}'" for path in sorted(set(matching_paths))
])
print(f"DEBUG: Generated odata_filters for selected folder: {odata_filters}")
retriever = index.as_retriever(
filters=folder_filter,
similarity_top_k=50,
vector_store_query_mode=VectorStoreQueryMode.SEMANTIC_HYBRID,
verbose=False,
)
chat_engine = create_chat_engine(retriever, memory)
try:
start_time_for_response = time.time()
agent_response = chat_engine.chat(request.query)
Error 2
{ “detail”: “Session.request() got an unexpected keyword argument ‘odata_filters’” }
code is same for the error 2 also but i just removed folder_filter in if condition and replaced with odatafilter like this so now odatafilter take the filter string which contains all relative_paths with or condition and passed it to the retriever with this parameter vector_store_kwargs={“odata_filters”: odata_filters} which is throwing me error 2
as per AzureAIsearch offical llama index documentation they are too passing the same parameter for odatafilter but its throwing me error so to prevent the error i just replace odatafilter with only filter in this parameter like this vector_store_kwargs={“filter”: odata_filters} now this is not throwing the error code is getting executed successfully without errors but filtering is not getting applied on only selected folders to limit the search space by this vector_store_kwargs={“filter”: odata_filters} when i passed to retriever , i will provide the below example which demonistrate the odatafilter is not filtering the ai search vector store
if user_selected_folders == dynamic_all_possible_folders: print(“if part used”) matching_paths = all_paths print("the if part matching _paths are non fall back search ",len(matching_paths))
odata_filters = " or ".join([
f"relative_path eq '{path}'" for path in sorted(set(matching_paths))
])
else: print(“else part used”) matching_paths = [] for folder in user_selected_folders: # Use cached all_paths here
matching_paths.extend(cached_folder_paths_map.get(folder, []))
print("the else part matching _paths are non fall back search ",len(matching_paths))
odata_filters = " or ".join([
f"relative_path eq '{path}'" for path in sorted(set(matching_paths))
])
print(f"DEBUG: Generated odata_filters for selected folder: {odata_filters}")
retriever = index.as_retriever(
similarity_top_k=50,
vector_store_query_mode=VectorStoreQueryMode.SEMANTIC_HYBRID,
vector_store_kwargs={"odata_filters": odata_filters},
verbose=False,
)
chat_engine = create_chat_engine(retriever, memory)
try:
start_time_for_response = time.time()
agent_response = chat_engine.chat(request.query)
Example
Actually i selected only few folders to test odata filter and asked the query of folder which is unselected but rather than getting response like iam sorry or something else by when it does not find relevant chunks to the llm , the llm is answering the unselected folder also, it means the unselected folder chunks got passed to the llm and its producing final answer so Error 1 folder filter is accurately performing filtering fwhen it got only few relative paths with or condition but it has limitation of passing all relative paths more recursively so i used odata filter to prevent the error error1 which is now not filtering the chunks as per metadata fields relative_paths
cmd logs of vscode
length of all paths59071 The length of cached_all_paths are 59071 The cached dynamic all possible folder are {‘Sample1’, ‘green’, ‘Support’, ‘Data’, ‘Base’, ‘paper’, ‘sample2’, ‘test_folder’} Folder: ‘Sample1’ contains 33 paths. Folder: ‘green’ contains 530 paths. Folder: ‘Support’ contains 603 paths. Folder: ‘Data’ contains 15 paths. Folder: ‘Base’ contains 6 paths. Folder: ‘paper’ contains 5 paths. Folder: ‘sample2’ contains 2 paths. Folder: ‘test_folder’ contains 4 paths. INFO: Application startup complete. INFO: 127.0.0.1:61238 - “GET /docs HTTP/1.1” 200 OK INFO: 127.0.0.1:61238 - “GET /openapi.json HTTP/1.1” 200 OK INFO: 127.0.0.1:61240 - “GET /folders HTTP/1.1” 200 OK these are all dynamic-all_possible_folders {‘Sample1’, ‘green’, ‘Support’, ‘Data’, ‘Base’, ‘paper’, ‘sample2’, ‘test_folder’} these are user selected_folders {‘Sample1’, ‘green’, ‘Support’, ‘Data’}
else part used the else part matching _paths are non fall back search 668
Fast api UI
{ “folders”: [ ‘Sample1’, ‘green’, ‘Support’, ‘Data’ ], “query”: “what is score formula”,
}
response by llm which it took from Base folder for answering but i did not select the folder at all
{
“answer”: “html\n<h1> Score Formula</h1>\n<p>The formula for calculating the Score is:</p>\n<p><strong>Score = 10 *x</sup></strong></p>\n”,
“sources”: [
{
“relative_path”: “Base/samp.docx”,
}
],
}
So answer me why odatafilter is not filtering chunks or limiting search space to only selected folder when filtering has unique relative paths which are filtereable by azure ai search vector store by putting or condition
Version
llama-index 0.12.47
Steps to Reproduce
Bug Report: odata_filters not limiting chunk filtering by relative_path in AzureAISearchVectorStore retriever
📋 Summary
When using a large number of relative_path values in the filter (e.g., >5000), the MetadataFilters object hits the recursion limit (Error 1). To avoid this, I switched to odata_filters, but the odata_filters query does not actually filter results as expected — chunks from non-selected folders still appear in the response.
Furthermore, the official documentation suggests using odata_filters, but it seems to be ignored during retrieval.
✅ Steps to Reproduce
-
Use
AzureAISearchVectorStorewith metadata fieldrelative_pathset as filterable. -
Have an index with ~12,00 unique
relative_pathvalues. -
In
index.as_retriever, set:vector_store_kwargs = { "odata_filters": " or ".join([ f"relative_path eq '{path}'" for path in selected_paths ]) } -
Query the retriever for something that only exists in non-selected paths (e.g., a file in a folder not included in
selected_paths). -
Observe that the LLM still retrieves and uses content from non-selected paths.
🤔 Expected Behavior
The retriever should strictly limit retrieved chunks to the subset of documents whose relative_path values match those in odata_filters, just like MetadataFilters does (before hitting recursion depth limit).
❌ Actual Behavior
- Chunks from non-matching
relative_pathvalues are retrieved. - The response contains content from folders the user did not select.
- This breaks the isolation logic intended to scope the RAG search to specific folders.
📄 Error Logs
When using MetadataFilters:
"detail": "() Invalid expression: Recursion depth exceeded allowed limit.\r\nParameter name: $filter"
When using odata_filters:
"answer": "<h1> Score Formula</h1>...",
"sources": [
{ "relative_path": "Base/samp.docx" } // This folder was *not* selected
]
💻 Code Snippet
odata_filters = " or ".join([
f"relative_path eq '{path}'" for path in selected_relative_paths
])
retriever = index.as_retriever(
similarity_top_k=50,
vector_store_query_mode=VectorStoreQueryMode.SEMANTIC_HYBRID,
vector_store_kwargs={"odata_filters": odata_filters}, # doesn't work as expected
verbose=False,
)
Note: Using
"filter": odata_filtersinstead of"odata_filters"does not raise an error but also does not apply filtering.
🧠 Additional Context
-
MetadataFiltersworks correctly but cannot handle more than a few hundredORconditions due to OData recursion limits. -
odata_filtersshould solve this limitation but instead:- Throws error (when passed as
odata_filters) - Is ignored (when passed as
filter)
- Throws error (when passed as
-
Documentation suggests
odata_filtersis valid invector_store_kwargs, but it does not behave as expected.
❓ Questions for Maintainers
- Is
odata_filtersofficially supported inAzureAISearchVectorStore.as_retriever(..., vector_store_kwargs=...)? - Is there a different or correct parameter name we should use?
- How can we apply large-scale filtering using metadata values like
relative_pathwithout hitting OData recursion limits?
Relevant Logs/Tracbacks
About this issue
- Original URL
- State: open
- Created a year ago
- Comments: 19
@dosu
output which confirms that my folder_path metadata is filterable by azure ai search index
Name: id, Type: Edm.String, Filterable: True, Sortable: False, Searchable: False Name: chunk, Type: Edm.String, Filterable: False, Sortable: False, Searchable: True Name: embedding, Type: Collection(Edm.Single), Filterable: False, Sortable: False, Searchable: True Name: metadata, Type: Edm.String, Filterable: False, Sortable: False, Searchable: False Name: doc_id, Type: Edm.String, Filterable: True, Sortable: False, Searchable: False Name: file_name, Type: Edm.String, Filterable: True, Sortable: False, Searchable: False Name: relative_path, Type: Edm.String, Filterable: True, Sortable: False, Searchable: False Name: folder_path, Type: Edm.String, Filterable: True, Sortable: False, Searchable: False
i have printed available folder_paths which acknowledges that each chunks as its respected folder_path metadata associated with it Set of all unique relative paths: Presentation/hart_modules/pre_deep_nested Q&A/FY 2015 Presentation/hart_modules Q&A/complex_customer Presentation/client_procedures Q&A
i have assigned parameter verbose=True still iam getting mix of chunks instead limiting to selected folder chunks
why the filtering is limitng the chunks i provided you every data i have why still please resolve this issue
present code USER_QUERY = “what is config.py” parent_folder = ‘Presentation’
odata_filters = f"startswith(folder_path, ‘{parent_folder}’)"
retriever = index.as_retriever( vector_store_kwargs={“filter”: odata_filters}, VectorStoreQueryMode=VectorStoreQueryMode.SEMANTIC_HYBRID, verbose=True, similarity_top_k=50, )
response = retriever.retrieve(USER_QUERY)
context_text = “\n”.join( f"[Parent_Folder_name: {node.metadata.get(‘folder_path’, ‘N/A’)}] " f"[Source: {node.metadata.get(‘relative_path’, ‘Unknown’)}]" f"[File: {node.metadata.get(‘file_name’, ‘Unknown’)}]" f"[Page: {node.metadata.get(‘page_number’, ‘N/A’)}] " f"{node.text}" for node_with_score in response for node in [node_with_score.node] )
print(context_text)
prompt = f"""Answer the question based on the following content:
{context_text}
Question: {USER_QUERY}“”"
messages = [ ChatMessage( role=MessageRole.SYSTEM, content=“Answer using only the retrieved context. Do not hallucinate. include parent_folder_name ,page_number always” ), ChatMessage(role=MessageRole.USER, content=prompt) ]
llm_response = llm.chat(messages)
print(“Final Answer:\n”, llm_response.message.content)
@dosu
Subject: How to filter by parent folder and retrieve documents from nested subfolders?
Description I’m currently indexing documents in LlamaIndex, extracting folder_path metadata using a custom function. My goal is to filter results by a parent folder (e.g., ‘Q&A’) and have the retriever return relevant chunks from files within that parent folder and its nested subfolders (e.g., ‘Q&A/complex_customer’).
Currently, using FilterOperator.EQ with folder_path=“Q&A” only retrieves chunks directly assigned to the ‘Q&A’ path, not those within its subdirectories like ‘Q&A/complex_customer’.
Setup Details I’m using SimpleDirectoryReader with recursive=True and a custom_file_metadata function to extract folder_path.
custom_file_metadata function:
Python
import os def custom_file_metadata(file_path): base_dir = “C:\NEW_ALL_DATASCIENCE_PROJECTS\Latest_deployment_opt_AI_search\Ovation_chatbot\Files\chatbot” relative_path = os.path.relpath(file_path, base_dir) folder_path = os.path.dirname(relative_path).replace(os.sep, “/”) return { “file_name”: os.path.basename(file_path), “relative_path”: relative_path, “folder_path”: folder_path # e.g., “Q&A” or “Q&A/complex_customer” } Indexing process:
Python
from llama_index.readers.simple import SimpleDirectoryReader from llama_index.readers.docling import DoclingReader from llama_index.node_parser.docling import DoclingNodeParser
Assuming necessary imports for index, retriever, etc. are present
reader = SimpleDirectoryReader( input_dir=“C:\NEW_ALL_DATASCIENCE_PROJECTS\Latest_deployment_opt_AI_search\Ovation_chatbot\Files\chatbot”, required_exts=[“.pdf”, “.docx”, “.pptx”], recursive=True, file_metadata=custom_file_metadata ) docling_reader = DoclingReader(export_type=DoclingReader.ExportType.JSON) docling_node_parser = DoclingNodeParser() documents = [] for file_path in reader.input_files: parsed_docs = docling_reader.load_data(file_path) nodes = docling_node_parser.get_nodes_from_documents(parsed_docs) for node in nodes: prov = node.metadata.get(‘doc_items’, [()])[0].get(‘prov’, []) if prov and ‘page_no’ in prov[0]: node.metadata[‘page_number’] = prov[0][‘page_no’] node.metadata.update(reader.file_metadata(file_path)) documents.extend(nodes)
Example folder_paths generated:
‘Presentation/hart_modules/pre_deep_nested’
‘Q&A/FY 2015’
‘Q&A/complex_customer’
‘Presentation/client_procedures’
‘Q&A’ (for files directly in the Q&A folder)
Problem When I try to filter using FilterOperator.EQ with selected_folder = ‘Q&A’, it only retrieves chunks whose folder_path is exactly ‘Q&A’. I want to retrieve chunks from all files within ‘Q&A’ and its subfolders, such as ‘Q&A/FY 2015’ and ‘Q&A/complex_customer’.
what filter condition or operator i should use as per azure ai search vector database
Current filtering logic:
Python
from llama_index.core.vector_stores import MetadataFilter, FilterOperator, MetadataFilters from llama_index.core.query_engine import VectorStoreQueryMode # Assuming this import
Assuming
indexis already defined fromdocumentsUSER_QUERY = “what are the all families values of all years from 1989 to 1998” selected_folder = ‘Q&A’ filters = MetadataFilters( filters=[ MetadataFilter(key=“folder_path”, value=selected_folder, operator=FilterOperator.EQ), ] ) retriever = index.as_retriever( VectorStoreQueryMode=VectorStoreQueryMode.SEMANTIC_HYBRID, verbose=True, similarity_top_k=50, filters=filters, ) response = retriever.retrieve(USER_QUERY)
and while performing indexing each folder and subfolder file chunks has their respected folder metadata example actually parent folder Q&A has this file sample_table_Q_A and those chunks we added folder_path metadata as Q&A and for the complex_customer folder file which is complex_customer_kba chunks we added folder path metadata as Q&A/complex_customer so how can i get these complex_customer_kba file chunks just by selecting only folder_path as Q&A how can i achieve this by filtering
Question How can I modify the filter condition or use a different operator/approach to achieve a broader filter where selecting a parent folder_path (e.g., ‘Q&A’) also includes all documents within its nested subfolders (e.g., ‘Q&A/complex_customer’)? Is there a “starts-with” or “contains” operator for metadata filtering, or a recommended way to handle hierarchical folder structures for filtering in LlamaIndex?
Which filters azure ai search supports does any filter of azure ai search achieves what iam looking for