milvus: [Bug]: offset and limit params works not correct in some cases
Is there an existing issue for this?
- I have searched the existing issues
Environment
- Milvus version: 2.2.11
- Deployment mode(standalone or cluster): Cluster on GCP
- MQ type(rocksmq, pulsar or kafka): Kafka
- SDK version(e.g. pymilvus v2.0.0rc2): pymilvus 2.2.11
- OS(Ubuntu or CentOS):
- CPU/Memory:
- GPU:
- Others:
Current Behavior
When performing a vector query using collection.query() , it seems the offset and limit params has a trange effect on the results.
Test data: Dataset with num_entities 35520 Dataset schema:
{ ‘auto_id’: False, ‘description’: ‘’, ‘fields’: [{ ‘name’: ‘id’, ‘description’: ‘’, ‘type’: < DataType.VARCHAR: 21 > , ‘params’: { ‘max_length’: 50 }, ‘is_primary’: True, ‘auto_id’: False }, { ‘name’: ‘text_field0’, ‘description’: ‘’, ‘type’: < DataType.VARCHAR: 21 > , ‘params’: { ‘max_length’: 50 } }, { ‘name’: ‘text_field1’, ‘description’: ‘’, ‘type’: < DataType.VARCHAR: 21 > , ‘params’: { ‘max_length’: 500 } }, { ‘name’: ‘text_field2’, ‘description’: ‘’, ‘type’: < DataType.VARCHAR: 21 > , ‘params’: { ‘max_length’: 500 }, ‘is_partition_key’: True }, { ‘name’: ‘images’, ‘description’: ‘’, ‘type’: < DataType.FLOAT_VECTOR: 101 > , ‘params’: { ‘dim’: 512 } } ] }
Examples: expression = “text_field1 != ’ '” limit = 30 offset = 30 result = dataset.query(output_fields=[‘text_field0’, ‘text_field1’, ‘text_field2’], expr=expression, limit=limit, offset=offset)
result is empty instead of 30 records.
expression = “text_field1 != ’ '” limit = 60 offset = 30 result = dataset.query(output_fields=[‘text_field0’, ‘text_field1’, ‘text_field2’], expr=expression, limit=limit, offset=offset)
result has 15 records instead of 60.
expression = “id in [list of 5 ids]” limit = 5 offset = 0 result = dataset.query(output_fields=[‘text_field0’, ‘text_field1’, ‘text_field2’], expr=expression, limit=limit, offset=offset)
result has 3 records instead of 5.
expression = “id in [list of 5 ids]” limit = 10 offset = 0 result = dataset.query(output_fields=[‘text_field0’, ‘text_field1’, ‘text_field2’], expr=expression, limit=limit, offset=offset)
result has 5 records instead of 10.
Expected Behavior
Milvus searchs offset+limit results and return limit results.
Steps To Reproduce
See examples
Milvus Log
No response
Anything else?
No response
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 19 (14 by maintainers)
Hey Milvus team, do you have any estimation when this bug can be fixed?