milvus: [Bug]: offset and limit params works not correct in some cases

Is there an existing issue for this?

  • I have searched the existing issues

Environment

- Milvus version: 2.2.11
- Deployment mode(standalone or cluster): Cluster on GCP
- MQ type(rocksmq, pulsar or kafka): Kafka      
- SDK version(e.g. pymilvus v2.0.0rc2): pymilvus 2.2.11
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

When performing a vector query using collection.query() , it seems the offset and limit params has a trange effect on the results.

Test data: Dataset with num_entities 35520 Dataset schema:

{ ‘auto_id’: False, ‘description’: ‘’, ‘fields’: [{ ‘name’: ‘id’, ‘description’: ‘’, ‘type’: < DataType.VARCHAR: 21 > , ‘params’: { ‘max_length’: 50 }, ‘is_primary’: True, ‘auto_id’: False }, { ‘name’: ‘text_field0’, ‘description’: ‘’, ‘type’: < DataType.VARCHAR: 21 > , ‘params’: { ‘max_length’: 50 } }, { ‘name’: ‘text_field1’, ‘description’: ‘’, ‘type’: < DataType.VARCHAR: 21 > , ‘params’: { ‘max_length’: 500 } }, { ‘name’: ‘text_field2’, ‘description’: ‘’, ‘type’: < DataType.VARCHAR: 21 > , ‘params’: { ‘max_length’: 500 }, ‘is_partition_key’: True }, { ‘name’: ‘images’, ‘description’: ‘’, ‘type’: < DataType.FLOAT_VECTOR: 101 > , ‘params’: { ‘dim’: 512 } } ] }

Examples: expression = “text_field1 != ’ '” limit = 30 offset = 30 result = dataset.query(output_fields=[‘text_field0’, ‘text_field1’, ‘text_field2’], expr=expression, limit=limit, offset=offset)

result is empty instead of 30 records.

expression = “text_field1 != ’ '” limit = 60 offset = 30 result = dataset.query(output_fields=[‘text_field0’, ‘text_field1’, ‘text_field2’], expr=expression, limit=limit, offset=offset)

result has 15 records instead of 60.

expression = “id in [list of 5 ids]” limit = 5 offset = 0 result = dataset.query(output_fields=[‘text_field0’, ‘text_field1’, ‘text_field2’], expr=expression, limit=limit, offset=offset)

result has 3 records instead of 5.

expression = “id in [list of 5 ids]” limit = 10 offset = 0 result = dataset.query(output_fields=[‘text_field0’, ‘text_field1’, ‘text_field2’], expr=expression, limit=limit, offset=offset)

result has 5 records instead of 10.

Expected Behavior

Milvus searchs offset+limit results and return limit results.

Steps To Reproduce

See examples

Milvus Log

No response

Anything else?

No response

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 19 (14 by maintainers)

Most upvoted comments

Hey Milvus team, do you have any estimation when this bug can be fixed?