milvus: [Bug]: Query results length is not as expected after upgrading
Is there an existing issue for this?
- I have searched the existing issues
Environment
- Milvus version: 2.1.4--> master-20221110-174310a1
- Deployment mode(standalone or cluster): cluster
- SDK version(e.g. pymilvus v2.0.0rc2):
- OS(Ubuntu or CentOS):
- CPU/Memory:
- GPU:
- Others:
Current Behavior
The code of query
t0 = time.time()
expr = "count in [2,4,6,8]"
output_fields = ["count", "random_value"]
res = c.query(expr, output_fields, timeout=120)
sorted_res = sorted(res, key=lambda k: k['count'])
for r in sorted_res:
print(r)
t1 = time.time()
assert len(res) == 4
The query results length should be 4, but the length of the actual result is only 2
[2022-11-10T09:43:09.896Z] collection name: task_5_IVF_PQ
[2022-11-10T09:43:09.896Z] load collection
[2022-11-10T09:43:09.896Z] release collection before load if replicas > 1
[2022-11-10T09:43:09.896Z] release time: 2.4116
[2022-11-10T09:43:09.896Z] load time: 3.5330
[2022-11-10T09:43:09.896Z] Replica groups:
[2022-11-10T09:43:09.896Z] - Group: <group_id:437275234056798363>, <group_nodes:(22, 23)>, <shards:[Shard: <channel_name:by-dev-rootcoord-dml_130_437275061354758145v0>, <shard_leader:22>, <shard_nodes:[22, 22, 22, 22, 22]>, Shard: <channel_name:by-dev-rootcoord-dml_131_437275061354758145v1>, <shard_leader:23>, <shard_nodes:[23, 22, 22, 22, 22]>]>
[2022-11-10T09:43:09.896Z] - Group: <group_id:437275234056798364>, <group_nodes:(21,)>, <shards:[Shard: <channel_name:by-dev-rootcoord-dml_130_437275061354758145v0>, <shard_leader:21>, <shard_nodes:[21, 21, 21, 21, 21]>, Shard: <channel_name:by-dev-rootcoord-dml_131_437275061354758145v1>, <shard_leader:21>, <shard_nodes:[21, 21, 21, 21, 21]>]>
[2022-11-10T09:43:09.896Z] {'metric_type': 'L2', 'params': {'nprobe': 10}}
[2022-11-10T09:43:09.896Z]
[2022-11-10T09:43:09.896Z] Search...
[2022-11-10T09:43:09.896Z] (distance: 30.237220764160156, id: 851) -20.0
[2022-11-10T09:43:09.896Z] (distance: 30.29398536682129, id: 2753) -15.0
[2022-11-10T09:43:09.896Z] (distance: 30.85041618347168, id: 816) -17.0
[2022-11-10T09:43:09.896Z] (distance: 31.285930633544922, id: 1850) -20.0
[2022-11-10T09:43:09.896Z] (distance: 31.405452728271484, id: 595) -14.0
[2022-11-10T09:43:09.896Z] [851, 2753, 816, 1850, 595]
[2022-11-10T09:43:09.896Z] search latency: 0.0098s
[2022-11-10T09:43:09.896Z] {'random_value': -14.0, 'count': 2}
[2022-11-10T09:43:09.896Z] {'random_value': -12.0, 'count': 4}
[2022-11-10T09:43:09.896Z] Traceback (most recent call last):
[2022-11-10T09:43:09.896Z] File "scripts/action_after_upgrade.py", line 112, in <module>
[2022-11-10T09:43:09.896Z] task_5(data_size, host)
[2022-11-10T09:43:09.896Z] File "scripts/action_after_upgrade.py", line 89, in task_5
[2022-11-10T09:43:09.896Z] load_and_search(prefix, replicas=NUM_REPLICAS)
[2022-11-10T09:43:09.896Z] File "/home/jenkins/agent/workspace/tests/python_client/deploy/scripts/utils.py", line 217, in load_and_search
[2022-11-10T09:43:09.896Z] assert len(res) == 4
[2022-11-10T09:43:09.896Z] AssertionError
script returned exit code 1
Expected Behavior
[2022-11-10T09:43:09.896Z] {'count': 2, 'random_value': -18.0}
[2022-11-10T09:43:09.896Z] {'count': 4, 'random_value': -17.0}
[2022-11-10T09:43:09.896Z] {'count': 6, 'random_value': -11.0}
[2022-11-10T09:43:09.896Z] {'count': 8, 'random_value': -14.0}
[2022-11-10T09:43:09.896Z] query latency: 0.0111s
Steps To Reproduce
No response
Milvus Log

failed job: https://qa-jenkins.milvus.io/blue/organizations/jenkins/deploy_test/detail/deploy_test/623/pipeline/307 log: artifacts-pulsar-cluster-upgrade-623-server-logs.tar.gz artifacts-pulsar-cluster-upgrade-623-pytest-logs.tar.gz
Anything else?
No response
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 17 (17 by maintainers)
This issue is related to replica, query, balance, load/reduce, relatively complicated. I will upload a document illustrating this problem in more detail lately