qdrant: Inconsistent results when querying with filters

Hello,

When executing the same query on the same index (unchanged) multiple times, the results are not consistent (not the same) like if the query execution wasn’t deterministic.

It’s very difficult in my context to identify the root cause (as I have something like 10+ must and must not filters in my query) but I think I found a minimal exemple that reproduce the issue (see below). It seems to be related to the filtering (didn’t notice the issue without filters in my query).

Note that the exemple give small variations (in term of score and occurence) but it my case it happens more frequently and sometime the score is very different (like 0.99 for the first execution of the query and -0.01 for the second execution for the top result).

Current Behavior

Results returned are not consistent, I don’t have the same results for each execution

Steps to Reproduce

1. Create an index and add random data using this Python script

from random import randint

import numpy as np
from qdrant_client import QdrantClient
from qdrant_client.http.models import VectorParams

qdrant_client: QdrantClient = QdrantClient('127.0.0.1', port=6333, grpc_port=6334, prefer_grpc=True)
qdrant_client.recreate_collection(
    collection_name='qdrant_issue',
    vectors_config=VectorParams(
        size=512,
        distance='Cosine'
    ),
    on_disk_payload=False
)
possible_languages = [
    ['fr'],
    ['es'],
    ['en'],
    ['fr', 'es'],
    ['fr', 'en'],
    ['es', 'en'],
    ['fr', 'es', 'en']
]
vectors = []
payloads = []
for i in range(100_000):
    vectors.append(np.random.uniform(0, 1, 512).tolist())
    payloads.append({
        'price': randint(0, 1000),
        'languages': possible_languages[randint(0, len(possible_languages) - 1)]
    })

qdrant_client.upload_collection(
    collection_name='qdrant_issue',
    vectors=vectors,
    payload=payloads,
    parallel=4
)

1. Run the following query (from dashboard UI) multiple time (like 20 and 30 times to see the variation sometimes)

POST collections/qdrant_issue/points/search
{
  "vector": [
    0.3710693418979645,
    0.06642328947782516,
    0.2810467481613159,
    0.3276609778404236,
    -0.494138240814209,
    0.469546914100647,
    -0.06745219230651855,
    -0.5583328604698181,
    0.5737641453742981,
    0.4667411148548126,
    0.3789732754230499,
    -0.38613274693489075,
    0.21777211129665375,
    0.3782398998737335,
    -0.17535682022571564,
    -0.03758164122700691,
    0.21561692655086517,
    0.19004908204078674,
    0.20649364590644836,
    0.5227827429771423,
    -0.21949194371700287,
    -0.609658420085907,
    -0.24385669827461243,
    0.4098377227783203,
    0.1374131590127945,
    0.2743236720561981,
    0.41385993361473083,
    -0.009203826077282429,
    0.44414022564888,
    0.2725498080253601,
    0.5890524387359619,
    0.39947938919067383,
    0.17512065172195435,
    -0.5291695594787598,
    -0.2502520680427551,
    -0.09344150125980377,
    -0.06048840656876564,
    0.48833906650543213,
    -0.54926598072052,
    -0.1934991329908371,
    0.24503175914287567,
    -0.36759766936302185,
    -0.12120242416858673,
    -0.3961760401725769,
    -0.1419602781534195,
    -0.188850998878479,
    0.4896467328071594,
    0.033387698233127594,
    -0.3955109119415283,
    -0.28260913491249084,
    0.03768128901720047,
    0.2803609073162079,
    0.09340129792690277,
    0.480867862701416,
    0.4721311628818512,
    0.44715842604637146,
    -0.4042094349861145,
    0.17255763709545135,
    -0.40601876378059387,
    0.06820644438266754,
    0.4697211682796478,
    0.37529537081718445,
    0.4232111871242523,
    -0.10453338921070099,
    -0.041331417858600616,
    0.22082379460334778,
    0.19731253385543823,
    0.19499245285987854,
    -0.23107758164405823,
    0.38675686717033386,
    -0.5106388926506042,
    -0.441979318857193,
    -0.3286953866481781,
    0.45299556851387024,
    0.5735219120979309,
    -0.48094677925109863,
    -0.40530064702033997,
    0.5109865069389343,
    0.5628646612167358,
    -0.1831115484237671,
    0.19255731999874115,
    0.5085017681121826,
    0.22506047785282135,
    -0.18357615172863007,
    -0.19840462505817413,
    -0.48293742537498474,
    0.31973475217819214,
    -0.19317026436328888,
    0.45976680517196655,
    0.1832597404718399,
    -0.2734256684780121,
    0.21361909806728363,
    -0.5652631521224976,
    -0.47899487614631653,
    -0.5182125568389893,
    0.21582023799419403,
    0.10821349173784256,
    -0.06147834286093712,
    -0.20216906070709229,
    0.3649701178073883,
    0.3071806728839874,
    0.2836838960647583,
    -0.09736554324626923,
    -0.524364173412323,
    -0.06411400437355042,
    0.3397457003593445,
    -0.41866421699523926,
    -0.32835888862609863,
    0.24862833321094513,
    0.4312220811843872,
    -0.17117567360401154,
    0.41165420413017273,
    -0.3240990936756134,
    -0.21373774111270905,
    0.011740312911570072,
    0.001962020294740796,
    0.20113226771354675,
    -0.4815387427806854,
    0.5897741317749023,
    0.33889853954315186,
    0.2394813746213913,
    -0.20469991862773895,
    -0.44459617137908936,
    -0.4091315269470215,
    0.1029294803738594,
    0.09159936755895615,
    0.15876410901546478,
    -0.5040865540504456,
    0.33774957060813904,
    -0.4341476857662201,
    -0.2279067039489746,
    -0.4204208254814148,
    0.02813168242573738,
    -0.0034707682207226753,
    -0.5011079907417297,
    -0.36517971754074097,
    -0.27571967244148254,
    0.23985733091831207,
    -0.22131754457950592,
    0.457368403673172,
    0.2597874701023102,
    -0.2914067208766937,
    -0.4266420602798462,
    0.2839112877845764,
    -0.14726409316062927,
    0.27810436487197876,
    0.12738099694252014,
    0.21796847879886627,
    0.28475627303123474,
    -0.014729432761669159,
    -0.43203094601631165,
    0.09629850834608078,
    0.3222324550151825,
    -0.5977743864059448,
    -0.23529309034347534,
    0.3358815312385559,
    -0.24901023507118225,
    0.08266223967075348,
    0.17324835062026978,
    -0.4760454595088959,
    0.24723045527935028,
    0.5028113722801208,
    0.06542569398880005,
    0.22501765191555023,
    -0.4366079866886139,
    0.11628778278827667,
    -0.14525337517261505,
    -0.39454078674316406,
    -0.3670209050178528,
    -0.40070033073425293,
    0.2533506155014038,
    0.2819904386997223,
    -0.12052939832210541,
    -0.5541168451309204,
    -0.5351543426513672,
    0.03961649164557457,
    0.4762345254421234,
    -0.16668589413166046,
    -0.3930352032184601,
    0.2957622706890106,
    -0.1186981126666069,
    -0.43253636360168457,
    0.5541649460792542,
    0.4702400267124176,
    0.24505382776260376,
    -0.37537673115730286,
    0.29184988141059875,
    0.2330573946237564,
    0.015326819382607937,
    0.22610585391521454,
    -0.2535123825073242,
    -0.3394162654876709,
    -0.40712109208106995,
    -0.46957436203956604,
    0.11691304296255112,
    -0.4147752523422241,
    0.393662691116333,
    -0.19320078194141388,
    -0.36137422919273376,
    -0.2333199381828308,
    0.2185746282339096,
    0.14688292145729065,
    0.2737440764904022,
    0.2624667286872864,
    0.4580828845500946,
    -0.11402329057455063,
    -0.47470802068710327,
    -0.3390156030654907,
    -0.005255725234746933,
    0.603532075881958,
    -0.41902637481689453,
    0.33064061403274536,
    -0.12663240730762482,
    -0.2885991930961609,
    -0.4161471426486969,
    0.2892448306083679,
    -0.2385479062795639,
    0.5033648610115051,
    -0.3708060383796692,
    -0.5479090809822083,
    -0.23156864941120148,
    -0.28989237546920776,
    0.3687075078487396,
    0.17985108494758606,
    0.0981837585568428,
    0.34839779138565063,
    0.4215945601463318,
    0.4452485144138336,
    -0.21525287628173828,
    -0.16504141688346863,
    -0.18654552102088928,
    0.5070935487747192,
    -0.3590397238731384,
    -0.01346271950751543,
    0.5883557200431824,
    -0.17709225416183472,
    0.3466012179851532,
    -0.22581566870212555,
    0.5030157566070557,
    0.5663403272628784,
    -0.4306854009628296,
    -0.24160942435264587,
    -0.26616713404655457,
    -0.17110127210617065,
    -0.4928770363330841,
    -0.2124696522951126,
    -0.3073999583721161,
    -0.45106077194213867,
    0.49604329466819763,
    -0.5252194404602051,
    0.40511804819107056,
    -0.174375519156456,
    -0.01153879426419735,
    0.05831225588917732,
    -0.4703374207019806,
    0.16403599083423615,
    -0.23077918589115143,
    -0.5042310953140259,
    0.3819766640663147,
    -0.13200107216835022,
    -0.28051891922950745,
    0.3440532088279724,
    0.021660858765244484,
    -0.05081227049231529,
    0.44980815052986145,
    -0.3431675434112549,
    -0.6024875640869141,
    0.44040945172309875,
    0.530968427658081,
    0.3361937701702118,
    -0.15851448476314545,
    0.0005797705962322652,
    0.11991538107395172,
    -0.5847152471542358,
    -0.2426372915506363,
    -0.2441706657409668,
    0.28669244050979614,
    -0.4061754047870636,
    -0.355652779340744,
    -0.3816211521625519,
    -0.07010698318481445,
    0.10076642781496048,
    0.39775824546813965,
    -0.25196096301078796,
    -0.08604700118303299,
    0.2874201834201813,
    -0.22195008397102356,
    0.2742477357387543,
    0.08676879107952118,
    -0.2809743583202362,
    0.08793063461780548,
    -0.0929449126124382,
    0.38596734404563904,
    -0.3892611861228943,
    0.5364879369735718,
    -0.5719305276870728,
    -0.10650250315666199,
    0.3110341429710388,
    -0.32905080914497375,
    0.18351607024669647,
    0.4214448630809784,
    0.10214200615882874,
    -0.2062770575284958,
    0.5013598799705505,
    -0.46007266640663147,
    0.3118341863155365,
    -0.18324995040893555,
    0.3800019323825836,
    -0.47918030619621277,
    0.43987202644348145,
    0.43580806255340576,
    0.20182137191295624,
    -0.17340834438800812,
    -0.45665639638900757,
    0.020332781597971916,
    0.4711951017379761,
    0.2715526819229126,
    -0.005494542885571718,
    -0.24079470336437225,
    0.47148397564888,
    0.3046533763408661,
    -0.36507129669189453,
    0.5213385224342346,
    -0.5565149188041687,
    -0.05611741915345192,
    -0.1021592989563942,
    0.37651291489601135,
    0.10140229016542435,
    0.2117488533258438,
    -0.48523643612861633,
    0.4346524775028229,
    -0.5334486961364746,
    -0.3006902039051056,
    0.2393081784248352,
    0.11824987083673477,
    0.13635829091072083,
    0.1274590939283371,
    0.36656633019447327,
    -0.4024696946144104,
    -0.06866607815027237,
    0.3843666613101959,
    -0.28440332412719727,
    -0.490739643573761,
    0.2925656735897064,
    -0.5079206228256226,
    0.1749175637960434,
    -0.41183003783226013,
    0.33096247911453247,
    0.38773009181022644,
    -0.11771278828382492,
    -0.4692631661891937,
    0.5228428244590759,
    0.3514528274536133,
    -0.47514772415161133,
    -0.1556738317012787,
    -0.09486229717731476,
    -0.4945237636566162,
    0.2584698498249054,
    0.3511196970939636,
    -0.10884279012680054,
    -0.17506985366344452,
    -0.5892227292060852,
    0.12465563416481018,
    -0.4375999867916107,
    -0.375838965177536,
    -0.026828071102499962,
    0.17912694811820984,
    0.033259790390729904,
    -0.22669856250286102,
    -0.4468538165092468,
    -0.28993505239486694,
    0.1830894500017166,
    -0.5758637189865112,
    0.08890960365533829,
    0.3340315520763397,
    -0.41574332118034363,
    0.46911269426345825,
    0.4583461284637451,
    0.40713125467300415,
    0.16072022914886475,
    0.38913336396217346,
    0.24672140181064606,
    0.22439509630203247,
    0.35975563526153564,
    0.49790769815444946,
    0.20908311009407043,
    -0.38461416959762573,
    -0.20063044130802155,
    0.22910483181476593,
    0.3068448007106781,
    -0.48460447788238525,
    -0.19395485520362854,
    0.4300477206707001,
    -0.4896939694881439,
    -0.059803079813718796,
    -0.5436601042747498,
    0.4849793314933777,
    -0.2965811789035797,
    -0.006951130926609039,
    0.31649190187454224,
    -0.4327892065048218,
    0.32259684801101685,
    -0.46070578694343567,
    0.2799196243286133,
    -0.3315366804599762,
    -0.5084371566772461,
    -0.4763864576816559,
    -0.5048060417175293,
    -0.42304256558418274,
    0.4224849343299866,
    0.07870078086853027,
    0.0671592727303505,
    -0.4817216396331787,
    0.48595741391181946,
    -0.29430899024009705,
    -0.3507298529148102,
    0.3033180832862854,
    -0.2841286063194275,
    -0.5314179062843323,
    0.3306940793991089,
    0.2627479135990143,
    0.2644122540950775,
    -0.012581211514770985,
    0.44302716851234436,
    0.4415266215801239,
    0.3880137801170349,
    -0.3092169761657715,
    -0.14105476438999176,
    -0.24590888619422913,
    0.07218480110168457,
    0.43385499715805054,
    -0.26493099331855774,
    0.42019641399383545,
    0.4744039475917816,
    0.20719872415065765,
    -0.10891271382570267,
    0.19471266865730286,
    0.166981041431427,
    -0.1153654232621193,
    0.16326428949832916,
    0.12045025080442429,
    -0.15714284777641296,
    0.35798463225364685,
    -0.1339985579252243,
    0.10156185179948807,
    0.3395368754863739,
    -0.5951840281486511,
    0.03629755973815918,
    0.32632482051849365,
    0.3587188720703125,
    -0.47592952847480774,
    0.4433431923389435,
    0.5005722641944885,
    -0.4020879864692688,
    -0.18998992443084717,
    -0.19025497138500214,
    0.17732374370098114,
    0.10072048008441925,
    0.36805856227874756,
    0.19115374982357025,
    -0.02854190208017826,
    0.287620484828949,
    0.056680042296648026,
    -0.2979012727737427,
    0.5821533203125,
    -0.21429066359996796,
    0.279401570558548,
    -0.2176329344511032,
    0.22281847894191742,
    -0.003370719263330102,
    -0.22053103148937225,
    -0.5627114176750183,
    0.5684434771537781,
    -0.09673493355512619,
    0.4843898415565491,
    -0.38741445541381836,
    0.39143794775009155,
    0.20813684165477753,
    -0.32795044779777527,
    -0.23728957772254944,
    -0.1308015137910843,
    -0.24011732637882233,
    -0.5175677537918091,
    -0.15964460372924805,
    0.33565473556518555,
    -0.3734162449836731,
    -0.016237366944551468,
    0.5922372937202454,
    -0.06207425519824028,
    0.3856859505176544,
    -0.14565140008926392,
    -0.09648784250020981,
    0.19736400246620178,
    0.3297894597053528,
    0.5224746465682983,
    0.46431979537010193,
    0.540341317653656,
    0.3546620309352875,
    0.4406343400478363,
    0.36092719435691833,
    -0.524186909198761,
    0.1004059761762619,
    0.16261401772499084,
    -0.3046920895576477,
    0.2742835283279419,
    -0.39581531286239624,
    -0.24131833016872406,
    0.42226678133010864,
    0.011217541061341763,
    -0.2443089634180069,
    0.42881155014038086,
    0.011226149275898933
  ],
    "filter": {
    "must": [
      {
        "key": "price",
        "range": {
          "gt": 0,
          "lt": 500
        }
      },
      {
        "key": "languages",
        "match": {
          "any": [
            "fr"
          ]
        }
      }
    ]
  },
  "limit": 1000,
  "with_payload": false,
  "with_vector": false
}

Note that the issue seems related to the combination between the range and the match filters. I think I didn’t see any issue without this combination

Expected Behavior

The results returned must always be the same, with the same scores

Possible Solution

I don’t have any but I could help your team to investigate further with my data if you want (the exemple use random data for privacy purposes)

Context (Environment)

• Issue reproduced using single node deployment (in local with the docker image):

docker pull qdrant/qdrant
docker run -p 6333:6333 -p 6334:6334 qdrant/qdrant

• Qdrant version is 1.3.2
• Note that my payload are stored in memory, I didn’t try using disk storage but in my case I need a very low latency and so memory storage
• Right now it impacts my application as the results are not consistent when reloading a page. Also I think it could impact the pagination (bring duplicates) as the query is executed each time and the results are different. Also, sometime the scores looks like very “broken”

Detailed Description

Possible Implementation

/