pinot: Add exception to broker response when not all segments are available (partial response)
There are 3 scenarios when the broker response might be partial:
- There are some segments without online server in external view -
numUnavailableSegmentsinBaseBrokerRequestHandlerline 718 - Some segments are not available on server -
numSegmentsQueried > numSegmentsAcquiredinServerQueryExecutorV1Implline 170 - Some servers not responded -
numServersQueried > numServersRespondedinSingleConnectionBrokerRequestHandlerline 128
Currently these partial responses are only tracked by metrics/query stats, but not modeled as an exception. We should add an exception to the broker response to inform the users that the response might be partial
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 22 (22 by maintainers)
+1
I also suggest we change the default behavior to disallow partial response by default, and provide the flag to turn it on. In many use cases, data quality and correctness is a serious issue. So it’s ok to fail the query but not so to return a partial result silently.
I’d suggest adding different errors for each scenario described above to the
QueryException, and users can decide on how to proceed based on the error type (error code). We already have similar errors such asBROKER_REQUEST_SEND_ERROR_CODEwhich is added when a server cannot be reached