presto: TPC-DS q29, q93, q97: VeloxRuntimeError: Partition index out of range

TPC-DS SF-10K:

VeloxRuntimeError:  Partition index out of range
	at Unknown.# 0  _ZN8facebook5velox7process10StackTraceC1Ei(Unknown Source)
	at Unknown.# 1  _ZN8facebook5velox14VeloxExceptionC2EPKcmS3_St17basic_string_viewIcSt11char_traitsIcEES7_S7_S7_bNS1_4TypeES7_(Unknown Source)
	at Unknown.# 2  _ZN8facebook5velox6detail14veloxCheckFailINS0_17VeloxRuntimeErrorEPKcEEvRKNS1_18VeloxCheckFailArgsET0_(Unknown Source)
	at Unknown.# 3  _ZN8facebook5velox4exec9HashTableILb0EE13partitionRowsERS3_(Unknown Source)
	at Unknown.# 4  _ZNSt17_Function_handlerIFSt10unique_ptrIbSt14default_deleteIbEEvEZN8facebook5velox4exec9HashTableILb0EE17parallelJoinBuildEvEUlvE0_E9_M_invokeERKSt9_Any_data(Unknown Source)
	at Unknown.# 5  _ZN8facebook5velox11AsyncSourceIbE7prepareEv(Unknown Source)
	at Unknown.# 6  _ZN5folly6detail8function14FunctionTraitsIFvvEEclEv(Unknown Source)
	at Unknown.# 7  _ZN5folly18ThreadPoolExecutor7runTaskERKSt10shared_ptrINS0_6ThreadEEONS0_4TaskE(Unknown Source)
	at Unknown.# 8  _ZN5folly21CPUThreadPoolExecutor9threadRunESt10shared_ptrINS_18ThreadPoolExecutor6ThreadEE(Unknown Source)
	at Unknown.# 9  _ZSt13__invoke_implIvRMN5folly18ThreadPoolExecutorEFvSt10shared_ptrINS1_6ThreadEEERPS1_JRS4_EET_St21__invoke_memfun_derefOT0_OT1_DpOT2_(Unknown Source)
	at Unknown.# 10 _ZSt8__invokeIRMN5folly18ThreadPoolExecutorEFvSt10shared_ptrINS1_6ThreadEEEJRPS1_RS4_EENSt15__invoke_resultIT_JDpT0_EE4typeEOSC_DpOSD_(Unknown Source)
	at Unknown.# 11 _ZNSt5_BindIFMN5folly18ThreadPoolExecutorEFvSt10shared_ptrINS1_6ThreadEEEPS1_S4_EE6__callIvJEJLm0ELm1EEEET_OSt5tupleIJDpT0_EESt12_Index_tupleIJXspT1_EEE(Unknown Source)
	at Unknown.# 12 _ZNSt5_BindIFMN5folly18ThreadPoolExecutorEFvSt10shared_ptrINS1_6ThreadEEEPS1_S4_EEclIJEvEET0_DpOT_(Unknown Source)
	at Unknown.# 13 _ZN5folly6detail8function14FunctionTraitsIFvvEE9callSmallISt5_BindIFMNS_18ThreadPoolExecutorEFvSt10shared_ptrINS7_6ThreadEEEPS7_SA_EEEEvRNS1_4DataE(Unknown Source)
	at Unknown.# 14 0x0000000000000000(Unknown Source)
	at Unknown.# 15 start_thread(Unknown Source)
	at Unknown.# 16 clone(Unknown Source)

In docker logs:

2023-06-14T23:06:18.827728539Z # 15 start_thread
2023-06-14T23:06:18.827742326Z # 16 clone
2023-06-14T23:06:18.910580946Z E0614 23:06:18.827117   870 HashTable.cpp:744] Error in async hash build: Exception: VeloxRuntimeError
2023-06-14T23:06:18.910614668Z Error Source: RUNTIME
2023-06-14T23:06:18.910650130Z Error Code: UNREACHABLE_CODE
2023-06-14T23:06:18.910684363Z Reason: Partition index out of range
2023-06-14T23:06:18.910707610Z Retriable: False
2023-06-14T23:06:18.910730478Z Function: findPartition
2023-06-14T23:06:18.910752809Z File: ../../velox/velox/exec/HashTable.cpp
2023-06-14T23:06:18.910773993Z Line: 864
2023-06-14T23:06:18.910787934Z Stack trace:
2023-06-14T23:06:18.910810939Z # 0  _ZN8facebook5velox7process10StackTraceC1Ei
2023-06-14T23:06:18.910834559Z # 1  _ZN8facebook5velox14VeloxExceptionC2EPKcmS3_St17basic_string_viewIcSt11char_traitsIcEES7_S7_S7_bNS1_4TypeES7_
2023-06-14T23:06:18.910872657Z # 2  _ZN8facebook5velox6detail14veloxCheckFailINS0_17VeloxRuntimeErrorEPKcEEvRKNS1_18VeloxCheckFailArgsET0_
2023-06-14T23:06:18.910887099Z # 3  _ZN8facebook5velox4exec9HashTableILb0EE13partitionRowsERS3_
2023-06-14T23:06:18.910901323Z # 4  _ZNSt17_Function_handlerIFSt10unique_ptrIbSt14default_deleteIbEEvEZN8facebook5velox4exec9HashTableILb0EE17parallelJoinBuildEvEUlvE0_E9_M_invokeERKSt9_Any_data
2023-06-14T23:06:18.910915709Z # 5  _ZN8facebook5velox11AsyncSourceIbE7prepareEv
2023-06-14T23:06:18.910929575Z # 6  _ZN5folly6detail8function14FunctionTraitsIFvvEEclEv
2023-06-14T23:06:18.910943482Z # 7  _ZN5folly18ThreadPoolExecutor7runTaskERKSt10shared_ptrINS0_6ThreadEEONS0_4TaskE
2023-06-14T23:06:18.910965741Z # 8  _ZN5folly21CPUThreadPoolExecutor9threadRunESt10shared_ptrINS_18ThreadPoolExecutor6ThreadEE
2023-06-14T23:06:18.910988587Z # 9  _ZSt13__invoke_implIvRMN5folly18ThreadPoolExecutorEFvSt10shared_ptrINS1_6ThreadEEERPS1_JRS4_EET_St21__invoke_memfun_derefOT0_OT1_DpOT2_
2023-06-14T23:06:18.911012963Z # 10 _ZSt8__invokeIRMN5folly18ThreadPoolExecutorEFvSt10shared_ptrINS1_6ThreadEEEJRPS1_RS4_EENSt15__invoke_resultIT_JDpT0_EE4typeEOSC_DpOSD_
2023-06-14T23:06:18.911032806Z # 11 _ZNSt5_BindIFMN5folly18ThreadPoolExecutorEFvSt10shared_ptrINS1_6ThreadEEEPS1_S4_EE6__callIvJEJLm0ELm1EEEET_OSt5tupleIJDpT0_EESt12_Index_tupleIJXspT1_EEE
2023-06-14T23:06:18.911047623Z # 12 _ZNSt5_BindIFMN5folly18ThreadPoolExecutorEFvSt10shared_ptrINS1_6ThreadEEEPS1_S4_EEclIJEvEET0_DpOT_
2023-06-14T23:06:18.911065954Z # 13 _ZN5folly6detail8function14FunctionTraitsIFvvEE9callSmallISt5_BindIFMNS_18ThreadPoolExecutorEFvSt10shared_ptrINS7_6ThreadEEEPS7_SA_EEEEvRNS1_4DataE
2023-06-14T23:06:18.911090859Z # 14 0x0000000000000000

About this issue

Original URL
State: closed
Created a year ago
Comments: 17 (15 by maintainers)

Most upvoted comments

@Yuhta : Thanks for the fix. Really appreciate it.

In the long run, a compilation time option would not work for us though. I don’t think we could have multiple builds based on query workload characteristic. The perf improvements should not be at a cost of correctness. I feel the build should use the int64 vector as a default.

What are Meta’s plans for this in the long term ? @mbasmanova

aditi-pandit on Jul 9, 2023