velox: Fuzzer fails on expression with regex_replace Spark function
Bug description
To reproduce, run
spark_expression_fuzzer_test --enable_variadic_signatures --lazy_vector_generation_ratio 0.2 --velox_fuzzer_enable_column_reuse --velox_fuzzer_enable_expression_reuse --max_expression_trees_per_step 2 --retry_with_try --duration_sec 1800 --logtostderr=1 --minloglevel=0 --seed 532472015
I20231204 08:41:41.080447 420554 ExpressionFuzzerVerifier.cpp:451] ==============================> Started iteration 0 (seed: 532472015)
I20231204 08:41:41.085498 420554 ExpressionVerifier.cpp:97] Executing expression 0 : regex_replace("c0",least(o'TB-#q4~K"Xc5sX{^Js-nVOHUAo?w:-.]sC^d&5Sn'eLzMwg4chYq-'w_|t[Ec[Ht\bh#/n@!tV>vdf,sha1("c1"),null),regex_replace("c2",least(o'TB-#q4~K"Xc5sX{^Js-nVOHUAo?w:-.]sC^d&5Sn'eLzMwg4chYq-'w_|t[Ec[Ht\bh#/n@!tV>vdf,sha1("c1"),null),sha1("c1"),7820211085474237582),4480153776208645342)
I20231204 08:41:41.112323 420554 FuzzerToolkit.cpp:103] Exceptions match.
I20231204 08:41:41.112435 420554 ExpressionFuzzerVerifier.cpp:501] Both paths failed with compatible exceptions. Retrying expression using try().
I20231204 08:41:41.112671 420554 ExpressionVerifier.cpp:97] Executing expression 0 : try(regex_replace("c0",least(o'TB-#q4~K"Xc5sX{^Js-nVOHUAo?w:-.]sC^d&5Sn'eLzMwg4chYq-'w_|t[Ec[Ht\bh#/n@!tV>vdf,sha1("c1"),null),regex_replace("c2",least(o'TB-#q4~K"Xc5sX{^Js-nVOHUAo?w:-.]sC^d&5Sn'eLzMwg4chYq-'w_|t[Ec[Ht\bh#/n@!tV>vdf,sha1("c1"),null),sha1("c1"),7820211085474237582),4480153776208645342))
E20231204 08:41:41.149097 420554 Exceptions.h:69] Line: /Users/mbasmanova/cpp/velox-1/velox/expression/tests/ExpressionVerifier.cpp:82, Function:operator(), Expression: left->equalValueAt(right.get(), row, row) Different results at idx '4': 'null' vs. 'fk2WNxd8Hs'DlQb}w{o>FN/w)jhf9u8_cZ6|i')KLA1l6vF-IKPY`)38TqAS7.rlzt', Source: RUNTIME, ErrorCode: INVALID_STATE
I20231204 08:41:41.149194 420554 ExpressionVerifier.cpp:263] Skipping persistence because repro path is empty.
libc++abi: terminating due to uncaught exception of type facebook::velox::VeloxRuntimeError: Exception: VeloxRuntimeError
Error Source: RUNTIME
Error Code: INVALID_STATE
Reason: Different results at idx '4': 'null' vs. 'fk2WNxd8Hs'DlQb}w{o>FN/w)jhf9u8_cZ6|i')KLA1l6vF-IKPY`)38TqAS7.rlzt'
Retriable: False
Expression: left->equalValueAt(right.get(), row, row)
Function: operator()
File: /Users/mbasmanova/cpp/velox-1/velox/expression/tests/ExpressionVerifier.cpp
Line: 82
System information
n/a
Relevant logs
No response
About this issue
- Original URL
- State: open
- Created 7 months ago
- Comments: 20 (20 by maintainers)
Commits related to this issue
- Remove sparksql regex_replace from registry Summary: This function is failing fuzzer (https://github.com/facebookincubator/velox/issues/7856), remove it from registry until the author fix it. Also t... — committed to Yuhta/velox by Yuhta 7 months ago
- Remove sparksql regex_replace from registry Summary: This function is failing fuzzer (https://github.com/facebookincubator/velox/issues/7856), remove it from registry until the author fix it. Also t... — committed to Yuhta/velox by Yuhta 7 months ago
- Remove sparksql regex_replace from registry (#8076) Summary: Pull Request resolved: https://github.com/facebookincubator/velox/pull/8076 This function is failing fuzzer (https://github.com/facebooki... — committed to Yuhta/velox by Yuhta 7 months ago
- Remove sparksql regex_replace from registry (#8076) Summary: Pull Request resolved: https://github.com/facebookincubator/velox/pull/8076 This function is failing fuzzer (https://github.com/facebooki... — committed to facebookincubator/velox by Yuhta 6 months ago
- Optimize and re-register regexp_replace Spark function (#8333) Summary: Addresses https://github.com/facebookincubator/velox/issues/7856 and https://github.com/facebookincubator/velox/issues/8438 ht... — committed to facebookincubator/velox by codyschierbeck 3 months ago
Hi @kagamiori, I would love to help out. It’s a part of the codebase that I am not familiar with, but your explanation in the https://github.com/facebookincubator/velox/issues/8157 is a great place for me to start.
I’ll get started tomorrow.
@mbasmanova @kagamiori
I have discovered what is going wrong, but I am still working on a solution.
It appears the underlying buffer pointer for “pattern” is being modified in the code at some point. So when the fuzzer passes in the buffer on its second and third attempts, it is passing in different input and thus we are getting different output. I am trying to find what part of the code is making these modifications because as far as I can tell everything is being passed as constant references.
Here are some screenshots that show this modification.
First fuzzer pass:
Second fuzzer pass:
Not included in these screenshots is the value of pattern.size_, which is 21. The value of pattern found in the second image occurs at index 22 of the first image, it appears the buffer is being iterated by exactly “size_” with each call.
@codyschierbeck We will remove it from registry while the function name & fuzzer is being fixed, is that ok for you?