datafusion: Attach `Diagnostic` to "wrong number of arguments" error
Is your feature request related to a problem or challenge?
For a query like:
SELECT sum(1, 2)
The only message that the end user of an application built atop of DataFusion sees is:
Error during planning: Execution error: User-defined coercion failed with Execution("SUM expects exactly one argument") No function matches the given name and argument types 'sum(Int64, Int64)'. You might need to add explicit type casts.
Candidate functions:
sum(UserDefined)
We want to provide a richer message that references and highlights locations in the original SQL query, and contextualises and helps the user understand the error. In the end, it would be possible to display errors in a fashion akin to what was enabled by #13664 for some errors:
See #14429 for more information.
Describe the solution you’d like
Attach a well crafted Diagnostic to the DataFusionError, building on top of the foundations laid in #13664. See #14429 for more information.
Describe alternatives you’ve considered
No response
Additional context
No response
About this issue
- Original URL
- State: open
- Created 2 years ago
- Comments: 15 (14 by maintainers)
Thank you for the care ❤️ Let me sync with @Chen-Yuan-Lai offline
@Chen-Yuan-Lai I only filed the ticket, but it’s @jsai28 working on that feature 😃 You’re definitely right though: these two tickets do have some overlap and we should make sure the resulting API makes sense overall.
In my personal view, #15276 is more about giving custom functions the ability to optionally implement a
diagnosefunction and optionally useDiagnostics ininvoke_with_argsto make pretty errors.But I think Datafusion should be able to return
Diagnostic-ed errors for “wrong number of arguments” even for all existing functions whoseSignaturealready brings the necessary information; without requiring them to implementdiagnose.For example, the
logfunction already today returns an error if you call it with three arguments:The goal of this ticket is to make similar errors
Diagnostic-ed.I think this ticket actually overlaps more with #14431 than with #15276 so perhaps you and @dentiny could align on it? 😊
The difficulty lies in
Signature::OneOfI think. For examplelogaccepts[Float]and[Float, Float]. So the error Datafusion returns today for[Int]is simply that it didn’t match any of the possibilities. While that’s correct, perhaps it’s not the best UX. Maybe that error message should be a fallback, more than anything.Proposal:
log(1,2,3), then we error out with “log cannot take 3 arguments”.log('foo')) then we error out with “log cannot take argument types [utf8]”Does that make sense? Sorry I realize that this must be very confusing. In general it’s going to be a bit difficult to map the concept of “arguments don’t match any of the available signatures” to a nicer user message.
My proposal is not really a complete plan but more of a spark to ignite a discussion together with @Chen-Yuan-Lai and @dentiny so we can find a good solution ❤️
Hi! I’m currently preparing my GSoC 2025 application and would love to contribute to this issue as a warm-up task. I understand this one involves attaching a
Diagnosticto the “wrong number of arguments” error, and I’m excited to explore how diagnostics are propagated in the codebase.I saw the previous discussion and understand there are some challenges with layered errors, but I’m happy to give it a try and ask questions as I go. Would it be okay if I start working on this?
Thanks in advance!
Hey @Chen-Yuan-Lai, thank you so much for your contributions 🙏 That is indeed an annoying issue. It seems like some of the error macros “flatten” the inner error to a string.
I think I’m in favour of option 2, in hope that there are not too many places where this flattening happens. I wouldn’t go for option 1 because, as you say, it would change all existing macro invocations.
Perhaps we want is a
DataFusionError::map_diagnosticed_errmethod? It would:&selfis aDataFusionError::Diagnostici.e. then you would do:
And now
err2would be aDiagnostic(ExecError)anderr3would be aDiagnostic(PlanError).I’m not sure if this would work or make for a good implementation, but I hope it can help in any way 😃
@eliaperantoni thanks for the advice! I will try that as soon as possible
Hey @Chen-Yuan-Lai, absolutely no problem! Was checking in to see if you needed any help 😊. I see, that’s a bit unfortunate. It seems like the first message is produced by a function called
take_function_argsand the second by one calledfunction_length_check.I see that
take_function_argsis receiving a lot of attention lately #14525. Perhaps you should attach aDiagnosticin the body of that function, and maybe you could even replace that wherefunction_length_checkis used, so that you can remove it?Then the problem would be to get the
Spantotake_function_args, but maybe you could make it take an optionalfunction_call_site: Option<Span>argument and then pass it in wherever possible, but without requiring all callers oftake_function_argsto pass it, because maybe sometimes it’s just being used for a quick assertion.That probably has something to do with the implementation of
DataFusionError::iterandDataFusionError::diagnostic: they might not be able to unwrap theDiagnosticfrom the layers that compose the error. Have you trieddbg!-ing the error that you get in the test, to see if it contains aDataFusionError::Diagnosticsomewhere? Perhaps the place where an error is created, that you attach aDiagnosticto, is not the first one that’s triggered for a bad query let’s say.take