OpenSearch: [BUG] "arithmetic_exception: long overflow" while running sort query on date field
Describe the bug
We are getting “arithmetic_exception: long overflow” intermittently when we sort the results on date field. This is the query we are using:
{
"query":{
"bool":{
"adjust_pure_negative":true,
"must":[
{
"match":{
"id":"7273617yygs2t7812ygsy"
}
},
{
"match":{
"scope":"2020"
}
},
{
"bool":{
"adjust_pure_negative":true,
"must_not":[
{
"match":{
"status":"COMPLETED"
}
}
],
"boost":1
}
}
],
"boost":1
}
},
"_source":{
"excludes":[
],
"includes":[
"dueDate",
"scope",
"id",
"status"
]
},
"sort":[
{
"dueDate":{
"order":"asc"
}
}
]
}
This is error message :
Related component
Search
To Reproduce
We are using this date format on dueDate field.
This is query
{
"query":{
"bool":{
"adjust_pure_negative":true,
"must":[
{
"match":{
"id": "7273617yygs2t7812ygsy"
}
},
{
"match":{
"scope": "2020"
}
},
{
"bool":{
"adjust_pure_negative":true,
"must_not":[
{
"match":{
"status": "COMPLETED"
}
}
],
"boost":1
}
}
],
"boost":1
}
},
"_source":{
"excludes":[
],
"includes":[
"dueDate",
"scope",
"id",
"status"
]
},
"sort":[
{
"dueDate":{
"order":"asc"
}
}
]
}
Note: Some of the documents/records that are inserted on the index do have value for dueDate field and some of them do.
We tried changing the date format to this yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis||strict_date_optional_time, but still we were getting arithmetic_exception.
Expected behavior
We want to sort on date field without having any issues.
Additional Details
Plugins Please list all plugins currently enabled.
Screenshots If applicable, add screenshots to help explain your problem.
Host/Environment (please complete the following information):
- OS: [e.g. iOS]
- Version [e.g. 22]
Additional context Add any other context about the problem here.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 1
- Comments: 16 (11 by maintainers)
@msfroh tagging you here since you’ve been looking into similar (or same?) issue before, may be missing some edge cases? Thanks @prudhvigodithi for looking into it.
Hello, everyone. @prudhvigodithi, I have tested again, using the same conditions shared for reproducing the error. Tried again with up to 20,000 documents with missing values for dueDate. From my end, the Arithmetic Exception was not triggered in any of the tests
Hello, @prudhvigodithi. I have cloned the fork, run the cluster, and tried to trigger the Arithmetic Exception. The exception was not triggered in any of the tests I ran. I tested with the provided conditions, ingesting increasing amounts of documents with missing dueDate values, up to 20,000 documents.
After multiple tries finally I was able to reproduce the error, I have created a small
error_hunter.shbash script for other to re-produce thisarithmetic_exception. Added some comments to the script to understand the flow.I can see this error across the
2.xline (2.18, 2.17, 2.15, 2,13) but not onmainbranch. I will debug further to find the root cause and will create a PR.Thank you.
Hello, everyone.
I replicated the issue by creating an index with different combinations of date formats for the dueDate field.
In my environment, the error was triggered when custom date formats were used (i.e., yyyy-MM-dd HH:mm:ss, yyyy-MM-dd), even when using each one of them alone.
When using “strict_date_optional_time” alone or paired with “epoch_millis” the issue did not show up.