arrow-rs: `NullArray::is_null()` returns `false` incorrectly

Describe the bug Found by @jonahgao in https://github.com/apache/arrow-datafusion/pull/7568

To Reproduce

fn main() {
    let arr = NullArray::new(1);
    println!("arr.is_null(0): {}", arr.is_null(0))
}

Prints

arr.is_null(0): false

Expected behavior NullArray::is_null() should return true for all elements

Additional context

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 1
  • Comments: 15 (15 by maintainers)

Most upvoted comments

Maybe we can solve this with some additional documentation

I’m open to suggestions, the current docs are pretty clear imo - https://docs.rs/arrow-array/latest/arrow_array/array/trait.Array.html#method.is_null

https://docs.rs/arrow-array/latest/arrow_array/array/trait.Array.html#method.logical_nulls

I can’t imagine we could do better performance-wise than return true for NullArray::is_null()

But what should be do for the other arrays like dictionaries and run arrays, we need to be consistent as to what these methods mean. We can’t return logical nullability for some arrays and not others

The problem is doing this would be inconsistent with the values returned by Array::nulls() so the following would not hold

If this property doesn’t hold for some arrays it isn’t an invariant, is it?

It is not clear that making NullArray::is_null return such a confusingly and seeming incorrect answer helps people understand the nuance