ibis: bug: inconsistencies in results when exporting `to_pandas` and `to_pyarrow` when there are nulls in a not-nullable column

What happened?

Hello,

I am experiencing some strange behavior when performing a join between two tables:

Depending on the method (right or inner, vs outer or left), I am getting an empty output when calling to_pyarrow on the result of the latter two methods, whereas I am getting the expected result when calling to_pandas. On the other hand, the first two methods give the same result in arrow or pandas.

One of the tables I’m joining is a slightly complex expression with self joins and aggregations.

import ibis.deferred as ibd

conn.table("TABLE_2", schema="SCHEMA")
    .join(
        table_1_filtered, # result of various self-joins
        [
            ibd["date"] == table_1_filtered["date"],
            ibd["client"] == table_1_filtered["client"]
        ],
        how="outer"
    )

I get the same issue when calling .outer_join or left_join

What version of ibis are you using?

Latest 8.x.x branch

What backend(s) are you using, if any?

mssql

Relevant log output

I can provide additional info on tables if needed, though will need to anonymize the info to post it, so lmk if you think it would be relevant.

Code of Conduct

• I agree to follow this project’s Code of Conduct