Describe the bug
Using df.sort(col) orders nulls first, while DataFusion's default is to order nulls last (see datafusion.sql_parser.default_null_ordering). Ordering nulls last is also the behavior of sort_by and ctx.sql("... order by ...").
To Reproduce
import datafusion
ctx = datafusion.SessionContext()
ctx.sql("create table t (a int)").collect()
ctx.sql("insert into t values (1), (null), (3), (null), (2)").collect()
ctx.table("t").sort(datafusion.Expr.column("a")).show()
ctx.table("t").sort_by(datafusion.Expr.column("a")).show()
ctx.sql("select * from t order by a").show()
DataFrame()
+---+
| a |
+---+
| |
| |
| 1 |
| 2 |
| 3 |
+---+
DataFrame()
+---+
| a |
+---+
| 1 |
| 2 |
| 3 |
| |
| |
+---+
DataFrame()
+---+
| a |
+---+
| 1 |
| 2 |
| 3 |
| |
| |
+---+
Expected behavior
Follow the same behavior as the default.
Describe the bug
Using
df.sort(col)orders nulls first, while DataFusion's default is to order nulls last (seedatafusion.sql_parser.default_null_ordering). Ordering nulls last is also the behavior ofsort_byandctx.sql("... order by ...").To Reproduce
Expected behavior
Follow the same behavior as the default.