(datafusion) 01/01: Update 53 upgrade guide to note release, other changes (#21449)

[gi...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch gh-readonly-queue/main/pr-21449-249c23c2b630fc1e43fc0de5e6ccc7d4c308a00b
in repository https://gitbox.apache.org/repos/asf/datafusion.git

commit 8d91fb0e233b540a323039dfb28a9f03984537f7
Author: Andrew Lamb <an...@nerdnetworks.org>
AuthorDate: Thu Apr 9 14:37:54 2026 -0400

    Update 53 upgrade guide to note release, other changes (#21449)
    
    ## Which issue does this PR close?
    
    - Related to https://github.com/apache/datafusion/issues/19692
    
    ## Rationale for this change
    
    @rluvaton noted some issues with the 53 upgrade guide
    https://github.com/apache/datafusion/issues/19692#issuecomment-4198963869:
    
    > FYI, the migration guide says 53.0.0 was not released yet and it miss
    the following breaking changes:
    >
    > https://github.com/apache/datafusion/pull/20319 - removed deprecated
    function from ExecutionPlan trait
    > https://github.com/apache/datafusion/pull/19635 - added argument to
    HashJoinExec::try_new function
    
    ## What changes are included in this PR?
    
    Fix the above
    
    ## Are these changes tested?
    Yes by CI
    
    ## Are there any user-facing changes?
    
    Better docs
---
 docs/source/library-user-guide/upgrading/53.0.0.md | 49 ++++++++++++++++++----
 1 file changed, 42 insertions(+), 7 deletions(-)

diff --git a/docs/source/library-user-guide/upgrading/53.0.0.md b/docs/source/library-user-guide/upgrading/53.0.0.md
index f616220778..f572b2a3f7 100644
--- a/docs/source/library-user-guide/upgrading/53.0.0.md
+++ b/docs/source/library-user-guide/upgrading/53.0.0.md
@@ -21,13 +21,6 @@
 
 ## DataFusion 53.0.0
 
-**Note:** DataFusion `53.0.0` has not been released yet. The information provided
-*in this section pertains to features and changes that have already been merged
-*to the main branch and are awaiting release in this version. See [#19692] for
-\*more details.
-
-[#19692]: https://github.com/apache/datafusion/issues/19692
-
 ### Upgrade arrow/parquet to 58.0.0 and object_store to 0.13.0
 
 DataFusion 53.0.0 uses `arrow` and `parquet` 58.0.0, and `object_store` 0.13.0.
@@ -39,6 +32,39 @@ See the [Arrow 58.0.0 release notes] and the [object_store 0.13.0 upgrade guide]
 [arrow 58.0.0 release notes]: https://github.com/apache/arrow-rs/releases/tag/58.0.0
 [object_store 0.13.0 upgrade guide]: https://github.com/apache/arrow-rs-object-store/blob/v0.13.0/CHANGELOG.md
 
+### `ExecutionPlan::statistics` removed
+
+The deprecated `ExecutionPlan::statistics()` method has been removed. If you
+implement custom `ExecutionPlan`s, remove that method from your impl and
+implement `partition_statistics()` instead.
+
+**Before:**
+
+```rust,ignore
+impl ExecutionPlan for MyExec {
+    // ...
+
+    fn statistics(&self) -> Result<Statistics> {
+        Ok(Statistics::new_unknown(&self.schema()))
+    }
+}
+```
+
+**After:**
+
+```rust,ignore
+impl ExecutionPlan for MyExec {
+    // ...
+
+    fn partition_statistics(&self, _partition: Option<usize>) -> Result<Statistics> {
+        Ok(Statistics::new_unknown(&self.schema()))
+    }
+}
+```
+
+If you do not have partition-specific statistics, return the same value for
+`None` and for any partition index.
+
 ### `ExecutionPlan::properties` now returns `&Arc<PlanProperties>`
 
 Now `ExecutionPlan::properties()` returns `&Arc<PlanProperties>` instead of a
@@ -125,6 +151,15 @@ let sub_plan = self.query_to_plan(subquery, planner_context)?;
 planner_context.pop_outer_query_schema();
 ```
 
+### `HashJoinExec::try_new` adds `null_aware`
+
+`HashJoinExec::try_new` now takes an extra `null_aware: bool` argument. This
+flag is used for null-aware anti joins, such as plans generated for `NOT IN`
+subqueries.
+
+Most callers should pass `false`, the previous behavior. Pass `true` only for null-aware
+`JoinType::LeftAnti` joins.
+
 ### `FileSinkConfig` adds `file_output_mode`
 
 `FileSinkConfig` now includes a `file_output_mode: FileOutputMode` field to control


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org