(spark) branch master updated: [SPARK-56868][SQL][FOLLOW-UP] Clarify scaladoc for V2 runtime filter helpers

[we...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

cloud-fan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 992290bc09c3 [SPARK-56868][SQL][FOLLOW-UP] Clarify scaladoc for V2 runtime filter helpers
992290bc09c3 is described below

commit 992290bc09c3b0974e3ab804bf515ba37d5065ef
Author: Szehon Ho <sz...@gmail.com>
AuthorDate: Thu May 21 17:07:00 2026 +0800

    [SPARK-56868][SQL][FOLLOW-UP] Clarify scaladoc for V2 runtime filter helpers
    
    ### What changes were proposed in this pull request?
    
    Clarify scaladoc for `PushDownUtils.pushRuntimeFilters` and `PushDownUtils.replanWithRuntimeFilters` introduced in [SPARK-56868](https://issues.apache.org/jira/browse/SPARK-56868) ([#55887](https://github.com/apache/spark/pull/55887)):
    
    - Document the mutating-scan constraint on `pushRuntimeFilters`, where `SupportsRuntimeV2Filtering.filter` is invoked.
    - Replace the mixed Note/Precondition text on `replanWithRuntimeFilters` with a concise Notes bullet list.
    
    ### Why are the changes needed?
    
    The original helper scaladoc mixed concerns (execute-time requirements, caller caching, and preconditions) and referenced `pushRuntimeFilters` from the wrong method. This follow-up makes the documentation easier to read without changing behavior.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    Documentation-only change; existing tests cover runtime filter behavior.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    No.
    
    Closes #55958 from szehon-ho/spark-56868-cleanup.
    
    Authored-by: Szehon Ho <sz...@gmail.com>
    Signed-off-by: Wenchen Fan <we...@databricks.com>
---
 .../sql/execution/datasources/v2/PushDownUtils.scala     | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/PushDownUtils.scala b/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/PushDownUtils.scala
index dc6de6f29af9..c54cc98014d7 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/PushDownUtils.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/PushDownUtils.scala
@@ -141,6 +141,9 @@ object PushDownUtils extends Logging {
    * the first pass are used to derive PartitionPredicates in the second pass, avoiding duplicate
    * pushdown.
    *
+   * Note: Do not call multiple times for the same `scan` instance;
+   * [[SupportsRuntimeV2Filtering.filter]] is mutating.
+   *
    * @return true if any filters were pushed to the data source
    */
   def pushRuntimeFilters(
@@ -195,14 +198,11 @@ object PushDownUtils extends Logging {
    * preserved the original partitioning and pads with `None` to preserve key alignment with the
    * pre-filter partition set.
    *
-   * Must be called at execute time: runtime filters carry [[DynamicPruningExpression]] and
-   * scalar-subquery references whose values are only resolved after their broadcast/subquery
-   * side completes. The mutating [[pushRuntimeFilters]] call must run at most once per scan
-   * instance; callers are responsible for caching the result.
-   *
-   * Precondition: when `outputPartitioning` is a [[KeyedPartitioning]], every element of
-   * `originalPartitions` (and every partition re-planned by the data source) must implement
-   * [[HasPartitionKey]].
+   * Notes:
+   *  - Do not call multiple times for the same `scan` instance;
+   *    [[SupportsRuntimeV2Filtering.filter]] is mutating.
+   *  - When `outputPartitioning` is a [[KeyedPartitioning]], every split from
+   *    `planInputPartitions()` used on this path must implement [[HasPartitionKey]].
    *
    * @param scan                the V2 scan to push filters into
    * @param runtimeFilters      runtime filters to translate and push


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org