[PR] [SPARK-59706][SQL] Hoist loop-invariant left row out of the AS-OF join inner scan [spark]

["david-mollitor-db (via GitHub)" <gi...@apache.org>]

david-mollitor-db opened a new pull request, #58959:
URL: https://github.com/apache/spark/pull/58959

   This backports #58879 (SPARK-59706) to `branch-4.x`. It is a clean cherry-pick of the master commit; the change is identical.
   
   ### What changes were proposed in this pull request?
   
   In `SortMergeAsOfJoinScanner`
   (`sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeAsOfJoinExec.scala`), the
   per-left-row scan over the buffered right group (`findBestBackwardForward` and
   `findBestForwardNearest`) rebinds both sides of the shared `JoinedRow` on every iteration via
   `joinedRow.withLeft(leftRow).withRight(rightRow)`. The left row is fixed for the duration of the
   scan, so this binds `joinedRow.withLeft(leftRow)` once before the loop and keeps only
   `joinedRow.withRight(rightRow)` inside it.
   
   ### Why are the changes needed?
   
   The inner scan runs once per buffered right row for every left row, so `withLeft` was called
   redundantly on each iteration with an argument that never changes within a scan. JFR profiling of
   `AsOfJoinBenchmark` showed `JoinedRow.withLeft`/`withRight` among the hotter frames in the
   (interpreted) scanner. Binding the left side once removes the redundant per-row call. This is
   behavior-preserving: `JoinedRow.withLeft` only stores the left-row reference, nothing mutates the
   left side within a scan, and `findNext` re-binds both sides before projecting the output row.
   
   ### Does this PR introduce _any_ user-facing change?
   
   No.
   
   ### How was this patch tested?
   
   Backport of #58879 (SPARK-59706); the change is identical to the master PR. Existing AS-OF join
   tests cover it (`SortMergeAsOfJoinSuite`, `DataFrameAsOfJoinSuite`, `AsOfJoinSQLSuite`, and
   `AsOfJoinSortMergeSQLSuite`), and `branch-4.x` CI runs them.
   
   ### Was this patch authored or co-authored using generative AI tooling?
   
   Generated-by: Isaac
   
   This pull request and its description were written by Isaac.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-59706][SQL] Hoist loop-invariant left row out of the AS-OF join inner scan [spark]

["david-mollitor-db (via GitHub)" <gi...@apache.org>]

david-mollitor-db closed pull request #58959: [SPARK-59706][SQL] Hoist loop-invariant left row out of the AS-OF join inner scan
URL: https://github.com/apache/spark/pull/58959


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org