[jira] [Created] (SPARK-59706) Hoist loop-invariant left row out of the AS-OF join inner scan

["David Mollitor (Jira)" <ji...@apache.org>]

David Mollitor created SPARK-59706:
--------------------------------------

             Summary: Hoist loop-invariant left row out of the AS-OF join inner scan
                 Key: SPARK-59706
                 URL: https://issues.apache.org/jira/browse/SPARK-59706
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 4.1.0
            Reporter: David Mollitor
             Fix For: 4.3.0


h3. What changes were proposed in this pull request?

In {{SortMergeAsOfJoinScanner}}
({{{}sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeAsOfJoinExec.scala{}}}), the per-left-row scan over the buffered right group ({{{}findBestBackwardForward{}}} and {{{}findBestForwardNearest{}}}) rebinds both sides of the shared {{JoinedRow}} on every iteration via {{{}joinedRow.withLeft(leftRow).withRight(rightRow){}}}. The left row is fixed for the duration of a scan, so this binds {{joinedRow.withLeft(leftRow)}} once before the loop and keeps only {{joinedRow.withRight(rightRow)}} inside it.
h3. Why are the changes needed?

The inner scan runs once per buffered right row for every left row, so {{withLeft}} was called redundantly on each iteration with an argument that never changes within a scan. JFR profiling of {{AsOfJoinBenchmark}} showed {{JoinedRow.withLeft}} / {{withRight}} among the hotter frames in the (interpreted) scanner. Binding the left side once removes the redundant per-row call.

This is behavior-preserving: {{JoinedRow.withLeft}} only stores the left-row reference, nothing mutates the left side within a scan, and {{findNext}} re-binds both sides before projecting the output row.
h3. Does this PR introduce any user-facing change?

No.
h3. How was this patch tested?

Existing tests pass: {{{}SortMergeAsOfJoinSuite{}}}, {{{}DataFrameAsOfJoinSuite{}}}, {{{}AsOfJoinSQLSuite{}}}, and {{AsOfJoinSortMergeSQLSuite}} (89 tests).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org