[PR] [AURON #2308] Fix native shuffle exchange NPE in shuffle write tasks [auron]

["Sigma-Ma (via GitHub)" <gi...@apache.org>]

Sigma-Ma opened a new pull request, #2487:
URL: https://github.com/apache/auron/pull/2487

   # Which issue does this PR close?
   
   Closes #2308
   
   # Rationale for this change
   
   The shuffle-write `NativeRDD` closure accessed `nativeHashExprs` and
   `nativeSortExecNode` through the enclosing `NativeShuffleExchangeBase`.
   
   This caused the task closure to retain the exchange plan. When Spark cleared
   the `SparkPlan` tree before task execution, RangePartitioning shuffle writes
   could dereference the cleared outer plan and throw a `NullPointerException`.
   
   # What changes are included in this PR?
   
   - Evaluate `nativeHashExprs` and `nativeSortExecNode` before constructing the
     shuffle-write `NativeRDD`, so the task closure only captures the evaluated
     native plan values.
   - Extend the existing binary RangePartitioning test to verify that the native
     plan closure no longer captures `NativeShuffleExchangeBase`.
   
   # Are there any user-facing changes?
   
   No. This is an internal execution correctness fix.
   
   # How was this patch tested?
   
   ```bash
   ./dev/reformat --check
   
   ./build/mvn -Pspark-3.5 -Pscala-2.12 -pl spark-extension -am install -DskipTests -DskipBuildNative
   
   ./build/mvn -Pspark-3.5 -Pscala-2.12 -pl spark-extension-shims-spark test -Dsuites=org.apache.auron.AuronQuerySuite -Dsuffixes=NoDiscoveredSuites
   ```
   
   # Was this patch authored or co-authored using generative AI tooling?
   - [x] Yes
   - [ ] No
   
   Generated-by: OpenAI Codex (GPT-5)
   
   ASF guidance: https://www.apache.org/legal/generative-tooling.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [AURON #2308] Fix native shuffle exchange NPE in shuffle write tasks [auron]

["slfan1989 (via GitHub)" <gi...@apache.org>]

slfan1989 merged PR #2487:
URL: https://github.com/apache/auron/pull/2487


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org