You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "cloud-fan (via GitHub)" <gi...@apache.org> on 2026/08/28 07:45:54 UTC

[PR] [SPARK-58933][SQL] Address dynamic INSERT review follow-ups [spark]

cloud-fan opened a new pull request, #58378:
URL: https://github.com/apache/spark/pull/58378

   ### What changes were proposed in this pull request?
   
   This follow-up to #58204 makes two corrections:
   
   * Distinguish INSERT targets from other target-table references in `parse_sql` lineage collection.
     INSERT targets continue to bypass CTE shadow filtering, while DELETE, UPDATE, and MERGE targets
     follow CTE substitution semantics.
   * Run the early dynamic INSERT target-resolution pass inside the analysis planning tracker and
     report failures through `QueryPlanningTracker.setAnalysisFailed`.
   
   Focused regression tests cover CTE-shadowed DML targets, rule timing for dynamic INSERT target
   resolution, and failure callback reporting.
   
   ### Why are the changes needed?
   
   The target-role exemption added by #58204 was broader than required. A CTE-shadowed DELETE,
   UPDATE, or MERGE target could be reported as a catalog target even though CTE substitution replaces
   that relation. Only INSERT targets are deliberately protected from CTE substitution.
   
   In addition, transaction discovery resolves a dynamic INSERT target before the normal analysis
   phase. That work was absent from rule and phase timing, and an exception from the pre-pass occurred
   before the normal analysis failure callback.
   
   ### Does this PR introduce _any_ user-facing change?
   
   Yes, within unreleased master only. `parse_sql` no longer reports CTE-shadowed DELETE, UPDATE, or
   MERGE targets as catalog target-table references. INSERT target reporting is unchanged.
   
   ### How was this patch tested?
   
   Added regression coverage and ran:
   
   ```bash
   build/sbt "sql/testOnly org.apache.spark.sql.catalyst.parser.ParseSqlResultSuite org.apache.spark.sql.execution.QueryExecutionSuite"
   ```
   
   All 39 tests passed.
   
   ### Was this patch authored or co-authored using generative AI tooling?
   
   Generated-by: OpenAI Codex (GPT-5)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-58933][SQL] Simplify and fix parsed INSERT target handling [spark]

Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan closed pull request #58378: [SPARK-58933][SQL] Simplify and fix parsed INSERT target handling
URL: https://github.com/apache/spark/pull/58378


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-58933][SQL] Simplify and fix parsed INSERT target handling [spark]

Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on PR #58378:
URL: https://github.com/apache/spark/pull/58378#issuecomment-5455160483

   thanks for review, merging to master/4.x/4.3/4.2


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-58933][SQL] Simplify and fix parsed INSERT target handling [spark]

Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on PR #58378:
URL: https://github.com/apache/spark/pull/58378#issuecomment-5455181632

   **Merge Summary:**
   - merged into master https://github.com/apache/spark/commit/900b44e22f03bdbf991038dbc97a48dab5c7ecb9
   - merged into branch-4.x https://github.com/apache/spark/commit/b2c7f1a14351095c23e1c97af5d24fa031869477
   
   *Posted by `merge_spark_pr.py`*


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org