You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "vladanvasi-db (via GitHub)" <gi...@apache.org> on 2026/10/20 15:59:16 UTC
[PR] [WIP][SQL] Support DROP TABLE in the single-pass analyzer [spark]
vladanvasi-db opened a new pull request, #59646:
URL: https://github.com/apache/spark/pull/59646
### What changes were proposed in this pull request?
Support `DROP TABLE` in the single-pass analyzer for V1 and V2 catalogs, including `IF EXISTS`, `PURGE`, qualified identifiers, and temporary views.
Share command-target identifier resolution and session-catalog lowering with the fixed-point analyzer, register the resolver in both core and Hive sessions, and add validator extensions for commands resolved outside Catalyst.
The SQL golden tests are covered separately by [SPARK-60197](https://issues.apache.org/jira/browse/SPARK-60197).
### Why are the changes needed?
`DROP TABLE` currently falls back to the fixed-point analyzer. Sharing the existing identifier and command conversion logic keeps both analyzers' plans and error behavior consistent.
### Does this PR introduce _any_ user-facing change?
No. SQL behavior is preserved; the single-pass analyzer can resolve an additional command.
### How was this patch tested?
Focused tests cover V1/V2 plan parity, temporary views, identifier origin and tags, namespace errors, guard routing, and catalog API execution with single-pass analysis forced on and validation enabled.
Independent subagent review completed with no remaining findings.
All 114 tests passed:
- `DropTableResolverSuite`, `ResolverGuardSuite`, and `ResolutionValidatorSuite` (90 tests).
- Existing V1/V2 `DropTableSuite` tests (16 tests) and Hive `DropTableSuite` (8 tests).
Scala style checks passed for Catalyst, SQL main/test sources, and Hive.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: OpenAI Codex (GPT-6)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-60422][SQL] Extract shared catalog identifier and DROP TABLE helpers [spark]
Posted by "vladanvasi-db (via GitHub)" <gi...@apache.org>.
vladanvasi-db commented on code in PR #59646:
URL: https://github.com/apache/spark/pull/59646#discussion_r4338206154
##########
sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala:
##########
@@ -1034,3 +1020,23 @@ class ResolveSessionCatalog(val catalogManager: CatalogManager)
}
}
+
+object ResolveSessionCatalog {
Review Comment:
Done in [e1fa89dbf96](https://github.com/apache/spark/commit/e1fa89dbf96aa563328e5d9bd1f30bc700c6d4d5). Extracted `V1CommandUtils` and `ResolvedV1CommandIdentifier` into separate Catalyst files, with callers using the helpers directly. Both helpers are independent of the fixed-point analyzer rule. All 120 existing tests and Catalyst/SQL style checks pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-60422][SQL] Extract shared catalog identifier and DROP TABLE helpers [spark]
Posted by "uros-b (via GitHub)" <gi...@apache.org>.
uros-b closed pull request #59646: [SPARK-60422][SQL] Extract shared catalog identifier and DROP TABLE helpers
URL: https://github.com/apache/spark/pull/59646
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-60422][SQL] Extract shared catalog identifier and DROP TABLE helpers [spark]
Posted by "uros-b (via GitHub)" <gi...@apache.org>.
uros-b commented on PR #59646:
URL: https://github.com/apache/spark/pull/59646#issuecomment-6335587837
Thank you @vladanvasi-db and @mihailoale-db!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [WIP][SQL] Extract shared catalog identifier and DROP TABLE helpers [spark]
Posted by "mihailoale-db (via GitHub)" <gi...@apache.org>.
mihailoale-db commented on code in PR #59646:
URL: https://github.com/apache/spark/pull/59646#discussion_r4336556145
##########
sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala:
##########
@@ -1034,3 +1020,23 @@ class ResolveSessionCatalog(val catalogManager: CatalogManager)
}
}
+
+object ResolveSessionCatalog {
Review Comment:
I would move both this and `ResolvedV1Identifier` to separate files to decouple from fixed-point analyzer rules and rename.
##########
sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala:
##########
@@ -1034,3 +1020,23 @@ class ResolveSessionCatalog(val catalogManager: CatalogManager)
}
}
+
+object ResolveSessionCatalog {
+ /** Whether commands on `catalog` can be converted to V1 session-catalog commands. */
+ def supportsV1Command(catalog: CatalogPlugin): Boolean = {
+ CatalogV2Util.isSessionCatalog(catalog) && (
+ SQLConf.get.getConf(SQLConf.V2_SESSION_CATALOG_IMPLEMENTATION) == "builtin" ||
+ catalog.isInstanceOf[CatalogExtension])
+ }
+
+ /** Extracts a V1 identifier, rejecting namespaces that cannot be represented in V1. */
+ object ResolvedV1Identifier {
+ import org.apache.spark.sql.connector.catalog.CatalogV2Implicits._
Review Comment:
Let's move imports to the top of the file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-60422][SQL] Extract shared catalog identifier and DROP TABLE helpers [spark]
Posted by "uros-b (via GitHub)" <gi...@apache.org>.
uros-b commented on PR #59646:
URL: https://github.com/apache/spark/pull/59646#issuecomment-6335598424
**Merge Summary:**
- merged into master https://github.com/apache/spark/commit/04328dee2b44a99dc78110c44d75e401a8d2fb82
- merged into branch-4.x https://github.com/apache/spark/commit/57458f4e319efec562dc5ad1da7bc4cdb309ac1e
*Posted by `merge_spark_pr.py`*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org