You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "Vamsi-klu (via GitHub)" <gi...@apache.org> on 2026/08/17 04:05:20 UTC
[PR] [SPARK-34254][SQL][DOCS] Document CREATE EXTERNAL TABLE for data source tables [spark]
Vamsi-klu opened a new pull request, #58037:
URL: https://github.com/apache/spark/pull/58037
### What changes were proposed in this pull request?
Document `CREATE EXTERNAL TABLE` on the data source CREATE TABLE page.
- Syntax diagram now shows `CREATE [ EXTERNAL ] TABLE`
- New EXTERNAL parameter next to LOCATION: the table uses the given path, DROP TABLE keeps the files, and `CREATE EXTERNAL TABLE` for a data source table must include `LOCATION` or an equivalent path option. A table created with `LOCATION` or a path option is also external without the `EXTERNAL` keyword.
- Parquet examples for `LOCATION` and `OPTIONS (PATH ...)`
`CREATE EXTERNAL TABLE ... USING ... LOCATION` has been legal since SPARK-33651. The Hive CREATE TABLE page already documents EXTERNAL. This page did not.
### Why are the changes needed?
The syntax has been supported since Spark 3.2.0, but `docs/sql-ref-syntax-ddl-create-table-datasource.md` still starts with `CREATE TABLE [ IF NOT EXISTS ]` and never shows an external example. Readers of this page would think EXTERNAL is Hive-only.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Docs only. No unit tests for this markdown. I did not run `SKIP_API=1 bundle exec jekyll build` here because Bundler was not available. Please check the rendered page for the syntax diagram, the EXTERNAL bullet next to LOCATION, and that the examples use `USING parquet`, not Hive `STORED AS` clauses.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Cursor Grok 4.6
Made with [Cursor](https://cursor.com)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-34254][SQL][DOCS] Document CREATE EXTERNAL TABLE for data source tables [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on PR #58037:
URL: https://github.com/apache/spark/pull/58037#issuecomment-5334129186
**Merge Summary:**
- merged into master https://github.com/apache/spark/commit/c854ed8b5b2c2907c78545a04b68a61564719c2c
- merged into branch-4.x https://github.com/apache/spark/commit/820db6e8ad797f7ecc3ac57ef362b2f2ee0b0732
- merged into branch-4.3 https://github.com/apache/spark/commit/687159acb8e79405691c4a9dc885db4879526e1f
*Posted by `merge_spark_pr.py`*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-34254][SQL][DOCS] Document CREATE EXTERNAL TABLE for data source tables [spark]
Posted by "Vamsi-klu (via GitHub)" <gi...@apache.org>.
Vamsi-klu commented on PR #58037:
URL: https://github.com/apache/spark/pull/58037#issuecomment-5333099923
> **1 addressed, 0 remaining, 1 new to this AI review.** (0 newly introduced, 1 late catch, 0 previously raised.)
> **0 blocking, 0 non-blocking, 1 nit.**
> The earlier correctness issue is fixed; only a small prose correction remains.
>
> **Nits**: 1 minor item (see inline comments).
>
> ## Verification
> I traced the V1 table path through `ResolveSessionCatalog`: `external || location.isDefined` selects `CatalogTableType.EXTERNAL`, `LOCATION` populates `storage.locationUri`, and `SessionCatalog.createTable` rejects external tables without that location. The catalog drop path deletes data only for managed tables, so the revised description is consistent with the implementation.
>
> ## PR metadata suggestions
> - Remove the stale claims that an equivalent path option is accepted and that the PR includes an `OPTIONS (PATH ...)` example; the current revision intentionally documents `LOCATION` only.
>
> <!-- ai-auto-review-authorized head=e91e0aae595321d464b7265efce3e30133289ff2 -->
>
> <!-- spark-dev-review-delivery:v1 sha256=2b57af1690e67f474d49c28fd001c039e132f3b5055ab04f002c11a8099fdba7 -->
>
> <!-- ai-code-review -->
Resolved, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-34254][SQL][DOCS] Document CREATE EXTERNAL TABLE for data source tables [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on code in PR #58037:
URL: https://github.com/apache/spark/pull/58037#discussion_r3807095147
##########
docs/sql-ref-syntax-ddl-create-table-datasource.md:
##########
@@ -78,6 +78,14 @@ as any order. For example, you can write COMMENT table_comment after TBLPROPERTI
Specifies buckets numbers, which is used in `CLUSTERED BY` clause.
+* **EXTERNAL**
+
+ Table is defined using the path provided as `LOCATION`, does not use default location for this table.
Review Comment:
**Nit:**
Please fix this comma splice; these are independent clauses.
```suggestion
The table is defined using the path provided as `LOCATION` and does not use the default location for this table.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-34254][SQL][DOCS] Document CREATE EXTERNAL TABLE for data source tables [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan closed pull request #58037: [SPARK-34254][SQL][DOCS] Document CREATE EXTERNAL TABLE for data source tables
URL: https://github.com/apache/spark/pull/58037
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-34254][SQL][DOCS] Document CREATE EXTERNAL TABLE for data source tables [spark]
Posted by "Vamsi-klu (via GitHub)" <gi...@apache.org>.
Vamsi-klu commented on PR #58037:
URL: https://github.com/apache/spark/pull/58037#issuecomment-5311680754
cc @HyukjinKwon @cloud-fan @dongjoon-hyun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-34254][SQL][DOCS] Document CREATE EXTERNAL TABLE for data source tables [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on code in PR #58037:
URL: https://github.com/apache/spark/pull/58037#discussion_r3805738741
##########
docs/sql-ref-syntax-ddl-create-table-datasource.md:
##########
@@ -78,6 +78,14 @@ as any order. For example, you can write COMMENT table_comment after TBLPROPERTI
Specifies buckets numbers, which is used in `CLUSTERED BY` clause.
+* **EXTERNAL**
+
+ Table is defined using the path provided as `LOCATION`, does not use default location for this table.
+ Dropping an external table removes catalog metadata and leaves the data files in place.
+ `CREATE EXTERNAL TABLE` for a data source table must include `LOCATION` or an equivalent path option.
Review Comment:
**Non-blocking:**
Please remove the path-option alternative and use `LOCATION` in the `student_ext` example. Parquet uses V1 by default, where `OPTIONS (PATH ...)` remains a storage property rather than `tableSpec.location`; with `EXTERNAL` set, `SessionCatalog.createTable` then rejects the table because `storage.locationUri` is empty.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org