You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/04/06 14:43:41 UTC

[GitHub] [spark] cloud-fan commented on issue #28026: [SPARK-31257][SQL] Unify create table syntax

cloud-fan commented on issue #28026: [SPARK-31257][SQL] Unify create table syntax
URL: https://github.com/apache/spark/pull/28026#issuecomment-609838028
 
 
   I'm a bit worried about merging this `+1661 −1146` huge PR to 3.0 at this stage. I agree that this may be in the right direction to implement Hive as a v2 catalog/data source, but this is way beyond what we need for 3.0.
   
   Can we have a more surgical patch that only unifies the CREATE TABLE syntax? Basically, for the items you listed in the PR description.
   
   These are still needed:
   - Unify the create table syntax in the parser by merging Hive and DataSource clauses
   - Add SerdeInfo and external boolean to statement plans and update AstBuilder to produce them
   - Add conversion from create statement plan to v1 create plans in ResolveSessionCatalog
   - Remove SparkSqlParser rules for Hive syntax
   
   With one more item: update ResolveCatalogs to fail if Hive specific clauses are specified in the create statement plan for v2 catalogs.
   
   These can be done later
   - Support new statement clauses in ResolveCatalogs conversion to v2 create plans
   - Add "option." namespace to distinguish SERDEPROPERTIES and OPTIONS in table properties
   
   BTW, the `Does this PR introduce any user-facing change?` section looks more like a new feature. Can we focus on the consequence of the syntax unification? For example:
   1. `CREATE TABLE ... USING ... PARTITIONED BY (colName colType)` is now supported.
   2. `CREATE EXTERNAL TABLE ...` without USING and STORED BY/AS now creates Hive table.
   3. ...

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org