You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (JIRA)" <ji...@apache.org> on 2019/03/08 20:36:00 UTC
[jira] [Assigned] (SPARK-27108) Add parsed CreateTable plans to
Catalyst
[ https://issues.apache.org/jira/browse/SPARK-27108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-27108:
------------------------------------
Assignee: Apache Spark
> Add parsed CreateTable plans to Catalyst
> ----------------------------------------
>
> Key: SPARK-27108
> URL: https://issues.apache.org/jira/browse/SPARK-27108
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 2.4.1
> Reporter: Ryan Blue
> Assignee: Apache Spark
> Priority: Major
>
> The abstract Catalyst SQL AST builder cannot currently parse {{CREATE TABLE}} commands. Creates are handled only by {{SparkSqlParser}} because the logical plans are defined in the v1 datasource package (org.apache.spark.sql.execution.datasources).
> The {{SparkSqlParser}} mixes parsing with logic that is specific to v1, like converting {{IF NOT EXISTS}} into a {{SaveMode}}. This makes it difficult (and error-prone) to produce v2 plans because it requires converting the AST to v1 and the converting v1 to v2.
> Instead, the catalyst parser should create plans that represent exactly what was parsed, after validation like ensuring no duplicate clauses. Then those plans should be converted to v1 or v2 plans in the analyzer. This structure will avoid errors caused by multiple layers of translation and keeps v1 and v2 plans separate to ensure that v1 has no behavior changes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org