You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Cheng Lian (JIRA)" <ji...@apache.org> on 2016/04/08 18:07:25 UTC
[jira] [Commented] (SPARK-14488) "CREATE TEMPORARY TABLE ... USING
... AS SELECT ..." creates persisted table
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15232401#comment-15232401 ]
Cheng Lian commented on SPARK-14488:
------------------------------------
Ah, sorry, the logical plan class {{CreateTableUsingAsSelect}} uses a boolean flag to indicate whether the table is temporary or not, while physical plan uses two different classes {{CreateTempTableUsingAsSelect}} and {{CreateTableUsingAsSelect}}. Then something is probably wrong in the planner.
> "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..." creates persisted table
> ----------------------------------------------------------------------------
>
> Key: SPARK-14488
> URL: https://issues.apache.org/jira/browse/SPARK-14488
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.0.0
> Reporter: Cheng Lian
> Assignee: Cheng Lian
>
> The following Spark shell snippet reproduces this bug:
> {code}
> sqlContext range 10 registerTempTable "x"
> // The problematic DDL statement:
> sqlContext sql "CREATE TEMPORARY TABLE y USING PARQUET AS SELECT * FROM x"
> sqlContext.tables().show()
> {code}
> It shows the following result:
> {noformat}
> +---------+-----------+
> |tableName|isTemporary|
> +---------+-----------+
> | y| false|
> | x| true|
> +---------+-----------+
> {noformat}
> Note that {{y}} is NOT temporary although it's created using {{CREATE TEMPORARY TABLE ...}}.
> Explain shows that the physical plan node is {{CreateTableUsingAsSelect}} rather than {{CreateTempTableUsingAsSelect}}.
> {noformat}
> == Parsed Logical Plan ==
> 'CreateTableUsingAsSelect `y`, PARQUET, true, [Ljava.lang.String;@4d001a14, None, Overwrite, Map()
> +- 'Project [*]
> +- 'UnresolvedRelation `x`, None
> == Analyzed Logical Plan ==
> CreateTableUsingAsSelect `y`, PARQUET, true, [Ljava.lang.String;@4d001a14, None, Overwrite, Map()
> +- Project [id#0L]
> +- SubqueryAlias x
> +- Range 0, 10, 1, 1, [id#0L]
> == Optimized Logical Plan ==
> CreateTableUsingAsSelect `y`, PARQUET, true, [Ljava.lang.String;@4d001a14, None, Overwrite, Map()
> +- Range 0, 10, 1, 1, [id#0L]
> == Physical Plan ==
> ExecutedCommand CreateMetastoreDataSourceAsSelect `y`, PARQUET, [Ljava.lang.String;@4d001a14, None, Overwrite, Map(), Range 0, 10, 1, 1, [id#0L]|
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org