You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Brian Zhang (JIRA)" <ji...@apache.org> on 2017/06/20 19:20:00 UTC
[jira] [Updated] (SPARK-21154) ParseException when Create View from
another View in Spark SQL
[ https://issues.apache.org/jira/browse/SPARK-21154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brian Zhang updated SPARK-21154:
--------------------------------
Description:
When creating View from another existing View in Spark SQL, we will see ParseException if existing View.
Here is the detail on how to reproduce it:
*Hive* (I'm using 1.1.0):
hive> *CREATE TABLE my_table (id int, name string);*
OK
Time taken: 0.107 seconds
hive> *CREATE VIEW my_view(view_id,view_name) AS SELECT * FROM my_table;*
OK
Time taken: 0.075 seconds
# View Information
View Original Text: SELECT * FROM my_table
View Expanded Text: SELECT `id` AS `view_id`, `name` AS `view_name` FROM (SELECT `my_table`.`id`, `my_table`.`name` FROM `default`.`my_table`) `default.my_view`
Time taken: 0.04 seconds, Fetched: 28 row(s)
*Spark *(Same behavior for spark 2.1.0 and 2.1.1):
scala> sqlContext.sql("CREATE VIEW my_view_spark AS SELECT * FROM my_view");
java.lang.RuntimeException: Failed to analyze the canonicalized SQL: SELECT `gen_attr_0` AS `view_id`, `gen_attr_1` AS `view_name` FROM (SELECT `gen_attr_0`, `gen_attr_1` FROM (SELECT `gen_attr_2` AS `gen_attr_0`, `gen_attr_3` AS `gen_attr_1` FROM (SELECT `gen_attr_2`, `gen_attr_3` FROM (SELECT `id` AS `gen_attr_2`, `name` AS `gen_attr_3` FROM `default`.`my_table`) AS gen_subquery_0) AS default.my_view) AS my_view) AS my_view
at org.apache.spark.sql.execution.command.CreateViewCommand.prepareTable(views.scala:222)
at org.apache.spark.sql.execution.command.CreateViewCommand.run(views.scala:176)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:74)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:114)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:114)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:135)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:132)
at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:113)
at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:87)
at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:87)
at org.apache.spark.sql.Dataset.<init>(Dataset.scala:185)
at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:64)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:592)
at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:699)
... 74 elided
Caused by: org.apache.spark.sql.catalyst.parser.ParseException:
mismatched input 'FROM' expecting {<EOF>, 'WHERE', 'GROUP', 'ORDER', 'HAVING', 'LIMIT', 'LATERAL', 'WINDOW', 'UNION', 'EXCEPT', 'MINUS', 'INTERSECT', 'SORT', 'CLUSTER', 'DISTRIBUTE'}(line 1, pos 62)
== SQL ==
SELECT `gen_attr_0` AS `view_id`, `gen_attr_1` AS `view_name` FROM (SELECT `gen_attr_0`, `gen_attr_1` FROM (SELECT `gen_attr_2` AS `gen_attr_0`, `gen_attr_3` AS `gen_attr_1` FROM (SELECT `gen_attr_2`, `gen_attr_3` FROM (SELECT `id` AS `gen_attr_2`, `name` AS `gen_attr_3` FROM `default`.`my_table`) AS gen_subquery_0) AS default.my_view) AS my_view) AS my_view
--------------------------------------------------------------^^^
at org.apache.spark.sql.catalyst.parser.ParseException.withCommand(ParseDriver.scala:197)
at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:99)
at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:45)
at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:53)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:592)
at org.apache.spark.sql.execution.command.CreateViewCommand.prepareTable(views.scala:219)
... 90 more
was:
When creating View from another existing View in Spark SQL, we will see ParseException if existing View.
Here is the detail on how to reproduce it:
*Hive* (I'm using 1.1.0):
hive> *CREATE TABLE my_table (id int, name string);*
OK
Time taken: 0.107 seconds
hive> *CREATE VIEW my_view(view_id,view_name) AS SELECT * FROM my_table;*
OK
Time taken: 0.075 seconds
# View Information
View Original Text: SELECT * FROM my_table
View Expanded Text: SELECT `id` AS `view_id`, `name` AS `view_name` FROM (SELECT `my_table`.`id`, `my_table`.`name` FROM `default`.`my_table`) `default.my_view`
Time taken: 0.04 seconds, Fetched: 28 row(s)
Spark (Same behavior for spark 2.1.0 and 2.1.1):
scala> sqlContext.sql("CREATE VIEW my_view_spark AS SELECT * FROM my_view");
java.lang.RuntimeException: Failed to analyze the canonicalized SQL: SELECT `gen_attr_0` AS `view_id`, `gen_attr_1` AS `view_name` FROM (SELECT `gen_attr_0`, `gen_attr_1` FROM (SELECT `gen_attr_2` AS `gen_attr_0`, `gen_attr_3` AS `gen_attr_1` FROM (SELECT `gen_attr_2`, `gen_attr_3` FROM (SELECT `id` AS `gen_attr_2`, `name` AS `gen_attr_3` FROM `default`.`my_table`) AS gen_subquery_0) AS default.my_view) AS my_view) AS my_view
at org.apache.spark.sql.execution.command.CreateViewCommand.prepareTable(views.scala:222)
at org.apache.spark.sql.execution.command.CreateViewCommand.run(views.scala:176)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:74)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:114)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:114)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:135)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:132)
at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:113)
at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:87)
at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:87)
at org.apache.spark.sql.Dataset.<init>(Dataset.scala:185)
at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:64)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:592)
at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:699)
... 74 elided
Caused by: org.apache.spark.sql.catalyst.parser.ParseException:
mismatched input 'FROM' expecting {<EOF>, 'WHERE', 'GROUP', 'ORDER', 'HAVING', 'LIMIT', 'LATERAL', 'WINDOW', 'UNION', 'EXCEPT', 'MINUS', 'INTERSECT', 'SORT', 'CLUSTER', 'DISTRIBUTE'}(line 1, pos 62)
== SQL ==
SELECT `gen_attr_0` AS `view_id`, `gen_attr_1` AS `view_name` FROM (SELECT `gen_attr_0`, `gen_attr_1` FROM (SELECT `gen_attr_2` AS `gen_attr_0`, `gen_attr_3` AS `gen_attr_1` FROM (SELECT `gen_attr_2`, `gen_attr_3` FROM (SELECT `id` AS `gen_attr_2`, `name` AS `gen_attr_3` FROM `default`.`my_table`) AS gen_subquery_0) AS default.my_view) AS my_view) AS my_view
--------------------------------------------------------------^^^
at org.apache.spark.sql.catalyst.parser.ParseException.withCommand(ParseDriver.scala:197)
at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:99)
at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:45)
at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:53)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:592)
at org.apache.spark.sql.execution.command.CreateViewCommand.prepareTable(views.scala:219)
... 90 more
> ParseException when Create View from another View in Spark SQL
> ---------------------------------------------------------------
>
> Key: SPARK-21154
> URL: https://issues.apache.org/jira/browse/SPARK-21154
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.1.0, 2.1.1
> Reporter: Brian Zhang
>
> When creating View from another existing View in Spark SQL, we will see ParseException if existing View.
> Here is the detail on how to reproduce it:
> *Hive* (I'm using 1.1.0):
> hive> *CREATE TABLE my_table (id int, name string);*
> OK
> Time taken: 0.107 seconds
> hive> *CREATE VIEW my_view(view_id,view_name) AS SELECT * FROM my_table;*
> OK
> Time taken: 0.075 seconds
> # View Information
> View Original Text: SELECT * FROM my_table
> View Expanded Text: SELECT `id` AS `view_id`, `name` AS `view_name` FROM (SELECT `my_table`.`id`, `my_table`.`name` FROM `default`.`my_table`) `default.my_view`
> Time taken: 0.04 seconds, Fetched: 28 row(s)
> *Spark *(Same behavior for spark 2.1.0 and 2.1.1):
> scala> sqlContext.sql("CREATE VIEW my_view_spark AS SELECT * FROM my_view");
> java.lang.RuntimeException: Failed to analyze the canonicalized SQL: SELECT `gen_attr_0` AS `view_id`, `gen_attr_1` AS `view_name` FROM (SELECT `gen_attr_0`, `gen_attr_1` FROM (SELECT `gen_attr_2` AS `gen_attr_0`, `gen_attr_3` AS `gen_attr_1` FROM (SELECT `gen_attr_2`, `gen_attr_3` FROM (SELECT `id` AS `gen_attr_2`, `name` AS `gen_attr_3` FROM `default`.`my_table`) AS gen_subquery_0) AS default.my_view) AS my_view) AS my_view
> at org.apache.spark.sql.execution.command.CreateViewCommand.prepareTable(views.scala:222)
> at org.apache.spark.sql.execution.command.CreateViewCommand.run(views.scala:176)
> at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58)
> at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56)
> at org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:74)
> at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:114)
> at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:114)
> at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:135)
> at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:132)
> at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:113)
> at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:87)
> at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:87)
> at org.apache.spark.sql.Dataset.<init>(Dataset.scala:185)
> at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:64)
> at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:592)
> at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:699)
> ... 74 elided
> Caused by: org.apache.spark.sql.catalyst.parser.ParseException:
> mismatched input 'FROM' expecting {<EOF>, 'WHERE', 'GROUP', 'ORDER', 'HAVING', 'LIMIT', 'LATERAL', 'WINDOW', 'UNION', 'EXCEPT', 'MINUS', 'INTERSECT', 'SORT', 'CLUSTER', 'DISTRIBUTE'}(line 1, pos 62)
> == SQL ==
> SELECT `gen_attr_0` AS `view_id`, `gen_attr_1` AS `view_name` FROM (SELECT `gen_attr_0`, `gen_attr_1` FROM (SELECT `gen_attr_2` AS `gen_attr_0`, `gen_attr_3` AS `gen_attr_1` FROM (SELECT `gen_attr_2`, `gen_attr_3` FROM (SELECT `id` AS `gen_attr_2`, `name` AS `gen_attr_3` FROM `default`.`my_table`) AS gen_subquery_0) AS default.my_view) AS my_view) AS my_view
> --------------------------------------------------------------^^^
> at org.apache.spark.sql.catalyst.parser.ParseException.withCommand(ParseDriver.scala:197)
> at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:99)
> at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:45)
> at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:53)
> at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:592)
> at org.apache.spark.sql.execution.command.CreateViewCommand.prepareTable(views.scala:219)
> ... 90 more
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org