You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by yh...@apache.org on 2016/03/18 21:40:57 UTC
spark git commit: [SPARK-13972][SQL][FOLLOW-UP] When creating the
query execution for a converted SQL query, we eagerly trigger analysis
Repository: spark
Updated Branches:
refs/heads/master 2e0c5284f -> 238fb485b
[SPARK-13972][SQL][FOLLOW-UP] When creating the query execution for a converted SQL query, we eagerly trigger analysis
## What changes were proposed in this pull request?
As part of testing generating SQL query from a analyzed SQL plan, we run the generated SQL for tests in HiveComparisonTest. This PR makes the generated SQL get eagerly analyzed. So, when a generated SQL has any analysis error, we can see the error message created by
```
case NonFatal(e) => fail(
s"""Failed to analyze the converted SQL string:
|
|# Original HiveQL query string:
|$queryString
|
|# Resolved query plan:
|${originalQuery.analyzed.treeString}
|
|# Converted SQL query string:
|$convertedSQL
""".stripMargin, e)
```
Right now, if we can parse a generated SQL but fail to analyze it, we will see error message generated by the following code (it only mentions that we cannot execute the original query, i.e. `queryString`).
```
case e: Throwable =>
val errorMessage =
s"""
|Failed to execute query using catalyst:
|Error: ${e.getMessage}
|${stackTraceToString(e)}
|$queryString
|$query
|== HIVE - ${hive.size} row(s) ==
|${hive.mkString("\n")}
""".stripMargin
```
## How was this patch tested?
Existing tests.
Author: Yin Huai <yh...@databricks.com>
Closes #11825 from yhuai/SPARK-13972-follow-up.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/238fb485
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/238fb485
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/238fb485
Branch: refs/heads/master
Commit: 238fb485be4cdf8337cacc58c31a9f885a99853c
Parents: 2e0c528
Author: Yin Huai <yh...@databricks.com>
Authored: Fri Mar 18 13:40:53 2016 -0700
Committer: Yin Huai <yh...@databricks.com>
Committed: Fri Mar 18 13:40:53 2016 -0700
----------------------------------------------------------------------
.../apache/spark/sql/hive/execution/HiveComparisonTest.scala | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/238fb485/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveComparisonTest.scala
----------------------------------------------------------------------
diff --git a/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveComparisonTest.scala b/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveComparisonTest.scala
index 019d752..cfca93b 100644
--- a/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveComparisonTest.scala
+++ b/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveComparisonTest.scala
@@ -409,7 +409,10 @@ abstract class HiveComparisonTest
}
try {
- new TestHive.QueryExecution(convertedSQL)
+ val queryExecution = new TestHive.QueryExecution(convertedSQL)
+ // Trigger the analysis of this converted SQL query.
+ queryExecution.analyzed
+ queryExecution
} catch {
case NonFatal(e) => fail(
s"""Failed to analyze the converted SQL string:
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org