You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by li...@apache.org on 2017/03/09 01:33:52 UTC
spark git commit: [MINOR][SQL] The analyzer rules are fired twice for
cases when AnalysisException is raised from analyzer.
Repository: spark
Updated Branches:
refs/heads/master a3648b5d4 -> d809ceed9
[MINOR][SQL] The analyzer rules are fired twice for cases when AnalysisException is raised from analyzer.
## What changes were proposed in this pull request?
In general we have a checkAnalysis phase which validates the logical plan and throws AnalysisException on semantic errors. However we also can throw AnalysisException from a few analyzer rules like ResolveSubquery.
I found that we fire up the analyzer rules twice for the queries that throw AnalysisException from one of the analyzer rules. This is a very minor fix. We don't have to strictly fix it. I just got confused seeing the rule getting fired two times when i was not expecting it.
## How was this patch tested?
Tested manually.
Author: Dilip Biswal <db...@us.ibm.com>
Closes #17214 from dilipbiswal/analyis_twice.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/d809ceed
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/d809ceed
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/d809ceed
Branch: refs/heads/master
Commit: d809ceed9762d5bbb04170e45f38751713112dd8
Parents: a3648b5
Author: Dilip Biswal <db...@us.ibm.com>
Authored: Wed Mar 8 17:33:49 2017 -0800
Committer: Xiao Li <ga...@gmail.com>
Committed: Wed Mar 8 17:33:49 2017 -0800
----------------------------------------------------------------------
.../org/apache/spark/sql/execution/QueryExecution.scala | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/d809ceed/sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala
----------------------------------------------------------------------
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala b/sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala
index 6ec2f4d..9a3656d 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala
@@ -46,9 +46,14 @@ class QueryExecution(val sparkSession: SparkSession, val logical: LogicalPlan) {
protected def planner = sparkSession.sessionState.planner
def assertAnalyzed(): Unit = {
- try sparkSession.sessionState.analyzer.checkAnalysis(analyzed) catch {
+ // Analyzer is invoked outside the try block to avoid calling it again from within the
+ // catch block below.
+ analyzed
+ try {
+ sparkSession.sessionState.analyzer.checkAnalysis(analyzed)
+ } catch {
case e: AnalysisException =>
- val ae = new AnalysisException(e.message, e.line, e.startPosition, Some(analyzed))
+ val ae = new AnalysisException(e.message, e.line, e.startPosition, Option(analyzed))
ae.setStackTrace(e.getStackTrace)
throw ae
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org