You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2019/05/21 03:59:21 UTC
[jira] [Updated] (SPARK-20122) Analyzer reports "unresolved
operator 'Aggregate" for nonexistent window specifications
[ https://issues.apache.org/jira/browse/SPARK-20122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon updated SPARK-20122:
---------------------------------
Labels: bulk-closed (was: )
> Analyzer reports "unresolved operator 'Aggregate" for nonexistent window specifications
> ---------------------------------------------------------------------------------------
>
> Key: SPARK-20122
> URL: https://issues.apache.org/jira/browse/SPARK-20122
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 2.2.0
> Reporter: Jacek Laskowski
> Priority: Minor
> Labels: bulk-closed
>
> {{Analyzer}} should be more elaborate on the root cause of the {{org.apache.spark.sql.AnalysisException}} when {{WindowSpecReference}} cannot be resolved.
> I think the real issue is that {{AstBuilder}} creates a lookup table for {{WindowSpecDefinition}}s per name and when it cannot resolve {{WindowSpecReference}} it simply fails analysis (it's too early in the SQL parsing).
> {code}
> scala> val table = spark.emptyDataset[Int]
> table: org.apache.spark.sql.Dataset[Int] = [value: int]
> scala> table.selectExpr("count() OVER nonexistentWindowSpec")
> org.apache.spark.sql.AnalysisException: unresolved operator 'Aggregate [unresolvedwindowexpression(count(), WindowSpecReference(nonexistentWindowSpec)) AS unresolvedwindowexpression(count())#15];;
> 'Aggregate [unresolvedwindowexpression(count(), WindowSpecReference(nonexistentWindowSpec)) AS unresolvedwindowexpression(count())#15]
> +- LocalRelation <empty>, [value#11]
> at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$class.failAnalysis(CheckAnalysis.scala:40)
> at org.apache.spark.sql.catalyst.analysis.Analyzer.failAnalysis(Analyzer.scala:90)
> at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$3.apply(CheckAnalysis.scala:408)
> at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$3.apply(CheckAnalysis.scala:407)
> at org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:127)
> at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$class.checkAnalysis(CheckAnalysis.scala:407)
> at org.apache.spark.sql.catalyst.analysis.Analyzer.checkAnalysis(Analyzer.scala:90)
> at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:53)
> at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:67)
> at org.apache.spark.sql.Dataset.org$apache$spark$sql$Dataset$$withPlan(Dataset.scala:2832)
> at org.apache.spark.sql.Dataset.select(Dataset.scala:1137)
> at org.apache.spark.sql.Dataset.selectExpr(Dataset.scala:1172)
> ... 48 elided
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org