You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by cloud-fan <gi...@git.apache.org> on 2016/04/30 09:39:45 UTC
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/12810
[SPARK-15029] improve error message for Generate
## What changes were proposed in this pull request?
This PR improve the error message for `Generate` in 3 cases:
1. generator is nested in expressions, e.g. `SELECT explode(list) + 1 FROM tbl`
2. generator appears more than one time in SELECT, e.g. `SELECT explode(list), explode(list) FROM tbl`
3. generator appears in other operator which is not project, e.g. `SELECT * FROM tbl SORT BY explode(list)`
## How was this patch tested?
new tests in `AnalysisErrorSuite`
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cloud-fan/spark bug
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/12810.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12810
----
commit 22f03029239b119da2c14faf525c660f02d01f72
Author: Wenchen Fan <we...@databricks.com>
Date: 2016-04-30T07:35:45Z
improve error message for Generate
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by hvanhovell <gi...@git.apache.org>.
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/12810#discussion_r61947445
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala ---
@@ -41,19 +41,17 @@ import org.apache.spark.sql.types._
*/
trait Generator extends Expression {
- // TODO ideally we should return the type of ArrayType(StructType),
- // however, we don't keep the output field names in the Generator.
- override def dataType: DataType = throw new UnsupportedOperationException
+ override def dataType: DataType = ArrayType(elementSchema)
override def foldable: Boolean = false
override def nullable: Boolean = false
/**
- * The output element data types in structure of Seq[(DataType, Nullable)]
+ * The output element schema.
* TODO we probably need to add more information like metadata etc.
--- End diff --
MINOR: is this TODO still valid?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-216435316
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-216752537
**[Test build #57713 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57713/consoleFull)** for PR 12810 at commit [`2e949c8`](https://github.com/apache/spark/commit/2e949c879a36a01590200b15c32f99378d94f1c9).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-216428982
**[Test build #57588 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57588/consoleFull)** for PR 12810 at commit [`4c284e5`](https://github.com/apache/spark/commit/4c284e5fb4e3f0e105f4093a526b8c76bf0162ee).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/12810#discussion_r61991132
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala ---
@@ -41,19 +41,17 @@ import org.apache.spark.sql.types._
*/
trait Generator extends Expression {
- // TODO ideally we should return the type of ArrayType(StructType),
- // however, we don't keep the output field names in the Generator.
- override def dataType: DataType = throw new UnsupportedOperationException
+ override def dataType: DataType = ArrayType(elementSchema)
override def foldable: Boolean = false
override def nullable: Boolean = false
/**
- * The output element data types in structure of Seq[(DataType, Nullable)]
+ * The output element schema.
* TODO we probably need to add more information like metadata etc.
--- End diff --
Let's just remove it. This is why TODOs in code are often not very useful :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-215950430
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-216456364
**[Test build #57608 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57608/consoleFull)** for PR 12810 at commit [`5a7ee32`](https://github.com/apache/spark/commit/5a7ee324664c85872fd0ce0f10698003b7543013).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-215945466
**[Test build #57426 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57426/consoleFull)** for PR 12810 at commit [`22f0302`](https://github.com/apache/spark/commit/22f03029239b119da2c14faf525c660f02d01f72).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-216470701
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-215950433
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57434/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-216762828
Merging in master/branch-2.0.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-216455641
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-215944609
cc @rxin
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by hvanhovell <gi...@git.apache.org>.
Github user hvanhovell commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-216651750
LGTM. Left a few tiny tiny comments. Feel free to address those.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-215956672
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57436/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-215985858
Looks like a legitimate test failure.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/12810#discussion_r61667078
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ---
@@ -165,6 +167,33 @@ class Analyzer(
}
}
+ object ValidateGenerator extends Rule[LogicalPlan] {
+ private def hasGenerator(expr: Expression): Boolean = {
+ expr.find(_.isInstanceOf[Generator]).isDefined
+ }
+
+ private def hasNestedGenerator(expr: NamedExpression): Boolean = expr match {
+ case UnresolvedAlias(_: Generator, _) => false
+ case Alias(_: Generator, _) => false
+ case MultiAlias(_: Generator, _) => false
+ case other if hasGenerator(other) => true
+ case _ => false
+ }
+
+ def apply(plan: LogicalPlan): LogicalPlan = plan resolveOperators {
+ case Project(projectList, _) if projectList.exists(hasNestedGenerator) =>
+ throw new AnalysisException("Generators are not supported when it's nested in expressions.")
--- End diff --
i think we need to show the generator itself, as "generator" is a very spark sql specific term. If users just put an explode, he/she wont' know it's a generator.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-216470515
**[Test build #57608 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57608/consoleFull)** for PR 12810 at commit [`5a7ee32`](https://github.com/apache/spark/commit/5a7ee324664c85872fd0ce0f10698003b7543013).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-216752641
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57713/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-216762179
**[Test build #57719 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57719/consoleFull)** for PR 12810 at commit [`041b7fe`](https://github.com/apache/spark/commit/041b7fef4702fd61ef497d0340664a2f616009e7).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-216455048
**[Test build #57606 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57606/consoleFull)** for PR 12810 at commit [`35b3c2b`](https://github.com/apache/spark/commit/35b3c2b13ac26c5017057d3863c28b3c69fd8861).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/12810#discussion_r61988177
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ---
@@ -1250,40 +1251,77 @@ class Analyzer(
}
/**
- * Rewrites table generating expressions that either need one or more of the following in order
- * to be resolved:
- * - concrete attribute references for their output.
- * - to be relocated from a SELECT clause (i.e. from a [[Project]]) into a [[Generate]]).
+ * Extracts [[Generator]] from the projectList of a [[Project]] operator and create [[Generate]]
+ * operator under [[Project]].
*
- * Names for the output [[Attribute]]s are extracted from [[Alias]] or [[MultiAlias]] expressions
- * that wrap the [[Generator]]. If more than one [[Generator]] is found in a Project, an
- * [[AnalysisException]] is throw.
+ * This rule will throw [[AnalysisException]] for following cases:
+ * 1. [[Generator]] is nested in expressions, e.g. `SELECT explode(list) + 1 FROM tbl`
+ * 2. more than one [[Generator]] is found in projectList,
+ * e.g. `SELECT explode(list), explode(list) FROM tbl`
+ * 3. [[Generator]] is found in other operators that are not [[Project]] or [[Generate]],
+ * e.g. `SELECT * FROM tbl SORT BY explode(list)`
*/
- object ResolveGenerate extends Rule[LogicalPlan] {
+ object ExtractGenerator extends Rule[LogicalPlan] {
+ private def hasGenerator(expr: Expression): Boolean = {
+ expr.find(_.isInstanceOf[Generator]).isDefined
+ }
+
+ private def hasNestedGenerator(expr: NamedExpression): Boolean = expr match {
+ case UnresolvedAlias(_: Generator, _) => false
+ case Alias(_: Generator, _) => false
+ case MultiAlias(_: Generator, _) => false
+ case other => hasGenerator(other)
+ }
+
+ private def trimAlias(expr: NamedExpression): Expression = expr match {
+ case UnresolvedAlias(child, _) => child
+ case Alias(child, _) => child
+ case MultiAlias(child, _) => child
+ case _ => expr
+ }
+
+ /** Extracts a [[Generator]] expression and any names assigned by aliases to their output. */
+ private object AliasedGenerator {
+ def unapply(e: Expression): Option[(Generator, Seq[String])] = e match {
+ case Alias(g: Generator, name) if g.resolved && g.elementSchema.length > 1 =>
+ // If not given the default names, and the TGF with multiple output columns
--- End diff --
yea, and this comment is just copied from the original code.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-215949555
**[Test build #57434 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57434/consoleFull)** for PR 12810 at commit [`71dd8dd`](https://github.com/apache/spark/commit/71dd8ddff0b6d655029f9ba07982e3993a5063ad).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by davies <gi...@git.apache.org>.
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/12810#discussion_r61991082
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala ---
@@ -41,19 +41,17 @@ import org.apache.spark.sql.types._
*/
trait Generator extends Expression {
- // TODO ideally we should return the type of ArrayType(StructType),
- // however, we don't keep the output field names in the Generator.
- override def dataType: DataType = throw new UnsupportedOperationException
+ override def dataType: DataType = ArrayType(elementSchema)
override def foldable: Boolean = false
override def nullable: Boolean = false
/**
- * The output element data types in structure of Seq[(DataType, Nullable)]
+ * The output element schema.
* TODO we probably need to add more information like metadata etc.
--- End diff --
I did not know either
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by hvanhovell <gi...@git.apache.org>.
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/12810#discussion_r61944296
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ---
@@ -1250,40 +1251,77 @@ class Analyzer(
}
/**
- * Rewrites table generating expressions that either need one or more of the following in order
- * to be resolved:
- * - concrete attribute references for their output.
- * - to be relocated from a SELECT clause (i.e. from a [[Project]]) into a [[Generate]]).
+ * Extracts [[Generator]] from the projectList of a [[Project]] operator and create [[Generate]]
+ * operator under [[Project]].
*
- * Names for the output [[Attribute]]s are extracted from [[Alias]] or [[MultiAlias]] expressions
- * that wrap the [[Generator]]. If more than one [[Generator]] is found in a Project, an
- * [[AnalysisException]] is throw.
+ * This rule will throw [[AnalysisException]] for following cases:
+ * 1. [[Generator]] is nested in expressions, e.g. `SELECT explode(list) + 1 FROM tbl`
+ * 2. more than one [[Generator]] is found in projectList,
+ * e.g. `SELECT explode(list), explode(list) FROM tbl`
+ * 3. [[Generator]] is found in other operators that are not [[Project]] or [[Generate]],
+ * e.g. `SELECT * FROM tbl SORT BY explode(list)`
*/
- object ResolveGenerate extends Rule[LogicalPlan] {
+ object ExtractGenerator extends Rule[LogicalPlan] {
+ private def hasGenerator(expr: Expression): Boolean = {
+ expr.find(_.isInstanceOf[Generator]).isDefined
+ }
+
+ private def hasNestedGenerator(expr: NamedExpression): Boolean = expr match {
+ case UnresolvedAlias(_: Generator, _) => false
+ case Alias(_: Generator, _) => false
+ case MultiAlias(_: Generator, _) => false
+ case other => hasGenerator(other)
+ }
+
+ private def trimAlias(expr: NamedExpression): Expression = expr match {
+ case UnresolvedAlias(child, _) => child
+ case Alias(child, _) => child
+ case MultiAlias(child, _) => child
+ case _ => expr
+ }
+
+ /** Extracts a [[Generator]] expression and any names assigned by aliases to their output. */
+ private object AliasedGenerator {
+ def unapply(e: Expression): Option[(Generator, Seq[String])] = e match {
+ case Alias(g: Generator, name) if g.resolved && g.elementSchema.length > 1 =>
+ // If not given the default names, and the TGF with multiple output columns
--- End diff --
Dumb question: what is TGF? Table Generating Function?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-216470706
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57608/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-216628489
cc @hvanhovell for review
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/12810
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-215945472
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-215945473
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57426/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-216752640
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-216435244
**[Test build #57588 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57588/consoleFull)** for PR 12810 at commit [`4c284e5`](https://github.com/apache/spark/commit/4c284e5fb4e3f0e105f4093a526b8c76bf0162ee).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-215944715
**[Test build #57426 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57426/consoleFull)** for PR 12810 at commit [`22f0302`](https://github.com/apache/spark/commit/22f03029239b119da2c14faf525c660f02d01f72).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-216455643
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57606/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-215956671
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-216762358
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57719/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-215956604
**[Test build #57436 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57436/consoleFull)** for PR 12810 at commit [`d986d3e`](https://github.com/apache/spark/commit/d986d3eafebb031aec0f1e151a81c48b9712f6a4).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-216749350
**[Test build #57719 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57719/consoleFull)** for PR 12810 at commit [`041b7fe`](https://github.com/apache/spark/commit/041b7fef4702fd61ef497d0340664a2f616009e7).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-216762355
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-216435317
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57588/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/12810#discussion_r61988203
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala ---
@@ -41,19 +41,17 @@ import org.apache.spark.sql.types._
*/
trait Generator extends Expression {
- // TODO ideally we should return the type of ArrayType(StructType),
- // however, we don't keep the output field names in the Generator.
- override def dataType: DataType = throw new UnsupportedOperationException
+ override def dataType: DataType = ArrayType(elementSchema)
override def foldable: Boolean = false
override def nullable: Boolean = false
/**
- * The output element data types in structure of Seq[(DataType, Nullable)]
+ * The output element schema.
* TODO we probably need to add more information like metadata etc.
--- End diff --
I'm not sure, cc @davies @marmbrus
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-216738769
**[Test build #57713 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57713/consoleFull)** for PR 12810 at commit [`2e949c8`](https://github.com/apache/spark/commit/2e949c879a36a01590200b15c32f99378d94f1c9).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by hvanhovell <gi...@git.apache.org>.
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/12810#discussion_r61944034
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ---
@@ -1250,40 +1251,77 @@ class Analyzer(
}
/**
- * Rewrites table generating expressions that either need one or more of the following in order
- * to be resolved:
- * - concrete attribute references for their output.
- * - to be relocated from a SELECT clause (i.e. from a [[Project]]) into a [[Generate]]).
+ * Extracts [[Generator]] from the projectList of a [[Project]] operator and create [[Generate]]
+ * operator under [[Project]].
*
- * Names for the output [[Attribute]]s are extracted from [[Alias]] or [[MultiAlias]] expressions
- * that wrap the [[Generator]]. If more than one [[Generator]] is found in a Project, an
- * [[AnalysisException]] is throw.
+ * This rule will throw [[AnalysisException]] for following cases:
+ * 1. [[Generator]] is nested in expressions, e.g. `SELECT explode(list) + 1 FROM tbl`
+ * 2. more than one [[Generator]] is found in projectList,
+ * e.g. `SELECT explode(list), explode(list) FROM tbl`
+ * 3. [[Generator]] is found in other operators that are not [[Project]] or [[Generate]],
+ * e.g. `SELECT * FROM tbl SORT BY explode(list)`
*/
- object ResolveGenerate extends Rule[LogicalPlan] {
+ object ExtractGenerator extends Rule[LogicalPlan] {
+ private def hasGenerator(expr: Expression): Boolean = {
+ expr.find(_.isInstanceOf[Generator]).isDefined
+ }
+
+ private def hasNestedGenerator(expr: NamedExpression): Boolean = expr match {
+ case UnresolvedAlias(_: Generator, _) => false
+ case Alias(_: Generator, _) => false
+ case MultiAlias(_: Generator, _) => false
+ case other => hasGenerator(other)
+ }
+
+ private def trimAlias(expr: NamedExpression): Expression = expr match {
+ case UnresolvedAlias(child, _) => child
+ case Alias(child, _) => child
+ case MultiAlias(child, _) => child
+ case _ => expr
+ }
+
+ /** Extracts a [[Generator]] expression and any names assigned by aliases to their output. */
+ private object AliasedGenerator {
+ def unapply(e: Expression): Option[(Generator, Seq[String])] = e match {
+ case Alias(g: Generator, name) if g.resolved && g.elementSchema.length > 1 =>
+ // If not given the default names, and the TGF with multiple output columns
+ failAnalysis(
+ s"""Expect multiple names given for ${g.getClass.getName},
+ |but only single name '$name' specified""".stripMargin)
+ case Alias(g: Generator, name) if g.resolved => Some((g, name :: Nil))
+ case MultiAlias(g: Generator, names) if g.resolved => Some(g, names)
+ case _ => None
+ }
+ }
+
def apply(plan: LogicalPlan): LogicalPlan = plan resolveOperators {
- case p: Generate if !p.child.resolved || !p.generator.resolved => p
- case g: Generate if !g.resolved =>
- g.copy(generatorOutput = makeGeneratorOutput(g.generator, g.generatorOutput.map(_.name)))
+ case Project(projectList, _) if projectList.exists(hasNestedGenerator) =>
+ val nestedGenerator = projectList.find(hasNestedGenerator).get
+ throw new AnalysisException("Generators are not supported when it's nested in " +
+ "expressions, but got: " + toPrettySQL(trimAlias(nestedGenerator)))
+
+ case Project(projectList, _) if projectList.count(hasGenerator) > 1 =>
+ val generators = projectList.flatMap(_.find(_.isInstanceOf[Generator]))
+ throw new AnalysisException("Only one generator allowed per select clause but found " +
+ generators.size + ": " + generators.map(toPrettySQL).mkString(", "))
--- End diff --
MINOR: Shouldn't we `trimAlias` here?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-215951648
**[Test build #57436 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57436/consoleFull)** for PR 12810 at commit [`d986d3e`](https://github.com/apache/spark/commit/d986d3eafebb031aec0f1e151a81c48b9712f6a4).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-216455636
**[Test build #57606 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57606/consoleFull)** for PR 12810 at commit [`35b3c2b`](https://github.com/apache/spark/commit/35b3c2b13ac26c5017057d3863c28b3c69fd8861).
* This patch **fails to build**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15029] improve error message for Genera...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12810#issuecomment-215950427
**[Test build #57434 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57434/consoleFull)** for PR 12810 at commit [`71dd8dd`](https://github.com/apache/spark/commit/71dd8ddff0b6d655029f9ba07982e3993a5063ad).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org