You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by chenghao-intel <gi...@git.apache.org> on 2015/04/16 20:46:29 UTC
[GitHub] spark pull request: [SPARK-4233] [SQL] [WIP] UDAF Interface Refact...
GitHub user chenghao-intel opened a pull request:
https://github.com/apache/spark/pull/5542
[SPARK-4233] [SQL] [WIP] UDAF Interface Refactoring
This PR will keep both old / new versions of UDAF, and switch them by
```
SET spark.sql.aggregate2=true/false;
```
The new interface is
```scala
trait AggregateFunction2 {
self: Product =>
// Specify the BoundReference for Aggregate Buffer
def initialize(buffers: Seq[BoundReference]): Unit
// Initialize (reinitialize) the aggregation buffer
def reset(buf: MutableRow): Unit
// Get the children value from the input row, and then
// merge it with the given aggregate buffer,
// `seen` is the set that the value showed up, that's will
// be useful for distinct aggregate. And it probably be
// null for non-distinct aggregate
def update(input: Row, buf: MutableRow, seen: JSet[Any]): Unit
// Merge 2 aggregation buffers, and write back to the later one
def merge(value: Row, buf: MutableRow): Unit
// Semantically we probably don't need this, however, we need it when
// integrating with Hive UDAF(GenericUDAF)
@deprecated
def terminatePartial(buf: MutableRow): Unit = {}
// Output the final result by feeding the aggregation buffer
def terminate(buffer: Row): Any
}
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/chenghao-intel/spark udaf_refactor
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5542.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5542
----
commit a6fdd419a63c241f2d21ca3bebc0222cf96e9280
Author: Cheng Hao <ha...@intel.com>
Date: 2015-04-11T06:08:23Z
migrate to support both version of UDAF
commit bfe30158023c6f876fd9112d456e749b35afd40c
Author: Cheng Hao <ha...@intel.com>
Date: 2015-04-11T06:33:12Z
Update the unit test to comment out the not support ones
commit 3a8232ccdf5abcfd6962be8dc6f3e50dbb8a6f88
Author: Cheng Hao <ha...@intel.com>
Date: 2015-04-13T23:05:37Z
update the interface name
commit dced96cde1f650f1fd098cc77504d6858febc104
Author: Cheng Hao <ha...@intel.com>
Date: 2015-04-15T02:43:40Z
change the update method from Any to Row
commit 29202bd89e028ce534b1b986c3444c362c29a3d8
Author: Cheng Hao <ha...@intel.com>
Date: 2015-04-15T18:29:51Z
move the distinct into the udaf
commit 967716b74d7cdbd3ea382e5b657a3952b3ef585f
Author: Cheng Hao <ha...@intel.com>
Date: 2015-04-16T07:50:31Z
simpify the aggregate expression by uing the Projection
commit 504fbe52abc191e873cc741fe9ef44d9e4013d7a
Author: Cheng Hao <ha...@intel.com>
Date: 2015-04-16T08:23:43Z
revert the uncessary changes
commit e9017ed23e946a3cc0d7e5142c05a19039491c58
Author: Cheng Hao <ha...@intel.com>
Date: 2015-04-16T17:45:12Z
Add Unit test
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] [WIP] UDAF Interface Refact...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5542#issuecomment-95870924
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30921/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] [WIP] UDAF Interface Refact...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5542#issuecomment-94633905
[Test build #30629 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30629/consoleFull) for PR 5542 at commit [`4aa56c2`](https://github.com/apache/spark/commit/4aa56c239cef347efdcc3312a0c567fafccca06e).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] [WIP] UDAF Interface Refact...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5542#issuecomment-93886910
[Test build #30451 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30451/consoleFull) for PR 5542 at commit [`e213e5e`](https://github.com/apache/spark/commit/e213e5ea9d744d5733335287874dcb92a771b65e).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `trait AggregateFunction2 `
* `trait AggregateExpression2 extends Expression with AggregateFunction2 `
* `abstract class UnaryAggregateExpression extends UnaryExpression with AggregateExpression2 `
* `case class Min(child: Expression) extends UnaryAggregateExpression `
* `case class Average(child: Expression, distinct: Boolean = false)`
* `case class Max(child: Expression) extends UnaryAggregateExpression `
* `case class Count(child: Expression)`
* `case class CountDistinct(children: Seq[Expression])`
* `case class Sum(child: Expression, distinct: Boolean = false)`
* `case class First(child: Expression, distinct: Boolean = false)`
* `case class Last(child: Expression, distinct: Boolean = false)`
* `class AggregateExpressionSubsitution `
* ` class HashAggregation2(aggrSubsitution: AggregateExpressionSubsitution) extends Strategy `
* `sealed class BufferSeens(var buffer: MutableRow, var seens: Array[JSet[Any]] = null) `
* `sealed class BufferAndKey(leftLen: Int, rightLen: Int)`
* `sealed trait Aggregate `
* `sealed trait PostShuffle extends Aggregate `
* `case class AggregatePreShuffle(`
* `case class AggregatePostShuffle(`
* `case class DistinctAggregate(`
* This patch does not change any dependencies.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] [WIP] UDAF Interface Refact...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5542#issuecomment-93813367
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30431/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] [WIP] UDAF Interface Refact...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5542#issuecomment-94634039
[Test build #30629 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30629/consoleFull) for PR 5542 at commit [`4aa56c2`](https://github.com/apache/spark/commit/4aa56c239cef347efdcc3312a0c567fafccca06e).
* This patch **fails Scala style tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `trait AggregateFunction2 `
* `trait AggregateExpression2 extends Expression with AggregateFunction2 `
* `abstract class UnaryAggregateExpression extends UnaryExpression with AggregateExpression2 `
* `case class Min(child: Expression) extends UnaryAggregateExpression `
* `case class Average(child: Expression, distinct: Boolean = false)`
* `case class Max(child: Expression) extends UnaryAggregateExpression `
* `case class Count(child: Expression)`
* `case class CountDistinct(children: Seq[Expression])`
* `case class Sum(child: Expression, distinct: Boolean = false)`
* `case class First(child: Expression, distinct: Boolean = false)`
* `case class Last(child: Expression, distinct: Boolean = false)`
* `class AggregateExpressionSubsitution `
* ` class HashAggregation2(aggrSubsitution: AggregateExpressionSubsitution) extends Strategy `
* `sealed class BufferSeens(var buffer: MutableRow, var seens: Array[JSet[Any]] = null) `
* `sealed class BufferAndKey(leftLen: Int, rightLen: Int)`
* `sealed trait Aggregate `
* `sealed trait PostShuffle extends Aggregate `
* `case class AggregatePreShuffle(`
* `case class AggregatePostShuffle(`
* `case class DistinctAggregate(`
* This patch does not change any dependencies.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] [WIP] UDAF Interface Refact...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5542#issuecomment-94647393
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30637/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] [WIP] UDAF Interface Refact...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5542#issuecomment-94634040
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30629/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] [WIP] UDAF Interface Refact...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5542#issuecomment-94687542
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30644/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] [WIP] UDAF Interface Refact...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5542#issuecomment-95870914
[Test build #30921 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30921/consoleFull) for PR 5542 at commit [`6b594f0`](https://github.com/apache/spark/commit/6b594f05ef2725aa5f6bed716dbac6eed64a1879).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `trait AggregateFunction2 `
* `trait AggregateExpression2 extends Expression with AggregateFunction2 `
* `abstract class UnaryAggregateExpression extends UnaryExpression with AggregateExpression2 `
* `case class Min(child: Expression) extends UnaryAggregateExpression `
* `case class Average(child: Expression, distinct: Boolean = false)`
* `case class Max(child: Expression) extends UnaryAggregateExpression `
* `case class Count(child: Expression)`
* `case class CountDistinct(children: Seq[Expression])`
* `case class Sum(child: Expression, distinct: Boolean = false)`
* `case class First(child: Expression, distinct: Boolean = false)`
* `case class Last(child: Expression, distinct: Boolean = false)`
* `class AggregateExpressionSubsitution `
* ` class HashAggregation2(aggrSubsitution: AggregateExpressionSubsitution) extends Strategy `
* `sealed class BufferSeens(var buffer: MutableRow, var seens: Array[JSet[Any]] = null) `
* `sealed class BufferAndKey(leftLen: Int, rightLen: Int)`
* `sealed trait Aggregate `
* `sealed trait PostShuffle extends Aggregate `
* `case class AggregatePreShuffle(`
* `case class AggregatePostShuffle(`
* `case class DistinctAggregate(`
* This patch does not change any dependencies.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] [WIP] UDAF Interface Refact...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5542#issuecomment-94687508
[Test build #30644 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30644/consoleFull) for PR 5542 at commit [`9806266`](https://github.com/apache/spark/commit/9806266ae0a68c943a2a7a7d65787eba0164db89).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `trait AggregateFunction2 `
* `trait AggregateExpression2 extends Expression with AggregateFunction2 `
* `abstract class UnaryAggregateExpression extends UnaryExpression with AggregateExpression2 `
* `case class Min(child: Expression) extends UnaryAggregateExpression `
* `case class Average(child: Expression, distinct: Boolean = false)`
* `case class Max(child: Expression) extends UnaryAggregateExpression `
* `case class Count(child: Expression)`
* `case class CountDistinct(children: Seq[Expression])`
* `case class Sum(child: Expression, distinct: Boolean = false)`
* `case class First(child: Expression, distinct: Boolean = false)`
* `case class Last(child: Expression, distinct: Boolean = false)`
* `class AggregateExpressionSubsitution `
* ` class HashAggregation2(aggrSubsitution: AggregateExpressionSubsitution) extends Strategy `
* `sealed class BufferSeens(var buffer: MutableRow, var seens: Array[JSet[Any]] = null) `
* `sealed class BufferAndKey(leftLen: Int, rightLen: Int)`
* `sealed trait Aggregate `
* `sealed trait PostShuffle extends Aggregate `
* `case class AggregatePreShuffle(`
* `case class AggregatePostShuffle(`
* `case class DistinctAggregate(`
* This patch does not change any dependencies.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] [WIP] UDAF Interface Refact...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5542#issuecomment-93876621
[Test build #30451 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30451/consoleFull) for PR 5542 at commit [`e213e5e`](https://github.com/apache/spark/commit/e213e5ea9d744d5733335287874dcb92a771b65e).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] [WIP] UDAF Interface Refact...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5542#issuecomment-95834836
[Test build #30921 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30921/consoleFull) for PR 5542 at commit [`6b594f0`](https://github.com/apache/spark/commit/6b594f05ef2725aa5f6bed716dbac6eed64a1879).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] [WIP] UDAF Interface Refact...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5542#issuecomment-93812712
[Test build #30431 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30431/consoleFull) for PR 5542 at commit [`e9017ed`](https://github.com/apache/spark/commit/e9017ed23e946a3cc0d7e5142c05a19039491c58).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] [WIP] UDAF Interface Refact...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5542#issuecomment-93886922
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30451/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] [WIP] UDAF Interface Refact...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5542#issuecomment-94647388
[Test build #30637 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30637/consoleFull) for PR 5542 at commit [`b45f487`](https://github.com/apache/spark/commit/b45f487fa269df60aa252a19d54140e652b0c7c0).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `trait AggregateFunction2 `
* `trait AggregateExpression2 extends Expression with AggregateFunction2 `
* `abstract class UnaryAggregateExpression extends UnaryExpression with AggregateExpression2 `
* `case class Min(child: Expression) extends UnaryAggregateExpression `
* `case class Average(child: Expression, distinct: Boolean = false)`
* `case class Max(child: Expression) extends UnaryAggregateExpression `
* `case class Count(child: Expression)`
* `case class CountDistinct(children: Seq[Expression])`
* `case class Sum(child: Expression, distinct: Boolean = false)`
* `case class First(child: Expression, distinct: Boolean = false)`
* `case class Last(child: Expression, distinct: Boolean = false)`
* `class AggregateExpressionSubsitution `
* ` class HashAggregation2(aggrSubsitution: AggregateExpressionSubsitution) extends Strategy `
* `sealed class BufferSeens(var buffer: MutableRow, var seens: Array[JSet[Any]] = null) `
* `sealed class BufferAndKey(leftLen: Int, rightLen: Int)`
* `sealed trait Aggregate `
* `sealed trait PostShuffle extends Aggregate `
* `case class AggregatePreShuffle(`
* `case class AggregatePostShuffle(`
* `case class DistinctAggregate(`
* This patch does not change any dependencies.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] [WIP] UDAF Interface Refact...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5542#issuecomment-94653175
[Test build #30644 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30644/consoleFull) for PR 5542 at commit [`9806266`](https://github.com/apache/spark/commit/9806266ae0a68c943a2a7a7d65787eba0164db89).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] [WIP] UDAF Interface Refact...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5542#issuecomment-95788877
[Test build #30901 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30901/consoleFull) for PR 5542 at commit [`71f1bd5`](https://github.com/apache/spark/commit/71f1bd538b3e0befead2d1d592ce12990cb9b417).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
* This patch does not change any dependencies.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] [WIP] UDAF Interface Refact...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5542#issuecomment-94642985
[Test build #30637 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30637/consoleFull) for PR 5542 at commit [`b45f487`](https://github.com/apache/spark/commit/b45f487fa269df60aa252a19d54140e652b0c7c0).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] [WIP] UDAF Interface Refact...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5542#issuecomment-95788883
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30901/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] [WIP] UDAF Interface Refact...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5542#issuecomment-93813358
[Test build #30431 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30431/consoleFull) for PR 5542 at commit [`e9017ed`](https://github.com/apache/spark/commit/e9017ed23e946a3cc0d7e5142c05a19039491c58).
* This patch **fails Scala style tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `trait AggregateFunction2 `
* `trait AggregateExpression2 extends Expression with AggregateFunction2 `
* `abstract class UnaryAggregateExpression extends UnaryExpression with AggregateExpression2 `
* `case class Min(`
* `case class Average(child: Expression, distinct: Boolean = false)`
* `case class Max(child: Expression)`
* `case class Count(child: Expression)`
* `case class CountDistinct(children: Seq[Expression])`
* `case class Sum(child: Expression, distinct: Boolean = false)`
* `case class First(child: Expression, distinct: Boolean = false)`
* `case class Last(child: Expression, distinct: Boolean = false)`
* `class AggregateExpressionSubsitution `
* ` class HashAggregation2(aggrSubsitution: AggregateExpressionSubsitution) extends Strategy `
* `sealed class BufferSeens(var buffer: MutableRow, var seens: Array[JSet[Any]] = null) `
* `sealed trait Aggregate `
* `sealed trait PostShuffle extends Aggregate `
* `case class AggregatePreShuffle(`
* `case class AggregatePostShuffle(`
* `case class DistinctAggregate(`
* This patch does not change any dependencies.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] [WIP] UDAF Interface Refact...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5542#issuecomment-95774441
[Test build #30901 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30901/consoleFull) for PR 5542 at commit [`71f1bd5`](https://github.com/apache/spark/commit/71f1bd538b3e0befead2d1d592ce12990cb9b417).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] [WIP] UDAF Interface Refact...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5542#issuecomment-96767683
[Test build #31007 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31007/consoleFull) for PR 5542 at commit [`6b594f0`](https://github.com/apache/spark/commit/6b594f05ef2725aa5f6bed716dbac6eed64a1879).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4233] [SQL] [WIP] UDAF Interface Refact...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/5542#discussion_r28945967
--- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala ---
@@ -562,3 +563,13 @@ class SQLQuerySuite extends QueryTest {
.queryExecution.analyzed
}
}
+
+class SQLQuerySuite2 extends SQLQuerySuite with BeforeAndAfter {
--- End diff --
you should name this something more explicit, maybe "SQLQueryNewUDAFSuite"
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org