You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by kai-zeng <gi...@git.apache.org> on 2015/02/09 13:25:23 UTC
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
GitHub user kai-zeng opened a pull request:
https://github.com/apache/spark/pull/4472
[SQL] Optimize arithmetic and predicate operators
Existing implementation of arithmetic operators and BinaryComparison operators have redundant type checking codes, e.g.:
Expression.n2 is used by Add/Subtract/Multiply.
(1) n2 always checks left.dataType == right.dataType. However, this checking should be done once when we resolve expression types;
(2) n2 requires dataType is a NumericType. This can be done once.
This PR optimize arithmetic and predicate operators by removing such redundant type-checking codes.
Some preliminary benchmarking on 10G TPC-H data over 5 r3.2xlarge EC2 machines show that this PR can reduce the query time by 5.5% to 11%.
The benchmark queries follow the template below, where OP is plus/minus/times/divide/remainder/bitwise and/bitwise or/bitwise xor:
SELECT l_returnflag, l_linestatus, SUM(l_quantity OP cnt1), SUM(l_quantity OP cnt2), ...., SUM(l_quantity OP cnt700)
FROM (
SELECT l_returnflag, l_linestatus, l_quantity, 1 AS cnt1, 2 AS cnt2, ..., 700 AS cnt700
FROM lineitem
WHERE l_shipdate <= '1998-09-01'
)
GROUP BY l_returnflag, l_linestatus;
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kai-zeng/spark arithmetic-optimize
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/4472.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4472
----
commit 24d062f62c4b714074fdde4844f4642a9a8749e7
Author: kai <ka...@eecs.berkeley.edu>
Date: 2015-02-07T04:51:32Z
test suite
commit fd95823e8486ddfb857650c5f351ccf416593ae5
Author: kai <ka...@eecs.berkeley.edu>
Date: 2015-02-07T04:52:12Z
remove unnecessary type checking
commit 16fd84c0ff88386e99c1a5b685d0e7974ef882a9
Author: kai <ka...@eecs.berkeley.edu>
Date: 2015-02-07T13:25:15Z
optimize + - * / % -(unary) abs < > <= >=
commit 03fd0c3737aa40c9fcafa6a15df9d3ff17ce72bf
Author: kai <ka...@eecs.berkeley.edu>
Date: 2015-02-07T14:47:28Z
fix predicate
commit 1cd7571a43eece267244badf57c28565f44ba076
Author: kai <ka...@eecs.berkeley.edu>
Date: 2015-02-08T02:18:59Z
fix sqrt and maxof
commit 12c5b3257a90180250cc92079891e7c83799c91f
Author: kai <ka...@eecs.berkeley.edu>
Date: 2015-02-08T05:06:57Z
override eval
commit b34d58d83781c19f10c7911c21d26d1227c42e67
Author: kai <ka...@eecs.berkeley.edu>
Date: 2015-02-08T05:26:12Z
add caching and benmark option
commit 62abbbc4aefb17558e99872b86c68349059ad418
Author: kai <ka...@eecs.berkeley.edu>
Date: 2015-02-09T02:41:41Z
clean up predicate and arithmetic
commit 0906c39d3f597c8b8293c1e4519fcd64d6e60701
Author: kai <ka...@eecs.berkeley.edu>
Date: 2015-02-09T02:50:11Z
add bitwise test
commit 97a7d6c5d30160c8fb19e94a17f38926d866298c
Author: kai <ka...@eecs.berkeley.edu>
Date: 2015-02-09T03:52:32Z
bitwise-and: override evalInternal using and func
commit cb92ae170acf612e7dfc23c474a77dccdf2ec397
Author: kai <ka...@eecs.berkeley.edu>
Date: 2015-02-09T03:54:33Z
bitwise-and: override eval
commit 86297e272a281ec4d0fff668d4b4fdc293343dde
Author: kai <ka...@eecs.berkeley.edu>
Date: 2015-02-09T06:24:12Z
generalized
commit 31ccdd400e39503f6bd279d7af501a9faac0ded5
Author: kai <ka...@eecs.berkeley.edu>
Date: 2015-02-09T06:57:48Z
rewrite all bitwise op and remove evalInternal
commit f8eba24b8ad120e65ff3ebb2e066105ce7473b79
Author: kai <ka...@eecs.berkeley.edu>
Date: 2015-02-09T07:58:53Z
override evalInternal
commit 6892fc48ab0a5b01f1ba3ab29db25e5b734deab5
Author: kai <ka...@eecs.berkeley.edu>
Date: 2015-02-09T08:56:05Z
revert override evalInternal
commit 8fa84a13b5d85b55a3f21f3c6a80a1048766b7d7
Author: kai <ka...@eecs.berkeley.edu>
Date: 2015-02-09T09:01:17Z
add bitwise or and xor
commit ca4780144501d0fb4c6f99f74d23817bfa618d7b
Author: kai <ka...@eecs.berkeley.edu>
Date: 2015-02-09T11:58:55Z
override evalInternal for bitwise ops
commit 3cbd3636f999a69f54e616b4070a1541fa421849
Author: kai <ka...@eecs.berkeley.edu>
Date: 2015-02-09T11:59:48Z
clean up test code
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by kai-zeng <gi...@git.apache.org>.
Github user kai-zeng commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-73523394
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-73510687
[Test build #27102 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27102/consoleFull) for PR 4472 at commit [`9c40bc0`](https://github.com/apache/spark/commit/9c40bc07e7d6c7695d4e90926c1e5d5328d15010).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-74479736
`aggregates.scala` is the only file in folder `expressions` which is executable, I think it was a mistake and we should make it un-executable.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-73510697
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27102/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-73523672
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27104/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by kai-zeng <gi...@git.apache.org>.
Github user kai-zeng commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-74474525
@chenghao-intel Done rebasing
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-73501771
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27101/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-73629667
[Test build #27142 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27142/consoleFull) for PR 4472 at commit [`7f6e968`](https://github.com/apache/spark/commit/7f6e968285e4091e295ee6aa0ceb0d5970d5bbff).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-73513477
[Test build #27103 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27103/consoleFull) for PR 4472 at commit [`7490dbc`](https://github.com/apache/spark/commit/7490dbc8f41374aff260a5643f3b38924d1d0813).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-74478946
[Test build #27548 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27548/consoleFull) for PR 4472 at commit [`5a41e49`](https://github.com/apache/spark/commit/5a41e496b3413dc68d1df1dfdb2c39428e05441a).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-74485193
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27548/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-73505392
[Test build #27102 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27102/consoleFull) for PR 4472 at commit [`9c40bc0`](https://github.com/apache/spark/commit/9c40bc07e7d6c7695d4e90926c1e5d5328d15010).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-74576851
[Test build #27576 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27576/consoleFull) for PR 4472 at commit [`fef0cf1`](https://github.com/apache/spark/commit/fef0cf18401970eaf64e76b1fd05f803a4e43729).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-74585173
[Test build #27576 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27576/consoleFull) for PR 4472 at commit [`fef0cf1`](https://github.com/apache/spark/commit/fef0cf18401970eaf64e76b1fd05f803a4e43729).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-73500659
[Test build #27101 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27101/consoleFull) for PR 4472 at commit [`3cbd363`](https://github.com/apache/spark/commit/3cbd3636f999a69f54e616b4070a1541fa421849).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-73538427
[Test build #27105 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27105/consoleFull) for PR 4472 at commit [`6cddb46`](https://github.com/apache/spark/commit/6cddb46962ac24bfdf42068637f40ac8098979dc).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-74485187
[Test build #27548 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27548/consoleFull) for PR 4472 at commit [`5a41e49`](https://github.com/apache/spark/commit/5a41e496b3413dc68d1df1dfdb2c39428e05441a).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-73513591
[Test build #27103 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27103/consoleFull) for PR 4472 at commit [`7490dbc`](https://github.com/apache/spark/commit/7490dbc8f41374aff260a5643f3b38924d1d0813).
* This patch **fails Scala style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/4472
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by marmbrus <gi...@git.apache.org>.
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-74593651
Thanks! merged to master and branch 1.3
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-73513595
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27103/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by kai-zeng <gi...@git.apache.org>.
Github user kai-zeng commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-74576543
@cloud-fan Yeah, aggregates.scala is executable in spark master. And I have made it un-executable in this PR. BTW, these files are also executable in spark master (I chmod them too):
./catalyst/src/main/scala/org/apache/spark/sql/catalyst/dsl/package.scala
./catalyst/src/main/scala/org/apache/spark/sql/catalyst/SqlParser.scala
./catalyst/src/main/scala/org/apache/spark/sql/types/Metadata.scala
./catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/MetadataSuite.scala
./core/src/main/scala/org/apache/spark/sql/execution/Aggregate.scala
./hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-74585177
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27576/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by kai-zeng <gi...@git.apache.org>.
Github user kai-zeng commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-74478451
@chenghao-intel you mean revert aggregates.scala to executable?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-73501760
[Test build #27101 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27101/consoleFull) for PR 4472 at commit [`3cbd363`](https://github.com/apache/spark/commit/3cbd3636f999a69f54e616b4070a1541fa421849).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-73621509
[Test build #27142 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27142/consoleFull) for PR 4472 at commit [`7f6e968`](https://github.com/apache/spark/commit/7f6e968285e4091e295ee6aa0ceb0d5970d5bbff).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by chenghao-intel <gi...@git.apache.org>.
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-74476383
@kai-zeng can you also revert the file permission bit for `aggregates.scala`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by chenghao-intel <gi...@git.apache.org>.
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-74463234
@kai-zeng The code LGTM, and I've tested the `Add` in an aggregation benchmark, it shows about 20% performance improved. So can you rebase the code?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-73524507
[Test build #27105 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27105/consoleFull) for PR 4472 at commit [`6cddb46`](https://github.com/apache/spark/commit/6cddb46962ac24bfdf42068637f40ac8098979dc).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-74479214
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27545/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-73629671
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27142/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by marmbrus <gi...@git.apache.org>.
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-73620361
Thanks for doing this. Can you undo the permission changes on the files `100644 → 100755`. I think you made them executable.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-73523669
[Test build #27104 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27104/consoleFull) for PR 4472 at commit [`7490dbc`](https://github.com/apache/spark/commit/7490dbc8f41374aff260a5643f3b38924d1d0813).
* This patch **fails Scala style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-73523508
[Test build #27104 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27104/consoleFull) for PR 4472 at commit [`7490dbc`](https://github.com/apache/spark/commit/7490dbc8f41374aff260a5643f3b38924d1d0813).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-74472313
[Test build #27545 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27545/consoleFull) for PR 4472 at commit [`cb37c94`](https://github.com/apache/spark/commit/cb37c94258f8fbde40b7f55c092e7efeb9bb822e).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-73538438
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27105/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SQL] Optimize arithmetic and predicate operat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4472#issuecomment-74479209
[Test build #27545 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27545/consoleFull) for PR 4472 at commit [`cb37c94`](https://github.com/apache/spark/commit/cb37c94258f8fbde40b7f55c092e7efeb9bb822e).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org