You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by tdas <gi...@git.apache.org> on 2014/11/17 21:02:32 UTC
[GitHub] spark pull request: [SPARK-4458][Build] Eliminate compilation of t...
GitHub user tdas opened a pull request:
https://github.com/apache/spark/pull/3324
[SPARK-4458][Build] Eliminate compilation of test classes in make distribution.
The make-distribution generates Spark distributions, and therefore does not require building of test classes. Maven -DskipTests only eliminates running of tests, but does not eliminate compilation of tests (see [Maven docs](http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#skipTests)). To prevent compilation of tests (reduces compilation time), we need to add -Dmaven.test.skip (again see [Maven docs](http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#skip)).
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tdas/spark SPARK-4458
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/3324.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3324
----
commit ea8e249ad6d22b7086c2849c7da76c646e53c172
Author: Tathagata Das <ta...@gmail.com>
Date: 2014-11-17T19:59:03Z
Eliminate compilation of test classes in make distribution.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4458][Build] Eliminate compilation of t...
Posted by tdas <gi...@git.apache.org>.
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/3324#issuecomment-63719582
Well, it doesnt add much. But we upgraded our spark-perf testsuite to use make-distribution, and these compilations add up a little and slow down the test cycle.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4458][Build] Eliminate compilation of t...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3324#issuecomment-63367328
[Test build #23493 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23493/consoleFull) for PR 3324 at commit [`ea8e249`](https://github.com/apache/spark/commit/ea8e249ad6d22b7086c2849c7da76c646e53c172).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4458][Build] Eliminate compilation of t...
Posted by pwendell <gi...@git.apache.org>.
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/3324#issuecomment-66530164
Yeah actually I like the fact that we build these as a "sanity test" for making sure the distribution is working. Given that it's very small compared with the build time of Spark I'd like to keep it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4458][Build] Eliminate compilation of t...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3324#issuecomment-63379511
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23493/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4458][Build] Eliminate compilation of t...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/3324#issuecomment-63379907
Just curious, is the time overhead large and a burden? Distributions are rarely created. I wonder if compiling the tests at least is still a good sanity check in a distribution build process.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4458][Build] Eliminate compilation of t...
Posted by tdas <gi...@git.apache.org>.
Github user tdas closed the pull request at:
https://github.com/apache/spark/pull/3324
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4458][Build] Eliminate compilation of t...
Posted by tdas <gi...@git.apache.org>.
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/3324#issuecomment-63366345
@pwendell Take a look.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4458][Build] Eliminate compilation of t...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3324#issuecomment-63379505
[Test build #23493 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23493/consoleFull) for PR 3324 at commit [`ea8e249`](https://github.com/apache/spark/commit/ea8e249ad6d22b7086c2849c7da76c646e53c172).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4458][Build] Eliminate compilation of t...
Posted by tdas <gi...@git.apache.org>.
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/3324#issuecomment-66613560
Alright, I am going to close this PR then.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org