You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by vanzin <gi...@git.apache.org> on 2018/11/16 21:54:39 UTC
[GitHub] spark pull request #23061: [SPARK-26095][build] Disable parallelization in m...
GitHub user vanzin opened a pull request:
https://github.com/apache/spark/pull/23061
[SPARK-26095][build] Disable parallelization in make-distibution.sh.
It makes the build slower, but at least it doesn't hang. Seems that
maven-shade-plugin has some issue with parallelization.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/vanzin/spark SPARK-26095
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/23061.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #23061
----
commit 2bf6e61d390ad3f4abc28a141bbdf11f71fbd08f
Author: Marcelo Vanzin <va...@...>
Date: 2018-11-16T21:52:49Z
[SPARK-26095][build] Disable parallelization in make-distibution.sh.
It makes the build slower, but at least it doesn't hang. Seems that
maven-shade-plugin has some issue with parallelization.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #23061: [SPARK-26095][build] Disable parallelization in make-dis...
Posted by shaneknapp <gi...@git.apache.org>.
Github user shaneknapp commented on the issue:
https://github.com/apache/spark/pull/23061
thanks @vanzin
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #23061: [SPARK-26095][build] Disable parallelization in make-dis...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23061
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #23061: [SPARK-26095][build] Disable parallelization in make-dis...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23061
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #23061: [SPARK-26095][build] Disable parallelization in make-dis...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/23061
BTW noticed this:
```
Step 6/9 : COPY R ${SPARK_HOME}/R
COPY failed: stat /var/lib/docker/tmp/docker-builder004084591/R: no such file or directory
Failed to build SparkR Docker image, please refer to Docker build output for details.
```
Are the R profile and the R integration tests intentionally disabled? `KubernetesSuite` doesn't mix in `RTestsSuite`, which is why it hasn't failed...
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #23061: [SPARK-26095][build] Disable parallelization in make-dis...
Posted by shaneknapp <gi...@git.apache.org>.
Github user shaneknapp commented on the issue:
https://github.com/apache/spark/pull/23061
edit the subject, add [k8s], then ask jenkins to trigger a build.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #23061: [SPARK-26095][build] Disable parallelization in make-dis...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23061
**[Test build #98934 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98934/testReport)** for PR 23061 at commit [`2bf6e61`](https://github.com/apache/spark/commit/2bf6e61d390ad3f4abc28a141bbdf11f71fbd08f).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #23061: [SPARK-26095][build] Disable parallelization in make-dis...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23061
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5091/
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #23061: [SPARK-26095][build] Disable parallelization in m...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/23061
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #23061: [SPARK-26095][build][k8s] Disable parallelization in mak...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23061
**[Test build #98934 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98934/testReport)** for PR 23061 at commit [`2bf6e61`](https://github.com/apache/spark/commit/2bf6e61d390ad3f4abc28a141bbdf11f71fbd08f).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #23061: [SPARK-26095][build] Disable parallelization in make-dis...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23061
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5090/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #23061: [SPARK-26095][build] Disable parallelization in make-dis...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23061
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98933/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #23061: [SPARK-26095][build] Disable parallelization in make-dis...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23061
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5091/
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #23061: [SPARK-26095][build] Disable parallelization in make-dis...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23061
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5091/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #23061: [SPARK-26095][build] Disable parallelization in make-dis...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23061
**[Test build #98933 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98933/testReport)** for PR 23061 at commit [`2bf6e61`](https://github.com/apache/spark/commit/2bf6e61d390ad3f4abc28a141bbdf11f71fbd08f).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #23061: [SPARK-26095][build] Disable parallelization in make-dis...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23061
**[Test build #98933 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98933/testReport)** for PR 23061 at commit [`2bf6e61`](https://github.com/apache/spark/commit/2bf6e61d390ad3f4abc28a141bbdf11f71fbd08f).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #23061: [SPARK-26095][build] Disable parallelization in make-dis...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/23061
@shaneknapp not sure if need something special to kick a k8s build.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #23061: [SPARK-26095][build] Disable parallelization in make-dis...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23061
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98934/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #23061: [SPARK-26095][build] Disable parallelization in make-dis...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23061
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #23061: [SPARK-26095][build] Disable parallelization in make-dis...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23061
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #23061: [SPARK-26095][build] Disable parallelization in make-dis...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/23061
Seems that k8s is not needed and jenkins was just backed up? I killed the stuck jobs.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #23061: [SPARK-26095][build] Disable parallelization in make-dis...
Posted by shaneknapp <gi...@git.apache.org>.
Github user shaneknapp commented on the issue:
https://github.com/apache/spark/pull/23061
> BTW noticed this:
>
> ```
> Step 6/9 : COPY R ${SPARK_HOME}/R
> COPY failed: stat /var/lib/docker/tmp/docker-builder004084591/R: no such file or directory
> Failed to build SparkR Docker image, please refer to Docker build output for details.
> ```
> Are the R profile and the R integration tests intentionally disabled? `KubernetesSuite` doesn't mix in `RTestsSuite`, which is why it hasn't failed...
yeah, intentionally disabled.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #23061: [SPARK-26095][build] Disable parallelization in make-dis...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/23061
I'm going to merge this to unblock k8s integration tests. If needed we can figure out the underlying issue some other time.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #23061: [SPARK-26095][build][k8s] Disable parallelization in mak...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/23061
retest this please
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #23061: [SPARK-26095][build] Disable parallelization in make-dis...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/23061
(ok it does need the title change, but https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5091/ is running with that.)
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org