You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by gaborgsomogyi <gi...@git.apache.org> on 2018/02/14 13:06:44 UTC
[GitHub] spark pull request #20608: [SPARK-23422][Core] YarnShuffleIntegrationSuite f...
GitHub user gaborgsomogyi opened a pull request:
https://github.com/apache/spark/pull/20608
[SPARK-23422][Core] YarnShuffleIntegrationSuite fix when SPARK_PREPEN…
…D_CLASSES set to 1
## What changes were proposed in this pull request?
YarnShuffleIntegrationSuite fails when SPARK_PREPEND_CLASSES set to 1.
Normally mllib built before yarn module. When SPARK_PREPEND_CLASSES used mllib classes are on yarn test classpath.
Before 2.3 that did not cause issues. But 2.3 has SPARK-22450, which registered some mllib classes with the kryo serializer. Now it dies with the following error:
`
18/02/13 07:33:29 INFO SparkContext: Starting job: collect at YarnShuffleIntegrationSuite.scala:143
Exception in thread "dag-scheduler-event-loop" java.lang.NoClassDefFoundError: breeze/linalg/DenseMatrix
`
In this PR NoClassDefFoundError caught only in case of testing and then do nothing.
## How was this patch tested?
Automated: Pass the Jenkins.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gaborgsomogyi/spark SPARK-23422
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20608.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20608
----
commit 9c1c415e4c58b334bc1339754fae75f2e991dfa2
Author: Gabor Somogyi <ga...@...>
Date: 2018-02-14T12:55:05Z
[SPARK-23422][Core] YarnShuffleIntegrationSuite fix when SPARK_PREPEND_CLASSES set to 1
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20608: [SPARK-23422][Core] YarnShuffleIntegrationSuite fix when...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20608
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20608: [SPARK-23422][Core] YarnShuffleIntegrationSuite fix when...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20608
ok to test
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20608: [SPARK-23422][Core] YarnShuffleIntegrationSuite fix when...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20608
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20608: [SPARK-23422][Core] YarnShuffleIntegrationSuite fix when...
Posted by gaborgsomogyi <gi...@git.apache.org>.
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/20608
cc @ConeyLiu @vanzin
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20608: [SPARK-23422][Core] YarnShuffleIntegrationSuite fix when...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20608
**[Test build #87449 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87449/testReport)** for PR 20608 at commit [`9c1c415`](https://github.com/apache/spark/commit/9c1c415e4c58b334bc1339754fae75f2e991dfa2).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20608: [SPARK-23422][Core] YarnShuffleIntegrationSuite fix when...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20608
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20608: [SPARK-23422][Core] YarnShuffleIntegrationSuite fix when...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20608
**[Test build #87449 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87449/testReport)** for PR 20608 at commit [`9c1c415`](https://github.com/apache/spark/commit/9c1c415e4c58b334bc1339754fae75f2e991dfa2).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20608: [SPARK-23422][Core] YarnShuffleIntegrationSuite f...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/20608
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20608: [SPARK-23422][Core] YarnShuffleIntegrationSuite fix when...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20608
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87449/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20608: [SPARK-23422][Core] YarnShuffleIntegrationSuite fix when...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20608
Merging to master / 2.3.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org