You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by ifilonenko <gi...@git.apache.org> on 2018/10/19 23:02:38 UTC
[GitHub] spark pull request #22777: [SPARK-25299][K8S][WIP] Enabling Remote Shuffle B...
GitHub user ifilonenko opened a pull request:
https://github.com/apache/spark/pull/22777
[SPARK-25299][K8S][WIP] Enabling Remote Shuffle Backup
## What changes were proposed in this pull request?
TODO
## How was this patch tested?
TODO
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ifilonenko/spark SPARK-25299-experimental
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22777.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22777
----
commit 73d3ac7d0dd876d436aa5e6723dbd75952f5a93c
Author: mcheah <mc...@...>
Date: 2018-10-02T23:21:59Z
Initial experiment for backing up shuffle files.
Instead of expecting the shuffle service to just pick up the data written to local disk, executors can upload the data explicitly to the shuffle service. Shuffle services do not need to be colocated with executors this way.
This can extend to arbitrary numbers of replicas.
commit 8281e1b7d736a72d1e3bbb9114136380bf5c4488
Author: mcheah <mc...@...>
Date: 2018-10-10T16:54:14Z
merge conflicts
commit 1aa6d656e0e38099e576f8de4550c0d8fcace4ca
Author: Ilan Filonenko <if...@...>
Date: 2018-10-19T22:44:25Z
fixing issue of client connections and refactoring for heartbeat
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22777: [SPARK-25299][K8S][WIP] Enabling Remote Shuffle B...
Posted by ifilonenko <gi...@git.apache.org>.
Github user ifilonenko closed the pull request at:
https://github.com/apache/spark/pull/22777
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22777: [SPARK-25299][K8S][WIP] Enabling Remote Shuffle Backup
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22777
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22777: [SPARK-25299][K8S][WIP] Enabling Remote Shuffle Backup
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22777
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97626/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org