You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by liutang123 <gi...@git.apache.org> on 2017/11/24 08:47:11 UTC
[GitHub] spark pull request #19812: [SPARK-22598][CORE] ExecutorAllocationManager doe...
GitHub user liutang123 opened a pull request:
https://github.com/apache/spark/pull/19812
[SPARK-22598][CORE] ExecutorAllocationManager does not requests new executors when executor has failed and target has not changed
## What changes were proposed in this pull request?
Check if the number of current executors when target has not changed.
## How was this patch tested?
manual test.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/liutang123/spark SPARK-22598
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/19812.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #19812
----
commit 6db1b8687060de9a67c0225d42e32afcfb60faf8
Author: liutang123 <li...@yeah.net>
Date: 2017-11-24T08:40:40Z
[SPARK-22598][CORE]ExecutorAllocationManager does not requests new executors when executor has failed and target has not changed.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19812: [SPARK-22598][CORE] ExecutorAllocationManager does not r...
Posted by jerryshao <gi...@git.apache.org>.
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/19812
Hi @liutang123 would you mind explaining us the issue you met and how to reproduce it? Currently we don't know what actual issue it is and how to evaluate your changes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19812: [SPARK-22598][CORE] ExecutorAllocationManager does not r...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19812
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19812: [SPARK-22598][CORE] ExecutorAllocationManager does not r...
Posted by liutang123 <gi...@git.apache.org>.
Github user liutang123 commented on the issue:
https://github.com/apache/spark/pull/19812
Hi @jerryshao , I modified the info of this PR.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19812: [SPARK-22598][CORE] ExecutorAllocationManager does not r...
Posted by liutang123 <gi...@git.apache.org>.
Github user liutang123 commented on the issue:
https://github.com/apache/spark/pull/19812
Jenkins, retest this please
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19812: [SPARK-22598][CORE] ExecutorAllocationManager does not r...
Posted by jerryshao <gi...@git.apache.org>.
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/19812
Does this failure ". For some reason, all of the 3 executors failed. " happened during task running or before task submission? Besides, if you're running on yarn, yarn will bring new executors to meet the requirement, also `ExecutorAllocationManager` will be notified with executor lost/register. Can you please tell us how to reproduce your scenario?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19812: [SPARK-22598][CORE] ExecutorAllocationManager does not r...
Posted by liutang123 <gi...@git.apache.org>.
Github user liutang123 commented on the issue:
https://github.com/apache/spark/pull/19812
Sorry, I can not reproduce it now. But, sometimes, `ExecutorAllocationManager ` did not request new executors and `YarnSchedulerBackend.requestedTotalExecutors` is 0. I will close this PR now and reopen it next time when I encounter this situation.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19812: [SPARK-22598][CORE] ExecutorAllocationManager doe...
Posted by liutang123 <gi...@git.apache.org>.
Github user liutang123 closed the pull request at:
https://github.com/apache/spark/pull/19812
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19812: [SPARK-22598][CORE] ExecutorAllocationManager does not r...
Posted by liutang123 <gi...@git.apache.org>.
Github user liutang123 commented on the issue:
https://github.com/apache/spark/pull/19812
@srowen Would you please look at this when you have time?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org