You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by seayi <gi...@git.apache.org> on 2016/04/20 11:03:04 UTC
[GitHub] spark pull request: Update DAGScheduler.scala
GitHub user seayi opened a pull request:
https://github.com/apache/spark/pull/12524
Update DAGScheduler.scala
## What changes were proposed in this pull request?
when task finished from failed stage ,it shouldn't. remove from pending rdd partition list.
## How was this patch tested?
manual tests
when executor lost DagScheduer may submit one stage twice even if the first running taskset for this stage is not finished,because the finished's task from the failed stage should not remove from the pending partition list.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/seayi/spark branch-1.6
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/12524.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12524
----
commit 85b0725134edaaa178dd125babd0724272935b20
Author: seayi <40...@qq.com>
Date: 2016-04-20T09:00:34Z
Update DAGScheduler.scala
when executor lost DagScheduer may submit one stage twice even if the first running taskset for this stage is not finished,because the finished's task from the failed stage should not remove from the pending partition list.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Update DAGScheduler.scala
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12524#issuecomment-212339993
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #12524: [SPARK-12524][Core]DagScheduler may submit a task set fo...
Posted by mridulm <gi...@git.apache.org>.
Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/12524
@seayi any progress on this ? Would be good to add this in if consistently reproducible.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Update DAGScheduler.scala
Posted by hvanhovell <gi...@git.apache.org>.
Github user hvanhovell commented on the pull request:
https://github.com/apache/spark/pull/12524#issuecomment-212347291
@seayi Could you provide a proper title to the PR, this should contain the JIRA ticket, the Spark component and descriptive title. Something like this: `[SPARK-14658][Core] Resubmit tasks on failure`
Manual test are typically not sufficient. Please think of a way of capturing this in a test.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #12524: [SPARK-12524][Core]DagScheduler may submit a task set fo...
Posted by kayousterhout <gi...@git.apache.org>.
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/12524
Also, the approach in this PR was discussed and rejected in #16620 (see https://github.com/apache/spark/pull/16620#issuecomment-279125227 for a description of why; the approach here will also fail the DAGSchedulerSuite unit tests).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #12524: [SPARK-12524][Core]DagScheduler may submit a task set fo...
Posted by mridulm <gi...@git.apache.org>.
Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/12524
@JoshRosen This is interesting : thanks for the details !
On the face of it, I think @markhamstra's comment about #16620 should apply - but given the additional details, it might possible to reproduce it consistently ?
I am hoping we can create a repeatable test to trigger this : which should greatly speed up the debugging. The earlier case was not reproducible when I tried, but we have more info now ...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Update DAGScheduler.scala
Posted by seayi <gi...@git.apache.org>.
Github user seayi commented on the pull request:
https://github.com/apache/spark/pull/12524#issuecomment-212340756
https://issues.apache.org/jira/browse/SPARK-14658
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12524][Core]DagScheduler may submit a t...
Posted by suyanNone <gi...@git.apache.org>.
Github user suyanNone commented on the pull request:
https://github.com/apache/spark/pull/12524#issuecomment-213351412
can you refer this, and have a look, https://github.com/apache/spark/pull/8927
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #12524: [SPARK-12524][Core]DagScheduler may submit a task set fo...
Posted by kayousterhout <gi...@git.apache.org>.
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/12524
@seayi -- can you close this PR, since it's a duplicate of #16620?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12524][Core]DagScheduler may submit a t...
Posted by seayi <gi...@git.apache.org>.
Github user seayi commented on the pull request:
https://github.com/apache/spark/pull/12524#issuecomment-213324738
@hvanhovell thanks for your attention, ok i will try to write the test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Update DAGScheduler.scala
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/12524#issuecomment-212766480
Yes I think this PR should be closed, and discussion merged to https://github.com/apache/spark/pull/12436
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #12524: [SPARK-12524][Core]DagScheduler may submit a task set fo...
Posted by kayousterhout <gi...@git.apache.org>.
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/12524
I just closed the PR as a duplicate and agree with @markhamstra that this duplicates #16620 (let's move discussion about whether this is a duplicate to the JIRA so it's recorded)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #12524: [SPARK-12524][Core]DagScheduler may submit a task set fo...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/12524
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Update DAGScheduler.scala
Posted by jodersky <gi...@git.apache.org>.
Github user jodersky commented on the pull request:
https://github.com/apache/spark/pull/12524#issuecomment-212754822
is this related to #12436 ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #12524: [SPARK-12524][Core]DagScheduler may submit a task set fo...
Posted by kayousterhout <gi...@git.apache.org>.
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/12524
Can you update the PR description here to have the JIRA number (SPARK-14658), not the PR number?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #12524: [SPARK-12524][Core]DagScheduler may submit a task...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/12524
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Update DAGScheduler.scala
Posted by seayi <gi...@git.apache.org>.
Github user seayi commented on the pull request:
https://github.com/apache/spark/pull/12524#issuecomment-213319167
thanks for your attention ,but i think it is not the same,the 14649 jira is running duplicate tasks, but this jira is submit a task set for a stage even if there are another active tast set for this stage is active which can cause the spark context exit.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #12524: [SPARK-12524][Core]DagScheduler may submit a task set fo...
Posted by markhamstra <gi...@git.apache.org>.
Github user markhamstra commented on the issue:
https://github.com/apache/spark/pull/12524
@JoshRosen I haven't tried to walk through the logs in your JIRA comment, but it wouldn't surprise me at all if this is the same issue that we've been working through in https://github.com/apache/spark/pull/16620
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #12524: [SPARK-12524][Core]DagScheduler may submit a task set fo...
Posted by JoshRosen <gi...@git.apache.org>.
Github user JoshRosen commented on the issue:
https://github.com/apache/spark/pull/12524
Per [my comment on the JIRA](https://issues.apache.org/jira/browse/SPARK-14658?focusedCommentId=15870777&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15870777), I believe that this is _not_ a duplicate of #12436 as was originally suggested, so I'd propose that we revive discussion and review of this.
@mridulm, I have logs from a reproduction which occurred on a Spark 2.1.0 production cluster, which I posted on the JIRA (https://issues.apache.org/jira/browse/SPARK-14658). I'm still not entirely sure what's happening here, but one clue comes from the fact that it's the _third_ submission of the task set which is failing. My hunch is that there's an invariant regarding overlapping original attempts and re-attempts which is violated when a re-attempt itself fails and is re-attempted again.
/cc @kayousterhout and @markhamstra for review of this scheduler-related patch.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org