[jira] [Commented] (SPARK-19565) After fetching failed, success of old attempt of stage should be taken as valid.

["Kay Ousterhout (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-19565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15930910#comment-15930910 ] 

Kay Ousterhout commented on SPARK-19565:
----------------------------------------

[~jinxing6042@126.com] I closed this because it looks like a duplicate with the word you did for SPARK-19263.  Feel free to re-open if I've misunderstood.

> After fetching failed, success of old attempt of stage should be taken as valid.
> --------------------------------------------------------------------------------
>
>                 Key: SPARK-19565
>                 URL: https://issues.apache.org/jira/browse/SPARK-19565
>             Project: Spark
>          Issue Type: Test
>          Components: Scheduler
>    Affects Versions: 2.1.0
>            Reporter: jin xing
>
> This is related to SPARK-19263. 
> When fetch failed, stage will be resubmitted. There can be running tasks from both old and new stage attempts. Success of tasks from old stage attempt should be taken as valid and partitionId should be removed from stage's pendingPartitions accordingly. When pending partitions is empty, downstream stage can be scheduled, even though there's still running tasks in the active(new) stage attempt.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org