You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "Jeff Zhang (JIRA)" <ji...@apache.org> on 2015/09/01 10:38:45 UTC
[jira] [Comment Edited] (TEZ-2758) Remove append API in
RecoveryService after TEZ-1909
[ https://issues.apache.org/jira/browse/TEZ-2758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14724983#comment-14724983 ]
Jeff Zhang edited comment on TEZ-2758 at 9/1/15 8:38 AM:
---------------------------------------------------------
Ideally append should never be used, but there's one special case. DAGFinishedEvent happens multiple times (SUCCEEDED --> ERROR). After DAGFinishedEvent is seen, output stream for recovery file is closed. So if we see the next DAGFinishedEvent, append is needed if we want to record this event.
* Do we need to record the next DAGFinsihedEvent if dag has already finished ? (If yes, it would be hard to decide when dag is really finished (when to close the outputstream), maybe consider the dag is finished if the next dag is started)
[~hitesh] Any thoughts ?
was (Author: zjffdu):
Ideally append should never be used, but there's one special case. DAGFinishedEvent happens multiple times (SUCCEEDED --> ERROR). After DAGFinishedEvent is seen, output stream for recovery file is closed. So if we see the next DAGFinishedEvent, append is needed if we want to record this event.
* Should we need to record the next DAGFinsihedEvent if dag has already finished ? (If yes, it would be hard to decide when dag is really finished (when to close the outputstream), maybe consider the dag is finished if the next dag is started)
[~hitesh] Any thoughts ?
> Remove append API in RecoveryService after TEZ-1909
> ---------------------------------------------------
>
> Key: TEZ-2758
> URL: https://issues.apache.org/jira/browse/TEZ-2758
> Project: Apache Tez
> Issue Type: Improvement
> Affects Versions: 0.6.2
> Reporter: Jeff Zhang
> Assignee: Jeff Zhang
>
> After TEZ-1909, there would be no case for append recovery file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)