[jira] [Commented] (TEZ-993) Remove application logic from RecoveryService

["Hitesh Shah (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294409#comment-14294409 ] 

Hitesh Shah commented on TEZ-993:
---------------------------------

I think there are a couple of issues which need to be clarified first. 

  - One of the concerns [~bikassaha] had was that the decision of whether to log recovery events for a DAG was not being done inside the DAGImpl but in the recovery layer. Tomorrow, if we support a config that says a particular DAG ( not a pre-warm dag ) should not be recovered, how would this re-factor support this?
  - Apart from that, the recovery process needed to know the internals of the DAGAppMaster such as dagCounter, etc. Currently, any relevant changes in the DAGAppMaster will also need corresponding changes to the recovery restore layer. 




> Remove application logic from RecoveryService
> ---------------------------------------------
>
>                 Key: TEZ-993
>                 URL: https://issues.apache.org/jira/browse/TEZ-993
>             Project: Apache Tez
>          Issue Type: Sub-task
>            Reporter: Bikas Saha
>            Assignee: Jeff Zhang
>         Attachments: TEZ-993-3.patch, Tez-993-2.patch, Tez-993.patch
>
>
> Currently RecoveryService storage logic knows a lot about the DAG like which dag is pre-warm and does not need to be stored, which events needs special treatment etc. This kind of logic couples the DAG and the storage more than is probably necessary and can be a source of complications down the road. The storage should ideally be simply storing a sequence of arbitrary records delimited by a marker.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)