You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Arun C Murthy (JIRA)" <ji...@apache.org> on 2011/08/28 10:05:37 UTC
[jira] [Created] (MAPREDUCE-2892) Improvements to AM apis
Improvements to AM apis
-----------------------
Key: MAPREDUCE-2892
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2892
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am, mrv2, resourcemanager
Affects Versions: 0.23.0
Reporter: Arun C Murthy
Priority: Blocker
Fix For: 0.23.0
Some api changes before we declare yarn apis as stable:
# FinishApplicationMasterRequest doesn't need to send out 'tracking url'.
# FinishApplicationMasterRequest shouldn't use 'string' as final state - it's got to be an enum and we need to use that enum appropriately in the RM's state-machines.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2892) Improvements to AM apis
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13112733#comment-13112733 ]
Arun C Murthy commented on MAPREDUCE-2892:
------------------------------------------
Anupam - thanks for taking this up.
To be clear - we just need to add a new enum for the final state of the application - FAILED/SUCCESS/KILLED.
Vinod - SUCCESS/FAILED/KILLED seems like a reasonable set of final states for all applications, no?
> Improvements to AM apis
> -----------------------
>
> Key: MAPREDUCE-2892
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2892
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2, resourcemanager
> Affects Versions: 0.23.0
> Reporter: Arun C Murthy
> Assignee: Anupam Seth
> Priority: Blocker
> Fix For: 0.23.0
>
>
> Some api changes before we declare yarn apis as stable:
> # FinishApplicationMasterRequest doesn't need to send out 'tracking url'.
> # FinishApplicationMasterRequest shouldn't use 'string' as final state - it's got to be an enum and we need to use that enum appropriately in the RM's state-machines.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2892) Improvements to AM apis
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13108042#comment-13108042 ]
Mahadev konar commented on MAPREDUCE-2892:
------------------------------------------
@Vinod,
I think the final state should not be a blob. There are only a few final states that an application can get to - FAILED/SUCCESS/KILLED. Strings/binary blob would be confusing.
> Improvements to AM apis
> -----------------------
>
> Key: MAPREDUCE-2892
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2892
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2, resourcemanager
> Affects Versions: 0.23.0
> Reporter: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.0
>
>
> Some api changes before we declare yarn apis as stable:
> # FinishApplicationMasterRequest doesn't need to send out 'tracking url'.
> # FinishApplicationMasterRequest shouldn't use 'string' as final state - it's got to be an enum and we need to use that enum appropriately in the RM's state-machines.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2892) Improvements to AM apis
Posted by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13108381#comment-13108381 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-2892:
----------------------------------------------------
We differentiate ApplicationState from the final state of the framework's AM. For e.g., Job Kill is handled by MR AM and from the platform point of view, the Application has simply finished, not killed. In this case, we will have Application.State as FINISHED, with the auxiliary information of Framework.State to be KILLED. Each framework can have arbitrary final state, like say ABORTED.
Today, we call Framework.State as finalState, may be we should name it better.
I saw that in MAPREDUCE-3017, when an application finishes, you used the Application.State column for printing the Framework.State when the application finishes. IMO, we need a separate column for Framework.State.
> Improvements to AM apis
> -----------------------
>
> Key: MAPREDUCE-2892
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2892
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2, resourcemanager
> Affects Versions: 0.23.0
> Reporter: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.0
>
>
> Some api changes before we declare yarn apis as stable:
> # FinishApplicationMasterRequest doesn't need to send out 'tracking url'.
> # FinishApplicationMasterRequest shouldn't use 'string' as final state - it's got to be an enum and we need to use that enum appropriately in the RM's state-machines.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-2892) Improvements to AM apis
Posted by "Hitesh Shah (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hitesh Shah reassigned MAPREDUCE-2892:
--------------------------------------
Assignee: Hitesh Shah (was: Anupam Seth)
> Improvements to AM apis
> -----------------------
>
> Key: MAPREDUCE-2892
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2892
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2, resourcemanager
> Affects Versions: 0.23.0
> Reporter: Arun C Murthy
> Assignee: Hitesh Shah
> Priority: Blocker
> Fix For: 0.23.0
>
>
> Some api changes before we declare yarn apis as stable:
> # FinishApplicationMasterRequest doesn't need to send out 'tracking url'.
> # FinishApplicationMasterRequest shouldn't use 'string' as final state - it's got to be an enum and we need to use that enum appropriately in the RM's state-machines.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2892) Improvements to AM apis
Posted by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13093642#comment-13093642 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-2892:
----------------------------------------------------
bq. FinishApplicationMasterRequest doesn't need to send out 'tracking url'.
The tracking url for an App can change when it finishes. For e.g., in mapreduce, the tracking-url for a job is the AM web UI while the AM is up, and will change to the history server URL on AM finish.
bq. FinishApplicationMasterRequest shouldn't use 'string' as final state - it's got to be an enum and we need to use that enum appropriately in the RM's state-machines.
This final state is framework specific (mapreduce specific for e.g.) and has no bearing on the final state of the ApplicationMaster in the RM. This state is *only used* to be displayed on the UI as the final state of the job specific to the framework and for nothing else. Sharad an I discussed this and concluded that this can best be a binary blob or a string, an ENUM isn't possible and doesn't make sense for enumerating all possible final states of all possible frameworks.
We can just document both these fields with the above descriptions.
> Improvements to AM apis
> -----------------------
>
> Key: MAPREDUCE-2892
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2892
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2, resourcemanager
> Affects Versions: 0.23.0
> Reporter: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.0
>
>
> Some api changes before we declare yarn apis as stable:
> # FinishApplicationMasterRequest doesn't need to send out 'tracking url'.
> # FinishApplicationMasterRequest shouldn't use 'string' as final state - it's got to be an enum and we need to use that enum appropriately in the RM's state-machines.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2892) Improvements to AM apis
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-2892:
-------------------------------------
Issue Type: Sub-task (was: Bug)
Parent: MAPREDUCE-2894
> Improvements to AM apis
> -----------------------
>
> Key: MAPREDUCE-2892
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2892
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2, resourcemanager
> Affects Versions: 0.23.0
> Reporter: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.0
>
>
> Some api changes before we declare yarn apis as stable:
> # FinishApplicationMasterRequest doesn't need to send out 'tracking url'.
> # FinishApplicationMasterRequest shouldn't use 'string' as final state - it's got to be an enum and we need to use that enum appropriately in the RM's state-machines.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-2892) Improvements to AM apis
Posted by "Vinod Kumar Vavilapalli (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli resolved MAPREDUCE-2892.
------------------------------------------------
Resolution: Duplicate
Being fixed as part of MAPREDUCE-3098.
> Improvements to AM apis
> -----------------------
>
> Key: MAPREDUCE-2892
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2892
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2, resourcemanager
> Affects Versions: 0.23.0
> Reporter: Arun C Murthy
> Assignee: Hitesh Shah
> Priority: Blocker
> Fix For: 0.23.0
>
>
> Some api changes before we declare yarn apis as stable:
> # FinishApplicationMasterRequest doesn't need to send out 'tracking url'.
> # FinishApplicationMasterRequest shouldn't use 'string' as final state - it's got to be an enum and we need to use that enum appropriately in the RM's state-machines.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2892) Improvements to AM apis
Posted by "Anupam Seth (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115501#comment-13115501 ]
Anupam Seth commented on MAPREDUCE-2892:
----------------------------------------
Yes - I have not started . No problems if Hitesh would like to club it with MR-3098.
> Improvements to AM apis
> -----------------------
>
> Key: MAPREDUCE-2892
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2892
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2, resourcemanager
> Affects Versions: 0.23.0
> Reporter: Arun C Murthy
> Assignee: Anupam Seth
> Priority: Blocker
> Fix For: 0.23.0
>
>
> Some api changes before we declare yarn apis as stable:
> # FinishApplicationMasterRequest doesn't need to send out 'tracking url'.
> # FinishApplicationMasterRequest shouldn't use 'string' as final state - it's got to be an enum and we need to use that enum appropriately in the RM's state-machines.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2892) Improvements to AM apis
Posted by "Arun C Murthy (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13115180#comment-13115180 ]
Arun C Murthy commented on MAPREDUCE-2892:
------------------------------------------
Anupam - if you havne't started on this, can I ask Hitesh to fix this as a part of MAPREDUCE-3098? Thanks.
> Improvements to AM apis
> -----------------------
>
> Key: MAPREDUCE-2892
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2892
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2, resourcemanager
> Affects Versions: 0.23.0
> Reporter: Arun C Murthy
> Assignee: Anupam Seth
> Priority: Blocker
> Fix For: 0.23.0
>
>
> Some api changes before we declare yarn apis as stable:
> # FinishApplicationMasterRequest doesn't need to send out 'tracking url'.
> # FinishApplicationMasterRequest shouldn't use 'string' as final state - it's got to be an enum and we need to use that enum appropriately in the RM's state-machines.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-2892) Improvements to AM apis
Posted by "Anupam Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anupam Seth reassigned MAPREDUCE-2892:
--------------------------------------
Assignee: Anupam Seth
> Improvements to AM apis
> -----------------------
>
> Key: MAPREDUCE-2892
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2892
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mr-am, mrv2, resourcemanager
> Affects Versions: 0.23.0
> Reporter: Arun C Murthy
> Assignee: Anupam Seth
> Priority: Blocker
> Fix For: 0.23.0
>
>
> Some api changes before we declare yarn apis as stable:
> # FinishApplicationMasterRequest doesn't need to send out 'tracking url'.
> # FinishApplicationMasterRequest shouldn't use 'string' as final state - it's got to be an enum and we need to use that enum appropriately in the RM's state-machines.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira