[jira] [Commented] (TEZ-828) TaskScheduler to get all node info from YARN when starting up

["Siddharth Seth (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13899453#comment-13899453 ] 

Siddharth Seth commented on TEZ-828:
------------------------------------

I think this will end up breaking Tez on secure clusters; the AM does not have credentials to talk to the RM on the ClientRMProtocol afaik.

I can see how getting node reports can be helpful up front - especially if we're pre-launching containers across nodes, except for the prelaunch case does it  add an unnecessary delay to AM startup ? Assuming this will only be called on startup, and we'll rely on updates from YARN for changes to the cluster topology.

Other than that, the patch itself looks good to me.

> TaskScheduler to get all node info from YARN when starting up
> -------------------------------------------------------------
>
>                 Key: TEZ-828
>                 URL: https://issues.apache.org/jira/browse/TEZ-828
>             Project: Apache Tez
>          Issue Type: Improvement
>            Reporter: Bikas Saha
>            Assignee: Bikas Saha
>         Attachments: TEZ-828.1.patch
>
>
> Without this Tez does not know about nodes until it gets them via an allocation from the RM. This is weird + hinders features like being able to start containers on all nodes in the cluster.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)