[jira] [Commented] (YARN-7494) Add muti node lookup support for better placement

["Wangda Tan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16280902#comment-16280902 ] 

Wangda Tan commented on YARN-7494:
----------------------------------

Thanks [~sunilg] for working on this. 

Looked at the implementation, some thoughts/suggestions: 

1) This patch added a flag to ASC, I think we have many other requirements like this, for example, applications may want to choose its own delay scheduling parameters, etc. Instead of adding this to protocol, how about reading such fields in AM launch context's environment (which is same as how we handle docker container image, etc.). Part of the reason is, we don't need to change applications to consume this feature. Most of the apps should be able to specify customized envars for AM. (Any other ideas here?)

2) This patch added a getPreferredNodeIterator method to CandidateNodeSet, however, I think we should keep CandidateNodeSet returns all available nodes (like all nodes under the partition or whole cluster). Different AppPlacementAllocator should implement its own sorting mechanism to allocate on nodes.

3) To help other folks reviewing the implementation, I suggest adding a sample multi-node placement allocator. Such as trying to make the allocation as packed as possible (which essentially trying to allocate on nodes with less utilization).

+ [~asuresh] / [~kkaranasos].

> Add muti node lookup support for better placement
> -------------------------------------------------
>
>                 Key: YARN-7494
>                 URL: https://issues.apache.org/jira/browse/YARN-7494
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacity scheduler
>            Reporter: Sunil G
>            Assignee: Sunil G
>         Attachments: YARN-7494.v0.patch
>
>
> Instead of single node, for effectiveness we can consider a multi node lookup based on partition to start with.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org