You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Tao Yang (JIRA)" <ji...@apache.org> on 2017/12/08 11:44:01 UTC
[jira] [Comment Edited] (YARN-7494) Add muti node lookup support
for better placement
[ https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16283419#comment-16283419 ]
Tao Yang edited comment on YARN-7494 at 12/8/17 11:43 AM:
----------------------------------------------------------
Thanks for the patch. [~sunilg]
Some thoughts from my side:
* Agree with 2) from [~leftnoteasy]. Sorting requirement may be different in some scenarios. For example, opportunistic containers would prefer considering node utilization to unallocated resource. I think we should support expandable sorting library and AppPlacementAllocator can choose from it.
* This patch iterates all partition nodes to create new PartitionBasedCandidateNodeSet for every schedule process in CapacityScheduler#getCandidateNodeSet. I think we can keep a single instance to avoid always creating the same set.
* This patch remain as it is to iterates all nodes and trigger the schedule process for every node in CapacityScheduler#schedule. Is it better to move multiNodePlacementEnabled condition branch from CapacityScheduler#getCandidateNodeSet to CapacityScheduler#schedule and iterates all partitions to trigger the schedule process when multi-node-placement enabled ?
was (Author: tao yang):
Thanks for the patch. [~sunilg]
Some thoughts from my side:
* Agree with 2) from [~leftnoteasy]. Sorting requirement may be different in some scenarios. For example, opportunistic containers would prefer considering node utilization to unallocated resource. I think we should support expandable sorting library and AppPlacementAllocator can choose from it.
* This patch iterates all partition nodes to create new PartitionBasedCandidateNodeSet for every schedule process in CapacityScheduler#getCandidateNodeSet. I think we can keep a single instance to avoid always creating the same set.
* This patch remain as it is to iterates all nodes and trigger the schedule process for every node in CapacityScheduler#schedule. IIUC Is it better to move multiNodePlacementEnabled condition branch from CapacityScheduler#getCandidateNodeSet to CapacityScheduler#schedule and iterates all partitions to trigger the schedule process when multi-node-placement enabled ?
> Add muti node lookup support for better placement
> -------------------------------------------------
>
> Key: YARN-7494
> URL: https://issues.apache.org/jira/browse/YARN-7494
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
> Reporter: Sunil G
> Assignee: Sunil G
> Attachments: YARN-7494.v0.patch
>
>
> Instead of single node, for effectiveness we can consider a multi node lookup based on partition to start with.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org