You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Jian He (JIRA)" <ji...@apache.org> on 2015/07/30 07:14:05 UTC
[jira] [Comment Edited] (YARN-3983) Make CapacityScheduler to
easier extend application allocation logic
[ https://issues.apache.org/jira/browse/YARN-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14647150#comment-14647150 ]
Jian He edited comment on YARN-3983 at 7/30/15 5:13 AM:
--------------------------------------------------------
thanks Wangda ! some comments on the patch
- ApplicationResourceAllocator -> ContainerAllocator
- NewContainerAllocator -> RegularContainerAllocator
- internalPreAllocation - > preAllocate
- move the assingContainersOnNode into internalApplyAllocation
- internalApplyAllocation -> doAllocation
- doAllocation -> allocate
- AllocatorAllocationResult -> ContainerAllocation
- SKIPPED_APP -> APP_SKIPPED; similarly for others
- this.resourceToBeAllocated can be set null; the caller can check whether null or not
{code}
if (resourceToBeAllocated == null) {
this.resourceToBeAllocated = Resources.none();
} else {
this.resourceToBeAllocated = resourceToBeAllocated;
}
{code}
- AllocatorAllocationResult#allocateNodeType -> AllocatorAllocationResult#containerNodeType
- Fix FiCaSchedulerApp#assignContainer method format and remove the unused createdContainer parameter
- handleNewContainerReservation does not need be a separate method;
- getCSAssignmentFromAllocateResult can be part of doAllocation.
was (Author: jianhe):
thanks Wangda ! some comments on the patch
- ApplicationResourceAllocator -> ContainerAllocator
- NewContainerAllocator -> RegularContainerAllocator
- internalPreAllocation - > preAllocate
- move the assingContainersOnNode into internalApplyAllocation
- internalApplyAllocation -> doAllocation
- doAllocation -> allocate
- AllocatorAllocationResult -> ContainerAllocation
- SKIPPED_APP -> SKIP_APP; similarly for others
- this.resourceToBeAllocated can be set null; the caller can check whether null or not
{code}
if (resourceToBeAllocated == null) {
this.resourceToBeAllocated = Resources.none();
} else {
this.resourceToBeAllocated = resourceToBeAllocated;
}
{code}
- AllocatorAllocationResult#allocateNodeType -> AllocatorAllocationResult#containerNodeType
- Fix FiCaSchedulerApp#assignContainer method format and remove the unused createdContainer parameter
- handleNewContainerReservation does not need be a separate method;
- getCSAssignmentFromAllocateResult can be part of doAllocation.
> Make CapacityScheduler to easier extend application allocation logic
> --------------------------------------------------------------------
>
> Key: YARN-3983
> URL: https://issues.apache.org/jira/browse/YARN-3983
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Wangda Tan
> Assignee: Wangda Tan
> Attachments: YARN-3983.1.patch
>
>
> While working on YARN-1651 (resource allocation for increasing container), I found it is very hard to extend existing CapacityScheduler resource allocation logic to support different types of resource allocation.
> For example, there's a lot of differences between increasing a container and allocating a container:
> - Increasing a container doesn't need to check locality delay.
> - Increasing a container doesn't need to build/modify a resource request tree (ANY->RACK/HOST).
> - Increasing a container doesn't need to check allocation/reservation starvation (see {{shouldAllocOrReserveNewContainer}}).
> - After increasing a container is approved by scheduler, it need to update an existing container token instead of creating new container.
> And there're lots of similarities when allocating different types of resources.
> - User-limit/queue-limit will be enforced for both of them.
> - Both of them needs resource reservation logic. (Maybe continuous reservation looking is needed for both of them).
> The purpose of this JIRA is to make easier extending CapacityScheduler resource allocation logic to support different types of resource allocation, make common code reusable, and also better code organization.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)