[jira] [Comment Edited] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

["Arun Suresh (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15311530#comment-15311530 ] 

Arun Suresh edited comment on YARN-5171 at 6/2/16 1:10 AM:
-----------------------------------------------------------

Thanks for the patch [~elgoiri]..

The general approach looks good, We would also need to ensure the following though :
# Since the container is created outside the RM, the {{attemptResourceUsage}} field in the {{SchedulerApplicationAttempt}} is not incremented (This is done by the specific schedulers and is found in the specific implementation of the SchedulerApplicationAttempt). I guess we should have a separate {{attemptResourceUsageOpportunistic}} field introduced in the SchedulerApplicationAttempt class, that is incremented when the DistributedSchedulingService creates the RMConatiner.
# Similarly, we should modify the {{completeContainer()}} method in the {{AbstractYarnScheduler}} to also decrement the {{attemptResourceUsageOpportunistic}} of the corresponding app attempt if the completed container was opportunistic (we can get that from the {{ContainerStatus}}). We don't have to worry about NOT decrementing {{attemptResourceUsage}}, since that is done in the specific Scheduler, and we are modifying the {{AbstractYarnScheduler::completedContainer()}}.


was (Author: asuresh):
Thanks for the patch [~elgoiri]..

The general approach looks good, We would also need to ensure the following though :
# Since the container is created outside the RM, the {{attemptResourceUsage}} field in the {{SchedulerApplicationAttempt}} is not incremented (This is done by the specific schedulers and is found in the specific implementation of the SchedulerApplicationAttempt). I guess we should have a separate {{attemptResourceUsageOpportunistic}} field introduced in the SchedulerApplicationAttempt class, that is incremented when the DistributedSchedulingService creates the RMConatiner.
# Similarly, we should modify the {{completeContainer()}} method in the {{AbstractYarnScheduler}} to also decrement the {{attemptResourceUsageOpportunistic}} of the corresponding app attempt if the completed container was opportunistic (we can get that from the {{ContainerStatus}})

> Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node
> ------------------------------------------------------------------------------------
>
>                 Key: YARN-5171
>                 URL: https://issues.apache.org/jira/browse/YARN-5171
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Arun Suresh
>            Assignee: Inigo Goiri
>         Attachments: YARN-5171.000.patch, YARN-5171.001.patch
>
>
> Currently, the RM does not know about Containers allocated by the OpportunisticContainerAllocator on the NM. This JIRA proposes to extend the Distributed Scheduler request interceptor and the protocol to notify the RM of new containers as and when they are allocated at the NM. The {{RMContainer}} should also be extended to expose the {{ExecutionType}} of the container.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org