[jira] [Commented] (SLIDER-955) fail to track the outstandingRequest when submit an application that yarn.memory is not a multiple of minimum-allocation-mb

["Steve Loughran (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SLIDER-955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14980713#comment-14980713 ] 

Steve Loughran commented on SLIDER-955:
---------------------------------------

well done for finding this problem!

I'll have to look @ the slider code here, as we really try to track outstanding requests by container priority; I guess the issue is that when we try to cancel a request, the requirements of the request to cancel don't match the (multiplied) view.

I think we go for fix #2 here, multiply up. With fix #1 things will fail depending on the cluster size, for no obvious reason. And as you node -yarn is multiplying up automatically anyway.

What slider would do just premultiply the value from resources.json when it loads in the resources

> fail to track the outstandingRequest when submit an application that yarn.memory is not a multiple of minimum-allocation-mb
> ---------------------------------------------------------------------------------------------------------------------------
>
>                 Key: SLIDER-955
>                 URL: https://issues.apache.org/jira/browse/SLIDER-955
>             Project: Slider
>          Issue Type: Improvement
>          Components: appmaster
>    Affects Versions: Slider 0.80
>            Reporter: kyungwan nam
>
> "yarn.scheduler.minimum-allocation-mb” is set to 256 in my cluster.
> I submit an application that “yarn.memory” is set to 7169 (not a multiple of 256)
> the resource of allocated container will be 7424(256*29) memory because the resource requirement is normalized to a multiple of "yarn.scheduler.minimum-allocation-mb” by RM.
> the resource requirement of outstandingRequest that is tracked by Slider AM does not match the resource of the allocated container.
> therefore, Slider AM can’t clean up the outstandingRequest and requests in AMRMClient.
> I think it should fix as following 1 or 2
> 1. an application should be able to submit only if “yarn.memory” is a multiple of "yarn.scheduler.minimum-allocation-mb”
> 2. the resource requirement of outstandingRequest is normalized to a multiple of "yarn.scheduler.minimum-allocation-mb”



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)