[jira] [Commented] (YARN-10168) FS-CS Convert: Converter tool doesn't handle min/max resource conversion correct

["Peter Bacsko (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-10168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17046406#comment-17046406 ] 

Peter Bacsko commented on YARN-10168:
-------------------------------------

BTW is this really a blocker for the converter?

> FS-CS Convert: Converter tool doesn't handle min/max resource conversion correct
> --------------------------------------------------------------------------------
>
>                 Key: YARN-10168
>                 URL: https://issues.apache.org/jira/browse/YARN-10168
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Wangda Tan
>            Priority: Blocker
>
> Trying to understand logics of convert min and max resource from FS to CS, and found some issues:
> 1)
> In FSQueueConverter#emitMaximumCapacity
> Existing logic in FS is to either specify a maximum percentage for queues against cluster resources. Or, specify an absolute valued maximum resource.
> In the existing FS2CS converter, when a percentage-based maximum resource is specified, the converter takes a global resource from fs2cs CLI, and applies percentages to that. It is not correct since the percentage-based value will get lost, and in the future when cluster resources go up and down, the maximum resource cannot be changed.
> 2)
> The logic to deal with min/weight resource is also questionable:
> The existing fs2cs tool, it takes precedence of percentage over absoluteResource, and could set both to a queue config. See FSQueueConverter.Capacity#toString
> However, in CS, comparing to FS, the weights/min resource is quite different:
> CS use the same queue.capacity to specify both percentage-based or absolute-resource-based configs (Similar to how FS deal with maximum Resource).
>  The capacity defines guaranteed resource, which also impact fairshare of the queue. (The more guaranteed resource a queue has, the larger "pie" the queue can get if there's any additional available resource).
>  In FS, minResource defined the guaranteed resource, and weight defined how much the pie can grow to.
> So to me, in FS, we should pick and choose either weight or minResource to generate CS.
> 3)
> In FS, mix-use of absolute-resource configs (like min/maxResource), and percentage-based (like weight) is allowed. But in CS, it is not allowed. The reason is discussed on YARN-5881, and find [a]Should we support specifying a mix of percentage ...
> The existing fs2cs doesn't handle the issue, which could set mixed absolute resource and percentage-based resources.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org