You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@yunikorn.apache.org by "Peter Bacsko (Jira)" <ji...@apache.org> on 2023/03/28 14:44:00 UTC
[jira] [Updated] (YUNIKORN-1665) Group tracked resources/counters to a separate type from Queue
[ https://issues.apache.org/jira/browse/YUNIKORN-1665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Peter Bacsko updated YUNIKORN-1665:
-----------------------------------
Description:
In core, we track a bunch of things inside a queue. Sometimes it's tracked automatically with the parent, sometimes not:
||Tracked item||Type||Updated in the parent||
|Applications|map[string]*Application|no|
|Allocation|*resource.Resource|yes|
|Reserved Apps|map[string]int|no|
|Pending resource|*resource.Resource|yes|
|Preempted resource|*resource.Resource|yes|
|Reserved apps|map[string]int|no|
|Running apps|uint64|yes|
|AllocatingAcceptedApps|map[string]bool|yes|
When we move an application to one leaf to another, all of these data structures need to be updated.
For example, if an application is moved from "root.users.qa.alice" to "root.dev", then its "pending" resources tracked in "root.users.qa.alice" must be subtracted from the two parents + leaf except root and it need to be added to "root.dev".
Whenever we add something new to track, it's easy to forget that it affects the app moving logic. So we can do two things:
1) We re-organize the current source code a bit better to make it clear that code in "queue_moveapp.go" needs to be updated if we add something new to track.
2) We extract everything to a separate Tracker type and this becomes even more obvious. The Tracker can be similar to {{ugm.QueueTracker}}, where the trackers are built in a hierarchy which accurately reflects the queue hierarchy, ie. all trackers have child trackers.
was:
In core, we track a bunch of things inside a queue. Sometimes it's tracked automatically with the parent, sometimes not:
||Tracked item||Type||Updated in the parent||
|Applications|map[string]*Application|no|
|Allocation|*resource.Resource|yes|
|Reserved Apps|map[string]int|no|
|Pending resource|*resource.Resource|yes|
|Preempted resource|*resource.Resource|yes|
|Reserved apps|map[string]int|no|
|Running apps|uint64|yes|
|AllocatingAcceptedApps|map[string]bool|yes|
When we move an application to one leaf to another, all of these data structures need to be updated.
For example, if an application is moved from "root.users.qa.alice" to "root.dev", then its "pending" resources tracked in "root.users.qa.alice" must be subtracted from the two parents + leaf except root and it need to be added to "root.dev".
Whenever we add something new to track, it's easy to forget that it affects the app moving logic. So we can do two things:
1) We re-organize the current source code a bit better to make it clear that code in "queue_moveapp.go" needs to be updated if we add something new to track.
2) We extract everything to a separate Tracker type and this becomes even more obvious. The Tracker can be similar to the {{QueueTracker}}, where the trackers are built in a hierarchy which accurately reflects the queue hierarchy, ie. all trackers have child trackers.
> Group tracked resources/counters to a separate type from Queue
> --------------------------------------------------------------
>
> Key: YUNIKORN-1665
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1665
> Project: Apache YuniKorn
> Issue Type: Sub-task
> Components: core - scheduler
> Reporter: Peter Bacsko
> Priority: Major
>
> In core, we track a bunch of things inside a queue. Sometimes it's tracked automatically with the parent, sometimes not:
>
> ||Tracked item||Type||Updated in the parent||
> |Applications|map[string]*Application|no|
> |Allocation|*resource.Resource|yes|
> |Reserved Apps|map[string]int|no|
> |Pending resource|*resource.Resource|yes|
> |Preempted resource|*resource.Resource|yes|
> |Reserved apps|map[string]int|no|
> |Running apps|uint64|yes|
> |AllocatingAcceptedApps|map[string]bool|yes|
> When we move an application to one leaf to another, all of these data structures need to be updated.
> For example, if an application is moved from "root.users.qa.alice" to "root.dev", then its "pending" resources tracked in "root.users.qa.alice" must be subtracted from the two parents + leaf except root and it need to be added to "root.dev".
> Whenever we add something new to track, it's easy to forget that it affects the app moving logic. So we can do two things:
> 1) We re-organize the current source code a bit better to make it clear that code in "queue_moveapp.go" needs to be updated if we add something new to track.
> 2) We extract everything to a separate Tracker type and this becomes even more obvious. The Tracker can be similar to {{ugm.QueueTracker}}, where the trackers are built in a hierarchy which accurately reflects the queue hierarchy, ie. all trackers have child trackers.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: issues-help@yunikorn.apache.org