You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@yunikorn.apache.org by GitBox <gi...@apache.org> on 2022/12/14 11:19:32 UTC
[GitHub] [yunikorn-core] pbacsko opened a new pull request, #479: [YUNIKORN-1466] Track dynamic preemption priority for queues
pbacsko opened a new pull request, #479:
URL: https://github.com/apache/yunikorn-core/pull/479
### What is this PR for?
We want to track the lowest priority application so that we can always return this value without unecessary calculations.
### What type of PR is it?
* [ ] - Bug Fix
* [x] - Improvement
* [ ] - Feature
* [ ] - Documentation
* [ ] - Hot Fix
* [ ] - Refactoring
### Todos
* [ ] - Task
### What is the Jira issue?
https://issues.apache.org/jira/browse/YUNIKORN-1466
### How should this be tested?
### Screenshots (if appropriate)
### Questions:
* [ ] - The licenses files need update.
* [ ] - There is breaking changes for older versions.
* [ ] - It needs documentation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [yunikorn-core] craigcondit closed pull request #479: [YUNIKORN-1466] Track dynamic preemption priority for queues
Posted by GitBox <gi...@apache.org>.
craigcondit closed pull request #479: [YUNIKORN-1466] Track dynamic preemption priority for queues
URL: https://github.com/apache/yunikorn-core/pull/479
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [yunikorn-core] pbacsko commented on a diff in pull request #479: [YUNIKORN-1466] Track dynamic preemption priority for queues
Posted by GitBox <gi...@apache.org>.
pbacsko commented on code in PR #479:
URL: https://github.com/apache/yunikorn-core/pull/479#discussion_r1061278188
##########
pkg/scheduler/objects/queue.go:
##########
@@ -628,6 +635,8 @@ func (sq *Queue) AddApplication(app *Application) {
defer sq.Unlock()
sq.applications[app.ApplicationID] = app
sq.appPriorities[app.ApplicationID] = app.GetAskMaxPriority()
+ sq.appPreemptionPriorities[app.ApplicationID] = app.GetAllocationMinPriority()
+
Review Comment:
Technically it's true, I still prefer this approach because it slightly reduces coupling. cc @craigcondit
##########
pkg/scheduler/objects/queue.go:
##########
@@ -691,11 +700,14 @@ func (sq *Queue) RemoveApplication(app *Application) {
sq.Lock()
delete(sq.applications, appID)
delete(sq.appPriorities, appID)
+ delete(sq.appPreemptionPriorities, appID)
delete(sq.allocatingAcceptedApps, appID)
priority := sq.recalculatePriority()
+ preemptionPriority := sq.recalculatePreemptionPriority()
sq.Unlock()
Review Comment:
Done
##########
pkg/scheduler/objects/queue.go:
##########
@@ -1571,3 +1593,84 @@ func (sq *Queue) recalculatePriority() int32 {
sq.currentPriority = curr
return priorityValueByPolicy(sq.priorityPolicy, sq.priorityOffset, curr)
}
+
+func (sq *Queue) UpdateApplicationPreemptionPriority(applicationID string, priority int32) {
+ if sq == nil || !sq.isLeaf {
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [yunikorn-core] pbacsko commented on a diff in pull request #479: [YUNIKORN-1466] Track dynamic preemption priority for queues
Posted by GitBox <gi...@apache.org>.
pbacsko commented on code in PR #479:
URL: https://github.com/apache/yunikorn-core/pull/479#discussion_r1061282100
##########
pkg/scheduler/objects/queue.go:
##########
@@ -1571,3 +1593,84 @@ func (sq *Queue) recalculatePriority() int32 {
sq.currentPriority = curr
return priorityValueByPolicy(sq.priorityPolicy, sq.priorityOffset, curr)
}
+
+func (sq *Queue) UpdateApplicationPreemptionPriority(applicationID string, priority int32) {
+ if sq == nil || !sq.isLeaf {
+ return
+ }
+ value := sq.updateApplicationPreemptionPriorityInternal(applicationID, priority)
+ sq.parent.UpdateQueuePreemptionPriority(sq.Name, value)
+}
+
+func (sq *Queue) UpdateQueuePreemptionPriority(queueName string, priority int32) {
+ if sq == nil || sq.isLeaf {
+ return
+ }
+ value := sq.updateQueuePreemptionPriorityInternal(queueName, priority)
+ sq.parent.UpdateQueuePreemptionPriority(sq.Name, value)
+}
+
+func (sq *Queue) updateQueuePreemptionPriorityInternal(queueName string, priority int32) int32 {
+ sq.Lock()
+ defer sq.Unlock()
+
+ if _, ok := sq.children[queueName]; !ok {
+ log.Logger().Debug("Unknown queue", zap.String("queueName", queueName))
+ return sq.preemptionPriority
+ }
+
+ sq.childPreemptionPriorities[queueName] = priority
+ return sq.recalculatePreemptionPriority()
+}
+
+func (sq *Queue) updateApplicationPreemptionPriorityInternal(applicationID string, priority int32) int32 {
+ sq.Lock()
+ defer sq.Unlock()
+
+ if _, ok := sq.applications[applicationID]; !ok {
+ log.Logger().Debug("Unknown application", zap.String("applicationID", applicationID))
+ return sq.preemptionPriority
+ }
+
+ sq.appPreemptionPriorities[applicationID] = priority
+ return sq.recalculatePreemptionPriority()
+}
+
+func (sq *Queue) recalculatePreemptionPriority() int32 {
+ var items *map[string]int32
Review Comment:
I guess the idea here is that by using a pointer it's a tad cheaper to iterate over the map because we avoid copying.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [yunikorn-core] wilfred-s commented on a diff in pull request #479: [YUNIKORN-1466] Track dynamic preemption priority for queues
Posted by GitBox <gi...@apache.org>.
wilfred-s commented on code in PR #479:
URL: https://github.com/apache/yunikorn-core/pull/479#discussion_r1062038862
##########
pkg/scheduler/objects/queue.go:
##########
@@ -1571,3 +1593,84 @@ func (sq *Queue) recalculatePriority() int32 {
sq.currentPriority = curr
return priorityValueByPolicy(sq.priorityPolicy, sq.priorityOffset, curr)
}
+
+func (sq *Queue) UpdateApplicationPreemptionPriority(applicationID string, priority int32) {
+ if sq == nil || !sq.isLeaf {
+ return
+ }
+ value := sq.updateApplicationPreemptionPriorityInternal(applicationID, priority)
+ sq.parent.UpdateQueuePreemptionPriority(sq.Name, value)
+}
+
+func (sq *Queue) UpdateQueuePreemptionPriority(queueName string, priority int32) {
+ if sq == nil || sq.isLeaf {
+ return
+ }
+ value := sq.updateQueuePreemptionPriorityInternal(queueName, priority)
+ sq.parent.UpdateQueuePreemptionPriority(sq.Name, value)
+}
+
+func (sq *Queue) updateQueuePreemptionPriorityInternal(queueName string, priority int32) int32 {
+ sq.Lock()
+ defer sq.Unlock()
+
+ if _, ok := sq.children[queueName]; !ok {
+ log.Logger().Debug("Unknown queue", zap.String("queueName", queueName))
+ return sq.preemptionPriority
+ }
+
+ sq.childPreemptionPriorities[queueName] = priority
+ return sq.recalculatePreemptionPriority()
+}
+
+func (sq *Queue) updateApplicationPreemptionPriorityInternal(applicationID string, priority int32) int32 {
+ sq.Lock()
+ defer sq.Unlock()
+
+ if _, ok := sq.applications[applicationID]; !ok {
+ log.Logger().Debug("Unknown application", zap.String("applicationID", applicationID))
+ return sq.preemptionPriority
+ }
+
+ sq.appPreemptionPriorities[applicationID] = priority
+ return sq.recalculatePreemptionPriority()
+}
+
+func (sq *Queue) recalculatePreemptionPriority() int32 {
+ var items *map[string]int32
Review Comment:
There is no copy when you do the assign, see this example code:
https://go.dev/play/p/qcJ9P-4ZCdB
Both point to one and the same data structure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [yunikorn-core] pbacsko commented on a diff in pull request #479: [YUNIKORN-1466] Track dynamic preemption priority for queues
Posted by GitBox <gi...@apache.org>.
pbacsko commented on code in PR #479:
URL: https://github.com/apache/yunikorn-core/pull/479#discussion_r1061278496
##########
pkg/scheduler/objects/queue.go:
##########
@@ -1571,3 +1593,84 @@ func (sq *Queue) recalculatePriority() int32 {
sq.currentPriority = curr
return priorityValueByPolicy(sq.priorityPolicy, sq.priorityOffset, curr)
}
+
+func (sq *Queue) UpdateApplicationPreemptionPriority(applicationID string, priority int32) {
+ if sq == nil || !sq.isLeaf {
+ return
+ }
+ value := sq.updateApplicationPreemptionPriorityInternal(applicationID, priority)
+ sq.parent.UpdateQueuePreemptionPriority(sq.Name, value)
+}
+
+func (sq *Queue) UpdateQueuePreemptionPriority(queueName string, priority int32) {
+ if sq == nil || sq.isLeaf {
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [yunikorn-core] wilfred-s commented on a diff in pull request #479: [YUNIKORN-1466] Track dynamic preemption priority for queues
Posted by GitBox <gi...@apache.org>.
wilfred-s commented on code in PR #479:
URL: https://github.com/apache/yunikorn-core/pull/479#discussion_r1062038862
##########
pkg/scheduler/objects/queue.go:
##########
@@ -1571,3 +1593,84 @@ func (sq *Queue) recalculatePriority() int32 {
sq.currentPriority = curr
return priorityValueByPolicy(sq.priorityPolicy, sq.priorityOffset, curr)
}
+
+func (sq *Queue) UpdateApplicationPreemptionPriority(applicationID string, priority int32) {
+ if sq == nil || !sq.isLeaf {
+ return
+ }
+ value := sq.updateApplicationPreemptionPriorityInternal(applicationID, priority)
+ sq.parent.UpdateQueuePreemptionPriority(sq.Name, value)
+}
+
+func (sq *Queue) UpdateQueuePreemptionPriority(queueName string, priority int32) {
+ if sq == nil || sq.isLeaf {
+ return
+ }
+ value := sq.updateQueuePreemptionPriorityInternal(queueName, priority)
+ sq.parent.UpdateQueuePreemptionPriority(sq.Name, value)
+}
+
+func (sq *Queue) updateQueuePreemptionPriorityInternal(queueName string, priority int32) int32 {
+ sq.Lock()
+ defer sq.Unlock()
+
+ if _, ok := sq.children[queueName]; !ok {
+ log.Logger().Debug("Unknown queue", zap.String("queueName", queueName))
+ return sq.preemptionPriority
+ }
+
+ sq.childPreemptionPriorities[queueName] = priority
+ return sq.recalculatePreemptionPriority()
+}
+
+func (sq *Queue) updateApplicationPreemptionPriorityInternal(applicationID string, priority int32) int32 {
+ sq.Lock()
+ defer sq.Unlock()
+
+ if _, ok := sq.applications[applicationID]; !ok {
+ log.Logger().Debug("Unknown application", zap.String("applicationID", applicationID))
+ return sq.preemptionPriority
+ }
+
+ sq.appPreemptionPriorities[applicationID] = priority
+ return sq.recalculatePreemptionPriority()
+}
+
+func (sq *Queue) recalculatePreemptionPriority() int32 {
+ var items *map[string]int32
Review Comment:
There is no copy when you do the assign a map is a reference type: https://go.dev/blog/maps
Also see this example code:
https://go.dev/play/p/qcJ9P-4ZCdB
Both maps variables point to one and the same data structure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [yunikorn-core] wilfred-s commented on pull request #479: [YUNIKORN-1466] Track dynamic preemption priority for queues
Posted by GitBox <gi...@apache.org>.
wilfred-s commented on PR #479:
URL: https://github.com/apache/yunikorn-core/pull/479#issuecomment-1369311493
@pbacsko Please fix the conflicts so we can do a final review on the change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [yunikorn-core] wilfred-s commented on a diff in pull request #479: [YUNIKORN-1466] Track dynamic preemption priority for queues
Posted by GitBox <gi...@apache.org>.
wilfred-s commented on code in PR #479:
URL: https://github.com/apache/yunikorn-core/pull/479#discussion_r1062084065
##########
pkg/scheduler/objects/queue.go:
##########
@@ -628,6 +635,8 @@ func (sq *Queue) AddApplication(app *Application) {
defer sq.Unlock()
sq.applications[app.ApplicationID] = app
sq.appPriorities[app.ApplicationID] = app.GetAskMaxPriority()
+ sq.appPreemptionPriorities[app.ApplicationID] = app.GetAllocationMinPriority()
+
Review Comment:
The application object cannot be accessed by anything until the end of this function itself. The queue is locked.
The overhead is 2 function calls, and 2 lock acquire and release actions. Locking cannot be contested as the application object cannot be accessed yet. That overhead is less than the Tag check...
OK to leave it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [yunikorn-core] pbacsko commented on a diff in pull request #479: [YUNIKORN-1466] Track dynamic preemption priority for queues
Posted by GitBox <gi...@apache.org>.
pbacsko commented on code in PR #479:
URL: https://github.com/apache/yunikorn-core/pull/479#discussion_r1062282285
##########
pkg/scheduler/objects/queue.go:
##########
@@ -1571,3 +1593,84 @@ func (sq *Queue) recalculatePriority() int32 {
sq.currentPriority = curr
return priorityValueByPolicy(sq.priorityPolicy, sq.priorityOffset, curr)
}
+
+func (sq *Queue) UpdateApplicationPreemptionPriority(applicationID string, priority int32) {
+ if sq == nil || !sq.isLeaf {
+ return
+ }
+ value := sq.updateApplicationPreemptionPriorityInternal(applicationID, priority)
+ sq.parent.UpdateQueuePreemptionPriority(sq.Name, value)
+}
+
+func (sq *Queue) UpdateQueuePreemptionPriority(queueName string, priority int32) {
+ if sq == nil || sq.isLeaf {
+ return
+ }
+ value := sq.updateQueuePreemptionPriorityInternal(queueName, priority)
+ sq.parent.UpdateQueuePreemptionPriority(sq.Name, value)
+}
+
+func (sq *Queue) updateQueuePreemptionPriorityInternal(queueName string, priority int32) int32 {
+ sq.Lock()
+ defer sq.Unlock()
+
+ if _, ok := sq.children[queueName]; !ok {
+ log.Logger().Debug("Unknown queue", zap.String("queueName", queueName))
+ return sq.preemptionPriority
+ }
+
+ sq.childPreemptionPriorities[queueName] = priority
+ return sq.recalculatePreemptionPriority()
+}
+
+func (sq *Queue) updateApplicationPreemptionPriorityInternal(applicationID string, priority int32) int32 {
+ sq.Lock()
+ defer sq.Unlock()
+
+ if _, ok := sq.applications[applicationID]; !ok {
+ log.Logger().Debug("Unknown application", zap.String("applicationID", applicationID))
+ return sq.preemptionPriority
+ }
+
+ sq.appPreemptionPriorities[applicationID] = priority
+ return sq.recalculatePreemptionPriority()
+}
+
+func (sq *Queue) recalculatePreemptionPriority() int32 {
+ var items *map[string]int32
Review Comment:
Oh, that's great. Makes sense to change then.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [yunikorn-core] pbacsko commented on pull request #479: [YUNIKORN-1466] Track dynamic preemption priority for queues
Posted by GitBox <gi...@apache.org>.
pbacsko commented on PR #479:
URL: https://github.com/apache/yunikorn-core/pull/479#issuecomment-1351855869
> This is a good start, but we need to update the priorities when app priorities (for leaf queues) or subqueue priorities (for parent queues) change. Additionally, this needs to propagate up the tree and take into account priority offsets as well.
Updated the PR with the new approach, no tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [yunikorn-core] codecov[bot] commented on pull request #479: [YUNIKORN-1466] Track dynamic preemption priority for queues
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on PR #479:
URL: https://github.com/apache/yunikorn-core/pull/479#issuecomment-1351044623
# [Codecov](https://codecov.io/gh/apache/yunikorn-core/pull/479?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#479](https://codecov.io/gh/apache/yunikorn-core/pull/479?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7fa8321) into [master](https://codecov.io/gh/apache/yunikorn-core/commit/24fceecbee525dbb58479587346f23e9c676e0e4?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (24fceec) will **increase** coverage by `0.07%`.
> The diff coverage is `100.00%`.
```diff
@@ Coverage Diff @@
## master #479 +/- ##
==========================================
+ Coverage 72.52% 72.59% +0.07%
==========================================
Files 67 67
Lines 10114 10142 +28
==========================================
+ Hits 7335 7363 +28
Misses 2533 2533
Partials 246 246
```
| [Impacted Files](https://codecov.io/gh/apache/yunikorn-core/pull/479?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [pkg/scheduler/objects/application.go](https://codecov.io/gh/apache/yunikorn-core/pull/479/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL2FwcGxpY2F0aW9uLmdv) | `56.18% <100.00%> (+0.06%)` | :arrow_up: |
| [pkg/scheduler/objects/queue.go](https://codecov.io/gh/apache/yunikorn-core/pull/479/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL3F1ZXVlLmdv) | `71.64% <100.00%> (+0.80%)` | :arrow_up: |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [yunikorn-core] wilfred-s commented on a diff in pull request #479: [YUNIKORN-1466] Track dynamic preemption priority for queues
Posted by GitBox <gi...@apache.org>.
wilfred-s commented on code in PR #479:
URL: https://github.com/apache/yunikorn-core/pull/479#discussion_r1061089397
##########
pkg/scheduler/objects/queue.go:
##########
@@ -1571,3 +1593,84 @@ func (sq *Queue) recalculatePriority() int32 {
sq.currentPriority = curr
return priorityValueByPolicy(sq.priorityPolicy, sq.priorityOffset, curr)
}
+
+func (sq *Queue) UpdateApplicationPreemptionPriority(applicationID string, priority int32) {
+ if sq == nil || !sq.isLeaf {
Review Comment:
there is a locked method to check `sq.isLeaf` should be using that one
##########
pkg/scheduler/objects/queue.go:
##########
@@ -1571,3 +1593,84 @@ func (sq *Queue) recalculatePriority() int32 {
sq.currentPriority = curr
return priorityValueByPolicy(sq.priorityPolicy, sq.priorityOffset, curr)
}
+
+func (sq *Queue) UpdateApplicationPreemptionPriority(applicationID string, priority int32) {
+ if sq == nil || !sq.isLeaf {
+ return
+ }
+ value := sq.updateApplicationPreemptionPriorityInternal(applicationID, priority)
+ sq.parent.UpdateQueuePreemptionPriority(sq.Name, value)
+}
+
+func (sq *Queue) UpdateQueuePreemptionPriority(queueName string, priority int32) {
+ if sq == nil || sq.isLeaf {
Review Comment:
there is a locked method to check `sq.isLeaf` should be using that one
##########
pkg/scheduler/objects/queue.go:
##########
@@ -628,6 +635,8 @@ func (sq *Queue) AddApplication(app *Application) {
defer sq.Unlock()
sq.applications[app.ApplicationID] = app
sq.appPriorities[app.ApplicationID] = app.GetAskMaxPriority()
+ sq.appPreemptionPriorities[app.ApplicationID] = app.GetAllocationMinPriority()
+
Review Comment:
I did not see this when we added priorities earlier: when an application gets added to a queue there are no asks or allocations yet. The `queue.AddApplication()` call is triggered from a new application request from a shim and that request cannot have an ask/allocation. The init can be a simple default value for both.
##########
pkg/scheduler/objects/queue.go:
##########
@@ -1571,3 +1593,84 @@ func (sq *Queue) recalculatePriority() int32 {
sq.currentPriority = curr
return priorityValueByPolicy(sq.priorityPolicy, sq.priorityOffset, curr)
}
+
+func (sq *Queue) UpdateApplicationPreemptionPriority(applicationID string, priority int32) {
+ if sq == nil || !sq.isLeaf {
+ return
+ }
+ value := sq.updateApplicationPreemptionPriorityInternal(applicationID, priority)
+ sq.parent.UpdateQueuePreemptionPriority(sq.Name, value)
+}
+
+func (sq *Queue) UpdateQueuePreemptionPriority(queueName string, priority int32) {
+ if sq == nil || sq.isLeaf {
+ return
+ }
+ value := sq.updateQueuePreemptionPriorityInternal(queueName, priority)
+ sq.parent.UpdateQueuePreemptionPriority(sq.Name, value)
+}
+
+func (sq *Queue) updateQueuePreemptionPriorityInternal(queueName string, priority int32) int32 {
+ sq.Lock()
+ defer sq.Unlock()
+
+ if _, ok := sq.children[queueName]; !ok {
+ log.Logger().Debug("Unknown queue", zap.String("queueName", queueName))
+ return sq.preemptionPriority
+ }
+
+ sq.childPreemptionPriorities[queueName] = priority
+ return sq.recalculatePreemptionPriority()
+}
+
+func (sq *Queue) updateApplicationPreemptionPriorityInternal(applicationID string, priority int32) int32 {
+ sq.Lock()
+ defer sq.Unlock()
+
+ if _, ok := sq.applications[applicationID]; !ok {
+ log.Logger().Debug("Unknown application", zap.String("applicationID", applicationID))
+ return sq.preemptionPriority
+ }
+
+ sq.appPreemptionPriorities[applicationID] = priority
+ return sq.recalculatePreemptionPriority()
+}
+
+func (sq *Queue) recalculatePreemptionPriority() int32 {
+ var items *map[string]int32
Review Comment:
I am not sure that I understand this indirection. If I say:
```
items = sq.appPreemptionPriorities
```
both will point to the same data structure it will not copy the data.
##########
pkg/scheduler/objects/queue.go:
##########
@@ -691,11 +700,14 @@ func (sq *Queue) RemoveApplication(app *Application) {
sq.Lock()
delete(sq.applications, appID)
delete(sq.appPriorities, appID)
+ delete(sq.appPreemptionPriorities, appID)
delete(sq.allocatingAcceptedApps, appID)
priority := sq.recalculatePriority()
+ preemptionPriority := sq.recalculatePreemptionPriority()
sq.Unlock()
Review Comment:
Can we refactor all these calls into a separate locked function `internalRemoveApp(appID string) int32` makes it much cleaner and a defer can be used to guarantee the unlock even if we add things later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org