You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@yunikorn.apache.org by "pbacsko (via GitHub)" <gi...@apache.org> on 2023/05/15 12:51:13 UTC
[GitHub] [yunikorn-core] pbacsko opened a new pull request, #547: [YUNIKORN-1719] Improve the performance of Application.sortRequests()
pbacsko opened a new pull request, #547:
URL: https://github.com/apache/yunikorn-core/pull/547
### What is this PR for?
Avoid calling `Application.sortRequests()` constantly. We only need to sort if a new request is added. If `pendingAskRepeat` becomes 0 for a request, we simply skip it.
After doing various benchmarks, this approach is simpler and even faster than maintaining requests in a sorted BTree.
### What type of PR is it?
* [ ] - Bug Fix
* [x] - Improvement
* [ ] - Feature
* [ ] - Documentation
* [ ] - Hot Fix
* [ ] - Refactoring
### Todos
* [ ] - Task
### What is the Jira issue?
https://issues.apache.org/jira/browse/YUNIKORN-1719
### How should this be tested?
### Screenshots (if appropriate)
### Questions:
* [ ] - The licenses files need update.
* [ ] - There is breaking changes for older versions.
* [ ] - It needs documentation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [yunikorn-core] craigcondit closed pull request #547: [YUNIKORN-1719] Improve the performance of Application.sortRequests()
Posted by "craigcondit (via GitHub)" <gi...@apache.org>.
craigcondit closed pull request #547: [YUNIKORN-1719] Improve the performance of Application.sortRequests()
URL: https://github.com/apache/yunikorn-core/pull/547
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [yunikorn-core] pbacsko commented on pull request #547: [YUNIKORN-1719] Improve the performance of Application.sortRequests()
Posted by "pbacsko (via GitHub)" <gi...@apache.org>.
pbacsko commented on PR #547:
URL: https://github.com/apache/yunikorn-core/pull/547#issuecomment-1549722570
> I see that we also have a small leak for the case that we call removeAsksInternal with an empty string. We remove all requests but we do not reset the sortedRequests to nil (line 516) we should add `sa.sortedRequests = nil` It is a leak that eventually will fix itself but we need to be consistent.
Thinking about creating some new unit tests. Now we maintain more state and we probably should focus on proper coverage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [yunikorn-core] wilfred-s commented on pull request #547: [YUNIKORN-1719] Improve the performance of Application.sortRequests()
Posted by "wilfred-s (via GitHub)" <gi...@apache.org>.
wilfred-s commented on PR #547:
URL: https://github.com/apache/yunikorn-core/pull/547#issuecomment-1550518824
> > (sa *Application) RecoverAllocationAsk --> ignore as all asks have a repeat of 0
>
> Interesting, but it means that the following branch in the same method is never taken:
>
> ```
> // update app priority
> repeat := ask.GetPendingAskRepeat()
> priority := ask.GetPriority()
> if repeat > 0 && priority > sa.askMaxPriority {
> sa.askMaxPriority = priority
> sa.queue.UpdateApplicationPriority(sa.ApplicationID, sa.askMaxPriority)
> }
> ```
Correct that can be removed. The request is purely there for consistency. It has been allocated already and does not influence priority anymore.
We should file a follow up jira to remove that bit of code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [yunikorn-core] codecov[bot] commented on pull request #547: [YUNIKORN-1719] Improve the performance of Application.sortRequests()
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #547:
URL: https://github.com/apache/yunikorn-core/pull/547#issuecomment-1547808462
## [Codecov](https://app.codecov.io/gh/apache/yunikorn-core/pull/547?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
> Merging [#547](https://app.codecov.io/gh/apache/yunikorn-core/pull/547?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (1000099) into [master](https://app.codecov.io/gh/apache/yunikorn-core/commit/b1c04fdf47a0bf963a687718841798e61796aa84?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (b1c04fd) will **decrease** coverage by `0.02%`.
> The diff coverage is `100.00%`.
```diff
@@ Coverage Diff @@
## master #547 +/- ##
==========================================
- Coverage 74.97% 74.96% -0.02%
==========================================
Files 70 70
Lines 11371 11374 +3
==========================================
+ Hits 8525 8526 +1
- Misses 2545 2546 +1
- Partials 301 302 +1
```
| [Impacted Files](https://app.codecov.io/gh/apache/yunikorn-core/pull/547?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [pkg/scheduler/objects/application.go](https://app.codecov.io/gh/apache/yunikorn-core/pull/547?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL2FwcGxpY2F0aW9uLmdv) | `64.35% <100.00%> (-0.07%)` | :arrow_down: |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [yunikorn-core] pbacsko commented on pull request #547: [YUNIKORN-1719] Improve the performance of Application.sortRequests()
Posted by "pbacsko (via GitHub)" <gi...@apache.org>.
pbacsko commented on PR #547:
URL: https://github.com/apache/yunikorn-core/pull/547#issuecomment-1549616225
> (sa *Application) RecoverAllocationAsk --> ignore as all asks have a repeat of 0
Interesting, but it means that the following branch in the same method is never taken:
```
// update app priority
repeat := ask.GetPendingAskRepeat()
priority := ask.GetPriority()
if repeat > 0 && priority > sa.askMaxPriority {
sa.askMaxPriority = priority
sa.queue.UpdateApplicationPriority(sa.ApplicationID, sa.askMaxPriority)
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [yunikorn-core] wilfred-s commented on pull request #547: [YUNIKORN-1719] Improve the performance of Application.sortRequests()
Posted by "wilfred-s (via GitHub)" <gi...@apache.org>.
wilfred-s commented on PR #547:
URL: https://github.com/apache/yunikorn-core/pull/547#issuecomment-1548925776
I see that we also have a small leak for the case that we call removeAsksInternal with an empty string. We remove all requests but we do not reset the sortedRequests to nil (line 516) we should add `sa.sortedRequests = nil` It is a leak that eventually will fix itself but we need to be consistent.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org