You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@yunikorn.apache.org by GitBox <gi...@apache.org> on 2021/11/09 17:57:00 UTC
[GitHub] [incubator-yunikorn-core] pbacsko opened a new pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
pbacsko opened a new pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336
### What is this PR for?
Add new REST interfaces to the web service:
* request a full state dump from Yunikorn
* enable writing full state information periodically to an external file
* disable periodic state dumps
### What type of PR is it?
* [ ] - Bug Fix
* [x] - Improvement
* [ ] - Feature
* [ ] - Documentation
* [ ] - Hot Fix
* [ ] - Refactoring
### What is the Jira issue?
https://issues.apache.org/jira/browse/YUNIKORN-940
### How should this be tested?
### Screenshots (if appropriate)
### Questions:
* [ ] - The licenses files need update.
* [ ] - There is breaking changes for older versions.
* [ ] - It needs documentation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] pbacsko commented on a change in pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
pbacsko commented on a change in pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#discussion_r748286930
##########
File path: pkg/webservice/routes.go
##########
@@ -170,6 +170,24 @@ var webRoutes = routes{
"/ws/v1/partition/{partition}/queue/{queue}/applications",
getQueueApplications,
},
+ route{
+ "Scheduler",
+ "GET",
+ "/ws/v1/fullstatedump",
+ getFullStateDump,
+ },
+ route{
+ "Scheduler",
+ "PUT",
+ "/ws/v1/enableperiodicstatedump/{period}",
+ enablePeriodicStateDump,
+ },
+ route{
+ "Scheduler",
+ "PUT",
+ "/ws/v1/disableperiodicstatedump",
+ disablePeriodicStateDump,
+ },
Review comment:
Any ideas? Like `"/ws/v1/statedump/{switch}/{period}" or what?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] codecov[bot] edited a comment on pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#issuecomment-966414547
# [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#336](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e9496d2) into [master](https://codecov.io/gh/apache/incubator-yunikorn-core/commit/5a1c19e280f66d6cb4fc206a675116260d6f2ff1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5a1c19e) will **increase** coverage by `4.32%`.
> The diff coverage is `69.06%`.
[](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #336 +/- ##
==========================================
+ Coverage 63.46% 67.79% +4.32%
==========================================
Files 60 64 +4
Lines 5220 6198 +978
==========================================
+ Hits 3313 4202 +889
- Misses 1747 1773 +26
- Partials 160 223 +63
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [pkg/metrics/init.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvaW5pdC5nbw==) | `62.96% <ø> (ø)` | |
| [pkg/scheduler/drf\_preemption\_policy.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9kcmZfcHJlZW1wdGlvbl9wb2xpY3kuZ28=) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/nodes\_usage\_monitor.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9ub2Rlc191c2FnZV9tb25pdG9yLmdv) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/objects/allocation.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL2FsbG9jYXRpb24uZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/node\_iterator.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL25vZGVfaXRlcmF0b3IuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/sorters.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL3NvcnRlcnMuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/webservice/webservice.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3dlYnNlcnZpY2Uvd2Vic2VydmljZS5nbw==) | `32.25% <ø> (+19.35%)` | :arrow_up: |
| [pkg/scheduler/context.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9jb250ZXh0Lmdv) | `16.55% <17.82%> (+10.28%)` | :arrow_up: |
| [pkg/metrics/metrics\_collector.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvbWV0cmljc19jb2xsZWN0b3IuZ28=) | `64.51% <45.00%> (-9.40%)` | :arrow_down: |
| [pkg/metrics/scheduler.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3Mvc2NoZWR1bGVyLmdv) | `60.89% <54.65%> (-0.69%)` | :arrow_down: |
| ... and [35 more](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [66eaecc...e9496d2](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] codecov[bot] edited a comment on pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#issuecomment-966414547
# [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#336](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (bc63b04) into [master](https://codecov.io/gh/apache/incubator-yunikorn-core/commit/5a1c19e280f66d6cb4fc206a675116260d6f2ff1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5a1c19e) will **increase** coverage by `4.34%`.
> The diff coverage is `69.12%`.
[](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #336 +/- ##
==========================================
+ Coverage 63.46% 67.81% +4.34%
==========================================
Files 60 64 +4
Lines 5220 6214 +994
==========================================
+ Hits 3313 4214 +901
- Misses 1747 1778 +31
- Partials 160 222 +62
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [pkg/metrics/init.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvaW5pdC5nbw==) | `62.96% <ø> (ø)` | |
| [pkg/scheduler/drf\_preemption\_policy.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9kcmZfcHJlZW1wdGlvbl9wb2xpY3kuZ28=) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/nodes\_usage\_monitor.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9ub2Rlc191c2FnZV9tb25pdG9yLmdv) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/objects/allocation.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL2FsbG9jYXRpb24uZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/node\_iterator.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL25vZGVfaXRlcmF0b3IuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/sorters.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL3NvcnRlcnMuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/webservice/webservice.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3dlYnNlcnZpY2Uvd2Vic2VydmljZS5nbw==) | `32.25% <ø> (+19.35%)` | :arrow_up: |
| [pkg/scheduler/context.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9jb250ZXh0Lmdv) | `16.55% <17.82%> (+10.28%)` | :arrow_up: |
| [pkg/metrics/metrics\_collector.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvbWV0cmljc19jb2xsZWN0b3IuZ28=) | `64.51% <45.00%> (-9.40%)` | :arrow_down: |
| [pkg/metrics/scheduler.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3Mvc2NoZWR1bGVyLmdv) | `60.89% <54.65%> (-0.69%)` | :arrow_down: |
| ... and [35 more](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [66eaecc...bc63b04](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] codecov[bot] edited a comment on pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#issuecomment-966414547
# [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#336](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e9496d2) into [master](https://codecov.io/gh/apache/incubator-yunikorn-core/commit/5a1c19e280f66d6cb4fc206a675116260d6f2ff1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5a1c19e) will **increase** coverage by `4.32%`.
> The diff coverage is `69.06%`.
> :exclamation: Current head e9496d2 differs from pull request most recent head 4a88a47. Consider uploading reports for the commit 4a88a47 to get more accurate results
[](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #336 +/- ##
==========================================
+ Coverage 63.46% 67.79% +4.32%
==========================================
Files 60 64 +4
Lines 5220 6198 +978
==========================================
+ Hits 3313 4202 +889
- Misses 1747 1773 +26
- Partials 160 223 +63
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [pkg/metrics/init.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvaW5pdC5nbw==) | `62.96% <ø> (ø)` | |
| [pkg/scheduler/drf\_preemption\_policy.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9kcmZfcHJlZW1wdGlvbl9wb2xpY3kuZ28=) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/nodes\_usage\_monitor.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9ub2Rlc191c2FnZV9tb25pdG9yLmdv) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/objects/allocation.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL2FsbG9jYXRpb24uZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/node\_iterator.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL25vZGVfaXRlcmF0b3IuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/sorters.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL3NvcnRlcnMuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/webservice/webservice.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3dlYnNlcnZpY2Uvd2Vic2VydmljZS5nbw==) | `32.25% <ø> (+19.35%)` | :arrow_up: |
| [pkg/scheduler/context.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9jb250ZXh0Lmdv) | `16.55% <17.82%> (+10.28%)` | :arrow_up: |
| [pkg/metrics/metrics\_collector.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvbWV0cmljc19jb2xsZWN0b3IuZ28=) | `64.51% <45.00%> (-9.40%)` | :arrow_down: |
| [pkg/metrics/scheduler.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3Mvc2NoZWR1bGVyLmdv) | `60.89% <54.65%> (-0.69%)` | :arrow_down: |
| ... and [35 more](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [66eaecc...4a88a47](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] codecov[bot] edited a comment on pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#issuecomment-966414547
# [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#336](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (44a805d) into [master](https://codecov.io/gh/apache/incubator-yunikorn-core/commit/5a1c19e280f66d6cb4fc206a675116260d6f2ff1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5a1c19e) will **increase** coverage by `4.32%`.
> The diff coverage is `69.06%`.
> :exclamation: Current head 44a805d differs from pull request most recent head cafdb27. Consider uploading reports for the commit cafdb27 to get more accurate results
[](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #336 +/- ##
==========================================
+ Coverage 63.46% 67.79% +4.32%
==========================================
Files 60 64 +4
Lines 5220 6198 +978
==========================================
+ Hits 3313 4202 +889
- Misses 1747 1773 +26
- Partials 160 223 +63
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [pkg/metrics/init.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvaW5pdC5nbw==) | `62.96% <ø> (ø)` | |
| [pkg/scheduler/drf\_preemption\_policy.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9kcmZfcHJlZW1wdGlvbl9wb2xpY3kuZ28=) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/nodes\_usage\_monitor.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9ub2Rlc191c2FnZV9tb25pdG9yLmdv) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/objects/allocation.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL2FsbG9jYXRpb24uZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/node\_iterator.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL25vZGVfaXRlcmF0b3IuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/sorters.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL3NvcnRlcnMuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/webservice/webservice.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3dlYnNlcnZpY2Uvd2Vic2VydmljZS5nbw==) | `32.25% <ø> (+19.35%)` | :arrow_up: |
| [pkg/scheduler/context.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9jb250ZXh0Lmdv) | `16.55% <17.82%> (+10.28%)` | :arrow_up: |
| [pkg/metrics/metrics\_collector.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvbWV0cmljc19jb2xsZWN0b3IuZ28=) | `64.51% <45.00%> (-9.40%)` | :arrow_down: |
| [pkg/metrics/scheduler.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3Mvc2NoZWR1bGVyLmdv) | `60.89% <54.65%> (-0.69%)` | :arrow_down: |
| ... and [35 more](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [66eaecc...cafdb27](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] codecov[bot] edited a comment on pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#issuecomment-966414547
# [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#336](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (cafdb27) into [master](https://codecov.io/gh/apache/incubator-yunikorn-core/commit/5a1c19e280f66d6cb4fc206a675116260d6f2ff1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5a1c19e) will **increase** coverage by `4.32%`.
> The diff coverage is `69.06%`.
> :exclamation: Current head cafdb27 differs from pull request most recent head e9496d2. Consider uploading reports for the commit e9496d2 to get more accurate results
[](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #336 +/- ##
==========================================
+ Coverage 63.46% 67.79% +4.32%
==========================================
Files 60 64 +4
Lines 5220 6198 +978
==========================================
+ Hits 3313 4202 +889
- Misses 1747 1773 +26
- Partials 160 223 +63
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [pkg/metrics/init.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvaW5pdC5nbw==) | `62.96% <ø> (ø)` | |
| [pkg/scheduler/drf\_preemption\_policy.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9kcmZfcHJlZW1wdGlvbl9wb2xpY3kuZ28=) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/nodes\_usage\_monitor.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9ub2Rlc191c2FnZV9tb25pdG9yLmdv) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/objects/allocation.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL2FsbG9jYXRpb24uZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/node\_iterator.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL25vZGVfaXRlcmF0b3IuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/sorters.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL3NvcnRlcnMuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/webservice/webservice.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3dlYnNlcnZpY2Uvd2Vic2VydmljZS5nbw==) | `32.25% <ø> (+19.35%)` | :arrow_up: |
| [pkg/scheduler/context.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9jb250ZXh0Lmdv) | `16.55% <17.82%> (+10.28%)` | :arrow_up: |
| [pkg/metrics/metrics\_collector.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvbWV0cmljc19jb2xsZWN0b3IuZ28=) | `64.51% <45.00%> (-9.40%)` | :arrow_down: |
| [pkg/metrics/scheduler.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3Mvc2NoZWR1bGVyLmdv) | `60.89% <54.65%> (-0.69%)` | :arrow_down: |
| ... and [35 more](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [66eaecc...e9496d2](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] pbacsko commented on a change in pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
pbacsko commented on a change in pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#discussion_r748279872
##########
File path: pkg/webservice/state_dump.go
##########
@@ -0,0 +1,207 @@
+/*
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+package webservice
+
+import (
+ "encoding/json"
+ "fmt"
+ "io"
+ "net/http"
+ "os"
+ "strconv"
+ "sync"
+ "time"
+
+ "github.com/apache/incubator-yunikorn-core/pkg/log"
+ metrics2 "github.com/apache/incubator-yunikorn-core/pkg/metrics"
+ "github.com/apache/incubator-yunikorn-core/pkg/scheduler"
+ "github.com/apache/incubator-yunikorn-core/pkg/webservice/dao"
+
+ "github.com/gorilla/mux"
+ "go.uber.org/zap"
+)
+
+const (
+ defaultStateDumpPeriod = 60
Review comment:
> Creating the state dump by default every minute seems a little too aggressive for me. It will make huge files. I think we should print the dumps less freqvently. What is the default value in yarn for the state dump creeation?
Fair Scheduler performs this step in every 25 `update()` call (update is called for every heartbeat). To me, a state dump per minute does not sound that aggressive.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] codecov[bot] edited a comment on pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#issuecomment-966414547
# [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#336](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e9496d2) into [master](https://codecov.io/gh/apache/incubator-yunikorn-core/commit/5a1c19e280f66d6cb4fc206a675116260d6f2ff1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5a1c19e) will **increase** coverage by `4.32%`.
> The diff coverage is `69.06%`.
> :exclamation: Current head e9496d2 differs from pull request most recent head 6a54d56. Consider uploading reports for the commit 6a54d56 to get more accurate results
[](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #336 +/- ##
==========================================
+ Coverage 63.46% 67.79% +4.32%
==========================================
Files 60 64 +4
Lines 5220 6198 +978
==========================================
+ Hits 3313 4202 +889
- Misses 1747 1773 +26
- Partials 160 223 +63
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [pkg/metrics/init.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvaW5pdC5nbw==) | `62.96% <ø> (ø)` | |
| [pkg/scheduler/drf\_preemption\_policy.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9kcmZfcHJlZW1wdGlvbl9wb2xpY3kuZ28=) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/nodes\_usage\_monitor.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9ub2Rlc191c2FnZV9tb25pdG9yLmdv) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/objects/allocation.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL2FsbG9jYXRpb24uZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/node\_iterator.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL25vZGVfaXRlcmF0b3IuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/sorters.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL3NvcnRlcnMuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/webservice/webservice.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3dlYnNlcnZpY2Uvd2Vic2VydmljZS5nbw==) | `32.25% <ø> (+19.35%)` | :arrow_up: |
| [pkg/scheduler/context.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9jb250ZXh0Lmdv) | `16.55% <17.82%> (+10.28%)` | :arrow_up: |
| [pkg/metrics/metrics\_collector.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvbWV0cmljc19jb2xsZWN0b3IuZ28=) | `64.51% <45.00%> (-9.40%)` | :arrow_down: |
| [pkg/metrics/scheduler.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3Mvc2NoZWR1bGVyLmdv) | `60.89% <54.65%> (-0.69%)` | :arrow_down: |
| ... and [35 more](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [66eaecc...6a54d56](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] pbacsko commented on a change in pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
pbacsko commented on a change in pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#discussion_r748274190
##########
File path: pkg/webservice/handlers_test.go
##########
@@ -1279,3 +1282,159 @@ func TestGetNodesUtilization(t *testing.T) {
assert.NilError(t, err)
assert.Equal(t, len(nodesDao), 0)
}
+
+func TestGetFullStateDump(t *testing.T) {
+ schedulerContext = prepareSchedulerContext(t)
+
+ imHistory = history.NewInternalMetricsHistory(5)
+ req, err := http.NewRequest("GET", "/ws/v1/getfullstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ getFullStateDump(resp, req)
+ receivedBytes := resp.outputBytes
+ statusCode := resp.statusCode
+ assert.Assert(t, len(receivedBytes) > 0, "json response is empty")
+ assert.Check(t, statusCode != http.StatusInternalServerError, "response status code")
+ var aggregated AggregatedStateInfo
+ if err := json.Unmarshal(receivedBytes, &aggregated); err != nil {
+ t.Fatal(err)
+ }
+ verifyStateDumpJSON(t, &aggregated)
+}
+
+func TestEnableDisablePeriodicStateDump(t *testing.T) {
+ schedulerContext = prepareSchedulerContext(t)
+ defer deleteStateDumpFile()
+ defer terminateGoroutine()
+
+ imHistory = history.NewInternalMetricsHistory(5)
+ req, err := http.NewRequest("GET", "/ws/v1/enableperiodicstatedump", strings.NewReader(""))
+ vars := map[string]string{
+ "period": "3",
+ }
+ req = mux.SetURLVars(req, vars)
+
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ // enable state dump, check file contents
+ enablePeriodicStateDump(resp, req)
+ statusCode := resp.statusCode
+ assert.Check(t, statusCode != http.StatusInternalServerError, "response status code")
+
+ waitForStateDumpFile(t)
+ fileContents, err2 := ioutil.ReadFile(stateDumpFilePath)
+ if err2 != nil {
+ t.Fatal(err2)
+ }
+ var aggregated AggregatedStateInfo
+ if err3 := json.Unmarshal(fileContents, &aggregated); err3 != nil {
+ t.Fatal(err3)
+ }
+ verifyStateDumpJSON(t, &aggregated)
+
+ // disable
+ req, err = http.NewRequest("GET", "/ws/v1/disableperiodicstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp = &MockResponseWriter{}
+ disablePeriodicStateDump(resp, req)
+ statusCode = resp.statusCode
+ assert.Assert(t, statusCode != http.StatusInternalServerError, "response status code")
+}
+
+func TestTryEnableStateDumpTwice(t *testing.T) {
+ defer deleteStateDumpFile()
+ defer terminateGoroutine()
+
+ req, err := http.NewRequest("GET", "/ws/v1/enableperiodicstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ enablePeriodicStateDump(resp, req)
+ enablePeriodicStateDump(resp, req)
+
+ statusCode := resp.statusCode
+ assert.Assert(t, statusCode == http.StatusInternalServerError, "response status code")
+}
+
+func TestTryDisableNotRunningStateDump(t *testing.T) {
+ req, err := http.NewRequest("GET", "/ws/v1/disableperiodicstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ disablePeriodicStateDump(resp, req)
+
+ statusCode := resp.statusCode
+ assert.Assert(t, statusCode == http.StatusInternalServerError, "response status code")
+}
+
+func prepareSchedulerContext(t *testing.T) *scheduler.ClusterContext {
+ configs.MockSchedulerConfigByData([]byte(configDefault))
+ var err error
+ schedulerContext, err = scheduler.NewClusterContext(rmID, policyGroup)
+ assert.NilError(t, err, "Error when load clusterInfo from config")
+ assert.Equal(t, 1, len(schedulerContext.GetPartitionMapClone()))
+
+ return schedulerContext
+}
+
+func waitForStateDumpFile(t *testing.T) {
+ for {
+ var attempts int
+
+ info, err := os.Stat(stateDumpFilePath)
+
+ // tolerate only "file not found" errors
+ if err != nil && !os.IsNotExist(err) {
Review comment:
Other error mean different things (eg. permission denied) and it does not make sense to continue (I don't expect it to happen though). "File not found" is normal, we have to wait until it is created by the background goroutine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] codecov[bot] edited a comment on pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#issuecomment-966414547
# [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#336](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (cafdb27) into [master](https://codecov.io/gh/apache/incubator-yunikorn-core/commit/5a1c19e280f66d6cb4fc206a675116260d6f2ff1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5a1c19e) will **increase** coverage by `4.32%`.
> The diff coverage is `69.06%`.
> :exclamation: Current head cafdb27 differs from pull request most recent head a1b394f. Consider uploading reports for the commit a1b394f to get more accurate results
[](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #336 +/- ##
==========================================
+ Coverage 63.46% 67.79% +4.32%
==========================================
Files 60 64 +4
Lines 5220 6198 +978
==========================================
+ Hits 3313 4202 +889
- Misses 1747 1773 +26
- Partials 160 223 +63
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [pkg/metrics/init.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvaW5pdC5nbw==) | `62.96% <ø> (ø)` | |
| [pkg/scheduler/drf\_preemption\_policy.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9kcmZfcHJlZW1wdGlvbl9wb2xpY3kuZ28=) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/nodes\_usage\_monitor.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9ub2Rlc191c2FnZV9tb25pdG9yLmdv) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/objects/allocation.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL2FsbG9jYXRpb24uZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/node\_iterator.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL25vZGVfaXRlcmF0b3IuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/sorters.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL3NvcnRlcnMuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/webservice/webservice.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3dlYnNlcnZpY2Uvd2Vic2VydmljZS5nbw==) | `32.25% <ø> (+19.35%)` | :arrow_up: |
| [pkg/scheduler/context.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9jb250ZXh0Lmdv) | `16.55% <17.82%> (+10.28%)` | :arrow_up: |
| [pkg/metrics/metrics\_collector.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvbWV0cmljc19jb2xsZWN0b3IuZ28=) | `64.51% <45.00%> (-9.40%)` | :arrow_down: |
| [pkg/metrics/scheduler.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3Mvc2NoZWR1bGVyLmdv) | `60.89% <54.65%> (-0.69%)` | :arrow_down: |
| ... and [35 more](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [66eaecc...a1b394f](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] codecov[bot] edited a comment on pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#issuecomment-966414547
# [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#336](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e9496d2) into [master](https://codecov.io/gh/apache/incubator-yunikorn-core/commit/5a1c19e280f66d6cb4fc206a675116260d6f2ff1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5a1c19e) will **increase** coverage by `4.32%`.
> The diff coverage is `69.06%`.
> :exclamation: Current head e9496d2 differs from pull request most recent head afa336f. Consider uploading reports for the commit afa336f to get more accurate results
[](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #336 +/- ##
==========================================
+ Coverage 63.46% 67.79% +4.32%
==========================================
Files 60 64 +4
Lines 5220 6198 +978
==========================================
+ Hits 3313 4202 +889
- Misses 1747 1773 +26
- Partials 160 223 +63
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [pkg/metrics/init.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvaW5pdC5nbw==) | `62.96% <ø> (ø)` | |
| [pkg/scheduler/drf\_preemption\_policy.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9kcmZfcHJlZW1wdGlvbl9wb2xpY3kuZ28=) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/nodes\_usage\_monitor.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9ub2Rlc191c2FnZV9tb25pdG9yLmdv) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/objects/allocation.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL2FsbG9jYXRpb24uZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/node\_iterator.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL25vZGVfaXRlcmF0b3IuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/sorters.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL3NvcnRlcnMuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/webservice/webservice.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3dlYnNlcnZpY2Uvd2Vic2VydmljZS5nbw==) | `32.25% <ø> (+19.35%)` | :arrow_up: |
| [pkg/scheduler/context.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9jb250ZXh0Lmdv) | `16.55% <17.82%> (+10.28%)` | :arrow_up: |
| [pkg/metrics/metrics\_collector.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvbWV0cmljc19jb2xsZWN0b3IuZ28=) | `64.51% <45.00%> (-9.40%)` | :arrow_down: |
| [pkg/metrics/scheduler.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3Mvc2NoZWR1bGVyLmdv) | `60.89% <54.65%> (-0.69%)` | :arrow_down: |
| ... and [35 more](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [66eaecc...afa336f](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] codecov[bot] edited a comment on pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#issuecomment-966414547
# [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#336](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (cafdb27) into [master](https://codecov.io/gh/apache/incubator-yunikorn-core/commit/5a1c19e280f66d6cb4fc206a675116260d6f2ff1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5a1c19e) will **increase** coverage by `4.32%`.
> The diff coverage is `69.06%`.
[](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #336 +/- ##
==========================================
+ Coverage 63.46% 67.79% +4.32%
==========================================
Files 60 64 +4
Lines 5220 6198 +978
==========================================
+ Hits 3313 4202 +889
- Misses 1747 1773 +26
- Partials 160 223 +63
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [pkg/metrics/init.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvaW5pdC5nbw==) | `62.96% <ø> (ø)` | |
| [pkg/scheduler/drf\_preemption\_policy.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9kcmZfcHJlZW1wdGlvbl9wb2xpY3kuZ28=) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/nodes\_usage\_monitor.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9ub2Rlc191c2FnZV9tb25pdG9yLmdv) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/objects/allocation.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL2FsbG9jYXRpb24uZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/node\_iterator.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL25vZGVfaXRlcmF0b3IuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/sorters.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL3NvcnRlcnMuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/webservice/webservice.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3dlYnNlcnZpY2Uvd2Vic2VydmljZS5nbw==) | `32.25% <ø> (+19.35%)` | :arrow_up: |
| [pkg/scheduler/context.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9jb250ZXh0Lmdv) | `16.55% <17.82%> (+10.28%)` | :arrow_up: |
| [pkg/metrics/metrics\_collector.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvbWV0cmljc19jb2xsZWN0b3IuZ28=) | `64.51% <45.00%> (-9.40%)` | :arrow_down: |
| [pkg/metrics/scheduler.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3Mvc2NoZWR1bGVyLmdv) | `60.89% <54.65%> (-0.69%)` | :arrow_down: |
| ... and [35 more](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [66eaecc...cafdb27](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] codecov[bot] commented on pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#issuecomment-966414547
# [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#336](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (44a805d) into [master](https://codecov.io/gh/apache/incubator-yunikorn-core/commit/5a1c19e280f66d6cb4fc206a675116260d6f2ff1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5a1c19e) will **increase** coverage by `4.32%`.
> The diff coverage is `69.06%`.
[](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #336 +/- ##
==========================================
+ Coverage 63.46% 67.79% +4.32%
==========================================
Files 60 64 +4
Lines 5220 6198 +978
==========================================
+ Hits 3313 4202 +889
- Misses 1747 1773 +26
- Partials 160 223 +63
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [pkg/metrics/init.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvaW5pdC5nbw==) | `62.96% <ø> (ø)` | |
| [pkg/scheduler/drf\_preemption\_policy.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9kcmZfcHJlZW1wdGlvbl9wb2xpY3kuZ28=) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/nodes\_usage\_monitor.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9ub2Rlc191c2FnZV9tb25pdG9yLmdv) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/objects/allocation.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL2FsbG9jYXRpb24uZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/node\_iterator.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL25vZGVfaXRlcmF0b3IuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/sorters.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL3NvcnRlcnMuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/webservice/webservice.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3dlYnNlcnZpY2Uvd2Vic2VydmljZS5nbw==) | `32.25% <ø> (+19.35%)` | :arrow_up: |
| [pkg/scheduler/context.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9jb250ZXh0Lmdv) | `16.55% <17.82%> (+10.28%)` | :arrow_up: |
| [pkg/metrics/metrics\_collector.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvbWV0cmljc19jb2xsZWN0b3IuZ28=) | `64.51% <45.00%> (-9.40%)` | :arrow_down: |
| [pkg/metrics/scheduler.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3Mvc2NoZWR1bGVyLmdv) | `60.89% <54.65%> (-0.69%)` | :arrow_down: |
| ... and [35 more](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [66eaecc...44a805d](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] codecov[bot] edited a comment on pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#issuecomment-966414547
# [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#336](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e37b1f0) into [master](https://codecov.io/gh/apache/incubator-yunikorn-core/commit/5a1c19e280f66d6cb4fc206a675116260d6f2ff1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5a1c19e) will **increase** coverage by `4.40%`.
> The diff coverage is `69.33%`.
[](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #336 +/- ##
==========================================
+ Coverage 63.46% 67.86% +4.40%
==========================================
Files 60 64 +4
Lines 5220 6212 +992
==========================================
+ Hits 3313 4216 +903
- Misses 1747 1773 +26
- Partials 160 223 +63
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [pkg/metrics/init.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvaW5pdC5nbw==) | `62.96% <ø> (ø)` | |
| [pkg/scheduler/drf\_preemption\_policy.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9kcmZfcHJlZW1wdGlvbl9wb2xpY3kuZ28=) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/nodes\_usage\_monitor.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9ub2Rlc191c2FnZV9tb25pdG9yLmdv) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/objects/allocation.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL2FsbG9jYXRpb24uZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/node\_iterator.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL25vZGVfaXRlcmF0b3IuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/sorters.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL3NvcnRlcnMuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/webservice/webservice.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3dlYnNlcnZpY2Uvd2Vic2VydmljZS5nbw==) | `32.25% <ø> (+19.35%)` | :arrow_up: |
| [pkg/scheduler/context.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9jb250ZXh0Lmdv) | `16.55% <17.82%> (+10.28%)` | :arrow_up: |
| [pkg/metrics/metrics\_collector.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvbWV0cmljc19jb2xsZWN0b3IuZ28=) | `64.51% <45.00%> (-9.40%)` | :arrow_down: |
| [pkg/metrics/scheduler.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3Mvc2NoZWR1bGVyLmdv) | `60.89% <54.65%> (-0.69%)` | :arrow_down: |
| ... and [35 more](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [66eaecc...e37b1f0](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] kingamarton merged pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
kingamarton merged pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] codecov[bot] edited a comment on pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#issuecomment-966414547
# [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#336](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (cafdb27) into [master](https://codecov.io/gh/apache/incubator-yunikorn-core/commit/5a1c19e280f66d6cb4fc206a675116260d6f2ff1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5a1c19e) will **increase** coverage by `4.32%`.
> The diff coverage is `69.06%`.
> :exclamation: Current head cafdb27 differs from pull request most recent head b3d4f85. Consider uploading reports for the commit b3d4f85 to get more accurate results
[](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #336 +/- ##
==========================================
+ Coverage 63.46% 67.79% +4.32%
==========================================
Files 60 64 +4
Lines 5220 6198 +978
==========================================
+ Hits 3313 4202 +889
- Misses 1747 1773 +26
- Partials 160 223 +63
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [pkg/metrics/init.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvaW5pdC5nbw==) | `62.96% <ø> (ø)` | |
| [pkg/scheduler/drf\_preemption\_policy.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9kcmZfcHJlZW1wdGlvbl9wb2xpY3kuZ28=) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/nodes\_usage\_monitor.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9ub2Rlc191c2FnZV9tb25pdG9yLmdv) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/objects/allocation.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL2FsbG9jYXRpb24uZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/node\_iterator.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL25vZGVfaXRlcmF0b3IuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/sorters.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL3NvcnRlcnMuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/webservice/webservice.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3dlYnNlcnZpY2Uvd2Vic2VydmljZS5nbw==) | `32.25% <ø> (+19.35%)` | :arrow_up: |
| [pkg/scheduler/context.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9jb250ZXh0Lmdv) | `16.55% <17.82%> (+10.28%)` | :arrow_up: |
| [pkg/metrics/metrics\_collector.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvbWV0cmljc19jb2xsZWN0b3IuZ28=) | `64.51% <45.00%> (-9.40%)` | :arrow_down: |
| [pkg/metrics/scheduler.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3Mvc2NoZWR1bGVyLmdv) | `60.89% <54.65%> (-0.69%)` | :arrow_down: |
| ... and [35 more](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [66eaecc...b3d4f85](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] pbacsko commented on a change in pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
pbacsko commented on a change in pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#discussion_r748286930
##########
File path: pkg/webservice/routes.go
##########
@@ -170,6 +170,24 @@ var webRoutes = routes{
"/ws/v1/partition/{partition}/queue/{queue}/applications",
getQueueApplications,
},
+ route{
+ "Scheduler",
+ "GET",
+ "/ws/v1/fullstatedump",
+ getFullStateDump,
+ },
+ route{
+ "Scheduler",
+ "PUT",
+ "/ws/v1/enableperiodicstatedump/{period}",
+ enablePeriodicStateDump,
+ },
+ route{
+ "Scheduler",
+ "PUT",
+ "/ws/v1/disableperiodicstatedump",
+ disablePeriodicStateDump,
+ },
Review comment:
Any ideas? Like `"/ws/v1/statedump/{switch}/{period}"`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] codecov[bot] edited a comment on pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#issuecomment-966414547
# [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#336](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e9496d2) into [master](https://codecov.io/gh/apache/incubator-yunikorn-core/commit/5a1c19e280f66d6cb4fc206a675116260d6f2ff1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5a1c19e) will **increase** coverage by `4.32%`.
> The diff coverage is `69.06%`.
> :exclamation: Current head e9496d2 differs from pull request most recent head ff1d4e3. Consider uploading reports for the commit ff1d4e3 to get more accurate results
[](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #336 +/- ##
==========================================
+ Coverage 63.46% 67.79% +4.32%
==========================================
Files 60 64 +4
Lines 5220 6198 +978
==========================================
+ Hits 3313 4202 +889
- Misses 1747 1773 +26
- Partials 160 223 +63
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [pkg/metrics/init.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvaW5pdC5nbw==) | `62.96% <ø> (ø)` | |
| [pkg/scheduler/drf\_preemption\_policy.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9kcmZfcHJlZW1wdGlvbl9wb2xpY3kuZ28=) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/nodes\_usage\_monitor.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9ub2Rlc191c2FnZV9tb25pdG9yLmdv) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/objects/allocation.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL2FsbG9jYXRpb24uZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/node\_iterator.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL25vZGVfaXRlcmF0b3IuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/sorters.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL3NvcnRlcnMuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/webservice/webservice.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3dlYnNlcnZpY2Uvd2Vic2VydmljZS5nbw==) | `32.25% <ø> (+19.35%)` | :arrow_up: |
| [pkg/scheduler/context.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9jb250ZXh0Lmdv) | `16.55% <17.82%> (+10.28%)` | :arrow_up: |
| [pkg/metrics/metrics\_collector.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvbWV0cmljc19jb2xsZWN0b3IuZ28=) | `64.51% <45.00%> (-9.40%)` | :arrow_down: |
| [pkg/metrics/scheduler.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3Mvc2NoZWR1bGVyLmdv) | `60.89% <54.65%> (-0.69%)` | :arrow_down: |
| ... and [35 more](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [66eaecc...ff1d4e3](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] codecov[bot] edited a comment on pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#issuecomment-966414547
# [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#336](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e9496d2) into [master](https://codecov.io/gh/apache/incubator-yunikorn-core/commit/5a1c19e280f66d6cb4fc206a675116260d6f2ff1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5a1c19e) will **increase** coverage by `4.32%`.
> The diff coverage is `69.06%`.
> :exclamation: Current head e9496d2 differs from pull request most recent head e37b1f0. Consider uploading reports for the commit e37b1f0 to get more accurate results
[](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #336 +/- ##
==========================================
+ Coverage 63.46% 67.79% +4.32%
==========================================
Files 60 64 +4
Lines 5220 6198 +978
==========================================
+ Hits 3313 4202 +889
- Misses 1747 1773 +26
- Partials 160 223 +63
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [pkg/metrics/init.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvaW5pdC5nbw==) | `62.96% <ø> (ø)` | |
| [pkg/scheduler/drf\_preemption\_policy.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9kcmZfcHJlZW1wdGlvbl9wb2xpY3kuZ28=) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/nodes\_usage\_monitor.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9ub2Rlc191c2FnZV9tb25pdG9yLmdv) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/objects/allocation.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL2FsbG9jYXRpb24uZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/node\_iterator.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL25vZGVfaXRlcmF0b3IuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/sorters.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL3NvcnRlcnMuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/webservice/webservice.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3dlYnNlcnZpY2Uvd2Vic2VydmljZS5nbw==) | `32.25% <ø> (+19.35%)` | :arrow_up: |
| [pkg/scheduler/context.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9jb250ZXh0Lmdv) | `16.55% <17.82%> (+10.28%)` | :arrow_up: |
| [pkg/metrics/metrics\_collector.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvbWV0cmljc19jb2xsZWN0b3IuZ28=) | `64.51% <45.00%> (-9.40%)` | :arrow_down: |
| [pkg/metrics/scheduler.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3Mvc2NoZWR1bGVyLmdv) | `60.89% <54.65%> (-0.69%)` | :arrow_down: |
| ... and [35 more](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [66eaecc...e37b1f0](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] codecov[bot] edited a comment on pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#issuecomment-966414547
# [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#336](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e37b1f0) into [master](https://codecov.io/gh/apache/incubator-yunikorn-core/commit/5a1c19e280f66d6cb4fc206a675116260d6f2ff1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5a1c19e) will **increase** coverage by `4.40%`.
> The diff coverage is `69.33%`.
> :exclamation: Current head e37b1f0 differs from pull request most recent head 1508415. Consider uploading reports for the commit 1508415 to get more accurate results
[](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #336 +/- ##
==========================================
+ Coverage 63.46% 67.86% +4.40%
==========================================
Files 60 64 +4
Lines 5220 6212 +992
==========================================
+ Hits 3313 4216 +903
- Misses 1747 1773 +26
- Partials 160 223 +63
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [pkg/metrics/init.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvaW5pdC5nbw==) | `62.96% <ø> (ø)` | |
| [pkg/scheduler/drf\_preemption\_policy.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9kcmZfcHJlZW1wdGlvbl9wb2xpY3kuZ28=) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/nodes\_usage\_monitor.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9ub2Rlc191c2FnZV9tb25pdG9yLmdv) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/objects/allocation.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL2FsbG9jYXRpb24uZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/node\_iterator.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL25vZGVfaXRlcmF0b3IuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/sorters.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL3NvcnRlcnMuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/webservice/webservice.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3dlYnNlcnZpY2Uvd2Vic2VydmljZS5nbw==) | `32.25% <ø> (+19.35%)` | :arrow_up: |
| [pkg/scheduler/context.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9jb250ZXh0Lmdv) | `16.55% <17.82%> (+10.28%)` | :arrow_up: |
| [pkg/metrics/metrics\_collector.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvbWV0cmljc19jb2xsZWN0b3IuZ28=) | `64.51% <45.00%> (-9.40%)` | :arrow_down: |
| [pkg/metrics/scheduler.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3Mvc2NoZWR1bGVyLmdv) | `60.89% <54.65%> (-0.69%)` | :arrow_down: |
| ... and [35 more](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [66eaecc...1508415](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] kingamarton commented on a change in pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
kingamarton commented on a change in pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#discussion_r748076690
##########
File path: pkg/webservice/handlers.go
##########
@@ -150,20 +141,7 @@ func getNodesInfo(w http.ResponseWriter, r *http.Request) {
writeHeaders(w)
lists := schedulerContext.GetPartitionMapClone()
- result := make([]*dao.NodesDAOInfo, 0, len(lists))
- for _, partition := range lists {
- ns := partition.GetNodes()
- nodesDao := make([]*dao.NodeDAOInfo, 0, len(ns))
- for _, node := range ns {
- nodeDao := getNodeJSON(node)
- nodesDao = append(nodesDao, nodeDao)
- }
- result = append(result, &dao.NodesDAOInfo{
- PartitionName: common.GetPartitionNameWithoutClusterID(partition.Name),
- Nodes: nodesDao,
- })
- }
-
+ result := getNodesDao(lists)
Review comment:
nit: please try to be consistent in the naming. Use everywhere DAO, or Dao, but use it in the same way (getClustersUtilDAOInfo, vs getNodesDao)
##########
File path: pkg/webservice/handlers.go
##########
@@ -744,3 +665,151 @@ func getLogLevel(w http.ResponseWriter, r *http.Request) {
buildJSONErrorResponse(w, err.Error(), http.StatusInternalServerError)
}
}
+
+func getPartitionInfoDao(lists map[string]*scheduler.PartitionContext) []*dao.PartitionInfo {
+ result := make([]*dao.PartitionInfo, 0, len(lists))
+
+ for _, partitionContext := range lists {
+ partitionInfo := &dao.PartitionInfo{}
+ partitionInfo.ClusterID = partitionContext.RmID
+ partitionInfo.Name = common.GetPartitionNameWithoutClusterID(partitionContext.Name)
+ partitionInfo.State = partitionContext.GetCurrentState()
+ partitionInfo.LastStateTransitionTime = partitionContext.GetStateTime().String()
+
+ capacityInfo := dao.PartitionCapacity{}
+ capacityInfo.Capacity = partitionContext.GetTotalPartitionResource().DAOString()
+ capacityInfo.UsedCapacity = partitionContext.GetAllocatedResource().DAOString()
+ partitionInfo.Capacity = capacityInfo
+ partitionInfo.NodeSortingPolicy = dao.NodeSortingPolicy{
+ Type: partitionContext.GetNodeSortingPolicyType().String(),
+ ResourceWeights: partitionContext.GetNodeSortingResourceWeights(),
+ }
+
+ appList := partitionContext.GetApplications()
+ appList = append(appList, partitionContext.GetCompletedApplications()...)
+ applicationsState := make(map[string]int)
+ totalApplications := 0
+ for _, app := range appList {
+ applicationsState[app.CurrentState()]++
+ totalApplications++
+ }
+ applicationsState["total"] = totalApplications
+ partitionInfo.Applications = applicationsState
+ result = append(result, partitionInfo)
+ }
+
+ return result
+}
+
+func getAppHistoryDao(records []*history.MetricsRecord) []*dao.ApplicationHistoryDAOInfo {
+ result := make([]*dao.ApplicationHistoryDAOInfo, 0, len(records))
+ for _, record := range records {
+ if record == nil {
+ continue
+ }
+ element := &dao.ApplicationHistoryDAOInfo{
+ Timestamp: record.Timestamp.UnixNano(),
+ TotalApplications: strconv.Itoa(record.TotalApplications),
+ }
+ result = append(result, element)
+ }
+
+ return result
+}
+
+func getNodesDao(lists map[string]*scheduler.PartitionContext) []*dao.NodesDAOInfo {
+ result := make([]*dao.NodesDAOInfo, 0, len(lists))
+ for _, partition := range lists {
+ ns := partition.GetNodes()
+ nodesDao := make([]*dao.NodeDAOInfo, 0, len(ns))
+ for _, node := range ns {
+ nodeDao := getNodeJSON(node)
+ nodesDao = append(nodesDao, nodeDao)
+ }
+ result = append(result, &dao.NodesDAOInfo{
+ PartitionName: common.GetPartitionNameWithoutClusterID(partition.Name),
+ Nodes: nodesDao,
+ })
+ }
+
+ return result
+}
+
+func getContainerHistoryDao(records []*history.MetricsRecord) []*dao.ContainerHistoryDAOInfo {
+ result := make([]*dao.ContainerHistoryDAOInfo, 0, len(records))
+ for _, record := range records {
+ if record == nil {
+ continue
+ }
+ element := &dao.ContainerHistoryDAOInfo{
+ Timestamp: record.Timestamp.UnixNano(),
+ TotalContainers: strconv.Itoa(record.TotalContainers),
+ }
+ result = append(result, element)
+ }
+
+ return result
+}
+
+func getClustersUtilDAOInfo(lists map[string]*scheduler.PartitionContext) []*dao.ClustersUtilDAOInfo {
+ result := make([]*dao.ClustersUtilDAOInfo, 0, len(lists))
+ for _, partition := range lists {
+ result = append(result, &dao.ClustersUtilDAOInfo{
+ PartitionName: common.GetPartitionNameWithoutClusterID(partition.Name),
+ ClustersUtil: getClusterUtilJSON(partition),
+ })
+ }
+
+ return result
+}
+func getNodesUtilizationDao(lists map[string]*scheduler.PartitionContext) []*dao.NodesUtilDAOInfo {
+ var result []*dao.NodesUtilDAOInfo
+
+ for _, partition := range lists {
+ partitionResource := partition.GetTotalPartitionResource()
+ // partitionResource can be null if the partition has no node
+ if partitionResource != nil {
+ for name := range partitionResource.Resources {
+ result = append(result, getNodesUtilJSON(partition, name))
+ }
+ }
+ }
+
+ return result
+}
+
+func getApplicationsDao(lists map[string]*scheduler.PartitionContext) []*dao.ApplicationDAOInfo {
+ result := make([]*dao.ApplicationDAOInfo, 0, 32)
+
+ for _, partition := range lists {
+ size := partition.GetTotalCompletedApplicationCount() + partition.GetTotalApplicationCount()
+ appList := make([]*objects.Application, size)
+ appList = append(appList, partition.GetApplications()...)
+ appList = append(appList, partition.GetCompletedApplications()...)
+
+ for _, app := range appList {
+ result = append(result, getApplicationJSON(app))
+ }
+ }
+
+ return result
+}
+
+func getPartitionDAOInfo(lists map[string]*scheduler.PartitionContext) []*dao.PartitionDAOInfo {
+ queues := make([]*dao.PartitionDAOInfo, len(lists))
Review comment:
please rename the variable here, since it is storing partitiondata, not queues.
##########
File path: pkg/webservice/handlers_test.go
##########
@@ -1279,3 +1282,159 @@ func TestGetNodesUtilization(t *testing.T) {
assert.NilError(t, err)
assert.Equal(t, len(nodesDao), 0)
}
+
+func TestGetFullStateDump(t *testing.T) {
+ schedulerContext = prepareSchedulerContext(t)
+
+ imHistory = history.NewInternalMetricsHistory(5)
+ req, err := http.NewRequest("GET", "/ws/v1/getfullstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ getFullStateDump(resp, req)
+ receivedBytes := resp.outputBytes
+ statusCode := resp.statusCode
+ assert.Assert(t, len(receivedBytes) > 0, "json response is empty")
+ assert.Check(t, statusCode != http.StatusInternalServerError, "response status code")
+ var aggregated AggregatedStateInfo
+ if err := json.Unmarshal(receivedBytes, &aggregated); err != nil {
+ t.Fatal(err)
+ }
+ verifyStateDumpJSON(t, &aggregated)
+}
+
+func TestEnableDisablePeriodicStateDump(t *testing.T) {
+ schedulerContext = prepareSchedulerContext(t)
+ defer deleteStateDumpFile()
+ defer terminateGoroutine()
+
+ imHistory = history.NewInternalMetricsHistory(5)
+ req, err := http.NewRequest("GET", "/ws/v1/enableperiodicstatedump", strings.NewReader(""))
+ vars := map[string]string{
+ "period": "3",
+ }
+ req = mux.SetURLVars(req, vars)
+
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ // enable state dump, check file contents
+ enablePeriodicStateDump(resp, req)
+ statusCode := resp.statusCode
+ assert.Check(t, statusCode != http.StatusInternalServerError, "response status code")
+
+ waitForStateDumpFile(t)
+ fileContents, err2 := ioutil.ReadFile(stateDumpFilePath)
+ if err2 != nil {
+ t.Fatal(err2)
+ }
Review comment:
Please use assert.NilError
##########
File path: pkg/webservice/handlers_test.go
##########
@@ -1279,3 +1282,159 @@ func TestGetNodesUtilization(t *testing.T) {
assert.NilError(t, err)
assert.Equal(t, len(nodesDao), 0)
}
+
+func TestGetFullStateDump(t *testing.T) {
+ schedulerContext = prepareSchedulerContext(t)
+
+ imHistory = history.NewInternalMetricsHistory(5)
+ req, err := http.NewRequest("GET", "/ws/v1/getfullstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ getFullStateDump(resp, req)
+ receivedBytes := resp.outputBytes
+ statusCode := resp.statusCode
+ assert.Assert(t, len(receivedBytes) > 0, "json response is empty")
+ assert.Check(t, statusCode != http.StatusInternalServerError, "response status code")
+ var aggregated AggregatedStateInfo
+ if err := json.Unmarshal(receivedBytes, &aggregated); err != nil {
+ t.Fatal(err)
+ }
+ verifyStateDumpJSON(t, &aggregated)
+}
+
+func TestEnableDisablePeriodicStateDump(t *testing.T) {
+ schedulerContext = prepareSchedulerContext(t)
+ defer deleteStateDumpFile()
+ defer terminateGoroutine()
+
+ imHistory = history.NewInternalMetricsHistory(5)
+ req, err := http.NewRequest("GET", "/ws/v1/enableperiodicstatedump", strings.NewReader(""))
+ vars := map[string]string{
+ "period": "3",
+ }
+ req = mux.SetURLVars(req, vars)
+
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ // enable state dump, check file contents
+ enablePeriodicStateDump(resp, req)
+ statusCode := resp.statusCode
+ assert.Check(t, statusCode != http.StatusInternalServerError, "response status code")
+
+ waitForStateDumpFile(t)
+ fileContents, err2 := ioutil.ReadFile(stateDumpFilePath)
+ if err2 != nil {
+ t.Fatal(err2)
+ }
+ var aggregated AggregatedStateInfo
+ if err3 := json.Unmarshal(fileContents, &aggregated); err3 != nil {
+ t.Fatal(err3)
+ }
+ verifyStateDumpJSON(t, &aggregated)
+
+ // disable
+ req, err = http.NewRequest("GET", "/ws/v1/disableperiodicstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp = &MockResponseWriter{}
+ disablePeriodicStateDump(resp, req)
+ statusCode = resp.statusCode
+ assert.Assert(t, statusCode != http.StatusInternalServerError, "response status code")
+}
+
+func TestTryEnableStateDumpTwice(t *testing.T) {
+ defer deleteStateDumpFile()
+ defer terminateGoroutine()
+
+ req, err := http.NewRequest("GET", "/ws/v1/enableperiodicstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ enablePeriodicStateDump(resp, req)
+ enablePeriodicStateDump(resp, req)
+
+ statusCode := resp.statusCode
+ assert.Assert(t, statusCode == http.StatusInternalServerError, "response status code")
+}
+
+func TestTryDisableNotRunningStateDump(t *testing.T) {
+ req, err := http.NewRequest("GET", "/ws/v1/disableperiodicstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ disablePeriodicStateDump(resp, req)
+
+ statusCode := resp.statusCode
+ assert.Assert(t, statusCode == http.StatusInternalServerError, "response status code")
+}
+
+func prepareSchedulerContext(t *testing.T) *scheduler.ClusterContext {
+ configs.MockSchedulerConfigByData([]byte(configDefault))
+ var err error
+ schedulerContext, err = scheduler.NewClusterContext(rmID, policyGroup)
+ assert.NilError(t, err, "Error when load clusterInfo from config")
+ assert.Equal(t, 1, len(schedulerContext.GetPartitionMapClone()))
+
+ return schedulerContext
+}
+
+func waitForStateDumpFile(t *testing.T) {
+ for {
+ var attempts int
+
+ info, err := os.Stat(stateDumpFilePath)
+
+ // tolerate only "file not found" errors
+ if err != nil && !os.IsNotExist(err) {
+ t.Fatal(err)
+ }
+
+ if info != nil && info.Size() > 0 {
+ break
+ }
+
+ if attempts++; attempts > 10 {
+ t.Fatal("state dump file has not been created")
+ }
+ time.Sleep(1 * time.Second)
+ }
+}
+
+func deleteStateDumpFile() {
+ _ = os.Remove(stateDumpFilePath)
Review comment:
I think we should keep and return the error here instead of suppressing it. Then, it can be checked if no error was thrown
##########
File path: pkg/webservice/handlers_test.go
##########
@@ -1279,3 +1282,159 @@ func TestGetNodesUtilization(t *testing.T) {
assert.NilError(t, err)
assert.Equal(t, len(nodesDao), 0)
}
+
+func TestGetFullStateDump(t *testing.T) {
+ schedulerContext = prepareSchedulerContext(t)
+
+ imHistory = history.NewInternalMetricsHistory(5)
+ req, err := http.NewRequest("GET", "/ws/v1/getfullstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ getFullStateDump(resp, req)
+ receivedBytes := resp.outputBytes
+ statusCode := resp.statusCode
+ assert.Assert(t, len(receivedBytes) > 0, "json response is empty")
+ assert.Check(t, statusCode != http.StatusInternalServerError, "response status code")
+ var aggregated AggregatedStateInfo
+ if err := json.Unmarshal(receivedBytes, &aggregated); err != nil {
+ t.Fatal(err)
+ }
+ verifyStateDumpJSON(t, &aggregated)
+}
+
+func TestEnableDisablePeriodicStateDump(t *testing.T) {
+ schedulerContext = prepareSchedulerContext(t)
+ defer deleteStateDumpFile()
+ defer terminateGoroutine()
+
+ imHistory = history.NewInternalMetricsHistory(5)
+ req, err := http.NewRequest("GET", "/ws/v1/enableperiodicstatedump", strings.NewReader(""))
+ vars := map[string]string{
+ "period": "3",
+ }
+ req = mux.SetURLVars(req, vars)
+
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ // enable state dump, check file contents
+ enablePeriodicStateDump(resp, req)
+ statusCode := resp.statusCode
+ assert.Check(t, statusCode != http.StatusInternalServerError, "response status code")
+
+ waitForStateDumpFile(t)
+ fileContents, err2 := ioutil.ReadFile(stateDumpFilePath)
+ if err2 != nil {
+ t.Fatal(err2)
+ }
+ var aggregated AggregatedStateInfo
+ if err3 := json.Unmarshal(fileContents, &aggregated); err3 != nil {
+ t.Fatal(err3)
+ }
+ verifyStateDumpJSON(t, &aggregated)
+
+ // disable
+ req, err = http.NewRequest("GET", "/ws/v1/disableperiodicstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp = &MockResponseWriter{}
+ disablePeriodicStateDump(resp, req)
+ statusCode = resp.statusCode
+ assert.Assert(t, statusCode != http.StatusInternalServerError, "response status code")
+}
+
+func TestTryEnableStateDumpTwice(t *testing.T) {
+ defer deleteStateDumpFile()
+ defer terminateGoroutine()
+
+ req, err := http.NewRequest("GET", "/ws/v1/enableperiodicstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ enablePeriodicStateDump(resp, req)
+ enablePeriodicStateDump(resp, req)
+
+ statusCode := resp.statusCode
+ assert.Assert(t, statusCode == http.StatusInternalServerError, "response status code")
Review comment:
IN this case you can use assert.Equal as well, maybe this one the more appropiate in this case.
##########
File path: pkg/webservice/handlers_test.go
##########
@@ -1279,3 +1282,159 @@ func TestGetNodesUtilization(t *testing.T) {
assert.NilError(t, err)
assert.Equal(t, len(nodesDao), 0)
}
+
+func TestGetFullStateDump(t *testing.T) {
+ schedulerContext = prepareSchedulerContext(t)
+
+ imHistory = history.NewInternalMetricsHistory(5)
+ req, err := http.NewRequest("GET", "/ws/v1/getfullstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ getFullStateDump(resp, req)
+ receivedBytes := resp.outputBytes
+ statusCode := resp.statusCode
+ assert.Assert(t, len(receivedBytes) > 0, "json response is empty")
+ assert.Check(t, statusCode != http.StatusInternalServerError, "response status code")
+ var aggregated AggregatedStateInfo
+ if err := json.Unmarshal(receivedBytes, &aggregated); err != nil {
+ t.Fatal(err)
+ }
+ verifyStateDumpJSON(t, &aggregated)
+}
+
+func TestEnableDisablePeriodicStateDump(t *testing.T) {
+ schedulerContext = prepareSchedulerContext(t)
+ defer deleteStateDumpFile()
+ defer terminateGoroutine()
+
+ imHistory = history.NewInternalMetricsHistory(5)
+ req, err := http.NewRequest("GET", "/ws/v1/enableperiodicstatedump", strings.NewReader(""))
+ vars := map[string]string{
+ "period": "3",
+ }
+ req = mux.SetURLVars(req, vars)
+
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ // enable state dump, check file contents
+ enablePeriodicStateDump(resp, req)
+ statusCode := resp.statusCode
+ assert.Check(t, statusCode != http.StatusInternalServerError, "response status code")
+
+ waitForStateDumpFile(t)
+ fileContents, err2 := ioutil.ReadFile(stateDumpFilePath)
+ if err2 != nil {
+ t.Fatal(err2)
+ }
+ var aggregated AggregatedStateInfo
+ if err3 := json.Unmarshal(fileContents, &aggregated); err3 != nil {
+ t.Fatal(err3)
+ }
+ verifyStateDumpJSON(t, &aggregated)
+
+ // disable
+ req, err = http.NewRequest("GET", "/ws/v1/disableperiodicstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp = &MockResponseWriter{}
+ disablePeriodicStateDump(resp, req)
+ statusCode = resp.statusCode
+ assert.Assert(t, statusCode != http.StatusInternalServerError, "response status code")
+}
+
+func TestTryEnableStateDumpTwice(t *testing.T) {
+ defer deleteStateDumpFile()
+ defer terminateGoroutine()
+
+ req, err := http.NewRequest("GET", "/ws/v1/enableperiodicstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ enablePeriodicStateDump(resp, req)
+ enablePeriodicStateDump(resp, req)
+
+ statusCode := resp.statusCode
+ assert.Assert(t, statusCode == http.StatusInternalServerError, "response status code")
+}
+
+func TestTryDisableNotRunningStateDump(t *testing.T) {
+ req, err := http.NewRequest("GET", "/ws/v1/disableperiodicstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ disablePeriodicStateDump(resp, req)
+
+ statusCode := resp.statusCode
+ assert.Assert(t, statusCode == http.StatusInternalServerError, "response status code")
Review comment:
nit: maybe assert.Equals would be better
##########
File path: pkg/webservice/handlers_test.go
##########
@@ -1279,3 +1282,159 @@ func TestGetNodesUtilization(t *testing.T) {
assert.NilError(t, err)
assert.Equal(t, len(nodesDao), 0)
}
+
+func TestGetFullStateDump(t *testing.T) {
+ schedulerContext = prepareSchedulerContext(t)
+
+ imHistory = history.NewInternalMetricsHistory(5)
+ req, err := http.NewRequest("GET", "/ws/v1/getfullstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ getFullStateDump(resp, req)
+ receivedBytes := resp.outputBytes
+ statusCode := resp.statusCode
+ assert.Assert(t, len(receivedBytes) > 0, "json response is empty")
+ assert.Check(t, statusCode != http.StatusInternalServerError, "response status code")
+ var aggregated AggregatedStateInfo
+ if err := json.Unmarshal(receivedBytes, &aggregated); err != nil {
+ t.Fatal(err)
+ }
+ verifyStateDumpJSON(t, &aggregated)
+}
+
+func TestEnableDisablePeriodicStateDump(t *testing.T) {
+ schedulerContext = prepareSchedulerContext(t)
+ defer deleteStateDumpFile()
+ defer terminateGoroutine()
+
+ imHistory = history.NewInternalMetricsHistory(5)
+ req, err := http.NewRequest("GET", "/ws/v1/enableperiodicstatedump", strings.NewReader(""))
+ vars := map[string]string{
+ "period": "3",
+ }
+ req = mux.SetURLVars(req, vars)
+
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ // enable state dump, check file contents
+ enablePeriodicStateDump(resp, req)
+ statusCode := resp.statusCode
+ assert.Check(t, statusCode != http.StatusInternalServerError, "response status code")
Review comment:
How this assert.Check works? Why is it better to use this one hete than assert.Assert?
##########
File path: pkg/webservice/handlers_test.go
##########
@@ -1279,3 +1282,159 @@ func TestGetNodesUtilization(t *testing.T) {
assert.NilError(t, err)
assert.Equal(t, len(nodesDao), 0)
}
+
+func TestGetFullStateDump(t *testing.T) {
+ schedulerContext = prepareSchedulerContext(t)
+
+ imHistory = history.NewInternalMetricsHistory(5)
+ req, err := http.NewRequest("GET", "/ws/v1/getfullstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ getFullStateDump(resp, req)
+ receivedBytes := resp.outputBytes
+ statusCode := resp.statusCode
+ assert.Assert(t, len(receivedBytes) > 0, "json response is empty")
+ assert.Check(t, statusCode != http.StatusInternalServerError, "response status code")
+ var aggregated AggregatedStateInfo
+ if err := json.Unmarshal(receivedBytes, &aggregated); err != nil {
+ t.Fatal(err)
+ }
+ verifyStateDumpJSON(t, &aggregated)
+}
+
+func TestEnableDisablePeriodicStateDump(t *testing.T) {
+ schedulerContext = prepareSchedulerContext(t)
+ defer deleteStateDumpFile()
+ defer terminateGoroutine()
+
+ imHistory = history.NewInternalMetricsHistory(5)
Review comment:
Do we need this variable here? It is not used in this test case.
##########
File path: pkg/webservice/handlers_test.go
##########
@@ -1279,3 +1282,159 @@ func TestGetNodesUtilization(t *testing.T) {
assert.NilError(t, err)
assert.Equal(t, len(nodesDao), 0)
}
+
+func TestGetFullStateDump(t *testing.T) {
+ schedulerContext = prepareSchedulerContext(t)
+
+ imHistory = history.NewInternalMetricsHistory(5)
+ req, err := http.NewRequest("GET", "/ws/v1/getfullstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ getFullStateDump(resp, req)
+ receivedBytes := resp.outputBytes
+ statusCode := resp.statusCode
+ assert.Assert(t, len(receivedBytes) > 0, "json response is empty")
+ assert.Check(t, statusCode != http.StatusInternalServerError, "response status code")
+ var aggregated AggregatedStateInfo
+ if err := json.Unmarshal(receivedBytes, &aggregated); err != nil {
+ t.Fatal(err)
+ }
+ verifyStateDumpJSON(t, &aggregated)
+}
+
+func TestEnableDisablePeriodicStateDump(t *testing.T) {
+ schedulerContext = prepareSchedulerContext(t)
+ defer deleteStateDumpFile()
+ defer terminateGoroutine()
+
+ imHistory = history.NewInternalMetricsHistory(5)
+ req, err := http.NewRequest("GET", "/ws/v1/enableperiodicstatedump", strings.NewReader(""))
+ vars := map[string]string{
+ "period": "3",
+ }
+ req = mux.SetURLVars(req, vars)
+
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ // enable state dump, check file contents
+ enablePeriodicStateDump(resp, req)
+ statusCode := resp.statusCode
+ assert.Check(t, statusCode != http.StatusInternalServerError, "response status code")
+
+ waitForStateDumpFile(t)
+ fileContents, err2 := ioutil.ReadFile(stateDumpFilePath)
+ if err2 != nil {
+ t.Fatal(err2)
+ }
+ var aggregated AggregatedStateInfo
+ if err3 := json.Unmarshal(fileContents, &aggregated); err3 != nil {
+ t.Fatal(err3)
+ }
Review comment:
Please use assert.NilError
##########
File path: pkg/webservice/handlers_test.go
##########
@@ -1279,3 +1282,159 @@ func TestGetNodesUtilization(t *testing.T) {
assert.NilError(t, err)
assert.Equal(t, len(nodesDao), 0)
}
+
+func TestGetFullStateDump(t *testing.T) {
+ schedulerContext = prepareSchedulerContext(t)
+
+ imHistory = history.NewInternalMetricsHistory(5)
+ req, err := http.NewRequest("GET", "/ws/v1/getfullstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ getFullStateDump(resp, req)
+ receivedBytes := resp.outputBytes
+ statusCode := resp.statusCode
+ assert.Assert(t, len(receivedBytes) > 0, "json response is empty")
+ assert.Check(t, statusCode != http.StatusInternalServerError, "response status code")
+ var aggregated AggregatedStateInfo
+ if err := json.Unmarshal(receivedBytes, &aggregated); err != nil {
+ t.Fatal(err)
+ }
+ verifyStateDumpJSON(t, &aggregated)
+}
+
+func TestEnableDisablePeriodicStateDump(t *testing.T) {
+ schedulerContext = prepareSchedulerContext(t)
+ defer deleteStateDumpFile()
+ defer terminateGoroutine()
+
+ imHistory = history.NewInternalMetricsHistory(5)
+ req, err := http.NewRequest("GET", "/ws/v1/enableperiodicstatedump", strings.NewReader(""))
+ vars := map[string]string{
+ "period": "3",
+ }
+ req = mux.SetURLVars(req, vars)
+
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ // enable state dump, check file contents
+ enablePeriodicStateDump(resp, req)
+ statusCode := resp.statusCode
+ assert.Check(t, statusCode != http.StatusInternalServerError, "response status code")
+
+ waitForStateDumpFile(t)
+ fileContents, err2 := ioutil.ReadFile(stateDumpFilePath)
+ if err2 != nil {
+ t.Fatal(err2)
+ }
+ var aggregated AggregatedStateInfo
+ if err3 := json.Unmarshal(fileContents, &aggregated); err3 != nil {
+ t.Fatal(err3)
+ }
+ verifyStateDumpJSON(t, &aggregated)
+
+ // disable
+ req, err = http.NewRequest("GET", "/ws/v1/disableperiodicstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp = &MockResponseWriter{}
+ disablePeriodicStateDump(resp, req)
+ statusCode = resp.statusCode
+ assert.Assert(t, statusCode != http.StatusInternalServerError, "response status code")
+}
+
+func TestTryEnableStateDumpTwice(t *testing.T) {
+ defer deleteStateDumpFile()
+ defer terminateGoroutine()
+
+ req, err := http.NewRequest("GET", "/ws/v1/enableperiodicstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ enablePeriodicStateDump(resp, req)
Review comment:
You can add an assert here as well, to check if the first call is successfull
##########
File path: pkg/webservice/handlers_test.go
##########
@@ -1279,3 +1282,159 @@ func TestGetNodesUtilization(t *testing.T) {
assert.NilError(t, err)
assert.Equal(t, len(nodesDao), 0)
}
+
+func TestGetFullStateDump(t *testing.T) {
+ schedulerContext = prepareSchedulerContext(t)
+
+ imHistory = history.NewInternalMetricsHistory(5)
+ req, err := http.NewRequest("GET", "/ws/v1/getfullstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ getFullStateDump(resp, req)
+ receivedBytes := resp.outputBytes
+ statusCode := resp.statusCode
+ assert.Assert(t, len(receivedBytes) > 0, "json response is empty")
+ assert.Check(t, statusCode != http.StatusInternalServerError, "response status code")
+ var aggregated AggregatedStateInfo
+ if err := json.Unmarshal(receivedBytes, &aggregated); err != nil {
+ t.Fatal(err)
+ }
+ verifyStateDumpJSON(t, &aggregated)
+}
+
+func TestEnableDisablePeriodicStateDump(t *testing.T) {
+ schedulerContext = prepareSchedulerContext(t)
+ defer deleteStateDumpFile()
+ defer terminateGoroutine()
+
+ imHistory = history.NewInternalMetricsHistory(5)
+ req, err := http.NewRequest("GET", "/ws/v1/enableperiodicstatedump", strings.NewReader(""))
+ vars := map[string]string{
+ "period": "3",
+ }
+ req = mux.SetURLVars(req, vars)
+
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ // enable state dump, check file contents
+ enablePeriodicStateDump(resp, req)
+ statusCode := resp.statusCode
+ assert.Check(t, statusCode != http.StatusInternalServerError, "response status code")
+
+ waitForStateDumpFile(t)
+ fileContents, err2 := ioutil.ReadFile(stateDumpFilePath)
+ if err2 != nil {
+ t.Fatal(err2)
+ }
+ var aggregated AggregatedStateInfo
+ if err3 := json.Unmarshal(fileContents, &aggregated); err3 != nil {
+ t.Fatal(err3)
+ }
+ verifyStateDumpJSON(t, &aggregated)
+
+ // disable
+ req, err = http.NewRequest("GET", "/ws/v1/disableperiodicstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp = &MockResponseWriter{}
+ disablePeriodicStateDump(resp, req)
+ statusCode = resp.statusCode
+ assert.Assert(t, statusCode != http.StatusInternalServerError, "response status code")
Review comment:
instead of asserting to a code what should not be the status code, better check if the code is the expected one.
##########
File path: pkg/webservice/handlers_test.go
##########
@@ -1279,3 +1282,159 @@ func TestGetNodesUtilization(t *testing.T) {
assert.NilError(t, err)
assert.Equal(t, len(nodesDao), 0)
}
+
+func TestGetFullStateDump(t *testing.T) {
+ schedulerContext = prepareSchedulerContext(t)
+
+ imHistory = history.NewInternalMetricsHistory(5)
+ req, err := http.NewRequest("GET", "/ws/v1/getfullstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ getFullStateDump(resp, req)
+ receivedBytes := resp.outputBytes
+ statusCode := resp.statusCode
+ assert.Assert(t, len(receivedBytes) > 0, "json response is empty")
+ assert.Check(t, statusCode != http.StatusInternalServerError, "response status code")
+ var aggregated AggregatedStateInfo
+ if err := json.Unmarshal(receivedBytes, &aggregated); err != nil {
+ t.Fatal(err)
+ }
+ verifyStateDumpJSON(t, &aggregated)
+}
+
+func TestEnableDisablePeriodicStateDump(t *testing.T) {
+ schedulerContext = prepareSchedulerContext(t)
+ defer deleteStateDumpFile()
+ defer terminateGoroutine()
+
+ imHistory = history.NewInternalMetricsHistory(5)
+ req, err := http.NewRequest("GET", "/ws/v1/enableperiodicstatedump", strings.NewReader(""))
+ vars := map[string]string{
+ "period": "3",
+ }
+ req = mux.SetURLVars(req, vars)
+
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ // enable state dump, check file contents
+ enablePeriodicStateDump(resp, req)
+ statusCode := resp.statusCode
+ assert.Check(t, statusCode != http.StatusInternalServerError, "response status code")
+
+ waitForStateDumpFile(t)
+ fileContents, err2 := ioutil.ReadFile(stateDumpFilePath)
+ if err2 != nil {
+ t.Fatal(err2)
+ }
+ var aggregated AggregatedStateInfo
+ if err3 := json.Unmarshal(fileContents, &aggregated); err3 != nil {
+ t.Fatal(err3)
+ }
+ verifyStateDumpJSON(t, &aggregated)
+
+ // disable
+ req, err = http.NewRequest("GET", "/ws/v1/disableperiodicstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp = &MockResponseWriter{}
+ disablePeriodicStateDump(resp, req)
+ statusCode = resp.statusCode
+ assert.Assert(t, statusCode != http.StatusInternalServerError, "response status code")
+}
+
+func TestTryEnableStateDumpTwice(t *testing.T) {
+ defer deleteStateDumpFile()
+ defer terminateGoroutine()
+
+ req, err := http.NewRequest("GET", "/ws/v1/enableperiodicstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ enablePeriodicStateDump(resp, req)
+ enablePeriodicStateDump(resp, req)
+
+ statusCode := resp.statusCode
+ assert.Assert(t, statusCode == http.StatusInternalServerError, "response status code")
+}
+
+func TestTryDisableNotRunningStateDump(t *testing.T) {
+ req, err := http.NewRequest("GET", "/ws/v1/disableperiodicstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ disablePeriodicStateDump(resp, req)
+
+ statusCode := resp.statusCode
+ assert.Assert(t, statusCode == http.StatusInternalServerError, "response status code")
+}
+
+func prepareSchedulerContext(t *testing.T) *scheduler.ClusterContext {
+ configs.MockSchedulerConfigByData([]byte(configDefault))
+ var err error
+ schedulerContext, err = scheduler.NewClusterContext(rmID, policyGroup)
+ assert.NilError(t, err, "Error when load clusterInfo from config")
+ assert.Equal(t, 1, len(schedulerContext.GetPartitionMapClone()))
+
+ return schedulerContext
+}
+
+func waitForStateDumpFile(t *testing.T) {
+ for {
+ var attempts int
+
+ info, err := os.Stat(stateDumpFilePath)
+
+ // tolerate only "file not found" errors
+ if err != nil && !os.IsNotExist(err) {
Review comment:
Why do we need to handle this error case differently?
##########
File path: pkg/webservice/state_dump.go
##########
@@ -0,0 +1,207 @@
+/*
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+package webservice
+
+import (
+ "encoding/json"
+ "fmt"
+ "io"
+ "net/http"
+ "os"
+ "strconv"
+ "sync"
+ "time"
+
+ "github.com/apache/incubator-yunikorn-core/pkg/log"
+ metrics2 "github.com/apache/incubator-yunikorn-core/pkg/metrics"
+ "github.com/apache/incubator-yunikorn-core/pkg/scheduler"
+ "github.com/apache/incubator-yunikorn-core/pkg/webservice/dao"
+
+ "github.com/gorilla/mux"
+ "go.uber.org/zap"
+)
+
+const (
+ defaultStateDumpPeriod = 60
Review comment:
Creating the state dump by default every minute seems a little too aggressive for me. It will make huge files. I think we should print the dumps less freqvently. What is the default value in yarn for the state dump creeation?
##########
File path: pkg/webservice/state_dump.go
##########
@@ -0,0 +1,207 @@
+/*
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+package webservice
+
+import (
+ "encoding/json"
+ "fmt"
+ "io"
+ "net/http"
+ "os"
+ "strconv"
+ "sync"
+ "time"
+
+ "github.com/apache/incubator-yunikorn-core/pkg/log"
+ metrics2 "github.com/apache/incubator-yunikorn-core/pkg/metrics"
+ "github.com/apache/incubator-yunikorn-core/pkg/scheduler"
+ "github.com/apache/incubator-yunikorn-core/pkg/webservice/dao"
+
+ "github.com/gorilla/mux"
+ "go.uber.org/zap"
+)
+
+const (
+ defaultStateDumpPeriod = 60
Review comment:
From this line we cannot know what this 60 is? mins? seconds? Since this is a duration, please use Duration type instead of int, or if it is not possible please rename the variable to mirror the exact value.
##########
File path: pkg/webservice/state_dump.go
##########
@@ -0,0 +1,207 @@
+/*
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+package webservice
+
+import (
+ "encoding/json"
+ "fmt"
+ "io"
+ "net/http"
+ "os"
+ "strconv"
+ "sync"
+ "time"
+
+ "github.com/apache/incubator-yunikorn-core/pkg/log"
+ metrics2 "github.com/apache/incubator-yunikorn-core/pkg/metrics"
+ "github.com/apache/incubator-yunikorn-core/pkg/scheduler"
+ "github.com/apache/incubator-yunikorn-core/pkg/webservice/dao"
+
+ "github.com/gorilla/mux"
+ "go.uber.org/zap"
+)
+
+const (
+ defaultStateDumpPeriod = 60
+ stateDumpFilePath = "yunikorn-state.txt"
+)
+
+var (
+ periodicStateDump bool
+ abort chan struct{}
+ startStop sync.Mutex
+ stateDump sync.Mutex // guards against simultaneous periodic vs web request
+)
+
+type AggregatedStateInfo struct {
+ Timestamp string
+ Partitions []*dao.PartitionInfo
+ Applications []*dao.ApplicationDAOInfo
+ AppHistory []*dao.ApplicationHistoryDAOInfo
+ Nodes []*dao.NodesDAOInfo
+ NodesUtilization []*dao.NodesUtilDAOInfo
+ ClusterInfo []*dao.ClusterDAOInfo
+ ClusterUtilization []*dao.ClustersUtilDAOInfo
+ ContainerHistory []*dao.ContainerHistoryDAOInfo
+ Queues []*dao.PartitionDAOInfo
+ SchedulerHealth *dao.SchedulerHealthDAOInfo
Review comment:
I am not sure if the health status should be included or not. Now it is set as liveness probe, so when the scheduler becomes unhealthy, it is logged and also it will be restarted.
##########
File path: pkg/webservice/routes.go
##########
@@ -170,6 +170,24 @@ var webRoutes = routes{
"/ws/v1/partition/{partition}/queue/{queue}/applications",
getQueueApplications,
},
+ route{
+ "Scheduler",
+ "GET",
+ "/ws/v1/fullstatedump",
+ getFullStateDump,
+ },
+ route{
+ "Scheduler",
+ "PUT",
+ "/ws/v1/enableperiodicstatedump/{period}",
+ enablePeriodicStateDump,
+ },
+ route{
+ "Scheduler",
+ "PUT",
+ "/ws/v1/disableperiodicstatedump",
+ disablePeriodicStateDump,
+ },
Review comment:
Can we merge this 2 into one? I am thinking about using a parameter for enabling/disabling it, and one for the period.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] pbacsko commented on a change in pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
pbacsko commented on a change in pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#discussion_r750343566
##########
File path: pkg/webservice/state_dump.go
##########
@@ -0,0 +1,225 @@
+/*
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+package webservice
+
+import (
+ "encoding/json"
+ "fmt"
+ "io"
+ "net/http"
+ "os"
+ "strconv"
+ "strings"
+ "sync"
+ "time"
+
+ "github.com/apache/incubator-yunikorn-core/pkg/log"
+ "github.com/apache/incubator-yunikorn-core/pkg/webservice/dao"
+
+ "github.com/gorilla/mux"
+ "go.uber.org/zap"
+)
+
+const (
+ defaultStateDumpPeriodSeconds time.Duration = 60 //nolint
+ stateDumpFilePath string = "yunikorn-state.txt"
+)
+
+var (
+ periodicStateDump bool
+ abort chan struct{}
+ startStop sync.Mutex
+ stateDump sync.Mutex // guards against simultaneous periodic vs web request
+)
+
+type AggregatedStateInfo struct {
+ Timestamp string
+ Partitions []*dao.PartitionInfo
+ Applications []*dao.ApplicationDAOInfo
+ AppHistory []*dao.ApplicationHistoryDAOInfo
+ Nodes []*dao.NodesDAOInfo
+ NodesUtilization []*dao.NodesUtilDAOInfo
+ ClusterInfo []*dao.ClusterDAOInfo
+ ClusterUtilization []*dao.ClustersUtilDAOInfo
+ ContainerHistory []*dao.ContainerHistoryDAOInfo
+ Queues []*dao.PartitionDAOInfo
+ LogLevel string
+}
+
+func getFullStateDump(w http.ResponseWriter, r *http.Request) {
+ writeHeaders(w)
+ if err := doStateDump(w); err != nil {
+ buildJSONErrorResponse(w, err.Error(), http.StatusInternalServerError)
+ }
+}
+
+func handlePeriodicStateDump(w http.ResponseWriter, r *http.Request) {
+ vars := mux.Vars(r)
+ writeHeaders(w)
+ if len(vars["switch"]) == 0 {
+ buildJSONErrorResponse(w, "required parameter enabled/disabled is missing",
+ http.StatusBadRequest)
+ return
+ }
+
+ enabledSwitch := strings.ToLower(vars["switch"])
+ switch enabledSwitch {
+ case "enable":
+ enablePeriodicStateDump(w, r)
+ case "disable":
+ disablePeriodicStateDump(w, r)
+ default:
+ buildJSONErrorResponse(w, fmt.Sprintf("required parameter enable/disable is illegal: %s", enabledSwitch),
+ http.StatusBadRequest)
+ }
+}
+
+func enablePeriodicStateDump(w http.ResponseWriter, r *http.Request) {
+ vars := mux.Vars(r)
+ const defaultPeriod = int(defaultStateDumpPeriodSeconds)
+ var periodFromHeader int
+ var period time.Duration
+ var err error
+ var zapField = zap.Duration("defaultStateDumpPeriodSeconds", defaultStateDumpPeriodSeconds)
+
+ if len(vars["period"]) == 0 {
+ log.Logger().Info("using the default period for state dump",
+ zapField)
+ periodFromHeader = defaultPeriod
+ } else if periodFromHeader, err = strconv.Atoi(vars["period"]); err != nil {
+ log.Logger().Warn("illegal value for period, using the default",
+ zapField)
+ periodFromHeader = defaultPeriod
+ }
+
+ if periodFromHeader < 0 {
+ log.Logger().Warn("period value is negative, using the default",
+ zapField)
+ periodFromHeader = defaultPeriod
+ }
+
+ period = time.Duration(periodFromHeader)
+ if err = startBackGroundStateDump(period); err != nil {
+ buildJSONErrorResponse(w, err.Error(), http.StatusInternalServerError)
+ }
+}
+
+func disablePeriodicStateDump(w http.ResponseWriter, r *http.Request) {
+ if err := stopBackGroundStateDump(); err != nil {
+ buildJSONErrorResponse(w, err.Error(), http.StatusInternalServerError)
+ }
+}
+
+func doStateDump(w io.Writer) error {
+ stateDump.Lock()
+ defer stateDump.Unlock()
+
+ partitionContext := schedulerContext.GetPartitionMapClone()
+ records := imHistory.GetRecords()
+ zapConfig := log.GetConfig()
+
+ var aggregated = AggregatedStateInfo{
+ Timestamp: time.Now().Format(time.RFC3339),
+ Partitions: getPartitionInfoDAO(partitionContext),
+ Applications: getApplicationsDAO(partitionContext),
+ AppHistory: getAppHistoryDAO(records),
+ Nodes: getNodesDAO(partitionContext),
+ NodesUtilization: getNodesUtilDAO(partitionContext),
+ ClusterInfo: getClusterDAO(partitionContext),
+ ClusterUtilization: getClustersUtilDAO(partitionContext),
+ ContainerHistory: getContainerHistoryDAO(records),
+ Queues: getPartitionDAO(partitionContext),
+ LogLevel: zapConfig.Level.Level().String(),
+ }
+
+ var prettyJSON []byte
+ var err error
+ prettyJSON, err = json.MarshalIndent(aggregated, "", " ")
+ if err != nil {
+ return err
+ }
+ if _, err = w.Write(prettyJSON); err != nil {
+ return err
+ }
+
+ return nil
+}
+
+func startBackGroundStateDump(period time.Duration) error {
+ startStop.Lock()
+ defer startStop.Unlock()
+
+ if periodicStateDump {
+ var errMsg = "state dump already running"
+ log.Logger().Error(errMsg)
+ return fmt.Errorf(errMsg)
+ }
+
+ file, err := os.OpenFile(stateDumpFilePath, os.O_APPEND|os.O_CREATE|os.O_RDWR, 0666)
+ if err != nil {
+ log.Logger().Error("unable to open/create file",
+ zap.Error(err))
+ return err
+ }
+ abort = make(chan struct{})
+ periodicStateDump = true
+
+ go func() {
+ ticker := time.NewTicker(period * time.Second)
+
+ for {
+ select {
+ case <-abort:
+ ticker.Stop()
+ file.Close()
+ return
+ case <-ticker.C:
+ if err := doStateDump(file); err != nil {
+ log.Logger().Error("state dump failed", zap.Error(err))
+ if err := stopBackGroundStateDump(); err != nil {
Review comment:
This means a file I/O error, which is unlikely to be recoverable. I'd rather stop when that happens (that's most likely a full or failed disk).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] codecov[bot] edited a comment on pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#issuecomment-966414547
# [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#336](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e37b1f0) into [master](https://codecov.io/gh/apache/incubator-yunikorn-core/commit/5a1c19e280f66d6cb4fc206a675116260d6f2ff1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5a1c19e) will **increase** coverage by `4.40%`.
> The diff coverage is `69.33%`.
> :exclamation: Current head e37b1f0 differs from pull request most recent head bc63b04. Consider uploading reports for the commit bc63b04 to get more accurate results
[](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #336 +/- ##
==========================================
+ Coverage 63.46% 67.86% +4.40%
==========================================
Files 60 64 +4
Lines 5220 6212 +992
==========================================
+ Hits 3313 4216 +903
- Misses 1747 1773 +26
- Partials 160 223 +63
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [pkg/metrics/init.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvaW5pdC5nbw==) | `62.96% <ø> (ø)` | |
| [pkg/scheduler/drf\_preemption\_policy.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9kcmZfcHJlZW1wdGlvbl9wb2xpY3kuZ28=) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/nodes\_usage\_monitor.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9ub2Rlc191c2FnZV9tb25pdG9yLmdv) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/objects/allocation.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL2FsbG9jYXRpb24uZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/node\_iterator.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL25vZGVfaXRlcmF0b3IuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/sorters.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL3NvcnRlcnMuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/webservice/webservice.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3dlYnNlcnZpY2Uvd2Vic2VydmljZS5nbw==) | `32.25% <ø> (+19.35%)` | :arrow_up: |
| [pkg/scheduler/context.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9jb250ZXh0Lmdv) | `16.55% <17.82%> (+10.28%)` | :arrow_up: |
| [pkg/metrics/metrics\_collector.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvbWV0cmljc19jb2xsZWN0b3IuZ28=) | `64.51% <45.00%> (-9.40%)` | :arrow_down: |
| [pkg/metrics/scheduler.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3Mvc2NoZWR1bGVyLmdv) | `60.89% <54.65%> (-0.69%)` | :arrow_down: |
| ... and [35 more](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [66eaecc...bc63b04](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] pbacsko commented on a change in pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
pbacsko commented on a change in pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#discussion_r748270371
##########
File path: pkg/webservice/handlers_test.go
##########
@@ -1279,3 +1282,159 @@ func TestGetNodesUtilization(t *testing.T) {
assert.NilError(t, err)
assert.Equal(t, len(nodesDao), 0)
}
+
+func TestGetFullStateDump(t *testing.T) {
+ schedulerContext = prepareSchedulerContext(t)
+
+ imHistory = history.NewInternalMetricsHistory(5)
+ req, err := http.NewRequest("GET", "/ws/v1/getfullstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ getFullStateDump(resp, req)
+ receivedBytes := resp.outputBytes
+ statusCode := resp.statusCode
+ assert.Assert(t, len(receivedBytes) > 0, "json response is empty")
+ assert.Check(t, statusCode != http.StatusInternalServerError, "response status code")
+ var aggregated AggregatedStateInfo
+ if err := json.Unmarshal(receivedBytes, &aggregated); err != nil {
+ t.Fatal(err)
+ }
+ verifyStateDumpJSON(t, &aggregated)
+}
+
+func TestEnableDisablePeriodicStateDump(t *testing.T) {
+ schedulerContext = prepareSchedulerContext(t)
+ defer deleteStateDumpFile()
+ defer terminateGoroutine()
+
+ imHistory = history.NewInternalMetricsHistory(5)
+ req, err := http.NewRequest("GET", "/ws/v1/enableperiodicstatedump", strings.NewReader(""))
+ vars := map[string]string{
+ "period": "3",
+ }
+ req = mux.SetURLVars(req, vars)
+
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ // enable state dump, check file contents
+ enablePeriodicStateDump(resp, req)
+ statusCode := resp.statusCode
+ assert.Check(t, statusCode != http.StatusInternalServerError, "response status code")
Review comment:
It does not stop the execution of the test if it fails. But we can switch back to the normal assert.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] codecov[bot] edited a comment on pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#issuecomment-966414547
# [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#336](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e9496d2) into [master](https://codecov.io/gh/apache/incubator-yunikorn-core/commit/5a1c19e280f66d6cb4fc206a675116260d6f2ff1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5a1c19e) will **increase** coverage by `4.32%`.
> The diff coverage is `69.06%`.
> :exclamation: Current head e9496d2 differs from pull request most recent head 95d0e1c. Consider uploading reports for the commit 95d0e1c to get more accurate results
[](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #336 +/- ##
==========================================
+ Coverage 63.46% 67.79% +4.32%
==========================================
Files 60 64 +4
Lines 5220 6198 +978
==========================================
+ Hits 3313 4202 +889
- Misses 1747 1773 +26
- Partials 160 223 +63
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [pkg/metrics/init.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvaW5pdC5nbw==) | `62.96% <ø> (ø)` | |
| [pkg/scheduler/drf\_preemption\_policy.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9kcmZfcHJlZW1wdGlvbl9wb2xpY3kuZ28=) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/nodes\_usage\_monitor.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9ub2Rlc191c2FnZV9tb25pdG9yLmdv) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/objects/allocation.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL2FsbG9jYXRpb24uZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/node\_iterator.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL25vZGVfaXRlcmF0b3IuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/sorters.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL3NvcnRlcnMuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/webservice/webservice.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3dlYnNlcnZpY2Uvd2Vic2VydmljZS5nbw==) | `32.25% <ø> (+19.35%)` | :arrow_up: |
| [pkg/scheduler/context.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9jb250ZXh0Lmdv) | `16.55% <17.82%> (+10.28%)` | :arrow_up: |
| [pkg/metrics/metrics\_collector.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvbWV0cmljc19jb2xsZWN0b3IuZ28=) | `64.51% <45.00%> (-9.40%)` | :arrow_down: |
| [pkg/metrics/scheduler.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3Mvc2NoZWR1bGVyLmdv) | `60.89% <54.65%> (-0.69%)` | :arrow_down: |
| ... and [35 more](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [66eaecc...95d0e1c](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] kingamarton commented on a change in pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
kingamarton commented on a change in pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#discussion_r746564854
##########
File path: pkg/webservice/handlers.go
##########
@@ -744,3 +746,140 @@ func getLogLevel(w http.ResponseWriter, r *http.Request) {
buildJSONErrorResponse(w, err.Error(), http.StatusInternalServerError)
}
}
+
+func getPartitionInfoDao(lists map[string]*scheduler.PartitionContext) []*dao.PartitionInfo {
Review comment:
we have getPartition() method. Let's reuse that one.
##########
File path: pkg/webservice/handlers.go
##########
@@ -744,3 +746,140 @@ func getLogLevel(w http.ResponseWriter, r *http.Request) {
buildJSONErrorResponse(w, err.Error(), http.StatusInternalServerError)
}
}
+
+func getPartitionInfoDao(lists map[string]*scheduler.PartitionContext) []*dao.PartitionInfo {
+ result := make([]*dao.PartitionInfo, 0, len(lists))
+
+ for _, partitionContext := range lists {
+ partitionInfo := &dao.PartitionInfo{}
+ partitionInfo.ClusterID = partitionContext.RmID
+ partitionInfo.Name = common.GetPartitionNameWithoutClusterID(partitionContext.Name)
+ partitionInfo.State = partitionContext.GetCurrentState()
+ partitionInfo.LastStateTransitionTime = partitionContext.GetStateTime().String()
+
+ capacityInfo := dao.PartitionCapacity{}
+ capacityInfo.Capacity = partitionContext.GetTotalPartitionResource().DAOString()
+ capacityInfo.UsedCapacity = partitionContext.GetAllocatedResource().DAOString()
+ partitionInfo.Capacity = capacityInfo
+ partitionInfo.NodeSortingPolicy = dao.NodeSortingPolicy{
+ Type: partitionContext.GetNodeSortingPolicyType().String(),
+ ResourceWeights: partitionContext.GetNodeSortingResourceWeights(),
+ }
+
+ appList := partitionContext.GetApplications()
+ appList = append(appList, partitionContext.GetCompletedApplications()...)
+ applicationsState := make(map[string]int)
+ totalApplications := 0
+ for _, app := range appList {
+ applicationsState[app.CurrentState()]++
+ totalApplications++
+ }
+ applicationsState["total"] = totalApplications
+ partitionInfo.Applications = applicationsState
+ result = append(result, partitionInfo)
+ }
+
+ return result
+}
+
+func getAppHistoryDao(records []*history.MetricsRecord) []*dao.ApplicationHistoryDAOInfo {
+ result := make([]*dao.ApplicationHistoryDAOInfo, 0, len(records))
+ for _, record := range records {
+ if record == nil {
+ continue
+ }
+ element := &dao.ApplicationHistoryDAOInfo{
+ Timestamp: record.Timestamp.UnixNano(),
+ TotalApplications: strconv.Itoa(record.TotalApplications),
+ }
+ result = append(result, element)
+ }
+
+ return result
+}
+
+func getNodesDao(lists map[string]*scheduler.PartitionContext) []*dao.NodesDAOInfo {
Review comment:
Please reuse `func getPartitionNodes(w http.ResponseWriter, r *http.Request)` instead of defining a new one
##########
File path: pkg/webservice/handlers.go
##########
@@ -744,3 +746,140 @@ func getLogLevel(w http.ResponseWriter, r *http.Request) {
buildJSONErrorResponse(w, err.Error(), http.StatusInternalServerError)
}
}
+
+func getPartitionInfoDao(lists map[string]*scheduler.PartitionContext) []*dao.PartitionInfo {
+ result := make([]*dao.PartitionInfo, 0, len(lists))
+
+ for _, partitionContext := range lists {
+ partitionInfo := &dao.PartitionInfo{}
+ partitionInfo.ClusterID = partitionContext.RmID
+ partitionInfo.Name = common.GetPartitionNameWithoutClusterID(partitionContext.Name)
+ partitionInfo.State = partitionContext.GetCurrentState()
+ partitionInfo.LastStateTransitionTime = partitionContext.GetStateTime().String()
+
+ capacityInfo := dao.PartitionCapacity{}
+ capacityInfo.Capacity = partitionContext.GetTotalPartitionResource().DAOString()
+ capacityInfo.UsedCapacity = partitionContext.GetAllocatedResource().DAOString()
+ partitionInfo.Capacity = capacityInfo
+ partitionInfo.NodeSortingPolicy = dao.NodeSortingPolicy{
+ Type: partitionContext.GetNodeSortingPolicyType().String(),
+ ResourceWeights: partitionContext.GetNodeSortingResourceWeights(),
+ }
+
+ appList := partitionContext.GetApplications()
+ appList = append(appList, partitionContext.GetCompletedApplications()...)
+ applicationsState := make(map[string]int)
+ totalApplications := 0
+ for _, app := range appList {
+ applicationsState[app.CurrentState()]++
+ totalApplications++
+ }
+ applicationsState["total"] = totalApplications
+ partitionInfo.Applications = applicationsState
+ result = append(result, partitionInfo)
+ }
+
+ return result
+}
+
+func getAppHistoryDao(records []*history.MetricsRecord) []*dao.ApplicationHistoryDAOInfo {
+ result := make([]*dao.ApplicationHistoryDAOInfo, 0, len(records))
+ for _, record := range records {
+ if record == nil {
+ continue
+ }
+ element := &dao.ApplicationHistoryDAOInfo{
+ Timestamp: record.Timestamp.UnixNano(),
+ TotalApplications: strconv.Itoa(record.TotalApplications),
+ }
+ result = append(result, element)
+ }
+
+ return result
+}
+
+func getNodesDao(lists map[string]*scheduler.PartitionContext) []*dao.NodesDAOInfo {
+ result := make([]*dao.NodesDAOInfo, 0, len(lists))
+ for _, partition := range lists {
+ ns := partition.GetNodes()
+ nodesDao := make([]*dao.NodeDAOInfo, 0, len(ns))
+ for _, node := range ns {
+ nodeDao := getNodeJSON(node)
+ nodesDao = append(nodesDao, nodeDao)
+ }
+ result = append(result, &dao.NodesDAOInfo{
+ PartitionName: common.GetPartitionNameWithoutClusterID(partition.Name),
+ Nodes: nodesDao,
+ })
+ }
+
+ return result
+}
+
+func getContainerHistoryDao(records []*history.MetricsRecord) []*dao.ContainerHistoryDAOInfo {
Review comment:
PLease use `func getContainerHistory(w http.ResponseWriter, r *http.Request) {`
##########
File path: pkg/webservice/handlers.go
##########
@@ -744,3 +746,140 @@ func getLogLevel(w http.ResponseWriter, r *http.Request) {
buildJSONErrorResponse(w, err.Error(), http.StatusInternalServerError)
}
}
+
+func getPartitionInfoDao(lists map[string]*scheduler.PartitionContext) []*dao.PartitionInfo {
+ result := make([]*dao.PartitionInfo, 0, len(lists))
+
+ for _, partitionContext := range lists {
+ partitionInfo := &dao.PartitionInfo{}
+ partitionInfo.ClusterID = partitionContext.RmID
+ partitionInfo.Name = common.GetPartitionNameWithoutClusterID(partitionContext.Name)
+ partitionInfo.State = partitionContext.GetCurrentState()
+ partitionInfo.LastStateTransitionTime = partitionContext.GetStateTime().String()
+
+ capacityInfo := dao.PartitionCapacity{}
+ capacityInfo.Capacity = partitionContext.GetTotalPartitionResource().DAOString()
+ capacityInfo.UsedCapacity = partitionContext.GetAllocatedResource().DAOString()
+ partitionInfo.Capacity = capacityInfo
+ partitionInfo.NodeSortingPolicy = dao.NodeSortingPolicy{
+ Type: partitionContext.GetNodeSortingPolicyType().String(),
+ ResourceWeights: partitionContext.GetNodeSortingResourceWeights(),
+ }
+
+ appList := partitionContext.GetApplications()
+ appList = append(appList, partitionContext.GetCompletedApplications()...)
+ applicationsState := make(map[string]int)
+ totalApplications := 0
+ for _, app := range appList {
+ applicationsState[app.CurrentState()]++
+ totalApplications++
+ }
+ applicationsState["total"] = totalApplications
+ partitionInfo.Applications = applicationsState
+ result = append(result, partitionInfo)
+ }
+
+ return result
+}
+
+func getAppHistoryDao(records []*history.MetricsRecord) []*dao.ApplicationHistoryDAOInfo {
Review comment:
Please reuse `func getApplicationHistory(w http.ResponseWriter, r *http.Request)` method
##########
File path: pkg/webservice/handlers.go
##########
@@ -744,3 +746,140 @@ func getLogLevel(w http.ResponseWriter, r *http.Request) {
buildJSONErrorResponse(w, err.Error(), http.StatusInternalServerError)
}
}
+
+func getPartitionInfoDao(lists map[string]*scheduler.PartitionContext) []*dao.PartitionInfo {
+ result := make([]*dao.PartitionInfo, 0, len(lists))
+
+ for _, partitionContext := range lists {
+ partitionInfo := &dao.PartitionInfo{}
+ partitionInfo.ClusterID = partitionContext.RmID
+ partitionInfo.Name = common.GetPartitionNameWithoutClusterID(partitionContext.Name)
+ partitionInfo.State = partitionContext.GetCurrentState()
+ partitionInfo.LastStateTransitionTime = partitionContext.GetStateTime().String()
+
+ capacityInfo := dao.PartitionCapacity{}
+ capacityInfo.Capacity = partitionContext.GetTotalPartitionResource().DAOString()
+ capacityInfo.UsedCapacity = partitionContext.GetAllocatedResource().DAOString()
+ partitionInfo.Capacity = capacityInfo
+ partitionInfo.NodeSortingPolicy = dao.NodeSortingPolicy{
+ Type: partitionContext.GetNodeSortingPolicyType().String(),
+ ResourceWeights: partitionContext.GetNodeSortingResourceWeights(),
+ }
+
+ appList := partitionContext.GetApplications()
+ appList = append(appList, partitionContext.GetCompletedApplications()...)
+ applicationsState := make(map[string]int)
+ totalApplications := 0
+ for _, app := range appList {
+ applicationsState[app.CurrentState()]++
+ totalApplications++
+ }
+ applicationsState["total"] = totalApplications
+ partitionInfo.Applications = applicationsState
+ result = append(result, partitionInfo)
+ }
+
+ return result
+}
+
+func getAppHistoryDao(records []*history.MetricsRecord) []*dao.ApplicationHistoryDAOInfo {
+ result := make([]*dao.ApplicationHistoryDAOInfo, 0, len(records))
+ for _, record := range records {
+ if record == nil {
+ continue
+ }
+ element := &dao.ApplicationHistoryDAOInfo{
+ Timestamp: record.Timestamp.UnixNano(),
+ TotalApplications: strconv.Itoa(record.TotalApplications),
+ }
+ result = append(result, element)
+ }
+
+ return result
+}
+
+func getNodesDao(lists map[string]*scheduler.PartitionContext) []*dao.NodesDAOInfo {
+ result := make([]*dao.NodesDAOInfo, 0, len(lists))
+ for _, partition := range lists {
+ ns := partition.GetNodes()
+ nodesDao := make([]*dao.NodeDAOInfo, 0, len(ns))
+ for _, node := range ns {
+ nodeDao := getNodeJSON(node)
+ nodesDao = append(nodesDao, nodeDao)
+ }
+ result = append(result, &dao.NodesDAOInfo{
+ PartitionName: common.GetPartitionNameWithoutClusterID(partition.Name),
+ Nodes: nodesDao,
+ })
+ }
+
+ return result
+}
+
+func getContainerHistoryDao(records []*history.MetricsRecord) []*dao.ContainerHistoryDAOInfo {
+ result := make([]*dao.ContainerHistoryDAOInfo, 0, len(records))
+ for _, record := range records {
+ if record == nil {
+ continue
+ }
+ element := &dao.ContainerHistoryDAOInfo{
+ Timestamp: record.Timestamp.UnixNano(),
+ TotalContainers: strconv.Itoa(record.TotalContainers),
+ }
+ result = append(result, element)
+ }
+
+ return result
+}
+
+func getNodesUtilizationDao(lists map[string]*scheduler.PartitionContext) []*dao.NodesUtilDAOInfo {
+ var result []*dao.NodesUtilDAOInfo
+
+ for _, partition := range lists {
+ partitionResource := partition.GetTotalPartitionResource()
+ // partitionResource can be null if the partition has no node
+ if partitionResource != nil {
+ for name := range partitionResource.Resources {
+ result = append(result, getNodesUtilJSON(partition, name))
+ }
+ }
+ }
+
+ return result
+}
+
+func getApplicationsDao(lists map[string]*scheduler.PartitionContext) []*dao.ApplicationDAOInfo {
+ result := make([]*dao.ApplicationDAOInfo, 0, 32)
+
+ for _, partition := range lists {
+ size := partition.GetTotalCompletedApplicationCount() + partition.GetTotalApplicationCount()
+ appList := make([]*objects.Application, size)
+ appList = append(appList, partition.GetApplications()...)
+ appList = append(appList, partition.GetCompletedApplications()...)
+
+ for _, app := range appList {
+ result = append(result, getApplicationJSON(app))
+ }
+ }
+
+ return result
+}
+
+func getPartitionDAOInfo(lists map[string]*scheduler.PartitionContext) []*dao.PartitionDAOInfo {
Review comment:
Please reuse `func getApplicationJSON(app *objects.Application) *dao.ApplicationDAOInfo {`
##########
File path: pkg/webservice/handlers.go
##########
@@ -744,3 +746,140 @@ func getLogLevel(w http.ResponseWriter, r *http.Request) {
buildJSONErrorResponse(w, err.Error(), http.StatusInternalServerError)
}
}
+
+func getPartitionInfoDao(lists map[string]*scheduler.PartitionContext) []*dao.PartitionInfo {
+ result := make([]*dao.PartitionInfo, 0, len(lists))
+
+ for _, partitionContext := range lists {
+ partitionInfo := &dao.PartitionInfo{}
+ partitionInfo.ClusterID = partitionContext.RmID
+ partitionInfo.Name = common.GetPartitionNameWithoutClusterID(partitionContext.Name)
+ partitionInfo.State = partitionContext.GetCurrentState()
+ partitionInfo.LastStateTransitionTime = partitionContext.GetStateTime().String()
+
+ capacityInfo := dao.PartitionCapacity{}
+ capacityInfo.Capacity = partitionContext.GetTotalPartitionResource().DAOString()
+ capacityInfo.UsedCapacity = partitionContext.GetAllocatedResource().DAOString()
+ partitionInfo.Capacity = capacityInfo
+ partitionInfo.NodeSortingPolicy = dao.NodeSortingPolicy{
+ Type: partitionContext.GetNodeSortingPolicyType().String(),
+ ResourceWeights: partitionContext.GetNodeSortingResourceWeights(),
+ }
+
+ appList := partitionContext.GetApplications()
+ appList = append(appList, partitionContext.GetCompletedApplications()...)
+ applicationsState := make(map[string]int)
+ totalApplications := 0
+ for _, app := range appList {
+ applicationsState[app.CurrentState()]++
+ totalApplications++
+ }
+ applicationsState["total"] = totalApplications
+ partitionInfo.Applications = applicationsState
+ result = append(result, partitionInfo)
+ }
+
+ return result
+}
+
+func getAppHistoryDao(records []*history.MetricsRecord) []*dao.ApplicationHistoryDAOInfo {
+ result := make([]*dao.ApplicationHistoryDAOInfo, 0, len(records))
+ for _, record := range records {
+ if record == nil {
+ continue
+ }
+ element := &dao.ApplicationHistoryDAOInfo{
+ Timestamp: record.Timestamp.UnixNano(),
+ TotalApplications: strconv.Itoa(record.TotalApplications),
+ }
+ result = append(result, element)
+ }
+
+ return result
+}
+
+func getNodesDao(lists map[string]*scheduler.PartitionContext) []*dao.NodesDAOInfo {
+ result := make([]*dao.NodesDAOInfo, 0, len(lists))
+ for _, partition := range lists {
+ ns := partition.GetNodes()
+ nodesDao := make([]*dao.NodeDAOInfo, 0, len(ns))
+ for _, node := range ns {
+ nodeDao := getNodeJSON(node)
+ nodesDao = append(nodesDao, nodeDao)
+ }
+ result = append(result, &dao.NodesDAOInfo{
+ PartitionName: common.GetPartitionNameWithoutClusterID(partition.Name),
+ Nodes: nodesDao,
+ })
+ }
+
+ return result
+}
+
+func getContainerHistoryDao(records []*history.MetricsRecord) []*dao.ContainerHistoryDAOInfo {
+ result := make([]*dao.ContainerHistoryDAOInfo, 0, len(records))
+ for _, record := range records {
+ if record == nil {
+ continue
+ }
+ element := &dao.ContainerHistoryDAOInfo{
+ Timestamp: record.Timestamp.UnixNano(),
+ TotalContainers: strconv.Itoa(record.TotalContainers),
+ }
+ result = append(result, element)
+ }
+
+ return result
+}
+
+func getNodesUtilizationDao(lists map[string]*scheduler.PartitionContext) []*dao.NodesUtilDAOInfo {
Review comment:
PLease reuse `func getNodesUtilization(w http.ResponseWriter, r *http.Request) {`
##########
File path: pkg/webservice/handlers.go
##########
@@ -744,3 +746,140 @@ func getLogLevel(w http.ResponseWriter, r *http.Request) {
buildJSONErrorResponse(w, err.Error(), http.StatusInternalServerError)
}
}
+
+func getPartitionInfoDao(lists map[string]*scheduler.PartitionContext) []*dao.PartitionInfo {
+ result := make([]*dao.PartitionInfo, 0, len(lists))
+
+ for _, partitionContext := range lists {
+ partitionInfo := &dao.PartitionInfo{}
+ partitionInfo.ClusterID = partitionContext.RmID
+ partitionInfo.Name = common.GetPartitionNameWithoutClusterID(partitionContext.Name)
+ partitionInfo.State = partitionContext.GetCurrentState()
+ partitionInfo.LastStateTransitionTime = partitionContext.GetStateTime().String()
+
+ capacityInfo := dao.PartitionCapacity{}
+ capacityInfo.Capacity = partitionContext.GetTotalPartitionResource().DAOString()
+ capacityInfo.UsedCapacity = partitionContext.GetAllocatedResource().DAOString()
+ partitionInfo.Capacity = capacityInfo
+ partitionInfo.NodeSortingPolicy = dao.NodeSortingPolicy{
+ Type: partitionContext.GetNodeSortingPolicyType().String(),
+ ResourceWeights: partitionContext.GetNodeSortingResourceWeights(),
+ }
+
+ appList := partitionContext.GetApplications()
+ appList = append(appList, partitionContext.GetCompletedApplications()...)
+ applicationsState := make(map[string]int)
+ totalApplications := 0
+ for _, app := range appList {
+ applicationsState[app.CurrentState()]++
+ totalApplications++
+ }
+ applicationsState["total"] = totalApplications
+ partitionInfo.Applications = applicationsState
+ result = append(result, partitionInfo)
+ }
+
+ return result
+}
+
+func getAppHistoryDao(records []*history.MetricsRecord) []*dao.ApplicationHistoryDAOInfo {
+ result := make([]*dao.ApplicationHistoryDAOInfo, 0, len(records))
+ for _, record := range records {
+ if record == nil {
+ continue
+ }
+ element := &dao.ApplicationHistoryDAOInfo{
+ Timestamp: record.Timestamp.UnixNano(),
+ TotalApplications: strconv.Itoa(record.TotalApplications),
+ }
+ result = append(result, element)
+ }
+
+ return result
+}
+
+func getNodesDao(lists map[string]*scheduler.PartitionContext) []*dao.NodesDAOInfo {
+ result := make([]*dao.NodesDAOInfo, 0, len(lists))
+ for _, partition := range lists {
+ ns := partition.GetNodes()
+ nodesDao := make([]*dao.NodeDAOInfo, 0, len(ns))
+ for _, node := range ns {
+ nodeDao := getNodeJSON(node)
+ nodesDao = append(nodesDao, nodeDao)
+ }
+ result = append(result, &dao.NodesDAOInfo{
+ PartitionName: common.GetPartitionNameWithoutClusterID(partition.Name),
+ Nodes: nodesDao,
+ })
+ }
+
+ return result
+}
+
+func getContainerHistoryDao(records []*history.MetricsRecord) []*dao.ContainerHistoryDAOInfo {
+ result := make([]*dao.ContainerHistoryDAOInfo, 0, len(records))
+ for _, record := range records {
+ if record == nil {
+ continue
+ }
+ element := &dao.ContainerHistoryDAOInfo{
+ Timestamp: record.Timestamp.UnixNano(),
+ TotalContainers: strconv.Itoa(record.TotalContainers),
+ }
+ result = append(result, element)
+ }
+
+ return result
+}
+
+func getNodesUtilizationDao(lists map[string]*scheduler.PartitionContext) []*dao.NodesUtilDAOInfo {
+ var result []*dao.NodesUtilDAOInfo
+
+ for _, partition := range lists {
+ partitionResource := partition.GetTotalPartitionResource()
+ // partitionResource can be null if the partition has no node
+ if partitionResource != nil {
+ for name := range partitionResource.Resources {
+ result = append(result, getNodesUtilJSON(partition, name))
+ }
+ }
+ }
+
+ return result
+}
+
+func getApplicationsDao(lists map[string]*scheduler.PartitionContext) []*dao.ApplicationDAOInfo {
+ result := make([]*dao.ApplicationDAOInfo, 0, 32)
+
+ for _, partition := range lists {
+ size := partition.GetTotalCompletedApplicationCount() + partition.GetTotalApplicationCount()
+ appList := make([]*objects.Application, size)
+ appList = append(appList, partition.GetApplications()...)
+ appList = append(appList, partition.GetCompletedApplications()...)
+
+ for _, app := range appList {
+ result = append(result, getApplicationJSON(app))
+ }
+ }
+
+ return result
+}
+
+func getPartitionDAOInfo(lists map[string]*scheduler.PartitionContext) []*dao.PartitionDAOInfo {
+ queues := make([]*dao.PartitionDAOInfo, len(lists))
+
+ for _, partition := range lists {
+ queues = append(queues, getPartitionJSON(partition))
+ }
+
+ return queues
+}
+
+func getClusterInfoDao(lists map[string]*scheduler.PartitionContext) []*dao.ClusterDAOInfo {
Review comment:
Please reuse `func getClusterInfo(w http.ResponseWriter, r *http.Request) {`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] codecov[bot] edited a comment on pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#issuecomment-966414547
# [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#336](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (cafdb27) into [master](https://codecov.io/gh/apache/incubator-yunikorn-core/commit/5a1c19e280f66d6cb4fc206a675116260d6f2ff1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5a1c19e) will **increase** coverage by `4.32%`.
> The diff coverage is `69.06%`.
> :exclamation: Current head cafdb27 differs from pull request most recent head ccae02b. Consider uploading reports for the commit ccae02b to get more accurate results
[](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #336 +/- ##
==========================================
+ Coverage 63.46% 67.79% +4.32%
==========================================
Files 60 64 +4
Lines 5220 6198 +978
==========================================
+ Hits 3313 4202 +889
- Misses 1747 1773 +26
- Partials 160 223 +63
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [pkg/metrics/init.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvaW5pdC5nbw==) | `62.96% <ø> (ø)` | |
| [pkg/scheduler/drf\_preemption\_policy.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9kcmZfcHJlZW1wdGlvbl9wb2xpY3kuZ28=) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/nodes\_usage\_monitor.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9ub2Rlc191c2FnZV9tb25pdG9yLmdv) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/objects/allocation.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL2FsbG9jYXRpb24uZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/node\_iterator.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL25vZGVfaXRlcmF0b3IuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/sorters.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL3NvcnRlcnMuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/webservice/webservice.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3dlYnNlcnZpY2Uvd2Vic2VydmljZS5nbw==) | `32.25% <ø> (+19.35%)` | :arrow_up: |
| [pkg/scheduler/context.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9jb250ZXh0Lmdv) | `16.55% <17.82%> (+10.28%)` | :arrow_up: |
| [pkg/metrics/metrics\_collector.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvbWV0cmljc19jb2xsZWN0b3IuZ28=) | `64.51% <45.00%> (-9.40%)` | :arrow_down: |
| [pkg/metrics/scheduler.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3Mvc2NoZWR1bGVyLmdv) | `60.89% <54.65%> (-0.69%)` | :arrow_down: |
| ... and [35 more](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [66eaecc...ccae02b](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] codecov[bot] edited a comment on pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#issuecomment-966414547
# [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#336](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e9496d2) into [master](https://codecov.io/gh/apache/incubator-yunikorn-core/commit/5a1c19e280f66d6cb4fc206a675116260d6f2ff1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5a1c19e) will **increase** coverage by `4.32%`.
> The diff coverage is `69.06%`.
> :exclamation: Current head e9496d2 differs from pull request most recent head 7762c5b. Consider uploading reports for the commit 7762c5b to get more accurate results
[](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #336 +/- ##
==========================================
+ Coverage 63.46% 67.79% +4.32%
==========================================
Files 60 64 +4
Lines 5220 6198 +978
==========================================
+ Hits 3313 4202 +889
- Misses 1747 1773 +26
- Partials 160 223 +63
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [pkg/metrics/init.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvaW5pdC5nbw==) | `62.96% <ø> (ø)` | |
| [pkg/scheduler/drf\_preemption\_policy.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9kcmZfcHJlZW1wdGlvbl9wb2xpY3kuZ28=) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/nodes\_usage\_monitor.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9ub2Rlc191c2FnZV9tb25pdG9yLmdv) | `0.00% <0.00%> (ø)` | |
| [pkg/scheduler/objects/allocation.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL2FsbG9jYXRpb24uZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/node\_iterator.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL25vZGVfaXRlcmF0b3IuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/scheduler/objects/sorters.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9vYmplY3RzL3NvcnRlcnMuZ28=) | `100.00% <ø> (ø)` | |
| [pkg/webservice/webservice.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3dlYnNlcnZpY2Uvd2Vic2VydmljZS5nbw==) | `32.25% <ø> (+19.35%)` | :arrow_up: |
| [pkg/scheduler/context.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL3NjaGVkdWxlci9jb250ZXh0Lmdv) | `16.55% <17.82%> (+10.28%)` | :arrow_up: |
| [pkg/metrics/metrics\_collector.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3MvbWV0cmljc19jb2xsZWN0b3IuZ28=) | `64.51% <45.00%> (-9.40%)` | :arrow_down: |
| [pkg/metrics/scheduler.go](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL21ldHJpY3Mvc2NoZWR1bGVyLmdv) | `60.89% <54.65%> (-0.69%)` | :arrow_down: |
| ... and [35 more](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [66eaecc...7762c5b](https://codecov.io/gh/apache/incubator-yunikorn-core/pull/336?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] pbacsko commented on a change in pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
pbacsko commented on a change in pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#discussion_r748293174
##########
File path: pkg/webservice/state_dump.go
##########
@@ -0,0 +1,207 @@
+/*
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+package webservice
+
+import (
+ "encoding/json"
+ "fmt"
+ "io"
+ "net/http"
+ "os"
+ "strconv"
+ "sync"
+ "time"
+
+ "github.com/apache/incubator-yunikorn-core/pkg/log"
+ metrics2 "github.com/apache/incubator-yunikorn-core/pkg/metrics"
+ "github.com/apache/incubator-yunikorn-core/pkg/scheduler"
+ "github.com/apache/incubator-yunikorn-core/pkg/webservice/dao"
+
+ "github.com/gorilla/mux"
+ "go.uber.org/zap"
+)
+
+const (
+ defaultStateDumpPeriod = 60
+ stateDumpFilePath = "yunikorn-state.txt"
+)
+
+var (
+ periodicStateDump bool
+ abort chan struct{}
+ startStop sync.Mutex
+ stateDump sync.Mutex // guards against simultaneous periodic vs web request
+)
+
+type AggregatedStateInfo struct {
+ Timestamp string
+ Partitions []*dao.PartitionInfo
+ Applications []*dao.ApplicationDAOInfo
+ AppHistory []*dao.ApplicationHistoryDAOInfo
+ Nodes []*dao.NodesDAOInfo
+ NodesUtilization []*dao.NodesUtilDAOInfo
+ ClusterInfo []*dao.ClusterDAOInfo
+ ClusterUtilization []*dao.ClustersUtilDAOInfo
+ ContainerHistory []*dao.ContainerHistoryDAOInfo
+ Queues []*dao.PartitionDAOInfo
+ SchedulerHealth *dao.SchedulerHealthDAOInfo
Review comment:
Makes sense, removing it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] pbacsko commented on a change in pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
pbacsko commented on a change in pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#discussion_r748246338
##########
File path: pkg/webservice/handlers_test.go
##########
@@ -1279,3 +1282,159 @@ func TestGetNodesUtilization(t *testing.T) {
assert.NilError(t, err)
assert.Equal(t, len(nodesDao), 0)
}
+
+func TestGetFullStateDump(t *testing.T) {
+ schedulerContext = prepareSchedulerContext(t)
+
+ imHistory = history.NewInternalMetricsHistory(5)
+ req, err := http.NewRequest("GET", "/ws/v1/getfullstatedump", strings.NewReader(""))
+ req = mux.SetURLVars(req, make(map[string]string))
+ assert.NilError(t, err)
+ resp := &MockResponseWriter{}
+
+ getFullStateDump(resp, req)
+ receivedBytes := resp.outputBytes
+ statusCode := resp.statusCode
+ assert.Assert(t, len(receivedBytes) > 0, "json response is empty")
+ assert.Check(t, statusCode != http.StatusInternalServerError, "response status code")
+ var aggregated AggregatedStateInfo
+ if err := json.Unmarshal(receivedBytes, &aggregated); err != nil {
+ t.Fatal(err)
+ }
+ verifyStateDumpJSON(t, &aggregated)
+}
+
+func TestEnableDisablePeriodicStateDump(t *testing.T) {
+ schedulerContext = prepareSchedulerContext(t)
+ defer deleteStateDumpFile()
+ defer terminateGoroutine()
+
+ imHistory = history.NewInternalMetricsHistory(5)
Review comment:
Yes, we do. There is a part which turns to `imHistory`, so it has to be initialized.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-yunikorn-core] kingamarton commented on a change in pull request #336: [YUNIKORN-940] Periodic & on-demand state dump in Yunikorn
Posted by GitBox <gi...@apache.org>.
kingamarton commented on a change in pull request #336:
URL: https://github.com/apache/incubator-yunikorn-core/pull/336#discussion_r750294140
##########
File path: pkg/webservice/state_dump.go
##########
@@ -0,0 +1,225 @@
+/*
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+package webservice
+
+import (
+ "encoding/json"
+ "fmt"
+ "io"
+ "net/http"
+ "os"
+ "strconv"
+ "strings"
+ "sync"
+ "time"
+
+ "github.com/apache/incubator-yunikorn-core/pkg/log"
+ "github.com/apache/incubator-yunikorn-core/pkg/webservice/dao"
+
+ "github.com/gorilla/mux"
+ "go.uber.org/zap"
+)
+
+const (
+ defaultStateDumpPeriodSeconds time.Duration = 60 //nolint
+ stateDumpFilePath string = "yunikorn-state.txt"
+)
+
+var (
+ periodicStateDump bool
+ abort chan struct{}
+ startStop sync.Mutex
+ stateDump sync.Mutex // guards against simultaneous periodic vs web request
+)
+
+type AggregatedStateInfo struct {
+ Timestamp string
+ Partitions []*dao.PartitionInfo
+ Applications []*dao.ApplicationDAOInfo
+ AppHistory []*dao.ApplicationHistoryDAOInfo
+ Nodes []*dao.NodesDAOInfo
+ NodesUtilization []*dao.NodesUtilDAOInfo
+ ClusterInfo []*dao.ClusterDAOInfo
+ ClusterUtilization []*dao.ClustersUtilDAOInfo
+ ContainerHistory []*dao.ContainerHistoryDAOInfo
+ Queues []*dao.PartitionDAOInfo
+ LogLevel string
+}
+
+func getFullStateDump(w http.ResponseWriter, r *http.Request) {
+ writeHeaders(w)
+ if err := doStateDump(w); err != nil {
+ buildJSONErrorResponse(w, err.Error(), http.StatusInternalServerError)
+ }
+}
+
+func handlePeriodicStateDump(w http.ResponseWriter, r *http.Request) {
+ vars := mux.Vars(r)
+ writeHeaders(w)
+ if len(vars["switch"]) == 0 {
+ buildJSONErrorResponse(w, "required parameter enabled/disabled is missing",
+ http.StatusBadRequest)
+ return
+ }
+
+ enabledSwitch := strings.ToLower(vars["switch"])
+ switch enabledSwitch {
+ case "enable":
+ enablePeriodicStateDump(w, r)
+ case "disable":
+ disablePeriodicStateDump(w, r)
+ default:
+ buildJSONErrorResponse(w, fmt.Sprintf("required parameter enable/disable is illegal: %s", enabledSwitch),
+ http.StatusBadRequest)
+ }
+}
+
+func enablePeriodicStateDump(w http.ResponseWriter, r *http.Request) {
+ vars := mux.Vars(r)
+ const defaultPeriod = int(defaultStateDumpPeriodSeconds)
+ var periodFromHeader int
Review comment:
I think this should be Duration instead of int, then you can avoid a lot of conversions.
##########
File path: pkg/webservice/state_dump.go
##########
@@ -0,0 +1,225 @@
+/*
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+package webservice
+
+import (
+ "encoding/json"
+ "fmt"
+ "io"
+ "net/http"
+ "os"
+ "strconv"
+ "strings"
+ "sync"
+ "time"
+
+ "github.com/apache/incubator-yunikorn-core/pkg/log"
+ "github.com/apache/incubator-yunikorn-core/pkg/webservice/dao"
+
+ "github.com/gorilla/mux"
+ "go.uber.org/zap"
+)
+
+const (
+ defaultStateDumpPeriodSeconds time.Duration = 60 //nolint
+ stateDumpFilePath string = "yunikorn-state.txt"
+)
+
+var (
+ periodicStateDump bool
+ abort chan struct{}
+ startStop sync.Mutex
+ stateDump sync.Mutex // guards against simultaneous periodic vs web request
+)
+
+type AggregatedStateInfo struct {
+ Timestamp string
+ Partitions []*dao.PartitionInfo
+ Applications []*dao.ApplicationDAOInfo
+ AppHistory []*dao.ApplicationHistoryDAOInfo
+ Nodes []*dao.NodesDAOInfo
+ NodesUtilization []*dao.NodesUtilDAOInfo
+ ClusterInfo []*dao.ClusterDAOInfo
+ ClusterUtilization []*dao.ClustersUtilDAOInfo
+ ContainerHistory []*dao.ContainerHistoryDAOInfo
+ Queues []*dao.PartitionDAOInfo
+ LogLevel string
+}
+
+func getFullStateDump(w http.ResponseWriter, r *http.Request) {
+ writeHeaders(w)
+ if err := doStateDump(w); err != nil {
+ buildJSONErrorResponse(w, err.Error(), http.StatusInternalServerError)
+ }
+}
+
+func handlePeriodicStateDump(w http.ResponseWriter, r *http.Request) {
+ vars := mux.Vars(r)
+ writeHeaders(w)
+ if len(vars["switch"]) == 0 {
+ buildJSONErrorResponse(w, "required parameter enabled/disabled is missing",
+ http.StatusBadRequest)
+ return
+ }
+
+ enabledSwitch := strings.ToLower(vars["switch"])
+ switch enabledSwitch {
+ case "enable":
+ enablePeriodicStateDump(w, r)
+ case "disable":
+ disablePeriodicStateDump(w, r)
+ default:
+ buildJSONErrorResponse(w, fmt.Sprintf("required parameter enable/disable is illegal: %s", enabledSwitch),
+ http.StatusBadRequest)
+ }
+}
+
+func enablePeriodicStateDump(w http.ResponseWriter, r *http.Request) {
+ vars := mux.Vars(r)
+ const defaultPeriod = int(defaultStateDumpPeriodSeconds)
+ var periodFromHeader int
+ var period time.Duration
+ var err error
+ var zapField = zap.Duration("defaultStateDumpPeriodSeconds", defaultStateDumpPeriodSeconds)
+
+ if len(vars["period"]) == 0 {
+ log.Logger().Info("using the default period for state dump",
+ zapField)
+ periodFromHeader = defaultPeriod
+ } else if periodFromHeader, err = strconv.Atoi(vars["period"]); err != nil {
+ log.Logger().Warn("illegal value for period, using the default",
+ zapField)
+ periodFromHeader = defaultPeriod
+ }
+
+ if periodFromHeader < 0 {
+ log.Logger().Warn("period value is negative, using the default",
+ zapField)
+ periodFromHeader = defaultPeriod
+ }
+
+ period = time.Duration(periodFromHeader)
+ if err = startBackGroundStateDump(period); err != nil {
+ buildJSONErrorResponse(w, err.Error(), http.StatusInternalServerError)
+ }
+}
+
+func disablePeriodicStateDump(w http.ResponseWriter, r *http.Request) {
+ if err := stopBackGroundStateDump(); err != nil {
+ buildJSONErrorResponse(w, err.Error(), http.StatusInternalServerError)
+ }
+}
+
+func doStateDump(w io.Writer) error {
+ stateDump.Lock()
+ defer stateDump.Unlock()
+
+ partitionContext := schedulerContext.GetPartitionMapClone()
+ records := imHistory.GetRecords()
+ zapConfig := log.GetConfig()
+
+ var aggregated = AggregatedStateInfo{
+ Timestamp: time.Now().Format(time.RFC3339),
+ Partitions: getPartitionInfoDAO(partitionContext),
+ Applications: getApplicationsDAO(partitionContext),
+ AppHistory: getAppHistoryDAO(records),
+ Nodes: getNodesDAO(partitionContext),
+ NodesUtilization: getNodesUtilDAO(partitionContext),
+ ClusterInfo: getClusterDAO(partitionContext),
+ ClusterUtilization: getClustersUtilDAO(partitionContext),
+ ContainerHistory: getContainerHistoryDAO(records),
+ Queues: getPartitionDAO(partitionContext),
+ LogLevel: zapConfig.Level.Level().String(),
+ }
+
+ var prettyJSON []byte
+ var err error
+ prettyJSON, err = json.MarshalIndent(aggregated, "", " ")
+ if err != nil {
+ return err
+ }
+ if _, err = w.Write(prettyJSON); err != nil {
+ return err
+ }
+
+ return nil
+}
+
+func startBackGroundStateDump(period time.Duration) error {
+ startStop.Lock()
+ defer startStop.Unlock()
+
+ if periodicStateDump {
+ var errMsg = "state dump already running"
+ log.Logger().Error(errMsg)
+ return fmt.Errorf(errMsg)
+ }
+
+ file, err := os.OpenFile(stateDumpFilePath, os.O_APPEND|os.O_CREATE|os.O_RDWR, 0666)
+ if err != nil {
+ log.Logger().Error("unable to open/create file",
+ zap.Error(err))
+ return err
+ }
+ abort = make(chan struct{})
+ periodicStateDump = true
+
+ go func() {
+ ticker := time.NewTicker(period * time.Second)
+
+ for {
+ select {
+ case <-abort:
+ ticker.Stop()
+ file.Close()
+ return
+ case <-ticker.C:
+ if err := doStateDump(file); err != nil {
+ log.Logger().Error("state dump failed", zap.Error(err))
+ if err := stopBackGroundStateDump(); err != nil {
Review comment:
shouldn't we restart it in case of some error? If it failed once for some reason it is sure that it will fail for the next calls as well?
##########
File path: pkg/webservice/state_dump.go
##########
@@ -0,0 +1,225 @@
+/*
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+package webservice
+
+import (
+ "encoding/json"
+ "fmt"
+ "io"
+ "net/http"
+ "os"
+ "strconv"
+ "strings"
+ "sync"
+ "time"
+
+ "github.com/apache/incubator-yunikorn-core/pkg/log"
+ "github.com/apache/incubator-yunikorn-core/pkg/webservice/dao"
+
+ "github.com/gorilla/mux"
+ "go.uber.org/zap"
+)
+
+const (
+ defaultStateDumpPeriodSeconds time.Duration = 60 //nolint
Review comment:
please add the exact lint check you want to skip. However I think this way it should pass the lint check. Have you tried to run the lint checks after this change? If it fails, please check the other examples of ignoring it.
What is the default unit for the duration? I am not sure that this line will be equal to 60 seconds.
`defaultStateDumpPeriodSeconds time.Duration = 60 * time.Seconds` - this is for sure defined in seconds. Can you please modify it?
##########
File path: pkg/webservice/routes.go
##########
@@ -170,6 +170,18 @@ var webRoutes = routes{
"/ws/v1/partition/{partition}/queue/{queue}/applications",
getQueueApplications,
},
+ route{
+ "Scheduler",
+ "GET",
+ "/ws/v1/fullstatedump",
+ getFullStateDump,
+ },
+ route{
+ "Scheduler",
+ "PUT",
+ "/ws/v1/periodicstatedump/{switch}/{period}",
Review comment:
Here in the variable name I would include the unit as well. Otherwise the user might be confused by this period.
##########
File path: pkg/webservice/state_dump.go
##########
@@ -0,0 +1,225 @@
+/*
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+package webservice
+
+import (
+ "encoding/json"
+ "fmt"
+ "io"
+ "net/http"
+ "os"
+ "strconv"
+ "strings"
+ "sync"
+ "time"
+
+ "github.com/apache/incubator-yunikorn-core/pkg/log"
+ "github.com/apache/incubator-yunikorn-core/pkg/webservice/dao"
+
+ "github.com/gorilla/mux"
+ "go.uber.org/zap"
+)
+
+const (
+ defaultStateDumpPeriodSeconds time.Duration = 60 //nolint
+ stateDumpFilePath string = "yunikorn-state.txt"
+)
+
+var (
+ periodicStateDump bool
+ abort chan struct{}
+ startStop sync.Mutex
+ stateDump sync.Mutex // guards against simultaneous periodic vs web request
+)
+
+type AggregatedStateInfo struct {
+ Timestamp string
+ Partitions []*dao.PartitionInfo
+ Applications []*dao.ApplicationDAOInfo
+ AppHistory []*dao.ApplicationHistoryDAOInfo
+ Nodes []*dao.NodesDAOInfo
+ NodesUtilization []*dao.NodesUtilDAOInfo
+ ClusterInfo []*dao.ClusterDAOInfo
+ ClusterUtilization []*dao.ClustersUtilDAOInfo
+ ContainerHistory []*dao.ContainerHistoryDAOInfo
+ Queues []*dao.PartitionDAOInfo
+ LogLevel string
+}
+
+func getFullStateDump(w http.ResponseWriter, r *http.Request) {
+ writeHeaders(w)
+ if err := doStateDump(w); err != nil {
+ buildJSONErrorResponse(w, err.Error(), http.StatusInternalServerError)
+ }
+}
+
+func handlePeriodicStateDump(w http.ResponseWriter, r *http.Request) {
+ vars := mux.Vars(r)
+ writeHeaders(w)
+ if len(vars["switch"]) == 0 {
+ buildJSONErrorResponse(w, "required parameter enabled/disabled is missing",
+ http.StatusBadRequest)
+ return
+ }
+
+ enabledSwitch := strings.ToLower(vars["switch"])
+ switch enabledSwitch {
+ case "enable":
+ enablePeriodicStateDump(w, r)
+ case "disable":
+ disablePeriodicStateDump(w, r)
+ default:
+ buildJSONErrorResponse(w, fmt.Sprintf("required parameter enable/disable is illegal: %s", enabledSwitch),
+ http.StatusBadRequest)
+ }
+}
+
+func enablePeriodicStateDump(w http.ResponseWriter, r *http.Request) {
+ vars := mux.Vars(r)
+ const defaultPeriod = int(defaultStateDumpPeriodSeconds)
Review comment:
If you handle the variable as duration everywhere, you will not need this constant.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@yunikorn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org