You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by GitBox <gi...@apache.org> on 2021/12/18 08:17:11 UTC
[GitHub] [cloudstack] DaanHoogland opened a new pull request #5588: Mshost stats
DaanHoogland opened a new pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588
### Description
This PR implements statistics for `ManagementServer`s. The code queries the local `mxBeans` and the filesystem `/proc` for information on the system. if then publishes this to all `ManagementSever`s that keep a list of the recent state of each other. Static data is stored in the database, so if this `ManagementServer` is down others can still query it's last boot/start/stop/shutdown data (and possibly more)
the `listManagementServerMetrics` API gathers the data from the internal list and augments it with any static data from the DB.
<!--- Describe your changes in DETAIL - And how has behaviour functionally changed. -->
<!-- For new features, provide link to FS, dev ML discussion etc. -->
<!-- In case of bug fix, the expected and actual behaviours, steps to reproduce. -->
<!-- When "Fixes: #<id>" is specified, the issue/PR will automatically be closed when this PR gets merged -->
<!-- For addressing multiple issues/PRs, use multiple "Fixes: #<id>" -->
<!-- Fixes: # -->
<!--- ********************************************************************************* -->
<!--- NOTE: AUTOMATATION USES THE DESCRIPTIONS TO SET LABELS AND PRODUCE DOCUMENTATION. -->
<!--- PLEASE PUT AN 'X' in only **ONE** box -->
<!--- ********************************************************************************* -->
### Types of changes
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
### Feature/Enhancement Scale or Bug Severity
#### Feature/Enhancement Scale
- [x] Major
- [ ] Minor
#### Bug Severity
- [ ] BLOCKER
- [ ] Critical
- [x] Major
- [ ] Minor
- [ ] Trivial
### Screenshots (if appropriate):
### How Has This Been Tested?
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
<!-- see how your change affects other areas of the code, etc. -->
so far only manual verification in a lab environment has been done.
<!-- Please read the [CONTRIBUTING](https://github.com/apache/cloudstack/blob/main/CONTRIBUTING.md) document -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1010017163
Wow, thanks @GutoVeronezi , really appreciated. I'll go through the comments one by one over the coming days ;).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1024169626
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 2365
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1016424656
@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1022871617
@sureshanaparti a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1028092470
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 2437
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1061238655
<b>Trillian test result (tid-3517)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 35290 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5588-t3517-kvm-centos7.zip
Smoke tests completed. 90 look OK, 2 have errors
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_list_management_server_metrics | `Failure` | 0.05 | test_metrics_api.py
test_02_redundant_VPC_default_routes | `Failure` | 368.69 | test_vpc_redundant.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1067980727
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1061238655
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1069269900
@DaanHoogland a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
nvazquez commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1071988540
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1069125068
@DaanHoogland a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-973940087
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1745
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-969218665
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1712
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-969041536
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-972698789
thanks for the review @weizhouapache , those sugestions are on my list :pray: ;)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-995989685
@sureshanaparti a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-996763336
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-999241257
<b>Trillian test result (tid-2708)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 33979 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5588-t2708-kvm-centos7.zip
Smoke tests completed. 91 look OK, 0 have errors
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r737373039
##########
File path: engine/schema/src/main/resources/META-INF/db/schema-41520to41600.sql
##########
@@ -807,3 +807,21 @@ ALTER TABLE `cloud_usage`.`user_statistics` DROP INDEX `account_id`, ADD UNIQUE
ALTER TABLE `cloud`.`vm_work_job` ADD COLUMN `secondary_object` char(100) COMMENT 'any additional item that must be checked during queueing' AFTER `vm_instance_id`;
ALTER TABLE cloud.vm_work_job ADD CONSTRAINT vm_work_job_step_and_objects UNIQUE KEY (step,vm_instance_id,secondary_object);
+
+CREATE TABLE `cloud`.`mshost_status` (
+ `id` bigint unsigned NOT NULL auto_increment COMMENT 'id',
+ `ms_id` varchar(40) DEFAULT NULL COMMENT 'the id of the management server record',
+ `last_start` datetime COMMENT 'the last start time for this MS',
+ `last_stop` datetime COMMENT 'the last stop time for this MS',
+ `last_boot` datetime COMMENT 'the last system boot time for the host of this MS',
+ `last_down` datetime COMMENT 'the last system shutdown time for the host of this MS',
+ `os_name` varchar(64) DEFAULT null COMMENT 'the name of the os type running on the host of this MS',
+ `os_version` varchar(64) DEFAULT null COMMENT 'the version of the os running on the host of this MS',
+ `java_name` varchar(64) DEFAULT null COMMENT 'the name of the java distribution running this MS',
+ `java_version` varchar(64) DEFAULT null COMMENT 'the version of the java distribution running this MS',
+ `updated` datetime DEFAULT NULL,
+ `created` datetime DEFAULT NULL,
+ `removed` datetime DEFAULT NULL,
+ PRIMARY KEY (`id`),
+ CONSTRAINT `uc_ms_id` UNIQUE (`ms_id`)
Review comment:
agree. most of the db columns, which are referencing a column in the other table through a unique identifier, are referenced through 'id' (not 'uuid').
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-950942111
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1612
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland closed pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland closed pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-954123825
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r774645930
##########
File path: ui/public/locales/en.json
##########
@@ -1272,8 +1283,11 @@
"label.lang.spanish": "Spanish",
"label.last.updated": "Last Update",
"label.lastannotated": "Last annotation date",
+"label.lastboottime": "Boot time of the management server machine",
"label.lastname": "Last Name",
"label.lastname.lower": "lastname",
+"label.lastserverstart": "Last start time for this management server",
Review comment:
how about `Last Management Server Process start time`? We discussed before that it must be clear whether it is the machine or the java process.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-998937208
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1005096547
<b>Trillian test result (tid-2779)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 31641 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5588-t2779-kvm-centos7.zip
Smoke tests completed. 91 look OK, 0 have errors
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-949748478
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1605
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-972011376
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1721
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r736337724
##########
File path: framework/cluster/src/main/java/com/cloud/cluster/ClusterManager.java
##########
@@ -21,27 +21,36 @@
import com.cloud.utils.component.Manager;
+/**
+ * the definition of the framework for inter MS communication
+ */
public interface ClusterManager extends Manager {
static final String ALERT_SUBJECT = "cluster-alert";
final ConfigKey<Integer> HeartbeatInterval = new ConfigKey<Integer>(Integer.class, "cluster.heartbeat.interval", "management-server", "1500",
"Interval to check for the heart beat between management server nodes", false);
final ConfigKey<Integer> HeartbeatThreshold = new ConfigKey<Integer>(Integer.class, "cluster.heartbeat.threshold", "management-server", "150000",
"Threshold before self-fence the management server", true);
+ /**
+ * adds a new pakket to the incoming queue
Review comment:
```suggestion
* adds a new packet to the incoming queue
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-962043397
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1677
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland closed pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland closed pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-962043397
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1072420804
<b>Trillian test result (tid-3658)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 35904 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5588-t3658-kvm-centos7.zip
Smoke tests completed. 92 look OK, 0 have errors
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1070177845
@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1069268484
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1063770084
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1082887507
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 3018
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1083904851
<b>Trillian test result (tid-3761)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 34604 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5588-t3761-kvm-centos7.zip
Smoke tests completed. 79 look OK, 1 have errors
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_01_add_primary_storage_disabled_host | `Error` | 0.71 | test_primary_storage.py
test_01_primary_storage_nfs | `Error` | 0.18 | test_primary_storage.py
ContextSuite context=TestStorageTags>:setup | `Error` | 0.31 | test_primary_storage.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1084526338
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1004563300
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1004563181
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] GutoVeronezi commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
GutoVeronezi commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r792721037
##########
File path: framework/cluster/src/main/java/com/cloud/cluster/ClusterManager.java
##########
@@ -53,19 +62,44 @@
void registerDispatcher(Dispatcher dispatcher);
+ /**
+ * register a listener for incoming status changes of ManagementServers
+ *
+ * @param administrator the object administrating statusses
+ */
+ void registerStatusAdministrator(StatusAdministrator administrator);
+
ManagementServerHost getPeer(String peerName);
+ /**
+ *
+ * @return {code}Long.toString({code}{@see getManagementNodeId()}{code}){code}
+ */
String getSelfPeerName();
long getManagementNodeId();
+ /**
+ * determined by the time
+ * @return start time as {code}System.currentTimeMillis(){code}
+ */
long getCurrentRunId();
- public long getManagementRunId(long msId);
+ /**
+ * the other guy's is derived from start time as stored in the db
+ */
+ long getManagementRunId(long msId);
- public interface Dispatcher {
+ interface Dispatcher {
String getName();
String dispatch(ClusterServicePdu pdu);
}
+
+ /**
+ * what the client of {@see registerStatusAdministrator()} should implement
+ */
+ interface StatusAdministrator {
+ String newStatus(ClusterServicePdu pdu);
+ }
Review comment:
Just some improvements in writing and punctuation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] borisstoyanov removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
borisstoyanov removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1024026955
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1063770851
@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1067196827
<b>Trillian test result (tid-3594)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 34968 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5588-t3594-kvm-centos7.zip
Smoke tests completed. 86 look OK, 6 have errors
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_01_add_primary_storage_disabled_host | `Error` | 0.69 | test_primary_storage.py
test_01_primary_storage_nfs | `Error` | 0.17 | test_primary_storage.py
ContextSuite context=TestStorageTags>:setup | `Error` | 0.30 | test_primary_storage.py
test_list_management_server_metrics | `Failure` | 0.05 | test_metrics_api.py
test_03_deploy_and_scale_kubernetes_cluster | `Failure` | 29.87 | test_kubernetes_clusters.py
test_07_deploy_kubernetes_ha_cluster | `Failure` | 56.71 | test_kubernetes_clusters.py
test_08_upgrade_kubernetes_ha_cluster | `Failure` | 37.36 | test_kubernetes_clusters.py
test_09_delete_kubernetes_ha_cluster | `Failure` | 30.02 | test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown | `Error` | 114.32 | test_kubernetes_clusters.py
test_01_secure_vm_migration | `Error` | 153.31 | test_vm_life_cycle.py
test_02_unsecure_vm_migration | `Error` | 269.63 | test_vm_life_cycle.py
test_03_secured_to_nonsecured_vm_migration | `Error` | 145.31 | test_vm_life_cycle.py
test_08_migrate_vm | `Error` | 44.98 | test_vm_life_cycle.py
test_02_list_snapshots_with_removed_data_store | `Error` | 8.49 | test_snapshots.py
test_02_list_snapshots_with_removed_data_store | `Error` | 8.49 | test_snapshots.py
test_hostha_enable_ha_when_host_in_maintenance | `Error` | 303.29 | test_hostha_kvm.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-972676761
Faking result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID fake-job
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-973940087
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1745
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-974221108
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1750
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] weizhouapache commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
weizhouapache commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r752012019
##########
File path: framework/cluster/src/main/java/com/cloud/cluster/ClusterManagerImpl.java
##########
@@ -97,10 +98,14 @@
@Inject
private ManagementServerHostDao _mshostDao;
@Inject
+ private ManagementServerStatusDao mshostStatusDao;
Review comment:
according to java code convention, should it start with "_" ?
##########
File path: framework/cluster/src/main/java/com/cloud/cluster/ClusterManagerImpl.java
##########
@@ -1005,8 +1050,11 @@ public boolean stop() {
if (_mshostId != null) {
final ManagementServerHostVO mshost = _mshostDao.findByMsid(_msId);
+ final ManagementServerStatusVO mshostStatus = mshostStatusDao.findByMsId(mshost.getUuid());
mshost.setState(ManagementServerHost.State.Down);
+ mshostStatus.setLastStop(new Date());
Review comment:
remove 4 spaces ?
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -564,11 +657,352 @@ protected Point creteInfluxDbPoint(Object metricsObject) {
}
}
+ Gson gson;
+
+ class ManagementServerCollector extends AbstractStatsCollector {
+ @Override
+ protected void runInContext() {
+ LOGGER.debug(String.format("%s is running...", this.getClass().getSimpleName()));
+ long msid = ManagementServerNode.getManagementServerId();
+ ManagementServerHostVO mshost = null;
+ ManagementServerHostStatsEntry hostStatsEntry = null;
+ try {
+ mshost = managementServerHostDao.findByMsid(msid);
+ // get local data
+ hostStatsEntry = getDataFrom(mshost);
+ managementServerHostStats.put(mshost.getUuid(), hostStatsEntry);
+ // send to other hosts
+ clusterManager.publishStatus(gson.toJson(hostStatsEntry));
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to retrieve host stats", t);
+ }
+ try {
+ // send to DB
+ storeStatus(hostStatsEntry, mshost);
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to store host state", t);
+ }
+ }
+
+ private void storeStatus(ManagementServerHostStatsEntry hostStatsEntry, ManagementServerHostVO mshost) {
+ if (hostStatsEntry == null || mshost == null) {
+ return;
+ }
+ ManagementServerStatusVO msStats = managementServerStatusDao.findByMsId(hostStatsEntry.getManagementServerHostUuid());
+ if (msStats == null) {
+ LOGGER.info(String.format("creating new status info record for host %s - %s",
+ mshost.getName(),
+ hostStatsEntry.getManagementServerHostUuid()));
+ msStats = new ManagementServerStatusVO();
+ msStats.setMsId(hostStatsEntry.getManagementServerHostUuid());
+ }
+ msStats.setOsDistribution(hostStatsEntry.getOsDistribution()); // for now just the bunch details come later
+ msStats.setJavaName(hostStatsEntry.getJvmVendor());
+ msStats.setJavaVersion(hostStatsEntry.getJvmVersion());
+ Date startTime = new Date(hostStatsEntry.getStartTime());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("reporting starttime %s", startTime));
+ }
+ msStats.setLastStart(startTime);
+ msStats.setUpdated(new Date());
+ managementServerStatusDao.persist(msStats);
+ }
+
+ @NotNull
+ private ManagementServerHostStatsEntry getDataFrom(ManagementServerHostVO mshost) {
+ ManagementServerHostStatsEntry newEntry = new ManagementServerHostStatsEntry();
+ LOGGER.debug("Metrics collection start...");
+ newEntry.setManagementServerHostId(mshost.getId());
+ newEntry.setManagementServerHostUuid(mshost.getUuid());
+ retrieveSession(newEntry);
+ getJvmDimensions(newEntry);
+ LOGGER.debug("Metrics collection extra...");
+ getRuntimeData(newEntry);
+ getCpuData(newEntry);
+ getMemoryData(newEntry);
+ // newEntry must now include a pid!
+ getProcFsData(newEntry);
+ getFsData(newEntry);
+ getDataBaseStatistics(newEntry, mshost.getMsid());
+ gatherAllMetrics(newEntry);
+ LOGGER.debug("Metrics collection end!");
+ return newEntry;
+ }
+
+ private void retrieveSession(ManagementServerHostStatsEntry newEntry) {
+ long sessions = ApiSessionListener.getSessionCount();
+ newEntry.setSessions(sessions);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("Sessions found in Api %d vs context %d", sessions,ApiSessionListener.getNumberOfSessions()));
+ } else {
+ LOGGER.debug("Sessions active: " + sessions);
+ }
+ }
+
+ private void getDataBaseStatistics(ManagementServerHostStatsEntry newEntry, long msid) {
+ int count = _hostDao.countByMs(msid);
+ newEntry.setAgentCount(count);
+ }
+
+ private void getMemoryData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ MemoryMXBean mxBean = ManagementFactory.getMemoryMXBean();
+ newEntry.setTotalInit(mxBean.getHeapMemoryUsage().getInit() + mxBean.getNonHeapMemoryUsage().getInit());
+ newEntry.setTotalUsed(mxBean.getHeapMemoryUsage().getUsed() + mxBean.getNonHeapMemoryUsage().getUsed());
+ newEntry.setTotalMax(mxBean.getHeapMemoryUsage().getMax() + mxBean.getNonHeapMemoryUsage().getMax());
+ newEntry.setTotalCommitted(mxBean.getHeapMemoryUsage().getCommitted() + mxBean.getNonHeapMemoryUsage().getCommitted());
+ }
+
+ private void getCpuData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final OperatingSystemMXBean mxBean = ManagementFactory.getOperatingSystemMXBean();
+ newEntry.setAvailableProcessors(mxBean.getAvailableProcessors());
+ newEntry.setLoadAverage(mxBean.getSystemLoadAverage());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics processors - %d , loadavg - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getLoadAverage()));
+ }
+ }
+
+ private void getRuntimeData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final RuntimeMXBean mxBean = ManagementFactory.getRuntimeMXBean();
+ newEntry.setUptime(mxBean.getUptime());
+ newEntry.setStartTime(mxBean.getStartTime());
+ newEntry.setProcessId(mxBean.getPid());
+ newEntry.setJvmName(mxBean.getName());
+ newEntry.setJvmVendor(mxBean.getVmVendor());
+ newEntry.setJvmVersion(mxBean.getVmVersion());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics uptime - %d , starttime - %d",
+ newEntry.getUptime(),
+ newEntry.getStartTime()));
+ }
+ }
+
+ private void getJvmDimensions(@NotNull ManagementServerHostStatsEntry newEntry) {
+ Runtime runtime = Runtime.getRuntime();
+ newEntry.setTotalMemoryBytes(runtime.totalMemory());
+ newEntry.setFreeMemoryBytes(runtime.freeMemory());
+ newEntry.setAvailableProcessors(runtime.availableProcessors());
+ newEntry.setTotalMax(runtime.maxMemory());
+ //long maxMem = runtime.maxMemory();
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics proc - %d , maxMem - %d , totalMemory - %d , freeMemory - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getTotalMax(),
+ newEntry.getTotalMemoryBytes(),
+ newEntry.getFreeMemoryBytes()));
+ }
+ }
+
+ /**
+ * As for data from outside the JVM, we only rely on /proc/ contained data.
+ *
+ * @param newEntry
+ */
+ private void getProcFsData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ String OS = Script.runSimpleBashScript("cat /proc/version");
Review comment:
or OperatingSystemMXBean.getVersion() ?
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/response/ManagementServerMetricsResponse.java
##########
@@ -0,0 +1,177 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.response;
+
+import com.cloud.serializer.Param;
+import com.google.gson.annotations.SerializedName;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.api.response.ManagementServerResponse;
+
+public class ManagementServerMetricsResponse extends ManagementServerResponse {
+
+ @SerializedName(ApiConstants.AVAILABLE_PROCESSORS)
+ @Param(description = "the number of processors available to the JVM")
+ private int availableProcessors;
Review comment:
more info with processors, vendor/model/freq ? just a suggestion.
##########
File path: ui/.env.remote
##########
@@ -0,0 +1 @@
+CS_URL=http://10.0.32.234:8080
Review comment:
revert the changes with these files ?
ui/.env.remote
ui/package-lock.json
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -564,11 +657,352 @@ protected Point creteInfluxDbPoint(Object metricsObject) {
}
}
+ Gson gson;
+
+ class ManagementServerCollector extends AbstractStatsCollector {
+ @Override
+ protected void runInContext() {
+ LOGGER.debug(String.format("%s is running...", this.getClass().getSimpleName()));
+ long msid = ManagementServerNode.getManagementServerId();
+ ManagementServerHostVO mshost = null;
+ ManagementServerHostStatsEntry hostStatsEntry = null;
+ try {
+ mshost = managementServerHostDao.findByMsid(msid);
+ // get local data
+ hostStatsEntry = getDataFrom(mshost);
+ managementServerHostStats.put(mshost.getUuid(), hostStatsEntry);
+ // send to other hosts
+ clusterManager.publishStatus(gson.toJson(hostStatsEntry));
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to retrieve host stats", t);
+ }
+ try {
+ // send to DB
+ storeStatus(hostStatsEntry, mshost);
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to store host state", t);
+ }
+ }
+
+ private void storeStatus(ManagementServerHostStatsEntry hostStatsEntry, ManagementServerHostVO mshost) {
+ if (hostStatsEntry == null || mshost == null) {
+ return;
+ }
+ ManagementServerStatusVO msStats = managementServerStatusDao.findByMsId(hostStatsEntry.getManagementServerHostUuid());
+ if (msStats == null) {
+ LOGGER.info(String.format("creating new status info record for host %s - %s",
+ mshost.getName(),
+ hostStatsEntry.getManagementServerHostUuid()));
+ msStats = new ManagementServerStatusVO();
+ msStats.setMsId(hostStatsEntry.getManagementServerHostUuid());
+ }
+ msStats.setOsDistribution(hostStatsEntry.getOsDistribution()); // for now just the bunch details come later
+ msStats.setJavaName(hostStatsEntry.getJvmVendor());
+ msStats.setJavaVersion(hostStatsEntry.getJvmVersion());
+ Date startTime = new Date(hostStatsEntry.getStartTime());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("reporting starttime %s", startTime));
+ }
+ msStats.setLastStart(startTime);
+ msStats.setUpdated(new Date());
+ managementServerStatusDao.persist(msStats);
+ }
+
+ @NotNull
+ private ManagementServerHostStatsEntry getDataFrom(ManagementServerHostVO mshost) {
+ ManagementServerHostStatsEntry newEntry = new ManagementServerHostStatsEntry();
+ LOGGER.debug("Metrics collection start...");
+ newEntry.setManagementServerHostId(mshost.getId());
+ newEntry.setManagementServerHostUuid(mshost.getUuid());
+ retrieveSession(newEntry);
+ getJvmDimensions(newEntry);
+ LOGGER.debug("Metrics collection extra...");
+ getRuntimeData(newEntry);
+ getCpuData(newEntry);
+ getMemoryData(newEntry);
+ // newEntry must now include a pid!
+ getProcFsData(newEntry);
+ getFsData(newEntry);
+ getDataBaseStatistics(newEntry, mshost.getMsid());
+ gatherAllMetrics(newEntry);
+ LOGGER.debug("Metrics collection end!");
+ return newEntry;
+ }
+
+ private void retrieveSession(ManagementServerHostStatsEntry newEntry) {
+ long sessions = ApiSessionListener.getSessionCount();
+ newEntry.setSessions(sessions);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("Sessions found in Api %d vs context %d", sessions,ApiSessionListener.getNumberOfSessions()));
+ } else {
+ LOGGER.debug("Sessions active: " + sessions);
+ }
+ }
+
+ private void getDataBaseStatistics(ManagementServerHostStatsEntry newEntry, long msid) {
+ int count = _hostDao.countByMs(msid);
+ newEntry.setAgentCount(count);
+ }
+
+ private void getMemoryData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ MemoryMXBean mxBean = ManagementFactory.getMemoryMXBean();
+ newEntry.setTotalInit(mxBean.getHeapMemoryUsage().getInit() + mxBean.getNonHeapMemoryUsage().getInit());
+ newEntry.setTotalUsed(mxBean.getHeapMemoryUsage().getUsed() + mxBean.getNonHeapMemoryUsage().getUsed());
+ newEntry.setTotalMax(mxBean.getHeapMemoryUsage().getMax() + mxBean.getNonHeapMemoryUsage().getMax());
+ newEntry.setTotalCommitted(mxBean.getHeapMemoryUsage().getCommitted() + mxBean.getNonHeapMemoryUsage().getCommitted());
+ }
+
+ private void getCpuData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final OperatingSystemMXBean mxBean = ManagementFactory.getOperatingSystemMXBean();
+ newEntry.setAvailableProcessors(mxBean.getAvailableProcessors());
+ newEntry.setLoadAverage(mxBean.getSystemLoadAverage());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics processors - %d , loadavg - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getLoadAverage()));
+ }
+ }
+
+ private void getRuntimeData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final RuntimeMXBean mxBean = ManagementFactory.getRuntimeMXBean();
+ newEntry.setUptime(mxBean.getUptime());
+ newEntry.setStartTime(mxBean.getStartTime());
+ newEntry.setProcessId(mxBean.getPid());
+ newEntry.setJvmName(mxBean.getName());
+ newEntry.setJvmVendor(mxBean.getVmVendor());
+ newEntry.setJvmVersion(mxBean.getVmVersion());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics uptime - %d , starttime - %d",
+ newEntry.getUptime(),
+ newEntry.getStartTime()));
+ }
+ }
+
+ private void getJvmDimensions(@NotNull ManagementServerHostStatsEntry newEntry) {
+ Runtime runtime = Runtime.getRuntime();
+ newEntry.setTotalMemoryBytes(runtime.totalMemory());
+ newEntry.setFreeMemoryBytes(runtime.freeMemory());
+ newEntry.setAvailableProcessors(runtime.availableProcessors());
+ newEntry.setTotalMax(runtime.maxMemory());
+ //long maxMem = runtime.maxMemory();
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics proc - %d , maxMem - %d , totalMemory - %d , freeMemory - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getTotalMax(),
+ newEntry.getTotalMemoryBytes(),
+ newEntry.getFreeMemoryBytes()));
+ }
+ }
+
+ /**
+ * As for data from outside the JVM, we only rely on /proc/ contained data.
+ *
+ * @param newEntry
+ */
+ private void getProcFsData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ String OS = Script.runSimpleBashScript("cat /proc/version");
+ newEntry.setOsDistribution(OS);
+ String mem = Script.runSimpleBashScript("cat /proc/meminfo | grep MemTotal | cut -f 2 -d ':' | tr -d 'a-zA-z '").trim();
+ newEntry.setSystemMemoryTotal(Long.parseLong(mem) * 1024);
+ String free = Script.runSimpleBashScript("cat /proc/meminfo | grep MemFree | cut -f 2 -d ':' | tr -d 'a-zA-z '").trim();
+ newEntry.setSystemMemoryFree(Long.parseLong(free) * 1024);
+ String used = Script.runSimpleBashScript(String.format("ps -o rss= %d", newEntry.getPid()));
+ newEntry.setSystemMemoryUsed(Long.parseLong(used));
+ String maxuse = Script.runSimpleBashScript(String.format("ps -o vsz= %d", newEntry.getPid()));
+ newEntry.setSystemMemoryVirtualSize(Long.parseLong(maxuse));
+
+ newEntry.setSystemTotalCpuCycles(getSystemCpuCyclesTotal());
+ newEntry.setSystemLoadAverages(getCpuLoads());
+ newEntry.setSystemCyclesUsage(getSystemCpuUsage(newEntry));
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.debug(
+ String.format("cpu\ncapacities: %f\n loads: %s ; %s ; %s\n stats: %f ; %f ; %f",
+ newEntry.getSystemTotalCpuCycles(),
+ newEntry.getSystemLoadAverages()[0], newEntry.getSystemLoadAverages()[1], newEntry.getSystemLoadAverages()[2],
+ newEntry.getSystemCyclesUsage()[0], newEntry.getSystemCyclesUsage()[1], newEntry.getSystemCyclesUsage()[2]
+ )
+ );
+ }
+ }
+
+ @NotNull
+ private double[] getCpuLoads() {
+ String[] cpuloadString = Script.runSimpleBashScript("cat /proc/loadavg").split(" ");
+ double[] cpuloads = {Double.parseDouble(cpuloadString[0]), Double.parseDouble(cpuloadString[1]), Double.parseDouble(cpuloadString[2])};
+ return cpuloads;
+ }
+
+ private double [] getSystemCpuUsage(@NotNull ManagementServerHostStatsEntry newEntry) {
+ String[] cpustats = Script.runSimpleBashScript("cat /proc/stat | grep \"cpu \" | tr -d \"cpu\"").trim().split(" ");
+ double [] cycleUsage = {Double.parseDouble(cpustats[0]) + Double.parseDouble(cpustats[1]), Double.parseDouble(cpustats[2]), Double.parseDouble(cpustats[3])};
+ return cycleUsage;
+ }
+
+ private double getSystemCpuCyclesTotal() {
+ String cpucaps = Script.runSimpleBashScript("cat /proc/cpuinfo | grep \"cpu MHz\" | grep \"cpu MHz\" | cut -f 2 -d : | tr -d ' '| tr '\\n' \" \"");
+ double totalcpucap = 0;
+ for (String cpucap : cpucaps.split(" ")) {
+ totalcpucap += Double.parseDouble(cpucap);
+ }
+ return totalcpucap;
+ }
+
+ private void getFsData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ Set<String> logFileNames = LogUtils.getLogFileNames();
+ StringBuilder logInfoBuilder = new StringBuilder();
+ for (String fileName : logFileNames) {
+ String du = Script.runSimpleBashScript(String.format("du -sk %s", fileName));
+ String df = Script.runSimpleBashScript(String.format("df -k %s | grep -v Filesystem", fileName));
+ logInfoBuilder.append(fileName).append('\n').append("usage: ").append(du).append('\n').append("disk :").append(df);
Review comment:
in kilobytes ?
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -564,11 +657,352 @@ protected Point creteInfluxDbPoint(Object metricsObject) {
}
}
+ Gson gson;
+
+ class ManagementServerCollector extends AbstractStatsCollector {
+ @Override
+ protected void runInContext() {
+ LOGGER.debug(String.format("%s is running...", this.getClass().getSimpleName()));
+ long msid = ManagementServerNode.getManagementServerId();
+ ManagementServerHostVO mshost = null;
+ ManagementServerHostStatsEntry hostStatsEntry = null;
+ try {
+ mshost = managementServerHostDao.findByMsid(msid);
+ // get local data
+ hostStatsEntry = getDataFrom(mshost);
+ managementServerHostStats.put(mshost.getUuid(), hostStatsEntry);
+ // send to other hosts
+ clusterManager.publishStatus(gson.toJson(hostStatsEntry));
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to retrieve host stats", t);
+ }
+ try {
+ // send to DB
+ storeStatus(hostStatsEntry, mshost);
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to store host state", t);
+ }
+ }
+
+ private void storeStatus(ManagementServerHostStatsEntry hostStatsEntry, ManagementServerHostVO mshost) {
+ if (hostStatsEntry == null || mshost == null) {
+ return;
+ }
+ ManagementServerStatusVO msStats = managementServerStatusDao.findByMsId(hostStatsEntry.getManagementServerHostUuid());
+ if (msStats == null) {
+ LOGGER.info(String.format("creating new status info record for host %s - %s",
+ mshost.getName(),
+ hostStatsEntry.getManagementServerHostUuid()));
+ msStats = new ManagementServerStatusVO();
+ msStats.setMsId(hostStatsEntry.getManagementServerHostUuid());
+ }
+ msStats.setOsDistribution(hostStatsEntry.getOsDistribution()); // for now just the bunch details come later
+ msStats.setJavaName(hostStatsEntry.getJvmVendor());
+ msStats.setJavaVersion(hostStatsEntry.getJvmVersion());
+ Date startTime = new Date(hostStatsEntry.getStartTime());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("reporting starttime %s", startTime));
+ }
+ msStats.setLastStart(startTime);
+ msStats.setUpdated(new Date());
+ managementServerStatusDao.persist(msStats);
+ }
+
+ @NotNull
+ private ManagementServerHostStatsEntry getDataFrom(ManagementServerHostVO mshost) {
+ ManagementServerHostStatsEntry newEntry = new ManagementServerHostStatsEntry();
+ LOGGER.debug("Metrics collection start...");
+ newEntry.setManagementServerHostId(mshost.getId());
+ newEntry.setManagementServerHostUuid(mshost.getUuid());
+ retrieveSession(newEntry);
+ getJvmDimensions(newEntry);
+ LOGGER.debug("Metrics collection extra...");
+ getRuntimeData(newEntry);
+ getCpuData(newEntry);
+ getMemoryData(newEntry);
+ // newEntry must now include a pid!
+ getProcFsData(newEntry);
+ getFsData(newEntry);
+ getDataBaseStatistics(newEntry, mshost.getMsid());
+ gatherAllMetrics(newEntry);
+ LOGGER.debug("Metrics collection end!");
+ return newEntry;
+ }
+
+ private void retrieveSession(ManagementServerHostStatsEntry newEntry) {
+ long sessions = ApiSessionListener.getSessionCount();
+ newEntry.setSessions(sessions);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("Sessions found in Api %d vs context %d", sessions,ApiSessionListener.getNumberOfSessions()));
+ } else {
+ LOGGER.debug("Sessions active: " + sessions);
+ }
+ }
+
+ private void getDataBaseStatistics(ManagementServerHostStatsEntry newEntry, long msid) {
+ int count = _hostDao.countByMs(msid);
+ newEntry.setAgentCount(count);
+ }
+
+ private void getMemoryData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ MemoryMXBean mxBean = ManagementFactory.getMemoryMXBean();
+ newEntry.setTotalInit(mxBean.getHeapMemoryUsage().getInit() + mxBean.getNonHeapMemoryUsage().getInit());
+ newEntry.setTotalUsed(mxBean.getHeapMemoryUsage().getUsed() + mxBean.getNonHeapMemoryUsage().getUsed());
+ newEntry.setTotalMax(mxBean.getHeapMemoryUsage().getMax() + mxBean.getNonHeapMemoryUsage().getMax());
+ newEntry.setTotalCommitted(mxBean.getHeapMemoryUsage().getCommitted() + mxBean.getNonHeapMemoryUsage().getCommitted());
+ }
+
+ private void getCpuData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final OperatingSystemMXBean mxBean = ManagementFactory.getOperatingSystemMXBean();
+ newEntry.setAvailableProcessors(mxBean.getAvailableProcessors());
+ newEntry.setLoadAverage(mxBean.getSystemLoadAverage());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics processors - %d , loadavg - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getLoadAverage()));
+ }
+ }
+
+ private void getRuntimeData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final RuntimeMXBean mxBean = ManagementFactory.getRuntimeMXBean();
+ newEntry.setUptime(mxBean.getUptime());
+ newEntry.setStartTime(mxBean.getStartTime());
+ newEntry.setProcessId(mxBean.getPid());
+ newEntry.setJvmName(mxBean.getName());
+ newEntry.setJvmVendor(mxBean.getVmVendor());
+ newEntry.setJvmVersion(mxBean.getVmVersion());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics uptime - %d , starttime - %d",
+ newEntry.getUptime(),
+ newEntry.getStartTime()));
+ }
+ }
+
+ private void getJvmDimensions(@NotNull ManagementServerHostStatsEntry newEntry) {
+ Runtime runtime = Runtime.getRuntime();
+ newEntry.setTotalMemoryBytes(runtime.totalMemory());
+ newEntry.setFreeMemoryBytes(runtime.freeMemory());
+ newEntry.setAvailableProcessors(runtime.availableProcessors());
+ newEntry.setTotalMax(runtime.maxMemory());
+ //long maxMem = runtime.maxMemory();
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics proc - %d , maxMem - %d , totalMemory - %d , freeMemory - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getTotalMax(),
+ newEntry.getTotalMemoryBytes(),
+ newEntry.getFreeMemoryBytes()));
+ }
+ }
+
+ /**
+ * As for data from outside the JVM, we only rely on /proc/ contained data.
+ *
+ * @param newEntry
+ */
+ private void getProcFsData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ String OS = Script.runSimpleBashScript("cat /proc/version");
+ newEntry.setOsDistribution(OS);
+ String mem = Script.runSimpleBashScript("cat /proc/meminfo | grep MemTotal | cut -f 2 -d ':' | tr -d 'a-zA-z '").trim();
+ newEntry.setSystemMemoryTotal(Long.parseLong(mem) * 1024);
+ String free = Script.runSimpleBashScript("cat /proc/meminfo | grep MemFree | cut -f 2 -d ':' | tr -d 'a-zA-z '").trim();
+ newEntry.setSystemMemoryFree(Long.parseLong(free) * 1024);
Review comment:
OperatingSystemMXBean.getFreePhysicalMemorySize() ?
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -564,11 +657,352 @@ protected Point creteInfluxDbPoint(Object metricsObject) {
}
}
+ Gson gson;
+
+ class ManagementServerCollector extends AbstractStatsCollector {
+ @Override
+ protected void runInContext() {
+ LOGGER.debug(String.format("%s is running...", this.getClass().getSimpleName()));
+ long msid = ManagementServerNode.getManagementServerId();
+ ManagementServerHostVO mshost = null;
+ ManagementServerHostStatsEntry hostStatsEntry = null;
+ try {
+ mshost = managementServerHostDao.findByMsid(msid);
+ // get local data
+ hostStatsEntry = getDataFrom(mshost);
+ managementServerHostStats.put(mshost.getUuid(), hostStatsEntry);
+ // send to other hosts
+ clusterManager.publishStatus(gson.toJson(hostStatsEntry));
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to retrieve host stats", t);
+ }
+ try {
+ // send to DB
+ storeStatus(hostStatsEntry, mshost);
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to store host state", t);
+ }
+ }
+
+ private void storeStatus(ManagementServerHostStatsEntry hostStatsEntry, ManagementServerHostVO mshost) {
+ if (hostStatsEntry == null || mshost == null) {
+ return;
+ }
+ ManagementServerStatusVO msStats = managementServerStatusDao.findByMsId(hostStatsEntry.getManagementServerHostUuid());
+ if (msStats == null) {
+ LOGGER.info(String.format("creating new status info record for host %s - %s",
+ mshost.getName(),
+ hostStatsEntry.getManagementServerHostUuid()));
+ msStats = new ManagementServerStatusVO();
+ msStats.setMsId(hostStatsEntry.getManagementServerHostUuid());
+ }
+ msStats.setOsDistribution(hostStatsEntry.getOsDistribution()); // for now just the bunch details come later
+ msStats.setJavaName(hostStatsEntry.getJvmVendor());
+ msStats.setJavaVersion(hostStatsEntry.getJvmVersion());
+ Date startTime = new Date(hostStatsEntry.getStartTime());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("reporting starttime %s", startTime));
+ }
+ msStats.setLastStart(startTime);
+ msStats.setUpdated(new Date());
+ managementServerStatusDao.persist(msStats);
+ }
+
+ @NotNull
+ private ManagementServerHostStatsEntry getDataFrom(ManagementServerHostVO mshost) {
+ ManagementServerHostStatsEntry newEntry = new ManagementServerHostStatsEntry();
+ LOGGER.debug("Metrics collection start...");
+ newEntry.setManagementServerHostId(mshost.getId());
+ newEntry.setManagementServerHostUuid(mshost.getUuid());
+ retrieveSession(newEntry);
+ getJvmDimensions(newEntry);
+ LOGGER.debug("Metrics collection extra...");
+ getRuntimeData(newEntry);
+ getCpuData(newEntry);
+ getMemoryData(newEntry);
+ // newEntry must now include a pid!
+ getProcFsData(newEntry);
+ getFsData(newEntry);
+ getDataBaseStatistics(newEntry, mshost.getMsid());
+ gatherAllMetrics(newEntry);
+ LOGGER.debug("Metrics collection end!");
+ return newEntry;
+ }
+
+ private void retrieveSession(ManagementServerHostStatsEntry newEntry) {
+ long sessions = ApiSessionListener.getSessionCount();
+ newEntry.setSessions(sessions);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("Sessions found in Api %d vs context %d", sessions,ApiSessionListener.getNumberOfSessions()));
+ } else {
+ LOGGER.debug("Sessions active: " + sessions);
+ }
+ }
+
+ private void getDataBaseStatistics(ManagementServerHostStatsEntry newEntry, long msid) {
+ int count = _hostDao.countByMs(msid);
+ newEntry.setAgentCount(count);
+ }
+
+ private void getMemoryData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ MemoryMXBean mxBean = ManagementFactory.getMemoryMXBean();
+ newEntry.setTotalInit(mxBean.getHeapMemoryUsage().getInit() + mxBean.getNonHeapMemoryUsage().getInit());
+ newEntry.setTotalUsed(mxBean.getHeapMemoryUsage().getUsed() + mxBean.getNonHeapMemoryUsage().getUsed());
+ newEntry.setTotalMax(mxBean.getHeapMemoryUsage().getMax() + mxBean.getNonHeapMemoryUsage().getMax());
+ newEntry.setTotalCommitted(mxBean.getHeapMemoryUsage().getCommitted() + mxBean.getNonHeapMemoryUsage().getCommitted());
+ }
+
+ private void getCpuData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final OperatingSystemMXBean mxBean = ManagementFactory.getOperatingSystemMXBean();
+ newEntry.setAvailableProcessors(mxBean.getAvailableProcessors());
+ newEntry.setLoadAverage(mxBean.getSystemLoadAverage());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics processors - %d , loadavg - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getLoadAverage()));
+ }
+ }
+
+ private void getRuntimeData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final RuntimeMXBean mxBean = ManagementFactory.getRuntimeMXBean();
+ newEntry.setUptime(mxBean.getUptime());
+ newEntry.setStartTime(mxBean.getStartTime());
+ newEntry.setProcessId(mxBean.getPid());
+ newEntry.setJvmName(mxBean.getName());
+ newEntry.setJvmVendor(mxBean.getVmVendor());
+ newEntry.setJvmVersion(mxBean.getVmVersion());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics uptime - %d , starttime - %d",
+ newEntry.getUptime(),
+ newEntry.getStartTime()));
+ }
+ }
+
+ private void getJvmDimensions(@NotNull ManagementServerHostStatsEntry newEntry) {
+ Runtime runtime = Runtime.getRuntime();
+ newEntry.setTotalMemoryBytes(runtime.totalMemory());
+ newEntry.setFreeMemoryBytes(runtime.freeMemory());
+ newEntry.setAvailableProcessors(runtime.availableProcessors());
+ newEntry.setTotalMax(runtime.maxMemory());
+ //long maxMem = runtime.maxMemory();
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics proc - %d , maxMem - %d , totalMemory - %d , freeMemory - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getTotalMax(),
+ newEntry.getTotalMemoryBytes(),
+ newEntry.getFreeMemoryBytes()));
+ }
+ }
+
+ /**
+ * As for data from outside the JVM, we only rely on /proc/ contained data.
+ *
+ * @param newEntry
+ */
+ private void getProcFsData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ String OS = Script.runSimpleBashScript("cat /proc/version");
Review comment:
it might be better to get os from '/etc/os-release'
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -564,11 +657,352 @@ protected Point creteInfluxDbPoint(Object metricsObject) {
}
}
+ Gson gson;
+
+ class ManagementServerCollector extends AbstractStatsCollector {
+ @Override
+ protected void runInContext() {
+ LOGGER.debug(String.format("%s is running...", this.getClass().getSimpleName()));
+ long msid = ManagementServerNode.getManagementServerId();
+ ManagementServerHostVO mshost = null;
+ ManagementServerHostStatsEntry hostStatsEntry = null;
+ try {
+ mshost = managementServerHostDao.findByMsid(msid);
+ // get local data
+ hostStatsEntry = getDataFrom(mshost);
+ managementServerHostStats.put(mshost.getUuid(), hostStatsEntry);
+ // send to other hosts
+ clusterManager.publishStatus(gson.toJson(hostStatsEntry));
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to retrieve host stats", t);
+ }
+ try {
+ // send to DB
+ storeStatus(hostStatsEntry, mshost);
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to store host state", t);
+ }
+ }
+
+ private void storeStatus(ManagementServerHostStatsEntry hostStatsEntry, ManagementServerHostVO mshost) {
+ if (hostStatsEntry == null || mshost == null) {
+ return;
+ }
+ ManagementServerStatusVO msStats = managementServerStatusDao.findByMsId(hostStatsEntry.getManagementServerHostUuid());
+ if (msStats == null) {
+ LOGGER.info(String.format("creating new status info record for host %s - %s",
+ mshost.getName(),
+ hostStatsEntry.getManagementServerHostUuid()));
+ msStats = new ManagementServerStatusVO();
+ msStats.setMsId(hostStatsEntry.getManagementServerHostUuid());
+ }
+ msStats.setOsDistribution(hostStatsEntry.getOsDistribution()); // for now just the bunch details come later
+ msStats.setJavaName(hostStatsEntry.getJvmVendor());
+ msStats.setJavaVersion(hostStatsEntry.getJvmVersion());
+ Date startTime = new Date(hostStatsEntry.getStartTime());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("reporting starttime %s", startTime));
+ }
+ msStats.setLastStart(startTime);
+ msStats.setUpdated(new Date());
+ managementServerStatusDao.persist(msStats);
+ }
+
+ @NotNull
+ private ManagementServerHostStatsEntry getDataFrom(ManagementServerHostVO mshost) {
+ ManagementServerHostStatsEntry newEntry = new ManagementServerHostStatsEntry();
+ LOGGER.debug("Metrics collection start...");
+ newEntry.setManagementServerHostId(mshost.getId());
+ newEntry.setManagementServerHostUuid(mshost.getUuid());
+ retrieveSession(newEntry);
+ getJvmDimensions(newEntry);
+ LOGGER.debug("Metrics collection extra...");
+ getRuntimeData(newEntry);
+ getCpuData(newEntry);
+ getMemoryData(newEntry);
+ // newEntry must now include a pid!
+ getProcFsData(newEntry);
+ getFsData(newEntry);
+ getDataBaseStatistics(newEntry, mshost.getMsid());
+ gatherAllMetrics(newEntry);
+ LOGGER.debug("Metrics collection end!");
+ return newEntry;
+ }
+
+ private void retrieveSession(ManagementServerHostStatsEntry newEntry) {
+ long sessions = ApiSessionListener.getSessionCount();
+ newEntry.setSessions(sessions);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("Sessions found in Api %d vs context %d", sessions,ApiSessionListener.getNumberOfSessions()));
+ } else {
+ LOGGER.debug("Sessions active: " + sessions);
+ }
+ }
+
+ private void getDataBaseStatistics(ManagementServerHostStatsEntry newEntry, long msid) {
+ int count = _hostDao.countByMs(msid);
+ newEntry.setAgentCount(count);
+ }
+
+ private void getMemoryData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ MemoryMXBean mxBean = ManagementFactory.getMemoryMXBean();
+ newEntry.setTotalInit(mxBean.getHeapMemoryUsage().getInit() + mxBean.getNonHeapMemoryUsage().getInit());
+ newEntry.setTotalUsed(mxBean.getHeapMemoryUsage().getUsed() + mxBean.getNonHeapMemoryUsage().getUsed());
+ newEntry.setTotalMax(mxBean.getHeapMemoryUsage().getMax() + mxBean.getNonHeapMemoryUsage().getMax());
+ newEntry.setTotalCommitted(mxBean.getHeapMemoryUsage().getCommitted() + mxBean.getNonHeapMemoryUsage().getCommitted());
+ }
+
+ private void getCpuData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final OperatingSystemMXBean mxBean = ManagementFactory.getOperatingSystemMXBean();
+ newEntry.setAvailableProcessors(mxBean.getAvailableProcessors());
+ newEntry.setLoadAverage(mxBean.getSystemLoadAverage());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics processors - %d , loadavg - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getLoadAverage()));
+ }
+ }
+
+ private void getRuntimeData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final RuntimeMXBean mxBean = ManagementFactory.getRuntimeMXBean();
+ newEntry.setUptime(mxBean.getUptime());
+ newEntry.setStartTime(mxBean.getStartTime());
+ newEntry.setProcessId(mxBean.getPid());
+ newEntry.setJvmName(mxBean.getName());
+ newEntry.setJvmVendor(mxBean.getVmVendor());
+ newEntry.setJvmVersion(mxBean.getVmVersion());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics uptime - %d , starttime - %d",
+ newEntry.getUptime(),
+ newEntry.getStartTime()));
+ }
+ }
+
+ private void getJvmDimensions(@NotNull ManagementServerHostStatsEntry newEntry) {
+ Runtime runtime = Runtime.getRuntime();
+ newEntry.setTotalMemoryBytes(runtime.totalMemory());
+ newEntry.setFreeMemoryBytes(runtime.freeMemory());
+ newEntry.setAvailableProcessors(runtime.availableProcessors());
+ newEntry.setTotalMax(runtime.maxMemory());
+ //long maxMem = runtime.maxMemory();
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics proc - %d , maxMem - %d , totalMemory - %d , freeMemory - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getTotalMax(),
+ newEntry.getTotalMemoryBytes(),
+ newEntry.getFreeMemoryBytes()));
+ }
+ }
+
+ /**
+ * As for data from outside the JVM, we only rely on /proc/ contained data.
+ *
+ * @param newEntry
+ */
+ private void getProcFsData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ String OS = Script.runSimpleBashScript("cat /proc/version");
+ newEntry.setOsDistribution(OS);
+ String mem = Script.runSimpleBashScript("cat /proc/meminfo | grep MemTotal | cut -f 2 -d ':' | tr -d 'a-zA-z '").trim();
Review comment:
OperatingSystemMXBean.getTotalPhysicalMemorySize ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r752060696
##########
File path: ui/.env.remote
##########
@@ -0,0 +1 @@
+CS_URL=http://10.0.32.234:8080
Review comment:
The .env.remote was an accident, I will remove.
The package-lock.json file should be updated at times , right? It will not build without it, it seems. No problem to keep it out of scope here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r752094566
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -564,11 +657,352 @@ protected Point creteInfluxDbPoint(Object metricsObject) {
}
}
+ Gson gson;
+
+ class ManagementServerCollector extends AbstractStatsCollector {
+ @Override
+ protected void runInContext() {
+ LOGGER.debug(String.format("%s is running...", this.getClass().getSimpleName()));
+ long msid = ManagementServerNode.getManagementServerId();
+ ManagementServerHostVO mshost = null;
+ ManagementServerHostStatsEntry hostStatsEntry = null;
+ try {
+ mshost = managementServerHostDao.findByMsid(msid);
+ // get local data
+ hostStatsEntry = getDataFrom(mshost);
+ managementServerHostStats.put(mshost.getUuid(), hostStatsEntry);
+ // send to other hosts
+ clusterManager.publishStatus(gson.toJson(hostStatsEntry));
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to retrieve host stats", t);
+ }
+ try {
+ // send to DB
+ storeStatus(hostStatsEntry, mshost);
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to store host state", t);
+ }
+ }
+
+ private void storeStatus(ManagementServerHostStatsEntry hostStatsEntry, ManagementServerHostVO mshost) {
+ if (hostStatsEntry == null || mshost == null) {
+ return;
+ }
+ ManagementServerStatusVO msStats = managementServerStatusDao.findByMsId(hostStatsEntry.getManagementServerHostUuid());
+ if (msStats == null) {
+ LOGGER.info(String.format("creating new status info record for host %s - %s",
+ mshost.getName(),
+ hostStatsEntry.getManagementServerHostUuid()));
+ msStats = new ManagementServerStatusVO();
+ msStats.setMsId(hostStatsEntry.getManagementServerHostUuid());
+ }
+ msStats.setOsDistribution(hostStatsEntry.getOsDistribution()); // for now just the bunch details come later
+ msStats.setJavaName(hostStatsEntry.getJvmVendor());
+ msStats.setJavaVersion(hostStatsEntry.getJvmVersion());
+ Date startTime = new Date(hostStatsEntry.getStartTime());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("reporting starttime %s", startTime));
+ }
+ msStats.setLastStart(startTime);
+ msStats.setUpdated(new Date());
+ managementServerStatusDao.persist(msStats);
+ }
+
+ @NotNull
+ private ManagementServerHostStatsEntry getDataFrom(ManagementServerHostVO mshost) {
+ ManagementServerHostStatsEntry newEntry = new ManagementServerHostStatsEntry();
+ LOGGER.debug("Metrics collection start...");
+ newEntry.setManagementServerHostId(mshost.getId());
+ newEntry.setManagementServerHostUuid(mshost.getUuid());
+ retrieveSession(newEntry);
+ getJvmDimensions(newEntry);
+ LOGGER.debug("Metrics collection extra...");
+ getRuntimeData(newEntry);
+ getCpuData(newEntry);
+ getMemoryData(newEntry);
+ // newEntry must now include a pid!
+ getProcFsData(newEntry);
+ getFsData(newEntry);
+ getDataBaseStatistics(newEntry, mshost.getMsid());
+ gatherAllMetrics(newEntry);
+ LOGGER.debug("Metrics collection end!");
+ return newEntry;
+ }
+
+ private void retrieveSession(ManagementServerHostStatsEntry newEntry) {
+ long sessions = ApiSessionListener.getSessionCount();
+ newEntry.setSessions(sessions);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("Sessions found in Api %d vs context %d", sessions,ApiSessionListener.getNumberOfSessions()));
+ } else {
+ LOGGER.debug("Sessions active: " + sessions);
+ }
+ }
+
+ private void getDataBaseStatistics(ManagementServerHostStatsEntry newEntry, long msid) {
+ int count = _hostDao.countByMs(msid);
+ newEntry.setAgentCount(count);
+ }
+
+ private void getMemoryData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ MemoryMXBean mxBean = ManagementFactory.getMemoryMXBean();
+ newEntry.setTotalInit(mxBean.getHeapMemoryUsage().getInit() + mxBean.getNonHeapMemoryUsage().getInit());
+ newEntry.setTotalUsed(mxBean.getHeapMemoryUsage().getUsed() + mxBean.getNonHeapMemoryUsage().getUsed());
+ newEntry.setTotalMax(mxBean.getHeapMemoryUsage().getMax() + mxBean.getNonHeapMemoryUsage().getMax());
+ newEntry.setTotalCommitted(mxBean.getHeapMemoryUsage().getCommitted() + mxBean.getNonHeapMemoryUsage().getCommitted());
+ }
+
+ private void getCpuData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final OperatingSystemMXBean mxBean = ManagementFactory.getOperatingSystemMXBean();
+ newEntry.setAvailableProcessors(mxBean.getAvailableProcessors());
+ newEntry.setLoadAverage(mxBean.getSystemLoadAverage());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics processors - %d , loadavg - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getLoadAverage()));
+ }
+ }
+
+ private void getRuntimeData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final RuntimeMXBean mxBean = ManagementFactory.getRuntimeMXBean();
+ newEntry.setUptime(mxBean.getUptime());
+ newEntry.setStartTime(mxBean.getStartTime());
+ newEntry.setProcessId(mxBean.getPid());
+ newEntry.setJvmName(mxBean.getName());
+ newEntry.setJvmVendor(mxBean.getVmVendor());
+ newEntry.setJvmVersion(mxBean.getVmVersion());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics uptime - %d , starttime - %d",
+ newEntry.getUptime(),
+ newEntry.getStartTime()));
+ }
+ }
+
+ private void getJvmDimensions(@NotNull ManagementServerHostStatsEntry newEntry) {
+ Runtime runtime = Runtime.getRuntime();
+ newEntry.setTotalMemoryBytes(runtime.totalMemory());
+ newEntry.setFreeMemoryBytes(runtime.freeMemory());
+ newEntry.setAvailableProcessors(runtime.availableProcessors());
+ newEntry.setTotalMax(runtime.maxMemory());
+ //long maxMem = runtime.maxMemory();
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics proc - %d , maxMem - %d , totalMemory - %d , freeMemory - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getTotalMax(),
+ newEntry.getTotalMemoryBytes(),
+ newEntry.getFreeMemoryBytes()));
+ }
+ }
+
+ /**
+ * As for data from outside the JVM, we only rely on /proc/ contained data.
+ *
+ * @param newEntry
+ */
+ private void getProcFsData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ String OS = Script.runSimpleBashScript("cat /proc/version");
+ newEntry.setOsDistribution(OS);
+ String mem = Script.runSimpleBashScript("cat /proc/meminfo | grep MemTotal | cut -f 2 -d ':' | tr -d 'a-zA-z '").trim();
+ newEntry.setSystemMemoryTotal(Long.parseLong(mem) * 1024);
+ String free = Script.runSimpleBashScript("cat /proc/meminfo | grep MemFree | cut -f 2 -d ':' | tr -d 'a-zA-z '").trim();
+ newEntry.setSystemMemoryFree(Long.parseLong(free) * 1024);
Review comment:
not in java 11 either
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r752118291
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -564,11 +657,352 @@ protected Point creteInfluxDbPoint(Object metricsObject) {
}
}
+ Gson gson;
+
+ class ManagementServerCollector extends AbstractStatsCollector {
+ @Override
+ protected void runInContext() {
+ LOGGER.debug(String.format("%s is running...", this.getClass().getSimpleName()));
+ long msid = ManagementServerNode.getManagementServerId();
+ ManagementServerHostVO mshost = null;
+ ManagementServerHostStatsEntry hostStatsEntry = null;
+ try {
+ mshost = managementServerHostDao.findByMsid(msid);
+ // get local data
+ hostStatsEntry = getDataFrom(mshost);
+ managementServerHostStats.put(mshost.getUuid(), hostStatsEntry);
+ // send to other hosts
+ clusterManager.publishStatus(gson.toJson(hostStatsEntry));
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to retrieve host stats", t);
+ }
+ try {
+ // send to DB
+ storeStatus(hostStatsEntry, mshost);
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to store host state", t);
+ }
+ }
+
+ private void storeStatus(ManagementServerHostStatsEntry hostStatsEntry, ManagementServerHostVO mshost) {
+ if (hostStatsEntry == null || mshost == null) {
+ return;
+ }
+ ManagementServerStatusVO msStats = managementServerStatusDao.findByMsId(hostStatsEntry.getManagementServerHostUuid());
+ if (msStats == null) {
+ LOGGER.info(String.format("creating new status info record for host %s - %s",
+ mshost.getName(),
+ hostStatsEntry.getManagementServerHostUuid()));
+ msStats = new ManagementServerStatusVO();
+ msStats.setMsId(hostStatsEntry.getManagementServerHostUuid());
+ }
+ msStats.setOsDistribution(hostStatsEntry.getOsDistribution()); // for now just the bunch details come later
+ msStats.setJavaName(hostStatsEntry.getJvmVendor());
+ msStats.setJavaVersion(hostStatsEntry.getJvmVersion());
+ Date startTime = new Date(hostStatsEntry.getStartTime());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("reporting starttime %s", startTime));
+ }
+ msStats.setLastStart(startTime);
+ msStats.setUpdated(new Date());
+ managementServerStatusDao.persist(msStats);
+ }
+
+ @NotNull
+ private ManagementServerHostStatsEntry getDataFrom(ManagementServerHostVO mshost) {
+ ManagementServerHostStatsEntry newEntry = new ManagementServerHostStatsEntry();
+ LOGGER.debug("Metrics collection start...");
+ newEntry.setManagementServerHostId(mshost.getId());
+ newEntry.setManagementServerHostUuid(mshost.getUuid());
+ retrieveSession(newEntry);
+ getJvmDimensions(newEntry);
+ LOGGER.debug("Metrics collection extra...");
+ getRuntimeData(newEntry);
+ getCpuData(newEntry);
+ getMemoryData(newEntry);
+ // newEntry must now include a pid!
+ getProcFsData(newEntry);
+ getFsData(newEntry);
+ getDataBaseStatistics(newEntry, mshost.getMsid());
+ gatherAllMetrics(newEntry);
+ LOGGER.debug("Metrics collection end!");
+ return newEntry;
+ }
+
+ private void retrieveSession(ManagementServerHostStatsEntry newEntry) {
+ long sessions = ApiSessionListener.getSessionCount();
+ newEntry.setSessions(sessions);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("Sessions found in Api %d vs context %d", sessions,ApiSessionListener.getNumberOfSessions()));
+ } else {
+ LOGGER.debug("Sessions active: " + sessions);
+ }
+ }
+
+ private void getDataBaseStatistics(ManagementServerHostStatsEntry newEntry, long msid) {
+ int count = _hostDao.countByMs(msid);
+ newEntry.setAgentCount(count);
+ }
+
+ private void getMemoryData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ MemoryMXBean mxBean = ManagementFactory.getMemoryMXBean();
+ newEntry.setTotalInit(mxBean.getHeapMemoryUsage().getInit() + mxBean.getNonHeapMemoryUsage().getInit());
+ newEntry.setTotalUsed(mxBean.getHeapMemoryUsage().getUsed() + mxBean.getNonHeapMemoryUsage().getUsed());
+ newEntry.setTotalMax(mxBean.getHeapMemoryUsage().getMax() + mxBean.getNonHeapMemoryUsage().getMax());
+ newEntry.setTotalCommitted(mxBean.getHeapMemoryUsage().getCommitted() + mxBean.getNonHeapMemoryUsage().getCommitted());
+ }
+
+ private void getCpuData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final OperatingSystemMXBean mxBean = ManagementFactory.getOperatingSystemMXBean();
+ newEntry.setAvailableProcessors(mxBean.getAvailableProcessors());
+ newEntry.setLoadAverage(mxBean.getSystemLoadAverage());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics processors - %d , loadavg - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getLoadAverage()));
+ }
+ }
+
+ private void getRuntimeData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final RuntimeMXBean mxBean = ManagementFactory.getRuntimeMXBean();
+ newEntry.setUptime(mxBean.getUptime());
+ newEntry.setStartTime(mxBean.getStartTime());
+ newEntry.setProcessId(mxBean.getPid());
+ newEntry.setJvmName(mxBean.getName());
+ newEntry.setJvmVendor(mxBean.getVmVendor());
+ newEntry.setJvmVersion(mxBean.getVmVersion());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics uptime - %d , starttime - %d",
+ newEntry.getUptime(),
+ newEntry.getStartTime()));
+ }
+ }
+
+ private void getJvmDimensions(@NotNull ManagementServerHostStatsEntry newEntry) {
+ Runtime runtime = Runtime.getRuntime();
+ newEntry.setTotalMemoryBytes(runtime.totalMemory());
+ newEntry.setFreeMemoryBytes(runtime.freeMemory());
+ newEntry.setAvailableProcessors(runtime.availableProcessors());
+ newEntry.setTotalMax(runtime.maxMemory());
+ //long maxMem = runtime.maxMemory();
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics proc - %d , maxMem - %d , totalMemory - %d , freeMemory - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getTotalMax(),
+ newEntry.getTotalMemoryBytes(),
+ newEntry.getFreeMemoryBytes()));
+ }
+ }
+
+ /**
+ * As for data from outside the JVM, we only rely on /proc/ contained data.
+ *
+ * @param newEntry
+ */
+ private void getProcFsData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ String OS = Script.runSimpleBashScript("cat /proc/version");
+ newEntry.setOsDistribution(OS);
+ String mem = Script.runSimpleBashScript("cat /proc/meminfo | grep MemTotal | cut -f 2 -d ':' | tr -d 'a-zA-z '").trim();
Review comment:
the extra methods are in com.sun.management.OperatingSystemMXBean. I'll be exploring a bit if it is worth changing. (doesn't seem to be so to me now)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r761916957
##########
File path: api/src/main/java/org/apache/cloudstack/api/ApiConstants.java
##########
@@ -226,10 +226,14 @@
public static final String IS_RECURSIVE = "isrecursive";
public static final String ISO_FILTER = "isofilter";
public static final String ISO_GUEST_OS_NONE = "None";
+ public static final String JAVA_DISTRIBUTION = "javadistribution";
+ public static final String JAVA_VERSION = "javaversion";
public static final String JOB_ID = "jobid";
public static final String JOB_STATUS = "jobstatus";
public static final String LABEL = "label";
public static final String LASTNAME = "lastname";
+ public static final String LAST_STOP = "laststop";
+ public static final String LAST_START = "laststart";
Review comment:
```suggestion
public static final String LAST_SYSTEM_START = "lastsystemstart";
```
Update related parameters/methods if needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r778859376
##########
File path: framework/cluster/src/main/java/com/cloud/cluster/ClusterManager.java
##########
@@ -53,19 +62,44 @@
void registerDispatcher(Dispatcher dispatcher);
+ /**
+ * register a listener for incoming status changes of ManagementServers
+ *
+ * @param administrator the object administrating statusses
Review comment:
:) Yes the double ss. thanks
```suggestion
* @param administrator the object administrating statuses
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r782287591
##########
File path: framework/cluster/src/main/java/com/cloud/cluster/ClusterManager.java
##########
@@ -53,19 +62,44 @@
void registerDispatcher(Dispatcher dispatcher);
+ /**
+ * register a listener for incoming status changes of ManagementServers
+ *
+ * @param administrator the object administrating statusses
+ */
+ void registerStatusAdministrator(StatusAdministrator administrator);
+
ManagementServerHost getPeer(String peerName);
+ /**
+ *
+ * @return {code}Long.toString({code}{@see getManagementNodeId()}{code}){code}
+ */
String getSelfPeerName();
long getManagementNodeId();
+ /**
+ * determined by the time
+ * @return start time as {code}System.currentTimeMillis(){code}
+ */
long getCurrentRunId();
- public long getManagementRunId(long msId);
+ /**
+ * the other guy's is derived from start time as stored in the db
+ */
+ long getManagementRunId(long msId);
- public interface Dispatcher {
+ interface Dispatcher {
String getName();
String dispatch(ClusterServicePdu pdu);
}
+
+ /**
+ * what the client of {@see registerStatusAdministrator()} should implement
+ */
+ interface StatusAdministrator {
+ String newStatus(ClusterServicePdu pdu);
+ }
Review comment:
kind of, some of these are mostly the results of my code analysis. Any suggestions?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r736348541
##########
File path: framework/cluster/src/main/java/com/cloud/cluster/ClusterManager.java
##########
@@ -21,27 +21,36 @@
import com.cloud.utils.component.Manager;
+/**
+ * the definition of the framework for inter MS communication
+ */
public interface ClusterManager extends Manager {
static final String ALERT_SUBJECT = "cluster-alert";
final ConfigKey<Integer> HeartbeatInterval = new ConfigKey<Integer>(Integer.class, "cluster.heartbeat.interval", "management-server", "1500",
"Interval to check for the heart beat between management server nodes", false);
final ConfigKey<Integer> HeartbeatThreshold = new ConfigKey<Integer>(Integer.class, "cluster.heartbeat.threshold", "management-server", "150000",
"Threshold before self-fence the management server", true);
+ /**
+ * adds a new pakket to the incoming queue
Review comment:
:) Dutch spelling
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] GutoVeronezi commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
GutoVeronezi commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r732977090
##########
File path: framework/cluster/src/main/java/com/cloud/cluster/ManagementServerStatusVO.java
##########
@@ -0,0 +1,215 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.cluster;
+
+import com.cloud.utils.db.GenericDao;
+import org.apache.cloudstack.management.ManagementServerStatus;
+
+import javax.persistence.Column;
+import javax.persistence.Entity;
+import javax.persistence.GeneratedValue;
+import javax.persistence.GenerationType;
+import javax.persistence.Id;
+import javax.persistence.Table;
+import javax.persistence.Temporal;
+import javax.persistence.TemporalType;
+import java.util.Date;
+
+@Entity
+@Table(name = "mshost_status")
+public class ManagementServerStatusVO implements ManagementServerStatus {
+
+ @Id
+ @GeneratedValue(strategy = GenerationType.IDENTITY)
+ @Column(name = "id")
+ private long id;
+
+ @Column(name = "ms_id", updatable = true, nullable = false)
+ private long msId;
+
+ @Column(name="last_start")
+ private Date lastStart;
+
+ @Column(name="last_stop")
+ private Date lastStop;
+
+ @Column(name="last_boot")
+ private Date lastBoot;
+
+ @Column(name="last_down")
+ private Date lastDown;
+
+ @Column(name="os_name")
+ private String osName;
+
+ @Column(name="os_version")
+ private String osVersion;
+
+ @Column(name="java_name")
+ private String javaName;
+
+ @Column(name="java_version")
+ private String javaVersion;
+
+ @Temporal(TemporalType.TIMESTAMP)
+ @Column(name = "updated")
+ private Date updated;
+
+ @Column(name = GenericDao.CREATED_COLUMN)
+ private Date created;
+
+ @Column(name = GenericDao.REMOVED_COLUMN)
+ private Date removed;
+
+
+ public ManagementServerStatusVO() {
+ }
+
+ @Override
+ public long getId() {
+ return id;
+ }
+
+ public void setId(long id) {
+ this.id = id;
+ }
+
+ @Override
+ public long getMsId() {
+ return msId;
+ }
+
+ public void setMsId(long msId) {
+ this.msId = msId;
+ }
+
+ @Override
+ public Date getLastStart() {
+ return lastStart;
+ }
+
+ public void setLastStart(Date lastStart) {
+ this.lastStart = lastStart;
+ }
+
+ @Override
+ public Date getLastStop() {
+ return lastStop;
+ }
+
+ public void setLastStop(Date lastStop) {
+ this.lastStop = lastStop;
+ }
+
+ @Override
+ public Date getLastBoot() {
+ return lastBoot;
+ }
+
+ public void setLastBoot(Date lastBoot) {
+ this.lastBoot = lastBoot;
+ }
+
+ @Override
+ public Date getLastDown() {
+ return lastDown;
+ }
+
+ public void setLastDown(Date lastDown) {
+ this.lastDown = lastDown;
+ }
+
+ @Override
+ public String getOsName() {
+ return osName;
+ }
+
+ public void setOsName(String osName) {
+ this.osName = osName;
+ }
+
+ @Override
+ public String getOsVersion() {
+ return osVersion;
+ }
+
+ public void setOsVersion(String osVersion) {
+ this.osVersion = osVersion;
+ }
+
+ @Override
+ public String getJavaName() {
+ return javaName;
+ }
+
+ public void setJavaName(String javaName) {
+ this.javaName = javaName;
+ }
+
+ @Override
+ public String getJavaVersion() {
+ return javaVersion;
+ }
+
+ public void setJavaVersion(String javaVersion) {
+ this.javaVersion = javaVersion;
+ }
+
+ @Override
+ public Date getUpdated() {
+ return updated;
+ }
+
+ public void setUpdated(Date updated) {
+ this.updated = updated;
+ }
+
+ @Override
+ public Date getCreated() {
+ return created;
+ }
+
+ public void setCreated(Date created) {
+ this.created = created;
+ }
+
+ @Override
+ public Date getRemoved() {
+ return removed;
+ }
+
+ public void setRemoved(Date removedTime) {
+ removed = removedTime;
+ }
+
+ @Override
+ public String toString() {
+ return String.join("-",
Review comment:
We could use `ReflectionToStringBuilderUtils` or `ReflectionToStringBuilder` here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r736462732
##########
File path: engine/schema/src/main/resources/META-INF/db/schema-41520to41600.sql
##########
@@ -807,3 +807,21 @@ ALTER TABLE `cloud_usage`.`user_statistics` DROP INDEX `account_id`, ADD UNIQUE
ALTER TABLE `cloud`.`vm_work_job` ADD COLUMN `secondary_object` char(100) COMMENT 'any additional item that must be checked during queueing' AFTER `vm_instance_id`;
ALTER TABLE cloud.vm_work_job ADD CONSTRAINT vm_work_job_step_and_objects UNIQUE KEY (step,vm_instance_id,secondary_object);
+
+CREATE TABLE `cloud`.`mshost_status` (
+ `id` bigint unsigned NOT NULL auto_increment COMMENT 'id',
+ `ms_id` varchar(40) DEFAULT NULL COMMENT 'the id of the management server record',
+ `last_start` datetime COMMENT 'the last start time for this MS',
+ `last_stop` datetime COMMENT 'the last stop time for this MS',
+ `last_boot` datetime COMMENT 'the last system boot time for the host of this MS',
+ `last_down` datetime COMMENT 'the last system shutdown time for the host of this MS',
+ `os_name` varchar(64) DEFAULT null COMMENT 'the name of the os type running on the host of this MS',
+ `os_version` varchar(64) DEFAULT null COMMENT 'the version of the os running on the host of this MS',
+ `java_name` varchar(64) DEFAULT null COMMENT 'the name of the java distribution running this MS',
+ `java_version` varchar(64) DEFAULT null COMMENT 'the version of the java distribution running this MS',
+ `updated` datetime DEFAULT NULL,
+ `created` datetime DEFAULT NULL,
+ `removed` datetime DEFAULT NULL,
+ PRIMARY KEY (`id`),
+ CONSTRAINT `uc_ms_id` UNIQUE (`ms_id`)
Review comment:
The UUID is also used in referring inline. I trust the UUID more and also it is the one that needs to be returned. I see no advantage in using the ID.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r752057604
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -564,11 +657,352 @@ protected Point creteInfluxDbPoint(Object metricsObject) {
}
}
+ Gson gson;
+
+ class ManagementServerCollector extends AbstractStatsCollector {
+ @Override
+ protected void runInContext() {
+ LOGGER.debug(String.format("%s is running...", this.getClass().getSimpleName()));
+ long msid = ManagementServerNode.getManagementServerId();
+ ManagementServerHostVO mshost = null;
+ ManagementServerHostStatsEntry hostStatsEntry = null;
+ try {
+ mshost = managementServerHostDao.findByMsid(msid);
+ // get local data
+ hostStatsEntry = getDataFrom(mshost);
+ managementServerHostStats.put(mshost.getUuid(), hostStatsEntry);
+ // send to other hosts
+ clusterManager.publishStatus(gson.toJson(hostStatsEntry));
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to retrieve host stats", t);
+ }
+ try {
+ // send to DB
+ storeStatus(hostStatsEntry, mshost);
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to store host state", t);
+ }
+ }
+
+ private void storeStatus(ManagementServerHostStatsEntry hostStatsEntry, ManagementServerHostVO mshost) {
+ if (hostStatsEntry == null || mshost == null) {
+ return;
+ }
+ ManagementServerStatusVO msStats = managementServerStatusDao.findByMsId(hostStatsEntry.getManagementServerHostUuid());
+ if (msStats == null) {
+ LOGGER.info(String.format("creating new status info record for host %s - %s",
+ mshost.getName(),
+ hostStatsEntry.getManagementServerHostUuid()));
+ msStats = new ManagementServerStatusVO();
+ msStats.setMsId(hostStatsEntry.getManagementServerHostUuid());
+ }
+ msStats.setOsDistribution(hostStatsEntry.getOsDistribution()); // for now just the bunch details come later
+ msStats.setJavaName(hostStatsEntry.getJvmVendor());
+ msStats.setJavaVersion(hostStatsEntry.getJvmVersion());
+ Date startTime = new Date(hostStatsEntry.getStartTime());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("reporting starttime %s", startTime));
+ }
+ msStats.setLastStart(startTime);
+ msStats.setUpdated(new Date());
+ managementServerStatusDao.persist(msStats);
+ }
+
+ @NotNull
+ private ManagementServerHostStatsEntry getDataFrom(ManagementServerHostVO mshost) {
+ ManagementServerHostStatsEntry newEntry = new ManagementServerHostStatsEntry();
+ LOGGER.debug("Metrics collection start...");
+ newEntry.setManagementServerHostId(mshost.getId());
+ newEntry.setManagementServerHostUuid(mshost.getUuid());
+ retrieveSession(newEntry);
+ getJvmDimensions(newEntry);
+ LOGGER.debug("Metrics collection extra...");
+ getRuntimeData(newEntry);
+ getCpuData(newEntry);
+ getMemoryData(newEntry);
+ // newEntry must now include a pid!
+ getProcFsData(newEntry);
+ getFsData(newEntry);
+ getDataBaseStatistics(newEntry, mshost.getMsid());
+ gatherAllMetrics(newEntry);
+ LOGGER.debug("Metrics collection end!");
+ return newEntry;
+ }
+
+ private void retrieveSession(ManagementServerHostStatsEntry newEntry) {
+ long sessions = ApiSessionListener.getSessionCount();
+ newEntry.setSessions(sessions);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("Sessions found in Api %d vs context %d", sessions,ApiSessionListener.getNumberOfSessions()));
+ } else {
+ LOGGER.debug("Sessions active: " + sessions);
+ }
+ }
+
+ private void getDataBaseStatistics(ManagementServerHostStatsEntry newEntry, long msid) {
+ int count = _hostDao.countByMs(msid);
+ newEntry.setAgentCount(count);
+ }
+
+ private void getMemoryData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ MemoryMXBean mxBean = ManagementFactory.getMemoryMXBean();
+ newEntry.setTotalInit(mxBean.getHeapMemoryUsage().getInit() + mxBean.getNonHeapMemoryUsage().getInit());
+ newEntry.setTotalUsed(mxBean.getHeapMemoryUsage().getUsed() + mxBean.getNonHeapMemoryUsage().getUsed());
+ newEntry.setTotalMax(mxBean.getHeapMemoryUsage().getMax() + mxBean.getNonHeapMemoryUsage().getMax());
+ newEntry.setTotalCommitted(mxBean.getHeapMemoryUsage().getCommitted() + mxBean.getNonHeapMemoryUsage().getCommitted());
+ }
+
+ private void getCpuData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final OperatingSystemMXBean mxBean = ManagementFactory.getOperatingSystemMXBean();
+ newEntry.setAvailableProcessors(mxBean.getAvailableProcessors());
+ newEntry.setLoadAverage(mxBean.getSystemLoadAverage());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics processors - %d , loadavg - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getLoadAverage()));
+ }
+ }
+
+ private void getRuntimeData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final RuntimeMXBean mxBean = ManagementFactory.getRuntimeMXBean();
+ newEntry.setUptime(mxBean.getUptime());
+ newEntry.setStartTime(mxBean.getStartTime());
+ newEntry.setProcessId(mxBean.getPid());
+ newEntry.setJvmName(mxBean.getName());
+ newEntry.setJvmVendor(mxBean.getVmVendor());
+ newEntry.setJvmVersion(mxBean.getVmVersion());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics uptime - %d , starttime - %d",
+ newEntry.getUptime(),
+ newEntry.getStartTime()));
+ }
+ }
+
+ private void getJvmDimensions(@NotNull ManagementServerHostStatsEntry newEntry) {
+ Runtime runtime = Runtime.getRuntime();
+ newEntry.setTotalMemoryBytes(runtime.totalMemory());
+ newEntry.setFreeMemoryBytes(runtime.freeMemory());
+ newEntry.setAvailableProcessors(runtime.availableProcessors());
+ newEntry.setTotalMax(runtime.maxMemory());
+ //long maxMem = runtime.maxMemory();
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics proc - %d , maxMem - %d , totalMemory - %d , freeMemory - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getTotalMax(),
+ newEntry.getTotalMemoryBytes(),
+ newEntry.getFreeMemoryBytes()));
+ }
+ }
+
+ /**
+ * As for data from outside the JVM, we only rely on /proc/ contained data.
+ *
+ * @param newEntry
+ */
+ private void getProcFsData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ String OS = Script.runSimpleBashScript("cat /proc/version");
+ newEntry.setOsDistribution(OS);
+ String mem = Script.runSimpleBashScript("cat /proc/meminfo | grep MemTotal | cut -f 2 -d ':' | tr -d 'a-zA-z '").trim();
+ newEntry.setSystemMemoryTotal(Long.parseLong(mem) * 1024);
+ String free = Script.runSimpleBashScript("cat /proc/meminfo | grep MemFree | cut -f 2 -d ':' | tr -d 'a-zA-z '").trim();
+ newEntry.setSystemMemoryFree(Long.parseLong(free) * 1024);
+ String used = Script.runSimpleBashScript(String.format("ps -o rss= %d", newEntry.getPid()));
+ newEntry.setSystemMemoryUsed(Long.parseLong(used));
+ String maxuse = Script.runSimpleBashScript(String.format("ps -o vsz= %d", newEntry.getPid()));
+ newEntry.setSystemMemoryVirtualSize(Long.parseLong(maxuse));
+
+ newEntry.setSystemTotalCpuCycles(getSystemCpuCyclesTotal());
+ newEntry.setSystemLoadAverages(getCpuLoads());
+ newEntry.setSystemCyclesUsage(getSystemCpuUsage(newEntry));
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.debug(
+ String.format("cpu\ncapacities: %f\n loads: %s ; %s ; %s\n stats: %f ; %f ; %f",
+ newEntry.getSystemTotalCpuCycles(),
+ newEntry.getSystemLoadAverages()[0], newEntry.getSystemLoadAverages()[1], newEntry.getSystemLoadAverages()[2],
+ newEntry.getSystemCyclesUsage()[0], newEntry.getSystemCyclesUsage()[1], newEntry.getSystemCyclesUsage()[2]
+ )
+ );
+ }
+ }
+
+ @NotNull
+ private double[] getCpuLoads() {
+ String[] cpuloadString = Script.runSimpleBashScript("cat /proc/loadavg").split(" ");
+ double[] cpuloads = {Double.parseDouble(cpuloadString[0]), Double.parseDouble(cpuloadString[1]), Double.parseDouble(cpuloadString[2])};
+ return cpuloads;
+ }
+
+ private double [] getSystemCpuUsage(@NotNull ManagementServerHostStatsEntry newEntry) {
+ String[] cpustats = Script.runSimpleBashScript("cat /proc/stat | grep \"cpu \" | tr -d \"cpu\"").trim().split(" ");
+ double [] cycleUsage = {Double.parseDouble(cpustats[0]) + Double.parseDouble(cpustats[1]), Double.parseDouble(cpustats[2]), Double.parseDouble(cpustats[3])};
+ return cycleUsage;
+ }
+
+ private double getSystemCpuCyclesTotal() {
+ String cpucaps = Script.runSimpleBashScript("cat /proc/cpuinfo | grep \"cpu MHz\" | grep \"cpu MHz\" | cut -f 2 -d : | tr -d ' '| tr '\\n' \" \"");
+ double totalcpucap = 0;
+ for (String cpucap : cpucaps.split(" ")) {
+ totalcpucap += Double.parseDouble(cpucap);
+ }
+ return totalcpucap;
+ }
+
+ private void getFsData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ Set<String> logFileNames = LogUtils.getLogFileNames();
+ StringBuilder logInfoBuilder = new StringBuilder();
+ for (String fileName : logFileNames) {
+ String du = Script.runSimpleBashScript(String.format("du -sk %s", fileName));
+ String df = Script.runSimpleBashScript(String.format("df -k %s | grep -v Filesystem", fileName));
+ logInfoBuilder.append(fileName).append('\n').append("usage: ").append(du).append('\n').append("disk :").append(df);
Review comment:
I don't know. It is my habit @weizhouapache . do you sugest bytes or megabytes or...?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] weizhouapache commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
weizhouapache commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r752061390
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -564,11 +657,352 @@ protected Point creteInfluxDbPoint(Object metricsObject) {
}
}
+ Gson gson;
+
+ class ManagementServerCollector extends AbstractStatsCollector {
+ @Override
+ protected void runInContext() {
+ LOGGER.debug(String.format("%s is running...", this.getClass().getSimpleName()));
+ long msid = ManagementServerNode.getManagementServerId();
+ ManagementServerHostVO mshost = null;
+ ManagementServerHostStatsEntry hostStatsEntry = null;
+ try {
+ mshost = managementServerHostDao.findByMsid(msid);
+ // get local data
+ hostStatsEntry = getDataFrom(mshost);
+ managementServerHostStats.put(mshost.getUuid(), hostStatsEntry);
+ // send to other hosts
+ clusterManager.publishStatus(gson.toJson(hostStatsEntry));
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to retrieve host stats", t);
+ }
+ try {
+ // send to DB
+ storeStatus(hostStatsEntry, mshost);
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to store host state", t);
+ }
+ }
+
+ private void storeStatus(ManagementServerHostStatsEntry hostStatsEntry, ManagementServerHostVO mshost) {
+ if (hostStatsEntry == null || mshost == null) {
+ return;
+ }
+ ManagementServerStatusVO msStats = managementServerStatusDao.findByMsId(hostStatsEntry.getManagementServerHostUuid());
+ if (msStats == null) {
+ LOGGER.info(String.format("creating new status info record for host %s - %s",
+ mshost.getName(),
+ hostStatsEntry.getManagementServerHostUuid()));
+ msStats = new ManagementServerStatusVO();
+ msStats.setMsId(hostStatsEntry.getManagementServerHostUuid());
+ }
+ msStats.setOsDistribution(hostStatsEntry.getOsDistribution()); // for now just the bunch details come later
+ msStats.setJavaName(hostStatsEntry.getJvmVendor());
+ msStats.setJavaVersion(hostStatsEntry.getJvmVersion());
+ Date startTime = new Date(hostStatsEntry.getStartTime());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("reporting starttime %s", startTime));
+ }
+ msStats.setLastStart(startTime);
+ msStats.setUpdated(new Date());
+ managementServerStatusDao.persist(msStats);
+ }
+
+ @NotNull
+ private ManagementServerHostStatsEntry getDataFrom(ManagementServerHostVO mshost) {
+ ManagementServerHostStatsEntry newEntry = new ManagementServerHostStatsEntry();
+ LOGGER.debug("Metrics collection start...");
+ newEntry.setManagementServerHostId(mshost.getId());
+ newEntry.setManagementServerHostUuid(mshost.getUuid());
+ retrieveSession(newEntry);
+ getJvmDimensions(newEntry);
+ LOGGER.debug("Metrics collection extra...");
+ getRuntimeData(newEntry);
+ getCpuData(newEntry);
+ getMemoryData(newEntry);
+ // newEntry must now include a pid!
+ getProcFsData(newEntry);
+ getFsData(newEntry);
+ getDataBaseStatistics(newEntry, mshost.getMsid());
+ gatherAllMetrics(newEntry);
+ LOGGER.debug("Metrics collection end!");
+ return newEntry;
+ }
+
+ private void retrieveSession(ManagementServerHostStatsEntry newEntry) {
+ long sessions = ApiSessionListener.getSessionCount();
+ newEntry.setSessions(sessions);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("Sessions found in Api %d vs context %d", sessions,ApiSessionListener.getNumberOfSessions()));
+ } else {
+ LOGGER.debug("Sessions active: " + sessions);
+ }
+ }
+
+ private void getDataBaseStatistics(ManagementServerHostStatsEntry newEntry, long msid) {
+ int count = _hostDao.countByMs(msid);
+ newEntry.setAgentCount(count);
+ }
+
+ private void getMemoryData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ MemoryMXBean mxBean = ManagementFactory.getMemoryMXBean();
+ newEntry.setTotalInit(mxBean.getHeapMemoryUsage().getInit() + mxBean.getNonHeapMemoryUsage().getInit());
+ newEntry.setTotalUsed(mxBean.getHeapMemoryUsage().getUsed() + mxBean.getNonHeapMemoryUsage().getUsed());
+ newEntry.setTotalMax(mxBean.getHeapMemoryUsage().getMax() + mxBean.getNonHeapMemoryUsage().getMax());
+ newEntry.setTotalCommitted(mxBean.getHeapMemoryUsage().getCommitted() + mxBean.getNonHeapMemoryUsage().getCommitted());
+ }
+
+ private void getCpuData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final OperatingSystemMXBean mxBean = ManagementFactory.getOperatingSystemMXBean();
+ newEntry.setAvailableProcessors(mxBean.getAvailableProcessors());
+ newEntry.setLoadAverage(mxBean.getSystemLoadAverage());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics processors - %d , loadavg - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getLoadAverage()));
+ }
+ }
+
+ private void getRuntimeData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final RuntimeMXBean mxBean = ManagementFactory.getRuntimeMXBean();
+ newEntry.setUptime(mxBean.getUptime());
+ newEntry.setStartTime(mxBean.getStartTime());
+ newEntry.setProcessId(mxBean.getPid());
+ newEntry.setJvmName(mxBean.getName());
+ newEntry.setJvmVendor(mxBean.getVmVendor());
+ newEntry.setJvmVersion(mxBean.getVmVersion());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics uptime - %d , starttime - %d",
+ newEntry.getUptime(),
+ newEntry.getStartTime()));
+ }
+ }
+
+ private void getJvmDimensions(@NotNull ManagementServerHostStatsEntry newEntry) {
+ Runtime runtime = Runtime.getRuntime();
+ newEntry.setTotalMemoryBytes(runtime.totalMemory());
+ newEntry.setFreeMemoryBytes(runtime.freeMemory());
+ newEntry.setAvailableProcessors(runtime.availableProcessors());
+ newEntry.setTotalMax(runtime.maxMemory());
+ //long maxMem = runtime.maxMemory();
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics proc - %d , maxMem - %d , totalMemory - %d , freeMemory - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getTotalMax(),
+ newEntry.getTotalMemoryBytes(),
+ newEntry.getFreeMemoryBytes()));
+ }
+ }
+
+ /**
+ * As for data from outside the JVM, we only rely on /proc/ contained data.
+ *
+ * @param newEntry
+ */
+ private void getProcFsData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ String OS = Script.runSimpleBashScript("cat /proc/version");
+ newEntry.setOsDistribution(OS);
+ String mem = Script.runSimpleBashScript("cat /proc/meminfo | grep MemTotal | cut -f 2 -d ':' | tr -d 'a-zA-z '").trim();
+ newEntry.setSystemMemoryTotal(Long.parseLong(mem) * 1024);
+ String free = Script.runSimpleBashScript("cat /proc/meminfo | grep MemFree | cut -f 2 -d ':' | tr -d 'a-zA-z '").trim();
+ newEntry.setSystemMemoryFree(Long.parseLong(free) * 1024);
+ String used = Script.runSimpleBashScript(String.format("ps -o rss= %d", newEntry.getPid()));
+ newEntry.setSystemMemoryUsed(Long.parseLong(used));
+ String maxuse = Script.runSimpleBashScript(String.format("ps -o vsz= %d", newEntry.getPid()));
+ newEntry.setSystemMemoryVirtualSize(Long.parseLong(maxuse));
+
+ newEntry.setSystemTotalCpuCycles(getSystemCpuCyclesTotal());
+ newEntry.setSystemLoadAverages(getCpuLoads());
+ newEntry.setSystemCyclesUsage(getSystemCpuUsage(newEntry));
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.debug(
+ String.format("cpu\ncapacities: %f\n loads: %s ; %s ; %s\n stats: %f ; %f ; %f",
+ newEntry.getSystemTotalCpuCycles(),
+ newEntry.getSystemLoadAverages()[0], newEntry.getSystemLoadAverages()[1], newEntry.getSystemLoadAverages()[2],
+ newEntry.getSystemCyclesUsage()[0], newEntry.getSystemCyclesUsage()[1], newEntry.getSystemCyclesUsage()[2]
+ )
+ );
+ }
+ }
+
+ @NotNull
+ private double[] getCpuLoads() {
+ String[] cpuloadString = Script.runSimpleBashScript("cat /proc/loadavg").split(" ");
+ double[] cpuloads = {Double.parseDouble(cpuloadString[0]), Double.parseDouble(cpuloadString[1]), Double.parseDouble(cpuloadString[2])};
+ return cpuloads;
+ }
+
+ private double [] getSystemCpuUsage(@NotNull ManagementServerHostStatsEntry newEntry) {
+ String[] cpustats = Script.runSimpleBashScript("cat /proc/stat | grep \"cpu \" | tr -d \"cpu\"").trim().split(" ");
+ double [] cycleUsage = {Double.parseDouble(cpustats[0]) + Double.parseDouble(cpustats[1]), Double.parseDouble(cpustats[2]), Double.parseDouble(cpustats[3])};
+ return cycleUsage;
+ }
+
+ private double getSystemCpuCyclesTotal() {
+ String cpucaps = Script.runSimpleBashScript("cat /proc/cpuinfo | grep \"cpu MHz\" | grep \"cpu MHz\" | cut -f 2 -d : | tr -d ' '| tr '\\n' \" \"");
+ double totalcpucap = 0;
+ for (String cpucap : cpucaps.split(" ")) {
+ totalcpucap += Double.parseDouble(cpucap);
+ }
+ return totalcpucap;
+ }
+
+ private void getFsData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ Set<String> logFileNames = LogUtils.getLogFileNames();
+ StringBuilder logInfoBuilder = new StringBuilder();
+ for (String fileName : logFileNames) {
+ String du = Script.runSimpleBashScript(String.format("du -sk %s", fileName));
+ String df = Script.runSimpleBashScript(String.format("df -k %s | grep -v Filesystem", fileName));
+ logInfoBuilder.append(fileName).append('\n').append("usage: ").append(du).append('\n').append("disk :").append(df);
Review comment:
@DaanHoogland
all are fine. what I meant is, we'd better add "KB" or "kilobytes" in the log info so it will be more clear.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-972882714
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_multiplication_x: debian :heavy_multiplication_x: suse15. SL-JID 1736
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r736582188
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -344,52 +355,107 @@ public InfrastructureResponse listInfrastructure() {
// CPU and memory capacities
final CapacityDaoImpl.SummedCapacity cpuCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_CPU, null, clusterId);
final CapacityDaoImpl.SummedCapacity memoryCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_MEMORY, null, clusterId);
- final Metrics metrics = new Metrics(cpuCapacity, memoryCapacity);
+ final org.apache.cloudstack.metrics.MetricsServiceImpl.HostMetrics hostMetrics = new org.apache.cloudstack.metrics.MetricsServiceImpl.HostMetrics(cpuCapacity, memoryCapacity);
for (final Host host: hostDao.findByClusterId(clusterId)) {
if (host == null || host.getType() != Host.Type.Routing) {
continue;
}
if (host.getStatus() == Status.Up) {
- metrics.incrUpResources();
+ hostMetrics.incrUpResources();
}
- metrics.incrTotalResources();
- updateHostMetrics(metrics, hostJoinDao.findById(host.getId()));
+ hostMetrics.incrTotalResources();
+ updateHostMetrics(hostMetrics, hostJoinDao.findById(host.getId()));
}
metricsResponse.setState(clusterResponse.getAllocationState(), clusterResponse.getManagedState());
- metricsResponse.setResources(metrics.getUpResources(), metrics.getTotalResources());
+ metricsResponse.setResources(hostMetrics.getUpResources(), hostMetrics.getTotalResources());
// CPU
- metricsResponse.setCpuTotal(metrics.getTotalCpu());
- metricsResponse.setCpuAllocated(metrics.getCpuAllocated(), metrics.getTotalCpu());
- if (metrics.getCpuUsedPercentage() > 0L) {
- metricsResponse.setCpuUsed(metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
- metricsResponse.setCpuMaxDeviation(metrics.getMaximumCpuUsage(), metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
+ metricsResponse.setCpuTotal(hostMetrics.getTotalCpu());
+ metricsResponse.setCpuAllocated(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu());
+ if (hostMetrics.getCpuUsedPercentage() > 0L) {
+ metricsResponse.setCpuUsed(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ metricsResponse.setCpuMaxDeviation(hostMetrics.getMaximumCpuUsage(), hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
}
// Memory
- metricsResponse.setMemTotal(metrics.getTotalMemory());
- metricsResponse.setMemAllocated(metrics.getMemoryAllocated(), metrics.getTotalMemory());
- if (metrics.getMemoryUsed() > 0L) {
- metricsResponse.setMemUsed(metrics.getMemoryUsed(), metrics.getTotalMemory());
- metricsResponse.setMemMaxDeviation(metrics.getMaximumMemoryUsage(), metrics.getMemoryUsed(), metrics.getTotalHosts());
+ metricsResponse.setMemTotal(hostMetrics.getTotalMemory());
+ metricsResponse.setMemAllocated(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory());
+ if (hostMetrics.getMemoryUsed() > 0L) {
+ metricsResponse.setMemUsed(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory());
+ metricsResponse.setMemMaxDeviation(hostMetrics.getMaximumMemoryUsage(), hostMetrics.getMemoryUsed(), hostMetrics.getTotalHosts());
}
// CPU thresholds
- metricsResponse.setCpuUsageThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuThreshold);
- metricsResponse.setCpuUsageDisableThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuDisableThreshold);
- metricsResponse.setCpuAllocatedThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuThreshold);
- metricsResponse.setCpuAllocatedDisableThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuDisableThreshold);
+ metricsResponse.setCpuUsageThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuThreshold);
+ metricsResponse.setCpuUsageDisableThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuDisableThreshold);
+ metricsResponse.setCpuAllocatedThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuThreshold);
+ metricsResponse.setCpuAllocatedDisableThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuDisableThreshold);
// Memory thresholds
- metricsResponse.setMemoryUsageThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryUsageDisableThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setMemoryAllocatedThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryAllocatedDisableThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryUsageThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryUsageDisableThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryAllocatedThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryAllocatedDisableThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
Review comment:
yes, and I may find time for that, but this change only amounts to a refactor of the name `metrics` -> `hostMetrics` to avoid confusion. (will do with low prio)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-954123825
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1631
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1022872411
Packaging result: :heavy_multiplication_x: el7 :heavy_multiplication_x: el8 :heavy_multiplication_x: debian :heavy_multiplication_x: suse15. SL-JID 2330
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1028114410
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1022870703
Packaging result: :heavy_multiplication_x: el7 :heavy_multiplication_x: el8 :heavy_multiplication_x: debian :heavy_multiplication_x: suse15. SL-JID 2329
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-994872786
I'm not sure how you see a duplicate, @tampler. I agree there is a large overlap in functionality but the prometheus one would need prometheus installed while this is more minimal it always gives some statistics on a minimal (acs only) installation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland closed pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland closed pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-996763336
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-998908159
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1982
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r761916718
##########
File path: api/src/main/java/org/apache/cloudstack/api/ApiConstants.java
##########
@@ -226,10 +226,14 @@
public static final String IS_RECURSIVE = "isrecursive";
public static final String ISO_FILTER = "isofilter";
public static final String ISO_GUEST_OS_NONE = "None";
+ public static final String JAVA_DISTRIBUTION = "javadistribution";
+ public static final String JAVA_VERSION = "javaversion";
public static final String JOB_ID = "jobid";
public static final String JOB_STATUS = "jobstatus";
public static final String LABEL = "label";
public static final String LASTNAME = "lastname";
+ public static final String LAST_STOP = "laststop";
Review comment:
```suggestion
public static final String LAST_SYSTEM_STOP = "lastsystemstop";
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r761940732
##########
File path: utils/src/main/java/com/cloud/utils/LogUtils.java
##########
@@ -20,26 +20,56 @@
package com.cloud.utils;
import java.io.File;
+import java.util.Enumeration;
+import java.util.HashSet;
+import java.util.Set;
+import org.apache.log4j.Appender;
+import org.apache.log4j.FileAppender;
import org.apache.log4j.Logger;
import org.apache.log4j.xml.DOMConfigurator;
public class LogUtils {
- public static final Logger s_logger = Logger.getLogger(LogUtils.class);
+ public static final Logger LOGGER = Logger.getLogger(LogUtils.class);
+
+ private static String configFileLocation = null;
public static void initLog4j(String log4jConfigFileName) {
assert (log4jConfigFileName != null);
File file = PropertiesUtil.findConfigFile(log4jConfigFileName);
if (file != null) {
- s_logger.info("log4j configuration found at " + file.getAbsolutePath());
- DOMConfigurator.configureAndWatch(file.getAbsolutePath());
+ configFileLocation = file.getAbsolutePath();
+ DOMConfigurator.configureAndWatch(configFileLocation);
} else {
String nameWithoutExtension = log4jConfigFileName.substring(0, log4jConfigFileName.lastIndexOf('.'));
file = PropertiesUtil.findConfigFile(nameWithoutExtension + ".properties");
if (file != null) {
- s_logger.info("log4j configuration found at " + file.getAbsolutePath());
- DOMConfigurator.configureAndWatch(file.getAbsolutePath());
+ configFileLocation = file.getAbsolutePath();
+ DOMConfigurator.configureAndWatch(configFileLocation);
+ }
+ }
+ if(configFileLocation != null) {
+ LOGGER.info("log4j configuration found at " + configFileLocation);
+ }
+ }
+ public static Set<String> getLogFileNames() {
+ Set<String> fileNames = new HashSet<>();
+ Enumeration appenders = LOGGER.getRootLogger().getAllAppenders();
+ int appenderCount=0;
+ while(appenders.hasMoreElements()) {
+ ++appenderCount;
+ Appender currAppender = (Appender) appenders.nextElement();
+ if(currAppender instanceof FileAppender) {
Review comment:
```suggestion
if (currAppender instanceof FileAppender) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-983462313
Packaging result: :heavy_multiplication_x: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1808
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1056943100
@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1056942550
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1069124100
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1071093358
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1066543952
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1069212464
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1083240569
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 3026
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1084322248
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 3036
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1071093358
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1004669357
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1028211978
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1084528102
@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] RodrigoDLopez commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
RodrigoDLopez commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r778195353
##########
File path: framework/cluster/src/main/java/com/cloud/cluster/ClusterManager.java
##########
@@ -53,19 +62,44 @@
void registerDispatcher(Dispatcher dispatcher);
+ /**
+ * register a listener for incoming status changes of ManagementServers
+ *
+ * @param administrator the object administrating statusses
Review comment:
typo.
##########
File path: framework/cluster/src/main/java/com/cloud/cluster/dao/ManagementServerHostDao.java
##########
@@ -47,7 +47,12 @@
List<ManagementServerHostVO> listBy(ManagementServerHost.State... states);
- public List<Long> listOrphanMsids();
+ /**
+ * actually lists msids for whihc hosts are orphaned, i.e. misids that hosts refer as their owning ms whilst no mshost entry exists with those msids
Review comment:
typo.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r782326772
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -335,61 +362,144 @@ public InfrastructureResponse listInfrastructure() {
}
final Long clusterId = cluster.getId();
- // Thresholds
- final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
- final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
- final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
- final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
-
// CPU and memory capacities
final CapacityDaoImpl.SummedCapacity cpuCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_CPU, null, clusterId);
final CapacityDaoImpl.SummedCapacity memoryCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_MEMORY, null, clusterId);
- final Metrics metrics = new Metrics(cpuCapacity, memoryCapacity);
+ final HostMetrics hostMetrics = new HostMetrics(cpuCapacity, memoryCapacity);
for (final Host host: hostDao.findByClusterId(clusterId)) {
if (host == null || host.getType() != Host.Type.Routing) {
continue;
}
if (host.getStatus() == Status.Up) {
- metrics.incrUpResources();
+ hostMetrics.incrUpResources();
}
- metrics.incrTotalResources();
- updateHostMetrics(metrics, hostJoinDao.findById(host.getId()));
+ hostMetrics.incrTotalResources();
+ updateHostMetrics(hostMetrics, hostJoinDao.findById(host.getId()));
}
metricsResponse.setState(clusterResponse.getAllocationState(), clusterResponse.getManagedState());
- metricsResponse.setResources(metrics.getUpResources(), metrics.getTotalResources());
- // CPU
- metricsResponse.setCpuTotal(metrics.getTotalCpu());
- metricsResponse.setCpuAllocated(metrics.getCpuAllocated(), metrics.getTotalCpu());
- if (metrics.getCpuUsedPercentage() > 0L) {
- metricsResponse.setCpuUsed(metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
- metricsResponse.setCpuMaxDeviation(metrics.getMaximumCpuUsage(), metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
+ metricsResponse.setResources(hostMetrics.getUpResources(), hostMetrics.getTotalResources());
+ // add CPU and Memory metrics
+ addHostCpuMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+ addHostMemoryMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+
+ metricsResponse.setHasAnnotation(clusterResponse.hasAnnotation());
+ metricsResponses.add(metricsResponse);
+ }
+ return metricsResponses;
+ }
+
+ private void addHostMemoryMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setMemTotal(hostMetrics.getTotalMemory());
+ metricsResponse.setMemAllocated(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory());
+ if (hostMetrics.getMemoryUsed() > 0L) {
+ metricsResponse.setMemUsed(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory());
+ metricsResponse.setMemMaxDeviation(hostMetrics.getMaximumMemoryUsage(), hostMetrics.getMemoryUsed(), hostMetrics.getTotalHosts());
+ }
+ // Memory thresholds
+ final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
+ final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
+ metricsResponse.setMemoryUsageThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryUsageDisableThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryAllocatedThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryAllocatedDisableThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ }
+
+ private void addHostCpuMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setCpuTotal(hostMetrics.getTotalCpu());
+ metricsResponse.setCpuAllocated(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu());
+ if (hostMetrics.getCpuUsedPercentage() > 0L) {
+ metricsResponse.setCpuUsed(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ metricsResponse.setCpuMaxDeviation(hostMetrics.getMaximumCpuUsage(), hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ }
+ // CPU thresholds
+ final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
+ final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
+ metricsResponse.setCpuUsageThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuThreshold);
+ metricsResponse.setCpuUsageDisableThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuDisableThreshold);
+ metricsResponse.setCpuAllocatedThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuThreshold);
+ metricsResponse.setCpuAllocatedDisableThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuDisableThreshold);
+ }
+
+
+ @Override
+ public List<ManagementServerMetricsResponse> listManagementServerMetrics(List<ManagementServerResponse> managementServerResponses) {
+ final List<ManagementServerMetricsResponse> metricsResponses = new ArrayList<>();
+ if(LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("getting metrics for %d MS hosts", managementServerResponses.size()));
+ }
+ for (final ManagementServerResponse managementServerResponse: managementServerResponses) {
+ if(LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("processing metrics for MS hosts %s", managementServerResponse.getId()));
}
- // Memory
- metricsResponse.setMemTotal(metrics.getTotalMemory());
- metricsResponse.setMemAllocated(metrics.getMemoryAllocated(), metrics.getTotalMemory());
- if (metrics.getMemoryUsed() > 0L) {
- metricsResponse.setMemUsed(metrics.getMemoryUsed(), metrics.getTotalMemory());
- metricsResponse.setMemMaxDeviation(metrics.getMaximumMemoryUsage(), metrics.getMemoryUsed(), metrics.getTotalHosts());
+ ManagementServerMetricsResponse metricsResponse = new ManagementServerMetricsResponse();
+
+ try {
+ BeanUtils.copyProperties(metricsResponse, managementServerResponse);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("bean copy result %s", new ReflectionToStringBuilder(metricsResponse, ToStringStyle.SIMPLE_STYLE).toString()));
+ }
+ } catch (IllegalAccessException | InvocationTargetException e) {
+ throw new ServerApiException(ApiErrorCode.INTERNAL_ERROR, "Failed to generate zone metrics response");
}
- // CPU thresholds
- metricsResponse.setCpuUsageThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuThreshold);
- metricsResponse.setCpuUsageDisableThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuDisableThreshold);
- metricsResponse.setCpuAllocatedThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuThreshold);
- metricsResponse.setCpuAllocatedDisableThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuDisableThreshold);
- // Memory thresholds
- metricsResponse.setMemoryUsageThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryUsageDisableThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setMemoryAllocatedThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryAllocatedDisableThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setHasAnnotation(clusterResponse.hasAnnotation());
+ updateManagementServerMetrics(metricsResponse, managementServerResponse);
+
metricsResponses.add(metricsResponse);
}
return metricsResponses;
}
+ /**
+ * get the transient/in memory data
+ * @param metricsResponse
+ * @param managementServerResponse
Review comment:
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1063807135
Packaging result: :heavy_multiplication_x: el7 :heavy_multiplication_x: el8 :heavy_multiplication_x: debian :heavy_multiplication_x: suse15. SL-JID 2826
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
sureshanaparti removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1004563181
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1082463055
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r733369444
##########
File path: server/pom.xml
##########
@@ -170,6 +170,37 @@
<groupId>org.influxdb</groupId>
<artifactId>influxdb-java</artifactId>
</dependency>
+<!-- <dependency>-->
+<!-- <groupId>io.dropwizard.metrics</groupId>-->
+<!-- <artifactId>metrics-core</artifactId>-->
+<!-- <version>3.0.2</version>-->
+<!-- </dependency>-->
+<!-- <dependency>-->
+<!-- <groupId>io.dropwizard.metrics</groupId>-->
+<!-- <artifactId>metrics-jvm</artifactId>-->
+<!-- <version>3.0.2</version>-->
+<!-- </dependency>-->
Review comment:
no, I'll clean up the code before taking out of draft.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r774440619
##########
File path: ui/public/locales/en.json
##########
@@ -2164,6 +2180,13 @@
"label.system.vm.scaled.up": "System VM Scaled Up",
"label.system.vms": "System VMs",
"label.system.wide.capacity": "System-wide capacity",
+"label.systemcycleusage": "User-, System and Idle Cycles",
Review comment:
```suggestion
"label.systemcycleusage": "User, System and Idle Cycles",
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1028053548
@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] tampler commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
tampler commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-994728144
Hello Daan.
Am right that this PR tries to implement a kinda of Metrics Collector and Exporter, similar to [this](https://github.com/prometheus/node_exporter) one ?
As per your code, most features are already implemented and have a Prometheus endpoint. Is that correct ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r752093957
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -564,11 +657,352 @@ protected Point creteInfluxDbPoint(Object metricsObject) {
}
}
+ Gson gson;
+
+ class ManagementServerCollector extends AbstractStatsCollector {
+ @Override
+ protected void runInContext() {
+ LOGGER.debug(String.format("%s is running...", this.getClass().getSimpleName()));
+ long msid = ManagementServerNode.getManagementServerId();
+ ManagementServerHostVO mshost = null;
+ ManagementServerHostStatsEntry hostStatsEntry = null;
+ try {
+ mshost = managementServerHostDao.findByMsid(msid);
+ // get local data
+ hostStatsEntry = getDataFrom(mshost);
+ managementServerHostStats.put(mshost.getUuid(), hostStatsEntry);
+ // send to other hosts
+ clusterManager.publishStatus(gson.toJson(hostStatsEntry));
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to retrieve host stats", t);
+ }
+ try {
+ // send to DB
+ storeStatus(hostStatsEntry, mshost);
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to store host state", t);
+ }
+ }
+
+ private void storeStatus(ManagementServerHostStatsEntry hostStatsEntry, ManagementServerHostVO mshost) {
+ if (hostStatsEntry == null || mshost == null) {
+ return;
+ }
+ ManagementServerStatusVO msStats = managementServerStatusDao.findByMsId(hostStatsEntry.getManagementServerHostUuid());
+ if (msStats == null) {
+ LOGGER.info(String.format("creating new status info record for host %s - %s",
+ mshost.getName(),
+ hostStatsEntry.getManagementServerHostUuid()));
+ msStats = new ManagementServerStatusVO();
+ msStats.setMsId(hostStatsEntry.getManagementServerHostUuid());
+ }
+ msStats.setOsDistribution(hostStatsEntry.getOsDistribution()); // for now just the bunch details come later
+ msStats.setJavaName(hostStatsEntry.getJvmVendor());
+ msStats.setJavaVersion(hostStatsEntry.getJvmVersion());
+ Date startTime = new Date(hostStatsEntry.getStartTime());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("reporting starttime %s", startTime));
+ }
+ msStats.setLastStart(startTime);
+ msStats.setUpdated(new Date());
+ managementServerStatusDao.persist(msStats);
+ }
+
+ @NotNull
+ private ManagementServerHostStatsEntry getDataFrom(ManagementServerHostVO mshost) {
+ ManagementServerHostStatsEntry newEntry = new ManagementServerHostStatsEntry();
+ LOGGER.debug("Metrics collection start...");
+ newEntry.setManagementServerHostId(mshost.getId());
+ newEntry.setManagementServerHostUuid(mshost.getUuid());
+ retrieveSession(newEntry);
+ getJvmDimensions(newEntry);
+ LOGGER.debug("Metrics collection extra...");
+ getRuntimeData(newEntry);
+ getCpuData(newEntry);
+ getMemoryData(newEntry);
+ // newEntry must now include a pid!
+ getProcFsData(newEntry);
+ getFsData(newEntry);
+ getDataBaseStatistics(newEntry, mshost.getMsid());
+ gatherAllMetrics(newEntry);
+ LOGGER.debug("Metrics collection end!");
+ return newEntry;
+ }
+
+ private void retrieveSession(ManagementServerHostStatsEntry newEntry) {
+ long sessions = ApiSessionListener.getSessionCount();
+ newEntry.setSessions(sessions);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("Sessions found in Api %d vs context %d", sessions,ApiSessionListener.getNumberOfSessions()));
+ } else {
+ LOGGER.debug("Sessions active: " + sessions);
+ }
+ }
+
+ private void getDataBaseStatistics(ManagementServerHostStatsEntry newEntry, long msid) {
+ int count = _hostDao.countByMs(msid);
+ newEntry.setAgentCount(count);
+ }
+
+ private void getMemoryData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ MemoryMXBean mxBean = ManagementFactory.getMemoryMXBean();
+ newEntry.setTotalInit(mxBean.getHeapMemoryUsage().getInit() + mxBean.getNonHeapMemoryUsage().getInit());
+ newEntry.setTotalUsed(mxBean.getHeapMemoryUsage().getUsed() + mxBean.getNonHeapMemoryUsage().getUsed());
+ newEntry.setTotalMax(mxBean.getHeapMemoryUsage().getMax() + mxBean.getNonHeapMemoryUsage().getMax());
+ newEntry.setTotalCommitted(mxBean.getHeapMemoryUsage().getCommitted() + mxBean.getNonHeapMemoryUsage().getCommitted());
+ }
+
+ private void getCpuData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final OperatingSystemMXBean mxBean = ManagementFactory.getOperatingSystemMXBean();
+ newEntry.setAvailableProcessors(mxBean.getAvailableProcessors());
+ newEntry.setLoadAverage(mxBean.getSystemLoadAverage());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics processors - %d , loadavg - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getLoadAverage()));
+ }
+ }
+
+ private void getRuntimeData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final RuntimeMXBean mxBean = ManagementFactory.getRuntimeMXBean();
+ newEntry.setUptime(mxBean.getUptime());
+ newEntry.setStartTime(mxBean.getStartTime());
+ newEntry.setProcessId(mxBean.getPid());
+ newEntry.setJvmName(mxBean.getName());
+ newEntry.setJvmVendor(mxBean.getVmVendor());
+ newEntry.setJvmVersion(mxBean.getVmVersion());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics uptime - %d , starttime - %d",
+ newEntry.getUptime(),
+ newEntry.getStartTime()));
+ }
+ }
+
+ private void getJvmDimensions(@NotNull ManagementServerHostStatsEntry newEntry) {
+ Runtime runtime = Runtime.getRuntime();
+ newEntry.setTotalMemoryBytes(runtime.totalMemory());
+ newEntry.setFreeMemoryBytes(runtime.freeMemory());
+ newEntry.setAvailableProcessors(runtime.availableProcessors());
+ newEntry.setTotalMax(runtime.maxMemory());
+ //long maxMem = runtime.maxMemory();
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics proc - %d , maxMem - %d , totalMemory - %d , freeMemory - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getTotalMax(),
+ newEntry.getTotalMemoryBytes(),
+ newEntry.getFreeMemoryBytes()));
+ }
+ }
+
+ /**
+ * As for data from outside the JVM, we only rely on /proc/ contained data.
+ *
+ * @param newEntry
+ */
+ private void getProcFsData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ String OS = Script.runSimpleBashScript("cat /proc/version");
+ newEntry.setOsDistribution(OS);
+ String mem = Script.runSimpleBashScript("cat /proc/meminfo | grep MemTotal | cut -f 2 -d ':' | tr -d 'a-zA-z '").trim();
Review comment:
This is not available in java 11 :(
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] shwstppr commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
shwstppr commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r762840155
##########
File path: ui/src/components/view/Metrics.vue
##########
@@ -0,0 +1,196 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+<template>
+ <div>
+ <a-row :gutter="12">
+ <a-col :md="24">
+ <a-card class="breadcrumb-card">
+ <a-row>
+ <a-col :span="24" style="padding-left: 12px">
+ <breadcrumb>
+ <a-tooltip placement="bottom" slot="end">
+ <template slot="title">{{ $t('label.refresh') }}</template>
+ <a-button
+ style="margin-top: 4px"
+ :loading="loading"
+ shape="round"
+ size="small"
+ icon="reload"
+ @click="fetchDetails()"
+ >{{ $t('label.refresh') }}</a-button>
+ </a-tooltip>
+ </breadcrumb>
+ </a-col>
+ </a-row>
+ </a-card>
+ </a-col>
+ </a-row>
+ <a-divider/>
+ <a-row>
+ <a-col :md="24" :lg="12" :gutter="12">
+ <a-card>
+ <a-table
+ class="metric-card"
+ :columns="columns"
+ :loading="loading"
+ :data-source="dbMetrics"
+ :pagination="{ pageSize: 15}"
+ size="middle"
+ >
+ <template slot="title">
+ <a>{{ $t('label.desc.db.stats') }}</a>
+ </template>
Review comment:
Please move this out of `a-table` tag and remove `<a>` for consistency
```
<a-card>
<template slot="title">
{{ $t('label.desc.db.stats') }}
</template>
<a-table
class="metric-card"
:columns="columns"
:loading="loading"
:data-source="dbMetrics"
:pagination="{ pageSize: 15}"
size="middle"
>
```
##########
File path: ui/public/locales/en.json
##########
@@ -760,7 +764,9 @@
"label.deny": "Deny",
"label.deployasis":"Read VM settings from OVA",
"label.deploymentplanner": "Deployment planner",
+"label.desc.db.stats": "DataBase Statistics",
Review comment:
```suggestion
"label.desc.db.stats": "Database Statistics",
```
##########
File path: ui/src/components/view/Metrics.vue
##########
@@ -0,0 +1,196 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+<template>
+ <div>
+ <a-row :gutter="12">
+ <a-col :md="24">
+ <a-card class="breadcrumb-card">
+ <a-row>
+ <a-col :span="24" style="padding-left: 12px">
+ <breadcrumb>
+ <a-tooltip placement="bottom" slot="end">
+ <template slot="title">{{ $t('label.refresh') }}</template>
+ <a-button
+ style="margin-top: 4px"
+ :loading="loading"
+ shape="round"
+ size="small"
+ icon="reload"
+ @click="fetchDetails()"
+ >{{ $t('label.refresh') }}</a-button>
+ </a-tooltip>
+ </breadcrumb>
+ </a-col>
+ </a-row>
+ </a-card>
+ </a-col>
+ </a-row>
+ <a-divider/>
+ <a-row>
+ <a-col :md="24" :lg="12" :gutter="12">
+ <a-card>
+ <a-table
+ class="metric-card"
+ :columns="columns"
+ :loading="loading"
+ :data-source="dbMetrics"
+ :pagination="{ pageSize: 15}"
+ size="middle"
+ >
+ <template slot="title">
+ <a>{{ $t('label.desc.db.stats') }}</a>
+ </template>
+ </a-table>
+ </a-card>
+ </a-col>
+ <a-col :md="24" :lg="12" :gutter="12">
+ <a-card>
+ <a-table
+ class="metric-card"
+ :columns="columns"
+ :data-source="usageMetrics"
+ :pagination="false"
+ size="middle"
+ >
+ <template slot="title">
+ <a>{{ $t('label.desc.usage.stats') }}</a>
+ </template>
+ </a-table>
+ </a-card>
+ </a-col>
+ </a-row>
+ </div>
+</template>
+
+<script>
+import { api } from '@/api'
+import { genericCompare } from '@/utils/sort.js'
+import Breadcrumb from '@/components/widgets/Breadcrumb'
+
+export default {
+ name: 'Metrics',
+ components: {
+ Breadcrumb
+ },
+ props: {
+ loading: {
+ type: Boolean,
+ default: false
+ }
+ },
+ data () {
+ return {
+ dbMetrics: [],
+ usageMetrics: [],
+ columns: [
+ {
+ title: this.$t('label.name'),
+ sorter: true,
+ dataIndex: 'name',
+ width: '30%'
+ },
+ {
+ title: this.$t('label.value'),
+ dataIndex: 'value'
+ }
+ ]
+ }
+ },
+ mounted () {
+ this.fetchDetails()
+ },
+ watch: {
+ },
+ methods: {
+ fetchDbMetrics () {
+ var metrics
+ api('listDbMetrics').then(json => {
+ metrics = this.mapToArray(json.listdbmetricsresponse.dbMetrics)
+ this.dbMetrics = metrics
+ })
+ return metrics
+ },
+ fetchUsageMetrics () {
+ var metrics
+ api('listUsageServerMetrics').then(json => {
+ metrics = this.mapToArray(json.listusageservermetricsresponse.usageMetrics)
+ this.usageMetrics = metrics
+ })
+ return metrics
+ },
+ mapToArray (map) {
+ /* eslint-disable no-unused-vars */
+ var array = []
+ for (var key in map) {
+ if (key === 'replicas') {
+ map[key].forEach(function (value, i) {
+ var metric = {}
+ var name = 'replica[' + i + ']'
+ metric.name = name
+ metric.value = value
+ array.push(metric)
+ })
+ } else if (key === 'loadaverages') {
+ map[key].forEach(function (value, i) {
+ var metric = {}
+ metric.name = 'load-average[' + i + ']'
+ metric.value = value
+ array.push(metric)
+ })
+ } else {
+ var metric = {}
+ metric.name = key
+ metric.value = map[key]
+ array.push(metric)
+ }
+ }
+ /* eslint-enable no-unused-vars */
+ return array
+ },
+ fetchDetails () {
+ this.dbMetrics = this.fetchDbMetrics()
+ this.usageMetrics = this.fetchUsageMetrics()
+ },
+ fetchUsageListData () {
+ this.columns = []
+ this.columns.push({
+ dataIndex: 'name',
+ title: this.$t('label.name'),
+ sorter: function (a, b) { return genericCompare(a[this.dataIndex] || '', b[this.dataIndex] || '') }
Review comment:
Sorting didn't work for me in my test
##########
File path: ui/src/components/view/Metrics.vue
##########
@@ -0,0 +1,196 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+<template>
+ <div>
+ <a-row :gutter="12">
+ <a-col :md="24">
+ <a-card class="breadcrumb-card">
+ <a-row>
+ <a-col :span="24" style="padding-left: 12px">
+ <breadcrumb>
+ <a-tooltip placement="bottom" slot="end">
+ <template slot="title">{{ $t('label.refresh') }}</template>
+ <a-button
+ style="margin-top: 4px"
+ :loading="loading"
+ shape="round"
+ size="small"
+ icon="reload"
+ @click="fetchDetails()"
+ >{{ $t('label.refresh') }}</a-button>
+ </a-tooltip>
+ </breadcrumb>
+ </a-col>
+ </a-row>
+ </a-card>
+ </a-col>
+ </a-row>
+ <a-divider/>
+ <a-row>
+ <a-col :md="24" :lg="12" :gutter="12">
+ <a-card>
+ <a-table
+ class="metric-card"
+ :columns="columns"
+ :loading="loading"
+ :data-source="dbMetrics"
+ :pagination="{ pageSize: 15}"
Review comment:
Use pagination similar to other pages in UI (with option to select page size and left aligned) for consistency
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-998908159
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1982
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
sureshanaparti removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-995989389
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] tampler commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
tampler commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-994754407
Looks like a duplicate of already implemented feature #5778
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-995895958
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-972636823
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1722
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland closed pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland closed pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r736307987
##########
File path: api/src/main/java/org/apache/cloudstack/api/command/admin/management/ListMgmtsCmd.java
##########
@@ -75,4 +77,9 @@ public void execute() {
response.setResponseName(getCommandName());
this.setResponseObject(response);
}
+
+ protected void updateMngmntResponse(List<ManagementServerResponse> response) {
Review comment:
```suggestion
protected void updateMgmtServersResponse(List<ManagementServerResponse> response) {
```
(or) _updateManagementServersResponse()_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] GutoVeronezi commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
GutoVeronezi commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r736463244
##########
File path: framework/cluster/src/main/java/com/cloud/cluster/ClusterManagerImpl.java
##########
@@ -1141,7 +1183,10 @@ private boolean pingManagementNode(final ManagementServerHostVO mshost) {
return true;
} catch (final IOException e) {
if (e instanceof ConnectException) {
- s_logger.error("Unable to ping management server at " + targetIp + ":" + mshost.getServicePort() + " due to ConnectException", e);
+ s_logger.error("Unable to ping management server at " + targetIp + ":" + mshost.getServicePort() + " due to ConnectException");
+ if (s_logger.isDebugEnabled()) {
+ s_logger.debug("Unable to ping management server at " + targetIp + ":" + mshost.getServicePort() + " due to ConnectException", e);
Review comment:
Wouldn't be better to pass the exception in the `s_logger.error` call?
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -344,52 +355,107 @@ public InfrastructureResponse listInfrastructure() {
// CPU and memory capacities
final CapacityDaoImpl.SummedCapacity cpuCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_CPU, null, clusterId);
final CapacityDaoImpl.SummedCapacity memoryCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_MEMORY, null, clusterId);
- final Metrics metrics = new Metrics(cpuCapacity, memoryCapacity);
+ final org.apache.cloudstack.metrics.MetricsServiceImpl.HostMetrics hostMetrics = new org.apache.cloudstack.metrics.MetricsServiceImpl.HostMetrics(cpuCapacity, memoryCapacity);
for (final Host host: hostDao.findByClusterId(clusterId)) {
if (host == null || host.getType() != Host.Type.Routing) {
continue;
}
if (host.getStatus() == Status.Up) {
- metrics.incrUpResources();
+ hostMetrics.incrUpResources();
}
- metrics.incrTotalResources();
- updateHostMetrics(metrics, hostJoinDao.findById(host.getId()));
+ hostMetrics.incrTotalResources();
+ updateHostMetrics(hostMetrics, hostJoinDao.findById(host.getId()));
}
metricsResponse.setState(clusterResponse.getAllocationState(), clusterResponse.getManagedState());
- metricsResponse.setResources(metrics.getUpResources(), metrics.getTotalResources());
+ metricsResponse.setResources(hostMetrics.getUpResources(), hostMetrics.getTotalResources());
// CPU
- metricsResponse.setCpuTotal(metrics.getTotalCpu());
- metricsResponse.setCpuAllocated(metrics.getCpuAllocated(), metrics.getTotalCpu());
- if (metrics.getCpuUsedPercentage() > 0L) {
- metricsResponse.setCpuUsed(metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
- metricsResponse.setCpuMaxDeviation(metrics.getMaximumCpuUsage(), metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
+ metricsResponse.setCpuTotal(hostMetrics.getTotalCpu());
+ metricsResponse.setCpuAllocated(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu());
+ if (hostMetrics.getCpuUsedPercentage() > 0L) {
+ metricsResponse.setCpuUsed(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ metricsResponse.setCpuMaxDeviation(hostMetrics.getMaximumCpuUsage(), hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
}
// Memory
- metricsResponse.setMemTotal(metrics.getTotalMemory());
- metricsResponse.setMemAllocated(metrics.getMemoryAllocated(), metrics.getTotalMemory());
- if (metrics.getMemoryUsed() > 0L) {
- metricsResponse.setMemUsed(metrics.getMemoryUsed(), metrics.getTotalMemory());
- metricsResponse.setMemMaxDeviation(metrics.getMaximumMemoryUsage(), metrics.getMemoryUsed(), metrics.getTotalHosts());
+ metricsResponse.setMemTotal(hostMetrics.getTotalMemory());
+ metricsResponse.setMemAllocated(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory());
+ if (hostMetrics.getMemoryUsed() > 0L) {
+ metricsResponse.setMemUsed(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory());
+ metricsResponse.setMemMaxDeviation(hostMetrics.getMaximumMemoryUsage(), hostMetrics.getMemoryUsed(), hostMetrics.getTotalHosts());
}
// CPU thresholds
- metricsResponse.setCpuUsageThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuThreshold);
- metricsResponse.setCpuUsageDisableThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuDisableThreshold);
- metricsResponse.setCpuAllocatedThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuThreshold);
- metricsResponse.setCpuAllocatedDisableThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuDisableThreshold);
+ metricsResponse.setCpuUsageThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuThreshold);
+ metricsResponse.setCpuUsageDisableThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuDisableThreshold);
+ metricsResponse.setCpuAllocatedThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuThreshold);
+ metricsResponse.setCpuAllocatedDisableThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuDisableThreshold);
// Memory thresholds
- metricsResponse.setMemoryUsageThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryUsageDisableThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setMemoryAllocatedThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryAllocatedDisableThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryUsageThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryUsageDisableThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryAllocatedThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryAllocatedDisableThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
Review comment:
We could split this code in several methods with proper names, avoiding long methods and comments.
##########
File path: framework/cluster/src/main/java/com/cloud/cluster/ManagementServerStatusVO.java
##########
@@ -0,0 +1,195 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.cluster;
+
+import com.cloud.utils.db.GenericDao;
+import org.apache.cloudstack.management.ManagementServerStatus;
+
+import javax.persistence.Column;
+import javax.persistence.Entity;
+import javax.persistence.GeneratedValue;
+import javax.persistence.GenerationType;
+import javax.persistence.Id;
+import javax.persistence.Table;
+import javax.persistence.Temporal;
+import javax.persistence.TemporalType;
+import java.util.Date;
+
+@Entity
+@Table(name = "mshost_status")
+public class ManagementServerStatusVO implements ManagementServerStatus {
+
+ @Id
+ @GeneratedValue(strategy = GenerationType.IDENTITY)
+ @Column(name = "id")
+ private long id;
+
+ @Column(name = "ms_id", nullable = false)
+ private String msId;
+
+ @Temporal(TemporalType.TIMESTAMP)
+ @Column(name="last_start")
+ private Date lastStart;
+
+ @Temporal(TemporalType.TIMESTAMP)
+ @Column(name="last_stop")
+ private Date lastStop;
+
+ @Temporal(TemporalType.TIMESTAMP)
+ @Column(name="last_boot")
+ private Date lastBoot;
+
+ @Temporal(TemporalType.TIMESTAMP)
+ @Column(name="last_down")
+ private Date lastDown;
+
+ @Column(name="os_distribution")
+ private String osDistribution;
+
+ @Column(name="java_name")
+ private String javaName;
+
+ @Column(name="java_version")
+ private String javaVersion;
+
+ @Temporal(TemporalType.TIMESTAMP)
+ @Column(name = "updated")
+ private Date updated;
+
+ @Column(name = GenericDao.CREATED_COLUMN)
+ private Date created;
+
+ @Column(name = GenericDao.REMOVED_COLUMN)
+ private Date removed;
+
+
+ public ManagementServerStatusVO() {
+ }
+
+ @Override
+ public long getId() {
+ return id;
+ }
+
+ public void setId(long id) {
+ this.id = id;
+ }
+
+ @Override
+ public String getMsId() {
+ return msId;
+ }
+
+ public void setMsId(String msId) {
+ this.msId = msId;
+ }
+
+ @Override
+ public Date getLastStart() {
+ return lastStart;
+ }
+
+ public void setLastStart(Date lastStart) {
+ this.lastStart = lastStart;
+ }
+
+ @Override
+ public Date getLastStop() {
+ return lastStop;
+ }
+
+ public void setLastStop(Date lastStop) {
+ this.lastStop = lastStop;
+ }
+
+ @Override
+ public Date getLastBoot() {
+ return lastBoot;
+ }
+
+ public void setLastBoot(Date lastBoot) {
+ this.lastBoot = lastBoot;
+ }
+
+ @Override
+ public Date getLastDown() {
+ return lastDown;
+ }
+
+ public void setLastDown(Date lastDown) {
+ this.lastDown = lastDown;
+ }
+
+ @Override
+ public String getOsDistribution() {
+ return osDistribution;
+ }
+
+ public void setOsDistribution(String osDistribution) {
+ this.osDistribution = osDistribution;
+ }
+
+ @Override
+ public String getJavaName() {
+ return javaName;
+ }
+
+ public void setJavaName(String javaName) {
+ this.javaName = javaName;
+ }
+
+ @Override
+ public String getJavaVersion() {
+ return javaVersion;
+ }
+
+ public void setJavaVersion(String javaVersion) {
+ this.javaVersion = javaVersion;
+ }
+
+ @Override
+ public Date getUpdated() {
+ return updated;
+ }
+
+ public void setUpdated(Date updated) {
+ this.updated = updated;
+ }
+
+ @Override
+ public Date getCreated() {
+ return created;
+ }
+
+ public void setCreated(Date created) {
+ this.created = created;
+ }
+
+ @Override
+ public Date getRemoved() {
+ return removed;
+ }
+
+ public void setRemoved(Date removedTime) {
+ removed = removedTime;
+ }
+
+ @Override
+ public String toString() {
+ return new org.apache.commons.lang3.builder.ReflectionToStringBuilder(this, org.apache.commons.lang3.builder.ToStringStyle.SIMPLE_STYLE).toString();
Review comment:
Could we reference the path in the import and only use the method here?
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -122,16 +133,16 @@ private Double findRatioValue(final String value) {
return 1.0;
}
- private void updateHostMetrics(final Metrics metrics, final HostJoinVO host) {
- metrics.incrTotalHosts();
- metrics.addCpuAllocated(host.getCpuReservedCapacity() + host.getCpuUsedCapacity());
- metrics.addMemoryAllocated(host.getMemReservedCapacity() + host.getMemUsedCapacity());
+ private void updateHostMetrics(final org.apache.cloudstack.metrics.MetricsServiceImpl.HostMetrics hostMetrics, final HostJoinVO host) {
Review comment:
@DaanHoogland, is there any technical reason to use the full path of this subclass, along this class?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1069309056
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 2907
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1067196827
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1071988873
@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1069269900
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
nvazquez removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1070175235
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1069309056
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r761915391
##########
File path: api/src/main/java/com/cloud/server/ManagementServerHostStats.java
##########
@@ -0,0 +1,105 @@
+//
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+package com.cloud.server;
+
+/**
+ * management server related stats
+ */
+public interface ManagementServerHostStats {
+ long getManagementServerHostId();
+
+ String getManagementServerHostUuid();
+
+ long getSessions();
+
+ double getCpuUtilization();
+
+ long getTotalMemoryBytes();
+
+ double getFreeMemoryBytes();
+
+ double getProcessMemoryBytes();
+
+ long getUptime();
+
+ long getStartTime();
Review comment:
Is this host start time?
```suggestion
long getSystemStartTime();
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r761916957
##########
File path: api/src/main/java/org/apache/cloudstack/api/ApiConstants.java
##########
@@ -226,10 +226,14 @@
public static final String IS_RECURSIVE = "isrecursive";
public static final String ISO_FILTER = "isofilter";
public static final String ISO_GUEST_OS_NONE = "None";
+ public static final String JAVA_DISTRIBUTION = "javadistribution";
+ public static final String JAVA_VERSION = "javaversion";
public static final String JOB_ID = "jobid";
public static final String JOB_STATUS = "jobstatus";
public static final String LABEL = "label";
public static final String LASTNAME = "lastname";
+ public static final String LAST_STOP = "laststop";
+ public static final String LAST_START = "laststart";
Review comment:
```suggestion
public static final String LAST_SYSTEM_START = "lastsystemstart";
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1001872137
<b>Trillian test result (tid-2740)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 31276 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5588-t2740-kvm-centos7.zip
Smoke tests completed. 91 look OK, 0 have errors
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1004630098
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 2084
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1016343498
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 2247
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1016419713
the rat test started failing without any files being added or the root pom.xml being edited :interrobang: :roll_eyes:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r783083555
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -417,73 +527,186 @@ public InfrastructureResponse listInfrastructure() {
// CPU and memory capacities
final CapacityDaoImpl.SummedCapacity cpuCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_CPU, zoneId, null);
final CapacityDaoImpl.SummedCapacity memoryCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_MEMORY, zoneId, null);
- final Metrics metrics = new Metrics(cpuCapacity, memoryCapacity);
+ final HostMetrics hostMetrics = new HostMetrics(cpuCapacity, memoryCapacity);
for (final Cluster cluster : clusterDao.listClustersByDcId(zoneId)) {
if (cluster == null) {
continue;
}
- metrics.incrTotalResources();
+ hostMetrics.incrTotalResources();
if (cluster.getAllocationState() == Grouping.AllocationState.Enabled
&& cluster.getManagedState() == Managed.ManagedState.Managed) {
- metrics.incrUpResources();
+ hostMetrics.incrUpResources();
}
for (final Host host: hostDao.findByClusterId(cluster.getId())) {
if (host == null || host.getType() != Host.Type.Routing) {
continue;
}
- updateHostMetrics(metrics, hostJoinDao.findById(host.getId()));
+ updateHostMetrics(hostMetrics, hostJoinDao.findById(host.getId()));
}
}
metricsResponse.setHasAnnotation(zoneResponse.hasAnnotation());
metricsResponse.setState(zoneResponse.getAllocationState());
- metricsResponse.setResource(metrics.getUpResources(), metrics.getTotalResources());
+ metricsResponse.setResource(hostMetrics.getUpResources(), hostMetrics.getTotalResources());
// CPU
- metricsResponse.setCpuTotal(metrics.getTotalCpu());
- metricsResponse.setCpuAllocated(metrics.getCpuAllocated(), metrics.getTotalCpu());
- if (metrics.getCpuUsedPercentage() > 0L) {
- metricsResponse.setCpuUsed(metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
- metricsResponse.setCpuMaxDeviation(metrics.getMaximumCpuUsage(), metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
+ metricsResponse.setCpuTotal(hostMetrics.getTotalCpu());
+ metricsResponse.setCpuAllocated(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu());
+ if (hostMetrics.getCpuUsedPercentage() > 0L) {
+ metricsResponse.setCpuUsed(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ metricsResponse.setCpuMaxDeviation(hostMetrics.getMaximumCpuUsage(), hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
}
// Memory
- metricsResponse.setMemTotal(metrics.getTotalMemory());
- metricsResponse.setMemAllocated(metrics.getMemoryAllocated(), metrics.getTotalMemory());
- if (metrics.getMemoryUsed() > 0L) {
- metricsResponse.setMemUsed(metrics.getMemoryUsed(), metrics.getTotalMemory());
- metricsResponse.setMemMaxDeviation(metrics.getMaximumMemoryUsage(), metrics.getMemoryUsed(), metrics.getTotalHosts());
+ metricsResponse.setMemTotal(hostMetrics.getTotalMemory());
+ metricsResponse.setMemAllocated(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory());
+ if (hostMetrics.getMemoryUsed() > 0L) {
+ metricsResponse.setMemUsed(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory());
+ metricsResponse.setMemMaxDeviation(hostMetrics.getMaximumMemoryUsage(), hostMetrics.getMemoryUsed(), hostMetrics.getTotalHosts());
}
// CPU thresholds
- metricsResponse.setCpuUsageThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuThreshold);
- metricsResponse.setCpuUsageDisableThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuDisableThreshold);
- metricsResponse.setCpuAllocatedThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuThreshold);
- metricsResponse.setCpuAllocatedDisableThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuDisableThreshold);
+ metricsResponse.setCpuUsageThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuThreshold);
+ metricsResponse.setCpuUsageDisableThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuDisableThreshold);
+ metricsResponse.setCpuAllocatedThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuThreshold);
+ metricsResponse.setCpuAllocatedDisableThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuDisableThreshold);
// Memory thresholds
- metricsResponse.setMemoryUsageThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryUsageDisableThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setMemoryAllocatedThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryAllocatedDisableThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryUsageThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryUsageDisableThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryAllocatedThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryAllocatedDisableThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
Review comment:
extracted and unified with the ones used in listClusterMetrics.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] GutoVeronezi commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
GutoVeronezi commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r781481450
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -335,61 +362,144 @@ public InfrastructureResponse listInfrastructure() {
}
final Long clusterId = cluster.getId();
- // Thresholds
- final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
- final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
- final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
- final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
-
// CPU and memory capacities
final CapacityDaoImpl.SummedCapacity cpuCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_CPU, null, clusterId);
final CapacityDaoImpl.SummedCapacity memoryCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_MEMORY, null, clusterId);
- final Metrics metrics = new Metrics(cpuCapacity, memoryCapacity);
+ final HostMetrics hostMetrics = new HostMetrics(cpuCapacity, memoryCapacity);
for (final Host host: hostDao.findByClusterId(clusterId)) {
if (host == null || host.getType() != Host.Type.Routing) {
continue;
}
if (host.getStatus() == Status.Up) {
- metrics.incrUpResources();
+ hostMetrics.incrUpResources();
}
- metrics.incrTotalResources();
- updateHostMetrics(metrics, hostJoinDao.findById(host.getId()));
+ hostMetrics.incrTotalResources();
+ updateHostMetrics(hostMetrics, hostJoinDao.findById(host.getId()));
}
metricsResponse.setState(clusterResponse.getAllocationState(), clusterResponse.getManagedState());
- metricsResponse.setResources(metrics.getUpResources(), metrics.getTotalResources());
- // CPU
- metricsResponse.setCpuTotal(metrics.getTotalCpu());
- metricsResponse.setCpuAllocated(metrics.getCpuAllocated(), metrics.getTotalCpu());
- if (metrics.getCpuUsedPercentage() > 0L) {
- metricsResponse.setCpuUsed(metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
- metricsResponse.setCpuMaxDeviation(metrics.getMaximumCpuUsage(), metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
+ metricsResponse.setResources(hostMetrics.getUpResources(), hostMetrics.getTotalResources());
+ // add CPU and Memory metrics
+ addHostCpuMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+ addHostMemoryMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+
+ metricsResponse.setHasAnnotation(clusterResponse.hasAnnotation());
+ metricsResponses.add(metricsResponse);
+ }
+ return metricsResponses;
+ }
+
+ private void addHostMemoryMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setMemTotal(hostMetrics.getTotalMemory());
+ metricsResponse.setMemAllocated(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory());
+ if (hostMetrics.getMemoryUsed() > 0L) {
+ metricsResponse.setMemUsed(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory());
+ metricsResponse.setMemMaxDeviation(hostMetrics.getMaximumMemoryUsage(), hostMetrics.getMemoryUsed(), hostMetrics.getTotalHosts());
+ }
+ // Memory thresholds
+ final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
+ final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
+ metricsResponse.setMemoryUsageThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryUsageDisableThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryAllocatedThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryAllocatedDisableThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ }
+
+ private void addHostCpuMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setCpuTotal(hostMetrics.getTotalCpu());
+ metricsResponse.setCpuAllocated(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu());
+ if (hostMetrics.getCpuUsedPercentage() > 0L) {
+ metricsResponse.setCpuUsed(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ metricsResponse.setCpuMaxDeviation(hostMetrics.getMaximumCpuUsage(), hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ }
+ // CPU thresholds
+ final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
+ final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
+ metricsResponse.setCpuUsageThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuThreshold);
+ metricsResponse.setCpuUsageDisableThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuDisableThreshold);
+ metricsResponse.setCpuAllocatedThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuThreshold);
+ metricsResponse.setCpuAllocatedDisableThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuDisableThreshold);
+ }
+
+
+ @Override
+ public List<ManagementServerMetricsResponse> listManagementServerMetrics(List<ManagementServerResponse> managementServerResponses) {
+ final List<ManagementServerMetricsResponse> metricsResponses = new ArrayList<>();
+ if(LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("getting metrics for %d MS hosts", managementServerResponses.size()));
+ }
+ for (final ManagementServerResponse managementServerResponse: managementServerResponses) {
+ if(LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("processing metrics for MS hosts %s", managementServerResponse.getId()));
}
- // Memory
- metricsResponse.setMemTotal(metrics.getTotalMemory());
- metricsResponse.setMemAllocated(metrics.getMemoryAllocated(), metrics.getTotalMemory());
- if (metrics.getMemoryUsed() > 0L) {
- metricsResponse.setMemUsed(metrics.getMemoryUsed(), metrics.getTotalMemory());
- metricsResponse.setMemMaxDeviation(metrics.getMaximumMemoryUsage(), metrics.getMemoryUsed(), metrics.getTotalHosts());
+ ManagementServerMetricsResponse metricsResponse = new ManagementServerMetricsResponse();
+
+ try {
+ BeanUtils.copyProperties(metricsResponse, managementServerResponse);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("bean copy result %s", new ReflectionToStringBuilder(metricsResponse, ToStringStyle.SIMPLE_STYLE).toString()));
+ }
+ } catch (IllegalAccessException | InvocationTargetException e) {
+ throw new ServerApiException(ApiErrorCode.INTERNAL_ERROR, "Failed to generate zone metrics response");
}
- // CPU thresholds
- metricsResponse.setCpuUsageThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuThreshold);
- metricsResponse.setCpuUsageDisableThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuDisableThreshold);
- metricsResponse.setCpuAllocatedThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuThreshold);
- metricsResponse.setCpuAllocatedDisableThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuDisableThreshold);
- // Memory thresholds
- metricsResponse.setMemoryUsageThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryUsageDisableThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setMemoryAllocatedThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryAllocatedDisableThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setHasAnnotation(clusterResponse.hasAnnotation());
+ updateManagementServerMetrics(metricsResponse, managementServerResponse);
+
metricsResponses.add(metricsResponse);
}
return metricsResponses;
}
+ /**
+ * get the transient/in memory data
+ * @param metricsResponse
+ * @param managementServerResponse
+ */
+ private void updateManagementServerMetrics(ManagementServerMetricsResponse metricsResponse, ManagementServerResponse managementServerResponse) {
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("getting stats for %s", managementServerResponse.getId()));
Review comment:
```suggestion
LOGGER.debug(String.format("Getting stats for MS %s.", managementServerResponse.getId()));
```
Is it possible to log more than only the id?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1011175066
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1004669656
@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland closed pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland closed pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1022857125
> the rat test started failing without any files being added or the root pom.xml being edited interrobang roll_eyes
finally it reporrted the right file and it was in there for a while already :(
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1016293266
@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1022321065
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-995897413
@sureshanaparti a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-970150206
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1715
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1004563300
@sureshanaparti a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] GutoVeronezi commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
GutoVeronezi commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1004784087
@DaanHoogland, is this PR ready for review?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] GutoVeronezi commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
GutoVeronezi commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r780488144
##########
File path: api/src/main/java/org/apache/cloudstack/api/response/ManagementServerResponse.java
##########
@@ -35,12 +38,80 @@
@SerializedName(ApiConstants.STATE)
@Param(description = "the state of the management server")
- private ManagementServerHost.State state;
+ private State state;
@SerializedName(ApiConstants.VERSION)
@Param(description = "the version of the management server")
private String version;
+ @SerializedName(ApiConstants.JAVA_DISTRIBUTION)
+ @Param(description = "the java distribution name running the management server process")
+ private String javaDistribution;
+
+ @SerializedName(ApiConstants.JAVA_VERSION)
+ @Param(description = "the version of the java distribution running the management server process")
+ private String javaVersion;
+
+ @SerializedName(ApiConstants.OS_DISTRIBUTION)
+ @Param(description = "the name of the OS distribution running on the management server")
+ private String osDistribution;
+
+ @SerializedName(ApiConstants.LAST_SERVER_START)
+ @Param(description = "the last time this Management Server was started")
+ private Date lastServerStart;
+
+ @SerializedName(ApiConstants.LAST_SERVER_STOP)
+ @Param(description = "the last time this Management Server was stopped")
+ private Date lastServerStop;
+
+ @SerializedName(ApiConstants.LAST_BOOT)
+ @Param(description = "the last time the host on which this Management Server runs, was booted")
+ private Date lastBoot;
+
+ @SerializedName(ApiConstants.KERNEL_VERSION)
+ @Param(description = "the running os kernel version for this Management Server")
Review comment:
```suggestion
@Param(description = "the running OS kernel version for this Management Server")
```
##########
File path: api/src/main/java/org/apache/cloudstack/api/response/ManagementServerResponse.java
##########
@@ -35,12 +38,80 @@
@SerializedName(ApiConstants.STATE)
@Param(description = "the state of the management server")
- private ManagementServerHost.State state;
+ private State state;
@SerializedName(ApiConstants.VERSION)
@Param(description = "the version of the management server")
private String version;
+ @SerializedName(ApiConstants.JAVA_DISTRIBUTION)
+ @Param(description = "the java distribution name running the management server process")
+ private String javaDistribution;
+
+ @SerializedName(ApiConstants.JAVA_VERSION)
+ @Param(description = "the version of the java distribution running the management server process")
+ private String javaVersion;
+
+ @SerializedName(ApiConstants.OS_DISTRIBUTION)
+ @Param(description = "the name of the OS distribution running on the management server")
+ private String osDistribution;
+
+ @SerializedName(ApiConstants.LAST_SERVER_START)
+ @Param(description = "the last time this Management Server was started")
+ private Date lastServerStart;
+
+ @SerializedName(ApiConstants.LAST_SERVER_STOP)
+ @Param(description = "the last time this Management Server was stopped")
+ private Date lastServerStop;
+
+ @SerializedName(ApiConstants.LAST_BOOT)
+ @Param(description = "the last time the host on which this Management Server runs, was booted")
Review comment:
```suggestion
@Param(description = "the last time the host on which this Management Server runs was booted")
```
##########
File path: engine/schema/src/main/java/com/cloud/host/dao/HostDaoImpl.java
##########
@@ -1287,6 +1292,13 @@ private String getHostIdsByComputeTags(List<String> offeringTags){
}
}
+ @Override
+ public int countByMs(long msid) {
Review comment:
```suggestion
public int countByMs(long msId) {
```
##########
File path: framework/cluster/src/main/java/com/cloud/cluster/ClusterManager.java
##########
@@ -21,27 +21,36 @@
import com.cloud.utils.component.Manager;
+/**
+ * the definition of the framework for inter MS communication
+ */
public interface ClusterManager extends Manager {
static final String ALERT_SUBJECT = "cluster-alert";
final ConfigKey<Integer> HeartbeatInterval = new ConfigKey<Integer>(Integer.class, "cluster.heartbeat.interval", "management-server", "1500",
"Interval to check for the heart beat between management server nodes", false);
final ConfigKey<Integer> HeartbeatThreshold = new ConfigKey<Integer>(Integer.class, "cluster.heartbeat.threshold", "management-server", "150000",
"Threshold before self-fence the management server", true);
+ /**
+ * adds a new packet to the incoming queue
Review comment:
```suggestion
* Adds a new packet to the incoming queue.
```
##########
File path: framework/cluster/src/main/java/com/cloud/cluster/ClusterManager.java
##########
@@ -21,27 +21,36 @@
import com.cloud.utils.component.Manager;
+/**
+ * the definition of the framework for inter MS communication
Review comment:
```suggestion
* The definition of the framework for inter MS communication.
```
##########
File path: framework/cluster/src/main/java/com/cloud/cluster/ClusterManager.java
##########
@@ -53,19 +62,44 @@
void registerDispatcher(Dispatcher dispatcher);
+ /**
+ * register a listener for incoming status changes of ManagementServers
Review comment:
```suggestion
* Registers a listener for incoming status changes of ManagementServers.
```
##########
File path: framework/cluster/src/main/java/com/cloud/cluster/dao/ManagementServerHostDao.java
##########
@@ -47,7 +47,12 @@
List<ManagementServerHostVO> listBy(ManagementServerHost.State... states);
- public List<Long> listOrphanMsids();
+ /**
+ * actually lists msids for which hosts are orphaned, i.e. msids that hosts refer as their owning ms whilst no mshost entry exists with those msids
Review comment:
```suggestion
* Lists msids for which hosts are orphaned, i.e. msids that hosts refer as their owning ms whilst no mshost entry exists with those msids
```
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/api/ListMgmtsMetricsCmd.java
##########
@@ -0,0 +1,56 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.cloudstack.api;
+
+import org.apache.cloudstack.acl.RoleType;
+import org.apache.cloudstack.api.command.admin.management.ListMgmtsCmd;
+import org.apache.cloudstack.api.response.ListResponse;
+import org.apache.cloudstack.api.response.ManagementServerResponse;
+import org.apache.cloudstack.metrics.MetricsService;
+import org.apache.cloudstack.response.ManagementServerMetricsResponse;
+
+import javax.inject.Inject;
+import java.util.List;
+
+@APICommand(name = ListMgmtsMetricsCmd.APINAME, description = "Lists Management Server metrics", responseObject = ManagementServerMetricsResponse.class,
+ requestHasSensitiveInfo = false, responseHasSensitiveInfo = false, responseView = ResponseObject.ResponseView.Full,
+ since = "4.17.0", authorized = {RoleType.Admin})
+public class ListMgmtsMetricsCmd extends ListMgmtsCmd {
+ public static final String APINAME = "listManagementServersMetrics";
+
+ @Parameter(name = MetricConstants.SYSTEM, type = CommandType.BOOLEAN, entityType = ManagementServerMetricsResponse.class, description = "include system level stats")
+ private boolean system;
+
+ @Inject
+ private MetricsService metricsService;
+
+ @Override
+ public String getCommandName() {
+ return APINAME.toLowerCase() + BaseCmd.RESPONSE_SUFFIX;
Review comment:
This behavior repeats along several classes (not only the changed ones), could we create an utility with this logic? Or just a method in the `BaseCmd`, passing the `APINAME`, e.g.:
```java
protected String getResponseCommandName(String commandName) {
return commandName.toLowerCase() + RESPONSE_SUFFIX;
}
```
Then, in the subclass:
```java
@Override
public String getCommandName() {
return getResponseCommandName(APINAME);
}
```
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -335,61 +362,144 @@ public InfrastructureResponse listInfrastructure() {
}
final Long clusterId = cluster.getId();
- // Thresholds
- final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
- final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
- final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
- final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
-
// CPU and memory capacities
final CapacityDaoImpl.SummedCapacity cpuCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_CPU, null, clusterId);
final CapacityDaoImpl.SummedCapacity memoryCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_MEMORY, null, clusterId);
- final Metrics metrics = new Metrics(cpuCapacity, memoryCapacity);
+ final HostMetrics hostMetrics = new HostMetrics(cpuCapacity, memoryCapacity);
for (final Host host: hostDao.findByClusterId(clusterId)) {
if (host == null || host.getType() != Host.Type.Routing) {
continue;
}
if (host.getStatus() == Status.Up) {
- metrics.incrUpResources();
+ hostMetrics.incrUpResources();
}
- metrics.incrTotalResources();
- updateHostMetrics(metrics, hostJoinDao.findById(host.getId()));
+ hostMetrics.incrTotalResources();
+ updateHostMetrics(hostMetrics, hostJoinDao.findById(host.getId()));
}
metricsResponse.setState(clusterResponse.getAllocationState(), clusterResponse.getManagedState());
- metricsResponse.setResources(metrics.getUpResources(), metrics.getTotalResources());
- // CPU
- metricsResponse.setCpuTotal(metrics.getTotalCpu());
- metricsResponse.setCpuAllocated(metrics.getCpuAllocated(), metrics.getTotalCpu());
- if (metrics.getCpuUsedPercentage() > 0L) {
- metricsResponse.setCpuUsed(metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
- metricsResponse.setCpuMaxDeviation(metrics.getMaximumCpuUsage(), metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
+ metricsResponse.setResources(hostMetrics.getUpResources(), hostMetrics.getTotalResources());
+ // add CPU and Memory metrics
+ addHostCpuMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+ addHostMemoryMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+
+ metricsResponse.setHasAnnotation(clusterResponse.hasAnnotation());
+ metricsResponses.add(metricsResponse);
+ }
+ return metricsResponses;
+ }
+
+ private void addHostMemoryMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setMemTotal(hostMetrics.getTotalMemory());
+ metricsResponse.setMemAllocated(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory());
+ if (hostMetrics.getMemoryUsed() > 0L) {
+ metricsResponse.setMemUsed(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory());
+ metricsResponse.setMemMaxDeviation(hostMetrics.getMaximumMemoryUsage(), hostMetrics.getMemoryUsed(), hostMetrics.getTotalHosts());
+ }
+ // Memory thresholds
+ final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
+ final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
+ metricsResponse.setMemoryUsageThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryUsageDisableThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryAllocatedThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryAllocatedDisableThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ }
+
+ private void addHostCpuMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setCpuTotal(hostMetrics.getTotalCpu());
+ metricsResponse.setCpuAllocated(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu());
+ if (hostMetrics.getCpuUsedPercentage() > 0L) {
+ metricsResponse.setCpuUsed(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ metricsResponse.setCpuMaxDeviation(hostMetrics.getMaximumCpuUsage(), hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ }
+ // CPU thresholds
+ final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
+ final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
+ metricsResponse.setCpuUsageThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuThreshold);
+ metricsResponse.setCpuUsageDisableThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuDisableThreshold);
+ metricsResponse.setCpuAllocatedThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuThreshold);
+ metricsResponse.setCpuAllocatedDisableThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuDisableThreshold);
+ }
+
+
+ @Override
+ public List<ManagementServerMetricsResponse> listManagementServerMetrics(List<ManagementServerResponse> managementServerResponses) {
+ final List<ManagementServerMetricsResponse> metricsResponses = new ArrayList<>();
+ if(LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("getting metrics for %d MS hosts", managementServerResponses.size()));
Review comment:
```suggestion
LOGGER.debug(String.format("Getting metrics for %d MS hosts.", managementServerResponses.size()));
```
##########
File path: engine/schema/src/main/java/com/cloud/host/dao/HostDao.java
##########
@@ -142,4 +142,11 @@
HostVO findByName(String name);
List<HostVO> listHostsWithActiveVMs(long offeringId);
+
+ /**
+ * get the number of hosts/agents this {@see ManagementServer} has responsibility over
Review comment:
```suggestion
* Retrieves the number of hosts/agents this {@see ManagementServer} has responsibility over.
```
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -417,73 +527,186 @@ public InfrastructureResponse listInfrastructure() {
// CPU and memory capacities
final CapacityDaoImpl.SummedCapacity cpuCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_CPU, zoneId, null);
final CapacityDaoImpl.SummedCapacity memoryCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_MEMORY, zoneId, null);
- final Metrics metrics = new Metrics(cpuCapacity, memoryCapacity);
+ final HostMetrics hostMetrics = new HostMetrics(cpuCapacity, memoryCapacity);
for (final Cluster cluster : clusterDao.listClustersByDcId(zoneId)) {
if (cluster == null) {
continue;
}
- metrics.incrTotalResources();
+ hostMetrics.incrTotalResources();
if (cluster.getAllocationState() == Grouping.AllocationState.Enabled
&& cluster.getManagedState() == Managed.ManagedState.Managed) {
- metrics.incrUpResources();
+ hostMetrics.incrUpResources();
}
for (final Host host: hostDao.findByClusterId(cluster.getId())) {
if (host == null || host.getType() != Host.Type.Routing) {
continue;
}
- updateHostMetrics(metrics, hostJoinDao.findById(host.getId()));
+ updateHostMetrics(hostMetrics, hostJoinDao.findById(host.getId()));
}
}
metricsResponse.setHasAnnotation(zoneResponse.hasAnnotation());
metricsResponse.setState(zoneResponse.getAllocationState());
- metricsResponse.setResource(metrics.getUpResources(), metrics.getTotalResources());
+ metricsResponse.setResource(hostMetrics.getUpResources(), hostMetrics.getTotalResources());
// CPU
- metricsResponse.setCpuTotal(metrics.getTotalCpu());
- metricsResponse.setCpuAllocated(metrics.getCpuAllocated(), metrics.getTotalCpu());
- if (metrics.getCpuUsedPercentage() > 0L) {
- metricsResponse.setCpuUsed(metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
- metricsResponse.setCpuMaxDeviation(metrics.getMaximumCpuUsage(), metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
+ metricsResponse.setCpuTotal(hostMetrics.getTotalCpu());
+ metricsResponse.setCpuAllocated(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu());
+ if (hostMetrics.getCpuUsedPercentage() > 0L) {
+ metricsResponse.setCpuUsed(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ metricsResponse.setCpuMaxDeviation(hostMetrics.getMaximumCpuUsage(), hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
}
// Memory
- metricsResponse.setMemTotal(metrics.getTotalMemory());
- metricsResponse.setMemAllocated(metrics.getMemoryAllocated(), metrics.getTotalMemory());
- if (metrics.getMemoryUsed() > 0L) {
- metricsResponse.setMemUsed(metrics.getMemoryUsed(), metrics.getTotalMemory());
- metricsResponse.setMemMaxDeviation(metrics.getMaximumMemoryUsage(), metrics.getMemoryUsed(), metrics.getTotalHosts());
+ metricsResponse.setMemTotal(hostMetrics.getTotalMemory());
+ metricsResponse.setMemAllocated(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory());
+ if (hostMetrics.getMemoryUsed() > 0L) {
+ metricsResponse.setMemUsed(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory());
+ metricsResponse.setMemMaxDeviation(hostMetrics.getMaximumMemoryUsage(), hostMetrics.getMemoryUsed(), hostMetrics.getTotalHosts());
}
// CPU thresholds
- metricsResponse.setCpuUsageThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuThreshold);
- metricsResponse.setCpuUsageDisableThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuDisableThreshold);
- metricsResponse.setCpuAllocatedThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuThreshold);
- metricsResponse.setCpuAllocatedDisableThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuDisableThreshold);
+ metricsResponse.setCpuUsageThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuThreshold);
+ metricsResponse.setCpuUsageDisableThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuDisableThreshold);
+ metricsResponse.setCpuAllocatedThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuThreshold);
+ metricsResponse.setCpuAllocatedDisableThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuDisableThreshold);
// Memory thresholds
- metricsResponse.setMemoryUsageThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryUsageDisableThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setMemoryAllocatedThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryAllocatedDisableThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryUsageThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryUsageDisableThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryAllocatedThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryAllocatedDisableThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
metricsResponses.add(metricsResponse);
}
return metricsResponses;
}
+ @Override
+ public UsageServerMetricsResponse listUsageServerMetrics() {
+ UsageServerMetricsResponse response = new UsageServerMetricsResponse();
+ response.setCollectionTime(new Date());
+ TransactionLegacy txn = TransactionLegacy.open(TransactionLegacy.USAGE_DB);
+ try {
+ response.setLastHeartbeat(usageJobDao.getLastHeartbeat());
+ response.setState(isUsageRunning()? State.Up : State.Down);
+ UsageJobVO job = usageJobDao.getNextImmediateJob();
+ if (job == null) {
+ job = usageJobDao.getLastJob();
+ }
+ response.setHostname(job == null? "N/A": job.getHost());
+ response.setLastSuccessfulJob(new Date(usageJobDao.getLastJobSuccessDateMillis()));
+ } finally {
+ txn.close();
+ TransactionLegacy swap = TransactionLegacy.open(TransactionLegacy.CLOUD_DB);
+ swap.close();
+ }
+ response.setObjectName("usageMetrics");
+ return response;
+ }
+
+ /**
+ TODO
+ â—¦ State (Up / Down) , makes no sense (no db no cloudstack)
+ â—¦ Average Queries Per Second
+ â—¦ Buffer Pool Utilization (buffer pool is used to cache the table data in memory and is accessed repeatedly by queries without requiring any disk I/O).
+ â—¦ any other relevant stats (if useful) to the response from the sql status variables.
+ * @return
+ */
+ @Override
+ public DbMetricsResponse listDbMetrics() {
+ DbMetricsResponse response = new DbMetricsResponse();
+
+ response.setCollectionTime(new Date());
+ response.setHostname(dbHostName());
+ response.setReplicas(dbReplicas());
+ getDynamicDataFromDB(response);
+ getStaticDataFromDB(response);
+
+ getQueryHistory(response);
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(new ReflectionToStringBuilder(response));
+ }
+
+ response.setObjectName("dbMetrics");
+ return response;
+ }
+
+ private void getQueryHistory(DbMetricsResponse response) {
+ Map<String, Object> dbStats = ApiDBUtils.getDbStatistics();
+ if (dbStats != null) {
+ response.setQueries((Integer)dbStats.get(DbStatsCollection.queries));
+ response.setUptime((Integer)dbStats.get(DbStatsCollection.uptime));
+ }
+
+ List<Double> loadHistory = (List<Double>) dbStats.get(DbStatsCollection.loadAvarages);
+ double[] loadAverages = new double[loadHistory.size()];
+
+ int index =0;
+ for (Double d : loadHistory) {
+ loadAverages[index++] = d;
+ }
+
+ response.setLoadAverages(loadAverages);
+ }
+
+ private void getStaticDataFromDB(DbMetricsResponse response) {
+ Map<String, String> vars = DbUtil.getDbInfo(DbStatsCollection.variables, DbStatsCollection.version, DbStatsCollection.versionComment);
+ response.setVersion(vars.get(DbStatsCollection.version));
+ response.setVersionComment(vars.get(DbStatsCollection.versionComment));
+ }
+
+ private void getDynamicDataFromDB(DbMetricsResponse response) {
+ Map<String, String> stats = DbUtil.getDbInfo(DbStatsCollection.status, DbStatsCollection.connections, DbStatsCollection.currentTlsVersion);
+ response.setConnections(Integer.parseInt(stats.get(DbStatsCollection.connections)));
+ response.setTlsVersions(stats.get(DbStatsCollection.currentTlsVersion));
+ }
+
+ private String dbHostName() {
+ Properties p = DbProperties.getDbProperties();
+ return p.getProperty("db.cloud.host");
+ }
+
+ private String[] dbReplicas() {
+ Properties p = DbProperties.getDbProperties();
+ return p.getProperty("db.cloud.replicas","").split(",");
+ }
+
+ /**
+ * returns whether a local usage server is running.
Review comment:
```suggestion
* Returns whether a local usage server is running.
```
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -335,61 +362,144 @@ public InfrastructureResponse listInfrastructure() {
}
final Long clusterId = cluster.getId();
- // Thresholds
- final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
- final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
- final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
- final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
-
// CPU and memory capacities
final CapacityDaoImpl.SummedCapacity cpuCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_CPU, null, clusterId);
final CapacityDaoImpl.SummedCapacity memoryCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_MEMORY, null, clusterId);
- final Metrics metrics = new Metrics(cpuCapacity, memoryCapacity);
+ final HostMetrics hostMetrics = new HostMetrics(cpuCapacity, memoryCapacity);
for (final Host host: hostDao.findByClusterId(clusterId)) {
if (host == null || host.getType() != Host.Type.Routing) {
continue;
}
if (host.getStatus() == Status.Up) {
- metrics.incrUpResources();
+ hostMetrics.incrUpResources();
}
- metrics.incrTotalResources();
- updateHostMetrics(metrics, hostJoinDao.findById(host.getId()));
+ hostMetrics.incrTotalResources();
+ updateHostMetrics(hostMetrics, hostJoinDao.findById(host.getId()));
}
metricsResponse.setState(clusterResponse.getAllocationState(), clusterResponse.getManagedState());
- metricsResponse.setResources(metrics.getUpResources(), metrics.getTotalResources());
- // CPU
- metricsResponse.setCpuTotal(metrics.getTotalCpu());
- metricsResponse.setCpuAllocated(metrics.getCpuAllocated(), metrics.getTotalCpu());
- if (metrics.getCpuUsedPercentage() > 0L) {
- metricsResponse.setCpuUsed(metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
- metricsResponse.setCpuMaxDeviation(metrics.getMaximumCpuUsage(), metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
+ metricsResponse.setResources(hostMetrics.getUpResources(), hostMetrics.getTotalResources());
+ // add CPU and Memory metrics
+ addHostCpuMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+ addHostMemoryMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+
+ metricsResponse.setHasAnnotation(clusterResponse.hasAnnotation());
+ metricsResponses.add(metricsResponse);
+ }
+ return metricsResponses;
+ }
+
+ private void addHostMemoryMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setMemTotal(hostMetrics.getTotalMemory());
+ metricsResponse.setMemAllocated(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory());
+ if (hostMetrics.getMemoryUsed() > 0L) {
+ metricsResponse.setMemUsed(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory());
+ metricsResponse.setMemMaxDeviation(hostMetrics.getMaximumMemoryUsage(), hostMetrics.getMemoryUsed(), hostMetrics.getTotalHosts());
+ }
+ // Memory thresholds
+ final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
+ final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
+ metricsResponse.setMemoryUsageThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryUsageDisableThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryAllocatedThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryAllocatedDisableThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ }
+
+ private void addHostCpuMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setCpuTotal(hostMetrics.getTotalCpu());
+ metricsResponse.setCpuAllocated(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu());
+ if (hostMetrics.getCpuUsedPercentage() > 0L) {
+ metricsResponse.setCpuUsed(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ metricsResponse.setCpuMaxDeviation(hostMetrics.getMaximumCpuUsage(), hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ }
+ // CPU thresholds
+ final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
+ final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
+ metricsResponse.setCpuUsageThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuThreshold);
+ metricsResponse.setCpuUsageDisableThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuDisableThreshold);
+ metricsResponse.setCpuAllocatedThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuThreshold);
+ metricsResponse.setCpuAllocatedDisableThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuDisableThreshold);
+ }
+
+
+ @Override
+ public List<ManagementServerMetricsResponse> listManagementServerMetrics(List<ManagementServerResponse> managementServerResponses) {
+ final List<ManagementServerMetricsResponse> metricsResponses = new ArrayList<>();
+ if(LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("getting metrics for %d MS hosts", managementServerResponses.size()));
+ }
+ for (final ManagementServerResponse managementServerResponse: managementServerResponses) {
+ if(LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("processing metrics for MS hosts %s", managementServerResponse.getId()));
}
- // Memory
- metricsResponse.setMemTotal(metrics.getTotalMemory());
- metricsResponse.setMemAllocated(metrics.getMemoryAllocated(), metrics.getTotalMemory());
- if (metrics.getMemoryUsed() > 0L) {
- metricsResponse.setMemUsed(metrics.getMemoryUsed(), metrics.getTotalMemory());
- metricsResponse.setMemMaxDeviation(metrics.getMaximumMemoryUsage(), metrics.getMemoryUsed(), metrics.getTotalHosts());
+ ManagementServerMetricsResponse metricsResponse = new ManagementServerMetricsResponse();
+
+ try {
+ BeanUtils.copyProperties(metricsResponse, managementServerResponse);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("bean copy result %s", new ReflectionToStringBuilder(metricsResponse, ToStringStyle.SIMPLE_STYLE).toString()));
+ }
+ } catch (IllegalAccessException | InvocationTargetException e) {
+ throw new ServerApiException(ApiErrorCode.INTERNAL_ERROR, "Failed to generate zone metrics response");
}
- // CPU thresholds
- metricsResponse.setCpuUsageThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuThreshold);
- metricsResponse.setCpuUsageDisableThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuDisableThreshold);
- metricsResponse.setCpuAllocatedThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuThreshold);
- metricsResponse.setCpuAllocatedDisableThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuDisableThreshold);
- // Memory thresholds
- metricsResponse.setMemoryUsageThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryUsageDisableThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setMemoryAllocatedThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryAllocatedDisableThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setHasAnnotation(clusterResponse.hasAnnotation());
+ updateManagementServerMetrics(metricsResponse, managementServerResponse);
+
metricsResponses.add(metricsResponse);
}
return metricsResponses;
}
+ /**
+ * get the transient/in memory data
+ * @param metricsResponse
+ * @param managementServerResponse
+ */
+ private void updateManagementServerMetrics(ManagementServerMetricsResponse metricsResponse, ManagementServerResponse managementServerResponse) {
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("getting stats for %s", managementServerResponse.getId()));
Review comment:
```suggestion
LOGGER.debug(String.format("Ggetting stats for MS %s.", managementServerResponse.getId()));
```
Is it possible to log more than only the id?
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -417,73 +527,186 @@ public InfrastructureResponse listInfrastructure() {
// CPU and memory capacities
final CapacityDaoImpl.SummedCapacity cpuCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_CPU, zoneId, null);
final CapacityDaoImpl.SummedCapacity memoryCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_MEMORY, zoneId, null);
- final Metrics metrics = new Metrics(cpuCapacity, memoryCapacity);
+ final HostMetrics hostMetrics = new HostMetrics(cpuCapacity, memoryCapacity);
for (final Cluster cluster : clusterDao.listClustersByDcId(zoneId)) {
if (cluster == null) {
continue;
}
- metrics.incrTotalResources();
+ hostMetrics.incrTotalResources();
if (cluster.getAllocationState() == Grouping.AllocationState.Enabled
&& cluster.getManagedState() == Managed.ManagedState.Managed) {
- metrics.incrUpResources();
+ hostMetrics.incrUpResources();
}
for (final Host host: hostDao.findByClusterId(cluster.getId())) {
if (host == null || host.getType() != Host.Type.Routing) {
continue;
}
- updateHostMetrics(metrics, hostJoinDao.findById(host.getId()));
+ updateHostMetrics(hostMetrics, hostJoinDao.findById(host.getId()));
}
}
metricsResponse.setHasAnnotation(zoneResponse.hasAnnotation());
metricsResponse.setState(zoneResponse.getAllocationState());
- metricsResponse.setResource(metrics.getUpResources(), metrics.getTotalResources());
+ metricsResponse.setResource(hostMetrics.getUpResources(), hostMetrics.getTotalResources());
// CPU
- metricsResponse.setCpuTotal(metrics.getTotalCpu());
- metricsResponse.setCpuAllocated(metrics.getCpuAllocated(), metrics.getTotalCpu());
- if (metrics.getCpuUsedPercentage() > 0L) {
- metricsResponse.setCpuUsed(metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
- metricsResponse.setCpuMaxDeviation(metrics.getMaximumCpuUsage(), metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
+ metricsResponse.setCpuTotal(hostMetrics.getTotalCpu());
+ metricsResponse.setCpuAllocated(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu());
+ if (hostMetrics.getCpuUsedPercentage() > 0L) {
+ metricsResponse.setCpuUsed(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ metricsResponse.setCpuMaxDeviation(hostMetrics.getMaximumCpuUsage(), hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
}
// Memory
- metricsResponse.setMemTotal(metrics.getTotalMemory());
- metricsResponse.setMemAllocated(metrics.getMemoryAllocated(), metrics.getTotalMemory());
- if (metrics.getMemoryUsed() > 0L) {
- metricsResponse.setMemUsed(metrics.getMemoryUsed(), metrics.getTotalMemory());
- metricsResponse.setMemMaxDeviation(metrics.getMaximumMemoryUsage(), metrics.getMemoryUsed(), metrics.getTotalHosts());
+ metricsResponse.setMemTotal(hostMetrics.getTotalMemory());
+ metricsResponse.setMemAllocated(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory());
+ if (hostMetrics.getMemoryUsed() > 0L) {
+ metricsResponse.setMemUsed(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory());
+ metricsResponse.setMemMaxDeviation(hostMetrics.getMaximumMemoryUsage(), hostMetrics.getMemoryUsed(), hostMetrics.getTotalHosts());
}
// CPU thresholds
- metricsResponse.setCpuUsageThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuThreshold);
- metricsResponse.setCpuUsageDisableThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuDisableThreshold);
- metricsResponse.setCpuAllocatedThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuThreshold);
- metricsResponse.setCpuAllocatedDisableThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuDisableThreshold);
+ metricsResponse.setCpuUsageThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuThreshold);
+ metricsResponse.setCpuUsageDisableThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuDisableThreshold);
+ metricsResponse.setCpuAllocatedThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuThreshold);
+ metricsResponse.setCpuAllocatedDisableThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuDisableThreshold);
// Memory thresholds
- metricsResponse.setMemoryUsageThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryUsageDisableThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setMemoryAllocatedThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryAllocatedDisableThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryUsageThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryUsageDisableThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryAllocatedThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryAllocatedDisableThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
Review comment:
These `hostMetrics.get...` are called several times in this method, we could extract them to variables.
##########
File path: server/src/main/java/com/cloud/api/query/vo/ManagementServerJoinVO.java
##########
@@ -0,0 +1,173 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.api.query.vo;
+
+import com.cloud.utils.db.GenericDao;
+import org.apache.cloudstack.management.ManagementServerHost;
+import org.apache.cloudstack.management.ManagementServerHost.State;
+
+import javax.persistence.Column;
+import javax.persistence.Entity;
+import javax.persistence.EnumType;
+import javax.persistence.Enumerated;
+import javax.persistence.GeneratedValue;
+import javax.persistence.GenerationType;
+import javax.persistence.Id;
+import javax.persistence.Table;
+import javax.persistence.Temporal;
+import javax.persistence.TemporalType;
+import java.util.Date;
+
+/**
+ * api response utility join view
Review comment:
Could we improve this javadoc? Don't sounds too intuitive to me.
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -207,22 +259,34 @@ public String toString() {
private static final String INFLUXDB_HOST_MEASUREMENT = "host_stats";
private static final String INFLUXDB_VM_MEASUREMENT = "vm_stats";
- private static final ConfigKey<Integer> vmDiskStatsInterval = new ConfigKey<Integer>("Advanced", Integer.class, "vm.disk.stats.interval", "0",
+ private static final ConfigKey<Integer> MANAGEMENT_SERVER_STATUS_COLLECTION_INTERVAL = new ConfigKey<>("Advanced",
+ Integer.class, "management.server.stats.interval", "60",
+ "time interval for management servers stats. Set to <= 0 to disable management servers stats", false);
Review comment:
```suggestion
"Time interval for management servers stats. Set to <= 0 to disable management servers stats.", false);
```
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -207,22 +259,34 @@ public String toString() {
private static final String INFLUXDB_HOST_MEASUREMENT = "host_stats";
private static final String INFLUXDB_VM_MEASUREMENT = "vm_stats";
- private static final ConfigKey<Integer> vmDiskStatsInterval = new ConfigKey<Integer>("Advanced", Integer.class, "vm.disk.stats.interval", "0",
+ private static final ConfigKey<Integer> MANAGEMENT_SERVER_STATUS_COLLECTION_INTERVAL = new ConfigKey<>("Advanced",
+ Integer.class, "management.server.stats.interval", "60",
+ "time interval for management servers stats. Set to <= 0 to disable management servers stats", false);
+ private static final ConfigKey<Integer> USAGE_SERVER_STATUS_COLLECTION_INTERVAL = new ConfigKey<>("Advanced",
+ Integer.class, "usage.server.stats.interval", "60",
+ "time interval for usage servers stats. Set to <= 0 to disable usage servers stats", false);
Review comment:
```suggestion
"Time interval for usage servers stats. Set to <= 0 to disable usage servers stats.", false);
```
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -417,73 +527,186 @@ public InfrastructureResponse listInfrastructure() {
// CPU and memory capacities
final CapacityDaoImpl.SummedCapacity cpuCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_CPU, zoneId, null);
final CapacityDaoImpl.SummedCapacity memoryCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_MEMORY, zoneId, null);
- final Metrics metrics = new Metrics(cpuCapacity, memoryCapacity);
+ final HostMetrics hostMetrics = new HostMetrics(cpuCapacity, memoryCapacity);
for (final Cluster cluster : clusterDao.listClustersByDcId(zoneId)) {
if (cluster == null) {
continue;
}
- metrics.incrTotalResources();
+ hostMetrics.incrTotalResources();
if (cluster.getAllocationState() == Grouping.AllocationState.Enabled
&& cluster.getManagedState() == Managed.ManagedState.Managed) {
- metrics.incrUpResources();
+ hostMetrics.incrUpResources();
}
for (final Host host: hostDao.findByClusterId(cluster.getId())) {
if (host == null || host.getType() != Host.Type.Routing) {
continue;
}
- updateHostMetrics(metrics, hostJoinDao.findById(host.getId()));
+ updateHostMetrics(hostMetrics, hostJoinDao.findById(host.getId()));
}
}
metricsResponse.setHasAnnotation(zoneResponse.hasAnnotation());
metricsResponse.setState(zoneResponse.getAllocationState());
- metricsResponse.setResource(metrics.getUpResources(), metrics.getTotalResources());
+ metricsResponse.setResource(hostMetrics.getUpResources(), hostMetrics.getTotalResources());
// CPU
- metricsResponse.setCpuTotal(metrics.getTotalCpu());
- metricsResponse.setCpuAllocated(metrics.getCpuAllocated(), metrics.getTotalCpu());
- if (metrics.getCpuUsedPercentage() > 0L) {
- metricsResponse.setCpuUsed(metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
- metricsResponse.setCpuMaxDeviation(metrics.getMaximumCpuUsage(), metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
+ metricsResponse.setCpuTotal(hostMetrics.getTotalCpu());
+ metricsResponse.setCpuAllocated(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu());
+ if (hostMetrics.getCpuUsedPercentage() > 0L) {
+ metricsResponse.setCpuUsed(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ metricsResponse.setCpuMaxDeviation(hostMetrics.getMaximumCpuUsage(), hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
}
// Memory
- metricsResponse.setMemTotal(metrics.getTotalMemory());
- metricsResponse.setMemAllocated(metrics.getMemoryAllocated(), metrics.getTotalMemory());
- if (metrics.getMemoryUsed() > 0L) {
- metricsResponse.setMemUsed(metrics.getMemoryUsed(), metrics.getTotalMemory());
- metricsResponse.setMemMaxDeviation(metrics.getMaximumMemoryUsage(), metrics.getMemoryUsed(), metrics.getTotalHosts());
+ metricsResponse.setMemTotal(hostMetrics.getTotalMemory());
+ metricsResponse.setMemAllocated(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory());
+ if (hostMetrics.getMemoryUsed() > 0L) {
+ metricsResponse.setMemUsed(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory());
+ metricsResponse.setMemMaxDeviation(hostMetrics.getMaximumMemoryUsage(), hostMetrics.getMemoryUsed(), hostMetrics.getTotalHosts());
}
// CPU thresholds
- metricsResponse.setCpuUsageThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuThreshold);
- metricsResponse.setCpuUsageDisableThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuDisableThreshold);
- metricsResponse.setCpuAllocatedThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuThreshold);
- metricsResponse.setCpuAllocatedDisableThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuDisableThreshold);
+ metricsResponse.setCpuUsageThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuThreshold);
+ metricsResponse.setCpuUsageDisableThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuDisableThreshold);
+ metricsResponse.setCpuAllocatedThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuThreshold);
+ metricsResponse.setCpuAllocatedDisableThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuDisableThreshold);
// Memory thresholds
- metricsResponse.setMemoryUsageThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryUsageDisableThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setMemoryAllocatedThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryAllocatedDisableThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryUsageThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryUsageDisableThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryAllocatedThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryAllocatedDisableThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
metricsResponses.add(metricsResponse);
}
return metricsResponses;
}
+ @Override
+ public UsageServerMetricsResponse listUsageServerMetrics() {
+ UsageServerMetricsResponse response = new UsageServerMetricsResponse();
+ response.setCollectionTime(new Date());
+ TransactionLegacy txn = TransactionLegacy.open(TransactionLegacy.USAGE_DB);
+ try {
+ response.setLastHeartbeat(usageJobDao.getLastHeartbeat());
+ response.setState(isUsageRunning()? State.Up : State.Down);
+ UsageJobVO job = usageJobDao.getNextImmediateJob();
+ if (job == null) {
+ job = usageJobDao.getLastJob();
+ }
+ response.setHostname(job == null? "N/A": job.getHost());
+ response.setLastSuccessfulJob(new Date(usageJobDao.getLastJobSuccessDateMillis()));
+ } finally {
+ txn.close();
+ TransactionLegacy swap = TransactionLegacy.open(TransactionLegacy.CLOUD_DB);
+ swap.close();
+ }
+ response.setObjectName("usageMetrics");
+ return response;
+ }
+
+ /**
+ TODO
+ â—¦ State (Up / Down) , makes no sense (no db no cloudstack)
+ â—¦ Average Queries Per Second
+ â—¦ Buffer Pool Utilization (buffer pool is used to cache the table data in memory and is accessed repeatedly by queries without requiring any disk I/O).
+ â—¦ any other relevant stats (if useful) to the response from the sql status variables.
+ * @return
+ */
+ @Override
+ public DbMetricsResponse listDbMetrics() {
+ DbMetricsResponse response = new DbMetricsResponse();
+
+ response.setCollectionTime(new Date());
+ response.setHostname(dbHostName());
+ response.setReplicas(dbReplicas());
+ getDynamicDataFromDB(response);
+ getStaticDataFromDB(response);
+
+ getQueryHistory(response);
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(new ReflectionToStringBuilder(response));
+ }
+
+ response.setObjectName("dbMetrics");
+ return response;
+ }
+
+ private void getQueryHistory(DbMetricsResponse response) {
+ Map<String, Object> dbStats = ApiDBUtils.getDbStatistics();
+ if (dbStats != null) {
+ response.setQueries((Integer)dbStats.get(DbStatsCollection.queries));
+ response.setUptime((Integer)dbStats.get(DbStatsCollection.uptime));
+ }
+
+ List<Double> loadHistory = (List<Double>) dbStats.get(DbStatsCollection.loadAvarages);
+ double[] loadAverages = new double[loadHistory.size()];
+
+ int index =0;
+ for (Double d : loadHistory) {
+ loadAverages[index++] = d;
+ }
+
+ response.setLoadAverages(loadAverages);
+ }
+
+ private void getStaticDataFromDB(DbMetricsResponse response) {
+ Map<String, String> vars = DbUtil.getDbInfo(DbStatsCollection.variables, DbStatsCollection.version, DbStatsCollection.versionComment);
+ response.setVersion(vars.get(DbStatsCollection.version));
+ response.setVersionComment(vars.get(DbStatsCollection.versionComment));
+ }
+
+ private void getDynamicDataFromDB(DbMetricsResponse response) {
+ Map<String, String> stats = DbUtil.getDbInfo(DbStatsCollection.status, DbStatsCollection.connections, DbStatsCollection.currentTlsVersion);
+ response.setConnections(Integer.parseInt(stats.get(DbStatsCollection.connections)));
+ response.setTlsVersions(stats.get(DbStatsCollection.currentTlsVersion));
+ }
+
+ private String dbHostName() {
+ Properties p = DbProperties.getDbProperties();
+ return p.getProperty("db.cloud.host");
+ }
+
+ private String[] dbReplicas() {
+ Properties p = DbProperties.getDbProperties();
+ return p.getProperty("db.cloud.replicas","").split(",");
+ }
+
+ /**
+ * returns whether a local usage server is running.
+ * Note that this might not be the one actually doing the usage aggregation at this moment.
+ * @return true if the service is active
Review comment:
```suggestion
* @return true if the service is active.
```
##########
File path: engine/schema/src/main/java/com/cloud/host/dao/HostDao.java
##########
@@ -142,4 +142,11 @@
HostVO findByName(String name);
List<HostVO> listHostsWithActiveVMs(long offeringId);
+
+ /**
+ * get the number of hosts/agents this {@see ManagementServer} has responsibility over
+ * @param msid the id of the {@see ManagementServer}
+ * @return the number of hosts/agents this {@see ManagementServer} has responsibility over
+ */
+ int countByMs(long msid);
Review comment:
```suggestion
int countByMs(long msId);
```
##########
File path: engine/schema/src/main/java/com/cloud/host/dao/HostDaoImpl.java
##########
@@ -1287,6 +1292,13 @@ private String getHostIdsByComputeTags(List<String> offeringTags){
}
}
+ @Override
+ public int countByMs(long msid) {
+ SearchCriteria<HostVO> sc = ResponsibleMsCountSearch.create();
+ sc.setParameters("managementServerId", msid);
Review comment:
```suggestion
sc.setParameters("managementServerId", msId);
```
##########
File path: framework/cluster/src/main/java/com/cloud/cluster/ClusterManager.java
##########
@@ -21,27 +21,36 @@
import com.cloud.utils.component.Manager;
+/**
+ * the definition of the framework for inter MS communication
+ */
public interface ClusterManager extends Manager {
static final String ALERT_SUBJECT = "cluster-alert";
final ConfigKey<Integer> HeartbeatInterval = new ConfigKey<Integer>(Integer.class, "cluster.heartbeat.interval", "management-server", "1500",
"Interval to check for the heart beat between management server nodes", false);
final ConfigKey<Integer> HeartbeatThreshold = new ConfigKey<Integer>(Integer.class, "cluster.heartbeat.threshold", "management-server", "150000",
"Threshold before self-fence the management server", true);
+ /**
+ * adds a new packet to the incoming queue
+ * @param pdu protocol data unit
+ */
void OnReceiveClusterServicePdu(ClusterServicePdu pdu);
+ void publishStatus(String status);
+
/**
- * This executes
- * @param strPeer
- * @param agentId
- * @param cmds
- * @param stopOnError
- * @return
+ * This creates pdu, registers it, notifies listeners, and waits on the pdu to be notified
Review comment:
```suggestion
* Creates and registers a PDU, notifies listeners, and waits on the PDU to be notified.
```
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -207,22 +259,34 @@ public String toString() {
private static final String INFLUXDB_HOST_MEASUREMENT = "host_stats";
private static final String INFLUXDB_VM_MEASUREMENT = "vm_stats";
- private static final ConfigKey<Integer> vmDiskStatsInterval = new ConfigKey<Integer>("Advanced", Integer.class, "vm.disk.stats.interval", "0",
+ private static final ConfigKey<Integer> MANAGEMENT_SERVER_STATUS_COLLECTION_INTERVAL = new ConfigKey<>("Advanced",
+ Integer.class, "management.server.stats.interval", "60",
+ "time interval for management servers stats. Set to <= 0 to disable management servers stats", false);
+ private static final ConfigKey<Integer> USAGE_SERVER_STATUS_COLLECTION_INTERVAL = new ConfigKey<>("Advanced",
+ Integer.class, "usage.server.stats.interval", "60",
+ "time interval for usage servers stats. Set to <= 0 to disable usage servers stats", false);
+ private static final ConfigKey<Integer> DATABASE_SERVER_STATUS_COLLECTION_INTERVAL = new ConfigKey<>("Advanced",
+ Integer.class, "database.server.stats.interval", "60",
+ "time interval for database servers stats. Set to <= 0 to disable database servers stats", false);
Review comment:
```suggestion
"Time interval for database servers stats. Set to <= 0 to disable database servers stats.", false);
```
##########
File path: framework/db/src/main/java/com/cloud/utils/db/DbUtil.java
##########
@@ -282,4 +282,25 @@ public static void closeConnection(final Connection connection) {
closeAutoCloseable(connection, "exception while close connection.");
}
+ public static Map<String, String> getDbInfo(String type, String ... var) {
+ String vars = String.join(",", var);
+ Map<String, String> result = new HashMap<>();
+ String sql = String.format("SHOW %s WHERE FIND_IN_SET(Variable_name,?)",type);
+ final TransactionLegacy txn = TransactionLegacy.open("metrics");
+ try {
+ PreparedStatement pstmt = txn.prepareAutoCloseStatement(sql);
+ pstmt.setString(1, vars);
+ final ResultSet rs = pstmt.executeQuery();
+ while (rs.next()) {
+ String variableName = rs.getString("Variable_name");
+ String value = rs.getString("value");
+ result.put(variableName, value);
+ }
+ } catch (SQLException e) {
+ LOGGER.error("failed to get the database status: " + e.getLocalizedMessage());
+ LOGGER.debug("failed to get the database status", e);
Review comment:
Is two log needed? We could log only in error:
```java
LOGGER.error(String.format("Failed to get the database status due to: ", e.getLocalizedMessage()) e);
```
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -564,11 +681,485 @@ protected Point creteInfluxDbPoint(Object metricsObject) {
}
}
+ class DbCollector extends AbstractStatsCollector {
+ List<Double> loadHistory = new ArrayList<>();
+ DbCollector() {
+ dbStats.put(loadAvarages, loadHistory);
+ }
+ @Override
+ protected void runInContext() {
+ LOGGER.debug(String.format("%s is running...", this.getClass().getSimpleName()));
+
+ try {
+ int lastUptime = (dbStats.containsKey(uptime) ? (Integer) dbStats.get(uptime) : 0);
+ int lastQueries = (dbStats.containsKey(queries) ? (Integer) dbStats.get(queries) : 0);
+ getDynamicDataFromDB();
+ int interval = (Integer) dbStats.get(uptime) - lastUptime;
+ int activity = (Integer) dbStats.get(queries) - lastQueries;
+ loadHistory.add(0, Double.valueOf(activity / interval));
+ int maxsize = DATABASE_SERVER_LOAD_HISTORY_RETENTION_NUMBER.value();
+ while (loadHistory.size() > maxsize) {
+ loadHistory.remove(maxsize - 1);
+ }
+ } catch (Throwable e) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("db statistics collection failed due to " + e.getLocalizedMessage());
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("db statistics collection failed.", e);
+ }
+ }
+ }
+
+ private void getDynamicDataFromDB() {
+ Map<String, String> stats = DbUtil.getDbInfo("STATUS", queries, uptime);
+ dbStats.put(queries, (Integer.valueOf(stats.get(queries))));
+ dbStats.put(uptime, (Integer.valueOf(stats.get(uptime))));
+ }
+
+
+ @Override
+ protected Point creteInfluxDbPoint(Object metricsObject) {
+ return null;
+ }
+ }
+
+ Gson gson;
+
+ class ManagementServerCollector extends AbstractStatsCollector {
+ @Override
+ protected void runInContext() {
+ LOGGER.debug(String.format("%s is running...", this.getClass().getSimpleName()));
+ long msid = ManagementServerNode.getManagementServerId();
+ ManagementServerHostVO mshost = null;
+ ManagementServerHostStatsEntry hostStatsEntry = null;
+ try {
+ mshost = managementServerHostDao.findByMsid(msid);
+ // get local data
+ hostStatsEntry = getDataFrom(mshost);
+ managementServerHostStats.put(mshost.getUuid(), hostStatsEntry);
+ // send to other hosts
+ clusterManager.publishStatus(gson.toJson(hostStatsEntry));
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to retrieve management server host statistics", t);
+ }
+ try {
+ // send to DB
+ storeStatus(hostStatsEntry, mshost);
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to store management server host statistics", t);
+ }
+ }
+
+ private void storeStatus(ManagementServerHostStatsEntry hostStatsEntry, ManagementServerHostVO mshost) {
+ if (hostStatsEntry == null || mshost == null) {
+ return;
+ }
+ ManagementServerStatusVO msStats = managementServerStatusDao.findByMsId(hostStatsEntry.getManagementServerHostUuid());
+ if (msStats == null) {
+ LOGGER.info(String.format("creating new status info record for host %s - %s",
+ mshost.getName(),
+ hostStatsEntry.getManagementServerHostUuid()));
+ msStats = new ManagementServerStatusVO();
+ msStats.setMsId(hostStatsEntry.getManagementServerHostUuid());
+ }
+ msStats.setOsDistribution(hostStatsEntry.getOsDistribution()); // for now just the bunch details come later
+ msStats.setJavaName(hostStatsEntry.getJvmVendor());
+ msStats.setJavaVersion(hostStatsEntry.getJvmVersion());
+ Date startTime = new Date(hostStatsEntry.getJvmStartTime());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("reporting starttime %s", startTime));
+ }
+ msStats.setLastJvmStart(startTime);
+ msStats.setLastSystemBoot(hostStatsEntry.getSystemBootTime());
+ msStats.setUpdated(new Date());
+ managementServerStatusDao.persist(msStats);
+ }
+
+ @NotNull
+ private ManagementServerHostStatsEntry getDataFrom(ManagementServerHostVO mshost) {
+ ManagementServerHostStatsEntry newEntry = new ManagementServerHostStatsEntry();
+ LOGGER.trace("Metrics collection start...");
+ newEntry.setManagementServerHostId(mshost.getId());
+ newEntry.setManagementServerHostUuid(mshost.getUuid());
+ newEntry.setDbLocal(isDbLocal());
+ newEntry.setUsageLocal(isUsageLocal());
+ retrieveSession(newEntry);
+ getJvmDimensions(newEntry);
+ LOGGER.trace("Metrics collection extra...");
+ getRuntimeData(newEntry);
+ getCpuData(newEntry);
+ getMemoryData(newEntry);
+ // newEntry must now include a pid!
+ getProcFileSystemData(newEntry);
+ getFileSystemData(newEntry);
+ getDataBaseStatistics(newEntry, mshost.getMsid());
+ gatherAllMetrics(newEntry);
+ LOGGER.trace("Metrics collection end!");
+ return newEntry;
+ }
+
+ private void retrieveSession(ManagementServerHostStatsEntry newEntry) {
+ long sessions = ApiSessionListener.getSessionCount();
+ newEntry.setSessions(sessions);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("Sessions found in Api %d vs context %d", sessions,ApiSessionListener.getNumberOfSessions()));
+ } else {
+ LOGGER.debug("Sessions active: " + sessions);
+ }
+ }
+
+ private void getDataBaseStatistics(ManagementServerHostStatsEntry newEntry, long msid) {
+ int count = _hostDao.countByMs(msid);
+ newEntry.setAgentCount(count);
+ }
+
+ private void getMemoryData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ MemoryMXBean mxBean = ManagementFactory.getMemoryMXBean();
+ newEntry.setTotalInit(mxBean.getHeapMemoryUsage().getInit() + mxBean.getNonHeapMemoryUsage().getInit());
+ newEntry.setTotalUsed(mxBean.getHeapMemoryUsage().getUsed() + mxBean.getNonHeapMemoryUsage().getUsed());
+ newEntry.setTotalMax(mxBean.getHeapMemoryUsage().getMax() + mxBean.getNonHeapMemoryUsage().getMax());
+ newEntry.setTotalCommitted(mxBean.getHeapMemoryUsage().getCommitted() + mxBean.getNonHeapMemoryUsage().getCommitted());
+ }
+
+ private void getCpuData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ java.lang.management.OperatingSystemMXBean bean = ManagementFactory.getOperatingSystemMXBean();
+ newEntry.setAvailableProcessors(bean.getAvailableProcessors());
+ newEntry.setLoadAverage(bean.getSystemLoadAverage());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics processors - %d , loadavg - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getLoadAverage()));
+ }
+ if (bean instanceof OperatingSystemMXBean) {
+ OperatingSystemMXBean mxBean = (OperatingSystemMXBean) bean;
+ newEntry.setSystemMemoryTotal(mxBean.getTotalPhysicalMemorySize());
+ newEntry.setSystemMemoryFree(mxBean.getFreePhysicalMemorySize());
+ newEntry.setSystemMemoryUsed(mxBean.getCommittedVirtualMemorySize());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("data from 'OperatingSystemMXBean': total mem: %d, free mem: %d, used mem: %d",
+ newEntry.getSystemMemoryTotal(),
+ newEntry.getSystemMemoryFree(),
+ newEntry.getSystemMemoryUsed()));
+ }
+ }
+ }
+
+ private void getRuntimeData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final RuntimeMXBean mxBean = ManagementFactory.getRuntimeMXBean();
+ newEntry.setJvmUptime(mxBean.getUptime());
+ newEntry.setJvmStartTime(mxBean.getStartTime());
+ newEntry.setProcessId(mxBean.getPid());
+ newEntry.setJvmName(mxBean.getName());
+ newEntry.setJvmVendor(mxBean.getVmVendor());
+ newEntry.setJvmVersion(mxBean.getVmVersion());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics uptime - %d , starttime - %d",
+ newEntry.getJvmUptime(),
+ newEntry.getJvmStartTime()));
+ }
+ }
+
+ private void getJvmDimensions(@NotNull ManagementServerHostStatsEntry newEntry) {
+ Runtime runtime = Runtime.getRuntime();
+ newEntry.setTotalMemoryBytes(runtime.totalMemory());
+ newEntry.setFreeMemoryBytes(runtime.freeMemory());
+ newEntry.setAvailableProcessors(runtime.availableProcessors());
+ newEntry.setTotalMax(runtime.maxMemory());
+ //long maxMem = runtime.maxMemory();
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics proc - %d , maxMem - %d , totalMemory - %d , freeMemory - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getTotalMax(),
+ newEntry.getTotalMemoryBytes(),
+ newEntry.getFreeMemoryBytes()));
+ }
+ }
+
+ /**
+ * As for data from outside the JVM, we only rely on /proc/ contained data.
+ *
+ * @param newEntry item to add the information to
+ */
+ private void getProcFileSystemData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ // this should be taken from ("cat /proc/version"), not sure how standard this /etc entry is
+ String OS = Script.runSimpleBashScript("cat /etc/os-release | grep PRETTY_NAME | cut -f2 -d '=' | tr -d '\"'");
+ newEntry.setOsDistribution(OS);
+ String kernel = Script.runSimpleBashScript("uname -r");
+ newEntry.setKernelVersion(kernel);
+ // if we got these from the bean, skip
+ if (newEntry.getSystemMemoryTotal() == 0) {
+ String mem = Script.runSimpleBashScript("cat /proc/meminfo | grep MemTotal | cut -f 2 -d ':' | tr -d 'a-zA-z '").trim();
+ newEntry.setSystemMemoryTotal(Long.parseLong(mem) * 1024);
Review comment:
We could use `ByteScaleUtils` here.
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -378,14 +475,34 @@ protected void init(Map<String, String> configs) {
}
} catch (URISyntaxException e) {
- s_logger.error("Failed to parse external statistics URI: ", e);
+ LOGGER.error("Failed to parse external statistics URI: ", e);
}
}
if (hostStatsInterval > 0) {
_executor.scheduleWithFixedDelay(new HostCollector(), 15000L, hostStatsInterval, TimeUnit.MILLISECONDS);
}
+ if (MANAGEMENT_SERVER_STATUS_COLLECTION_INTERVAL.value() > 0) {
+ _executor.scheduleAtFixedRate(new ManagementServerCollector(),
+ 1L,
+ MANAGEMENT_SERVER_STATUS_COLLECTION_INTERVAL.value(),
+ TimeUnit.SECONDS);
+ } else {
+ LOGGER.debug(String.format("%s - %d is 0 or less, so not scheduling the management host status collector thread",
+ MANAGEMENT_SERVER_STATUS_COLLECTION_INTERVAL.key(), MANAGEMENT_SERVER_STATUS_COLLECTION_INTERVAL.value()));
+ }
Review comment:
We could extract it to a method.
##########
File path: framework/cluster/src/main/java/com/cloud/cluster/ClusterManagerImpl.java
##########
@@ -385,6 +401,35 @@ public void executeAsync(final String strPeer, final long agentId, final String
addOutgoingClusterPdu(pdu);
}
+ @Override
+ public void publishStatus(final String status) {
+ final Date cutTime = DateUtil.currentGMTTime();
+
+ final List<ManagementServerHostVO> peers = _mshostDao.getActiveList(new Date(cutTime.getTime() - HeartbeatThreshold.value()));
+ for (final ManagementServerHostVO peer : peers) {
+ final String peerName = Long.toString(peer.getMsid());
+ try {
+ if (s_logger.isDebugEnabled()) {
+ s_logger.debug("Forwarding " + status + " to " + peer.getMsid());
+ }
+ sendStatus(peerName, status);
+ } catch (final Exception e) {
+ s_logger.warn("Caught exception while talking to " + peer.getMsid());
Review comment:
We can pass the exception as parameter of `s_logger.warn`.
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -335,61 +362,144 @@ public InfrastructureResponse listInfrastructure() {
}
final Long clusterId = cluster.getId();
- // Thresholds
- final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
- final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
- final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
- final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
-
// CPU and memory capacities
final CapacityDaoImpl.SummedCapacity cpuCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_CPU, null, clusterId);
final CapacityDaoImpl.SummedCapacity memoryCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_MEMORY, null, clusterId);
- final Metrics metrics = new Metrics(cpuCapacity, memoryCapacity);
+ final HostMetrics hostMetrics = new HostMetrics(cpuCapacity, memoryCapacity);
for (final Host host: hostDao.findByClusterId(clusterId)) {
if (host == null || host.getType() != Host.Type.Routing) {
continue;
}
if (host.getStatus() == Status.Up) {
- metrics.incrUpResources();
+ hostMetrics.incrUpResources();
}
- metrics.incrTotalResources();
- updateHostMetrics(metrics, hostJoinDao.findById(host.getId()));
+ hostMetrics.incrTotalResources();
+ updateHostMetrics(hostMetrics, hostJoinDao.findById(host.getId()));
}
metricsResponse.setState(clusterResponse.getAllocationState(), clusterResponse.getManagedState());
- metricsResponse.setResources(metrics.getUpResources(), metrics.getTotalResources());
- // CPU
- metricsResponse.setCpuTotal(metrics.getTotalCpu());
- metricsResponse.setCpuAllocated(metrics.getCpuAllocated(), metrics.getTotalCpu());
- if (metrics.getCpuUsedPercentage() > 0L) {
- metricsResponse.setCpuUsed(metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
- metricsResponse.setCpuMaxDeviation(metrics.getMaximumCpuUsage(), metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
+ metricsResponse.setResources(hostMetrics.getUpResources(), hostMetrics.getTotalResources());
+ // add CPU and Memory metrics
Review comment:
Is this comment needed?
##########
File path: framework/cluster/src/main/java/com/cloud/cluster/ClusterManager.java
##########
@@ -53,19 +62,44 @@
void registerDispatcher(Dispatcher dispatcher);
+ /**
+ * register a listener for incoming status changes of ManagementServers
+ *
+ * @param administrator the object administrating statusses
+ */
+ void registerStatusAdministrator(StatusAdministrator administrator);
+
ManagementServerHost getPeer(String peerName);
+ /**
+ *
+ * @return {code}Long.toString({code}{@see getManagementNodeId()}{code}){code}
+ */
String getSelfPeerName();
long getManagementNodeId();
+ /**
+ * determined by the time
+ * @return start time as {code}System.currentTimeMillis(){code}
+ */
long getCurrentRunId();
- public long getManagementRunId(long msId);
+ /**
+ * the other guy's is derived from start time as stored in the db
+ */
+ long getManagementRunId(long msId);
- public interface Dispatcher {
+ interface Dispatcher {
String getName();
String dispatch(ClusterServicePdu pdu);
}
+
+ /**
+ * what the client of {@see registerStatusAdministrator()} should implement
+ */
+ interface StatusAdministrator {
+ String newStatus(ClusterServicePdu pdu);
+ }
Review comment:
@DaanHoogland, these javadocs are already ready?
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -335,61 +362,144 @@ public InfrastructureResponse listInfrastructure() {
}
final Long clusterId = cluster.getId();
- // Thresholds
- final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
- final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
- final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
- final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
-
// CPU and memory capacities
final CapacityDaoImpl.SummedCapacity cpuCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_CPU, null, clusterId);
final CapacityDaoImpl.SummedCapacity memoryCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_MEMORY, null, clusterId);
- final Metrics metrics = new Metrics(cpuCapacity, memoryCapacity);
+ final HostMetrics hostMetrics = new HostMetrics(cpuCapacity, memoryCapacity);
for (final Host host: hostDao.findByClusterId(clusterId)) {
if (host == null || host.getType() != Host.Type.Routing) {
continue;
}
if (host.getStatus() == Status.Up) {
- metrics.incrUpResources();
+ hostMetrics.incrUpResources();
}
- metrics.incrTotalResources();
- updateHostMetrics(metrics, hostJoinDao.findById(host.getId()));
+ hostMetrics.incrTotalResources();
+ updateHostMetrics(hostMetrics, hostJoinDao.findById(host.getId()));
}
metricsResponse.setState(clusterResponse.getAllocationState(), clusterResponse.getManagedState());
- metricsResponse.setResources(metrics.getUpResources(), metrics.getTotalResources());
- // CPU
- metricsResponse.setCpuTotal(metrics.getTotalCpu());
- metricsResponse.setCpuAllocated(metrics.getCpuAllocated(), metrics.getTotalCpu());
- if (metrics.getCpuUsedPercentage() > 0L) {
- metricsResponse.setCpuUsed(metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
- metricsResponse.setCpuMaxDeviation(metrics.getMaximumCpuUsage(), metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
+ metricsResponse.setResources(hostMetrics.getUpResources(), hostMetrics.getTotalResources());
+ // add CPU and Memory metrics
+ addHostCpuMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+ addHostMemoryMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+
+ metricsResponse.setHasAnnotation(clusterResponse.hasAnnotation());
+ metricsResponses.add(metricsResponse);
+ }
+ return metricsResponses;
+ }
+
+ private void addHostMemoryMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setMemTotal(hostMetrics.getTotalMemory());
+ metricsResponse.setMemAllocated(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory());
+ if (hostMetrics.getMemoryUsed() > 0L) {
+ metricsResponse.setMemUsed(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory());
+ metricsResponse.setMemMaxDeviation(hostMetrics.getMaximumMemoryUsage(), hostMetrics.getMemoryUsed(), hostMetrics.getTotalHosts());
+ }
+ // Memory thresholds
+ final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
+ final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
+ metricsResponse.setMemoryUsageThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryUsageDisableThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryAllocatedThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryAllocatedDisableThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ }
+
+ private void addHostCpuMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setCpuTotal(hostMetrics.getTotalCpu());
Review comment:
These `hostMetrics.get...` are called several times in this method, we could extract them to variables.
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/api/ListUsageServerMetricsCmd.java
##########
@@ -0,0 +1,57 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.cloudstack.api;
+
+import com.cloud.exception.ConcurrentOperationException;
+import com.cloud.exception.InsufficientCapacityException;
+import com.cloud.exception.NetworkRuleConflictException;
+import com.cloud.exception.ResourceAllocationException;
+import com.cloud.exception.ResourceUnavailableException;
+import org.apache.cloudstack.acl.RoleType;
+import org.apache.cloudstack.context.CallContext;
+import org.apache.cloudstack.metrics.MetricsService;
+import org.apache.cloudstack.response.UsageServerMetricsResponse;
+
+import javax.inject.Inject;
+
+@APICommand(name = ListUsageServerMetricsCmd.APINAME, description = "Lists Usage Server metrics", responseObject = UsageServerMetricsResponse.class,
+ requestHasSensitiveInfo = false, responseHasSensitiveInfo = false, responseView = ResponseObject.ResponseView.Full,
+ since = "4.17.0", authorized = {RoleType.Admin})
+public class ListUsageServerMetricsCmd extends BaseCmd {
+ public static final String APINAME = "listUsageServerMetrics";
+
+ @Inject
+ private MetricsService metricsService;
+
+ @Override
+ public void execute() throws ResourceUnavailableException, InsufficientCapacityException, ServerApiException, ConcurrentOperationException, ResourceAllocationException, NetworkRuleConflictException {
+ UsageServerMetricsResponse usageServerMetrics = metricsService.listUsageServerMetrics();
+ usageServerMetrics.setResponseName(getCommandName());
+ setResponseObject(usageServerMetrics);
+ }
+
+ @Override
+ public String getCommandName() {
+ return APINAME.toLowerCase() + BaseCmd.RESPONSE_SUFFIX;
+ }
+
+ @Override
+ public long getEntityOwnerId() {
+ return CallContext.current().getCallingAccountId();
Review comment:
This logic could be a method of `BaseCmd` too.
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -335,61 +362,144 @@ public InfrastructureResponse listInfrastructure() {
}
final Long clusterId = cluster.getId();
- // Thresholds
- final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
- final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
- final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
- final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
-
// CPU and memory capacities
final CapacityDaoImpl.SummedCapacity cpuCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_CPU, null, clusterId);
final CapacityDaoImpl.SummedCapacity memoryCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_MEMORY, null, clusterId);
- final Metrics metrics = new Metrics(cpuCapacity, memoryCapacity);
+ final HostMetrics hostMetrics = new HostMetrics(cpuCapacity, memoryCapacity);
for (final Host host: hostDao.findByClusterId(clusterId)) {
if (host == null || host.getType() != Host.Type.Routing) {
continue;
}
if (host.getStatus() == Status.Up) {
- metrics.incrUpResources();
+ hostMetrics.incrUpResources();
}
- metrics.incrTotalResources();
- updateHostMetrics(metrics, hostJoinDao.findById(host.getId()));
+ hostMetrics.incrTotalResources();
+ updateHostMetrics(hostMetrics, hostJoinDao.findById(host.getId()));
}
metricsResponse.setState(clusterResponse.getAllocationState(), clusterResponse.getManagedState());
- metricsResponse.setResources(metrics.getUpResources(), metrics.getTotalResources());
- // CPU
- metricsResponse.setCpuTotal(metrics.getTotalCpu());
- metricsResponse.setCpuAllocated(metrics.getCpuAllocated(), metrics.getTotalCpu());
- if (metrics.getCpuUsedPercentage() > 0L) {
- metricsResponse.setCpuUsed(metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
- metricsResponse.setCpuMaxDeviation(metrics.getMaximumCpuUsage(), metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
+ metricsResponse.setResources(hostMetrics.getUpResources(), hostMetrics.getTotalResources());
+ // add CPU and Memory metrics
+ addHostCpuMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+ addHostMemoryMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+
+ metricsResponse.setHasAnnotation(clusterResponse.hasAnnotation());
+ metricsResponses.add(metricsResponse);
+ }
+ return metricsResponses;
+ }
+
+ private void addHostMemoryMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setMemTotal(hostMetrics.getTotalMemory());
Review comment:
These `hostMetrics.get...` are called several times in this method, we could extract them to variables.
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -335,61 +362,144 @@ public InfrastructureResponse listInfrastructure() {
}
final Long clusterId = cluster.getId();
- // Thresholds
- final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
- final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
- final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
- final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
-
// CPU and memory capacities
final CapacityDaoImpl.SummedCapacity cpuCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_CPU, null, clusterId);
final CapacityDaoImpl.SummedCapacity memoryCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_MEMORY, null, clusterId);
- final Metrics metrics = new Metrics(cpuCapacity, memoryCapacity);
+ final HostMetrics hostMetrics = new HostMetrics(cpuCapacity, memoryCapacity);
for (final Host host: hostDao.findByClusterId(clusterId)) {
if (host == null || host.getType() != Host.Type.Routing) {
continue;
}
if (host.getStatus() == Status.Up) {
- metrics.incrUpResources();
+ hostMetrics.incrUpResources();
}
- metrics.incrTotalResources();
- updateHostMetrics(metrics, hostJoinDao.findById(host.getId()));
+ hostMetrics.incrTotalResources();
+ updateHostMetrics(hostMetrics, hostJoinDao.findById(host.getId()));
}
metricsResponse.setState(clusterResponse.getAllocationState(), clusterResponse.getManagedState());
- metricsResponse.setResources(metrics.getUpResources(), metrics.getTotalResources());
- // CPU
- metricsResponse.setCpuTotal(metrics.getTotalCpu());
- metricsResponse.setCpuAllocated(metrics.getCpuAllocated(), metrics.getTotalCpu());
- if (metrics.getCpuUsedPercentage() > 0L) {
- metricsResponse.setCpuUsed(metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
- metricsResponse.setCpuMaxDeviation(metrics.getMaximumCpuUsage(), metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
+ metricsResponse.setResources(hostMetrics.getUpResources(), hostMetrics.getTotalResources());
+ // add CPU and Memory metrics
+ addHostCpuMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+ addHostMemoryMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+
+ metricsResponse.setHasAnnotation(clusterResponse.hasAnnotation());
+ metricsResponses.add(metricsResponse);
+ }
+ return metricsResponses;
+ }
+
+ private void addHostMemoryMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setMemTotal(hostMetrics.getTotalMemory());
+ metricsResponse.setMemAllocated(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory());
+ if (hostMetrics.getMemoryUsed() > 0L) {
+ metricsResponse.setMemUsed(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory());
+ metricsResponse.setMemMaxDeviation(hostMetrics.getMaximumMemoryUsage(), hostMetrics.getMemoryUsed(), hostMetrics.getTotalHosts());
+ }
+ // Memory thresholds
+ final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
+ final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
+ metricsResponse.setMemoryUsageThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryUsageDisableThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryAllocatedThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryAllocatedDisableThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ }
+
+ private void addHostCpuMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setCpuTotal(hostMetrics.getTotalCpu());
+ metricsResponse.setCpuAllocated(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu());
+ if (hostMetrics.getCpuUsedPercentage() > 0L) {
+ metricsResponse.setCpuUsed(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ metricsResponse.setCpuMaxDeviation(hostMetrics.getMaximumCpuUsage(), hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ }
+ // CPU thresholds
+ final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
+ final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
+ metricsResponse.setCpuUsageThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuThreshold);
+ metricsResponse.setCpuUsageDisableThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuDisableThreshold);
+ metricsResponse.setCpuAllocatedThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuThreshold);
+ metricsResponse.setCpuAllocatedDisableThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuDisableThreshold);
+ }
+
+
+ @Override
+ public List<ManagementServerMetricsResponse> listManagementServerMetrics(List<ManagementServerResponse> managementServerResponses) {
+ final List<ManagementServerMetricsResponse> metricsResponses = new ArrayList<>();
+ if(LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("getting metrics for %d MS hosts", managementServerResponses.size()));
+ }
+ for (final ManagementServerResponse managementServerResponse: managementServerResponses) {
+ if(LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("processing metrics for MS hosts %s", managementServerResponse.getId()));
}
- // Memory
- metricsResponse.setMemTotal(metrics.getTotalMemory());
- metricsResponse.setMemAllocated(metrics.getMemoryAllocated(), metrics.getTotalMemory());
- if (metrics.getMemoryUsed() > 0L) {
- metricsResponse.setMemUsed(metrics.getMemoryUsed(), metrics.getTotalMemory());
- metricsResponse.setMemMaxDeviation(metrics.getMaximumMemoryUsage(), metrics.getMemoryUsed(), metrics.getTotalHosts());
+ ManagementServerMetricsResponse metricsResponse = new ManagementServerMetricsResponse();
+
+ try {
+ BeanUtils.copyProperties(metricsResponse, managementServerResponse);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("bean copy result %s", new ReflectionToStringBuilder(metricsResponse, ToStringStyle.SIMPLE_STYLE).toString()));
+ }
+ } catch (IllegalAccessException | InvocationTargetException e) {
+ throw new ServerApiException(ApiErrorCode.INTERNAL_ERROR, "Failed to generate zone metrics response");
Review comment:
```suggestion
throw new ServerApiException(ApiErrorCode.INTERNAL_ERROR, "Failed to generate zone metrics response.");
```
Should we pass the exception as parameter of ServerApiException?
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -335,61 +362,144 @@ public InfrastructureResponse listInfrastructure() {
}
final Long clusterId = cluster.getId();
- // Thresholds
- final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
- final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
- final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
- final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
-
// CPU and memory capacities
final CapacityDaoImpl.SummedCapacity cpuCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_CPU, null, clusterId);
final CapacityDaoImpl.SummedCapacity memoryCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_MEMORY, null, clusterId);
- final Metrics metrics = new Metrics(cpuCapacity, memoryCapacity);
+ final HostMetrics hostMetrics = new HostMetrics(cpuCapacity, memoryCapacity);
for (final Host host: hostDao.findByClusterId(clusterId)) {
if (host == null || host.getType() != Host.Type.Routing) {
continue;
}
if (host.getStatus() == Status.Up) {
- metrics.incrUpResources();
+ hostMetrics.incrUpResources();
}
- metrics.incrTotalResources();
- updateHostMetrics(metrics, hostJoinDao.findById(host.getId()));
+ hostMetrics.incrTotalResources();
+ updateHostMetrics(hostMetrics, hostJoinDao.findById(host.getId()));
}
metricsResponse.setState(clusterResponse.getAllocationState(), clusterResponse.getManagedState());
- metricsResponse.setResources(metrics.getUpResources(), metrics.getTotalResources());
- // CPU
- metricsResponse.setCpuTotal(metrics.getTotalCpu());
- metricsResponse.setCpuAllocated(metrics.getCpuAllocated(), metrics.getTotalCpu());
- if (metrics.getCpuUsedPercentage() > 0L) {
- metricsResponse.setCpuUsed(metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
- metricsResponse.setCpuMaxDeviation(metrics.getMaximumCpuUsage(), metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
+ metricsResponse.setResources(hostMetrics.getUpResources(), hostMetrics.getTotalResources());
+ // add CPU and Memory metrics
+ addHostCpuMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+ addHostMemoryMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+
+ metricsResponse.setHasAnnotation(clusterResponse.hasAnnotation());
+ metricsResponses.add(metricsResponse);
+ }
+ return metricsResponses;
+ }
+
+ private void addHostMemoryMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setMemTotal(hostMetrics.getTotalMemory());
+ metricsResponse.setMemAllocated(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory());
+ if (hostMetrics.getMemoryUsed() > 0L) {
+ metricsResponse.setMemUsed(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory());
+ metricsResponse.setMemMaxDeviation(hostMetrics.getMaximumMemoryUsage(), hostMetrics.getMemoryUsed(), hostMetrics.getTotalHosts());
+ }
+ // Memory thresholds
+ final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
+ final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
+ metricsResponse.setMemoryUsageThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryUsageDisableThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryAllocatedThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryAllocatedDisableThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ }
+
+ private void addHostCpuMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setCpuTotal(hostMetrics.getTotalCpu());
+ metricsResponse.setCpuAllocated(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu());
+ if (hostMetrics.getCpuUsedPercentage() > 0L) {
+ metricsResponse.setCpuUsed(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ metricsResponse.setCpuMaxDeviation(hostMetrics.getMaximumCpuUsage(), hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ }
+ // CPU thresholds
+ final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
+ final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
+ metricsResponse.setCpuUsageThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuThreshold);
+ metricsResponse.setCpuUsageDisableThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuDisableThreshold);
+ metricsResponse.setCpuAllocatedThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuThreshold);
+ metricsResponse.setCpuAllocatedDisableThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuDisableThreshold);
+ }
+
+
+ @Override
+ public List<ManagementServerMetricsResponse> listManagementServerMetrics(List<ManagementServerResponse> managementServerResponses) {
+ final List<ManagementServerMetricsResponse> metricsResponses = new ArrayList<>();
+ if(LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("getting metrics for %d MS hosts", managementServerResponses.size()));
+ }
+ for (final ManagementServerResponse managementServerResponse: managementServerResponses) {
+ if(LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("processing metrics for MS hosts %s", managementServerResponse.getId()));
}
- // Memory
- metricsResponse.setMemTotal(metrics.getTotalMemory());
- metricsResponse.setMemAllocated(metrics.getMemoryAllocated(), metrics.getTotalMemory());
- if (metrics.getMemoryUsed() > 0L) {
- metricsResponse.setMemUsed(metrics.getMemoryUsed(), metrics.getTotalMemory());
- metricsResponse.setMemMaxDeviation(metrics.getMaximumMemoryUsage(), metrics.getMemoryUsed(), metrics.getTotalHosts());
+ ManagementServerMetricsResponse metricsResponse = new ManagementServerMetricsResponse();
+
+ try {
+ BeanUtils.copyProperties(metricsResponse, managementServerResponse);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("bean copy result %s", new ReflectionToStringBuilder(metricsResponse, ToStringStyle.SIMPLE_STYLE).toString()));
Review comment:
```suggestion
LOGGER.trace(String.format("Bean copy result %s.", new ReflectionToStringBuilder(metricsResponse, ToStringStyle.SIMPLE_STYLE).toString()));
```
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -335,61 +362,144 @@ public InfrastructureResponse listInfrastructure() {
}
final Long clusterId = cluster.getId();
- // Thresholds
- final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
- final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
- final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
- final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
-
// CPU and memory capacities
final CapacityDaoImpl.SummedCapacity cpuCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_CPU, null, clusterId);
final CapacityDaoImpl.SummedCapacity memoryCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_MEMORY, null, clusterId);
- final Metrics metrics = new Metrics(cpuCapacity, memoryCapacity);
+ final HostMetrics hostMetrics = new HostMetrics(cpuCapacity, memoryCapacity);
for (final Host host: hostDao.findByClusterId(clusterId)) {
if (host == null || host.getType() != Host.Type.Routing) {
continue;
}
if (host.getStatus() == Status.Up) {
- metrics.incrUpResources();
+ hostMetrics.incrUpResources();
}
- metrics.incrTotalResources();
- updateHostMetrics(metrics, hostJoinDao.findById(host.getId()));
+ hostMetrics.incrTotalResources();
+ updateHostMetrics(hostMetrics, hostJoinDao.findById(host.getId()));
}
metricsResponse.setState(clusterResponse.getAllocationState(), clusterResponse.getManagedState());
- metricsResponse.setResources(metrics.getUpResources(), metrics.getTotalResources());
- // CPU
- metricsResponse.setCpuTotal(metrics.getTotalCpu());
- metricsResponse.setCpuAllocated(metrics.getCpuAllocated(), metrics.getTotalCpu());
- if (metrics.getCpuUsedPercentage() > 0L) {
- metricsResponse.setCpuUsed(metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
- metricsResponse.setCpuMaxDeviation(metrics.getMaximumCpuUsage(), metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
+ metricsResponse.setResources(hostMetrics.getUpResources(), hostMetrics.getTotalResources());
+ // add CPU and Memory metrics
+ addHostCpuMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+ addHostMemoryMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+
+ metricsResponse.setHasAnnotation(clusterResponse.hasAnnotation());
+ metricsResponses.add(metricsResponse);
+ }
+ return metricsResponses;
+ }
+
+ private void addHostMemoryMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setMemTotal(hostMetrics.getTotalMemory());
+ metricsResponse.setMemAllocated(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory());
+ if (hostMetrics.getMemoryUsed() > 0L) {
+ metricsResponse.setMemUsed(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory());
+ metricsResponse.setMemMaxDeviation(hostMetrics.getMaximumMemoryUsage(), hostMetrics.getMemoryUsed(), hostMetrics.getTotalHosts());
+ }
+ // Memory thresholds
+ final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
+ final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
+ metricsResponse.setMemoryUsageThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryUsageDisableThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryAllocatedThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryAllocatedDisableThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ }
+
+ private void addHostCpuMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setCpuTotal(hostMetrics.getTotalCpu());
+ metricsResponse.setCpuAllocated(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu());
+ if (hostMetrics.getCpuUsedPercentage() > 0L) {
+ metricsResponse.setCpuUsed(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ metricsResponse.setCpuMaxDeviation(hostMetrics.getMaximumCpuUsage(), hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ }
+ // CPU thresholds
+ final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
+ final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
+ metricsResponse.setCpuUsageThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuThreshold);
+ metricsResponse.setCpuUsageDisableThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuDisableThreshold);
+ metricsResponse.setCpuAllocatedThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuThreshold);
+ metricsResponse.setCpuAllocatedDisableThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuDisableThreshold);
+ }
+
+
+ @Override
+ public List<ManagementServerMetricsResponse> listManagementServerMetrics(List<ManagementServerResponse> managementServerResponses) {
+ final List<ManagementServerMetricsResponse> metricsResponses = new ArrayList<>();
+ if(LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("getting metrics for %d MS hosts", managementServerResponses.size()));
+ }
+ for (final ManagementServerResponse managementServerResponse: managementServerResponses) {
+ if(LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("processing metrics for MS hosts %s", managementServerResponse.getId()));
Review comment:
```suggestion
LOGGER.debug(String.format("Processing metrics for MS hosts %s.", managementServerResponse.getId()));
```
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -335,61 +362,144 @@ public InfrastructureResponse listInfrastructure() {
}
final Long clusterId = cluster.getId();
- // Thresholds
- final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
- final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
- final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
- final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
-
// CPU and memory capacities
final CapacityDaoImpl.SummedCapacity cpuCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_CPU, null, clusterId);
final CapacityDaoImpl.SummedCapacity memoryCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_MEMORY, null, clusterId);
- final Metrics metrics = new Metrics(cpuCapacity, memoryCapacity);
+ final HostMetrics hostMetrics = new HostMetrics(cpuCapacity, memoryCapacity);
for (final Host host: hostDao.findByClusterId(clusterId)) {
if (host == null || host.getType() != Host.Type.Routing) {
continue;
}
if (host.getStatus() == Status.Up) {
- metrics.incrUpResources();
+ hostMetrics.incrUpResources();
}
- metrics.incrTotalResources();
- updateHostMetrics(metrics, hostJoinDao.findById(host.getId()));
+ hostMetrics.incrTotalResources();
+ updateHostMetrics(hostMetrics, hostJoinDao.findById(host.getId()));
}
metricsResponse.setState(clusterResponse.getAllocationState(), clusterResponse.getManagedState());
- metricsResponse.setResources(metrics.getUpResources(), metrics.getTotalResources());
- // CPU
- metricsResponse.setCpuTotal(metrics.getTotalCpu());
- metricsResponse.setCpuAllocated(metrics.getCpuAllocated(), metrics.getTotalCpu());
- if (metrics.getCpuUsedPercentage() > 0L) {
- metricsResponse.setCpuUsed(metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
- metricsResponse.setCpuMaxDeviation(metrics.getMaximumCpuUsage(), metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
+ metricsResponse.setResources(hostMetrics.getUpResources(), hostMetrics.getTotalResources());
+ // add CPU and Memory metrics
+ addHostCpuMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+ addHostMemoryMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+
+ metricsResponse.setHasAnnotation(clusterResponse.hasAnnotation());
+ metricsResponses.add(metricsResponse);
+ }
+ return metricsResponses;
+ }
+
+ private void addHostMemoryMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setMemTotal(hostMetrics.getTotalMemory());
+ metricsResponse.setMemAllocated(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory());
+ if (hostMetrics.getMemoryUsed() > 0L) {
+ metricsResponse.setMemUsed(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory());
+ metricsResponse.setMemMaxDeviation(hostMetrics.getMaximumMemoryUsage(), hostMetrics.getMemoryUsed(), hostMetrics.getTotalHosts());
+ }
+ // Memory thresholds
+ final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
+ final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
+ metricsResponse.setMemoryUsageThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryUsageDisableThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryAllocatedThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryAllocatedDisableThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ }
+
+ private void addHostCpuMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setCpuTotal(hostMetrics.getTotalCpu());
+ metricsResponse.setCpuAllocated(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu());
+ if (hostMetrics.getCpuUsedPercentage() > 0L) {
+ metricsResponse.setCpuUsed(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ metricsResponse.setCpuMaxDeviation(hostMetrics.getMaximumCpuUsage(), hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ }
+ // CPU thresholds
+ final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
+ final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
+ metricsResponse.setCpuUsageThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuThreshold);
+ metricsResponse.setCpuUsageDisableThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuDisableThreshold);
+ metricsResponse.setCpuAllocatedThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuThreshold);
+ metricsResponse.setCpuAllocatedDisableThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuDisableThreshold);
+ }
+
+
+ @Override
+ public List<ManagementServerMetricsResponse> listManagementServerMetrics(List<ManagementServerResponse> managementServerResponses) {
+ final List<ManagementServerMetricsResponse> metricsResponses = new ArrayList<>();
+ if(LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("getting metrics for %d MS hosts", managementServerResponses.size()));
+ }
+ for (final ManagementServerResponse managementServerResponse: managementServerResponses) {
+ if(LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("processing metrics for MS hosts %s", managementServerResponse.getId()));
}
- // Memory
- metricsResponse.setMemTotal(metrics.getTotalMemory());
- metricsResponse.setMemAllocated(metrics.getMemoryAllocated(), metrics.getTotalMemory());
- if (metrics.getMemoryUsed() > 0L) {
- metricsResponse.setMemUsed(metrics.getMemoryUsed(), metrics.getTotalMemory());
- metricsResponse.setMemMaxDeviation(metrics.getMaximumMemoryUsage(), metrics.getMemoryUsed(), metrics.getTotalHosts());
+ ManagementServerMetricsResponse metricsResponse = new ManagementServerMetricsResponse();
+
+ try {
+ BeanUtils.copyProperties(metricsResponse, managementServerResponse);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("bean copy result %s", new ReflectionToStringBuilder(metricsResponse, ToStringStyle.SIMPLE_STYLE).toString()));
+ }
+ } catch (IllegalAccessException | InvocationTargetException e) {
+ throw new ServerApiException(ApiErrorCode.INTERNAL_ERROR, "Failed to generate zone metrics response");
}
- // CPU thresholds
- metricsResponse.setCpuUsageThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuThreshold);
- metricsResponse.setCpuUsageDisableThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuDisableThreshold);
- metricsResponse.setCpuAllocatedThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuThreshold);
- metricsResponse.setCpuAllocatedDisableThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuDisableThreshold);
- // Memory thresholds
- metricsResponse.setMemoryUsageThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryUsageDisableThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setMemoryAllocatedThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryAllocatedDisableThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setHasAnnotation(clusterResponse.hasAnnotation());
+ updateManagementServerMetrics(metricsResponse, managementServerResponse);
+
metricsResponses.add(metricsResponse);
}
return metricsResponses;
}
+ /**
+ * get the transient/in memory data
Review comment:
```suggestion
* Get the transient/in memory data.
```
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -335,61 +362,144 @@ public InfrastructureResponse listInfrastructure() {
}
final Long clusterId = cluster.getId();
- // Thresholds
- final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
- final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
- final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
- final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
-
// CPU and memory capacities
final CapacityDaoImpl.SummedCapacity cpuCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_CPU, null, clusterId);
final CapacityDaoImpl.SummedCapacity memoryCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_MEMORY, null, clusterId);
- final Metrics metrics = new Metrics(cpuCapacity, memoryCapacity);
+ final HostMetrics hostMetrics = new HostMetrics(cpuCapacity, memoryCapacity);
for (final Host host: hostDao.findByClusterId(clusterId)) {
if (host == null || host.getType() != Host.Type.Routing) {
continue;
}
if (host.getStatus() == Status.Up) {
- metrics.incrUpResources();
+ hostMetrics.incrUpResources();
}
- metrics.incrTotalResources();
- updateHostMetrics(metrics, hostJoinDao.findById(host.getId()));
+ hostMetrics.incrTotalResources();
+ updateHostMetrics(hostMetrics, hostJoinDao.findById(host.getId()));
}
metricsResponse.setState(clusterResponse.getAllocationState(), clusterResponse.getManagedState());
- metricsResponse.setResources(metrics.getUpResources(), metrics.getTotalResources());
- // CPU
- metricsResponse.setCpuTotal(metrics.getTotalCpu());
- metricsResponse.setCpuAllocated(metrics.getCpuAllocated(), metrics.getTotalCpu());
- if (metrics.getCpuUsedPercentage() > 0L) {
- metricsResponse.setCpuUsed(metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
- metricsResponse.setCpuMaxDeviation(metrics.getMaximumCpuUsage(), metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
+ metricsResponse.setResources(hostMetrics.getUpResources(), hostMetrics.getTotalResources());
+ // add CPU and Memory metrics
+ addHostCpuMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+ addHostMemoryMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+
+ metricsResponse.setHasAnnotation(clusterResponse.hasAnnotation());
+ metricsResponses.add(metricsResponse);
+ }
+ return metricsResponses;
+ }
+
+ private void addHostMemoryMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setMemTotal(hostMetrics.getTotalMemory());
+ metricsResponse.setMemAllocated(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory());
+ if (hostMetrics.getMemoryUsed() > 0L) {
+ metricsResponse.setMemUsed(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory());
+ metricsResponse.setMemMaxDeviation(hostMetrics.getMaximumMemoryUsage(), hostMetrics.getMemoryUsed(), hostMetrics.getTotalHosts());
+ }
+ // Memory thresholds
+ final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
+ final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
+ metricsResponse.setMemoryUsageThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryUsageDisableThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryAllocatedThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryAllocatedDisableThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ }
+
+ private void addHostCpuMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setCpuTotal(hostMetrics.getTotalCpu());
+ metricsResponse.setCpuAllocated(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu());
+ if (hostMetrics.getCpuUsedPercentage() > 0L) {
+ metricsResponse.setCpuUsed(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ metricsResponse.setCpuMaxDeviation(hostMetrics.getMaximumCpuUsage(), hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ }
+ // CPU thresholds
+ final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
+ final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
+ metricsResponse.setCpuUsageThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuThreshold);
+ metricsResponse.setCpuUsageDisableThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuDisableThreshold);
+ metricsResponse.setCpuAllocatedThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuThreshold);
+ metricsResponse.setCpuAllocatedDisableThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuDisableThreshold);
+ }
+
+
+ @Override
+ public List<ManagementServerMetricsResponse> listManagementServerMetrics(List<ManagementServerResponse> managementServerResponses) {
+ final List<ManagementServerMetricsResponse> metricsResponses = new ArrayList<>();
+ if(LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("getting metrics for %d MS hosts", managementServerResponses.size()));
+ }
+ for (final ManagementServerResponse managementServerResponse: managementServerResponses) {
+ if(LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("processing metrics for MS hosts %s", managementServerResponse.getId()));
}
- // Memory
- metricsResponse.setMemTotal(metrics.getTotalMemory());
- metricsResponse.setMemAllocated(metrics.getMemoryAllocated(), metrics.getTotalMemory());
- if (metrics.getMemoryUsed() > 0L) {
- metricsResponse.setMemUsed(metrics.getMemoryUsed(), metrics.getTotalMemory());
- metricsResponse.setMemMaxDeviation(metrics.getMaximumMemoryUsage(), metrics.getMemoryUsed(), metrics.getTotalHosts());
+ ManagementServerMetricsResponse metricsResponse = new ManagementServerMetricsResponse();
+
+ try {
+ BeanUtils.copyProperties(metricsResponse, managementServerResponse);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("bean copy result %s", new ReflectionToStringBuilder(metricsResponse, ToStringStyle.SIMPLE_STYLE).toString()));
+ }
+ } catch (IllegalAccessException | InvocationTargetException e) {
+ throw new ServerApiException(ApiErrorCode.INTERNAL_ERROR, "Failed to generate zone metrics response");
}
- // CPU thresholds
- metricsResponse.setCpuUsageThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuThreshold);
- metricsResponse.setCpuUsageDisableThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuDisableThreshold);
- metricsResponse.setCpuAllocatedThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuThreshold);
- metricsResponse.setCpuAllocatedDisableThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuDisableThreshold);
- // Memory thresholds
- metricsResponse.setMemoryUsageThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryUsageDisableThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setMemoryAllocatedThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryAllocatedDisableThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setHasAnnotation(clusterResponse.hasAnnotation());
+ updateManagementServerMetrics(metricsResponse, managementServerResponse);
+
metricsResponses.add(metricsResponse);
}
return metricsResponses;
}
+ /**
+ * get the transient/in memory data
+ * @param metricsResponse
+ * @param managementServerResponse
+ */
+ private void updateManagementServerMetrics(ManagementServerMetricsResponse metricsResponse, ManagementServerResponse managementServerResponse) {
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("getting stats for %s", managementServerResponse.getId()));
+ }
+ ManagementServerHostStats status = ApiDBUtils.getManagementServerHostStatistics(managementServerResponse.getId());
+ if (status == null ) {
+ LOGGER.info(String.format("no status object found for %s - %s", managementServerResponse.getName(), managementServerResponse.getId()));
+ } else {
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("status object found for %s - %s", managementServerResponse.getName(), new ReflectionToStringBuilder(status)));
Review comment:
```suggestion
LOGGER.debug(String.format("Status object found for MS %s - %s.", managementServerResponse.getName(), new ReflectionToStringBuilder(status)));
```
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -335,61 +362,144 @@ public InfrastructureResponse listInfrastructure() {
}
final Long clusterId = cluster.getId();
- // Thresholds
- final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
- final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
- final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
- final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
-
// CPU and memory capacities
final CapacityDaoImpl.SummedCapacity cpuCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_CPU, null, clusterId);
final CapacityDaoImpl.SummedCapacity memoryCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_MEMORY, null, clusterId);
- final Metrics metrics = new Metrics(cpuCapacity, memoryCapacity);
+ final HostMetrics hostMetrics = new HostMetrics(cpuCapacity, memoryCapacity);
for (final Host host: hostDao.findByClusterId(clusterId)) {
if (host == null || host.getType() != Host.Type.Routing) {
continue;
}
if (host.getStatus() == Status.Up) {
- metrics.incrUpResources();
+ hostMetrics.incrUpResources();
}
- metrics.incrTotalResources();
- updateHostMetrics(metrics, hostJoinDao.findById(host.getId()));
+ hostMetrics.incrTotalResources();
+ updateHostMetrics(hostMetrics, hostJoinDao.findById(host.getId()));
}
metricsResponse.setState(clusterResponse.getAllocationState(), clusterResponse.getManagedState());
- metricsResponse.setResources(metrics.getUpResources(), metrics.getTotalResources());
- // CPU
- metricsResponse.setCpuTotal(metrics.getTotalCpu());
- metricsResponse.setCpuAllocated(metrics.getCpuAllocated(), metrics.getTotalCpu());
- if (metrics.getCpuUsedPercentage() > 0L) {
- metricsResponse.setCpuUsed(metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
- metricsResponse.setCpuMaxDeviation(metrics.getMaximumCpuUsage(), metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
+ metricsResponse.setResources(hostMetrics.getUpResources(), hostMetrics.getTotalResources());
+ // add CPU and Memory metrics
+ addHostCpuMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+ addHostMemoryMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+
+ metricsResponse.setHasAnnotation(clusterResponse.hasAnnotation());
+ metricsResponses.add(metricsResponse);
+ }
+ return metricsResponses;
+ }
+
+ private void addHostMemoryMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setMemTotal(hostMetrics.getTotalMemory());
+ metricsResponse.setMemAllocated(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory());
+ if (hostMetrics.getMemoryUsed() > 0L) {
+ metricsResponse.setMemUsed(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory());
+ metricsResponse.setMemMaxDeviation(hostMetrics.getMaximumMemoryUsage(), hostMetrics.getMemoryUsed(), hostMetrics.getTotalHosts());
+ }
+ // Memory thresholds
+ final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
+ final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
+ metricsResponse.setMemoryUsageThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryUsageDisableThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryAllocatedThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryAllocatedDisableThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ }
+
+ private void addHostCpuMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setCpuTotal(hostMetrics.getTotalCpu());
+ metricsResponse.setCpuAllocated(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu());
+ if (hostMetrics.getCpuUsedPercentage() > 0L) {
+ metricsResponse.setCpuUsed(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ metricsResponse.setCpuMaxDeviation(hostMetrics.getMaximumCpuUsage(), hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ }
+ // CPU thresholds
+ final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
+ final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
+ metricsResponse.setCpuUsageThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuThreshold);
+ metricsResponse.setCpuUsageDisableThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuDisableThreshold);
+ metricsResponse.setCpuAllocatedThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuThreshold);
+ metricsResponse.setCpuAllocatedDisableThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuDisableThreshold);
+ }
+
+
+ @Override
+ public List<ManagementServerMetricsResponse> listManagementServerMetrics(List<ManagementServerResponse> managementServerResponses) {
+ final List<ManagementServerMetricsResponse> metricsResponses = new ArrayList<>();
+ if(LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("getting metrics for %d MS hosts", managementServerResponses.size()));
+ }
+ for (final ManagementServerResponse managementServerResponse: managementServerResponses) {
+ if(LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("processing metrics for MS hosts %s", managementServerResponse.getId()));
}
- // Memory
- metricsResponse.setMemTotal(metrics.getTotalMemory());
- metricsResponse.setMemAllocated(metrics.getMemoryAllocated(), metrics.getTotalMemory());
- if (metrics.getMemoryUsed() > 0L) {
- metricsResponse.setMemUsed(metrics.getMemoryUsed(), metrics.getTotalMemory());
- metricsResponse.setMemMaxDeviation(metrics.getMaximumMemoryUsage(), metrics.getMemoryUsed(), metrics.getTotalHosts());
+ ManagementServerMetricsResponse metricsResponse = new ManagementServerMetricsResponse();
+
+ try {
+ BeanUtils.copyProperties(metricsResponse, managementServerResponse);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("bean copy result %s", new ReflectionToStringBuilder(metricsResponse, ToStringStyle.SIMPLE_STYLE).toString()));
+ }
+ } catch (IllegalAccessException | InvocationTargetException e) {
+ throw new ServerApiException(ApiErrorCode.INTERNAL_ERROR, "Failed to generate zone metrics response");
}
- // CPU thresholds
- metricsResponse.setCpuUsageThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuThreshold);
- metricsResponse.setCpuUsageDisableThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuDisableThreshold);
- metricsResponse.setCpuAllocatedThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuThreshold);
- metricsResponse.setCpuAllocatedDisableThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuDisableThreshold);
- // Memory thresholds
- metricsResponse.setMemoryUsageThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryUsageDisableThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setMemoryAllocatedThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryAllocatedDisableThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setHasAnnotation(clusterResponse.hasAnnotation());
+ updateManagementServerMetrics(metricsResponse, managementServerResponse);
+
metricsResponses.add(metricsResponse);
}
return metricsResponses;
}
+ /**
+ * get the transient/in memory data
+ * @param metricsResponse
+ * @param managementServerResponse
Review comment:
I think we could remove these lines if there will be nothing else written.
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -417,73 +527,186 @@ public InfrastructureResponse listInfrastructure() {
// CPU and memory capacities
final CapacityDaoImpl.SummedCapacity cpuCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_CPU, zoneId, null);
final CapacityDaoImpl.SummedCapacity memoryCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_MEMORY, zoneId, null);
- final Metrics metrics = new Metrics(cpuCapacity, memoryCapacity);
+ final HostMetrics hostMetrics = new HostMetrics(cpuCapacity, memoryCapacity);
for (final Cluster cluster : clusterDao.listClustersByDcId(zoneId)) {
if (cluster == null) {
continue;
}
- metrics.incrTotalResources();
+ hostMetrics.incrTotalResources();
if (cluster.getAllocationState() == Grouping.AllocationState.Enabled
&& cluster.getManagedState() == Managed.ManagedState.Managed) {
- metrics.incrUpResources();
+ hostMetrics.incrUpResources();
}
for (final Host host: hostDao.findByClusterId(cluster.getId())) {
if (host == null || host.getType() != Host.Type.Routing) {
continue;
}
- updateHostMetrics(metrics, hostJoinDao.findById(host.getId()));
+ updateHostMetrics(hostMetrics, hostJoinDao.findById(host.getId()));
}
}
metricsResponse.setHasAnnotation(zoneResponse.hasAnnotation());
metricsResponse.setState(zoneResponse.getAllocationState());
- metricsResponse.setResource(metrics.getUpResources(), metrics.getTotalResources());
+ metricsResponse.setResource(hostMetrics.getUpResources(), hostMetrics.getTotalResources());
// CPU
- metricsResponse.setCpuTotal(metrics.getTotalCpu());
- metricsResponse.setCpuAllocated(metrics.getCpuAllocated(), metrics.getTotalCpu());
- if (metrics.getCpuUsedPercentage() > 0L) {
- metricsResponse.setCpuUsed(metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
- metricsResponse.setCpuMaxDeviation(metrics.getMaximumCpuUsage(), metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
+ metricsResponse.setCpuTotal(hostMetrics.getTotalCpu());
+ metricsResponse.setCpuAllocated(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu());
+ if (hostMetrics.getCpuUsedPercentage() > 0L) {
+ metricsResponse.setCpuUsed(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ metricsResponse.setCpuMaxDeviation(hostMetrics.getMaximumCpuUsage(), hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
}
// Memory
- metricsResponse.setMemTotal(metrics.getTotalMemory());
- metricsResponse.setMemAllocated(metrics.getMemoryAllocated(), metrics.getTotalMemory());
- if (metrics.getMemoryUsed() > 0L) {
- metricsResponse.setMemUsed(metrics.getMemoryUsed(), metrics.getTotalMemory());
- metricsResponse.setMemMaxDeviation(metrics.getMaximumMemoryUsage(), metrics.getMemoryUsed(), metrics.getTotalHosts());
+ metricsResponse.setMemTotal(hostMetrics.getTotalMemory());
+ metricsResponse.setMemAllocated(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory());
+ if (hostMetrics.getMemoryUsed() > 0L) {
+ metricsResponse.setMemUsed(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory());
+ metricsResponse.setMemMaxDeviation(hostMetrics.getMaximumMemoryUsage(), hostMetrics.getMemoryUsed(), hostMetrics.getTotalHosts());
}
// CPU thresholds
- metricsResponse.setCpuUsageThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuThreshold);
- metricsResponse.setCpuUsageDisableThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuDisableThreshold);
- metricsResponse.setCpuAllocatedThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuThreshold);
- metricsResponse.setCpuAllocatedDisableThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuDisableThreshold);
+ metricsResponse.setCpuUsageThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuThreshold);
+ metricsResponse.setCpuUsageDisableThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuDisableThreshold);
+ metricsResponse.setCpuAllocatedThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuThreshold);
+ metricsResponse.setCpuAllocatedDisableThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuDisableThreshold);
// Memory thresholds
- metricsResponse.setMemoryUsageThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryUsageDisableThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setMemoryAllocatedThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryAllocatedDisableThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryUsageThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryUsageDisableThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryAllocatedThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryAllocatedDisableThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
metricsResponses.add(metricsResponse);
}
return metricsResponses;
}
+ @Override
+ public UsageServerMetricsResponse listUsageServerMetrics() {
+ UsageServerMetricsResponse response = new UsageServerMetricsResponse();
+ response.setCollectionTime(new Date());
+ TransactionLegacy txn = TransactionLegacy.open(TransactionLegacy.USAGE_DB);
+ try {
+ response.setLastHeartbeat(usageJobDao.getLastHeartbeat());
+ response.setState(isUsageRunning()? State.Up : State.Down);
Review comment:
```suggestion
response.setState(isUsageRunning() ? State.Up : State.Down);
```
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -417,73 +527,186 @@ public InfrastructureResponse listInfrastructure() {
// CPU and memory capacities
final CapacityDaoImpl.SummedCapacity cpuCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_CPU, zoneId, null);
final CapacityDaoImpl.SummedCapacity memoryCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_MEMORY, zoneId, null);
- final Metrics metrics = new Metrics(cpuCapacity, memoryCapacity);
+ final HostMetrics hostMetrics = new HostMetrics(cpuCapacity, memoryCapacity);
for (final Cluster cluster : clusterDao.listClustersByDcId(zoneId)) {
if (cluster == null) {
continue;
}
- metrics.incrTotalResources();
+ hostMetrics.incrTotalResources();
if (cluster.getAllocationState() == Grouping.AllocationState.Enabled
&& cluster.getManagedState() == Managed.ManagedState.Managed) {
- metrics.incrUpResources();
+ hostMetrics.incrUpResources();
}
for (final Host host: hostDao.findByClusterId(cluster.getId())) {
if (host == null || host.getType() != Host.Type.Routing) {
continue;
}
- updateHostMetrics(metrics, hostJoinDao.findById(host.getId()));
+ updateHostMetrics(hostMetrics, hostJoinDao.findById(host.getId()));
}
}
metricsResponse.setHasAnnotation(zoneResponse.hasAnnotation());
metricsResponse.setState(zoneResponse.getAllocationState());
- metricsResponse.setResource(metrics.getUpResources(), metrics.getTotalResources());
+ metricsResponse.setResource(hostMetrics.getUpResources(), hostMetrics.getTotalResources());
// CPU
- metricsResponse.setCpuTotal(metrics.getTotalCpu());
- metricsResponse.setCpuAllocated(metrics.getCpuAllocated(), metrics.getTotalCpu());
- if (metrics.getCpuUsedPercentage() > 0L) {
- metricsResponse.setCpuUsed(metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
- metricsResponse.setCpuMaxDeviation(metrics.getMaximumCpuUsage(), metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
+ metricsResponse.setCpuTotal(hostMetrics.getTotalCpu());
+ metricsResponse.setCpuAllocated(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu());
+ if (hostMetrics.getCpuUsedPercentage() > 0L) {
+ metricsResponse.setCpuUsed(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ metricsResponse.setCpuMaxDeviation(hostMetrics.getMaximumCpuUsage(), hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
}
// Memory
- metricsResponse.setMemTotal(metrics.getTotalMemory());
- metricsResponse.setMemAllocated(metrics.getMemoryAllocated(), metrics.getTotalMemory());
- if (metrics.getMemoryUsed() > 0L) {
- metricsResponse.setMemUsed(metrics.getMemoryUsed(), metrics.getTotalMemory());
- metricsResponse.setMemMaxDeviation(metrics.getMaximumMemoryUsage(), metrics.getMemoryUsed(), metrics.getTotalHosts());
+ metricsResponse.setMemTotal(hostMetrics.getTotalMemory());
+ metricsResponse.setMemAllocated(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory());
+ if (hostMetrics.getMemoryUsed() > 0L) {
+ metricsResponse.setMemUsed(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory());
+ metricsResponse.setMemMaxDeviation(hostMetrics.getMaximumMemoryUsage(), hostMetrics.getMemoryUsed(), hostMetrics.getTotalHosts());
}
// CPU thresholds
- metricsResponse.setCpuUsageThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuThreshold);
- metricsResponse.setCpuUsageDisableThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuDisableThreshold);
- metricsResponse.setCpuAllocatedThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuThreshold);
- metricsResponse.setCpuAllocatedDisableThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuDisableThreshold);
+ metricsResponse.setCpuUsageThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuThreshold);
+ metricsResponse.setCpuUsageDisableThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuDisableThreshold);
+ metricsResponse.setCpuAllocatedThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuThreshold);
+ metricsResponse.setCpuAllocatedDisableThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuDisableThreshold);
// Memory thresholds
- metricsResponse.setMemoryUsageThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryUsageDisableThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setMemoryAllocatedThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryAllocatedDisableThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryUsageThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryUsageDisableThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryAllocatedThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryAllocatedDisableThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
metricsResponses.add(metricsResponse);
}
return metricsResponses;
}
+ @Override
+ public UsageServerMetricsResponse listUsageServerMetrics() {
+ UsageServerMetricsResponse response = new UsageServerMetricsResponse();
+ response.setCollectionTime(new Date());
+ TransactionLegacy txn = TransactionLegacy.open(TransactionLegacy.USAGE_DB);
+ try {
+ response.setLastHeartbeat(usageJobDao.getLastHeartbeat());
+ response.setState(isUsageRunning()? State.Up : State.Down);
+ UsageJobVO job = usageJobDao.getNextImmediateJob();
+ if (job == null) {
+ job = usageJobDao.getLastJob();
+ }
+ response.setHostname(job == null? "N/A": job.getHost());
Review comment:
```suggestion
response.setHostname(job == null ? "N/A" : job.getHost());
```
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -335,61 +362,144 @@ public InfrastructureResponse listInfrastructure() {
}
final Long clusterId = cluster.getId();
- // Thresholds
- final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
- final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
- final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
- final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
-
// CPU and memory capacities
final CapacityDaoImpl.SummedCapacity cpuCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_CPU, null, clusterId);
final CapacityDaoImpl.SummedCapacity memoryCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_MEMORY, null, clusterId);
- final Metrics metrics = new Metrics(cpuCapacity, memoryCapacity);
+ final HostMetrics hostMetrics = new HostMetrics(cpuCapacity, memoryCapacity);
for (final Host host: hostDao.findByClusterId(clusterId)) {
if (host == null || host.getType() != Host.Type.Routing) {
continue;
}
if (host.getStatus() == Status.Up) {
- metrics.incrUpResources();
+ hostMetrics.incrUpResources();
}
- metrics.incrTotalResources();
- updateHostMetrics(metrics, hostJoinDao.findById(host.getId()));
+ hostMetrics.incrTotalResources();
+ updateHostMetrics(hostMetrics, hostJoinDao.findById(host.getId()));
}
metricsResponse.setState(clusterResponse.getAllocationState(), clusterResponse.getManagedState());
- metricsResponse.setResources(metrics.getUpResources(), metrics.getTotalResources());
- // CPU
- metricsResponse.setCpuTotal(metrics.getTotalCpu());
- metricsResponse.setCpuAllocated(metrics.getCpuAllocated(), metrics.getTotalCpu());
- if (metrics.getCpuUsedPercentage() > 0L) {
- metricsResponse.setCpuUsed(metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
- metricsResponse.setCpuMaxDeviation(metrics.getMaximumCpuUsage(), metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
+ metricsResponse.setResources(hostMetrics.getUpResources(), hostMetrics.getTotalResources());
+ // add CPU and Memory metrics
+ addHostCpuMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+ addHostMemoryMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+
+ metricsResponse.setHasAnnotation(clusterResponse.hasAnnotation());
+ metricsResponses.add(metricsResponse);
+ }
+ return metricsResponses;
+ }
+
+ private void addHostMemoryMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setMemTotal(hostMetrics.getTotalMemory());
+ metricsResponse.setMemAllocated(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory());
+ if (hostMetrics.getMemoryUsed() > 0L) {
+ metricsResponse.setMemUsed(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory());
+ metricsResponse.setMemMaxDeviation(hostMetrics.getMaximumMemoryUsage(), hostMetrics.getMemoryUsed(), hostMetrics.getTotalHosts());
+ }
+ // Memory thresholds
+ final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
+ final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
+ metricsResponse.setMemoryUsageThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryUsageDisableThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryAllocatedThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryAllocatedDisableThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ }
+
+ private void addHostCpuMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setCpuTotal(hostMetrics.getTotalCpu());
+ metricsResponse.setCpuAllocated(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu());
+ if (hostMetrics.getCpuUsedPercentage() > 0L) {
+ metricsResponse.setCpuUsed(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ metricsResponse.setCpuMaxDeviation(hostMetrics.getMaximumCpuUsage(), hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ }
+ // CPU thresholds
+ final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
+ final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
+ metricsResponse.setCpuUsageThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuThreshold);
+ metricsResponse.setCpuUsageDisableThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuDisableThreshold);
+ metricsResponse.setCpuAllocatedThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuThreshold);
+ metricsResponse.setCpuAllocatedDisableThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuDisableThreshold);
+ }
+
+
+ @Override
+ public List<ManagementServerMetricsResponse> listManagementServerMetrics(List<ManagementServerResponse> managementServerResponses) {
+ final List<ManagementServerMetricsResponse> metricsResponses = new ArrayList<>();
+ if(LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("getting metrics for %d MS hosts", managementServerResponses.size()));
+ }
+ for (final ManagementServerResponse managementServerResponse: managementServerResponses) {
+ if(LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("processing metrics for MS hosts %s", managementServerResponse.getId()));
}
- // Memory
- metricsResponse.setMemTotal(metrics.getTotalMemory());
- metricsResponse.setMemAllocated(metrics.getMemoryAllocated(), metrics.getTotalMemory());
- if (metrics.getMemoryUsed() > 0L) {
- metricsResponse.setMemUsed(metrics.getMemoryUsed(), metrics.getTotalMemory());
- metricsResponse.setMemMaxDeviation(metrics.getMaximumMemoryUsage(), metrics.getMemoryUsed(), metrics.getTotalHosts());
+ ManagementServerMetricsResponse metricsResponse = new ManagementServerMetricsResponse();
+
+ try {
+ BeanUtils.copyProperties(metricsResponse, managementServerResponse);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("bean copy result %s", new ReflectionToStringBuilder(metricsResponse, ToStringStyle.SIMPLE_STYLE).toString()));
+ }
+ } catch (IllegalAccessException | InvocationTargetException e) {
+ throw new ServerApiException(ApiErrorCode.INTERNAL_ERROR, "Failed to generate zone metrics response");
}
- // CPU thresholds
- metricsResponse.setCpuUsageThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuThreshold);
- metricsResponse.setCpuUsageDisableThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuDisableThreshold);
- metricsResponse.setCpuAllocatedThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuThreshold);
- metricsResponse.setCpuAllocatedDisableThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuDisableThreshold);
- // Memory thresholds
- metricsResponse.setMemoryUsageThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryUsageDisableThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setMemoryAllocatedThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryAllocatedDisableThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setHasAnnotation(clusterResponse.hasAnnotation());
+ updateManagementServerMetrics(metricsResponse, managementServerResponse);
+
metricsResponses.add(metricsResponse);
}
return metricsResponses;
}
+ /**
+ * get the transient/in memory data
+ * @param metricsResponse
+ * @param managementServerResponse
+ */
+ private void updateManagementServerMetrics(ManagementServerMetricsResponse metricsResponse, ManagementServerResponse managementServerResponse) {
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("getting stats for %s", managementServerResponse.getId()));
+ }
+ ManagementServerHostStats status = ApiDBUtils.getManagementServerHostStatistics(managementServerResponse.getId());
+ if (status == null ) {
+ LOGGER.info(String.format("no status object found for %s - %s", managementServerResponse.getName(), managementServerResponse.getId()));
Review comment:
```suggestion
LOGGER.info(String.format("No status object found for MS %s - %s.", managementServerResponse.getName(), managementServerResponse.getId()));
```
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -417,73 +527,186 @@ public InfrastructureResponse listInfrastructure() {
// CPU and memory capacities
final CapacityDaoImpl.SummedCapacity cpuCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_CPU, zoneId, null);
final CapacityDaoImpl.SummedCapacity memoryCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_MEMORY, zoneId, null);
- final Metrics metrics = new Metrics(cpuCapacity, memoryCapacity);
+ final HostMetrics hostMetrics = new HostMetrics(cpuCapacity, memoryCapacity);
for (final Cluster cluster : clusterDao.listClustersByDcId(zoneId)) {
if (cluster == null) {
continue;
}
- metrics.incrTotalResources();
+ hostMetrics.incrTotalResources();
if (cluster.getAllocationState() == Grouping.AllocationState.Enabled
&& cluster.getManagedState() == Managed.ManagedState.Managed) {
- metrics.incrUpResources();
+ hostMetrics.incrUpResources();
}
for (final Host host: hostDao.findByClusterId(cluster.getId())) {
if (host == null || host.getType() != Host.Type.Routing) {
continue;
}
- updateHostMetrics(metrics, hostJoinDao.findById(host.getId()));
+ updateHostMetrics(hostMetrics, hostJoinDao.findById(host.getId()));
}
}
metricsResponse.setHasAnnotation(zoneResponse.hasAnnotation());
metricsResponse.setState(zoneResponse.getAllocationState());
- metricsResponse.setResource(metrics.getUpResources(), metrics.getTotalResources());
+ metricsResponse.setResource(hostMetrics.getUpResources(), hostMetrics.getTotalResources());
// CPU
- metricsResponse.setCpuTotal(metrics.getTotalCpu());
- metricsResponse.setCpuAllocated(metrics.getCpuAllocated(), metrics.getTotalCpu());
- if (metrics.getCpuUsedPercentage() > 0L) {
- metricsResponse.setCpuUsed(metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
- metricsResponse.setCpuMaxDeviation(metrics.getMaximumCpuUsage(), metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
+ metricsResponse.setCpuTotal(hostMetrics.getTotalCpu());
+ metricsResponse.setCpuAllocated(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu());
+ if (hostMetrics.getCpuUsedPercentage() > 0L) {
+ metricsResponse.setCpuUsed(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ metricsResponse.setCpuMaxDeviation(hostMetrics.getMaximumCpuUsage(), hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
}
// Memory
- metricsResponse.setMemTotal(metrics.getTotalMemory());
- metricsResponse.setMemAllocated(metrics.getMemoryAllocated(), metrics.getTotalMemory());
- if (metrics.getMemoryUsed() > 0L) {
- metricsResponse.setMemUsed(metrics.getMemoryUsed(), metrics.getTotalMemory());
- metricsResponse.setMemMaxDeviation(metrics.getMaximumMemoryUsage(), metrics.getMemoryUsed(), metrics.getTotalHosts());
+ metricsResponse.setMemTotal(hostMetrics.getTotalMemory());
+ metricsResponse.setMemAllocated(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory());
+ if (hostMetrics.getMemoryUsed() > 0L) {
+ metricsResponse.setMemUsed(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory());
+ metricsResponse.setMemMaxDeviation(hostMetrics.getMaximumMemoryUsage(), hostMetrics.getMemoryUsed(), hostMetrics.getTotalHosts());
}
// CPU thresholds
- metricsResponse.setCpuUsageThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuThreshold);
- metricsResponse.setCpuUsageDisableThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuDisableThreshold);
- metricsResponse.setCpuAllocatedThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuThreshold);
- metricsResponse.setCpuAllocatedDisableThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuDisableThreshold);
+ metricsResponse.setCpuUsageThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuThreshold);
+ metricsResponse.setCpuUsageDisableThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuDisableThreshold);
+ metricsResponse.setCpuAllocatedThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuThreshold);
+ metricsResponse.setCpuAllocatedDisableThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuDisableThreshold);
// Memory thresholds
- metricsResponse.setMemoryUsageThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryUsageDisableThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setMemoryAllocatedThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryAllocatedDisableThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryUsageThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryUsageDisableThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryAllocatedThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryAllocatedDisableThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
metricsResponses.add(metricsResponse);
}
return metricsResponses;
}
+ @Override
+ public UsageServerMetricsResponse listUsageServerMetrics() {
+ UsageServerMetricsResponse response = new UsageServerMetricsResponse();
+ response.setCollectionTime(new Date());
+ TransactionLegacy txn = TransactionLegacy.open(TransactionLegacy.USAGE_DB);
+ try {
+ response.setLastHeartbeat(usageJobDao.getLastHeartbeat());
+ response.setState(isUsageRunning()? State.Up : State.Down);
+ UsageJobVO job = usageJobDao.getNextImmediateJob();
+ if (job == null) {
+ job = usageJobDao.getLastJob();
+ }
+ response.setHostname(job == null? "N/A": job.getHost());
+ response.setLastSuccessfulJob(new Date(usageJobDao.getLastJobSuccessDateMillis()));
+ } finally {
+ txn.close();
+ TransactionLegacy swap = TransactionLegacy.open(TransactionLegacy.CLOUD_DB);
+ swap.close();
+ }
+ response.setObjectName("usageMetrics");
+ return response;
+ }
+
+ /**
+ TODO
+ â—¦ State (Up / Down) , makes no sense (no db no cloudstack)
+ â—¦ Average Queries Per Second
+ â—¦ Buffer Pool Utilization (buffer pool is used to cache the table data in memory and is accessed repeatedly by queries without requiring any disk I/O).
+ â—¦ any other relevant stats (if useful) to the response from the sql status variables.
+ * @return
+ */
+ @Override
+ public DbMetricsResponse listDbMetrics() {
+ DbMetricsResponse response = new DbMetricsResponse();
+
+ response.setCollectionTime(new Date());
+ response.setHostname(dbHostName());
+ response.setReplicas(dbReplicas());
+ getDynamicDataFromDB(response);
+ getStaticDataFromDB(response);
+
+ getQueryHistory(response);
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(new ReflectionToStringBuilder(response));
+ }
+
+ response.setObjectName("dbMetrics");
+ return response;
+ }
+
+ private void getQueryHistory(DbMetricsResponse response) {
+ Map<String, Object> dbStats = ApiDBUtils.getDbStatistics();
+ if (dbStats != null) {
+ response.setQueries((Integer)dbStats.get(DbStatsCollection.queries));
+ response.setUptime((Integer)dbStats.get(DbStatsCollection.uptime));
+ }
+
+ List<Double> loadHistory = (List<Double>) dbStats.get(DbStatsCollection.loadAvarages);
+ double[] loadAverages = new double[loadHistory.size()];
+
+ int index =0;
+ for (Double d : loadHistory) {
+ loadAverages[index++] = d;
+ }
+
+ response.setLoadAverages(loadAverages);
+ }
+
+ private void getStaticDataFromDB(DbMetricsResponse response) {
+ Map<String, String> vars = DbUtil.getDbInfo(DbStatsCollection.variables, DbStatsCollection.version, DbStatsCollection.versionComment);
+ response.setVersion(vars.get(DbStatsCollection.version));
+ response.setVersionComment(vars.get(DbStatsCollection.versionComment));
+ }
+
+ private void getDynamicDataFromDB(DbMetricsResponse response) {
+ Map<String, String> stats = DbUtil.getDbInfo(DbStatsCollection.status, DbStatsCollection.connections, DbStatsCollection.currentTlsVersion);
+ response.setConnections(Integer.parseInt(stats.get(DbStatsCollection.connections)));
+ response.setTlsVersions(stats.get(DbStatsCollection.currentTlsVersion));
+ }
+
+ private String dbHostName() {
+ Properties p = DbProperties.getDbProperties();
+ return p.getProperty("db.cloud.host");
+ }
+
+ private String[] dbReplicas() {
+ Properties p = DbProperties.getDbProperties();
+ return p.getProperty("db.cloud.replicas","").split(",");
+ }
+
+ /**
+ * returns whether a local usage server is running.
+ * Note that this might not be the one actually doing the usage aggregation at this moment.
+ * @return true if the service is active
+ */
+ protected boolean isUsageRunning() {
+ boolean local = false;
+ String usageStatus = Script.runSimpleBashScript("systemctl status cloudstack-usage | grep \" Active:\"");
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("usage status: %s", usageStatus));
Review comment:
```suggestion
LOGGER.trace(String.format("The current usage status is: %s.", usageStatus));
```
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -417,73 +527,186 @@ public InfrastructureResponse listInfrastructure() {
// CPU and memory capacities
final CapacityDaoImpl.SummedCapacity cpuCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_CPU, zoneId, null);
final CapacityDaoImpl.SummedCapacity memoryCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_MEMORY, zoneId, null);
- final Metrics metrics = new Metrics(cpuCapacity, memoryCapacity);
+ final HostMetrics hostMetrics = new HostMetrics(cpuCapacity, memoryCapacity);
for (final Cluster cluster : clusterDao.listClustersByDcId(zoneId)) {
if (cluster == null) {
continue;
}
- metrics.incrTotalResources();
+ hostMetrics.incrTotalResources();
if (cluster.getAllocationState() == Grouping.AllocationState.Enabled
&& cluster.getManagedState() == Managed.ManagedState.Managed) {
- metrics.incrUpResources();
+ hostMetrics.incrUpResources();
}
for (final Host host: hostDao.findByClusterId(cluster.getId())) {
if (host == null || host.getType() != Host.Type.Routing) {
continue;
}
- updateHostMetrics(metrics, hostJoinDao.findById(host.getId()));
+ updateHostMetrics(hostMetrics, hostJoinDao.findById(host.getId()));
}
}
metricsResponse.setHasAnnotation(zoneResponse.hasAnnotation());
metricsResponse.setState(zoneResponse.getAllocationState());
- metricsResponse.setResource(metrics.getUpResources(), metrics.getTotalResources());
+ metricsResponse.setResource(hostMetrics.getUpResources(), hostMetrics.getTotalResources());
// CPU
- metricsResponse.setCpuTotal(metrics.getTotalCpu());
- metricsResponse.setCpuAllocated(metrics.getCpuAllocated(), metrics.getTotalCpu());
- if (metrics.getCpuUsedPercentage() > 0L) {
- metricsResponse.setCpuUsed(metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
- metricsResponse.setCpuMaxDeviation(metrics.getMaximumCpuUsage(), metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
+ metricsResponse.setCpuTotal(hostMetrics.getTotalCpu());
+ metricsResponse.setCpuAllocated(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu());
+ if (hostMetrics.getCpuUsedPercentage() > 0L) {
+ metricsResponse.setCpuUsed(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ metricsResponse.setCpuMaxDeviation(hostMetrics.getMaximumCpuUsage(), hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
}
// Memory
- metricsResponse.setMemTotal(metrics.getTotalMemory());
- metricsResponse.setMemAllocated(metrics.getMemoryAllocated(), metrics.getTotalMemory());
- if (metrics.getMemoryUsed() > 0L) {
- metricsResponse.setMemUsed(metrics.getMemoryUsed(), metrics.getTotalMemory());
- metricsResponse.setMemMaxDeviation(metrics.getMaximumMemoryUsage(), metrics.getMemoryUsed(), metrics.getTotalHosts());
+ metricsResponse.setMemTotal(hostMetrics.getTotalMemory());
+ metricsResponse.setMemAllocated(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory());
+ if (hostMetrics.getMemoryUsed() > 0L) {
+ metricsResponse.setMemUsed(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory());
+ metricsResponse.setMemMaxDeviation(hostMetrics.getMaximumMemoryUsage(), hostMetrics.getMemoryUsed(), hostMetrics.getTotalHosts());
}
// CPU thresholds
- metricsResponse.setCpuUsageThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuThreshold);
- metricsResponse.setCpuUsageDisableThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuDisableThreshold);
- metricsResponse.setCpuAllocatedThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuThreshold);
- metricsResponse.setCpuAllocatedDisableThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuDisableThreshold);
+ metricsResponse.setCpuUsageThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuThreshold);
+ metricsResponse.setCpuUsageDisableThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuDisableThreshold);
+ metricsResponse.setCpuAllocatedThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuThreshold);
+ metricsResponse.setCpuAllocatedDisableThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuDisableThreshold);
// Memory thresholds
- metricsResponse.setMemoryUsageThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryUsageDisableThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setMemoryAllocatedThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryAllocatedDisableThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryUsageThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryUsageDisableThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryAllocatedThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryAllocatedDisableThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
metricsResponses.add(metricsResponse);
}
return metricsResponses;
}
+ @Override
+ public UsageServerMetricsResponse listUsageServerMetrics() {
+ UsageServerMetricsResponse response = new UsageServerMetricsResponse();
+ response.setCollectionTime(new Date());
+ TransactionLegacy txn = TransactionLegacy.open(TransactionLegacy.USAGE_DB);
+ try {
+ response.setLastHeartbeat(usageJobDao.getLastHeartbeat());
+ response.setState(isUsageRunning()? State.Up : State.Down);
+ UsageJobVO job = usageJobDao.getNextImmediateJob();
+ if (job == null) {
+ job = usageJobDao.getLastJob();
+ }
+ response.setHostname(job == null? "N/A": job.getHost());
+ response.setLastSuccessfulJob(new Date(usageJobDao.getLastJobSuccessDateMillis()));
+ } finally {
+ txn.close();
+ TransactionLegacy swap = TransactionLegacy.open(TransactionLegacy.CLOUD_DB);
+ swap.close();
+ }
+ response.setObjectName("usageMetrics");
+ return response;
+ }
+
+ /**
+ TODO
+ â—¦ State (Up / Down) , makes no sense (no db no cloudstack)
+ â—¦ Average Queries Per Second
+ â—¦ Buffer Pool Utilization (buffer pool is used to cache the table data in memory and is accessed repeatedly by queries without requiring any disk I/O).
+ â—¦ any other relevant stats (if useful) to the response from the sql status variables.
+ * @return
+ */
+ @Override
+ public DbMetricsResponse listDbMetrics() {
+ DbMetricsResponse response = new DbMetricsResponse();
+
+ response.setCollectionTime(new Date());
+ response.setHostname(dbHostName());
+ response.setReplicas(dbReplicas());
+ getDynamicDataFromDB(response);
+ getStaticDataFromDB(response);
+
+ getQueryHistory(response);
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(new ReflectionToStringBuilder(response));
+ }
+
+ response.setObjectName("dbMetrics");
+ return response;
+ }
+
+ private void getQueryHistory(DbMetricsResponse response) {
+ Map<String, Object> dbStats = ApiDBUtils.getDbStatistics();
+ if (dbStats != null) {
+ response.setQueries((Integer)dbStats.get(DbStatsCollection.queries));
+ response.setUptime((Integer)dbStats.get(DbStatsCollection.uptime));
+ }
+
+ List<Double> loadHistory = (List<Double>) dbStats.get(DbStatsCollection.loadAvarages);
+ double[] loadAverages = new double[loadHistory.size()];
+
+ int index =0;
Review comment:
```suggestion
int index = 0;
```
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -207,22 +259,34 @@ public String toString() {
private static final String INFLUXDB_HOST_MEASUREMENT = "host_stats";
private static final String INFLUXDB_VM_MEASUREMENT = "vm_stats";
- private static final ConfigKey<Integer> vmDiskStatsInterval = new ConfigKey<Integer>("Advanced", Integer.class, "vm.disk.stats.interval", "0",
+ private static final ConfigKey<Integer> MANAGEMENT_SERVER_STATUS_COLLECTION_INTERVAL = new ConfigKey<>("Advanced",
+ Integer.class, "management.server.stats.interval", "60",
+ "time interval for management servers stats. Set to <= 0 to disable management servers stats", false);
+ private static final ConfigKey<Integer> USAGE_SERVER_STATUS_COLLECTION_INTERVAL = new ConfigKey<>("Advanced",
+ Integer.class, "usage.server.stats.interval", "60",
+ "time interval for usage servers stats. Set to <= 0 to disable usage servers stats", false);
+ private static final ConfigKey<Integer> DATABASE_SERVER_STATUS_COLLECTION_INTERVAL = new ConfigKey<>("Advanced",
+ Integer.class, "database.server.stats.interval", "60",
+ "time interval for database servers stats. Set to <= 0 to disable database servers stats", false);
+ private static final ConfigKey<Integer> DATABASE_SERVER_LOAD_HISTORY_RETENTION_NUMBER = new ConfigKey<>("Advanced",
+ Integer.class, "database.server.stats.retention", "10",
+ "the number of load averages to retain in history", true);
Review comment:
```suggestion
"The number of load averages to retain in history.", true);
```
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -334,8 +406,28 @@ public StatsCollector() {
@Override
public boolean start() {
init(_configDao.getConfiguration());
+ registerAll("gc", new GarbageCollectorMetricSet(), METRIC_REGISTRY);
+ registerAll("buffers", new BufferPoolMetricSet(ManagementFactory.getPlatformMBeanServer()), METRIC_REGISTRY);
+ registerAll("memory", new MemoryUsageGaugeSet(), METRIC_REGISTRY);
+ registerAll("threads", new ThreadStatesGaugeSet(), METRIC_REGISTRY);
+ registerAll("jvm", new JvmAttributeGaugeSet(), METRIC_REGISTRY);
return true;
}
+ @Override
+ public boolean stop() {
+ _executor.shutdown();
+ return true;
+ }
+
+ private void registerAll(String prefix, MetricSet metricSet, MetricRegistry registry) {
+ for (Map.Entry<String, Metric> entry : metricSet.getMetrics().entrySet()) {
+ if (entry.getValue() instanceof MetricSet) {
+ registerAll(prefix + "." + entry.getKey(), (MetricSet) entry.getValue(), registry);
+ } else {
+ registry.register(prefix + "." + entry.getKey(), entry.getValue());
+ }
Review comment:
```suggestion
String registryName = String.format("%s.%s", prefix, entry.getKey());
Metric entryValue = entry.getValue();
if (entryValue instanceof MetricSet) {
registerAll(registryName, (MetricSet) entryValue, registry);
} else {
registry.register(registryName, entryValue);
}
```
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -564,11 +681,485 @@ protected Point creteInfluxDbPoint(Object metricsObject) {
}
}
+ class DbCollector extends AbstractStatsCollector {
+ List<Double> loadHistory = new ArrayList<>();
+ DbCollector() {
+ dbStats.put(loadAvarages, loadHistory);
+ }
+ @Override
+ protected void runInContext() {
+ LOGGER.debug(String.format("%s is running...", this.getClass().getSimpleName()));
+
+ try {
+ int lastUptime = (dbStats.containsKey(uptime) ? (Integer) dbStats.get(uptime) : 0);
+ int lastQueries = (dbStats.containsKey(queries) ? (Integer) dbStats.get(queries) : 0);
+ getDynamicDataFromDB();
+ int interval = (Integer) dbStats.get(uptime) - lastUptime;
+ int activity = (Integer) dbStats.get(queries) - lastQueries;
+ loadHistory.add(0, Double.valueOf(activity / interval));
+ int maxsize = DATABASE_SERVER_LOAD_HISTORY_RETENTION_NUMBER.value();
+ while (loadHistory.size() > maxsize) {
+ loadHistory.remove(maxsize - 1);
+ }
+ } catch (Throwable e) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("db statistics collection failed due to " + e.getLocalizedMessage());
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("db statistics collection failed.", e);
+ }
Review comment:
Do we need 2 logs?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1030661368
@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1030736655
<b>Trillian test result (tid-3179)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 31526 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5588-t3179-kvm-centos7.zip
Smoke tests completed. 91 look OK, 1 have errors
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_list_management_server_metrics | `Failure` | 0.04 | test_metrics_api.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1056715357
@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1056714376
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-979358819
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1788
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-977656706
Packaging result: :heavy_multiplication_x: el7 :heavy_multiplication_x: el8 :heavy_multiplication_x: debian :heavy_multiplication_x: suse15. SL-JID 1776
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] weizhouapache commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
weizhouapache commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r752065441
##########
File path: ui/.env.remote
##########
@@ -0,0 +1 @@
+CS_URL=http://10.0.32.234:8080
Review comment:
the changes on package-lock.json is huge
```
32,591 additions, 440 deletions not shown because the diff is too large. Please use a local Git client to view these changes.
```
Do we need the new nodejs modules ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r752094566
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -564,11 +657,352 @@ protected Point creteInfluxDbPoint(Object metricsObject) {
}
}
+ Gson gson;
+
+ class ManagementServerCollector extends AbstractStatsCollector {
+ @Override
+ protected void runInContext() {
+ LOGGER.debug(String.format("%s is running...", this.getClass().getSimpleName()));
+ long msid = ManagementServerNode.getManagementServerId();
+ ManagementServerHostVO mshost = null;
+ ManagementServerHostStatsEntry hostStatsEntry = null;
+ try {
+ mshost = managementServerHostDao.findByMsid(msid);
+ // get local data
+ hostStatsEntry = getDataFrom(mshost);
+ managementServerHostStats.put(mshost.getUuid(), hostStatsEntry);
+ // send to other hosts
+ clusterManager.publishStatus(gson.toJson(hostStatsEntry));
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to retrieve host stats", t);
+ }
+ try {
+ // send to DB
+ storeStatus(hostStatsEntry, mshost);
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to store host state", t);
+ }
+ }
+
+ private void storeStatus(ManagementServerHostStatsEntry hostStatsEntry, ManagementServerHostVO mshost) {
+ if (hostStatsEntry == null || mshost == null) {
+ return;
+ }
+ ManagementServerStatusVO msStats = managementServerStatusDao.findByMsId(hostStatsEntry.getManagementServerHostUuid());
+ if (msStats == null) {
+ LOGGER.info(String.format("creating new status info record for host %s - %s",
+ mshost.getName(),
+ hostStatsEntry.getManagementServerHostUuid()));
+ msStats = new ManagementServerStatusVO();
+ msStats.setMsId(hostStatsEntry.getManagementServerHostUuid());
+ }
+ msStats.setOsDistribution(hostStatsEntry.getOsDistribution()); // for now just the bunch details come later
+ msStats.setJavaName(hostStatsEntry.getJvmVendor());
+ msStats.setJavaVersion(hostStatsEntry.getJvmVersion());
+ Date startTime = new Date(hostStatsEntry.getStartTime());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("reporting starttime %s", startTime));
+ }
+ msStats.setLastStart(startTime);
+ msStats.setUpdated(new Date());
+ managementServerStatusDao.persist(msStats);
+ }
+
+ @NotNull
+ private ManagementServerHostStatsEntry getDataFrom(ManagementServerHostVO mshost) {
+ ManagementServerHostStatsEntry newEntry = new ManagementServerHostStatsEntry();
+ LOGGER.debug("Metrics collection start...");
+ newEntry.setManagementServerHostId(mshost.getId());
+ newEntry.setManagementServerHostUuid(mshost.getUuid());
+ retrieveSession(newEntry);
+ getJvmDimensions(newEntry);
+ LOGGER.debug("Metrics collection extra...");
+ getRuntimeData(newEntry);
+ getCpuData(newEntry);
+ getMemoryData(newEntry);
+ // newEntry must now include a pid!
+ getProcFsData(newEntry);
+ getFsData(newEntry);
+ getDataBaseStatistics(newEntry, mshost.getMsid());
+ gatherAllMetrics(newEntry);
+ LOGGER.debug("Metrics collection end!");
+ return newEntry;
+ }
+
+ private void retrieveSession(ManagementServerHostStatsEntry newEntry) {
+ long sessions = ApiSessionListener.getSessionCount();
+ newEntry.setSessions(sessions);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("Sessions found in Api %d vs context %d", sessions,ApiSessionListener.getNumberOfSessions()));
+ } else {
+ LOGGER.debug("Sessions active: " + sessions);
+ }
+ }
+
+ private void getDataBaseStatistics(ManagementServerHostStatsEntry newEntry, long msid) {
+ int count = _hostDao.countByMs(msid);
+ newEntry.setAgentCount(count);
+ }
+
+ private void getMemoryData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ MemoryMXBean mxBean = ManagementFactory.getMemoryMXBean();
+ newEntry.setTotalInit(mxBean.getHeapMemoryUsage().getInit() + mxBean.getNonHeapMemoryUsage().getInit());
+ newEntry.setTotalUsed(mxBean.getHeapMemoryUsage().getUsed() + mxBean.getNonHeapMemoryUsage().getUsed());
+ newEntry.setTotalMax(mxBean.getHeapMemoryUsage().getMax() + mxBean.getNonHeapMemoryUsage().getMax());
+ newEntry.setTotalCommitted(mxBean.getHeapMemoryUsage().getCommitted() + mxBean.getNonHeapMemoryUsage().getCommitted());
+ }
+
+ private void getCpuData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final OperatingSystemMXBean mxBean = ManagementFactory.getOperatingSystemMXBean();
+ newEntry.setAvailableProcessors(mxBean.getAvailableProcessors());
+ newEntry.setLoadAverage(mxBean.getSystemLoadAverage());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics processors - %d , loadavg - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getLoadAverage()));
+ }
+ }
+
+ private void getRuntimeData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final RuntimeMXBean mxBean = ManagementFactory.getRuntimeMXBean();
+ newEntry.setUptime(mxBean.getUptime());
+ newEntry.setStartTime(mxBean.getStartTime());
+ newEntry.setProcessId(mxBean.getPid());
+ newEntry.setJvmName(mxBean.getName());
+ newEntry.setJvmVendor(mxBean.getVmVendor());
+ newEntry.setJvmVersion(mxBean.getVmVersion());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics uptime - %d , starttime - %d",
+ newEntry.getUptime(),
+ newEntry.getStartTime()));
+ }
+ }
+
+ private void getJvmDimensions(@NotNull ManagementServerHostStatsEntry newEntry) {
+ Runtime runtime = Runtime.getRuntime();
+ newEntry.setTotalMemoryBytes(runtime.totalMemory());
+ newEntry.setFreeMemoryBytes(runtime.freeMemory());
+ newEntry.setAvailableProcessors(runtime.availableProcessors());
+ newEntry.setTotalMax(runtime.maxMemory());
+ //long maxMem = runtime.maxMemory();
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics proc - %d , maxMem - %d , totalMemory - %d , freeMemory - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getTotalMax(),
+ newEntry.getTotalMemoryBytes(),
+ newEntry.getFreeMemoryBytes()));
+ }
+ }
+
+ /**
+ * As for data from outside the JVM, we only rely on /proc/ contained data.
+ *
+ * @param newEntry
+ */
+ private void getProcFsData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ String OS = Script.runSimpleBashScript("cat /proc/version");
+ newEntry.setOsDistribution(OS);
+ String mem = Script.runSimpleBashScript("cat /proc/meminfo | grep MemTotal | cut -f 2 -d ':' | tr -d 'a-zA-z '").trim();
+ newEntry.setSystemMemoryTotal(Long.parseLong(mem) * 1024);
+ String free = Script.runSimpleBashScript("cat /proc/meminfo | grep MemFree | cut -f 2 -d ':' | tr -d 'a-zA-z '").trim();
+ newEntry.setSystemMemoryFree(Long.parseLong(free) * 1024);
Review comment:
not in java 11 either
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r752257403
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -344,52 +355,107 @@ public InfrastructureResponse listInfrastructure() {
// CPU and memory capacities
final CapacityDaoImpl.SummedCapacity cpuCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_CPU, null, clusterId);
final CapacityDaoImpl.SummedCapacity memoryCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_MEMORY, null, clusterId);
- final Metrics metrics = new Metrics(cpuCapacity, memoryCapacity);
+ final org.apache.cloudstack.metrics.MetricsServiceImpl.HostMetrics hostMetrics = new org.apache.cloudstack.metrics.MetricsServiceImpl.HostMetrics(cpuCapacity, memoryCapacity);
for (final Host host: hostDao.findByClusterId(clusterId)) {
if (host == null || host.getType() != Host.Type.Routing) {
continue;
}
if (host.getStatus() == Status.Up) {
- metrics.incrUpResources();
+ hostMetrics.incrUpResources();
}
- metrics.incrTotalResources();
- updateHostMetrics(metrics, hostJoinDao.findById(host.getId()));
+ hostMetrics.incrTotalResources();
+ updateHostMetrics(hostMetrics, hostJoinDao.findById(host.getId()));
}
metricsResponse.setState(clusterResponse.getAllocationState(), clusterResponse.getManagedState());
- metricsResponse.setResources(metrics.getUpResources(), metrics.getTotalResources());
+ metricsResponse.setResources(hostMetrics.getUpResources(), hostMetrics.getTotalResources());
// CPU
- metricsResponse.setCpuTotal(metrics.getTotalCpu());
- metricsResponse.setCpuAllocated(metrics.getCpuAllocated(), metrics.getTotalCpu());
- if (metrics.getCpuUsedPercentage() > 0L) {
- metricsResponse.setCpuUsed(metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
- metricsResponse.setCpuMaxDeviation(metrics.getMaximumCpuUsage(), metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
+ metricsResponse.setCpuTotal(hostMetrics.getTotalCpu());
+ metricsResponse.setCpuAllocated(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu());
+ if (hostMetrics.getCpuUsedPercentage() > 0L) {
+ metricsResponse.setCpuUsed(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ metricsResponse.setCpuMaxDeviation(hostMetrics.getMaximumCpuUsage(), hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
}
// Memory
- metricsResponse.setMemTotal(metrics.getTotalMemory());
- metricsResponse.setMemAllocated(metrics.getMemoryAllocated(), metrics.getTotalMemory());
- if (metrics.getMemoryUsed() > 0L) {
- metricsResponse.setMemUsed(metrics.getMemoryUsed(), metrics.getTotalMemory());
- metricsResponse.setMemMaxDeviation(metrics.getMaximumMemoryUsage(), metrics.getMemoryUsed(), metrics.getTotalHosts());
+ metricsResponse.setMemTotal(hostMetrics.getTotalMemory());
+ metricsResponse.setMemAllocated(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory());
+ if (hostMetrics.getMemoryUsed() > 0L) {
+ metricsResponse.setMemUsed(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory());
+ metricsResponse.setMemMaxDeviation(hostMetrics.getMaximumMemoryUsage(), hostMetrics.getMemoryUsed(), hostMetrics.getTotalHosts());
}
// CPU thresholds
- metricsResponse.setCpuUsageThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuThreshold);
- metricsResponse.setCpuUsageDisableThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuDisableThreshold);
- metricsResponse.setCpuAllocatedThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuThreshold);
- metricsResponse.setCpuAllocatedDisableThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuDisableThreshold);
+ metricsResponse.setCpuUsageThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuThreshold);
+ metricsResponse.setCpuUsageDisableThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuDisableThreshold);
+ metricsResponse.setCpuAllocatedThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuThreshold);
+ metricsResponse.setCpuAllocatedDisableThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuDisableThreshold);
// Memory thresholds
- metricsResponse.setMemoryUsageThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryUsageDisableThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setMemoryAllocatedThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryAllocatedDisableThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryUsageThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryUsageDisableThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryAllocatedThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryAllocatedDisableThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
Review comment:
have a look @GutoVeronezi ;) i got to it (finally)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-973849363
Packaging result: :heavy_multiplication_x: el7 :heavy_multiplication_x: el8 :heavy_multiplication_x: debian :heavy_multiplication_x: suse15. SL-JID 1744
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-975523076
Packaging result: :heavy_check_mark: el7 :heavy_multiplication_x: el8 :heavy_check_mark: debian :heavy_multiplication_x: suse15. SL-JID 1762
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r752111216
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -564,11 +657,352 @@ protected Point creteInfluxDbPoint(Object metricsObject) {
}
}
+ Gson gson;
+
+ class ManagementServerCollector extends AbstractStatsCollector {
+ @Override
+ protected void runInContext() {
+ LOGGER.debug(String.format("%s is running...", this.getClass().getSimpleName()));
+ long msid = ManagementServerNode.getManagementServerId();
+ ManagementServerHostVO mshost = null;
+ ManagementServerHostStatsEntry hostStatsEntry = null;
+ try {
+ mshost = managementServerHostDao.findByMsid(msid);
+ // get local data
+ hostStatsEntry = getDataFrom(mshost);
+ managementServerHostStats.put(mshost.getUuid(), hostStatsEntry);
+ // send to other hosts
+ clusterManager.publishStatus(gson.toJson(hostStatsEntry));
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to retrieve host stats", t);
+ }
+ try {
+ // send to DB
+ storeStatus(hostStatsEntry, mshost);
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to store host state", t);
+ }
+ }
+
+ private void storeStatus(ManagementServerHostStatsEntry hostStatsEntry, ManagementServerHostVO mshost) {
+ if (hostStatsEntry == null || mshost == null) {
+ return;
+ }
+ ManagementServerStatusVO msStats = managementServerStatusDao.findByMsId(hostStatsEntry.getManagementServerHostUuid());
+ if (msStats == null) {
+ LOGGER.info(String.format("creating new status info record for host %s - %s",
+ mshost.getName(),
+ hostStatsEntry.getManagementServerHostUuid()));
+ msStats = new ManagementServerStatusVO();
+ msStats.setMsId(hostStatsEntry.getManagementServerHostUuid());
+ }
+ msStats.setOsDistribution(hostStatsEntry.getOsDistribution()); // for now just the bunch details come later
+ msStats.setJavaName(hostStatsEntry.getJvmVendor());
+ msStats.setJavaVersion(hostStatsEntry.getJvmVersion());
+ Date startTime = new Date(hostStatsEntry.getStartTime());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("reporting starttime %s", startTime));
+ }
+ msStats.setLastStart(startTime);
+ msStats.setUpdated(new Date());
+ managementServerStatusDao.persist(msStats);
+ }
+
+ @NotNull
+ private ManagementServerHostStatsEntry getDataFrom(ManagementServerHostVO mshost) {
+ ManagementServerHostStatsEntry newEntry = new ManagementServerHostStatsEntry();
+ LOGGER.debug("Metrics collection start...");
+ newEntry.setManagementServerHostId(mshost.getId());
+ newEntry.setManagementServerHostUuid(mshost.getUuid());
+ retrieveSession(newEntry);
+ getJvmDimensions(newEntry);
+ LOGGER.debug("Metrics collection extra...");
+ getRuntimeData(newEntry);
+ getCpuData(newEntry);
+ getMemoryData(newEntry);
+ // newEntry must now include a pid!
+ getProcFsData(newEntry);
+ getFsData(newEntry);
+ getDataBaseStatistics(newEntry, mshost.getMsid());
+ gatherAllMetrics(newEntry);
+ LOGGER.debug("Metrics collection end!");
+ return newEntry;
+ }
+
+ private void retrieveSession(ManagementServerHostStatsEntry newEntry) {
+ long sessions = ApiSessionListener.getSessionCount();
+ newEntry.setSessions(sessions);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("Sessions found in Api %d vs context %d", sessions,ApiSessionListener.getNumberOfSessions()));
+ } else {
+ LOGGER.debug("Sessions active: " + sessions);
+ }
+ }
+
+ private void getDataBaseStatistics(ManagementServerHostStatsEntry newEntry, long msid) {
+ int count = _hostDao.countByMs(msid);
+ newEntry.setAgentCount(count);
+ }
+
+ private void getMemoryData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ MemoryMXBean mxBean = ManagementFactory.getMemoryMXBean();
+ newEntry.setTotalInit(mxBean.getHeapMemoryUsage().getInit() + mxBean.getNonHeapMemoryUsage().getInit());
+ newEntry.setTotalUsed(mxBean.getHeapMemoryUsage().getUsed() + mxBean.getNonHeapMemoryUsage().getUsed());
+ newEntry.setTotalMax(mxBean.getHeapMemoryUsage().getMax() + mxBean.getNonHeapMemoryUsage().getMax());
+ newEntry.setTotalCommitted(mxBean.getHeapMemoryUsage().getCommitted() + mxBean.getNonHeapMemoryUsage().getCommitted());
+ }
+
+ private void getCpuData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final OperatingSystemMXBean mxBean = ManagementFactory.getOperatingSystemMXBean();
+ newEntry.setAvailableProcessors(mxBean.getAvailableProcessors());
+ newEntry.setLoadAverage(mxBean.getSystemLoadAverage());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics processors - %d , loadavg - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getLoadAverage()));
+ }
+ }
+
+ private void getRuntimeData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final RuntimeMXBean mxBean = ManagementFactory.getRuntimeMXBean();
+ newEntry.setUptime(mxBean.getUptime());
+ newEntry.setStartTime(mxBean.getStartTime());
+ newEntry.setProcessId(mxBean.getPid());
+ newEntry.setJvmName(mxBean.getName());
+ newEntry.setJvmVendor(mxBean.getVmVendor());
+ newEntry.setJvmVersion(mxBean.getVmVersion());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics uptime - %d , starttime - %d",
+ newEntry.getUptime(),
+ newEntry.getStartTime()));
+ }
+ }
+
+ private void getJvmDimensions(@NotNull ManagementServerHostStatsEntry newEntry) {
+ Runtime runtime = Runtime.getRuntime();
+ newEntry.setTotalMemoryBytes(runtime.totalMemory());
+ newEntry.setFreeMemoryBytes(runtime.freeMemory());
+ newEntry.setAvailableProcessors(runtime.availableProcessors());
+ newEntry.setTotalMax(runtime.maxMemory());
+ //long maxMem = runtime.maxMemory();
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics proc - %d , maxMem - %d , totalMemory - %d , freeMemory - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getTotalMax(),
+ newEntry.getTotalMemoryBytes(),
+ newEntry.getFreeMemoryBytes()));
+ }
+ }
+
+ /**
+ * As for data from outside the JVM, we only rely on /proc/ contained data.
+ *
+ * @param newEntry
+ */
+ private void getProcFsData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ String OS = Script.runSimpleBashScript("cat /proc/version");
+ newEntry.setOsDistribution(OS);
+ String mem = Script.runSimpleBashScript("cat /proc/meminfo | grep MemTotal | cut -f 2 -d ':' | tr -d 'a-zA-z '").trim();
Review comment:
never mind, it does, gort confused by the naming
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1016292541
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1022871415
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
nvazquez commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1070175235
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1066543952
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1069213404
@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-962043397
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1677
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r775432559
##########
File path: ui/public/locales/en.json
##########
@@ -2164,6 +2180,13 @@
"label.system.vm.scaled.up": "System VM Scaled Up",
"label.system.vms": "System VMs",
"label.system.wide.capacity": "System-wide capacity",
+"label.systemcycleusage": "User-, System and Idle Cycles",
+"label.systemloadaverages": "1, 5 and 15 mintute load averages",
+"label.systemmemoryfree": "Free System Memory",
+"label.systemmemorytotal": "Total System Memory",
+"label.systemmemoryused": "Used System Memory",
+"label.systemmemoryvirtualsize": "Virtual Process Size (when all shared libraries are loaded)",
Review comment:
how about
```suggestion
"label.systemmemoryvirtualsize": "Virtual Total Process Size",
```
?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-995989685
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r777446595
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/response/DbMetricsResponse.java
##########
@@ -0,0 +1,107 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.response;
+
+import com.cloud.serializer.Param;
+import com.google.gson.annotations.SerializedName;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.api.BaseResponse;
+import org.apache.cloudstack.api.MetricConstants;
+
+import java.util.Date;
+
+public class DbMetricsResponse extends BaseResponse {
+ @SerializedName(MetricConstants.COLLECTION_TIME)
+ @Param(description = "the time these statistics were collected")
+ private Date collectionTime;
+
+ @SerializedName(ApiConstants.HOST_NAME)
+ @Param(description = "the name of the active usage server")
+ private String hostname;
+
+ @SerializedName(MetricConstants.REPLICAS)
+ @Param(description = "the state of the usage server")
+ private String[] replicas;
+
+ @SerializedName(MetricConstants.CONNECTIONS)
+ @Param(description = "the number of connections to the DB")
+ private int connections;
+
+ @SerializedName(MetricConstants.UPTIME)
+ @Param(description = "the uptime of the DB")
+ private int uptime;
+
+ @SerializedName(MetricConstants.TLS_VERSIONS)
+ @Param(description = "the tls versions currently in use (accepted) by the DB")
+ private String tlsVersions;
+
+ @SerializedName(ApiConstants.VERSION)
+ @Param(description = "the version of the currently running DB")
+ private String version;
+
+ @SerializedName(MetricConstants.VERSION_COMMENT)
+ @Param(description = "the version of the currently running DB")
+ private String versionComment;
+
+ @SerializedName(MetricConstants.QUERIES)
+ @Param(description = "the number of queroes performed on the DB")
Review comment:
```suggestion
@Param(description = "the number of queries performed on the DB")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1004047467
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1063770084
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1064883397
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 2840
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1028114410
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1028114734
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r777442782
##########
File path: api/src/main/java/com/cloud/server/ManagementServerHostStats.java
##########
@@ -0,0 +1,113 @@
+//
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+package com.cloud.server;
+
+import java.util.Date;
+
+/**
+ * management server related stats
+ */
+public interface ManagementServerHostStats {
+ Date getCollectionTime();
+
+ Date getSystemBootTime();
+
+ long getManagementServerHostId();
+
+ String getManagementServerHostUuid();
+
+ long getSessions();
+
+ double getCpuUtilization();
+
+ long getTotalMemoryBytes();
+
+ double getFreeMemoryBytes();
+
+ double getProcessMemoryBytes();
+
+ long getJvmUptime();
+
+ long getJvmStartTime();
+
+ int getAvailableProcessors();
+
+ double getLoadAverage();
+
+ long getTotalInit();
+
+ long getTotalUsed();
+
+ long getTotalMax();
+
+ long getTotalCommitted();
+
+ long getPid();
+
+ String getJvmName();
+
+ String getJvmVendor();
+
+ String getJvmVersion();
+
+ String getOsDistribution();
+
+ int getAgentCount();
+
+ long getHeapMemoryUsed();
+
+ long getHeapMemoryTotal();
+
+ int getThreadsBlockedCount();
+
+ int getThreadsTotalCount();
+
+ int getThreadsDaemonCount();
+
+ int getThreadsRunnableCount();
+
+ int getThreadsTerminatedCount();
+
+ int getThreadsWaitingCount();
+
+ long getSystemMemoryTotal();
+
+ long getSystemMemoryFree();
+
+ long getSystemMemoryUsed();
+
+ long getSystemMemoryVirtualSize();
+
+ String getLogInfo();
+
+ /**
+ * @return in mega herz
Review comment:
```suggestion
* @return in mega hertz
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-975523076
Packaging result: :heavy_check_mark: el7 :heavy_multiplication_x: el8 :heavy_check_mark: debian :heavy_multiplication_x: suse15. SL-JID 1762
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r733369133
##########
File path: framework/cluster/src/main/java/com/cloud/cluster/dao/ManagementServerStatusDaoImpl.java
##########
@@ -0,0 +1,50 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.cluster.dao;
+
+import com.cloud.cluster.ManagementServerStatusVO;
+import com.cloud.utils.db.GenericDaoBase;
+import com.cloud.utils.db.SearchBuilder;
+import com.cloud.utils.db.SearchCriteria;
+import org.apache.log4j.Logger;
+
+import java.util.List;
+
+public class ManagementServerStatusDaoImpl extends GenericDaoBase<ManagementServerStatusVO, Long> implements ManagementServerStatusDao {
+ private static final Logger s_logger = Logger.getLogger(ManagementServerHostDaoImpl.class);
+
+ private final SearchBuilder<ManagementServerStatusVO> MsIdSearch;
+
+ public ManagementServerStatusDaoImpl() {
+ MsIdSearch = createSearchBuilder();
+ MsIdSearch.and("msid", MsIdSearch.entity().getMsId(), SearchCriteria.Op.EQ);
+ MsIdSearch.done();
+ }
+
+ @Override
+ public ManagementServerStatusVO findByMsId(String msId) {
+ SearchCriteria<ManagementServerStatusVO> sc = MsIdSearch.create();
+ sc.setParameters("msid", msId);
+
+ List<ManagementServerStatusVO> l = listIncludingRemovedBy(sc);
+ if (l != null && l.size() > 0) {
Review comment:
copy and paste tech, but I will consider once further in the dev process
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] GutoVeronezi commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
GutoVeronezi commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r732980376
##########
File path: server/pom.xml
##########
@@ -170,6 +170,37 @@
<groupId>org.influxdb</groupId>
<artifactId>influxdb-java</artifactId>
</dependency>
+<!-- <dependency>-->
+<!-- <groupId>io.dropwizard.metrics</groupId>-->
+<!-- <artifactId>metrics-core</artifactId>-->
+<!-- <version>3.0.2</version>-->
+<!-- </dependency>-->
+<!-- <dependency>-->
+<!-- <groupId>io.dropwizard.metrics</groupId>-->
+<!-- <artifactId>metrics-jvm</artifactId>-->
+<!-- <version>3.0.2</version>-->
+<!-- </dependency>-->
Review comment:
Will these commented dependencies be needed?
##########
File path: framework/cluster/src/main/java/com/cloud/cluster/ManagementServerStatusVO.java
##########
@@ -0,0 +1,215 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.cluster;
+
+import com.cloud.utils.db.GenericDao;
+import org.apache.cloudstack.management.ManagementServerStatus;
+
+import javax.persistence.Column;
+import javax.persistence.Entity;
+import javax.persistence.GeneratedValue;
+import javax.persistence.GenerationType;
+import javax.persistence.Id;
+import javax.persistence.Table;
+import javax.persistence.Temporal;
+import javax.persistence.TemporalType;
+import java.util.Date;
+
+@Entity
+@Table(name = "mshost_status")
+public class ManagementServerStatusVO implements ManagementServerStatus {
+
+ @Id
+ @GeneratedValue(strategy = GenerationType.IDENTITY)
+ @Column(name = "id")
+ private long id;
+
+ @Column(name = "ms_id", updatable = true, nullable = false)
+ private long msId;
+
+ @Column(name="last_start")
+ private Date lastStart;
+
+ @Column(name="last_stop")
+ private Date lastStop;
+
+ @Column(name="last_boot")
+ private Date lastBoot;
+
+ @Column(name="last_down")
+ private Date lastDown;
+
+ @Column(name="os_name")
+ private String osName;
+
+ @Column(name="os_version")
+ private String osVersion;
+
+ @Column(name="java_name")
+ private String javaName;
+
+ @Column(name="java_version")
+ private String javaVersion;
+
+ @Temporal(TemporalType.TIMESTAMP)
+ @Column(name = "updated")
+ private Date updated;
+
+ @Column(name = GenericDao.CREATED_COLUMN)
+ private Date created;
+
+ @Column(name = GenericDao.REMOVED_COLUMN)
+ private Date removed;
+
+
+ public ManagementServerStatusVO() {
+ }
+
+ @Override
+ public long getId() {
+ return id;
+ }
+
+ public void setId(long id) {
+ this.id = id;
+ }
+
+ @Override
+ public long getMsId() {
+ return msId;
+ }
+
+ public void setMsId(long msId) {
+ this.msId = msId;
+ }
+
+ @Override
+ public Date getLastStart() {
+ return lastStart;
+ }
+
+ public void setLastStart(Date lastStart) {
+ this.lastStart = lastStart;
+ }
+
+ @Override
+ public Date getLastStop() {
+ return lastStop;
+ }
+
+ public void setLastStop(Date lastStop) {
+ this.lastStop = lastStop;
+ }
+
+ @Override
+ public Date getLastBoot() {
+ return lastBoot;
+ }
+
+ public void setLastBoot(Date lastBoot) {
+ this.lastBoot = lastBoot;
+ }
+
+ @Override
+ public Date getLastDown() {
+ return lastDown;
+ }
+
+ public void setLastDown(Date lastDown) {
+ this.lastDown = lastDown;
+ }
+
+ @Override
+ public String getOsName() {
+ return osName;
+ }
+
+ public void setOsName(String osName) {
+ this.osName = osName;
+ }
+
+ @Override
+ public String getOsVersion() {
+ return osVersion;
+ }
+
+ public void setOsVersion(String osVersion) {
+ this.osVersion = osVersion;
+ }
+
+ @Override
+ public String getJavaName() {
+ return javaName;
+ }
+
+ public void setJavaName(String javaName) {
+ this.javaName = javaName;
+ }
+
+ @Override
+ public String getJavaVersion() {
+ return javaVersion;
+ }
+
+ public void setJavaVersion(String javaVersion) {
+ this.javaVersion = javaVersion;
+ }
+
+ @Override
+ public Date getUpdated() {
+ return updated;
+ }
+
+ public void setUpdated(Date updated) {
+ this.updated = updated;
+ }
+
+ @Override
+ public Date getCreated() {
+ return created;
+ }
+
+ public void setCreated(Date created) {
+ this.created = created;
+ }
+
+ @Override
+ public Date getRemoved() {
+ return removed;
+ }
+
+ public void setRemoved(Date removedTime) {
+ removed = removedTime;
+ }
+
+ @Override
+ public String toString() {
+ return String.join("-",
Review comment:
We could use `ReflectionToStringBuilder` or `ReflectionToStringBuilder` here.
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -334,9 +363,39 @@ public StatsCollector() {
@Override
public boolean start() {
init(_configDao.getConfiguration());
+ registerAll("gc", new GarbageCollectorMetricSet(), registry);
+ registerAll("buffers", new BufferPoolMetricSet(ManagementFactory.getPlatformMBeanServer()), registry);
+ registerAll("memory", new MemoryUsageGaugeSet(), registry);
+ registerAll("threads", new ThreadStatesGaugeSet(), registry);
+ //registerAll("threads", new CpuMetricSet(), registry);
+ //registerAll("files", new FileDescriptorRatioGauge(), registry);
+ //registerAll("threads", new RuntimeMetricSet(), registry);
+ //registerAll("threads", new StandardSystemMetricSet(), registry);
Review comment:
Will these commented lines be needed?
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -148,6 +154,18 @@
import static com.cloud.utils.NumbersUtil.toHumanReadableSize;
import org.apache.commons.io.FileUtils;
+import com.codahale.metrics.Metric;
+import com.codahale.metrics.MetricRegistry;
+import com.codahale.metrics.MetricSet;
+import com.codahale.metrics.jvm.BufferPoolMetricSet;
+import com.codahale.metrics.jvm.GarbageCollectorMetricSet;
+import com.codahale.metrics.jvm.MemoryUsageGaugeSet;
+import com.codahale.metrics.jvm.ThreadStatesGaugeSet;
+
+//import com.codahale.metrics.jvm.CpuMetricSet;
+//import com.codahale.metrics.jvm.FileDescriptorRatioGauge;
+//import com.codahale.metrics.jvm.RuntimeMetricSet;
+//import com.codahale.metrics.jvm.StandardSystemMetricSet;
Review comment:
Will these commented imports be needed?
##########
File path: server/pom.xml
##########
@@ -170,6 +170,37 @@
<groupId>org.influxdb</groupId>
<artifactId>influxdb-java</artifactId>
</dependency>
+<!-- <dependency>-->
+<!-- <groupId>io.dropwizard.metrics</groupId>-->
+<!-- <artifactId>metrics-core</artifactId>-->
+<!-- <version>3.0.2</version>-->
+<!-- </dependency>-->
+<!-- <dependency>-->
+<!-- <groupId>io.dropwizard.metrics</groupId>-->
+<!-- <artifactId>metrics-jvm</artifactId>-->
+<!-- <version>3.0.2</version>-->
+<!-- </dependency>-->
+ <dependency>
+ <groupId>com.codahale.metrics</groupId>
+ <artifactId>metrics-core</artifactId>
+ <version>3.0.2</version>
+ </dependency>
+ <dependency>
+ <groupId>com.codahale.metrics</groupId>
+ <artifactId>metrics-jvm</artifactId>
+ <version>3.0.2</version>
+ </dependency>
+<!-- <dependency>-->
+<!-- <groupId>org.apache.cloudstack</groupId>-->
+<!-- <artifactId>cloud-framework-spring-module</artifactId>-->
+<!-- <version>4.14.0.0</version>-->
+<!-- <scope>compile</scope>-->
+<!-- </dependency>-->
+ <!-- <dependency>-->
+<!-- <groupId>com.codahale.metrics</groupId>-->
+<!-- <artifactId>metrics-parent</artifactId>-->
+<!-- <version>3.0.2</version>-->
+<!-- </dependency>-->
Review comment:
Will these commented dependencies be needed?
##########
File path: framework/cluster/src/main/java/com/cloud/cluster/dao/ManagementServerStatusDaoImpl.java
##########
@@ -0,0 +1,50 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.cluster.dao;
+
+import com.cloud.cluster.ManagementServerStatusVO;
+import com.cloud.utils.db.GenericDaoBase;
+import com.cloud.utils.db.SearchBuilder;
+import com.cloud.utils.db.SearchCriteria;
+import org.apache.log4j.Logger;
+
+import java.util.List;
+
+public class ManagementServerStatusDaoImpl extends GenericDaoBase<ManagementServerStatusVO, Long> implements ManagementServerStatusDao {
+ private static final Logger s_logger = Logger.getLogger(ManagementServerHostDaoImpl.class);
+
+ private final SearchBuilder<ManagementServerStatusVO> MsIdSearch;
+
+ public ManagementServerStatusDaoImpl() {
+ MsIdSearch = createSearchBuilder();
+ MsIdSearch.and("msid", MsIdSearch.entity().getMsId(), SearchCriteria.Op.EQ);
+ MsIdSearch.done();
+ }
+
+ @Override
+ public ManagementServerStatusVO findByMsId(String msId) {
+ SearchCriteria<ManagementServerStatusVO> sc = MsIdSearch.create();
+ sc.setParameters("msid", msId);
+
+ List<ManagementServerStatusVO> l = listIncludingRemovedBy(sc);
+ if (l != null && l.size() > 0) {
Review comment:
We could use `org.apache.commons.collections.CollectionUtils` here:
```suggestion
if (CollectionUtils.isNotEmpty(l)) {
```
Also, could we rename the var to a name more intuitive?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-998937208
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-995943847
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1917
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-997168260
see if jenkins has other ideas, now...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1022857629
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1023639284
Packaging result: :heavy_multiplication_x: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_multiplication_x: suse15. SL-JID 2352
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-972011376
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1721
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-972836786
Packaging result: :heavy_multiplication_x: el7 :heavy_multiplication_x: el8 :heavy_multiplication_x: debian :heavy_multiplication_x: suse15. SL-JID 1730
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r752074905
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -564,11 +657,352 @@ protected Point creteInfluxDbPoint(Object metricsObject) {
}
}
+ Gson gson;
+
+ class ManagementServerCollector extends AbstractStatsCollector {
+ @Override
+ protected void runInContext() {
+ LOGGER.debug(String.format("%s is running...", this.getClass().getSimpleName()));
+ long msid = ManagementServerNode.getManagementServerId();
+ ManagementServerHostVO mshost = null;
+ ManagementServerHostStatsEntry hostStatsEntry = null;
+ try {
+ mshost = managementServerHostDao.findByMsid(msid);
+ // get local data
+ hostStatsEntry = getDataFrom(mshost);
+ managementServerHostStats.put(mshost.getUuid(), hostStatsEntry);
+ // send to other hosts
+ clusterManager.publishStatus(gson.toJson(hostStatsEntry));
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to retrieve host stats", t);
+ }
+ try {
+ // send to DB
+ storeStatus(hostStatsEntry, mshost);
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to store host state", t);
+ }
+ }
+
+ private void storeStatus(ManagementServerHostStatsEntry hostStatsEntry, ManagementServerHostVO mshost) {
+ if (hostStatsEntry == null || mshost == null) {
+ return;
+ }
+ ManagementServerStatusVO msStats = managementServerStatusDao.findByMsId(hostStatsEntry.getManagementServerHostUuid());
+ if (msStats == null) {
+ LOGGER.info(String.format("creating new status info record for host %s - %s",
+ mshost.getName(),
+ hostStatsEntry.getManagementServerHostUuid()));
+ msStats = new ManagementServerStatusVO();
+ msStats.setMsId(hostStatsEntry.getManagementServerHostUuid());
+ }
+ msStats.setOsDistribution(hostStatsEntry.getOsDistribution()); // for now just the bunch details come later
+ msStats.setJavaName(hostStatsEntry.getJvmVendor());
+ msStats.setJavaVersion(hostStatsEntry.getJvmVersion());
+ Date startTime = new Date(hostStatsEntry.getStartTime());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("reporting starttime %s", startTime));
+ }
+ msStats.setLastStart(startTime);
+ msStats.setUpdated(new Date());
+ managementServerStatusDao.persist(msStats);
+ }
+
+ @NotNull
+ private ManagementServerHostStatsEntry getDataFrom(ManagementServerHostVO mshost) {
+ ManagementServerHostStatsEntry newEntry = new ManagementServerHostStatsEntry();
+ LOGGER.debug("Metrics collection start...");
+ newEntry.setManagementServerHostId(mshost.getId());
+ newEntry.setManagementServerHostUuid(mshost.getUuid());
+ retrieveSession(newEntry);
+ getJvmDimensions(newEntry);
+ LOGGER.debug("Metrics collection extra...");
+ getRuntimeData(newEntry);
+ getCpuData(newEntry);
+ getMemoryData(newEntry);
+ // newEntry must now include a pid!
+ getProcFsData(newEntry);
+ getFsData(newEntry);
+ getDataBaseStatistics(newEntry, mshost.getMsid());
+ gatherAllMetrics(newEntry);
+ LOGGER.debug("Metrics collection end!");
+ return newEntry;
+ }
+
+ private void retrieveSession(ManagementServerHostStatsEntry newEntry) {
+ long sessions = ApiSessionListener.getSessionCount();
+ newEntry.setSessions(sessions);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("Sessions found in Api %d vs context %d", sessions,ApiSessionListener.getNumberOfSessions()));
+ } else {
+ LOGGER.debug("Sessions active: " + sessions);
+ }
+ }
+
+ private void getDataBaseStatistics(ManagementServerHostStatsEntry newEntry, long msid) {
+ int count = _hostDao.countByMs(msid);
+ newEntry.setAgentCount(count);
+ }
+
+ private void getMemoryData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ MemoryMXBean mxBean = ManagementFactory.getMemoryMXBean();
+ newEntry.setTotalInit(mxBean.getHeapMemoryUsage().getInit() + mxBean.getNonHeapMemoryUsage().getInit());
+ newEntry.setTotalUsed(mxBean.getHeapMemoryUsage().getUsed() + mxBean.getNonHeapMemoryUsage().getUsed());
+ newEntry.setTotalMax(mxBean.getHeapMemoryUsage().getMax() + mxBean.getNonHeapMemoryUsage().getMax());
+ newEntry.setTotalCommitted(mxBean.getHeapMemoryUsage().getCommitted() + mxBean.getNonHeapMemoryUsage().getCommitted());
+ }
+
+ private void getCpuData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final OperatingSystemMXBean mxBean = ManagementFactory.getOperatingSystemMXBean();
+ newEntry.setAvailableProcessors(mxBean.getAvailableProcessors());
+ newEntry.setLoadAverage(mxBean.getSystemLoadAverage());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics processors - %d , loadavg - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getLoadAverage()));
+ }
+ }
+
+ private void getRuntimeData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final RuntimeMXBean mxBean = ManagementFactory.getRuntimeMXBean();
+ newEntry.setUptime(mxBean.getUptime());
+ newEntry.setStartTime(mxBean.getStartTime());
+ newEntry.setProcessId(mxBean.getPid());
+ newEntry.setJvmName(mxBean.getName());
+ newEntry.setJvmVendor(mxBean.getVmVendor());
+ newEntry.setJvmVersion(mxBean.getVmVersion());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics uptime - %d , starttime - %d",
+ newEntry.getUptime(),
+ newEntry.getStartTime()));
+ }
+ }
+
+ private void getJvmDimensions(@NotNull ManagementServerHostStatsEntry newEntry) {
+ Runtime runtime = Runtime.getRuntime();
+ newEntry.setTotalMemoryBytes(runtime.totalMemory());
+ newEntry.setFreeMemoryBytes(runtime.freeMemory());
+ newEntry.setAvailableProcessors(runtime.availableProcessors());
+ newEntry.setTotalMax(runtime.maxMemory());
+ //long maxMem = runtime.maxMemory();
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics proc - %d , maxMem - %d , totalMemory - %d , freeMemory - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getTotalMax(),
+ newEntry.getTotalMemoryBytes(),
+ newEntry.getFreeMemoryBytes()));
+ }
+ }
+
+ /**
+ * As for data from outside the JVM, we only rely on /proc/ contained data.
+ *
+ * @param newEntry
+ */
+ private void getProcFsData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ String OS = Script.runSimpleBashScript("cat /proc/version");
+ newEntry.setOsDistribution(OS);
+ String mem = Script.runSimpleBashScript("cat /proc/meminfo | grep MemTotal | cut -f 2 -d ':' | tr -d 'a-zA-z '").trim();
+ newEntry.setSystemMemoryTotal(Long.parseLong(mem) * 1024);
+ String free = Script.runSimpleBashScript("cat /proc/meminfo | grep MemFree | cut -f 2 -d ':' | tr -d 'a-zA-z '").trim();
+ newEntry.setSystemMemoryFree(Long.parseLong(free) * 1024);
+ String used = Script.runSimpleBashScript(String.format("ps -o rss= %d", newEntry.getPid()));
+ newEntry.setSystemMemoryUsed(Long.parseLong(used));
+ String maxuse = Script.runSimpleBashScript(String.format("ps -o vsz= %d", newEntry.getPid()));
+ newEntry.setSystemMemoryVirtualSize(Long.parseLong(maxuse));
+
+ newEntry.setSystemTotalCpuCycles(getSystemCpuCyclesTotal());
+ newEntry.setSystemLoadAverages(getCpuLoads());
+ newEntry.setSystemCyclesUsage(getSystemCpuUsage(newEntry));
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.debug(
+ String.format("cpu\ncapacities: %f\n loads: %s ; %s ; %s\n stats: %f ; %f ; %f",
+ newEntry.getSystemTotalCpuCycles(),
+ newEntry.getSystemLoadAverages()[0], newEntry.getSystemLoadAverages()[1], newEntry.getSystemLoadAverages()[2],
+ newEntry.getSystemCyclesUsage()[0], newEntry.getSystemCyclesUsage()[1], newEntry.getSystemCyclesUsage()[2]
+ )
+ );
+ }
+ }
+
+ @NotNull
+ private double[] getCpuLoads() {
+ String[] cpuloadString = Script.runSimpleBashScript("cat /proc/loadavg").split(" ");
+ double[] cpuloads = {Double.parseDouble(cpuloadString[0]), Double.parseDouble(cpuloadString[1]), Double.parseDouble(cpuloadString[2])};
+ return cpuloads;
+ }
+
+ private double [] getSystemCpuUsage(@NotNull ManagementServerHostStatsEntry newEntry) {
+ String[] cpustats = Script.runSimpleBashScript("cat /proc/stat | grep \"cpu \" | tr -d \"cpu\"").trim().split(" ");
+ double [] cycleUsage = {Double.parseDouble(cpustats[0]) + Double.parseDouble(cpustats[1]), Double.parseDouble(cpustats[2]), Double.parseDouble(cpustats[3])};
+ return cycleUsage;
+ }
+
+ private double getSystemCpuCyclesTotal() {
+ String cpucaps = Script.runSimpleBashScript("cat /proc/cpuinfo | grep \"cpu MHz\" | grep \"cpu MHz\" | cut -f 2 -d : | tr -d ' '| tr '\\n' \" \"");
+ double totalcpucap = 0;
+ for (String cpucap : cpucaps.split(" ")) {
+ totalcpucap += Double.parseDouble(cpucap);
+ }
+ return totalcpucap;
+ }
+
+ private void getFsData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ Set<String> logFileNames = LogUtils.getLogFileNames();
+ StringBuilder logInfoBuilder = new StringBuilder();
+ for (String fileName : logFileNames) {
+ String du = Script.runSimpleBashScript(String.format("du -sk %s", fileName));
+ String df = Script.runSimpleBashScript(String.format("df -k %s | grep -v Filesystem", fileName));
+ logInfoBuilder.append(fileName).append('\n').append("usage: ").append(du).append('\n').append("disk :").append(df);
Review comment:
ah, sorry, I didn't understand, will do
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-949698667
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r736577034
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -122,16 +133,16 @@ private Double findRatioValue(final String value) {
return 1.0;
}
- private void updateHostMetrics(final Metrics metrics, final HostJoinVO host) {
- metrics.incrTotalHosts();
- metrics.addCpuAllocated(host.getCpuReservedCapacity() + host.getCpuUsedCapacity());
- metrics.addMemoryAllocated(host.getMemReservedCapacity() + host.getMemUsedCapacity());
+ private void updateHostMetrics(final org.apache.cloudstack.metrics.MetricsServiceImpl.HostMetrics hostMetrics, final HostJoinVO host) {
Review comment:
no @GutoVeronezi , thanks for pointing it out. It is an issue with my IDE, that has been haunting me since short. I'll fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r736578289
##########
File path: framework/cluster/src/main/java/com/cloud/cluster/ClusterManagerImpl.java
##########
@@ -1141,7 +1183,10 @@ private boolean pingManagementNode(final ManagementServerHostVO mshost) {
return true;
} catch (final IOException e) {
if (e instanceof ConnectException) {
- s_logger.error("Unable to ping management server at " + targetIp + ":" + mshost.getServicePort() + " due to ConnectException", e);
+ s_logger.error("Unable to ping management server at " + targetIp + ":" + mshost.getServicePort() + " due to ConnectException");
+ if (s_logger.isDebugEnabled()) {
+ s_logger.debug("Unable to ping management server at " + targetIp + ":" + mshost.getServicePort() + " due to ConnectException", e);
Review comment:
@GutoVeronezi I prefer to keep stacktraces at `debug` level and just clear messages at `error`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-982709184
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1803
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-983462313
Packaging result: :heavy_multiplication_x: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1808
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1011176357
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland closed pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland closed pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1011231761
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 2186
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r774438133
##########
File path: server/src/main/java/com/cloud/server/StatsCollection.java
##########
@@ -0,0 +1,30 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.server;
+
+public interface StatsCollection {
Review comment:
are these only DB stats?
```suggestion
public interface DBStatsCollection {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-998844371
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r761923724
##########
File path: engine/schema/src/main/resources/META-INF/db/schema-41610to41700.sql
##########
@@ -21,3 +21,50 @@
-- PR#5668 Change the type of the 'ipsec_psk' field to allow large PSK.
ALTER TABLE cloud.remote_access_vpn MODIFY ipsec_psk text NOT NULL;
+
+-- Management Server Status
+ALTER TABLE `cloud`.`mshost` ADD CONSTRAINT `mshost_UUID` UNIQUE KEY (`uuid`);
+CREATE TABLE `cloud`.`mshost_status` (
+ `id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
+ `ms_id` varchar(40) DEFAULT NULL COMMENT 'the uuid of the management server record',
+ `last_start` datetime DEFAULT NULL COMMENT 'the last start time for this MS',
+ `last_stop` datetime DEFAULT NULL COMMENT 'the last stop time for this MS',
+ `last_boot` datetime DEFAULT NULL COMMENT 'the last system boot time for the host of this MS',
+ `last_down` datetime DEFAULT NULL COMMENT 'the last system shutdown time for the host of this MS',
Review comment:
_last_system_down_ or _last_system_shutdown_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r761937272
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -564,11 +678,471 @@ protected Point creteInfluxDbPoint(Object metricsObject) {
}
}
+ class DbCollector extends AbstractStatsCollector {
+ List<Double> loadHistory = new ArrayList<>();
+ DbCollector() {
+ dbStats.put(loadAvarages, loadHistory);
+ }
+ @Override
+ protected void runInContext() {
+ LOGGER.debug(String.format("%s is running...", this.getClass().getSimpleName()));
+
+ try {
+ int lastUptime = (dbStats.containsKey(uptime) ? (Integer) dbStats.get(uptime) : 0);
+ int lastQueries = (dbStats.containsKey(queries) ? (Integer) dbStats.get(queries) : 0);
+ getDynamicDataFromDB();
+ int interval = (Integer) dbStats.get(uptime) - lastUptime;
+ int activity = (Integer) dbStats.get(queries) - lastQueries;
+ loadHistory.add(Double.valueOf(activity / interval));
+ while (loadHistory.size() > DATABASE_SERVER_LOAD_HISTORY_RETENTION_NUMBER.value()) {
+ loadHistory.remove(0);
+ }
+ } catch (Throwable e) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("db statistics collection failed due to " + e.getLocalizedMessage());
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("db statistics collection failed.", e);
+ }
+ }
+ }
+
+ private void getDynamicDataFromDB() {
+ Map<String, String> stats = DbUtil.getDbInfo("STATUS", queries, uptime);
+ dbStats.put(queries, (Integer.valueOf(stats.get(queries))));
+ dbStats.put(uptime, (Integer.valueOf(stats.get(uptime))));
+ }
+
+
+ @Override
+ protected Point creteInfluxDbPoint(Object metricsObject) {
+ return null;
+ }
+ }
+
+ Gson gson;
+
+ class ManagementServerCollector extends AbstractStatsCollector {
+ @Override
+ protected void runInContext() {
+ LOGGER.debug(String.format("%s is running...", this.getClass().getSimpleName()));
+ long msid = ManagementServerNode.getManagementServerId();
+ ManagementServerHostVO mshost = null;
+ ManagementServerHostStatsEntry hostStatsEntry = null;
+ try {
+ mshost = managementServerHostDao.findByMsid(msid);
+ // get local data
+ hostStatsEntry = getDataFrom(mshost);
+ managementServerHostStats.put(mshost.getUuid(), hostStatsEntry);
+ // send to other hosts
+ clusterManager.publishStatus(gson.toJson(hostStatsEntry));
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to retrieve management server host statistics", t);
+ }
+ try {
+ // send to DB
+ storeStatus(hostStatsEntry, mshost);
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to store management server host statistics", t);
+ }
+ }
+
+ private void storeStatus(ManagementServerHostStatsEntry hostStatsEntry, ManagementServerHostVO mshost) {
+ if (hostStatsEntry == null || mshost == null) {
+ return;
+ }
+ ManagementServerStatusVO msStats = managementServerStatusDao.findByMsId(hostStatsEntry.getManagementServerHostUuid());
+ if (msStats == null) {
+ LOGGER.info(String.format("creating new status info record for host %s - %s",
+ mshost.getName(),
+ hostStatsEntry.getManagementServerHostUuid()));
+ msStats = new ManagementServerStatusVO();
+ msStats.setMsId(hostStatsEntry.getManagementServerHostUuid());
+ }
+ msStats.setOsDistribution(hostStatsEntry.getOsDistribution()); // for now just the bunch details come later
+ msStats.setJavaName(hostStatsEntry.getJvmVendor());
+ msStats.setJavaVersion(hostStatsEntry.getJvmVersion());
+ Date startTime = new Date(hostStatsEntry.getStartTime());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("reporting starttime %s", startTime));
+ }
+ msStats.setLastStart(startTime);
+ msStats.setUpdated(new Date());
+ managementServerStatusDao.persist(msStats);
+ }
+
+ @NotNull
+ private ManagementServerHostStatsEntry getDataFrom(ManagementServerHostVO mshost) {
+ ManagementServerHostStatsEntry newEntry = new ManagementServerHostStatsEntry();
+ LOGGER.trace("Metrics collection start...");
+ newEntry.setManagementServerHostId(mshost.getId());
+ newEntry.setManagementServerHostUuid(mshost.getUuid());
+ newEntry.setDbLocal(isDbLocal());
+ newEntry.setUsageLocal(isUsageLocal());
+ retrieveSession(newEntry);
+ getJvmDimensions(newEntry);
+ LOGGER.trace("Metrics collection extra...");
+ getRuntimeData(newEntry);
+ getCpuData(newEntry);
+ getMemoryData(newEntry);
+ // newEntry must now include a pid!
+ getProcFsData(newEntry);
+ getFsData(newEntry);
+ getDataBaseStatistics(newEntry, mshost.getMsid());
+ gatherAllMetrics(newEntry);
+ LOGGER.trace("Metrics collection end!");
+ return newEntry;
+ }
+
+ private void retrieveSession(ManagementServerHostStatsEntry newEntry) {
+ long sessions = ApiSessionListener.getSessionCount();
+ newEntry.setSessions(sessions);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("Sessions found in Api %d vs context %d", sessions,ApiSessionListener.getNumberOfSessions()));
+ } else {
+ LOGGER.debug("Sessions active: " + sessions);
+ }
+ }
+
+ private void getDataBaseStatistics(ManagementServerHostStatsEntry newEntry, long msid) {
+ int count = _hostDao.countByMs(msid);
+ newEntry.setAgentCount(count);
+ }
+
+ private void getMemoryData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ MemoryMXBean mxBean = ManagementFactory.getMemoryMXBean();
+ newEntry.setTotalInit(mxBean.getHeapMemoryUsage().getInit() + mxBean.getNonHeapMemoryUsage().getInit());
+ newEntry.setTotalUsed(mxBean.getHeapMemoryUsage().getUsed() + mxBean.getNonHeapMemoryUsage().getUsed());
+ newEntry.setTotalMax(mxBean.getHeapMemoryUsage().getMax() + mxBean.getNonHeapMemoryUsage().getMax());
+ newEntry.setTotalCommitted(mxBean.getHeapMemoryUsage().getCommitted() + mxBean.getNonHeapMemoryUsage().getCommitted());
+ }
+
+ private void getCpuData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ java.lang.management.OperatingSystemMXBean bean = ManagementFactory.getOperatingSystemMXBean();
+ newEntry.setAvailableProcessors(bean.getAvailableProcessors());
+ newEntry.setLoadAverage(bean.getSystemLoadAverage());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics processors - %d , loadavg - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getLoadAverage()));
+ }
+ if (bean instanceof OperatingSystemMXBean) {
+ OperatingSystemMXBean mxBean = (OperatingSystemMXBean) bean;
+ newEntry.setSystemMemoryTotal(mxBean.getTotalPhysicalMemorySize());
+ newEntry.setSystemMemoryFree(mxBean.getFreePhysicalMemorySize());
+ newEntry.setSystemMemoryUsed(mxBean.getCommittedVirtualMemorySize());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("data from 'OperatingSystemMXBean': total mem: %d, free mem: %d, used mem: %d",
+ newEntry.getSystemMemoryTotal(),
+ newEntry.getSystemMemoryFree(),
+ newEntry.getSystemMemoryUsed()));
+ }
+ }
+ }
+
+ private void getRuntimeData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final RuntimeMXBean mxBean = ManagementFactory.getRuntimeMXBean();
+ newEntry.setUptime(mxBean.getUptime());
+ newEntry.setStartTime(mxBean.getStartTime());
+ newEntry.setProcessId(mxBean.getPid());
+ newEntry.setJvmName(mxBean.getName());
+ newEntry.setJvmVendor(mxBean.getVmVendor());
+ newEntry.setJvmVersion(mxBean.getVmVersion());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics uptime - %d , starttime - %d",
+ newEntry.getUptime(),
+ newEntry.getStartTime()));
+ }
+ }
+
+ private void getJvmDimensions(@NotNull ManagementServerHostStatsEntry newEntry) {
+ Runtime runtime = Runtime.getRuntime();
+ newEntry.setTotalMemoryBytes(runtime.totalMemory());
+ newEntry.setFreeMemoryBytes(runtime.freeMemory());
+ newEntry.setAvailableProcessors(runtime.availableProcessors());
+ newEntry.setTotalMax(runtime.maxMemory());
+ //long maxMem = runtime.maxMemory();
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics proc - %d , maxMem - %d , totalMemory - %d , freeMemory - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getTotalMax(),
+ newEntry.getTotalMemoryBytes(),
+ newEntry.getFreeMemoryBytes()));
+ }
+ }
+
+ /**
+ * As for data from outside the JVM, we only rely on /proc/ contained data.
+ *
+ * @param newEntry
+ */
+ private void getProcFsData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ String OS = Script.runSimpleBashScript("cat /proc/version");
+ newEntry.setOsDistribution(OS);
+ // if we got these from the bean, skip
+ if (newEntry.getSystemMemoryTotal() == 0) {
+ String mem = Script.runSimpleBashScript("cat /proc/meminfo | grep MemTotal | cut -f 2 -d ':' | tr -d 'a-zA-z '").trim();
+ newEntry.setSystemMemoryTotal(Long.parseLong(mem) * 1024);
+ LOGGER.info(String.format("system memory from /proc: %d", newEntry.getSystemMemoryTotal()));
+ }
+ if (newEntry.getSystemMemoryFree() == 0) {
+ String free = Script.runSimpleBashScript("cat /proc/meminfo | grep MemFree | cut -f 2 -d ':' | tr -d 'a-zA-z '").trim();
+ newEntry.setSystemMemoryFree(Long.parseLong(free) * 1024);
+ LOGGER.info(String.format("free memory from /proc: %d", newEntry.getSystemMemoryFree()));
+ }
+ if (newEntry.getSystemMemoryUsed() <= 0) {
+ String used = Script.runSimpleBashScript(String.format("ps -o rss= %d", newEntry.getPid()));
+ newEntry.setSystemMemoryUsed(Long.parseLong(used));
+ LOGGER.info(String.format("used memory from /proc: %d", newEntry.getSystemMemoryUsed()));
+ }
+ String maxuse = Script.runSimpleBashScript(String.format("ps -o vsz= %d", newEntry.getPid()));
+ newEntry.setSystemMemoryVirtualSize(Long.parseLong(maxuse));
+
+ newEntry.setSystemTotalCpuCycles(getSystemCpuCyclesTotal());
+ newEntry.setSystemLoadAverages(getCpuLoads());
+ newEntry.setSystemCyclesUsage(getSystemCpuUsage(newEntry));
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(
+ String.format("cpu\ncapacities: %f\n loads: %s ; %s ; %s\n stats: %f ; %f ; %f",
+ newEntry.getSystemTotalCpuCycles(),
+ newEntry.getSystemLoadAverages()[0], newEntry.getSystemLoadAverages()[1], newEntry.getSystemLoadAverages()[2],
+ newEntry.getSystemCyclesUsage()[0], newEntry.getSystemCyclesUsage()[1], newEntry.getSystemCyclesUsage()[2]
+ )
+ );
+ }
+ }
+
+ @NotNull
+ private double[] getCpuLoads() {
+ String[] cpuloadString = Script.runSimpleBashScript("cat /proc/loadavg").split(" ");
+ double[] cpuloads = {Double.parseDouble(cpuloadString[0]), Double.parseDouble(cpuloadString[1]), Double.parseDouble(cpuloadString[2])};
+ return cpuloads;
+ }
+
+ private double [] getSystemCpuUsage(@NotNull ManagementServerHostStatsEntry newEntry) {
+ String[] cpustats = Script.runSimpleBashScript("cat /proc/stat | grep \"cpu \" | tr -d \"cpu\"").trim().split(" ");
+ double [] cycleUsage = {Double.parseDouble(cpustats[0]) + Double.parseDouble(cpustats[1]), Double.parseDouble(cpustats[2]), Double.parseDouble(cpustats[3])};
+ return cycleUsage;
+ }
+
+ private double getSystemCpuCyclesTotal() {
+ String cpucaps = Script.runSimpleBashScript("cat /proc/cpuinfo | grep \"cpu MHz\" | grep \"cpu MHz\" | cut -f 2 -d : | tr -d ' '| tr '\\n' \" \"");
+ double totalcpucap = 0;
+ for (String cpucap : cpucaps.split(" ")) {
+ totalcpucap += Double.parseDouble(cpucap);
+ }
+ return totalcpucap;
+ }
+
+ private void getFsData(@NotNull ManagementServerHostStatsEntry newEntry) {
Review comment:
```suggestion
private void getFileSystemData(@NotNull ManagementServerHostStatsEntry newEntry) {
```
If this is specifically for logs, getFileSystemUsageDataForLogFiles
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1082463055
<b>Trillian test result (tid-3753)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 31444 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5588-t3753-kvm-centos7.zip
Smoke tests completed. 91 look OK, 1 have errors
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_hostha_enable_ha_when_host_disabled | `Error` | 1.89 | test_hostha_kvm.py
test_hostha_enable_ha_when_host_in_maintenance | `Error` | 301.02 | test_hostha_kvm.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1067981012
@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1064883397
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 2840
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1069268484
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1067981012
@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] borisstoyanov removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
borisstoyanov removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1060665590
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1069171289
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
nvazquez removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1070175235
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1056771979
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 2746
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] RodrigoDLopez commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
RodrigoDLopez commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r778777603
##########
File path: framework/cluster/src/main/java/com/cloud/cluster/ClusterManager.java
##########
@@ -53,19 +62,44 @@
void registerDispatcher(Dispatcher dispatcher);
+ /**
+ * register a listener for incoming status changes of ManagementServers
+ *
+ * @param administrator the object administrating statusses
Review comment:
I can swear I saw something like statusses yesterday.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r782311347
##########
File path: engine/schema/src/main/java/com/cloud/host/dao/HostDao.java
##########
@@ -142,4 +142,11 @@
HostVO findByName(String name);
List<HostVO> listHostsWithActiveVMs(long offeringId);
+
+ /**
+ * get the number of hosts/agents this {@see ManagementServer} has responsibility over
+ * @param msid the id of the {@see ManagementServer}
+ * @return the number of hosts/agents this {@see ManagementServer} has responsibility over
+ */
+ int countByMs(long msid);
Review comment:
I've taken over this name from somewhere and have no preference, also the lowercase `msid` is on the API as opposed to the camelcase `msId` so I'll leave as is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1001722568
@sureshanaparti a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1001606758
@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-999841455
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-962043397
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1677
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1070177845
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1064856724
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1056715357
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1004047467
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1028213078
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] GutoVeronezi commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
GutoVeronezi commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r792727079
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java
##########
@@ -335,61 +362,144 @@ public InfrastructureResponse listInfrastructure() {
}
final Long clusterId = cluster.getId();
- // Thresholds
- final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
- final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
- final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
- final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
-
// CPU and memory capacities
final CapacityDaoImpl.SummedCapacity cpuCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_CPU, null, clusterId);
final CapacityDaoImpl.SummedCapacity memoryCapacity = getCapacity((int) Capacity.CAPACITY_TYPE_MEMORY, null, clusterId);
- final Metrics metrics = new Metrics(cpuCapacity, memoryCapacity);
+ final HostMetrics hostMetrics = new HostMetrics(cpuCapacity, memoryCapacity);
for (final Host host: hostDao.findByClusterId(clusterId)) {
if (host == null || host.getType() != Host.Type.Routing) {
continue;
}
if (host.getStatus() == Status.Up) {
- metrics.incrUpResources();
+ hostMetrics.incrUpResources();
}
- metrics.incrTotalResources();
- updateHostMetrics(metrics, hostJoinDao.findById(host.getId()));
+ hostMetrics.incrTotalResources();
+ updateHostMetrics(hostMetrics, hostJoinDao.findById(host.getId()));
}
metricsResponse.setState(clusterResponse.getAllocationState(), clusterResponse.getManagedState());
- metricsResponse.setResources(metrics.getUpResources(), metrics.getTotalResources());
- // CPU
- metricsResponse.setCpuTotal(metrics.getTotalCpu());
- metricsResponse.setCpuAllocated(metrics.getCpuAllocated(), metrics.getTotalCpu());
- if (metrics.getCpuUsedPercentage() > 0L) {
- metricsResponse.setCpuUsed(metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
- metricsResponse.setCpuMaxDeviation(metrics.getMaximumCpuUsage(), metrics.getCpuUsedPercentage(), metrics.getTotalHosts());
+ metricsResponse.setResources(hostMetrics.getUpResources(), hostMetrics.getTotalResources());
+ // add CPU and Memory metrics
+ addHostCpuMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+ addHostMemoryMetricsToResponse(metricsResponse, clusterId, hostMetrics);
+
+ metricsResponse.setHasAnnotation(clusterResponse.hasAnnotation());
+ metricsResponses.add(metricsResponse);
+ }
+ return metricsResponses;
+ }
+
+ private void addHostMemoryMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setMemTotal(hostMetrics.getTotalMemory());
+ metricsResponse.setMemAllocated(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory());
+ if (hostMetrics.getMemoryUsed() > 0L) {
+ metricsResponse.setMemUsed(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory());
+ metricsResponse.setMemMaxDeviation(hostMetrics.getMaximumMemoryUsage(), hostMetrics.getMemoryUsed(), hostMetrics.getTotalHosts());
+ }
+ // Memory thresholds
+ final Double memoryThreshold = AlertManager.MemoryCapacityThreshold.valueIn(clusterId);
+ final Float memoryDisableThreshold = DeploymentClusterPlanner.ClusterMemoryCapacityDisableThreshold.valueIn(clusterId);
+ metricsResponse.setMemoryUsageThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryUsageDisableThreshold(hostMetrics.getMemoryUsed(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ metricsResponse.setMemoryAllocatedThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryThreshold);
+ metricsResponse.setMemoryAllocatedDisableThreshold(hostMetrics.getMemoryAllocated(), hostMetrics.getTotalMemory(), memoryDisableThreshold);
+ }
+
+ private void addHostCpuMetricsToResponse(ClusterMetricsResponse metricsResponse, Long clusterId, HostMetrics hostMetrics) {
+ metricsResponse.setCpuTotal(hostMetrics.getTotalCpu());
+ metricsResponse.setCpuAllocated(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu());
+ if (hostMetrics.getCpuUsedPercentage() > 0L) {
+ metricsResponse.setCpuUsed(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ metricsResponse.setCpuMaxDeviation(hostMetrics.getMaximumCpuUsage(), hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts());
+ }
+ // CPU thresholds
+ final Double cpuThreshold = AlertManager.CPUCapacityThreshold.valueIn(clusterId);
+ final Float cpuDisableThreshold = DeploymentClusterPlanner.ClusterCPUCapacityDisableThreshold.valueIn(clusterId);
+ metricsResponse.setCpuUsageThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuThreshold);
+ metricsResponse.setCpuUsageDisableThreshold(hostMetrics.getCpuUsedPercentage(), hostMetrics.getTotalHosts(), cpuDisableThreshold);
+ metricsResponse.setCpuAllocatedThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuThreshold);
+ metricsResponse.setCpuAllocatedDisableThreshold(hostMetrics.getCpuAllocated(), hostMetrics.getTotalCpu(), cpuDisableThreshold);
+ }
+
+
+ @Override
+ public List<ManagementServerMetricsResponse> listManagementServerMetrics(List<ManagementServerResponse> managementServerResponses) {
+ final List<ManagementServerMetricsResponse> metricsResponses = new ArrayList<>();
+ if(LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("getting metrics for %d MS hosts", managementServerResponses.size()));
+ }
+ for (final ManagementServerResponse managementServerResponse: managementServerResponses) {
+ if(LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("processing metrics for MS hosts %s", managementServerResponse.getId()));
}
- // Memory
- metricsResponse.setMemTotal(metrics.getTotalMemory());
- metricsResponse.setMemAllocated(metrics.getMemoryAllocated(), metrics.getTotalMemory());
- if (metrics.getMemoryUsed() > 0L) {
- metricsResponse.setMemUsed(metrics.getMemoryUsed(), metrics.getTotalMemory());
- metricsResponse.setMemMaxDeviation(metrics.getMaximumMemoryUsage(), metrics.getMemoryUsed(), metrics.getTotalHosts());
+ ManagementServerMetricsResponse metricsResponse = new ManagementServerMetricsResponse();
+
+ try {
+ BeanUtils.copyProperties(metricsResponse, managementServerResponse);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("bean copy result %s", new ReflectionToStringBuilder(metricsResponse, ToStringStyle.SIMPLE_STYLE).toString()));
+ }
+ } catch (IllegalAccessException | InvocationTargetException e) {
+ throw new ServerApiException(ApiErrorCode.INTERNAL_ERROR, "Failed to generate zone metrics response");
}
- // CPU thresholds
- metricsResponse.setCpuUsageThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuThreshold);
- metricsResponse.setCpuUsageDisableThreshold(metrics.getCpuUsedPercentage(), metrics.getTotalHosts(), cpuDisableThreshold);
- metricsResponse.setCpuAllocatedThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuThreshold);
- metricsResponse.setCpuAllocatedDisableThreshold(metrics.getCpuAllocated(), metrics.getTotalCpu(), cpuDisableThreshold);
- // Memory thresholds
- metricsResponse.setMemoryUsageThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryUsageDisableThreshold(metrics.getMemoryUsed(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setMemoryAllocatedThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryThreshold);
- metricsResponse.setMemoryAllocatedDisableThreshold(metrics.getMemoryAllocated(), metrics.getTotalMemory(), memoryDisableThreshold);
- metricsResponse.setHasAnnotation(clusterResponse.hasAnnotation());
+ updateManagementServerMetrics(metricsResponse, managementServerResponse);
+
metricsResponses.add(metricsResponse);
}
return metricsResponses;
}
+ /**
+ * get the transient/in memory data
+ * @param metricsResponse
+ * @param managementServerResponse
+ */
+ private void updateManagementServerMetrics(ManagementServerMetricsResponse metricsResponse, ManagementServerResponse managementServerResponse) {
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("getting stats for %s", managementServerResponse.getId()));
Review comment:
I think adding the name would be enough for this one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] GutoVeronezi commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
GutoVeronezi commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r792724664
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/api/ListMgmtsMetricsCmd.java
##########
@@ -0,0 +1,56 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.cloudstack.api;
+
+import org.apache.cloudstack.acl.RoleType;
+import org.apache.cloudstack.api.command.admin.management.ListMgmtsCmd;
+import org.apache.cloudstack.api.response.ListResponse;
+import org.apache.cloudstack.api.response.ManagementServerResponse;
+import org.apache.cloudstack.metrics.MetricsService;
+import org.apache.cloudstack.response.ManagementServerMetricsResponse;
+
+import javax.inject.Inject;
+import java.util.List;
+
+@APICommand(name = ListMgmtsMetricsCmd.APINAME, description = "Lists Management Server metrics", responseObject = ManagementServerMetricsResponse.class,
+ requestHasSensitiveInfo = false, responseHasSensitiveInfo = false, responseView = ResponseObject.ResponseView.Full,
+ since = "4.17.0", authorized = {RoleType.Admin})
+public class ListMgmtsMetricsCmd extends ListMgmtsCmd {
+ public static final String APINAME = "listManagementServersMetrics";
+
+ @Parameter(name = MetricConstants.SYSTEM, type = CommandType.BOOLEAN, entityType = ManagementServerMetricsResponse.class, description = "include system level stats")
+ private boolean system;
+
+ @Inject
+ private MetricsService metricsService;
+
+ @Override
+ public String getCommandName() {
+ return APINAME.toLowerCase() + BaseCmd.RESPONSE_SUFFIX;
Review comment:
@DaanHoogland I'm working on a proposal to this, maybe in the next week or the another I already can open a PR to share my thoughts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1022321065
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1022857629
@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1016343498
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 2247
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1024028739
@borisstoyanov a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-951914661
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1620
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r736578699
##########
File path: framework/cluster/src/main/java/com/cloud/cluster/ManagementServerStatusVO.java
##########
@@ -0,0 +1,195 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.cluster;
+
+import com.cloud.utils.db.GenericDao;
+import org.apache.cloudstack.management.ManagementServerStatus;
+
+import javax.persistence.Column;
+import javax.persistence.Entity;
+import javax.persistence.GeneratedValue;
+import javax.persistence.GenerationType;
+import javax.persistence.Id;
+import javax.persistence.Table;
+import javax.persistence.Temporal;
+import javax.persistence.TemporalType;
+import java.util.Date;
+
+@Entity
+@Table(name = "mshost_status")
+public class ManagementServerStatusVO implements ManagementServerStatus {
+
+ @Id
+ @GeneratedValue(strategy = GenerationType.IDENTITY)
+ @Column(name = "id")
+ private long id;
+
+ @Column(name = "ms_id", nullable = false)
+ private String msId;
+
+ @Temporal(TemporalType.TIMESTAMP)
+ @Column(name="last_start")
+ private Date lastStart;
+
+ @Temporal(TemporalType.TIMESTAMP)
+ @Column(name="last_stop")
+ private Date lastStop;
+
+ @Temporal(TemporalType.TIMESTAMP)
+ @Column(name="last_boot")
+ private Date lastBoot;
+
+ @Temporal(TemporalType.TIMESTAMP)
+ @Column(name="last_down")
+ private Date lastDown;
+
+ @Column(name="os_distribution")
+ private String osDistribution;
+
+ @Column(name="java_name")
+ private String javaName;
+
+ @Column(name="java_version")
+ private String javaVersion;
+
+ @Temporal(TemporalType.TIMESTAMP)
+ @Column(name = "updated")
+ private Date updated;
+
+ @Column(name = GenericDao.CREATED_COLUMN)
+ private Date created;
+
+ @Column(name = GenericDao.REMOVED_COLUMN)
+ private Date removed;
+
+
+ public ManagementServerStatusVO() {
+ }
+
+ @Override
+ public long getId() {
+ return id;
+ }
+
+ public void setId(long id) {
+ this.id = id;
+ }
+
+ @Override
+ public String getMsId() {
+ return msId;
+ }
+
+ public void setMsId(String msId) {
+ this.msId = msId;
+ }
+
+ @Override
+ public Date getLastStart() {
+ return lastStart;
+ }
+
+ public void setLastStart(Date lastStart) {
+ this.lastStart = lastStart;
+ }
+
+ @Override
+ public Date getLastStop() {
+ return lastStop;
+ }
+
+ public void setLastStop(Date lastStop) {
+ this.lastStop = lastStop;
+ }
+
+ @Override
+ public Date getLastBoot() {
+ return lastBoot;
+ }
+
+ public void setLastBoot(Date lastBoot) {
+ this.lastBoot = lastBoot;
+ }
+
+ @Override
+ public Date getLastDown() {
+ return lastDown;
+ }
+
+ public void setLastDown(Date lastDown) {
+ this.lastDown = lastDown;
+ }
+
+ @Override
+ public String getOsDistribution() {
+ return osDistribution;
+ }
+
+ public void setOsDistribution(String osDistribution) {
+ this.osDistribution = osDistribution;
+ }
+
+ @Override
+ public String getJavaName() {
+ return javaName;
+ }
+
+ public void setJavaName(String javaName) {
+ this.javaName = javaName;
+ }
+
+ @Override
+ public String getJavaVersion() {
+ return javaVersion;
+ }
+
+ public void setJavaVersion(String javaVersion) {
+ this.javaVersion = javaVersion;
+ }
+
+ @Override
+ public Date getUpdated() {
+ return updated;
+ }
+
+ public void setUpdated(Date updated) {
+ this.updated = updated;
+ }
+
+ @Override
+ public Date getCreated() {
+ return created;
+ }
+
+ public void setCreated(Date created) {
+ this.created = created;
+ }
+
+ @Override
+ public Date getRemoved() {
+ return removed;
+ }
+
+ public void setRemoved(Date removedTime) {
+ removed = removedTime;
+ }
+
+ @Override
+ public String toString() {
+ return new org.apache.commons.lang3.builder.ReflectionToStringBuilder(this, org.apache.commons.lang3.builder.ToStringStyle.SIMPLE_STYLE).toString();
Review comment:
yes, of course, tnx
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-974221108
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1750
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-972676761
Faking result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID fake-job
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r752058463
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/response/ManagementServerMetricsResponse.java
##########
@@ -0,0 +1,177 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.response;
+
+import com.cloud.serializer.Param;
+import com.google.gson.annotations.SerializedName;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.api.response.ManagementServerResponse;
+
+public class ManagementServerMetricsResponse extends ManagementServerResponse {
+
+ @SerializedName(ApiConstants.AVAILABLE_PROCESSORS)
+ @Param(description = "the number of processors available to the JVM")
+ private int availableProcessors;
Review comment:
might be a future enhancement? was not in scope for this change, but I will discuss as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] weizhouapache commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
weizhouapache commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r752061723
##########
File path: plugins/metrics/src/main/java/org/apache/cloudstack/response/ManagementServerMetricsResponse.java
##########
@@ -0,0 +1,177 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.response;
+
+import com.cloud.serializer.Param;
+import com.google.gson.annotations.SerializedName;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.api.response.ManagementServerResponse;
+
+public class ManagementServerMetricsResponse extends ManagementServerResponse {
+
+ @SerializedName(ApiConstants.AVAILABLE_PROCESSORS)
+ @Param(description = "the number of processors available to the JVM")
+ private int availableProcessors;
Review comment:
cool. thanks @DaanHoogland
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r752064103
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -564,11 +657,352 @@ protected Point creteInfluxDbPoint(Object metricsObject) {
}
}
+ Gson gson;
+
+ class ManagementServerCollector extends AbstractStatsCollector {
+ @Override
+ protected void runInContext() {
+ LOGGER.debug(String.format("%s is running...", this.getClass().getSimpleName()));
+ long msid = ManagementServerNode.getManagementServerId();
+ ManagementServerHostVO mshost = null;
+ ManagementServerHostStatsEntry hostStatsEntry = null;
+ try {
+ mshost = managementServerHostDao.findByMsid(msid);
+ // get local data
+ hostStatsEntry = getDataFrom(mshost);
+ managementServerHostStats.put(mshost.getUuid(), hostStatsEntry);
+ // send to other hosts
+ clusterManager.publishStatus(gson.toJson(hostStatsEntry));
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to retrieve host stats", t);
+ }
+ try {
+ // send to DB
+ storeStatus(hostStatsEntry, mshost);
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to store host state", t);
+ }
+ }
+
+ private void storeStatus(ManagementServerHostStatsEntry hostStatsEntry, ManagementServerHostVO mshost) {
+ if (hostStatsEntry == null || mshost == null) {
+ return;
+ }
+ ManagementServerStatusVO msStats = managementServerStatusDao.findByMsId(hostStatsEntry.getManagementServerHostUuid());
+ if (msStats == null) {
+ LOGGER.info(String.format("creating new status info record for host %s - %s",
+ mshost.getName(),
+ hostStatsEntry.getManagementServerHostUuid()));
+ msStats = new ManagementServerStatusVO();
+ msStats.setMsId(hostStatsEntry.getManagementServerHostUuid());
+ }
+ msStats.setOsDistribution(hostStatsEntry.getOsDistribution()); // for now just the bunch details come later
+ msStats.setJavaName(hostStatsEntry.getJvmVendor());
+ msStats.setJavaVersion(hostStatsEntry.getJvmVersion());
+ Date startTime = new Date(hostStatsEntry.getStartTime());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("reporting starttime %s", startTime));
+ }
+ msStats.setLastStart(startTime);
+ msStats.setUpdated(new Date());
+ managementServerStatusDao.persist(msStats);
+ }
+
+ @NotNull
+ private ManagementServerHostStatsEntry getDataFrom(ManagementServerHostVO mshost) {
+ ManagementServerHostStatsEntry newEntry = new ManagementServerHostStatsEntry();
+ LOGGER.debug("Metrics collection start...");
+ newEntry.setManagementServerHostId(mshost.getId());
+ newEntry.setManagementServerHostUuid(mshost.getUuid());
+ retrieveSession(newEntry);
+ getJvmDimensions(newEntry);
+ LOGGER.debug("Metrics collection extra...");
+ getRuntimeData(newEntry);
+ getCpuData(newEntry);
+ getMemoryData(newEntry);
+ // newEntry must now include a pid!
+ getProcFsData(newEntry);
+ getFsData(newEntry);
+ getDataBaseStatistics(newEntry, mshost.getMsid());
+ gatherAllMetrics(newEntry);
+ LOGGER.debug("Metrics collection end!");
+ return newEntry;
+ }
+
+ private void retrieveSession(ManagementServerHostStatsEntry newEntry) {
+ long sessions = ApiSessionListener.getSessionCount();
+ newEntry.setSessions(sessions);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("Sessions found in Api %d vs context %d", sessions,ApiSessionListener.getNumberOfSessions()));
+ } else {
+ LOGGER.debug("Sessions active: " + sessions);
+ }
+ }
+
+ private void getDataBaseStatistics(ManagementServerHostStatsEntry newEntry, long msid) {
+ int count = _hostDao.countByMs(msid);
+ newEntry.setAgentCount(count);
+ }
+
+ private void getMemoryData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ MemoryMXBean mxBean = ManagementFactory.getMemoryMXBean();
+ newEntry.setTotalInit(mxBean.getHeapMemoryUsage().getInit() + mxBean.getNonHeapMemoryUsage().getInit());
+ newEntry.setTotalUsed(mxBean.getHeapMemoryUsage().getUsed() + mxBean.getNonHeapMemoryUsage().getUsed());
+ newEntry.setTotalMax(mxBean.getHeapMemoryUsage().getMax() + mxBean.getNonHeapMemoryUsage().getMax());
+ newEntry.setTotalCommitted(mxBean.getHeapMemoryUsage().getCommitted() + mxBean.getNonHeapMemoryUsage().getCommitted());
+ }
+
+ private void getCpuData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final OperatingSystemMXBean mxBean = ManagementFactory.getOperatingSystemMXBean();
+ newEntry.setAvailableProcessors(mxBean.getAvailableProcessors());
+ newEntry.setLoadAverage(mxBean.getSystemLoadAverage());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics processors - %d , loadavg - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getLoadAverage()));
+ }
+ }
+
+ private void getRuntimeData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final RuntimeMXBean mxBean = ManagementFactory.getRuntimeMXBean();
+ newEntry.setUptime(mxBean.getUptime());
+ newEntry.setStartTime(mxBean.getStartTime());
+ newEntry.setProcessId(mxBean.getPid());
+ newEntry.setJvmName(mxBean.getName());
+ newEntry.setJvmVendor(mxBean.getVmVendor());
+ newEntry.setJvmVersion(mxBean.getVmVersion());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics uptime - %d , starttime - %d",
+ newEntry.getUptime(),
+ newEntry.getStartTime()));
+ }
+ }
+
+ private void getJvmDimensions(@NotNull ManagementServerHostStatsEntry newEntry) {
+ Runtime runtime = Runtime.getRuntime();
+ newEntry.setTotalMemoryBytes(runtime.totalMemory());
+ newEntry.setFreeMemoryBytes(runtime.freeMemory());
+ newEntry.setAvailableProcessors(runtime.availableProcessors());
+ newEntry.setTotalMax(runtime.maxMemory());
+ //long maxMem = runtime.maxMemory();
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics proc - %d , maxMem - %d , totalMemory - %d , freeMemory - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getTotalMax(),
+ newEntry.getTotalMemoryBytes(),
+ newEntry.getFreeMemoryBytes()));
+ }
+ }
+
+ /**
+ * As for data from outside the JVM, we only rely on /proc/ contained data.
+ *
+ * @param newEntry
+ */
+ private void getProcFsData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ String OS = Script.runSimpleBashScript("cat /proc/version");
+ newEntry.setOsDistribution(OS);
+ String mem = Script.runSimpleBashScript("cat /proc/meminfo | grep MemTotal | cut -f 2 -d ':' | tr -d 'a-zA-z '").trim();
Review comment:
is that as reliable? I don't mind changing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-977656706
Packaging result: :heavy_multiplication_x: el7 :heavy_multiplication_x: el8 :heavy_multiplication_x: debian :heavy_multiplication_x: suse15. SL-JID 1776
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-979358819
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1788
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-982709184
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1803
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r761917622
##########
File path: api/src/main/java/org/apache/cloudstack/api/response/ManagementServerResponse.java
##########
@@ -35,12 +38,68 @@
@SerializedName(ApiConstants.STATE)
@Param(description = "the state of the management server")
- private ManagementServerHost.State state;
+ private State state;
@SerializedName(ApiConstants.VERSION)
@Param(description = "the version of the management server")
private String version;
+ @SerializedName(ApiConstants.JAVA_DISTRIBUTION)
+ @Param(description = "the java distribution name running the management server process")
+ private String javaDistribution;
+
+ @SerializedName(ApiConstants.JAVA_VERSION)
+ @Param(description = "the version of the java distribution running the management server process")
+ private String javaVersion;
+
+ @SerializedName(ApiConstants.OS_DISTRIBUTION)
+ @Param(description = "the name of the OS distribution running on the management server")
+ private String osDistribution;
+
+ @SerializedName(ApiConstants.LAST_START)
+ @Param(description = "the last time this Management Server was started")
+ private Date lastStart;
+
+ @SerializedName(ApiConstants.LAST_STOP)
+ @Param(description = "the last time this Management Server was started")
Review comment:
typo, started => _stopped_
```suggestion
@Param(description = "the last time this Management Server was stopped")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-996795454
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1939
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-996764029
@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-995989389
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r770592011
##########
File path: api/src/main/java/org/apache/cloudstack/management/ManagementServerStatus.java
##########
@@ -0,0 +1,47 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.management;
+
+import org.apache.cloudstack.api.InternalIdentity;
+
+import java.util.Date;
+
+public interface ManagementServerStatus extends InternalIdentity {
+ long getId();
+
+ String getMsId();
+
+ Date getLastStart();
Review comment:
made it lastJvmStart, didn't think systemstart would be much clearer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-998844371
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-996795454
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
sureshanaparti removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-995895958
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-950942111
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1612
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1064856724
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1028052148
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1028211978
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-949698667
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-949699994
@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r736320369
##########
File path: api/src/main/java/org/apache/cloudstack/api/command/admin/management/ListMgmtsCmd.java
##########
@@ -75,4 +77,9 @@ public void execute() {
response.setResponseName(getCommandName());
this.setResponseObject(response);
}
+
+ protected void updateMngmntResponse(List<ManagementServerResponse> response) {
Review comment:
yes, I'll go for the full name(s)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r736311697
##########
File path: engine/schema/src/main/resources/META-INF/db/schema-41520to41600.sql
##########
@@ -807,3 +807,21 @@ ALTER TABLE `cloud_usage`.`user_statistics` DROP INDEX `account_id`, ADD UNIQUE
ALTER TABLE `cloud`.`vm_work_job` ADD COLUMN `secondary_object` char(100) COMMENT 'any additional item that must be checked during queueing' AFTER `vm_instance_id`;
ALTER TABLE cloud.vm_work_job ADD CONSTRAINT vm_work_job_step_and_objects UNIQUE KEY (step,vm_instance_id,secondary_object);
+
+CREATE TABLE `cloud`.`mshost_status` (
+ `id` bigint unsigned NOT NULL auto_increment COMMENT 'id',
+ `ms_id` varchar(40) DEFAULT NULL COMMENT 'the id of the management server record',
+ `last_start` datetime COMMENT 'the last start time for this MS',
+ `last_stop` datetime COMMENT 'the last stop time for this MS',
+ `last_boot` datetime COMMENT 'the last system boot time for the host of this MS',
+ `last_down` datetime COMMENT 'the last system shutdown time for the host of this MS',
+ `os_name` varchar(64) DEFAULT null COMMENT 'the name of the os type running on the host of this MS',
+ `os_version` varchar(64) DEFAULT null COMMENT 'the version of the os running on the host of this MS',
+ `java_name` varchar(64) DEFAULT null COMMENT 'the name of the java distribution running this MS',
+ `java_version` varchar(64) DEFAULT null COMMENT 'the version of the java distribution running this MS',
+ `updated` datetime DEFAULT NULL,
+ `created` datetime DEFAULT NULL,
+ `removed` datetime DEFAULT NULL,
+ PRIMARY KEY (`id`),
+ CONSTRAINT `uc_ms_id` UNIQUE (`ms_id`)
Review comment:
check and add constraint for 'ms_id' column, that references 'id' in mshost table.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] sureshanaparti commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
sureshanaparti commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r736410801
##########
File path: engine/schema/src/main/resources/META-INF/db/schema-41520to41600.sql
##########
@@ -807,3 +807,21 @@ ALTER TABLE `cloud_usage`.`user_statistics` DROP INDEX `account_id`, ADD UNIQUE
ALTER TABLE `cloud`.`vm_work_job` ADD COLUMN `secondary_object` char(100) COMMENT 'any additional item that must be checked during queueing' AFTER `vm_instance_id`;
ALTER TABLE cloud.vm_work_job ADD CONSTRAINT vm_work_job_step_and_objects UNIQUE KEY (step,vm_instance_id,secondary_object);
+
+CREATE TABLE `cloud`.`mshost_status` (
+ `id` bigint unsigned NOT NULL auto_increment COMMENT 'id',
+ `ms_id` varchar(40) DEFAULT NULL COMMENT 'the id of the management server record',
+ `last_start` datetime COMMENT 'the last start time for this MS',
+ `last_stop` datetime COMMENT 'the last stop time for this MS',
+ `last_boot` datetime COMMENT 'the last system boot time for the host of this MS',
+ `last_down` datetime COMMENT 'the last system shutdown time for the host of this MS',
+ `os_name` varchar(64) DEFAULT null COMMENT 'the name of the os type running on the host of this MS',
+ `os_version` varchar(64) DEFAULT null COMMENT 'the version of the os running on the host of this MS',
+ `java_name` varchar(64) DEFAULT null COMMENT 'the name of the java distribution running this MS',
+ `java_version` varchar(64) DEFAULT null COMMENT 'the version of the java distribution running this MS',
+ `updated` datetime DEFAULT NULL,
+ `created` datetime DEFAULT NULL,
+ `removed` datetime DEFAULT NULL,
+ PRIMARY KEY (`id`),
+ CONSTRAINT `uc_ms_id` UNIQUE (`ms_id`)
Review comment:
mshost table already has the UUID, so 'id' can be used here for referencing instead of UUID.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-949699994
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-954123825
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r752063840
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -564,11 +657,352 @@ protected Point creteInfluxDbPoint(Object metricsObject) {
}
}
+ Gson gson;
+
+ class ManagementServerCollector extends AbstractStatsCollector {
+ @Override
+ protected void runInContext() {
+ LOGGER.debug(String.format("%s is running...", this.getClass().getSimpleName()));
+ long msid = ManagementServerNode.getManagementServerId();
+ ManagementServerHostVO mshost = null;
+ ManagementServerHostStatsEntry hostStatsEntry = null;
+ try {
+ mshost = managementServerHostDao.findByMsid(msid);
+ // get local data
+ hostStatsEntry = getDataFrom(mshost);
+ managementServerHostStats.put(mshost.getUuid(), hostStatsEntry);
+ // send to other hosts
+ clusterManager.publishStatus(gson.toJson(hostStatsEntry));
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to retrieve host stats", t);
+ }
+ try {
+ // send to DB
+ storeStatus(hostStatsEntry, mshost);
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to store host state", t);
+ }
+ }
+
+ private void storeStatus(ManagementServerHostStatsEntry hostStatsEntry, ManagementServerHostVO mshost) {
+ if (hostStatsEntry == null || mshost == null) {
+ return;
+ }
+ ManagementServerStatusVO msStats = managementServerStatusDao.findByMsId(hostStatsEntry.getManagementServerHostUuid());
+ if (msStats == null) {
+ LOGGER.info(String.format("creating new status info record for host %s - %s",
+ mshost.getName(),
+ hostStatsEntry.getManagementServerHostUuid()));
+ msStats = new ManagementServerStatusVO();
+ msStats.setMsId(hostStatsEntry.getManagementServerHostUuid());
+ }
+ msStats.setOsDistribution(hostStatsEntry.getOsDistribution()); // for now just the bunch details come later
+ msStats.setJavaName(hostStatsEntry.getJvmVendor());
+ msStats.setJavaVersion(hostStatsEntry.getJvmVersion());
+ Date startTime = new Date(hostStatsEntry.getStartTime());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("reporting starttime %s", startTime));
+ }
+ msStats.setLastStart(startTime);
+ msStats.setUpdated(new Date());
+ managementServerStatusDao.persist(msStats);
+ }
+
+ @NotNull
+ private ManagementServerHostStatsEntry getDataFrom(ManagementServerHostVO mshost) {
+ ManagementServerHostStatsEntry newEntry = new ManagementServerHostStatsEntry();
+ LOGGER.debug("Metrics collection start...");
+ newEntry.setManagementServerHostId(mshost.getId());
+ newEntry.setManagementServerHostUuid(mshost.getUuid());
+ retrieveSession(newEntry);
+ getJvmDimensions(newEntry);
+ LOGGER.debug("Metrics collection extra...");
+ getRuntimeData(newEntry);
+ getCpuData(newEntry);
+ getMemoryData(newEntry);
+ // newEntry must now include a pid!
+ getProcFsData(newEntry);
+ getFsData(newEntry);
+ getDataBaseStatistics(newEntry, mshost.getMsid());
+ gatherAllMetrics(newEntry);
+ LOGGER.debug("Metrics collection end!");
+ return newEntry;
+ }
+
+ private void retrieveSession(ManagementServerHostStatsEntry newEntry) {
+ long sessions = ApiSessionListener.getSessionCount();
+ newEntry.setSessions(sessions);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("Sessions found in Api %d vs context %d", sessions,ApiSessionListener.getNumberOfSessions()));
+ } else {
+ LOGGER.debug("Sessions active: " + sessions);
+ }
+ }
+
+ private void getDataBaseStatistics(ManagementServerHostStatsEntry newEntry, long msid) {
+ int count = _hostDao.countByMs(msid);
+ newEntry.setAgentCount(count);
+ }
+
+ private void getMemoryData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ MemoryMXBean mxBean = ManagementFactory.getMemoryMXBean();
+ newEntry.setTotalInit(mxBean.getHeapMemoryUsage().getInit() + mxBean.getNonHeapMemoryUsage().getInit());
+ newEntry.setTotalUsed(mxBean.getHeapMemoryUsage().getUsed() + mxBean.getNonHeapMemoryUsage().getUsed());
+ newEntry.setTotalMax(mxBean.getHeapMemoryUsage().getMax() + mxBean.getNonHeapMemoryUsage().getMax());
+ newEntry.setTotalCommitted(mxBean.getHeapMemoryUsage().getCommitted() + mxBean.getNonHeapMemoryUsage().getCommitted());
+ }
+
+ private void getCpuData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final OperatingSystemMXBean mxBean = ManagementFactory.getOperatingSystemMXBean();
+ newEntry.setAvailableProcessors(mxBean.getAvailableProcessors());
+ newEntry.setLoadAverage(mxBean.getSystemLoadAverage());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics processors - %d , loadavg - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getLoadAverage()));
+ }
+ }
+
+ private void getRuntimeData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final RuntimeMXBean mxBean = ManagementFactory.getRuntimeMXBean();
+ newEntry.setUptime(mxBean.getUptime());
+ newEntry.setStartTime(mxBean.getStartTime());
+ newEntry.setProcessId(mxBean.getPid());
+ newEntry.setJvmName(mxBean.getName());
+ newEntry.setJvmVendor(mxBean.getVmVendor());
+ newEntry.setJvmVersion(mxBean.getVmVersion());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics uptime - %d , starttime - %d",
+ newEntry.getUptime(),
+ newEntry.getStartTime()));
+ }
+ }
+
+ private void getJvmDimensions(@NotNull ManagementServerHostStatsEntry newEntry) {
+ Runtime runtime = Runtime.getRuntime();
+ newEntry.setTotalMemoryBytes(runtime.totalMemory());
+ newEntry.setFreeMemoryBytes(runtime.freeMemory());
+ newEntry.setAvailableProcessors(runtime.availableProcessors());
+ newEntry.setTotalMax(runtime.maxMemory());
+ //long maxMem = runtime.maxMemory();
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics proc - %d , maxMem - %d , totalMemory - %d , freeMemory - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getTotalMax(),
+ newEntry.getTotalMemoryBytes(),
+ newEntry.getFreeMemoryBytes()));
+ }
+ }
+
+ /**
+ * As for data from outside the JVM, we only rely on /proc/ contained data.
+ *
+ * @param newEntry
+ */
+ private void getProcFsData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ String OS = Script.runSimpleBashScript("cat /proc/version");
Review comment:
I'll look at the bean, /etc/os-release is not reliable in my opinion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r752259918
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -564,11 +657,352 @@ protected Point creteInfluxDbPoint(Object metricsObject) {
}
}
+ Gson gson;
+
+ class ManagementServerCollector extends AbstractStatsCollector {
+ @Override
+ protected void runInContext() {
+ LOGGER.debug(String.format("%s is running...", this.getClass().getSimpleName()));
+ long msid = ManagementServerNode.getManagementServerId();
+ ManagementServerHostVO mshost = null;
+ ManagementServerHostStatsEntry hostStatsEntry = null;
+ try {
+ mshost = managementServerHostDao.findByMsid(msid);
+ // get local data
+ hostStatsEntry = getDataFrom(mshost);
+ managementServerHostStats.put(mshost.getUuid(), hostStatsEntry);
+ // send to other hosts
+ clusterManager.publishStatus(gson.toJson(hostStatsEntry));
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to retrieve host stats", t);
+ }
+ try {
+ // send to DB
+ storeStatus(hostStatsEntry, mshost);
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to store host state", t);
+ }
+ }
+
+ private void storeStatus(ManagementServerHostStatsEntry hostStatsEntry, ManagementServerHostVO mshost) {
+ if (hostStatsEntry == null || mshost == null) {
+ return;
+ }
+ ManagementServerStatusVO msStats = managementServerStatusDao.findByMsId(hostStatsEntry.getManagementServerHostUuid());
+ if (msStats == null) {
+ LOGGER.info(String.format("creating new status info record for host %s - %s",
+ mshost.getName(),
+ hostStatsEntry.getManagementServerHostUuid()));
+ msStats = new ManagementServerStatusVO();
+ msStats.setMsId(hostStatsEntry.getManagementServerHostUuid());
+ }
+ msStats.setOsDistribution(hostStatsEntry.getOsDistribution()); // for now just the bunch details come later
+ msStats.setJavaName(hostStatsEntry.getJvmVendor());
+ msStats.setJavaVersion(hostStatsEntry.getJvmVersion());
+ Date startTime = new Date(hostStatsEntry.getStartTime());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("reporting starttime %s", startTime));
+ }
+ msStats.setLastStart(startTime);
+ msStats.setUpdated(new Date());
+ managementServerStatusDao.persist(msStats);
+ }
+
+ @NotNull
+ private ManagementServerHostStatsEntry getDataFrom(ManagementServerHostVO mshost) {
+ ManagementServerHostStatsEntry newEntry = new ManagementServerHostStatsEntry();
+ LOGGER.debug("Metrics collection start...");
+ newEntry.setManagementServerHostId(mshost.getId());
+ newEntry.setManagementServerHostUuid(mshost.getUuid());
+ retrieveSession(newEntry);
+ getJvmDimensions(newEntry);
+ LOGGER.debug("Metrics collection extra...");
+ getRuntimeData(newEntry);
+ getCpuData(newEntry);
+ getMemoryData(newEntry);
+ // newEntry must now include a pid!
+ getProcFsData(newEntry);
+ getFsData(newEntry);
+ getDataBaseStatistics(newEntry, mshost.getMsid());
+ gatherAllMetrics(newEntry);
+ LOGGER.debug("Metrics collection end!");
+ return newEntry;
+ }
+
+ private void retrieveSession(ManagementServerHostStatsEntry newEntry) {
+ long sessions = ApiSessionListener.getSessionCount();
+ newEntry.setSessions(sessions);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("Sessions found in Api %d vs context %d", sessions,ApiSessionListener.getNumberOfSessions()));
+ } else {
+ LOGGER.debug("Sessions active: " + sessions);
+ }
+ }
+
+ private void getDataBaseStatistics(ManagementServerHostStatsEntry newEntry, long msid) {
+ int count = _hostDao.countByMs(msid);
+ newEntry.setAgentCount(count);
+ }
+
+ private void getMemoryData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ MemoryMXBean mxBean = ManagementFactory.getMemoryMXBean();
+ newEntry.setTotalInit(mxBean.getHeapMemoryUsage().getInit() + mxBean.getNonHeapMemoryUsage().getInit());
+ newEntry.setTotalUsed(mxBean.getHeapMemoryUsage().getUsed() + mxBean.getNonHeapMemoryUsage().getUsed());
+ newEntry.setTotalMax(mxBean.getHeapMemoryUsage().getMax() + mxBean.getNonHeapMemoryUsage().getMax());
+ newEntry.setTotalCommitted(mxBean.getHeapMemoryUsage().getCommitted() + mxBean.getNonHeapMemoryUsage().getCommitted());
+ }
+
+ private void getCpuData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final OperatingSystemMXBean mxBean = ManagementFactory.getOperatingSystemMXBean();
+ newEntry.setAvailableProcessors(mxBean.getAvailableProcessors());
+ newEntry.setLoadAverage(mxBean.getSystemLoadAverage());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics processors - %d , loadavg - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getLoadAverage()));
+ }
+ }
+
+ private void getRuntimeData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final RuntimeMXBean mxBean = ManagementFactory.getRuntimeMXBean();
+ newEntry.setUptime(mxBean.getUptime());
+ newEntry.setStartTime(mxBean.getStartTime());
+ newEntry.setProcessId(mxBean.getPid());
+ newEntry.setJvmName(mxBean.getName());
+ newEntry.setJvmVendor(mxBean.getVmVendor());
+ newEntry.setJvmVersion(mxBean.getVmVersion());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics uptime - %d , starttime - %d",
+ newEntry.getUptime(),
+ newEntry.getStartTime()));
+ }
+ }
+
+ private void getJvmDimensions(@NotNull ManagementServerHostStatsEntry newEntry) {
+ Runtime runtime = Runtime.getRuntime();
+ newEntry.setTotalMemoryBytes(runtime.totalMemory());
+ newEntry.setFreeMemoryBytes(runtime.freeMemory());
+ newEntry.setAvailableProcessors(runtime.availableProcessors());
+ newEntry.setTotalMax(runtime.maxMemory());
+ //long maxMem = runtime.maxMemory();
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics proc - %d , maxMem - %d , totalMemory - %d , freeMemory - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getTotalMax(),
+ newEntry.getTotalMemoryBytes(),
+ newEntry.getFreeMemoryBytes()));
+ }
+ }
+
+ /**
+ * As for data from outside the JVM, we only rely on /proc/ contained data.
+ *
+ * @param newEntry
+ */
+ private void getProcFsData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ String OS = Script.runSimpleBashScript("cat /proc/version");
+ newEntry.setOsDistribution(OS);
+ String mem = Script.runSimpleBashScript("cat /proc/meminfo | grep MemTotal | cut -f 2 -d ':' | tr -d 'a-zA-z '").trim();
Review comment:
did a conditional retrieve with fallback, @weizhouapache . the return type for `ManagementFactory.getOperatingSystemMXBean()` is documented to be platform dependent. So I retrieve if I can and scavenge /proc if I have no data as a result.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r752061903
##########
File path: framework/cluster/src/main/java/com/cloud/cluster/ClusterManagerImpl.java
##########
@@ -97,10 +98,14 @@
@Inject
private ManagementServerHostDao _mshostDao;
@Inject
+ private ManagementServerStatusDao mshostStatusDao;
Review comment:
hm, I don't subscribe to that and we didn't include that in our checkstyle at it was regarded an obsolete convention AFAICR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#discussion_r752107060
##########
File path: server/src/main/java/com/cloud/server/StatsCollector.java
##########
@@ -564,11 +657,352 @@ protected Point creteInfluxDbPoint(Object metricsObject) {
}
}
+ Gson gson;
+
+ class ManagementServerCollector extends AbstractStatsCollector {
+ @Override
+ protected void runInContext() {
+ LOGGER.debug(String.format("%s is running...", this.getClass().getSimpleName()));
+ long msid = ManagementServerNode.getManagementServerId();
+ ManagementServerHostVO mshost = null;
+ ManagementServerHostStatsEntry hostStatsEntry = null;
+ try {
+ mshost = managementServerHostDao.findByMsid(msid);
+ // get local data
+ hostStatsEntry = getDataFrom(mshost);
+ managementServerHostStats.put(mshost.getUuid(), hostStatsEntry);
+ // send to other hosts
+ clusterManager.publishStatus(gson.toJson(hostStatsEntry));
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to retrieve host stats", t);
+ }
+ try {
+ // send to DB
+ storeStatus(hostStatsEntry, mshost);
+ } catch (Throwable t) {
+ // pokemon catch to make sure the thread stays running
+ LOGGER.error("Error trying to store host state", t);
+ }
+ }
+
+ private void storeStatus(ManagementServerHostStatsEntry hostStatsEntry, ManagementServerHostVO mshost) {
+ if (hostStatsEntry == null || mshost == null) {
+ return;
+ }
+ ManagementServerStatusVO msStats = managementServerStatusDao.findByMsId(hostStatsEntry.getManagementServerHostUuid());
+ if (msStats == null) {
+ LOGGER.info(String.format("creating new status info record for host %s - %s",
+ mshost.getName(),
+ hostStatsEntry.getManagementServerHostUuid()));
+ msStats = new ManagementServerStatusVO();
+ msStats.setMsId(hostStatsEntry.getManagementServerHostUuid());
+ }
+ msStats.setOsDistribution(hostStatsEntry.getOsDistribution()); // for now just the bunch details come later
+ msStats.setJavaName(hostStatsEntry.getJvmVendor());
+ msStats.setJavaVersion(hostStatsEntry.getJvmVersion());
+ Date startTime = new Date(hostStatsEntry.getStartTime());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("reporting starttime %s", startTime));
+ }
+ msStats.setLastStart(startTime);
+ msStats.setUpdated(new Date());
+ managementServerStatusDao.persist(msStats);
+ }
+
+ @NotNull
+ private ManagementServerHostStatsEntry getDataFrom(ManagementServerHostVO mshost) {
+ ManagementServerHostStatsEntry newEntry = new ManagementServerHostStatsEntry();
+ LOGGER.debug("Metrics collection start...");
+ newEntry.setManagementServerHostId(mshost.getId());
+ newEntry.setManagementServerHostUuid(mshost.getUuid());
+ retrieveSession(newEntry);
+ getJvmDimensions(newEntry);
+ LOGGER.debug("Metrics collection extra...");
+ getRuntimeData(newEntry);
+ getCpuData(newEntry);
+ getMemoryData(newEntry);
+ // newEntry must now include a pid!
+ getProcFsData(newEntry);
+ getFsData(newEntry);
+ getDataBaseStatistics(newEntry, mshost.getMsid());
+ gatherAllMetrics(newEntry);
+ LOGGER.debug("Metrics collection end!");
+ return newEntry;
+ }
+
+ private void retrieveSession(ManagementServerHostStatsEntry newEntry) {
+ long sessions = ApiSessionListener.getSessionCount();
+ newEntry.setSessions(sessions);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("Sessions found in Api %d vs context %d", sessions,ApiSessionListener.getNumberOfSessions()));
+ } else {
+ LOGGER.debug("Sessions active: " + sessions);
+ }
+ }
+
+ private void getDataBaseStatistics(ManagementServerHostStatsEntry newEntry, long msid) {
+ int count = _hostDao.countByMs(msid);
+ newEntry.setAgentCount(count);
+ }
+
+ private void getMemoryData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ MemoryMXBean mxBean = ManagementFactory.getMemoryMXBean();
+ newEntry.setTotalInit(mxBean.getHeapMemoryUsage().getInit() + mxBean.getNonHeapMemoryUsage().getInit());
+ newEntry.setTotalUsed(mxBean.getHeapMemoryUsage().getUsed() + mxBean.getNonHeapMemoryUsage().getUsed());
+ newEntry.setTotalMax(mxBean.getHeapMemoryUsage().getMax() + mxBean.getNonHeapMemoryUsage().getMax());
+ newEntry.setTotalCommitted(mxBean.getHeapMemoryUsage().getCommitted() + mxBean.getNonHeapMemoryUsage().getCommitted());
+ }
+
+ private void getCpuData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final OperatingSystemMXBean mxBean = ManagementFactory.getOperatingSystemMXBean();
+ newEntry.setAvailableProcessors(mxBean.getAvailableProcessors());
+ newEntry.setLoadAverage(mxBean.getSystemLoadAverage());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics processors - %d , loadavg - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getLoadAverage()));
+ }
+ }
+
+ private void getRuntimeData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ final RuntimeMXBean mxBean = ManagementFactory.getRuntimeMXBean();
+ newEntry.setUptime(mxBean.getUptime());
+ newEntry.setStartTime(mxBean.getStartTime());
+ newEntry.setProcessId(mxBean.getPid());
+ newEntry.setJvmName(mxBean.getName());
+ newEntry.setJvmVendor(mxBean.getVmVendor());
+ newEntry.setJvmVersion(mxBean.getVmVersion());
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics uptime - %d , starttime - %d",
+ newEntry.getUptime(),
+ newEntry.getStartTime()));
+ }
+ }
+
+ private void getJvmDimensions(@NotNull ManagementServerHostStatsEntry newEntry) {
+ Runtime runtime = Runtime.getRuntime();
+ newEntry.setTotalMemoryBytes(runtime.totalMemory());
+ newEntry.setFreeMemoryBytes(runtime.freeMemory());
+ newEntry.setAvailableProcessors(runtime.availableProcessors());
+ newEntry.setTotalMax(runtime.maxMemory());
+ //long maxMem = runtime.maxMemory();
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format(
+ "Metrics proc - %d , maxMem - %d , totalMemory - %d , freeMemory - %f ",
+ newEntry.getAvailableProcessors(),
+ newEntry.getTotalMax(),
+ newEntry.getTotalMemoryBytes(),
+ newEntry.getFreeMemoryBytes()));
+ }
+ }
+
+ /**
+ * As for data from outside the JVM, we only rely on /proc/ contained data.
+ *
+ * @param newEntry
+ */
+ private void getProcFsData(@NotNull ManagementServerHostStatsEntry newEntry) {
+ String OS = Script.runSimpleBashScript("cat /proc/version");
Review comment:
the bean give considderably less information, so I'll stick with the /proc data for now. Open for discussion though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-972988374
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1056914159
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1056914159
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1060666349
@borisstoyanov a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1063807135
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1069213404
@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1069212464
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
nvazquez commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1083255816
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
DaanHoogland removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1056714376
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
nvazquez removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1083255816
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez removed a comment on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
nvazquez removed a comment on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1081928858
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1081930314
@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #5588: Mshost stats
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #5588:
URL: https://github.com/apache/cloudstack/pull/5588#issuecomment-1084805118
<b>Trillian Build Failed (tid-3781)<b/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org