You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by "abh1sar (via GitHub)" <gi...@apache.org> on 2025/08/29 05:57:54 UTC
[PR] Track volume usage data at a vm granularity as well [cloudstack]
abh1sar opened a new pull request, #11531:
URL: https://github.com/apache/cloudstack/pull/11531
### Description
This PR fixes #10985
Some users have asked to show the virtualmachineid also in volume usage, to track it per instance.
There will be two kinds of records now.
- Without the virtualmachineid - tracking the cumulative volume usage in the usage window. This is to be consistent with the current behaviour.
- With virtualmachineid - tracking the volume usage for the time it was attached to the vm in the usage window.
If the volume was attached to multiple instances in the same usage window, we will have separate entries for each VM.
#### Schema changes
- Add column vm_id to cloud.usage_event and cloud_usage.usage_event
- Add column vm_id to cloud_usage.usage_volume
I have also done some code refactoring in createVolumeHelperEvent() to make it easier to follow.
<!--- Describe your changes in DETAIL - And how has behaviour functionally changed. -->
<!-- For new features, provide link to FS, dev ML discussion etc. -->
<!-- In case of bug fix, the expected and actual behaviours, steps to reproduce. -->
<!-- When "Fixes: #<id>" is specified, the issue/PR will automatically be closed when this PR gets merged -->
<!-- For addressing multiple issues/PRs, use multiple "Fixes: #<id>" -->
<!-- Fixes: # -->
<!--- ******************************************************************************* -->
<!--- NOTE: AUTOMATION USES THE DESCRIPTIONS TO SET LABELS AND PRODUCE DOCUMENTATION. -->
<!--- PLEASE PUT AN 'X' in only **ONE** box -->
<!--- ******************************************************************************* -->
### Types of changes
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
- [ ] build/CI
- [ ] test (unit or integration test code)
### Feature/Enhancement Scale or Bug Severity
#### Feature/Enhancement Scale
- [ ] Major
- [x] Minor
#### Bug Severity
- [ ] BLOCKER
- [ ] Critical
- [ ] Major
- [ ] Minor
- [ ] Trivial
### Screenshots (if appropriate):
listUsageRecords for a volume in a 2 minute (0.033333 hours) window.
The volume was created, then attached to VM-00a42c28-74ae-4а0c-b38a-4e3b88f11241. It was then detached and attached to another VM QA-d6df4657-2fb4-4e26-b7de-21743c5463cd. So we have 3 records.
<img width="2226" height="1180" alt="Screenshot 2025-08-29 at 2 42 20 AM" src="https://github.com/user-attachments/assets/6ae299d6-269b-4b07-a80f-01023168ecd8" />
### How Has This Been Tested?
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
#### How did you try to break this feature and the system with this change?
<!-- see how your change affects other areas of the code, etc. -->
<!-- Please read the [CONTRIBUTING](https://github.com/apache/cloudstack/blob/main/CONTRIBUTING.md) document -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "vishesh92 (via GitHub)" <gi...@apache.org>.
vishesh92 commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3354813889
@blueorangutan test keepEnv
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3519962691
<b>[SF] Trillian test result (tid-14799)</b>
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 46691 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr11531-t14799-kvm-ol8.zip
Smoke tests completed. 149 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3352541058
Packaging result [SF]: ✖️ el8 ✖️ el9 ✔️ debian ✖️ suse15. SL-JID 15238
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3353177021
@vishesh92 a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3356518174
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3357103756
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15257
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3270191065
> code looks good in isolation @abh1sar , but I see some bears on the road: what if someone upgrades from 20.1 to 21 and then on to 22; How do we guarantee they get the DB updates? It may be wiser to put this in version 22.
>
> Sorry, I overlooked this when I asked you to rebase earlier.
Thanks @DaanHoogland. Rebased to main.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "sudo87 (via GitHub)" <gi...@apache.org>.
sudo87 commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3236636033
clgtm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3355218049
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3352163010
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3235822351
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on code in PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#discussion_r2309484476
##########
engine/schema/src/main/resources/META-INF/db/schema-41930to41940.sql:
##########
Review Comment:
should go in a different file
##########
engine/schema/src/main/java/com/cloud/upgrade/dao/Upgrade41930to41940.java:
##########
Review Comment:
@abh1sar please retarget 4.20? 4.19 will be eol in less than 3 days ;)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "vishesh92 (via GitHub)" <gi...@apache.org>.
vishesh92 commented on code in PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#discussion_r2393561885
##########
usage/src/main/java/com/cloud/usage/UsageManagerImpl.java:
##########
@@ -1424,92 +1429,112 @@ private void createIPHelperEvent(UsageEventVO event) {
}
}
+ private void deleteExistingSecondaryStorageUsageForVolume(long volId, long accountId, Date deletedDate) {
+ List<UsageStorageVO> storageVOs = _usageStorageDao.listById(accountId, volId, StorageTypes.VOLUME);
+ for (UsageStorageVO storageVO : storageVOs) {
+ logger.debug("Setting the volume with id: {} to 'deleted' in the usage_storage table for account: {}.", volId, accountId);
+ storageVO.setDeleted(deletedDate);
+ _usageStorageDao.update(storageVO);
+ }
+ }
+
+ private void deleteExistingInstanceVolumeUsage(long volId, long accountId, Date deletedDate) {
+ List<UsageVolumeVO> volumesVOs = _usageVolumeDao.listByVolumeId(volId, accountId);
+ for (UsageVolumeVO volumesVO : volumesVOs) {
+ if (volumesVO.getVmId() != null) {
+ logger.debug("Setting the volume with id: {} for instance id: {} to 'deleted' in the usage_volume table for account {}.",
+ volumesVO.getVolumeId(), volumesVO.getVmId(), accountId);
+ volumesVO.setDeleted(deletedDate);
+ _usageVolumeDao.update(volumesVO.getId(), volumesVO);
+ }
+ }
+ }
+
+ private void deleteExistingVolumeUsage(long volId, long accountId, Date deletedDate) {
+ List<UsageVolumeVO> volumesVOs = _usageVolumeDao.listByVolumeId(volId, accountId);
+ for (UsageVolumeVO volumesVO : volumesVOs) {
+ logger.debug("Setting the volume with id: {} to 'deleted' in the usage_volume table for account: {}.", volId, accountId);
+ volumesVO.setDeleted(deletedDate);
+ _usageVolumeDao.update(volumesVO.getId(), volumesVO);
+ }
+ }
+
private void createVolumeHelperEvent(UsageEventVO event) {
long volId = event.getResourceId();
+ Account acct = _accountDao.findByIdIncludingRemoved(event.getAccountId());
+ List<UsageVolumeVO> volumesVOs;
+ UsageVolumeVO volumeVO;
- if (EventTypes.EVENT_VOLUME_CREATE.equals(event.getType())) {
- //For volumes which are 'attached' successfully, set the 'deleted' column in the usage_storage table,
+ switch (event.getType()) {
+ case EventTypes.EVENT_VOLUME_CREATE:
+ //For volumes which are 'attached' successfully from uploaded state, set the 'deleted' column in the usage_storage table,
//so that the secondary storage should stop accounting and only primary will be accounted.
- SearchCriteria<UsageStorageVO> sc = _usageStorageDao.createSearchCriteria();
- sc.addAnd("entityId", SearchCriteria.Op.EQ, volId);
- sc.addAnd("storageType", SearchCriteria.Op.EQ, StorageTypes.VOLUME);
- List<UsageStorageVO> volumesVOs = _usageStorageDao.search(sc, null);
- if (volumesVOs != null) {
- if (volumesVOs.size() == 1) {
- logger.debug("Setting the volume with id: " + volId + " to 'deleted' in the usage_storage table.");
- volumesVOs.get(0).setDeleted(event.getCreateDate());
- _usageStorageDao.update(volumesVOs.get(0));
- }
- }
- }
- if (EventTypes.EVENT_VOLUME_CREATE.equals(event.getType()) || EventTypes.EVENT_VOLUME_RESIZE.equals(event.getType())) {
- SearchCriteria<UsageVolumeVO> sc = _usageVolumeDao.createSearchCriteria();
- sc.addAnd("accountId", SearchCriteria.Op.EQ, event.getAccountId());
- sc.addAnd("volumeId", SearchCriteria.Op.EQ, volId);
- sc.addAnd("deleted", SearchCriteria.Op.NULL);
- List<UsageVolumeVO> volumesVOs = _usageVolumeDao.search(sc, null);
+ deleteExistingSecondaryStorageUsageForVolume(volId, event.getAccountId(), event.getCreateDate());
+
+ volumesVOs = _usageVolumeDao.listByVolumeId(volId, event.getAccountId());
if (volumesVOs.size() > 0) {
//This is a safeguard to avoid double counting of volumes.
logger.error("Found duplicate usage entry for volume: " + volId + " assigned to account: " + event.getAccountId() + "; marking as deleted...");
+ deleteExistingVolumeUsage(volId, event.getAccountId(), event.getCreateDate());
}
- //an entry exists if it is a resize volume event. marking the existing deleted and creating a new one in the case of resize.
- for (UsageVolumeVO volumesVO : volumesVOs) {
- if (logger.isDebugEnabled()) {
- logger.debug("deleting volume: " + volumesVO.getId() + " from account: " + volumesVO.getAccountId());
- }
- volumesVO.setDeleted(event.getCreateDate());
- _usageVolumeDao.update(volumesVO);
- }
- if (logger.isDebugEnabled()) {
- logger.debug("create volume with id : " + volId + " for account: " + event.getAccountId());
- }
- Account acct = _accountDao.findByIdIncludingRemoved(event.getAccountId());
- UsageVolumeVO volumeVO = new UsageVolumeVO(volId, event.getZoneId(), event.getAccountId(), acct.getDomainId(), event.getOfferingId(), event.getTemplateId(), event.getSize(), event.getCreateDate(), null);
+
+ logger.debug("Creating a new entry in usage_volume for volume with id: {} for account: {}", volId, event.getAccountId());
+ volumeVO = new UsageVolumeVO(volId, event.getZoneId(), event.getAccountId(), acct.getDomainId(), event.getOfferingId(), event.getTemplateId(), null, event.getSize(), event.getCreateDate(), null);
_usageVolumeDao.persist(volumeVO);
- } else if (EventTypes.EVENT_VOLUME_DELETE.equals(event.getType())) {
- SearchCriteria<UsageVolumeVO> sc = _usageVolumeDao.createSearchCriteria();
- sc.addAnd("accountId", SearchCriteria.Op.EQ, event.getAccountId());
- sc.addAnd("volumeId", SearchCriteria.Op.EQ, volId);
- sc.addAnd("deleted", SearchCriteria.Op.NULL);
- List<UsageVolumeVO> volumesVOs = _usageVolumeDao.search(sc, null);
- if (volumesVOs.size() > 1) {
- logger.warn("More that one usage entry for volume: " + volId + " assigned to account: " + event.getAccountId() + "; marking them all as deleted...");
- }
- for (UsageVolumeVO volumesVO : volumesVOs) {
- if (logger.isDebugEnabled()) {
- logger.debug("deleting volume: " + volumesVO.getId() + " from account: " + volumesVO.getAccountId());
- }
- volumesVO.setDeleted(event.getCreateDate()); // there really shouldn't be more than one
- _usageVolumeDao.update(volumesVO);
- }
- } else if (EventTypes.EVENT_VOLUME_UPLOAD.equals(event.getType())) {
- //For Upload event add an entry to the usage_storage table.
- SearchCriteria<UsageStorageVO> sc = _usageStorageDao.createSearchCriteria();
- sc.addAnd("accountId", SearchCriteria.Op.EQ, event.getAccountId());
- sc.addAnd("entityId", SearchCriteria.Op.EQ, volId);
- sc.addAnd("storageType", SearchCriteria.Op.EQ, StorageTypes.VOLUME);
- sc.addAnd("deleted", SearchCriteria.Op.NULL);
- List<UsageStorageVO> volumesVOs = _usageStorageDao.search(sc, null);
- if (volumesVOs.size() > 0) {
- //This is a safeguard to avoid double counting of volumes.
- logger.error("Found duplicate usage entry for volume: " + volId + " assigned to account: " + event.getAccountId() + "; marking as deleted...");
+ if (event.getVmId() != null) {
+ volumeVO = new UsageVolumeVO(volId, event.getZoneId(), event.getAccountId(), acct.getDomainId(), event.getOfferingId(), event.getTemplateId(), event.getVmId(), event.getSize(), event.getCreateDate(), null);
+ _usageVolumeDao.persist(volumeVO);
}
- for (UsageStorageVO volumesVO : volumesVOs) {
- if (logger.isDebugEnabled()) {
- logger.debug("deleting volume: " + volumesVO.getId() + " from account: " + volumesVO.getAccountId());
+ break;
+
+ case EventTypes.EVENT_VOLUME_RESIZE:
+ volumesVOs = _usageVolumeDao.listByVolumeId(volId, event.getAccountId());
+ for (UsageVolumeVO volumesVO : volumesVOs) {
+ String delete_msg = String.format("Setting the volume with id: %s to 'deleted' in the usage_storage table for account: %s.", volId, event.getAccountId());
Review Comment:
```suggestion
String delete_msg = String.format("Setting the volume with id: %s to 'deleted' in the usage_volume table for account: %s.", volId, event.getAccountId());
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3241984126
code looks good in isolation @abh1sar , but I see some bears on the road: what if someone upgrades from 20.1 to 21 and then on to 22; How do we guarantee they get the DB updates?
It may be wiser to put this in version 22.
Sorry, I overlooked this when I asked you to rebase earlier.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar closed pull request #11531: Track volume usage data at a vm granularity as well
URL: https://github.com/apache/cloudstack/pull/11531
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3517361468
@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland merged PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3393542845
<b>[SF] Trillian test result (tid-14634)</b>
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 52645 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr11531-t14634-kvm-ol8.zip
Smoke tests completed. 149 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "vishesh92 (via GitHub)" <gi...@apache.org>.
vishesh92 commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3351837091
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3352957918
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15241
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3376596405
@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3235962994
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14760
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3235825633
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland closed pull request #11531: Track volume usage data at a vm granularity as well
URL: https://github.com/apache/cloudstack/pull/11531
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#discussion_r2309313215
##########
usage/src/main/java/com/cloud/usage/UsageManagerImpl.java:
##########
@@ -1534,92 +1539,113 @@ private void createIPHelperEvent(UsageEventVO event) {
}
}
+ private void deleteExistingSecondaryStorageUsageForVolume(long volId, long accountId, Date deletedDate) {
+ List<UsageStorageVO> storageVOs = _usageStorageDao.listById(accountId, volId, StorageTypes.VOLUME);
+ for (UsageStorageVO storageVO : storageVOs) {
+ s_logger.debug(String.format("Setting the volume with id: {} to 'deleted' in the usage_storage table for account: {}.", volId, accountId));
+ storageVO.setDeleted(deletedDate);
+ _usageStorageDao.update(storageVO);
+ }
+ }
+
+ private void deleteExistingInstanceVolumeUsage(long volId, long accountId, Date deletedDate) {
+ List<UsageVolumeVO> volumesVOs = _usageVolumeDao.listByVolumeId(volId, accountId);
+ for (UsageVolumeVO volumesVO : volumesVOs) {
+ if (volumesVO.getVmId() != null) {
+ s_logger.debug(String.format("Setting the volume with id: {} for instance id: {} to 'deleted' in the usage_volume table for account {}.",
+ volumesVO.getVolumeId(), volumesVO.getVmId(), accountId));
+ volumesVO.setDeleted(deletedDate);
+ _usageVolumeDao.update(volumesVO.getId(), volumesVO);
+ }
+ }
+ }
+
+ private void deleteExistingVolumeUsage(long volId, long accountId, Date deletedDate) {
+ List<UsageVolumeVO> volumesVOs = _usageVolumeDao.listByVolumeId(volId, accountId);
+ for (UsageVolumeVO volumesVO : volumesVOs) {
+ s_logger.debug(String.format("Setting the volume with id: {} to 'deleted' in the usage_storage table for account: {}.", volId, accountId));
+ volumesVO.setDeleted(deletedDate);
+ _usageVolumeDao.update(volumesVO.getId(), volumesVO);
+ }
+ }
+
private void createVolumeHelperEvent(UsageEventVO event) {
long volId = event.getResourceId();
+ Account acct = _accountDao.findByIdIncludingRemoved(event.getAccountId());
+ List<UsageVolumeVO> volumesVOs;
+ UsageVolumeVO volumeVO;
- if (EventTypes.EVENT_VOLUME_CREATE.equals(event.getType())) {
- //For volumes which are 'attached' successfully, set the 'deleted' column in the usage_storage table,
+ switch (event.getType()) {
+ case EventTypes.EVENT_VOLUME_CREATE:
+ //For volumes which are 'attached' successfully from uploaded state, set the 'deleted' column in the usage_storage table,
//so that the secondary storage should stop accounting and only primary will be accounted.
- SearchCriteria<UsageStorageVO> sc = _usageStorageDao.createSearchCriteria();
- sc.addAnd("entityId", SearchCriteria.Op.EQ, volId);
- sc.addAnd("storageType", SearchCriteria.Op.EQ, StorageTypes.VOLUME);
- List<UsageStorageVO> volumesVOs = _usageStorageDao.search(sc, null);
- if (volumesVOs != null) {
- if (volumesVOs.size() == 1) {
- s_logger.debug("Setting the volume with id: " + volId + " to 'deleted' in the usage_storage table.");
- volumesVOs.get(0).setDeleted(event.getCreateDate());
- _usageStorageDao.update(volumesVOs.get(0));
- }
- }
- }
- if (EventTypes.EVENT_VOLUME_CREATE.equals(event.getType()) || EventTypes.EVENT_VOLUME_RESIZE.equals(event.getType())) {
- SearchCriteria<UsageVolumeVO> sc = _usageVolumeDao.createSearchCriteria();
- sc.addAnd("accountId", SearchCriteria.Op.EQ, event.getAccountId());
- sc.addAnd("volumeId", SearchCriteria.Op.EQ, volId);
- sc.addAnd("deleted", SearchCriteria.Op.NULL);
- List<UsageVolumeVO> volumesVOs = _usageVolumeDao.search(sc, null);
+ deleteExistingSecondaryStorageUsageForVolume(volId, event.getAccountId(), event.getCreateDate());
+
+ volumesVOs = _usageVolumeDao.listByVolumeId(volId, event.getAccountId());
if (volumesVOs.size() > 0) {
//This is a safeguard to avoid double counting of volumes.
s_logger.error("Found duplicate usage entry for volume: " + volId + " assigned to account: " + event.getAccountId() + "; marking as deleted...");
+ deleteExistingVolumeUsage(volId, event.getAccountId(), event.getCreateDate());
}
- //an entry exists if it is a resize volume event. marking the existing deleted and creating a new one in the case of resize.
- for (UsageVolumeVO volumesVO : volumesVOs) {
- if (s_logger.isDebugEnabled()) {
- s_logger.debug("deleting volume: " + volumesVO.getId() + " from account: " + volumesVO.getAccountId());
- }
- volumesVO.setDeleted(event.getCreateDate());
- _usageVolumeDao.update(volumesVO);
- }
- if (s_logger.isDebugEnabled()) {
- s_logger.debug("create volume with id : " + volId + " for account: " + event.getAccountId());
- }
- Account acct = _accountDao.findByIdIncludingRemoved(event.getAccountId());
- UsageVolumeVO volumeVO = new UsageVolumeVO(volId, event.getZoneId(), event.getAccountId(), acct.getDomainId(), event.getOfferingId(), event.getTemplateId(), event.getSize(), event.getCreateDate(), null);
+
+ s_logger.debug(String.format("Creating a new entry in usage_volume for volume with id: {} for account: {}", volId, event.getVmId(), event.getAccountId()));
+ volumeVO = new UsageVolumeVO(volId, event.getZoneId(), event.getAccountId(), acct.getDomainId(), event.getOfferingId(), event.getTemplateId(), null, event.getSize(), event.getCreateDate(), null);
_usageVolumeDao.persist(volumeVO);
- } else if (EventTypes.EVENT_VOLUME_DELETE.equals(event.getType())) {
- SearchCriteria<UsageVolumeVO> sc = _usageVolumeDao.createSearchCriteria();
- sc.addAnd("accountId", SearchCriteria.Op.EQ, event.getAccountId());
- sc.addAnd("volumeId", SearchCriteria.Op.EQ, volId);
- sc.addAnd("deleted", SearchCriteria.Op.NULL);
- List<UsageVolumeVO> volumesVOs = _usageVolumeDao.search(sc, null);
- if (volumesVOs.size() > 1) {
- s_logger.warn("More that one usage entry for volume: " + volId + " assigned to account: " + event.getAccountId() + "; marking them all as deleted...");
- }
- for (UsageVolumeVO volumesVO : volumesVOs) {
- if (s_logger.isDebugEnabled()) {
- s_logger.debug("deleting volume: " + volumesVO.getId() + " from account: " + volumesVO.getAccountId());
- }
- volumesVO.setDeleted(event.getCreateDate()); // there really shouldn't be more than one
- _usageVolumeDao.update(volumesVO);
- }
- } else if (EventTypes.EVENT_VOLUME_UPLOAD.equals(event.getType())) {
- //For Upload event add an entry to the usage_storage table.
- SearchCriteria<UsageStorageVO> sc = _usageStorageDao.createSearchCriteria();
- sc.addAnd("accountId", SearchCriteria.Op.EQ, event.getAccountId());
- sc.addAnd("entityId", SearchCriteria.Op.EQ, volId);
- sc.addAnd("storageType", SearchCriteria.Op.EQ, StorageTypes.VOLUME);
- sc.addAnd("deleted", SearchCriteria.Op.NULL);
- List<UsageStorageVO> volumesVOs = _usageStorageDao.search(sc, null);
- if (volumesVOs.size() > 0) {
- //This is a safeguard to avoid double counting of volumes.
- s_logger.error("Found duplicate usage entry for volume: " + volId + " assigned to account: " + event.getAccountId() + "; marking as deleted...");
+ if (event.getVmId() != null) {
Review Comment:
VOLUME_CREATE event will contain the vmId also when it is created during Instance deployment. Need to add a vm specific entry also in that case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3352642300
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3376592751
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3390734232
@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "harikrishna-patnala (via GitHub)" <gi...@apache.org>.
harikrishna-patnala commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3553101627
@DaanHoogland @abh1sar there are DB changes made in this PR in schema-42100to42200.sql file which should be in newer sql file, this needs fixing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#discussion_r2309251761
##########
engine/schema/src/main/java/com/cloud/usage/dao/UsageVolumeDaoImpl.java:
##########
@@ -18,83 +18,48 @@
import java.sql.PreparedStatement;
import java.sql.ResultSet;
-import java.sql.SQLException;
import java.util.ArrayList;
import java.util.Date;
import java.util.List;
import java.util.TimeZone;
-import com.cloud.exception.CloudException;
+import javax.annotation.PostConstruct;
+
import org.apache.log4j.Logger;
import org.springframework.stereotype.Component;
import com.cloud.usage.UsageVolumeVO;
import com.cloud.utils.DateUtil;
import com.cloud.utils.db.GenericDaoBase;
+import com.cloud.utils.db.SearchBuilder;
+import com.cloud.utils.db.SearchCriteria;
import com.cloud.utils.db.TransactionLegacy;
@Component
public class UsageVolumeDaoImpl extends GenericDaoBase<UsageVolumeVO, Long> implements UsageVolumeDao {
public static final Logger s_logger = Logger.getLogger(UsageVolumeDaoImpl.class.getName());
- protected static final String REMOVE_BY_USERID_VOLID = "DELETE FROM usage_volume WHERE account_id = ? AND volume_id = ?";
- protected static final String UPDATE_DELETED = "UPDATE usage_volume SET deleted = ? WHERE account_id = ? AND volume_id = ? and deleted IS NULL";
- protected static final String GET_USAGE_RECORDS_BY_ACCOUNT = "SELECT volume_id, zone_id, account_id, domain_id, disk_offering_id, template_id, size, created, deleted "
+ protected static final String GET_USAGE_RECORDS_BY_ACCOUNT = "SELECT volume_id, zone_id, account_id, domain_id, disk_offering_id, template_id, vm_id, size, created, deleted "
+ "FROM usage_volume " + "WHERE account_id = ? AND ((deleted IS NULL) OR (created BETWEEN ? AND ?) OR "
+ " (deleted BETWEEN ? AND ?) OR ((created <= ?) AND (deleted >= ?)))";
- protected static final String GET_USAGE_RECORDS_BY_DOMAIN = "SELECT volume_id, zone_id, account_id, domain_id, disk_offering_id, template_id, size, created, deleted "
+ protected static final String GET_USAGE_RECORDS_BY_DOMAIN = "SELECT volume_id, zone_id, account_id, domain_id, disk_offering_id, template_id, vm_id, size, created, deleted "
+ "FROM usage_volume " + "WHERE domain_id = ? AND ((deleted IS NULL) OR (created BETWEEN ? AND ?) OR "
+ " (deleted BETWEEN ? AND ?) OR ((created <= ?) AND (deleted >= ?)))";
- protected static final String GET_ALL_USAGE_RECORDS = "SELECT volume_id, zone_id, account_id, domain_id, disk_offering_id, template_id, size, created, deleted "
+ protected static final String GET_ALL_USAGE_RECORDS = "SELECT volume_id, zone_id, account_id, domain_id, disk_offering_id, template_id, vm_id, size, created, deleted "
+ "FROM usage_volume " + "WHERE (deleted IS NULL) OR (created BETWEEN ? AND ?) OR " + " (deleted BETWEEN ? AND ?) OR ((created <= ?) AND (deleted >= ?))";
+ private SearchBuilder<UsageVolumeVO> volumeSearch;
public UsageVolumeDaoImpl() {
}
- @Override
Review Comment:
This kind of update method is not required since https://github.com/apache/cloudstack/pull/5785 added auto-increment primary keys. We can directly use the GenericDao.update()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "vishesh92 (via GitHub)" <gi...@apache.org>.
vishesh92 commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3353170574
@blueorangutan test keepEnv
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3356633694
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#discussion_r2391243571
##########
usage/src/main/java/com/cloud/usage/UsageManagerImpl.java:
##########
@@ -1424,92 +1429,113 @@ private void createIPHelperEvent(UsageEventVO event) {
}
}
+ private void deleteExistingSecondaryStorageUsageForVolume(long volId, long accountId, Date deletedDate) {
+ List<UsageStorageVO> storageVOs = _usageStorageDao.listById(accountId, volId, StorageTypes.VOLUME);
+ for (UsageStorageVO storageVO : storageVOs) {
+ logger.debug(String.format("Setting the volume with id: {} to 'deleted' in the usage_storage table for account: {}.", volId, accountId));
+ storageVO.setDeleted(deletedDate);
+ _usageStorageDao.update(storageVO);
+ }
+ }
+
+ private void deleteExistingInstanceVolumeUsage(long volId, long accountId, Date deletedDate) {
+ List<UsageVolumeVO> volumesVOs = _usageVolumeDao.listByVolumeId(volId, accountId);
+ for (UsageVolumeVO volumesVO : volumesVOs) {
+ if (volumesVO.getVmId() != null) {
+ logger.debug(String.format("Setting the volume with id: {} for instance id: {} to 'deleted' in the usage_volume table for account {}.",
+ volumesVO.getVolumeId(), volumesVO.getVmId(), accountId));
+ volumesVO.setDeleted(deletedDate);
+ _usageVolumeDao.update(volumesVO.getId(), volumesVO);
+ }
+ }
+ }
+
+ private void deleteExistingVolumeUsage(long volId, long accountId, Date deletedDate) {
+ List<UsageVolumeVO> volumesVOs = _usageVolumeDao.listByVolumeId(volId, accountId);
+ for (UsageVolumeVO volumesVO : volumesVOs) {
+ logger.debug(String.format("Setting the volume with id: {} to 'deleted' in the usage_storage table for account: {}.", volId, accountId));
+ volumesVO.setDeleted(deletedDate);
+ _usageVolumeDao.update(volumesVO.getId(), volumesVO);
+ }
+ }
+
private void createVolumeHelperEvent(UsageEventVO event) {
long volId = event.getResourceId();
+ Account acct = _accountDao.findByIdIncludingRemoved(event.getAccountId());
+ List<UsageVolumeVO> volumesVOs;
+ UsageVolumeVO volumeVO;
- if (EventTypes.EVENT_VOLUME_CREATE.equals(event.getType())) {
- //For volumes which are 'attached' successfully, set the 'deleted' column in the usage_storage table,
+ switch (event.getType()) {
+ case EventTypes.EVENT_VOLUME_CREATE:
+ //For volumes which are 'attached' successfully from uploaded state, set the 'deleted' column in the usage_storage table,
//so that the secondary storage should stop accounting and only primary will be accounted.
- SearchCriteria<UsageStorageVO> sc = _usageStorageDao.createSearchCriteria();
- sc.addAnd("entityId", SearchCriteria.Op.EQ, volId);
- sc.addAnd("storageType", SearchCriteria.Op.EQ, StorageTypes.VOLUME);
- List<UsageStorageVO> volumesVOs = _usageStorageDao.search(sc, null);
- if (volumesVOs != null) {
- if (volumesVOs.size() == 1) {
- logger.debug("Setting the volume with id: " + volId + " to 'deleted' in the usage_storage table.");
- volumesVOs.get(0).setDeleted(event.getCreateDate());
- _usageStorageDao.update(volumesVOs.get(0));
- }
- }
- }
- if (EventTypes.EVENT_VOLUME_CREATE.equals(event.getType()) || EventTypes.EVENT_VOLUME_RESIZE.equals(event.getType())) {
- SearchCriteria<UsageVolumeVO> sc = _usageVolumeDao.createSearchCriteria();
- sc.addAnd("accountId", SearchCriteria.Op.EQ, event.getAccountId());
- sc.addAnd("volumeId", SearchCriteria.Op.EQ, volId);
- sc.addAnd("deleted", SearchCriteria.Op.NULL);
- List<UsageVolumeVO> volumesVOs = _usageVolumeDao.search(sc, null);
+ deleteExistingSecondaryStorageUsageForVolume(volId, event.getAccountId(), event.getCreateDate());
+
+ volumesVOs = _usageVolumeDao.listByVolumeId(volId, event.getAccountId());
if (volumesVOs.size() > 0) {
//This is a safeguard to avoid double counting of volumes.
logger.error("Found duplicate usage entry for volume: " + volId + " assigned to account: " + event.getAccountId() + "; marking as deleted...");
+ deleteExistingVolumeUsage(volId, event.getAccountId(), event.getCreateDate());
}
- //an entry exists if it is a resize volume event. marking the existing deleted and creating a new one in the case of resize.
- for (UsageVolumeVO volumesVO : volumesVOs) {
- if (logger.isDebugEnabled()) {
- logger.debug("deleting volume: " + volumesVO.getId() + " from account: " + volumesVO.getAccountId());
- }
- volumesVO.setDeleted(event.getCreateDate());
- _usageVolumeDao.update(volumesVO);
- }
- if (logger.isDebugEnabled()) {
- logger.debug("create volume with id : " + volId + " for account: " + event.getAccountId());
- }
- Account acct = _accountDao.findByIdIncludingRemoved(event.getAccountId());
- UsageVolumeVO volumeVO = new UsageVolumeVO(volId, event.getZoneId(), event.getAccountId(), acct.getDomainId(), event.getOfferingId(), event.getTemplateId(), event.getSize(), event.getCreateDate(), null);
+
+ logger.debug(String.format("Creating a new entry in usage_volume for volume with id: {} for account: {}", volId, event.getVmId(), event.getAccountId()));
+ volumeVO = new UsageVolumeVO(volId, event.getZoneId(), event.getAccountId(), acct.getDomainId(), event.getOfferingId(), event.getTemplateId(), null, event.getSize(), event.getCreateDate(), null);
_usageVolumeDao.persist(volumeVO);
- } else if (EventTypes.EVENT_VOLUME_DELETE.equals(event.getType())) {
- SearchCriteria<UsageVolumeVO> sc = _usageVolumeDao.createSearchCriteria();
- sc.addAnd("accountId", SearchCriteria.Op.EQ, event.getAccountId());
- sc.addAnd("volumeId", SearchCriteria.Op.EQ, volId);
- sc.addAnd("deleted", SearchCriteria.Op.NULL);
- List<UsageVolumeVO> volumesVOs = _usageVolumeDao.search(sc, null);
- if (volumesVOs.size() > 1) {
- logger.warn("More that one usage entry for volume: " + volId + " assigned to account: " + event.getAccountId() + "; marking them all as deleted...");
- }
- for (UsageVolumeVO volumesVO : volumesVOs) {
- if (logger.isDebugEnabled()) {
- logger.debug("deleting volume: " + volumesVO.getId() + " from account: " + volumesVO.getAccountId());
- }
- volumesVO.setDeleted(event.getCreateDate()); // there really shouldn't be more than one
- _usageVolumeDao.update(volumesVO);
- }
- } else if (EventTypes.EVENT_VOLUME_UPLOAD.equals(event.getType())) {
- //For Upload event add an entry to the usage_storage table.
- SearchCriteria<UsageStorageVO> sc = _usageStorageDao.createSearchCriteria();
- sc.addAnd("accountId", SearchCriteria.Op.EQ, event.getAccountId());
- sc.addAnd("entityId", SearchCriteria.Op.EQ, volId);
- sc.addAnd("storageType", SearchCriteria.Op.EQ, StorageTypes.VOLUME);
- sc.addAnd("deleted", SearchCriteria.Op.NULL);
- List<UsageStorageVO> volumesVOs = _usageStorageDao.search(sc, null);
- if (volumesVOs.size() > 0) {
- //This is a safeguard to avoid double counting of volumes.
- logger.error("Found duplicate usage entry for volume: " + volId + " assigned to account: " + event.getAccountId() + "; marking as deleted...");
+ if (event.getVmId() != null) {
+ volumeVO = new UsageVolumeVO(volId, event.getZoneId(), event.getAccountId(), acct.getDomainId(), event.getOfferingId(), event.getTemplateId(), event.getVmId(), event.getSize(), event.getCreateDate(), null);
+ _usageVolumeDao.persist(volumeVO);
}
- for (UsageStorageVO volumesVO : volumesVOs) {
- if (logger.isDebugEnabled()) {
- logger.debug("deleting volume: " + volumesVO.getId() + " from account: " + volumesVO.getAccountId());
+ break;
+
+ case EventTypes.EVENT_VOLUME_RESIZE:
+ volumesVOs = _usageVolumeDao.listByVolumeId(volId, event.getAccountId());
+ for (UsageVolumeVO volumesVO : volumesVOs) {
+ String delete_msg = String.format("Setting the volume with id: {} to 'deleted' in the usage_storage table for account: {}.", volId, event.getAccountId());
+ String create_msg = String.format("Creating a new entry in usage_volume for volume with id: {} after resize for account: {}", volId, event.getAccountId());
+ Long vmId = null;
+ if (vmId != null) {
+ vmId = volumesVO.getVmId();
+ delete_msg = String.format("Setting the volume with id: {} for instance id: {} to 'deleted' in the usage_storage table for account: {}.",
Review Comment:
The log message incorrectly states 'usage_storage table' but this is handling volume usage records in the 'usage_volume' table. Change 'usage_storage' to 'usage_volume' for accuracy.
```suggestion
delete_msg = String.format("Setting the volume with id: {} for instance id: {} to 'deleted' in the usage_volume table for account: {}.",
```
##########
usage/src/main/java/com/cloud/usage/UsageManagerImpl.java:
##########
@@ -1424,92 +1429,113 @@ private void createIPHelperEvent(UsageEventVO event) {
}
}
+ private void deleteExistingSecondaryStorageUsageForVolume(long volId, long accountId, Date deletedDate) {
+ List<UsageStorageVO> storageVOs = _usageStorageDao.listById(accountId, volId, StorageTypes.VOLUME);
+ for (UsageStorageVO storageVO : storageVOs) {
+ logger.debug(String.format("Setting the volume with id: {} to 'deleted' in the usage_storage table for account: {}.", volId, accountId));
+ storageVO.setDeleted(deletedDate);
+ _usageStorageDao.update(storageVO);
+ }
+ }
+
+ private void deleteExistingInstanceVolumeUsage(long volId, long accountId, Date deletedDate) {
+ List<UsageVolumeVO> volumesVOs = _usageVolumeDao.listByVolumeId(volId, accountId);
+ for (UsageVolumeVO volumesVO : volumesVOs) {
+ if (volumesVO.getVmId() != null) {
+ logger.debug(String.format("Setting the volume with id: {} for instance id: {} to 'deleted' in the usage_volume table for account {}.",
+ volumesVO.getVolumeId(), volumesVO.getVmId(), accountId));
+ volumesVO.setDeleted(deletedDate);
+ _usageVolumeDao.update(volumesVO.getId(), volumesVO);
+ }
+ }
+ }
+
+ private void deleteExistingVolumeUsage(long volId, long accountId, Date deletedDate) {
+ List<UsageVolumeVO> volumesVOs = _usageVolumeDao.listByVolumeId(volId, accountId);
+ for (UsageVolumeVO volumesVO : volumesVOs) {
+ logger.debug(String.format("Setting the volume with id: {} to 'deleted' in the usage_storage table for account: {}.", volId, accountId));
+ volumesVO.setDeleted(deletedDate);
+ _usageVolumeDao.update(volumesVO.getId(), volumesVO);
+ }
+ }
+
private void createVolumeHelperEvent(UsageEventVO event) {
long volId = event.getResourceId();
+ Account acct = _accountDao.findByIdIncludingRemoved(event.getAccountId());
+ List<UsageVolumeVO> volumesVOs;
+ UsageVolumeVO volumeVO;
- if (EventTypes.EVENT_VOLUME_CREATE.equals(event.getType())) {
- //For volumes which are 'attached' successfully, set the 'deleted' column in the usage_storage table,
+ switch (event.getType()) {
+ case EventTypes.EVENT_VOLUME_CREATE:
+ //For volumes which are 'attached' successfully from uploaded state, set the 'deleted' column in the usage_storage table,
//so that the secondary storage should stop accounting and only primary will be accounted.
- SearchCriteria<UsageStorageVO> sc = _usageStorageDao.createSearchCriteria();
- sc.addAnd("entityId", SearchCriteria.Op.EQ, volId);
- sc.addAnd("storageType", SearchCriteria.Op.EQ, StorageTypes.VOLUME);
- List<UsageStorageVO> volumesVOs = _usageStorageDao.search(sc, null);
- if (volumesVOs != null) {
- if (volumesVOs.size() == 1) {
- logger.debug("Setting the volume with id: " + volId + " to 'deleted' in the usage_storage table.");
- volumesVOs.get(0).setDeleted(event.getCreateDate());
- _usageStorageDao.update(volumesVOs.get(0));
- }
- }
- }
- if (EventTypes.EVENT_VOLUME_CREATE.equals(event.getType()) || EventTypes.EVENT_VOLUME_RESIZE.equals(event.getType())) {
- SearchCriteria<UsageVolumeVO> sc = _usageVolumeDao.createSearchCriteria();
- sc.addAnd("accountId", SearchCriteria.Op.EQ, event.getAccountId());
- sc.addAnd("volumeId", SearchCriteria.Op.EQ, volId);
- sc.addAnd("deleted", SearchCriteria.Op.NULL);
- List<UsageVolumeVO> volumesVOs = _usageVolumeDao.search(sc, null);
+ deleteExistingSecondaryStorageUsageForVolume(volId, event.getAccountId(), event.getCreateDate());
+
+ volumesVOs = _usageVolumeDao.listByVolumeId(volId, event.getAccountId());
if (volumesVOs.size() > 0) {
//This is a safeguard to avoid double counting of volumes.
logger.error("Found duplicate usage entry for volume: " + volId + " assigned to account: " + event.getAccountId() + "; marking as deleted...");
+ deleteExistingVolumeUsage(volId, event.getAccountId(), event.getCreateDate());
}
- //an entry exists if it is a resize volume event. marking the existing deleted and creating a new one in the case of resize.
- for (UsageVolumeVO volumesVO : volumesVOs) {
- if (logger.isDebugEnabled()) {
- logger.debug("deleting volume: " + volumesVO.getId() + " from account: " + volumesVO.getAccountId());
- }
- volumesVO.setDeleted(event.getCreateDate());
- _usageVolumeDao.update(volumesVO);
- }
- if (logger.isDebugEnabled()) {
- logger.debug("create volume with id : " + volId + " for account: " + event.getAccountId());
- }
- Account acct = _accountDao.findByIdIncludingRemoved(event.getAccountId());
- UsageVolumeVO volumeVO = new UsageVolumeVO(volId, event.getZoneId(), event.getAccountId(), acct.getDomainId(), event.getOfferingId(), event.getTemplateId(), event.getSize(), event.getCreateDate(), null);
+
+ logger.debug(String.format("Creating a new entry in usage_volume for volume with id: {} for account: {}", volId, event.getVmId(), event.getAccountId()));
+ volumeVO = new UsageVolumeVO(volId, event.getZoneId(), event.getAccountId(), acct.getDomainId(), event.getOfferingId(), event.getTemplateId(), null, event.getSize(), event.getCreateDate(), null);
_usageVolumeDao.persist(volumeVO);
- } else if (EventTypes.EVENT_VOLUME_DELETE.equals(event.getType())) {
- SearchCriteria<UsageVolumeVO> sc = _usageVolumeDao.createSearchCriteria();
- sc.addAnd("accountId", SearchCriteria.Op.EQ, event.getAccountId());
- sc.addAnd("volumeId", SearchCriteria.Op.EQ, volId);
- sc.addAnd("deleted", SearchCriteria.Op.NULL);
- List<UsageVolumeVO> volumesVOs = _usageVolumeDao.search(sc, null);
- if (volumesVOs.size() > 1) {
- logger.warn("More that one usage entry for volume: " + volId + " assigned to account: " + event.getAccountId() + "; marking them all as deleted...");
- }
- for (UsageVolumeVO volumesVO : volumesVOs) {
- if (logger.isDebugEnabled()) {
- logger.debug("deleting volume: " + volumesVO.getId() + " from account: " + volumesVO.getAccountId());
- }
- volumesVO.setDeleted(event.getCreateDate()); // there really shouldn't be more than one
- _usageVolumeDao.update(volumesVO);
- }
- } else if (EventTypes.EVENT_VOLUME_UPLOAD.equals(event.getType())) {
- //For Upload event add an entry to the usage_storage table.
- SearchCriteria<UsageStorageVO> sc = _usageStorageDao.createSearchCriteria();
- sc.addAnd("accountId", SearchCriteria.Op.EQ, event.getAccountId());
- sc.addAnd("entityId", SearchCriteria.Op.EQ, volId);
- sc.addAnd("storageType", SearchCriteria.Op.EQ, StorageTypes.VOLUME);
- sc.addAnd("deleted", SearchCriteria.Op.NULL);
- List<UsageStorageVO> volumesVOs = _usageStorageDao.search(sc, null);
- if (volumesVOs.size() > 0) {
- //This is a safeguard to avoid double counting of volumes.
- logger.error("Found duplicate usage entry for volume: " + volId + " assigned to account: " + event.getAccountId() + "; marking as deleted...");
+ if (event.getVmId() != null) {
+ volumeVO = new UsageVolumeVO(volId, event.getZoneId(), event.getAccountId(), acct.getDomainId(), event.getOfferingId(), event.getTemplateId(), event.getVmId(), event.getSize(), event.getCreateDate(), null);
+ _usageVolumeDao.persist(volumeVO);
}
- for (UsageStorageVO volumesVO : volumesVOs) {
- if (logger.isDebugEnabled()) {
- logger.debug("deleting volume: " + volumesVO.getId() + " from account: " + volumesVO.getAccountId());
+ break;
+
+ case EventTypes.EVENT_VOLUME_RESIZE:
+ volumesVOs = _usageVolumeDao.listByVolumeId(volId, event.getAccountId());
+ for (UsageVolumeVO volumesVO : volumesVOs) {
+ String delete_msg = String.format("Setting the volume with id: {} to 'deleted' in the usage_storage table for account: {}.", volId, event.getAccountId());
+ String create_msg = String.format("Creating a new entry in usage_volume for volume with id: {} after resize for account: {}", volId, event.getAccountId());
+ Long vmId = null;
+ if (vmId != null) {
Review Comment:
The condition `if (vmId != null)` will never be true because `vmId` is initialized to `null` on line 1497. This should be `if (volumesVO.getVmId() != null)` to properly check if the volume has a VM ID.
```suggestion
if (volumesVO.getVmId() != null) {
```
##########
usage/src/main/java/com/cloud/usage/UsageManagerImpl.java:
##########
@@ -1424,92 +1429,113 @@ private void createIPHelperEvent(UsageEventVO event) {
}
}
+ private void deleteExistingSecondaryStorageUsageForVolume(long volId, long accountId, Date deletedDate) {
+ List<UsageStorageVO> storageVOs = _usageStorageDao.listById(accountId, volId, StorageTypes.VOLUME);
+ for (UsageStorageVO storageVO : storageVOs) {
+ logger.debug(String.format("Setting the volume with id: {} to 'deleted' in the usage_storage table for account: {}.", volId, accountId));
+ storageVO.setDeleted(deletedDate);
+ _usageStorageDao.update(storageVO);
+ }
+ }
+
+ private void deleteExistingInstanceVolumeUsage(long volId, long accountId, Date deletedDate) {
+ List<UsageVolumeVO> volumesVOs = _usageVolumeDao.listByVolumeId(volId, accountId);
+ for (UsageVolumeVO volumesVO : volumesVOs) {
+ if (volumesVO.getVmId() != null) {
+ logger.debug(String.format("Setting the volume with id: {} for instance id: {} to 'deleted' in the usage_volume table for account {}.",
+ volumesVO.getVolumeId(), volumesVO.getVmId(), accountId));
+ volumesVO.setDeleted(deletedDate);
+ _usageVolumeDao.update(volumesVO.getId(), volumesVO);
+ }
+ }
+ }
+
+ private void deleteExistingVolumeUsage(long volId, long accountId, Date deletedDate) {
+ List<UsageVolumeVO> volumesVOs = _usageVolumeDao.listByVolumeId(volId, accountId);
+ for (UsageVolumeVO volumesVO : volumesVOs) {
+ logger.debug(String.format("Setting the volume with id: {} to 'deleted' in the usage_storage table for account: {}.", volId, accountId));
Review Comment:
The log message incorrectly states 'usage_storage table' but this method is updating the 'usage_volume' table. Change 'usage_storage' to 'usage_volume' for accuracy.
```suggestion
logger.debug(String.format("Setting the volume with id: {} to 'deleted' in the usage_volume table for account: {}.", volId, accountId));
```
##########
usage/src/main/java/com/cloud/usage/UsageManagerImpl.java:
##########
@@ -1424,92 +1429,113 @@ private void createIPHelperEvent(UsageEventVO event) {
}
}
+ private void deleteExistingSecondaryStorageUsageForVolume(long volId, long accountId, Date deletedDate) {
+ List<UsageStorageVO> storageVOs = _usageStorageDao.listById(accountId, volId, StorageTypes.VOLUME);
+ for (UsageStorageVO storageVO : storageVOs) {
+ logger.debug(String.format("Setting the volume with id: {} to 'deleted' in the usage_storage table for account: {}.", volId, accountId));
+ storageVO.setDeleted(deletedDate);
+ _usageStorageDao.update(storageVO);
+ }
+ }
+
+ private void deleteExistingInstanceVolumeUsage(long volId, long accountId, Date deletedDate) {
+ List<UsageVolumeVO> volumesVOs = _usageVolumeDao.listByVolumeId(volId, accountId);
+ for (UsageVolumeVO volumesVO : volumesVOs) {
+ if (volumesVO.getVmId() != null) {
+ logger.debug(String.format("Setting the volume with id: {} for instance id: {} to 'deleted' in the usage_volume table for account {}.",
+ volumesVO.getVolumeId(), volumesVO.getVmId(), accountId));
+ volumesVO.setDeleted(deletedDate);
+ _usageVolumeDao.update(volumesVO.getId(), volumesVO);
+ }
+ }
+ }
+
+ private void deleteExistingVolumeUsage(long volId, long accountId, Date deletedDate) {
+ List<UsageVolumeVO> volumesVOs = _usageVolumeDao.listByVolumeId(volId, accountId);
+ for (UsageVolumeVO volumesVO : volumesVOs) {
+ logger.debug(String.format("Setting the volume with id: {} to 'deleted' in the usage_storage table for account: {}.", volId, accountId));
+ volumesVO.setDeleted(deletedDate);
+ _usageVolumeDao.update(volumesVO.getId(), volumesVO);
+ }
+ }
+
private void createVolumeHelperEvent(UsageEventVO event) {
long volId = event.getResourceId();
+ Account acct = _accountDao.findByIdIncludingRemoved(event.getAccountId());
+ List<UsageVolumeVO> volumesVOs;
+ UsageVolumeVO volumeVO;
- if (EventTypes.EVENT_VOLUME_CREATE.equals(event.getType())) {
- //For volumes which are 'attached' successfully, set the 'deleted' column in the usage_storage table,
+ switch (event.getType()) {
+ case EventTypes.EVENT_VOLUME_CREATE:
+ //For volumes which are 'attached' successfully from uploaded state, set the 'deleted' column in the usage_storage table,
//so that the secondary storage should stop accounting and only primary will be accounted.
- SearchCriteria<UsageStorageVO> sc = _usageStorageDao.createSearchCriteria();
- sc.addAnd("entityId", SearchCriteria.Op.EQ, volId);
- sc.addAnd("storageType", SearchCriteria.Op.EQ, StorageTypes.VOLUME);
- List<UsageStorageVO> volumesVOs = _usageStorageDao.search(sc, null);
- if (volumesVOs != null) {
- if (volumesVOs.size() == 1) {
- logger.debug("Setting the volume with id: " + volId + " to 'deleted' in the usage_storage table.");
- volumesVOs.get(0).setDeleted(event.getCreateDate());
- _usageStorageDao.update(volumesVOs.get(0));
- }
- }
- }
- if (EventTypes.EVENT_VOLUME_CREATE.equals(event.getType()) || EventTypes.EVENT_VOLUME_RESIZE.equals(event.getType())) {
- SearchCriteria<UsageVolumeVO> sc = _usageVolumeDao.createSearchCriteria();
- sc.addAnd("accountId", SearchCriteria.Op.EQ, event.getAccountId());
- sc.addAnd("volumeId", SearchCriteria.Op.EQ, volId);
- sc.addAnd("deleted", SearchCriteria.Op.NULL);
- List<UsageVolumeVO> volumesVOs = _usageVolumeDao.search(sc, null);
+ deleteExistingSecondaryStorageUsageForVolume(volId, event.getAccountId(), event.getCreateDate());
+
+ volumesVOs = _usageVolumeDao.listByVolumeId(volId, event.getAccountId());
if (volumesVOs.size() > 0) {
//This is a safeguard to avoid double counting of volumes.
logger.error("Found duplicate usage entry for volume: " + volId + " assigned to account: " + event.getAccountId() + "; marking as deleted...");
+ deleteExistingVolumeUsage(volId, event.getAccountId(), event.getCreateDate());
}
- //an entry exists if it is a resize volume event. marking the existing deleted and creating a new one in the case of resize.
- for (UsageVolumeVO volumesVO : volumesVOs) {
- if (logger.isDebugEnabled()) {
- logger.debug("deleting volume: " + volumesVO.getId() + " from account: " + volumesVO.getAccountId());
- }
- volumesVO.setDeleted(event.getCreateDate());
- _usageVolumeDao.update(volumesVO);
- }
- if (logger.isDebugEnabled()) {
- logger.debug("create volume with id : " + volId + " for account: " + event.getAccountId());
- }
- Account acct = _accountDao.findByIdIncludingRemoved(event.getAccountId());
- UsageVolumeVO volumeVO = new UsageVolumeVO(volId, event.getZoneId(), event.getAccountId(), acct.getDomainId(), event.getOfferingId(), event.getTemplateId(), event.getSize(), event.getCreateDate(), null);
+
+ logger.debug(String.format("Creating a new entry in usage_volume for volume with id: {} for account: {}", volId, event.getVmId(), event.getAccountId()));
Review Comment:
The debug message has a formatting mismatch - it expects 2 parameters but provides 3 (`volId`, `event.getVmId()`, and `event.getAccountId()`). Remove `event.getVmId()` from the parameters to match the format string.
```suggestion
logger.debug(String.format("Creating a new entry in usage_volume for volume with id: {} for account: {}", volId, event.getAccountId()));
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#discussion_r2313252003
##########
engine/schema/src/main/java/com/cloud/upgrade/dao/Upgrade41930to41940.java:
##########
Review Comment:
rebased to 4.20
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3517355952
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3379449047
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3391402829
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15409
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3392745932
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3389915308
Verified the usage, event and volumes records when volumes are attached to & detached from VMs. LGTM.
```
mysql> SELECT * FROM cloud.usage_event WHERE type LIKE '%VOLUME%' AND resource_id IN (8, 10) ORDER BY id;
+----+---------------+------------+---------------------+---------+-------------+---------------+-------------+-------------+------------+---------------+-----------+--------------+-------+
| id | type | account_id | created | zone_id | resource_id | resource_name | offering_id | template_id | size | resource_type | processed | virtual_size | vm_id |
+----+---------------+------------+---------------------+---------+-------------+---------------+-------------+-------------+------------+---------------+-----------+--------------+-------+
| 12 | VOLUME.CREATE | 2 | 2025-10-09 09:55:02 | 1 | 8 | DATA02 | 6 | NULL | 3221225472 | NULL | 0 | NULL | NULL |
| 20 | VOLUME.CREATE | 2 | 2025-10-09 09:55:24 | 1 | 10 | DATA04 | 6 | NULL | 1073741824 | NULL | 0 | NULL | NULL |
| 22 | VOLUME.ATTACH | 2 | 2025-10-09 09:55:55 | 1 | 8 | DATA02 | 6 | NULL | 3221225472 | NULL | 0 | NULL | 3 |
| 24 | VOLUME.ATTACH | 2 | 2025-10-09 09:56:07 | 1 | 10 | DATA04 | 6 | NULL | 1073741824 | NULL | 0 | NULL | 6 |
| 33 | VOLUME.DETACH | 2 | 2025-10-09 10:31:13 | 1 | 8 | DATA02 | 6 | NULL | 3221225472 | NULL | 0 | NULL | NULL |
| 34 | VOLUME.DETACH | 2 | 2025-10-09 10:31:17 | 1 | 10 | DATA04 | 6 | NULL | 1073741824 | NULL | 0 | NULL | NULL |
| 37 | VOLUME.ATTACH | 2 | 2025-10-10 08:42:33 | 1 | 8 | DATA02 | 6 | NULL | 3221225472 | NULL | 0 | NULL | 7 |
| 38 | VOLUME.DETACH | 2 | 2025-10-10 08:55:37 | 1 | 8 | DATA02 | 6 | NULL | 3221225472 | NULL | 0 | NULL | NULL |
| 39 | VOLUME.ATTACH | 2 | 2025-10-10 09:00:39 | 1 | 10 | DATA04 | 6 | NULL | 1073741824 | NULL | 0 | NULL | 3 |
| 40 | VOLUME.ATTACH | 2 | 2025-10-10 09:01:39 | 1 | 8 | DATA02 | 6 | NULL | 3221225472 | NULL | 0 | NULL | 6 |
| 41 | VOLUME.DETACH | 2 | 2025-10-10 09:05:23 | 1 | 8 | DATA02 | 6 | NULL | 3221225472 | NULL | 0 | NULL | NULL |
| 42 | VOLUME.DETACH | 2 | 2025-10-10 09:05:31 | 1 | 10 | DATA04 | 6 | NULL | 1073741824 | NULL | 0 | NULL | NULL |
+----+---------------+------------+---------------------+---------+-------------+---------------+-------------+-------------+------------+---------------+-----------+--------------+-------+
12 rows in set (0.00 sec)
mysql> SELECT * FROM cloud_usage.usage_event WHERE type LIKE '%VOLUME%' AND resource_id IN (8, 10) ORDER BY id;
+----+---------------+------------+---------------------+---------+-------------+---------------+-------------+-------------+------------+---------------+-----------+--------------+-------+
| id | type | account_id | created | zone_id | resource_id | resource_name | offering_id | template_id | size | resource_type | processed | virtual_size | vm_id |
+----+---------------+------------+---------------------+---------+-------------+---------------+-------------+-------------+------------+---------------+-----------+--------------+-------+
| 12 | VOLUME.CREATE | 2 | 2025-10-09 09:55:02 | 1 | 8 | DATA02 | 6 | NULL | 3221225472 | NULL | 1 | NULL | NULL |
| 20 | VOLUME.CREATE | 2 | 2025-10-09 09:55:24 | 1 | 10 | DATA04 | 6 | NULL | 1073741824 | NULL | 1 | NULL | NULL |
| 22 | VOLUME.ATTACH | 2 | 2025-10-09 09:55:55 | 1 | 8 | DATA02 | 6 | NULL | 3221225472 | NULL | 1 | NULL | 3 |
| 24 | VOLUME.ATTACH | 2 | 2025-10-09 09:56:07 | 1 | 10 | DATA04 | 6 | NULL | 1073741824 | NULL | 1 | NULL | 6 |
| 33 | VOLUME.DETACH | 2 | 2025-10-09 10:31:13 | 1 | 8 | DATA02 | 6 | NULL | 3221225472 | NULL | 1 | NULL | NULL |
| 34 | VOLUME.DETACH | 2 | 2025-10-09 10:31:17 | 1 | 10 | DATA04 | 6 | NULL | 1073741824 | NULL | 1 | NULL | NULL |
| 37 | VOLUME.ATTACH | 2 | 2025-10-10 08:42:33 | 1 | 8 | DATA02 | 6 | NULL | 3221225472 | NULL | 1 | NULL | 7 |
| 38 | VOLUME.DETACH | 2 | 2025-10-10 08:55:37 | 1 | 8 | DATA02 | 6 | NULL | 3221225472 | NULL | 1 | NULL | NULL |
| 39 | VOLUME.ATTACH | 2 | 2025-10-10 09:00:39 | 1 | 10 | DATA04 | 6 | NULL | 1073741824 | NULL | 1 | NULL | 3 |
| 40 | VOLUME.ATTACH | 2 | 2025-10-10 09:01:39 | 1 | 8 | DATA02 | 6 | NULL | 3221225472 | NULL | 1 | NULL | 6 |
| 41 | VOLUME.DETACH | 2 | 2025-10-10 09:05:23 | 1 | 8 | DATA02 | 6 | NULL | 3221225472 | NULL | 1 | NULL | NULL |
| 42 | VOLUME.DETACH | 2 | 2025-10-10 09:05:31 | 1 | 10 | DATA04 | 6 | NULL | 1073741824 | NULL | 1 | NULL | NULL |
+----+---------------+------------+---------------------+---------+-------------+---------------+-------------+-------------+------------+---------------+-----------+--------------+-------+
12 rows in set (0.01 sec)
mysql> SELECT * FROM cloud_usage.usage_volume WHERE volume_id IN (8, 10) ORDER BY id;
+----+-----------+---------+------------+-----------+------------------+-------------+------------+---------------------+---------------------+-------+
| id | volume_id | zone_id | account_id | domain_id | disk_offering_id | template_id | size | created | deleted | vm_id |
+----+-----------+---------+------------+-----------+------------------+-------------+------------+---------------------+---------------------+-------+
| 8 | 8 | 1 | 2 | 1 | 6 | NULL | 3221225472 | 2025-10-09 09:55:02 | NULL | NULL |
| 10 | 10 | 1 | 2 | 1 | 6 | NULL | 1073741824 | 2025-10-09 09:55:24 | NULL | NULL |
| 12 | 8 | 1 | 2 | 1 | 6 | NULL | 3221225472 | 2025-10-09 09:55:55 | 2025-10-09 10:31:13 | 3 |
| 14 | 10 | 1 | 2 | 1 | 6 | NULL | 1073741824 | 2025-10-09 09:56:07 | 2025-10-09 10:31:17 | 6 |
| 63 | 8 | 1 | 2 | 1 | 6 | NULL | 3221225472 | 2025-10-10 08:42:33 | 2025-10-10 08:55:37 | 7 |
| 66 | 10 | 1 | 2 | 1 | 6 | NULL | 1073741824 | 2025-10-10 09:00:39 | 2025-10-10 09:05:31 | 3 |
| 67 | 8 | 1 | 2 | 1 | 6 | NULL | 3221225472 | 2025-10-10 09:01:39 | 2025-10-10 09:05:23 | 6 |
+----+-----------+---------+------------+-----------+------------------+-------------+------------+---------------------+---------------------+-------+
7 rows in set (0.00 sec)
```
Usage records:
```
(cmk) 🐱 > list usagerecords startdate=2025-10-07 enddate=2025-10-10 type=6 usageid=2c218702-abff-46e8-bc8a-3e12fffa2188
{
"count": 436,
"usagerecord": [
{
"account": "admin",
"accountid": "f709bdb4-a4f1-11f0-9b4e-1e00e6000172",
"description": "Volume usage for DATA02 (2c218702-abff-46e8-bc8a-3e12fffa2188) attached to VM testvm01 (282033c0-18df-46cd-809a-470f1a38a800) with disk offering Custom (b528eb0d-0efd-4df7-bffe-f0db170ac2e1) and size (3.00 GB) 3221225472",
"domain": "ROOT",
"domainid": "9f57bbb6-a4f1-11f0-9b4e-1e00e6000172",
"domainpath": "/",
"enddate": "2025-10-09T09:59:59+0000",
"offeringid": "b528eb0d-0efd-4df7-bffe-f0db170ac2e1",
"rawusage": "0.068056",
"size": 3221225472,
"startdate": "2025-10-09T09:00:00+0000",
"tags": [],
"usage": "0.068056 Hrs",
"usageid": "2c218702-abff-46e8-bc8a-3e12fffa2188",
"usagetype": 6,
"virtualmachineid": "282033c0-18df-46cd-809a-470f1a38a800",
"zoneid": "1f5131c0-cb48-4dc4-bb7c-33f8a0de2ec9"
},
...
{
"account": "admin",
"accountid": "f709bdb4-a4f1-11f0-9b4e-1e00e6000172",
"description": "Volume usage for DATA02 (2c218702-abff-46e8-bc8a-3e12fffa2188) with disk offering Custom (b528eb0d-0efd-4df7-bffe-f0db170ac2e1) and size (3.00 GB) 3221225472",
"domain": "ROOT",
"domainid": "9f57bbb6-a4f1-11f0-9b4e-1e00e6000172",
"domainpath": "/",
"enddate": "2025-10-10T07:59:59+0000",
"offeringid": "b528eb0d-0efd-4df7-bffe-f0db170ac2e1",
"rawusage": "1",
"size": 3221225472,
"startdate": "2025-10-10T07:00:00+0000",
"tags": [],
"usage": "1 Hrs",
"usageid": "2c218702-abff-46e8-bc8a-3e12fffa2188",
"usagetype": 6,
"zoneid": "1f5131c0-cb48-4dc4-bb7c-33f8a0de2ec9"
},
{
"account": "admin",
"accountid": "f709bdb4-a4f1-11f0-9b4e-1e00e6000172",
"description": "Volume usage for DATA02 (2c218702-abff-46e8-bc8a-3e12fffa2188) attached to VM testvm04 (332284cf-f56a-4075-adf5-3ac4f4fc8033) with disk offering Custom (b528eb0d-0efd-4df7-bffe-f0db170ac2e1) and size (3.00 GB) 3221225472",
"domain": "ROOT",
"domainid": "9f57bbb6-a4f1-11f0-9b4e-1e00e6000172",
"domainpath": "/",
"enddate": "2025-10-10T08:59:59+0000",
"offeringid": "b528eb0d-0efd-4df7-bffe-f0db170ac2e1",
"rawusage": "0.217778",
"size": 3221225472,
"startdate": "2025-10-10T08:00:00+0000",
"tags": [],
"usage": "0.217778 Hrs",
"usageid": "2c218702-abff-46e8-bc8a-3e12fffa2188",
"usagetype": 6,
"virtualmachineid": "332284cf-f56a-4075-adf5-3ac4f4fc8033",
"zoneid": "1f5131c0-cb48-4dc4-bb7c-33f8a0de2ec9"
},
{
"account": "admin",
"accountid": "f709bdb4-a4f1-11f0-9b4e-1e00e6000172",
"description": "Volume usage for DATA02 (2c218702-abff-46e8-bc8a-3e12fffa2188) with disk offering Custom (b528eb0d-0efd-4df7-bffe-f0db170ac2e1) and size (3.00 GB) 3221225472",
"domain": "ROOT",
"domainid": "9f57bbb6-a4f1-11f0-9b4e-1e00e6000172",
"domainpath": "/",
"enddate": "2025-10-10T08:59:59+0000",
"offeringid": "b528eb0d-0efd-4df7-bffe-f0db170ac2e1",
"rawusage": "1",
"size": 3221225472,
"startdate": "2025-10-10T08:00:00+0000",
"tags": [],
"usage": "1 Hrs",
"usageid": "2c218702-abff-46e8-bc8a-3e12fffa2188",
"usagetype": 6,
"zoneid": "1f5131c0-cb48-4dc4-bb7c-33f8a0de2ec9"
},
{
"account": "admin",
"accountid": "f709bdb4-a4f1-11f0-9b4e-1e00e6000172",
"description": "Volume usage for DATA02 (2c218702-abff-46e8-bc8a-3e12fffa2188) attached to VM testvm02 (ca3309cd-15e7-4718-9b33-d2c3af12ecd8) with disk offering Custom (b528eb0d-0efd-4df7-bffe-f0db170ac2e1) and size (3.00 GB) 3221225472",
"domain": "ROOT",
"domainid": "9f57bbb6-a4f1-11f0-9b4e-1e00e6000172",
"domainpath": "/",
"enddate": "2025-10-10T09:59:59+0000",
"offeringid": "b528eb0d-0efd-4df7-bffe-f0db170ac2e1",
"rawusage": "0.062222",
"size": 3221225472,
"startdate": "2025-10-10T09:00:00+0000",
"tags": [],
"usage": "0.062222 Hrs",
"usageid": "2c218702-abff-46e8-bc8a-3e12fffa2188",
"usagetype": 6,
"virtualmachineid": "ca3309cd-15e7-4718-9b33-d2c3af12ecd8",
"zoneid": "1f5131c0-cb48-4dc4-bb7c-33f8a0de2ec9"
},
{
"account": "admin",
"accountid": "f709bdb4-a4f1-11f0-9b4e-1e00e6000172",
"description": "Volume usage for DATA02 (2c218702-abff-46e8-bc8a-3e12fffa2188) with disk offering Custom (b528eb0d-0efd-4df7-bffe-f0db170ac2e1) and size (3.00 GB) 3221225472",
"domain": "ROOT",
"domainid": "9f57bbb6-a4f1-11f0-9b4e-1e00e6000172",
"domainpath": "/",
"enddate": "2025-10-10T09:59:59+0000",
"offeringid": "b528eb0d-0efd-4df7-bffe-f0db170ac2e1",
"rawusage": "1",
"size": 3221225472,
"startdate": "2025-10-10T09:00:00+0000",
"tags": [],
"usage": "1 Hrs",
"usageid": "2c218702-abff-46e8-bc8a-3e12fffa2188",
"usagetype": 6,
"zoneid": "1f5131c0-cb48-4dc4-bb7c-33f8a0de2ec9"
},
{
"account": "admin",
"accountid": "f709bdb4-a4f1-11f0-9b4e-1e00e6000172",
"description": "Volume usage for DATA02 (2c218702-abff-46e8-bc8a-3e12fffa2188) with disk offering Custom (b528eb0d-0efd-4df7-bffe-f0db170ac2e1) and size (3.00 GB) 3221225472",
"domain": "ROOT",
"domainid": "9f57bbb6-a4f1-11f0-9b4e-1e00e6000172",
"domainpath": "/",
"enddate": "2025-10-10T10:59:59+0000",
"offeringid": "b528eb0d-0efd-4df7-bffe-f0db170ac2e1",
"rawusage": "1",
"size": 3221225472,
"startdate": "2025-10-10T10:00:00+0000",
"tags": [],
"usage": "1 Hrs",
"usageid": "2c218702-abff-46e8-bc8a-3e12fffa2188",
"usagetype": 6,
"zoneid": "1f5131c0-cb48-4dc4-bb7c-33f8a0de2ec9"
},
{
"account": "admin",
"accountid": "f709bdb4-a4f1-11f0-9b4e-1e00e6000172",
"description": "Volume usage for DATA02 (2c218702-abff-46e8-bc8a-3e12fffa2188) with disk offering Custom (b528eb0d-0efd-4df7-bffe-f0db170ac2e1) and size (3.00 GB) 3221225472",
"domain": "ROOT",
"domainid": "9f57bbb6-a4f1-11f0-9b4e-1e00e6000172",
"domainpath": "/",
"enddate": "2025-10-10T11:59:59+0000",
"offeringid": "b528eb0d-0efd-4df7-bffe-f0db170ac2e1",
"rawusage": "1",
"size": 3221225472,
"startdate": "2025-10-10T11:00:00+0000",
"tags": [],
"usage": "1 Hrs",
"usageid": "2c218702-abff-46e8-bc8a-3e12fffa2188",
"usagetype": 6,
"zoneid": "1f5131c0-cb48-4dc4-bb7c-33f8a0de2ec9"
}
]
}
(cmk) 🐱 > list usagerecords startdate=2025-10-07 enddate=2025-10-10 type=6 usageid=6c766c5c-1ab8-4428-a3ba-4b9a17b84502
{
"count": 432,
"usagerecord": [
{
"account": "admin",
"accountid": "f709bdb4-a4f1-11f0-9b4e-1e00e6000172",
"description": "Volume usage for DATA04 (6c766c5c-1ab8-4428-a3ba-4b9a17b84502) attached to VM testvm02 (ca3309cd-15e7-4718-9b33-d2c3af12ecd8) with disk offering Custom (b528eb0d-0efd-4df7-bffe-f0db170ac2e1) and size (1.00 GB) 1073741824",
"domain": "ROOT",
"domainid": "9f57bbb6-a4f1-11f0-9b4e-1e00e6000172",
"domainpath": "/",
"enddate": "2025-10-09T09:59:59+0000",
"offeringid": "b528eb0d-0efd-4df7-bffe-f0db170ac2e1",
"rawusage": "0.064722",
"size": 1073741824,
"startdate": "2025-10-09T09:00:00+0000",
"tags": [],
"usage": "0.064722 Hrs",
"usageid": "6c766c5c-1ab8-4428-a3ba-4b9a17b84502",
"usagetype": 6,
"virtualmachineid": "ca3309cd-15e7-4718-9b33-d2c3af12ecd8",
"zoneid": "1f5131c0-cb48-4dc4-bb7c-33f8a0de2ec9"
},
...
{
"account": "admin",
"accountid": "f709bdb4-a4f1-11f0-9b4e-1e00e6000172",
"description": "Volume usage for DATA04 (6c766c5c-1ab8-4428-a3ba-4b9a17b84502) with disk offering Custom (b528eb0d-0efd-4df7-bffe-f0db170ac2e1) and size (1.00 GB) 1073741824",
"domain": "ROOT",
"domainid": "9f57bbb6-a4f1-11f0-9b4e-1e00e6000172",
"domainpath": "/",
"enddate": "2025-10-10T08:59:59+0000",
"offeringid": "b528eb0d-0efd-4df7-bffe-f0db170ac2e1",
"rawusage": "1",
"size": 1073741824,
"startdate": "2025-10-10T08:00:00+0000",
"tags": [],
"usage": "1 Hrs",
"usageid": "6c766c5c-1ab8-4428-a3ba-4b9a17b84502",
"usagetype": 6,
"zoneid": "1f5131c0-cb48-4dc4-bb7c-33f8a0de2ec9"
},
{
"account": "admin",
"accountid": "f709bdb4-a4f1-11f0-9b4e-1e00e6000172",
"description": "Volume usage for DATA04 (6c766c5c-1ab8-4428-a3ba-4b9a17b84502) attached to VM testvm01 (282033c0-18df-46cd-809a-470f1a38a800) with disk offering Custom (b528eb0d-0efd-4df7-bffe-f0db170ac2e1) and size (1.00 GB) 1073741824",
"domain": "ROOT",
"domainid": "9f57bbb6-a4f1-11f0-9b4e-1e00e6000172",
"domainpath": "/",
"enddate": "2025-10-10T09:59:59+0000",
"offeringid": "b528eb0d-0efd-4df7-bffe-f0db170ac2e1",
"rawusage": "0.081111",
"size": 1073741824,
"startdate": "2025-10-10T09:00:00+0000",
"tags": [],
"usage": "0.081111 Hrs",
"usageid": "6c766c5c-1ab8-4428-a3ba-4b9a17b84502",
"usagetype": 6,
"virtualmachineid": "282033c0-18df-46cd-809a-470f1a38a800",
"zoneid": "1f5131c0-cb48-4dc4-bb7c-33f8a0de2ec9"
},
{
"account": "admin",
"accountid": "f709bdb4-a4f1-11f0-9b4e-1e00e6000172",
"description": "Volume usage for DATA04 (6c766c5c-1ab8-4428-a3ba-4b9a17b84502) with disk offering Custom (b528eb0d-0efd-4df7-bffe-f0db170ac2e1) and size (1.00 GB) 1073741824",
"domain": "ROOT",
"domainid": "9f57bbb6-a4f1-11f0-9b4e-1e00e6000172",
"domainpath": "/",
"enddate": "2025-10-10T09:59:59+0000",
"offeringid": "b528eb0d-0efd-4df7-bffe-f0db170ac2e1",
"rawusage": "1",
"size": 1073741824,
"startdate": "2025-10-10T09:00:00+0000",
"tags": [],
"usage": "1 Hrs",
"usageid": "6c766c5c-1ab8-4428-a3ba-4b9a17b84502",
"usagetype": 6,
"zoneid": "1f5131c0-cb48-4dc4-bb7c-33f8a0de2ec9"
},
{
"account": "admin",
"accountid": "f709bdb4-a4f1-11f0-9b4e-1e00e6000172",
"description": "Volume usage for DATA04 (6c766c5c-1ab8-4428-a3ba-4b9a17b84502) with disk offering Custom (b528eb0d-0efd-4df7-bffe-f0db170ac2e1) and size (1.00 GB) 1073741824",
"domain": "ROOT",
"domainid": "9f57bbb6-a4f1-11f0-9b4e-1e00e6000172",
"domainpath": "/",
"enddate": "2025-10-10T10:59:59+0000",
"offeringid": "b528eb0d-0efd-4df7-bffe-f0db170ac2e1",
"rawusage": "1",
"size": 1073741824,
"startdate": "2025-10-10T10:00:00+0000",
"tags": [],
"usage": "1 Hrs",
"usageid": "6c766c5c-1ab8-4428-a3ba-4b9a17b84502",
"usagetype": 6,
"zoneid": "1f5131c0-cb48-4dc4-bb7c-33f8a0de2ec9"
},
{
"account": "admin",
"accountid": "f709bdb4-a4f1-11f0-9b4e-1e00e6000172",
"description": "Volume usage for DATA04 (6c766c5c-1ab8-4428-a3ba-4b9a17b84502) with disk offering Custom (b528eb0d-0efd-4df7-bffe-f0db170ac2e1) and size (1.00 GB) 1073741824",
"domain": "ROOT",
"domainid": "9f57bbb6-a4f1-11f0-9b4e-1e00e6000172",
"domainpath": "/",
"enddate": "2025-10-10T11:59:59+0000",
"offeringid": "b528eb0d-0efd-4df7-bffe-f0db170ac2e1",
"rawusage": "1",
"size": 1073741824,
"startdate": "2025-10-10T11:00:00+0000",
"tags": [],
"usage": "1 Hrs",
"usageid": "6c766c5c-1ab8-4428-a3ba-4b9a17b84502",
"usagetype": 6,
"zoneid": "1f5131c0-cb48-4dc4-bb7c-33f8a0de2ec9"
}
]
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3390725401
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3354821288
@vishesh92 a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3352167335
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3355223960
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#discussion_r2313252805
##########
engine/schema/src/main/resources/META-INF/db/schema-41930to41940.sql:
##########
Review Comment:
added to file schema-42010to42020.sql
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3235836225
## [Codecov](https://app.codecov.io/gh/apache/cloudstack/pull/11531?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 4.28%. Comparing base ([`25f93b1`](https://app.codecov.io/gh/apache/cloudstack/commit/25f93b1d6b83acbdd14ba942c2f4086e9fdfd888?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cfe9746`](https://app.codecov.io/gh/apache/cloudstack/commit/cfe9746e3fd59e4ea5d89a277a585bcc83719919?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
> :exclamation: There is a different number of reports uploaded between BASE (25f93b1) and HEAD (cfe9746). Click for more details.
>
> <details><summary>HEAD has 1 upload less than BASE</summary>
>
>| Flag | BASE (25f93b1) | HEAD (cfe9746) |
>|------|------|------|
>|unittests|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## 4.19 #11531 +/- ##
=============================================
- Coverage 15.18% 4.28% -10.90%
=============================================
Files 5415 372 -5043
Lines 476066 29745 -446321
Branches 58123 5229 -52894
=============================================
- Hits 72281 1275 -71006
+ Misses 395698 28324 -367374
+ Partials 8087 146 -7941
```
| [Flag](https://app.codecov.io/gh/apache/cloudstack/pull/11531/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [uitests](https://app.codecov.io/gh/apache/cloudstack/pull/11531/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `4.28% <ø> (ø)` | |
| [unittests](https://app.codecov.io/gh/apache/cloudstack/pull/11531/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/cloudstack/pull/11531?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3332796444
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3353330789
<b>[SF] Trillian Build Failed (tid-14490)<b/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3356616362
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3517229777
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15717
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on code in PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#discussion_r2348098564
##########
server/src/main/java/com/cloud/api/ApiResponseHelper.java:
##########
@@ -4330,6 +4330,9 @@ public UsageRecordResponse createUsageResponse(Usage usageRecord, Map<String, Se
if (volume != null) {
builder.append("for ").append(volume.getName()).append(" (").append(volume.getUuid()).append(")");
}
+ if (vmInstance != null) {
+ builder.append(" attached to VM ").append(vmInstance.getHostName()).append(" (").append(vmInstance.getUuid()).append(")");
+ }
Review Comment:
this is adding only three lines to a 530 line method. No issue in the context but `public UsageRecordResponse createUsageResponse(Usage usageRecord)` needs to be refactorred (as many pieces of code).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3379446529
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3377006142
Packaging result [SF]: ✖️ el8 ✖️ el9 ✔️ debian ✖️ suse15. SL-JID 15320
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar closed pull request #11531: Track volume usage data at a vm granularity as well
URL: https://github.com/apache/cloudstack/pull/11531
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3358398612
<b>[SF] Trillian test result (tid-14492)</b>
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 58325 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr11531-t14492-kvm-ol8.zip
Smoke tests completed. 146 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_02_upgrade_kubernetes_cluster | `Failure` | 334.74 | test_kubernetes_clusters.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3355541498
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15249
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3379590171
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15331
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3516898298
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3516906453
@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3352650333
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Track volume usage data at a vm granularity as well [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #11531:
URL: https://github.com/apache/cloudstack/pull/11531#issuecomment-3392748046
@abh1sar a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org