You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by "sureshanaparti (via GitHub)" <gi...@apache.org> on 2026/01/19 12:37:11 UTC
[PR] Update physical size for the snapshots on ceph primary storage [cloudstack]
sureshanaparti opened a new pull request, #12465:
URL: https://github.com/apache/cloudstack/pull/12465
### Description
This PR updates the physical size of the snapshots on ceph primary storage.
<!--- Describe your changes in DETAIL - And how has behaviour functionally changed. -->
<!-- For new features, provide link to FS, dev ML discussion etc. -->
<!-- In case of bug fix, the expected and actual behaviours, steps to reproduce. -->
<!-- When "Fixes: #<id>" is specified, the issue/PR will automatically be closed when this PR gets merged -->
<!-- For addressing multiple issues/PRs, use multiple "Fixes: #<id>" -->
<!-- Fixes: # -->
<!--- ******************************************************************************* -->
<!--- NOTE: AUTOMATION USES THE DESCRIPTIONS TO SET LABELS AND PRODUCE DOCUMENTATION. -->
<!--- PLEASE PUT AN 'X' in only **ONE** box -->
<!--- ******************************************************************************* -->
### Types of changes
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
- [ ] Build/CI
- [ ] Test (unit or integration test code)
### Feature/Enhancement Scale or Bug Severity
#### Feature/Enhancement Scale
- [ ] Major
- [x] Minor
#### Bug Severity
- [ ] BLOCKER
- [ ] Critical
- [ ] Major
- [ ] Minor
- [ ] Trivial
### Screenshots (if appropriate):
### How Has This Been Tested?
Tested the snapshots for the volumes on ceph primary storage with the config 'snapshot.backup.to.secondary' - true & false. Notice the physical size is properly set in both the cases.
<img width="1450" height="643" alt="SnapshotsWithCephPrimaryStorage" src="https://github.com/user-attachments/assets/0c22b170-fcc8-406c-99c8-bd7c02e862ed" />
DB:
```
mysql> SELECT id, uuid, name, pool_id, instance_id, size, state, created FROM cloud.volumes WHERE id IN (8,9);
+----+--------------------------------------+--------+---------+-------------+------------+-------+---------------------+
| id | uuid | name | pool_id | instance_id | size | state | created |
+----+--------------------------------------+--------+---------+-------------+------------+-------+---------------------+
| 8 | e29110ce-ffab-4a27-b2ad-6d87905039d8 | ROOT-8 | 2 | 8 | 8589934592 | Ready | 2026-01-16 07:48:17 |
| 9 | d2a9e77a-9a41-4427-8511-f8c365d8b581 | ROOT-9 | 2 | 9 | 8589934592 | Ready | 2026-01-19 10:57:20 |
+----+--------------------------------------+--------+---------+-------------+------------+-------+---------------------+
2 rows in set (0.00 sec)
mysql> SELECT id, uuid, name, volume_id, size, status, created FROM cloud.snapshots WHERE removed IS NULL AND volume_id IN (8,9);
+----+--------------------------------------+-------------------+-----------+------------+----------+---------------------+
| id | uuid | name | volume_id | size | status | created |
+----+--------------------------------------+-------------------+-----------+------------+----------+---------------------+
| 1 | 55b8b9c4-0779-4614-8cfe-505d345f9e16 | ROOT-8-Snap01 | 8 | 8589934592 | BackedUp | 2026-01-16 08:03:57 |
| 2 | 9e6f7ef9-c278-4e1a-a6dc-5ad575c295b3 | ROOT-8-Snap02-Pri | 8 | 8589934592 | BackedUp | 2026-01-16 08:08:38 |
| 3 | 2eb7aef8-6b91-4cce-bf42-11756639f9f4 | ROOT-8-Snap03-Pri | 8 | 8589934592 | BackedUp | 2026-01-16 09:26:47 |
| 4 | f076ad82-e189-425e-857f-76a678622519 | ROOT-8-Snap04-Pri | 8 | 8589934592 | BackedUp | 2026-01-16 14:46:33 |
| 5 | 3319a00c-b22e-466d-93df-657bca3f75b3 | ROOT-8-Snap05-Pri | 8 | 8589934592 | BackedUp | 2026-01-16 21:19:59 |
| 6 | cab273ac-128b-4c22-a026-88afbaf1c617 | ROOT-8-Snap06-Pri | 8 | 8589934592 | BackedUp | 2026-01-16 22:07:08 |
| 7 | a645f0eb-bcb5-4207-983d-ceaa20586910 | ROOT-8-Snap07-Pri | 8 | 8589934592 | BackedUp | 2026-01-16 22:31:25 |
| 8 | 46798347-044a-482d-bbc8-ca9ef4d2b5e6 | ROOT-8-Snap08-Pri | 8 | 8589934592 | BackedUp | 2026-01-19 08:47:20 |
| 9 | 59719b8b-9d6f-431c-9f16-1979ee1742a6 | ROOT-8-Snap09-Sec | 8 | 8589934592 | BackedUp | 2026-01-19 08:52:45 |
| 10 | 269cf711-b766-4128-afb4-3d7eee137b80 | ROOT-8-Snap10-Sec | 8 | 8589934592 | BackedUp | 2026-01-19 08:54:31 |
| 11 | 63cae549-4e88-41a6-bebb-d75675d573b3 | ROOT-9-Snap01 | 9 | 8589934592 | BackedUp | 2026-01-19 11:02:51 |
| 12 | 5f7120f8-3adc-465d-92eb-608604f2ec75 | ROOT-9-Snap02-Pri | 9 | 8589934592 | BackedUp | 2026-01-19 11:07:21 |
+----+--------------------------------------+-------------------+-----------+------------+----------+---------------------+
12 rows in set (0.00 sec)
mysql> SELECT ssref.volume_id, s.name, snapshot_id, store_id, store_role,ssref.size, physical_size, install_path, state, ssref.created FROM cloud.snapshot_store_ref ssref JOIN cloud.snapshots s ON ssref.snapshot_id = s.id WHERE s.volume_id IN (8,9);
+-----------+-------------------+-------------+----------+------------+------------+---------------+--------------------------------------------------------------------------------------+-------+---------------------+
| volume_id | name | snapshot_id | store_id | store_role | size | physical_size | install_path | state | created |
+-----------+-------------------+-------------+----------+------------+------------+---------------+--------------------------------------------------------------------------------------+-------+---------------------+
| 8 | ROOT-8-Snap01 | 1 | 2 | Primary | 8589934592 | 8589934592 | cloudstack/e29110ce-ffab-4a27-b2ad-6d87905039d8/1d27eef6-2e9d-4456-a0e9-953a3bb3bcd1 | Ready | 2026-01-16 08:03:59 |
| 8 | ROOT-8-Snap01 | 1 | 1 | Image | 8589934592 | 1673134080 | snapshots/2/8/1d27eef6-2e9d-4456-a0e9-953a3bb3bcd1 | Ready | 2026-01-16 08:04:00 |
| 8 | ROOT-8-Snap02-Pri | 2 | 2 | Primary | 8589934592 | 8589934592 | cloudstack/e29110ce-ffab-4a27-b2ad-6d87905039d8/5cc25049-375f-4299-a541-ac2bd7ea0654 | Ready | 2026-01-16 08:08:40 |
| 8 | ROOT-8-Snap03-Pri | 3 | 2 | Primary | 8589934592 | 8589934592 | cloudstack/e29110ce-ffab-4a27-b2ad-6d87905039d8/082fa72d-2ec9-44e2-aea5-c13b87decf70 | Ready | 2026-01-16 09:26:48 |
| 8 | ROOT-8-Snap04-Pri | 4 | 2 | Primary | 8589934592 | 8589934592 | cloudstack/e29110ce-ffab-4a27-b2ad-6d87905039d8/d2e2d6d5-ff66-46f1-be7b-76e0504e965d | Ready | 2026-01-16 14:46:34 |
| 8 | ROOT-8-Snap05-Pri | 5 | 2 | Primary | 8589934592 | 8589934592 | cloudstack/e29110ce-ffab-4a27-b2ad-6d87905039d8/08453f18-c3f7-41db-bf9b-b6f87427726e | Ready | 2026-01-16 21:20:00 |
| 8 | ROOT-8-Snap06-Pri | 6 | 2 | Primary | 8589934592 | 8589934592 | cloudstack/e29110ce-ffab-4a27-b2ad-6d87905039d8/672c796a-edd5-45fd-ab5a-60283e5ce67a | Ready | 2026-01-16 22:07:10 |
| 8 | ROOT-8-Snap07-Pri | 7 | 2 | Primary | 8589934592 | 12582912 | cloudstack/e29110ce-ffab-4a27-b2ad-6d87905039d8/e06f1646-0561-411e-ab69-cbb1679bcd0b | Ready | 2026-01-16 22:31:26 |
| 8 | ROOT-8-Snap08-Pri | 8 | 2 | Primary | 8589934592 | 633339904 | cloudstack/e29110ce-ffab-4a27-b2ad-6d87905039d8/dce4c78e-04cc-40d7-a5c1-dec6e834c32b | Ready | 2026-01-19 08:47:22 |
| 8 | ROOT-8-Snap09-Sec | 9 | 2 | Primary | 8589934592 | 230686720 | cloudstack/e29110ce-ffab-4a27-b2ad-6d87905039d8/8af3d334-b609-4baf-88ef-a73505953217 | Ready | 2026-01-19 08:52:46 |
| 8 | ROOT-8-Snap09-Sec | 9 | 1 | Image | 8589934592 | 1791361024 | snapshots/2/8/8af3d334-b609-4baf-88ef-a73505953217 | Ready | 2026-01-19 08:53:17 |
| 8 | ROOT-8-Snap10-Sec | 10 | 2 | Primary | 8589934592 | 222298112 | cloudstack/e29110ce-ffab-4a27-b2ad-6d87905039d8/ddcc1b75-57e9-4f9a-b09a-34e784752543 | Ready | 2026-01-19 08:54:32 |
| 8 | ROOT-8-Snap10-Sec | 10 | 1 | Image | 8589934592 | 1793064960 | snapshots/2/8/ddcc1b75-57e9-4f9a-b09a-34e784752543 | Ready | 2026-01-19 08:55:03 |
| 9 | ROOT-9-Snap01 | 11 | 2 | Primary | 8589934592 | 247463936 | cloudstack/d2a9e77a-9a41-4427-8511-f8c365d8b581/f169b209-c16c-4691-ba04-52fb3699320b | Ready | 2026-01-19 11:02:52 |
| 9 | ROOT-9-Snap01 | 11 | 1 | Image | 8589934592 | 1673003008 | snapshots/4/9/f169b209-c16c-4691-ba04-52fb3699320b | Ready | 2026-01-19 11:02:53 |
| 9 | ROOT-9-Snap02-Pri | 12 | 2 | Primary | 8589934592 | 20971520 | cloudstack/d2a9e77a-9a41-4427-8511-f8c365d8b581/35d0b7f7-fbc9-405b-8796-903656085293 | Ready | 2026-01-19 11:07:23 |
+-----------+-------------------+-------------+----------+------------+------------+---------------+--------------------------------------------------------------------------------------+-------+---------------------+
16 rows in set (0.00 sec)
```
Ceph Dashboard:
<img width="1442" height="664" alt="CephVolume_e29110ce-ffab-4a27-b2ad-6d87905039d8_Snapshots" src="https://github.com/user-attachments/assets/fb97bcb6-0522-4dc7-99da-4e8eff07ea25" />
<img width="1444" height="393" alt="CephVolume_d2a9e77a-9a41-4427-8511-f8c365d8b581_Snapshots" src="https://github.com/user-attachments/assets/dc19001b-bb81-4c0f-b252-5eadc8f13cfa" />
Ceph cmd output:
```
root@ceph1:~# rbd du cloudstack/e29110ce-ffab-4a27-b2ad-6d87905039d8
NAME PROVISIONED USED
e29110ce-ffab-4a27-b2ad-6d87905039d8@1d27eef6-2e9d-4456-a0e9-953a3bb3bcd1 8 GiB 1.4 GiB
e29110ce-ffab-4a27-b2ad-6d87905039d8@5cc25049-375f-4299-a541-ac2bd7ea0654 8 GiB 12 MiB
e29110ce-ffab-4a27-b2ad-6d87905039d8@082fa72d-2ec9-44e2-aea5-c13b87decf70 8 GiB 760 MiB
e29110ce-ffab-4a27-b2ad-6d87905039d8@d2e2d6d5-ff66-46f1-be7b-76e0504e965d 8 GiB 96 MiB
e29110ce-ffab-4a27-b2ad-6d87905039d8@08453f18-c3f7-41db-bf9b-b6f87427726e 8 GiB 96 MiB
e29110ce-ffab-4a27-b2ad-6d87905039d8@672c796a-edd5-45fd-ab5a-60283e5ce67a 8 GiB 84 MiB
e29110ce-ffab-4a27-b2ad-6d87905039d8@e06f1646-0561-411e-ab69-cbb1679bcd0b 8 GiB 12 MiB
e29110ce-ffab-4a27-b2ad-6d87905039d8@dce4c78e-04cc-40d7-a5c1-dec6e834c32b 8 GiB 604 MiB
e29110ce-ffab-4a27-b2ad-6d87905039d8@8af3d334-b609-4baf-88ef-a73505953217 8 GiB 220 MiB
e29110ce-ffab-4a27-b2ad-6d87905039d8@ddcc1b75-57e9-4f9a-b09a-34e784752543 8 GiB 212 MiB
e29110ce-ffab-4a27-b2ad-6d87905039d8 8 GiB 144 MiB
<TOTAL> 8 GiB 3.6 GiB
root@ceph1:~# rbd du cloudstack/d2a9e77a-9a41-4427-8511-f8c365d8b581
NAME PROVISIONED USED
d2a9e77a-9a41-4427-8511-f8c365d8b581@f169b209-c16c-4691-ba04-52fb3699320b 8 GiB 696 MiB
d2a9e77a-9a41-4427-8511-f8c365d8b581@35d0b7f7-fbc9-405b-8796-903656085293 8 GiB 20 MiB
d2a9e77a-9a41-4427-8511-f8c365d8b581 8 GiB 700 MiB
<TOTAL> 8 GiB 1.4 GiB
```
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
#### How did you try to break this feature and the system with this change?
<!-- see how your change affects other areas of the code, etc. -->
<!-- Please read the [CONTRIBUTING](https://github.com/apache/cloudstack/blob/main/CONTRIBUTING.md) document -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update physical size for the snapshots on ceph primary storage [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on code in PR #12465:
URL: https://github.com/apache/cloudstack/pull/12465#discussion_r2704732115
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java:
##########
@@ -1873,6 +1882,31 @@ public Answer createSnapshot(final CreateObjectCommand cmd) {
}
}
+ private long getRdbSnapshotSize(String poolPath, String diskName, String snapshotName, String rbdMonitor, String authUser, String authSecret) {
+ logger.debug("Get RBD snapshot size for {}/{}@{}", poolPath, diskName, snapshotName);
+ //cmd: rbd du <pool>/<disk-name>@<snapshot-name> --format json --mon-host <monitor-host> --id <user> --key <key> 2>/dev/null
+ String snapshotDetailsInJson = Script.runSimpleBashScript(String.format("rbd du %s/%s@%s --format json --mon-host %s --id %s --key %s 2>/dev/null", poolPath, diskName, snapshotName, rbdMonitor, authUser, authSecret));
+ if (StringUtils.isNotBlank(snapshotDetailsInJson)) {
+ ObjectMapper mapper = new ObjectMapper();
+ try {
+ JsonNode root = mapper.readTree(snapshotDetailsInJson);
+ for (JsonNode image : root.path("images")) {
+ if (snapshotName.equals(image.path("snapshot").asText())) {
+ long usedSizeInBytes = image.path("used_size").asLong();
+ if (usedSizeInBytes > 0) {
+ logger.debug("RBD snapshot {}/{}@{} used size in bytes: {}", poolPath, diskName, snapshotName, usedSizeInBytes);
+ return usedSizeInBytes;
+ }
+ }
+ }
+ } catch (JsonProcessingException e) {
+ logger.error("Unable to get the RBD snapshot size", e);
+ }
+ }
+
+ return 0;
+ }
Review Comment:
couldn't get the physical size details with RbdImage, RbdSnapInfo.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update physical size for the snapshots on ceph primary storage [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on code in PR #12465:
URL: https://github.com/apache/cloudstack/pull/12465#discussion_r2704734064
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java:
##########
@@ -43,6 +43,10 @@
import javax.naming.ConfigurationException;
+import com.ceph.rbd.jna.RbdImageInfo;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.cloudstack.agent.directdownload.DirectDownloadAnswer;
import org.apache.cloudstack.agent.directdownload.DirectDownloadCommand;
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update physical size for the snapshots on ceph primary storage [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on code in PR #12465:
URL: https://github.com/apache/cloudstack/pull/12465#discussion_r2704733271
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java:
##########
@@ -1873,6 +1882,31 @@ public Answer createSnapshot(final CreateObjectCommand cmd) {
}
}
+ private long getRdbSnapshotSize(String poolPath, String diskName, String snapshotName, String rbdMonitor, String authUser, String authSecret) {
+ logger.debug("Get RBD snapshot size for {}/{}@{}", poolPath, diskName, snapshotName);
+ //cmd: rbd du <pool>/<disk-name>@<snapshot-name> --format json --mon-host <monitor-host> --id <user> --key <key> 2>/dev/null
+ String snapshotDetailsInJson = Script.runSimpleBashScript(String.format("rbd du %s/%s@%s --format json --mon-host %s --id %s --key %s 2>/dev/null", poolPath, diskName, snapshotName, rbdMonitor, authUser, authSecret));
+ if (StringUtils.isNotBlank(snapshotDetailsInJson)) {
+ ObjectMapper mapper = new ObjectMapper();
+ try {
+ JsonNode root = mapper.readTree(snapshotDetailsInJson);
+ for (JsonNode image : root.path("images")) {
+ if (snapshotName.equals(image.path("snapshot").asText())) {
+ long usedSizeInBytes = image.path("used_size").asLong();
+ if (usedSizeInBytes > 0) {
+ logger.debug("RBD snapshot {}/{}@{} used size in bytes: {}", poolPath, diskName, snapshotName, usedSizeInBytes);
+ return usedSizeInBytes;
+ }
+ }
+ }
+ } catch (JsonProcessingException e) {
+ logger.error("Unable to get the RBD snapshot size", e);
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update physical size for the snapshots on ceph primary storage [cloudstack]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #12465:
URL: https://github.com/apache/cloudstack/pull/12465#issuecomment-3768165503
## [Codecov](https://app.codecov.io/gh/apache/cloudstack/pull/12465?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 4.03%. Comparing base ([`f1f779a`](https://app.codecov.io/gh/apache/cloudstack/commit/f1f779a08d94037baf1aad0c8107eaaf219af80e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cd2ad36`](https://app.codecov.io/gh/apache/cloudstack/commit/cd2ad36eb0d3d0982355fe9f2b5768615b31485c?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 2 commits behind head on 4.20.
> :exclamation: There is a different number of reports uploaded between BASE (f1f779a) and HEAD (cd2ad36). Click for more details.
>
> <details><summary>HEAD has 1 upload less than BASE</summary>
>
>| Flag | BASE (f1f779a) | HEAD (cd2ad36) |
>|------|------|------|
>|unittests|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## 4.20 #12465 +/- ##
=============================================
- Coverage 16.23% 4.03% -12.21%
=============================================
Files 5657 402 -5255
Lines 499024 32712 -466312
Branches 60562 5831 -54731
=============================================
- Hits 81029 1319 -79710
+ Misses 408959 31238 -377721
+ Partials 9036 155 -8881
```
| [Flag](https://app.codecov.io/gh/apache/cloudstack/pull/12465/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [uitests](https://app.codecov.io/gh/apache/cloudstack/pull/12465/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `4.03% <ø> (-0.01%)` | :arrow_down: |
| [unittests](https://app.codecov.io/gh/apache/cloudstack/pull/12465/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/cloudstack/pull/12465?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update physical size for the snapshots of the volumes on ceph primary storage [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland merged PR #12465:
URL: https://github.com/apache/cloudstack/pull/12465
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update physical size for the snapshots on ceph primary storage [cloudstack]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #12465:
URL: https://github.com/apache/cloudstack/pull/12465#discussion_r2704662833
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java:
##########
@@ -1873,6 +1882,31 @@ public Answer createSnapshot(final CreateObjectCommand cmd) {
}
}
+ private long getRdbSnapshotSize(String poolPath, String diskName, String snapshotName, String rbdMonitor, String authUser, String authSecret) {
+ logger.debug("Get RBD snapshot size for {}/{}@{}", poolPath, diskName, snapshotName);
+ //cmd: rbd du <pool>/<disk-name>@<snapshot-name> --format json --mon-host <monitor-host> --id <user> --key <key> 2>/dev/null
+ String snapshotDetailsInJson = Script.runSimpleBashScript(String.format("rbd du %s/%s@%s --format json --mon-host %s --id %s --key %s 2>/dev/null", poolPath, diskName, snapshotName, rbdMonitor, authUser, authSecret));
+ if (StringUtils.isNotBlank(snapshotDetailsInJson)) {
+ ObjectMapper mapper = new ObjectMapper();
+ try {
+ JsonNode root = mapper.readTree(snapshotDetailsInJson);
+ for (JsonNode image : root.path("images")) {
+ if (snapshotName.equals(image.path("snapshot").asText())) {
+ long usedSizeInBytes = image.path("used_size").asLong();
+ if (usedSizeInBytes > 0) {
+ logger.debug("RBD snapshot {}/{}@{} used size in bytes: {}", poolPath, diskName, snapshotName, usedSizeInBytes);
+ return usedSizeInBytes;
+ }
+ }
+ }
+ } catch (JsonProcessingException e) {
+ logger.error("Unable to get the RBD snapshot size", e);
+ }
Review Comment:
When the rbd command fails or returns null/empty output, the method silently returns 0 without logging any details about the command execution failure. This makes it difficult to distinguish between a genuine 0-size snapshot and a failed command. Consider adding a debug or warning log when snapshotDetailsInJson is blank to help troubleshoot issues.
```suggestion
}
} else {
logger.warn("Failed to get RBD snapshot size for {}/{}@{}: 'rbd du' returned no output or an empty result", poolPath, diskName, snapshotName);
```
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java:
##########
@@ -43,6 +43,10 @@
import javax.naming.ConfigurationException;
+import com.ceph.rbd.jna.RbdImageInfo;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.cloudstack.agent.directdownload.DirectDownloadAnswer;
import org.apache.cloudstack.agent.directdownload.DirectDownloadCommand;
Review Comment:
The import 'com.ceph.rbd.jna.RbdImageInfo' is added but never used in the code. Remove this unused import to keep the code clean.
```suggestion
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.cloudstack.agent.directdownload.DirectDownloadAnswer;
import org.apache.cloudstack.agent.directdownload.DirectDownloadCommand;
import org.apache.cloudstack.agent.directdownload.DirectDownloadCommand;
```
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java:
##########
@@ -1873,6 +1882,31 @@ public Answer createSnapshot(final CreateObjectCommand cmd) {
}
}
+ private long getRdbSnapshotSize(String poolPath, String diskName, String snapshotName, String rbdMonitor, String authUser, String authSecret) {
Review Comment:
Method name has a typo: 'getRdbSnapshotSize' should be 'getRbdSnapshotSize' (RBD stands for RADOS Block Device). This inconsistency makes the code harder to maintain and understand.
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java:
##########
@@ -1873,6 +1882,31 @@ public Answer createSnapshot(final CreateObjectCommand cmd) {
}
}
+ private long getRdbSnapshotSize(String poolPath, String diskName, String snapshotName, String rbdMonitor, String authUser, String authSecret) {
+ logger.debug("Get RBD snapshot size for {}/{}@{}", poolPath, diskName, snapshotName);
+ //cmd: rbd du <pool>/<disk-name>@<snapshot-name> --format json --mon-host <monitor-host> --id <user> --key <key> 2>/dev/null
+ String snapshotDetailsInJson = Script.runSimpleBashScript(String.format("rbd du %s/%s@%s --format json --mon-host %s --id %s --key %s 2>/dev/null", poolPath, diskName, snapshotName, rbdMonitor, authUser, authSecret));
Review Comment:
The command construction using String.format with user-controlled parameters (poolPath, diskName, snapshotName, rbdMonitor, authUser, authSecret) is vulnerable to command injection. If any of these parameters contain shell metacharacters like semicolons, pipes, backticks, or dollar signs, they could be used to execute arbitrary commands. Consider using a more secure approach such as:
1. Using Script.runSimpleBashScriptWithParameters or similar method that properly escapes arguments
2. Validating and sanitizing input parameters before using them in shell commands
3. Using the Java RBD bindings (like the existing Rbd/RbdImage classes) instead of shell commands
```suggestion
String snapshotDetailsInJson = Script.runSimpleBashScriptWithParameters(
"rbd du %s/%s@%s --format json --mon-host %s --id %s --key %s 2>/dev/null",
poolPath, diskName, snapshotName, rbdMonitor, authUser, authSecret);
```
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java:
##########
@@ -1873,6 +1882,31 @@ public Answer createSnapshot(final CreateObjectCommand cmd) {
}
}
+ private long getRdbSnapshotSize(String poolPath, String diskName, String snapshotName, String rbdMonitor, String authUser, String authSecret) {
+ logger.debug("Get RBD snapshot size for {}/{}@{}", poolPath, diskName, snapshotName);
+ //cmd: rbd du <pool>/<disk-name>@<snapshot-name> --format json --mon-host <monitor-host> --id <user> --key <key> 2>/dev/null
+ String snapshotDetailsInJson = Script.runSimpleBashScript(String.format("rbd du %s/%s@%s --format json --mon-host %s --id %s --key %s 2>/dev/null", poolPath, diskName, snapshotName, rbdMonitor, authUser, authSecret));
+ if (StringUtils.isNotBlank(snapshotDetailsInJson)) {
+ ObjectMapper mapper = new ObjectMapper();
+ try {
+ JsonNode root = mapper.readTree(snapshotDetailsInJson);
+ for (JsonNode image : root.path("images")) {
+ if (snapshotName.equals(image.path("snapshot").asText())) {
+ long usedSizeInBytes = image.path("used_size").asLong();
+ if (usedSizeInBytes > 0) {
+ logger.debug("RBD snapshot {}/{}@{} used size in bytes: {}", poolPath, diskName, snapshotName, usedSizeInBytes);
+ return usedSizeInBytes;
+ }
+ }
+ }
+ } catch (JsonProcessingException e) {
+ logger.error("Unable to get the RBD snapshot size", e);
Review Comment:
The error handling silently logs an error and returns 0 when JSON parsing fails. This makes debugging difficult and could mask underlying issues. Consider logging the actual JSON content that failed to parse (taking care not to log sensitive credentials) to help with troubleshooting.
```suggestion
logger.error("Unable to get the RBD snapshot size. RBD JSON output: {}", snapshotDetailsInJson, e);
```
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java:
##########
@@ -1873,6 +1882,31 @@ public Answer createSnapshot(final CreateObjectCommand cmd) {
}
}
+ private long getRdbSnapshotSize(String poolPath, String diskName, String snapshotName, String rbdMonitor, String authUser, String authSecret) {
+ logger.debug("Get RBD snapshot size for {}/{}@{}", poolPath, diskName, snapshotName);
+ //cmd: rbd du <pool>/<disk-name>@<snapshot-name> --format json --mon-host <monitor-host> --id <user> --key <key> 2>/dev/null
+ String snapshotDetailsInJson = Script.runSimpleBashScript(String.format("rbd du %s/%s@%s --format json --mon-host %s --id %s --key %s 2>/dev/null", poolPath, diskName, snapshotName, rbdMonitor, authUser, authSecret));
+ if (StringUtils.isNotBlank(snapshotDetailsInJson)) {
+ ObjectMapper mapper = new ObjectMapper();
+ try {
+ JsonNode root = mapper.readTree(snapshotDetailsInJson);
+ for (JsonNode image : root.path("images")) {
+ if (snapshotName.equals(image.path("snapshot").asText())) {
+ long usedSizeInBytes = image.path("used_size").asLong();
+ if (usedSizeInBytes > 0) {
+ logger.debug("RBD snapshot {}/{}@{} used size in bytes: {}", poolPath, diskName, snapshotName, usedSizeInBytes);
+ return usedSizeInBytes;
+ }
+ }
+ }
+ } catch (JsonProcessingException e) {
+ logger.error("Unable to get the RBD snapshot size", e);
+ }
+ }
+
+ return 0;
+ }
Review Comment:
The method uses shell commands to get RBD snapshot size, while the createSnapshot method already uses Java RBD bindings (Rbd, RbdImage). Consider using RbdImage.stat() or similar Java API methods to get snapshot size information instead of relying on shell commands. This would be more consistent with the existing code, avoid command injection risks, and be more maintainable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update physical size for the snapshots on ceph primary storage [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on PR #12465:
URL: https://github.com/apache/cloudstack/pull/12465#issuecomment-3768333041
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update physical size for the snapshots on ceph primary storage [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on code in PR #12465:
URL: https://github.com/apache/cloudstack/pull/12465#discussion_r2704726474
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java:
##########
@@ -43,6 +43,10 @@
import javax.naming.ConfigurationException;
+import com.ceph.rbd.jna.RbdImageInfo;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.cloudstack.agent.directdownload.DirectDownloadAnswer;
import org.apache.cloudstack.agent.directdownload.DirectDownloadCommand;
Review Comment:
checkstyle doesn’t like you @sureshanaparti (or your code), but don’t apply it lit co-pilot suggests!!! You’d get double imports, that way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update physical size for the snapshots of the volumes on ceph primary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12465:
URL: https://github.com/apache/cloudstack/pull/12465#issuecomment-3771134237
<b>[SF] Trillian test result (tid-15212)</b>
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 49996 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12465-t15212-kvm-ol8.zip
Smoke tests completed. 141 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update physical size for the snapshots on ceph primary storage [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on code in PR #12465:
URL: https://github.com/apache/cloudstack/pull/12465#discussion_r2704738727
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java:
##########
@@ -1873,6 +1882,31 @@ public Answer createSnapshot(final CreateObjectCommand cmd) {
}
}
+ private long getRdbSnapshotSize(String poolPath, String diskName, String snapshotName, String rbdMonitor, String authUser, String authSecret) {
+ logger.debug("Get RBD snapshot size for {}/{}@{}", poolPath, diskName, snapshotName);
+ //cmd: rbd du <pool>/<disk-name>@<snapshot-name> --format json --mon-host <monitor-host> --id <user> --key <key> 2>/dev/null
+ String snapshotDetailsInJson = Script.runSimpleBashScript(String.format("rbd du %s/%s@%s --format json --mon-host %s --id %s --key %s 2>/dev/null", poolPath, diskName, snapshotName, rbdMonitor, authUser, authSecret));
+ if (StringUtils.isNotBlank(snapshotDetailsInJson)) {
+ ObjectMapper mapper = new ObjectMapper();
+ try {
+ JsonNode root = mapper.readTree(snapshotDetailsInJson);
+ for (JsonNode image : root.path("images")) {
+ if (snapshotName.equals(image.path("snapshot").asText())) {
+ long usedSizeInBytes = image.path("used_size").asLong();
+ if (usedSizeInBytes > 0) {
+ logger.debug("RBD snapshot {}/{}@{} used size in bytes: {}", poolPath, diskName, snapshotName, usedSizeInBytes);
+ return usedSizeInBytes;
+ }
+ }
+ }
+ } catch (JsonProcessingException e) {
+ logger.error("Unable to get the RBD snapshot size", e);
+ }
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update physical size for the snapshots of the volumes on ceph primary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12465:
URL: https://github.com/apache/cloudstack/pull/12465#issuecomment-3768609589
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16423
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update physical size for the snapshots of the volumes on ceph primary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12465:
URL: https://github.com/apache/cloudstack/pull/12465#issuecomment-3768769335
@sureshanaparti a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update physical size for the snapshots on ceph primary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12465:
URL: https://github.com/apache/cloudstack/pull/12465#issuecomment-3768316584
@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update physical size for the snapshots of the volumes on ceph primary storage [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on PR #12465:
URL: https://github.com/apache/cloudstack/pull/12465#issuecomment-3768764062
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update physical size for the snapshots on ceph primary storage [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on PR #12465:
URL: https://github.com/apache/cloudstack/pull/12465#issuecomment-3768306388
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Update physical size for the snapshots on ceph primary storage [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12465:
URL: https://github.com/apache/cloudstack/pull/12465#issuecomment-3768344371
@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org