You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by "Pearl1594 (via GitHub)" <gi...@apache.org> on 2026/02/09 19:47:42 UTC
[PR] CLVM enhancements and fixes [cloudstack]
Pearl1594 opened a new pull request, #12617:
URL: https://github.com/apache/cloudstack/pull/12617
### Description
This PR...
<!--- Describe your changes in DETAIL - And how has behaviour functionally changed. -->
<!-- For new features, provide link to FS, dev ML discussion etc. -->
<!-- In case of bug fix, the expected and actual behaviours, steps to reproduce. -->
<!-- When "Fixes: #<id>" is specified, the issue/PR will automatically be closed when this PR gets merged -->
<!-- For addressing multiple issues/PRs, use multiple "Fixes: #<id>" -->
<!-- Fixes: # -->
<!--- ******************************************************************************* -->
<!--- NOTE: AUTOMATION USES THE DESCRIPTIONS TO SET LABELS AND PRODUCE DOCUMENTATION. -->
<!--- PLEASE PUT AN 'X' in only **ONE** box -->
<!--- ******************************************************************************* -->
### Types of changes
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
- [ ] Build/CI
- [ ] Test (unit or integration test code)
### Feature/Enhancement Scale or Bug Severity
#### Feature/Enhancement Scale
- [ ] Major
- [ ] Minor
#### Bug Severity
- [ ] BLOCKER
- [ ] Critical
- [ ] Major
- [ ] Minor
- [ ] Trivial
### Screenshots (if appropriate):
### How Has This Been Tested?
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
#### How did you try to break this feature and the system with this change?
<!-- see how your change affects other areas of the code, etc. -->
<!-- Please read the [CONTRIBUTING](https://github.com/apache/cloudstack/blob/main/CONTRIBUTING.md) document -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-3946433600
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [2.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4093308926
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [8.5% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4168131299
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4169464788
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-3915636633
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4400191677
@Pearl1594 a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4405363613
<b>[SF] Trillian test result (tid-16043)</b>
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 49742 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12617-t16043-kvm-ol8.zip
Smoke tests completed. 151 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4390278059
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17742
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4391456649
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4391909294
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17743
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4444110871
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4444464809
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17847
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-3887033862
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4637607129
<b>[SF] Trillian test result (tid-16254)</b>
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 49433 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12617-t16254-kvm-ol8.zip
Smoke tests completed. 151 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4333437210
<b>[SF] Trillian test result (tid-15956)</b>
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 48926 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12617-t15956-kvm-ol8.zip
Smoke tests completed. 151 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r3162729179
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java:
##########
@@ -6686,4 +6692,242 @@ public String getHypervisorPath() {
public String getGuestCpuArch() {
return guestCpuArch;
}
+
+ /**
+ * CLVM volume state for migration operations on source host
+ */
+ public enum ClvmVolumeState {
+ /** Shared mode (-asy) - used before migration to allow both hosts to access volume */
+ SHARED("-asy", "shared", "Before migration: activating in shared mode"),
+
+ /** Deactivate (-an) - used after successful migration to release volume on source */
+ DEACTIVATE("-an", "deactivated", "After successful migration: deactivating volume"),
+
+ /** Exclusive mode (-aey) - used after failed migration to revert to original exclusive state */
+ EXCLUSIVE("-aey", "exclusive", "After failed migration: reverting to exclusive mode");
+
+ private final String lvchangeFlag;
+ private final String description;
+ private final String logMessage;
+
+ ClvmVolumeState(String lvchangeFlag, String description, String logMessage) {
+ this.lvchangeFlag = lvchangeFlag;
+ this.description = description;
+ this.logMessage = logMessage;
+ }
+
+ public String getLvchangeFlag() {
+ return lvchangeFlag;
+ }
+
+ public String getDescription() {
+ return description;
+ }
+
+ public String getLogMessage() {
+ return logMessage;
+ }
+ }
+
+ public static void modifyClvmVolumesStateForMigration(List<DiskDef> disks, LibvirtComputingResource resource,
+ VirtualMachineTO vmSpec, ClvmVolumeState state) {
+ for (DiskDef disk : disks) {
+ if (isClvmVolume(disk, resource, vmSpec)) {
+ String volumePath = disk.getDiskPath();
+ try {
+ modifyClvmVolumeState(volumePath, state.getLvchangeFlag(), state.getDescription(), state.getLogMessage());
+ } catch (Exception e) {
+ LOGGER.error("[CLVM Migration] Exception while setting volume [{}] to {} state: {}",
+ volumePath, state.getDescription(), e.getMessage(), e);
+ }
+ }
+ }
+ }
+
+ private static void modifyClvmVolumeState(String volumePath, String lvchangeFlag,
+ String stateDescription, String logMessage) {
+ try {
+ LOGGER.info("{} for volume [{}]", logMessage, volumePath);
+
+ Script cmd = new Script("lvchange", Duration.standardSeconds(300), LOGGER);
+ cmd.add(lvchangeFlag);
+ cmd.add(volumePath);
+
+ String result = cmd.execute();
+ if (result != null) {
+ String errorMsg = String.format(
+ "Failed to set volume [%s] to %s state. Command result: %s",
+ volumePath, stateDescription, result);
+ LOGGER.error(errorMsg);
+ throw new CloudRuntimeException(errorMsg);
+ } else {
+ LOGGER.info("Successfully set volume [{}] to {} state.",
+ volumePath, stateDescription);
+ }
+ } catch (CloudRuntimeException e) {
+ throw e;
+ } catch (Exception e) {
+ String errorMsg = String.format(
+ "Exception while setting volume [%s] to %s state: %s",
+ volumePath, stateDescription, e.getMessage());
+ LOGGER.error(errorMsg, e);
+ throw new CloudRuntimeException(errorMsg, e);
+ }
+ }
+
+ public static void activateClvmVolumeExclusive(String volumePath) {
+ modifyClvmVolumeState(volumePath, ClvmVolumeState.EXCLUSIVE.getLvchangeFlag(),
+ ClvmVolumeState.EXCLUSIVE.getDescription(),
+ "Activating CLVM volume in exclusive mode");
+ }
+
+ public static void deactivateClvmVolume(String volumePath) {
+ try {
+ modifyClvmVolumeState(volumePath, ClvmVolumeState.DEACTIVATE.getLvchangeFlag(),
+ ClvmVolumeState.DEACTIVATE.getDescription(),
+ "Deactivating CLVM volume");
+ } catch (Exception e) {
+ LOGGER.warn("Failed to deactivate CLVM volume {}: {}", volumePath, e.getMessage());
+ }
+ }
+
+ public static void setClvmVolumeToSharedMode(String volumePath) {
+ try {
+ modifyClvmVolumeState(volumePath, ClvmVolumeState.SHARED.getLvchangeFlag(),
+ ClvmVolumeState.SHARED.getDescription(),
+ "Setting CLVM volume to shared mode");
+ } catch (Exception e) {
+ LOGGER.warn("Failed to set CLVM volume {} to shared mode: {}", volumePath, e.getMessage());
+ }
+ }
+
+ /**
+ * Determines if a disk is on a CLVM storage pool by checking the actual pool type from VirtualMachineTO.
+ * This is the most reliable method as it uses CloudStack's own storage pool information.
+ *
+ * @param disk The disk definition to check
+ * @param resource The LibvirtComputingResource instance (unused but kept for compatibility)
+ * @param vmSpec The VirtualMachineTO specification containing disk and pool information
+ * @return true if the disk is on a CLVM storage pool, false otherwise
+ */
+ private static boolean isClvmVolume(DiskDef disk, LibvirtComputingResource resource, VirtualMachineTO vmSpec) {
+ String diskPath = disk.getDiskPath();
+ if (diskPath == null || vmSpec == null) {
+ return false;
+ }
+
+ try {
+ if (vmSpec.getDisks() != null) {
+ for (DiskTO diskTO : vmSpec.getDisks()) {
+ if (diskTO.getData() instanceof VolumeObjectTO) {
+ VolumeObjectTO volumeTO = (VolumeObjectTO) diskTO.getData();
+ if (diskPath.equals(volumeTO.getPath()) || diskPath.equals(diskTO.getPath())) {
+ DataStoreTO dataStore = volumeTO.getDataStore();
+ if (dataStore instanceof PrimaryDataStoreTO) {
+ PrimaryDataStoreTO primaryStore = (PrimaryDataStoreTO) dataStore;
+ boolean isClvm = StoragePoolType.CLVM == primaryStore.getPoolType() ||
+ StoragePoolType.CLVM_NG == primaryStore.getPoolType();
+ LOGGER.debug("Disk {} identified as CLVM/CLVM_NG={} via VirtualMachineTO pool type: {}",
+ diskPath, isClvm, primaryStore.getPoolType());
+ return isClvm;
+ }
+ }
+ }
+ }
+ }
+
+ // Fallback: Check VG attributes using vgs command (reliable)
+ // CLVM VGs have the 'c' (clustered) or 's' (shared) flag in their attributes
+ // Example: 'wz--ns' = shared, 'wz--n-' = not clustered
+ if (diskPath.startsWith("/dev/") && !diskPath.contains("/dev/mapper/")) {
+ String vgName = extractVolumeGroupFromPath(diskPath);
+ if (vgName != null) {
+ boolean isClustered = checkIfVolumeGroupIsClustered(vgName);
+ LOGGER.debug("Disk {} VG {} identified as clustered={} via vgs attribute check",
+ diskPath, vgName, isClustered);
+ return isClustered;
+ }
+ }
+
+ } catch (Exception e) {
+ LOGGER.error("Error determining if volume {} is CLVM: {}", diskPath, e.getMessage(), e);
+ }
+
+ return false;
+ }
+
+ /**
+ * Extracts the volume group name from a device path.
+ *
+ * @param devicePath The device path (e.g., /dev/vgname/lvname)
+ * @return The volume group name, or null if cannot be determined
+ */
+ static String extractVolumeGroupFromPath(String devicePath) {
+ if (devicePath == null || !devicePath.startsWith("/dev/")) {
+ return null;
+ }
+
+ // Format: /dev/<vgname>/<lvname>
+ String[] parts = devicePath.split("/");
+ if (parts.length >= 3) {
+ return parts[2]; // ["", "dev", "vgname", ...]
+ }
+
+ return null;
+ }
+
+ /**
+ * Checks if a volume group is clustered (CLVM) by examining its attributes.
+ * Uses 'vgs' command to check for the clustered/shared flag in VG attributes.
+ *
+ * VG Attr format (6 characters): wz--nc or wz--ns
+ * Position 6: Clustered flag - 'c' = CLVM (clustered), 's' = shared (lvmlockd), '-' = not clustered
+ *
+ * @param vgName The volume group name
+ * @return true if the VG is clustered or shared, false otherwise
+ */
+ static boolean checkIfVolumeGroupIsClustered(String vgName) {
+ if (vgName == null) {
+ return false;
+ }
+
+ try {
+ // Use vgs with --noheadings and -o attr to get VG attributes
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ Script vgsCmd = new Script("vgs", 5000, LOGGER);
+ vgsCmd.add("--noheadings");
+ vgsCmd.add("--unbuffered");
+ vgsCmd.add("-o");
+ vgsCmd.add("vg_attr");
+ vgsCmd.add(vgName);
+
+ String result = vgsCmd.execute(parser);
+
+ if (result == null && parser.getLines() != null) {
+ String output = parser.getLines();
+ if (output != null && !output.isEmpty()) {
+ // Parse VG attributes (format: wz--nc or wz--ns or wz--n-)
+ // Position 6 (0-indexed 5) indicates clustering/sharing:
+ // 'c' = clustered (CLVM) or 's' = shared (lvmlockd) or '-' = not clustered/shared
+ String vgAttr = output.trim();
+ if (vgAttr.length() >= 6) {
+ char clusterFlag = vgAttr.charAt(5); // Position 6 (0-indexed 5)
+ boolean isClustered = (clusterFlag == 'c' || clusterFlag == 's');
+ LOGGER.debug("VG {} has attributes '{}', cluster/shared flag '{}' = {}",
+ vgName, vgAttr, clusterFlag, isClustered);
+ return isClustered;
+ } else {
+ LOGGER.warn("VG {} attributes '{}' have unexpected format (expected 6+ chars)", vgName, vgAttr);
+ }
+ }
+ } else {
+ LOGGER.warn("Failed to get VG attributes for {}: {}", vgName, result);
+ }
+
+ } catch (Exception e) {
+ LOGGER.debug("Error checking if VG {} is clustered: {}", vgName, e.getMessage());
+ }
+
+ return false;
+ }
Review Comment:
I will address this as a separate PR .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4341148890
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4646240239
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "JoaoJandre (via GitHub)" <gi...@apache.org>.
JoaoJandre commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r3382315110
##########
engine/orchestration/src/main/java/com/cloud/vm/VirtualMachineManagerImpl.java:
##########
@@ -3150,6 +3158,11 @@ protected void migrate(final VMInstanceVO vm, final long srcHostId, final Deploy
updateOverCommitRatioForVmProfile(profile, dest.getHost().getClusterId());
final VirtualMachineTO to = toVmTO(profile);
+
+ if (vm.getHypervisorType() == HypervisorType.KVM && hasClvmVolumes(vm.getId())) {
+ executePreMigrationCommand(to, vm.getInstanceName(), srcHostId);
+ }
Review Comment:
I think this check could be inside the executePreMigrationCommand, like it is done for the executePostMigrationCommand method.
##########
engine/orchestration/src/main/java/com/cloud/vm/VirtualMachineManagerImpl.java:
##########
@@ -3320,6 +3334,29 @@ protected void migrate(final VMInstanceVO vm, final long srcHostId, final Deploy
}
}
+ private void executePostMigrationCommand(VMInstanceVO vm, VirtualMachineTO to, long dstHostId) {
+ if (vm.getHypervisorType() == HypervisorType.KVM && hasClvmVolumes(vm.getId())) {
Review Comment:
We could invert the logic and return early, reducing indentation.
##########
engine/orchestration/src/main/java/com/cloud/vm/VirtualMachineManagerImpl.java:
##########
@@ -3320,6 +3334,29 @@ protected void migrate(final VMInstanceVO vm, final long srcHostId, final Deploy
}
}
+ private void executePostMigrationCommand(VMInstanceVO vm, VirtualMachineTO to, long dstHostId) {
+ if (vm.getHypervisorType() == HypervisorType.KVM && hasClvmVolumes(vm.getId())) {
+ try {
+ logger.info("Executing post-migration tasks for VM {} with CLVM volumes on destination host {}", vm.getInstanceName(), dstHostId);
Review Comment:
Could we log the host uuid instead?
##########
engine/orchestration/src/main/java/com/cloud/vm/VirtualMachineManagerImpl.java:
##########
@@ -6441,6 +6507,32 @@ private Pair<Long, Long> findClusterAndHostIdForVm(VirtualMachine vm) {
return findClusterAndHostIdForVm(vm, false);
}
+ private boolean hasClvmVolumes(long vmId) {
+ List<VolumeVO> volumes = _volsDao.findByInstance(vmId);
+ return volumes.stream()
+ .map(v -> _storagePoolDao.findById(v.getPoolId()))
+ .anyMatch(pool -> pool != null && ClvmPoolManager.isClvmPoolType(pool.getPoolType()));
+ }
+
+ private void executePreMigrationCommand(VirtualMachineTO to, String vmInstanceName, long srcHostId) {
+ logger.info("Sending PreMigrationCommand to source host {} for VM {} with CLVM volumes", srcHostId, vmInstanceName);
Review Comment:
same about host uuid
##########
engine/orchestration/src/main/java/org/apache/cloudstack/engine/orchestration/VolumeOrchestrator.java:
##########
@@ -788,6 +805,121 @@ private String getVolumeIdentificationInfos(Volume volume) {
return String.format("uuid: %s, name: %s", volume.getUuid(), volume.getName());
}
+ /**
+ * Updates the CLVM_LOCK_HOST_ID for a migrated volume if applicable.
+ * For CLVM volumes that are attached to a VM, this updates the lock host tracking
+ * to point to the VM's current host after volume migration.
+ *
+ * @param migratedVolume The volume that was migrated
+ * @param destPool The destination storage pool
+ * @param operationType Description of the operation (e.g., "migrated", "live-migrated") for logging
+ */
+ private void updateClvmLockHostAfterMigration(Volume migratedVolume, StoragePool destPool, String operationType) {
+ if (migratedVolume == null || destPool == null) {
+ return;
+ }
+
+ StoragePoolVO pool = _storagePoolDao.findById(destPool.getId());
+ if (pool == null || !ClvmPoolManager.isClvmPoolType(pool.getPoolType())) {
+ return;
+ }
+
+ if (migratedVolume.getInstanceId() == null) {
+ return;
+ }
+
+ VMInstanceVO vm = vmInstanceDao.findById(migratedVolume.getInstanceId());
+ if (vm == null || vm.getHostId() == null) {
+ return;
+ }
+
+ clvmPoolManager.setClvmLockHostId(migratedVolume.getId(), vm.getHostId());
+ logger.debug("Updated CLVM_LOCK_HOST_ID for {} volume {} to host {} where VM {} is running",
+ operationType, migratedVolume.getUuid(), vm.getHostId(), vm.getInstanceName());
+ }
+
+ /**
+ * Retrieves the CLVM lock host ID from any existing volume of the specified VM.
+ * This is useful when attaching a new volume to a stopped VM - we want to maintain
+ * consistency by using the same host that manages the VM's other CLVM volumes.
+ *
+ * @param vmId The ID of the VM
+ * @return The host ID if found, null otherwise
+ */
+ private Long getClvmLockHostFromVmVolumes(Long vmId) {
+ if (vmId == null) {
+ return null;
+ }
+
+ List<VolumeVO> vmVolumes = _volsDao.findByInstance(vmId);
+ if (vmVolumes == null || vmVolumes.isEmpty()) {
+ return null;
+ }
+
+ for (VolumeVO volume : vmVolumes) {
+ if (volume.getPoolId() == null) {
+ continue;
+ }
+
+ StoragePoolVO pool = _storagePoolDao.findById(volume.getPoolId());
+ if (pool != null && ClvmPoolManager.isClvmPoolType(pool.getPoolType())) {
Review Comment:
we could invert the logic to reduce indentation.
##########
engine/orchestration/src/main/java/com/cloud/vm/VirtualMachineManagerImpl.java:
##########
@@ -3320,6 +3334,29 @@ protected void migrate(final VMInstanceVO vm, final long srcHostId, final Deploy
}
}
+ private void executePostMigrationCommand(VMInstanceVO vm, VirtualMachineTO to, long dstHostId) {
+ if (vm.getHypervisorType() == HypervisorType.KVM && hasClvmVolumes(vm.getId())) {
+ try {
+ logger.info("Executing post-migration tasks for VM {} with CLVM volumes on destination host {}", vm.getInstanceName(), dstHostId);
+ final PostMigrationCommand postMigrationCommand = new PostMigrationCommand(to, vm.getInstanceName());
+ final Answer postMigrationAnswer = _agentMgr.send(dstHostId, postMigrationCommand);
+
+ if (postMigrationAnswer == null || !postMigrationAnswer.getResult()) {
+ final String details = postMigrationAnswer != null ? postMigrationAnswer.getDetails() : "null answer returned";
+ logger.warn("Post-migration tasks failed for VM {} on destination host {}: {}. Migration completed but some cleanup may be needed.",
+ vm.getInstanceName(), dstHostId, details);
+ } else {
+ logger.info("Successfully completed post-migration tasks for VM {} on destination host {}", vm.getInstanceName(), dstHostId);
+ }
+ } catch (Exception e) {
+ logger.warn("Exception during post-migration tasks for VM {} on destination host {}: {}. Migration completed but some cleanup may be needed.",
+ vm.getInstanceName(), dstHostId, e.getMessage(), e);
+ }
+ }
+
+ updateClvmLockHostForVmVolumes(vm.getId(), dstHostId);
Review Comment:
If the result is false, or null, we still update the lock? How do we know if this update will be correct?
If the host does not execute the command (the command queue is full and the command times out, or the host is restarted) we will update the lock metadata to point to the wrong host, correct?
##########
engine/orchestration/src/main/java/com/cloud/vm/VirtualMachineManagerImpl.java:
##########
@@ -4922,6 +4980,14 @@ private void orchestrateMigrateForScale(final String vmUuid, final long srcHostI
volumeMgr.prepareForMigration(profile, dest);
final VirtualMachineTO to = toVmTO(profile);
+
+ // Step 1: Send PreMigrationCommand to source host to convert CLVM volumes to shared mode
+ // This must happen BEFORE PrepareForMigrationCommand on destination to avoid lock conflicts
+ if (vm.getHypervisorType() == HypervisorType.KVM && hasClvmVolumes(vm.getId())) {
+ executePreMigrationCommand(to, vm.getInstanceName(), srcHostId);
Review Comment:
In this case we only execute the premigration, but not the post migration?
##########
engine/storage/datamotion/src/main/java/org/apache/cloudstack/storage/motion/StorageSystemDataMotionStrategy.java:
##########
@@ -2132,6 +2157,23 @@ public void copyAsync(Map<VolumeInfo, DataStore> volumeDataStoreMap, VirtualMach
throw new AgentUnavailableException("Operation timed out", destHost.getId());
}
+ for (VolumeInfo vol : samePoolClvmVolumes) {
Review Comment:
shouldn't we check if the pool is clvm before executing this for?
##########
engine/storage/datamotion/src/main/java/org/apache/cloudstack/storage/motion/StorageSystemDataMotionStrategy.java:
##########
@@ -2132,6 +2157,23 @@ public void copyAsync(Map<VolumeInfo, DataStore> volumeDataStoreMap, VirtualMach
throw new AgentUnavailableException("Operation timed out", destHost.getId());
}
+ for (VolumeInfo vol : samePoolClvmVolumes) {
+ StoragePoolVO samePoolClvmPool = _storagePoolDao.findById(vol.getPoolId());
+ String vgName = samePoolClvmPool.getPath();
+ if (vgName.startsWith("/")) vgName = vgName.substring(1);
Review Comment:
this `if` does not follow the java coding conventions for ACS.
##########
engine/storage/datamotion/src/main/java/org/apache/cloudstack/storage/motion/AncientDataMotionStrategy.java:
##########
@@ -331,6 +336,27 @@ protected Answer copyVolumeFromSnapshot(DataObject snapObj, DataObject volObj) {
}
}
+ private void updateLockHostForVolume(EndPoint ep, DataObject volObj) {
+ if (ep != null && volObj instanceof VolumeInfo) {
+ VolumeInfo volumeInfo = (VolumeInfo) volObj;
+ StoragePool destPool = (StoragePool) volObj.getDataStore();
+ if (destPool != null && ClvmPoolManager.isClvmPoolType(destPool.getPoolType())) {
Review Comment:
we could invert both of these logics to reduce indentation
##########
engine/storage/datamotion/src/main/java/org/apache/cloudstack/storage/motion/StorageSystemDataMotionStrategy.java:
##########
@@ -2211,19 +2253,56 @@ public void copyAsync(Map<VolumeInfo, DataStore> volumeDataStoreMap, VirtualMach
}
}
+ private void prepareDisksForMigrationForClvm(VirtualMachineTO vmTO, Map<VolumeInfo, DataStore> volumeDataStoreMap, Host srcHost) {
+ // For CLVM/CLVM_NG source pools, convert volumes from exclusive to shared mode
+ // on the source host BEFORE PrepareForMigrationCommand on the destination.
+ boolean hasClvmSource = volumeDataStoreMap.keySet().stream()
+ .map(v -> _storagePoolDao.findById(v.getPoolId()))
+ .anyMatch(p -> p != null && (p.getPoolType() == StoragePoolType.CLVM || p.getPoolType() == StoragePoolType.CLVM_NG));
+ if (hasClvmSource && srcHost.getHypervisorType() == HypervisorType.KVM) {
+ logger.info("CLVM/CLVM_NG source pool detected for VM [{}], sending PreMigrationCommand to source host [{}] to convert volumes to shared mode.", vmTO.getName(), srcHost.getId());
+ PreMigrationCommand preMigCmd = new PreMigrationCommand(vmTO, vmTO.getName());
+ try {
+ Answer preMigAnswer = agentManager.send(srcHost.getId(), preMigCmd);
+ if (preMigAnswer == null || !preMigAnswer.getResult()) {
+ String details = preMigAnswer != null ? preMigAnswer.getDetails() : "null answer returned";
+ logger.warn("PreMigrationCommand failed for CLVM/CLVM_NG VM [{}] on source host [{}]: {}. Migration will continue but may fail if volumes are exclusively locked.", vmTO.getName(), srcHost.getId(), details);
Review Comment:
Shouldn't we abort the migration in this case?
If we continue and it does not work, it could lead to problems in the VM, no?
##########
engine/orchestration/src/main/java/com/cloud/vm/VirtualMachineManagerImpl.java:
##########
@@ -6441,6 +6507,32 @@ private Pair<Long, Long> findClusterAndHostIdForVm(VirtualMachine vm) {
return findClusterAndHostIdForVm(vm, false);
}
+ private boolean hasClvmVolumes(long vmId) {
+ List<VolumeVO> volumes = _volsDao.findByInstance(vmId);
+ return volumes.stream()
+ .map(v -> _storagePoolDao.findById(v.getPoolId()))
+ .anyMatch(pool -> pool != null && ClvmPoolManager.isClvmPoolType(pool.getPoolType()));
+ }
+
+ private void executePreMigrationCommand(VirtualMachineTO to, String vmInstanceName, long srcHostId) {
+ logger.info("Sending PreMigrationCommand to source host {} for VM {} with CLVM volumes", srcHostId, vmInstanceName);
+ final PreMigrationCommand preMigCmd = new PreMigrationCommand(to, vmInstanceName);
+ Answer preMigAnswer = null;
+ try {
+ preMigAnswer = _agentMgr.send(srcHostId, preMigCmd);
+ if (preMigAnswer == null || !preMigAnswer.getResult()) {
+ final String details = preMigAnswer != null ? preMigAnswer.getDetails() : "null answer returned";
Review Comment:
What will happen if the command was executed in the host, but the answer could not be delivered to the Management Server? Will the next try be able to repeat the process or will it be stuck ?
##########
engine/storage/datamotion/src/main/java/org/apache/cloudstack/storage/motion/KvmNonManagedStorageDataMotionStrategy.java:
##########
@@ -144,12 +144,16 @@ protected boolean isDestinationNfsPrimaryStorageClusterWide(Map<VolumeInfo, Data
}
/**
- * Configures a {@link MigrateDiskInfo} object configured for migrating a File System volume and calls rootImageProvisioning.
+ * Configures a {@link MigrateDiskInfo} object configured for migrating a File System volume.
*/
@Override
protected MigrateCommand.MigrateDiskInfo configureMigrateDiskInfo(VolumeInfo srcVolumeInfo, String destPath, String backingPath) {
- return new MigrateCommand.MigrateDiskInfo(srcVolumeInfo.getPath(), MigrateCommand.MigrateDiskInfo.DiskType.FILE, MigrateCommand.MigrateDiskInfo.DriverType.QCOW2,
- MigrateCommand.MigrateDiskInfo.Source.FILE, destPath, backingPath);
+ return new MigrateCommand.MigrateDiskInfo(srcVolumeInfo.getPath(),
+ MigrateCommand.MigrateDiskInfo.DiskType.FILE,
+ MigrateCommand.MigrateDiskInfo.DriverType.QCOW2,
+ MigrateCommand.MigrateDiskInfo.Source.FILE,
+ destPath,
+ backingPath);
Review Comment:
Is this a style change only? Is it needed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4700320581
<b>[SF] Trillian Build Failed (tid-16305)<b/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4700661864
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4549188360
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [33.5% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
<!-- sqra:qg_revision=b8d6924772be4ecfa7e6f37f0974e3cfbd02be9f353122e9586b82df580939c5 -->
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-3960059138
@Pearl1594 a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4121075576
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [8.5% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-3945018503
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-3886643814
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4221727313
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r3154363590
##########
engine/orchestration/src/main/java/com/cloud/vm/VirtualMachineManagerImpl.java:
##########
@@ -3274,6 +3287,26 @@ protected void migrate(final VMInstanceVO vm, final long srcHostId, final Deploy
logger.warn("Error while checking the vm {} on host {}", vm, dest.getHost(), e);
}
migrated = true;
+ if (vm.getHypervisorType() == HypervisorType.KVM && hasClvmVolumes(vm.getId())) {
+ try {
+ logger.info("Executing post-migration tasks for VM {} with CLVM volumes on destination host {}", vm.getInstanceName(), dstHostId);
+ final PostMigrationCommand postMigrationCommand = new PostMigrationCommand(to, vm.getInstanceName());
+ final Answer postMigrationAnswer = _agentMgr.send(dstHostId, postMigrationCommand);
+
+ if (postMigrationAnswer == null || !postMigrationAnswer.getResult()) {
+ final String details = postMigrationAnswer != null ? postMigrationAnswer.getDetails() : "null answer returned";
+ logger.warn("Post-migration tasks failed for VM {} on destination host {}: {}. Migration completed but some cleanup may be needed.",
+ vm.getInstanceName(), dstHostId, details);
+ } else {
+ logger.info("Successfully completed post-migration tasks for VM {} on destination host {}", vm.getInstanceName(), dstHostId);
+ }
+ } catch (Exception e) {
+ logger.warn("Exception during post-migration tasks for VM {} on destination host {}: {}. Migration completed but some cleanup may be needed.",
+ vm.getInstanceName(), dstHostId, e.getMessage(), e);
+ }
Review Comment:
can move this ^^^ to separate method - _executePostMigrationCommand()_ ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r3154502054
##########
server/src/main/java/com/cloud/storage/VolumeApiServiceImpl.java:
##########
@@ -414,6 +416,9 @@ public class VolumeApiServiceImpl extends ManagerBase implements VolumeApiServic
public static final ConfigKey<Boolean> AllowCheckAndRepairVolume = new ConfigKey<>("Advanced", Boolean.class, "volume.check.and.repair.leaks.before.use", "false",
"To check and repair the volume if it has any leaks before performing volume attach or VM start operations", true, ConfigKey.Scope.StoragePool);
+ public static final ConfigKey<Boolean> CLVMSecureZeroFill = new ConfigKey<>("Advanced", Boolean.class, "clvm.secure.zero.fill", "false",
Review Comment:
can move this config to ClvmLockManager?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r3154508583
##########
server/src/main/java/com/cloud/storage/ClvmLockManager.java:
##########
@@ -0,0 +1,334 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package com.cloud.storage;
+
+import java.util.Arrays;
+import java.util.List;
+import javax.inject.Inject;
+
+import com.cloud.agent.AgentManager;
+import com.cloud.agent.api.Answer;
+import com.cloud.exception.AgentUnavailableException;
+import com.cloud.exception.OperationTimedoutException;
+import com.cloud.host.Host;
+import com.cloud.host.HostVO;
+import com.cloud.host.Status;
+import com.cloud.host.dao.HostDao;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.dao.VolumeDetailsDao;
+import org.apache.cloudstack.engine.subsystem.api.storage.VolumeInfo;
+import org.apache.cloudstack.storage.command.ClvmLockTransferCommand;
+import org.apache.cloudstack.storage.command.ClvmLockTransferAnswer;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.springframework.stereotype.Component;
+
+@Component
+public class ClvmLockManager {
Review Comment:
```suggestion
public class ClvmPoolManager {
```
keep all CLVM related config/params here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4408219368
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [33.1% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
<!-- sqra:qg_revision=2810404f9bdbb08abd7243adeba83ed47ec8a40367ecfd74dc4787d0c336544b -->
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4498274930
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4549142986
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18048
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4548701185
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4597095671
<b>[SF] Trillian test result (tid-16231)</b>
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 51829 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12617-t16231-kvm-ol8.zip
Smoke tests completed. 151 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4245269210
<b>[SF] Trillian test result (tid-15862)</b>
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 51139 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12617-t15862-kvm-ol8.zip
Smoke tests completed. 151 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4240524815
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4654189643
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18198
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4673253876
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4700198302
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4700291207
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18251
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4685195430
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "JoaoJandre (via GitHub)" <gi...@apache.org>.
JoaoJandre commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r3388150985
##########
engine/storage/datamotion/src/main/java/org/apache/cloudstack/storage/motion/StorageSystemDataMotionStrategy.java:
##########
@@ -2320,7 +2434,26 @@ String getVolumeBackingFile(VolumeInfo srcVolumeInfo) {
return null;
}
- private void handlePostMigration(boolean success, Map<VolumeInfo, VolumeInfo> srcVolumeInfoToDestVolumeInfo, VirtualMachineTO vmTO, Host destHost) {
+ private void sendClvmLockCommand(long hostId, StoragePoolVO pool, VolumeInfo volumeInfo,
+ ClvmLockTransferCommand.Operation operation) {
+ String vgName = pool.getPath();
+ if (vgName.startsWith("/")) {
+ vgName = vgName.substring(1);
+ }
+ String lvPath = String.format("/dev/%s/%s", vgName, volumeInfo.getPath());
+ try {
+ Answer answer = agentManager.send(hostId,
+ new ClvmLockTransferCommand(operation, lvPath, volumeInfo.getUuid()));
+ if (answer == null || !answer.getResult()) {
+ String details = answer != null ? answer.getDetails() : "null answer";
+ logger.warn("CLVM lock command [{}] failed for LV [{}] on host [{}]: {}", operation, lvPath, hostId, details);
Review Comment:
What is the consequence of improper locking? If we get a false answer here we just continue, what happens then?
If there is no consequence, why are we locking and unlocking?
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/wrapper/LibvirtClvmLockTransferCommandWrapper.java:
##########
@@ -0,0 +1,182 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package com.cloud.hypervisor.kvm.resource.wrapper;
+
+import org.apache.logging.log4j.Logger;
+import org.apache.logging.log4j.LogManager;
+
+import com.cloud.agent.api.Answer;
+import org.apache.cloudstack.storage.clvm.command.ClvmLockTransferCommand;
+import org.apache.cloudstack.storage.clvm.command.ClvmLockTransferAnswer;
+import com.cloud.hypervisor.kvm.resource.LibvirtComputingResource;
+import com.cloud.resource.CommandWrapper;
+import com.cloud.resource.ResourceWrapper;
+import com.cloud.utils.script.Script;
+import com.cloud.utils.script.OutputInterpreter;
+
+@ResourceWrapper(handles = ClvmLockTransferCommand.class)
+public class LibvirtClvmLockTransferCommandWrapper
+ extends CommandWrapper<ClvmLockTransferCommand, Answer, LibvirtComputingResource> {
+
+ protected Logger logger = LogManager.getLogger(getClass());
+
+ @Override
+ public Answer execute(ClvmLockTransferCommand cmd, LibvirtComputingResource serverResource) {
+ String lvPath = cmd.getLvPath();
+ ClvmLockTransferCommand.Operation operation = cmd.getOperation();
+ String volumeUuid = cmd.getVolumeUuid();
+
+ logger.info("Executing CLVM lock transfer: operation={}, lv={}, volume={}",
+ operation, lvPath, volumeUuid);
+
+ try {
+
+ if (operation == ClvmLockTransferCommand.Operation.QUERY_LOCK_STATE) {
+ return handleQueryLockState(cmd, lvPath, volumeUuid);
+ }
+
+ String lvchangeOpt;
+ String operationDesc;
+ switch (operation) {
+ case DEACTIVATE:
+ lvchangeOpt = "-an";
+ operationDesc = "deactivated";
+ break;
+ case ACTIVATE_EXCLUSIVE:
+ lvchangeOpt = "-aey";
+ operationDesc = "activated exclusively";
+ break;
+ case ACTIVATE_SHARED:
+ lvchangeOpt = "-asy";
+ operationDesc = "activated in shared mode";
+ break;
+ default:
+ return new ClvmLockTransferAnswer(cmd, false, "Unknown operation: " + operation);
+ }
+
+ Script script = new Script("/usr/sbin/lvchange", 30000, logger);
+ script.add(lvchangeOpt);
+ script.add(lvPath);
+
+ String result = script.execute();
+
+ if (result != null) {
+ logger.error("CLVM lock transfer failed for volume {}: {}",
+ volumeUuid, result);
+ return new ClvmLockTransferAnswer(cmd, false,
+ String.format("lvchange %s %s failed: %s", lvchangeOpt, lvPath, result));
+ }
+
+ logger.info("Successfully executed CLVM lock transfer: {} {} for volume {}",
+ lvchangeOpt, lvPath, volumeUuid);
+
+ return new ClvmLockTransferAnswer(cmd, true,
+ String.format("Successfully %s CLVM volume %s", operationDesc, volumeUuid));
+
+ } catch (Exception e) {
+ logger.error("Exception during CLVM lock transfer for volume {}: {}",
+ volumeUuid, e.getMessage(), e);
+ return new ClvmLockTransferAnswer(cmd, false, "Exception: " + e.getMessage());
+ }
+ }
+
+ /**
+ * Query whether this host currently has the CLVM LV activated locally.
+ * Executes: lvs -o lv_attr,lv_host,lv_active --noheadings <lvPath>
+ *
+ * lv_attr[4]=='a' (isActive) is LOCAL and is the authoritative signal — true only on
+ * the host where the LV is currently activated. The management server fans out this
+ * query to all cluster hosts; the one returning isActive=true is the lock holder.
+ * lv_attr[5]=='o' (isOpen) means a VM has the device open on this host (doing I/O).
+ * lv_host is retained for diagnostic logging only — do NOT use it to identify the
+ * lock holder.
+ */
+ private Answer handleQueryLockState(ClvmLockTransferCommand cmd, String lvPath, String volumeUuid) {
+ try {
+ Script script = new Script("/usr/sbin/lvs", 10000, logger);
+ script.add("-o");
+ script.add("lv_attr,lv_host");
+ script.add("--noheadings");
+ script.add(lvPath);
+
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ String result = script.execute(parser);
+
+ if (result != null) {
+ logger.error("Failed to query lock state for volume {}: {}", volumeUuid, result);
+ return new ClvmLockTransferAnswer(cmd, false,
+ String.format("lvs command failed: %s", result));
+ }
+
+ String[] lines = parser.getLines().split("\n");
+ String dataLine = null;
+
+ for (String line : lines) {
+ String trimmed = line.trim();
+ if (!trimmed.isEmpty() &&
+ trimmed.length() >= 10 &&
+ (trimmed.charAt(0) == '-' || trimmed.charAt(0) == 'w' ||
+ trimmed.charAt(0) == 'r' || trimmed.charAt(0) == 's' ||
+ trimmed.charAt(0) == 'v' || trimmed.charAt(0) == 'm' ||
+ trimmed.charAt(0) == 'p' || trimmed.charAt(0) == 'c' ||
+ trimmed.charAt(0) == 'o')) {
Review Comment:
Could put these in a list and check if the char at 0 is in the list. Or even better, create a Enum so we know what each character means.
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/wrapper/LibvirtClvmLockTransferCommandWrapper.java:
##########
@@ -0,0 +1,182 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package com.cloud.hypervisor.kvm.resource.wrapper;
+
+import org.apache.logging.log4j.Logger;
+import org.apache.logging.log4j.LogManager;
+
+import com.cloud.agent.api.Answer;
+import org.apache.cloudstack.storage.clvm.command.ClvmLockTransferCommand;
+import org.apache.cloudstack.storage.clvm.command.ClvmLockTransferAnswer;
+import com.cloud.hypervisor.kvm.resource.LibvirtComputingResource;
+import com.cloud.resource.CommandWrapper;
+import com.cloud.resource.ResourceWrapper;
+import com.cloud.utils.script.Script;
+import com.cloud.utils.script.OutputInterpreter;
+
+@ResourceWrapper(handles = ClvmLockTransferCommand.class)
+public class LibvirtClvmLockTransferCommandWrapper
+ extends CommandWrapper<ClvmLockTransferCommand, Answer, LibvirtComputingResource> {
+
+ protected Logger logger = LogManager.getLogger(getClass());
Review Comment:
The CommandWrapper class already has a logger, which is inherited by this class. There is no need to define it again.
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/wrapper/LibvirtMigrateCommandWrapper.java:
##########
@@ -1066,4 +1079,118 @@ public static String maskSensitiveInfoInXML(String xmlDesc) {
return xmlDesc.replaceAll("(graphics\\s+[^>]*type=['\"]vnc['\"][^>]*passwd=['\"])([^'\"]*)(['\"])",
"$1*****$3");
}
+
+ /**
+ * Checks if any of the destination disks in the migration target a CLVM or CLVM_NG storage pool.
+ * This is used to determine if incremental migration should be disabled to avoid libvirt
+ * precreate errors with QCOW2-on-LVM setups.
+ *
+ * @param mapMigrateStorage the map containing migration disk information with destination pool types
+ * @return true if any destination disk targets CLVM or CLVM_NG, false otherwise
+ */
+ protected boolean hasClvmDestinationDisks(Map<String, MigrateCommand.MigrateDiskInfo> mapMigrateStorage) {
+ if (MapUtils.isEmpty(mapMigrateStorage)) {
+ return false;
+ }
+
+ try {
+ for (Map.Entry<String, MigrateCommand.MigrateDiskInfo> entry : mapMigrateStorage.entrySet()) {
+ MigrateCommand.MigrateDiskInfo diskInfo = entry.getValue();
+ if (isClvmBlockDevice(diskInfo)) {
+ logger.debug("Found disk targeting CLVM/CLVM_NG destination pool");
+ return true;
+ }
Review Comment:
```suggestion
if (isClvmBlockDevice(diskInfo)) {
logger.debug("Found disk targeting CLVM/CLVM_NG destination pool");
return true;
}
```
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java:
##########
@@ -6820,4 +6827,242 @@ public String getHypervisorPath() {
public String getGuestCpuArch() {
return guestCpuArch;
}
+
+ /**
+ * CLVM volume state for migration operations on source host
+ */
+ public enum ClvmVolumeState {
+ /** Shared mode (-asy) - used before migration to allow both hosts to access volume */
+ SHARED("-asy", "shared", "Before migration: activating in shared mode"),
+
+ /** Deactivate (-an) - used after successful migration to release volume on source */
+ DEACTIVATE("-an", "deactivated", "After successful migration: deactivating volume"),
+
+ /** Exclusive mode (-aey) - used after failed migration to revert to original exclusive state */
+ EXCLUSIVE("-aey", "exclusive", "After failed migration: reverting to exclusive mode");
+
+ private final String lvchangeFlag;
+ private final String description;
+ private final String logMessage;
+
+ ClvmVolumeState(String lvchangeFlag, String description, String logMessage) {
+ this.lvchangeFlag = lvchangeFlag;
+ this.description = description;
+ this.logMessage = logMessage;
+ }
+
+ public String getLvchangeFlag() {
+ return lvchangeFlag;
+ }
+
+ public String getDescription() {
+ return description;
+ }
+
+ public String getLogMessage() {
+ return logMessage;
+ }
+ }
+
+ public static void modifyClvmVolumesStateForMigration(List<DiskDef> disks, LibvirtComputingResource resource,
+ VirtualMachineTO vmSpec, ClvmVolumeState state) {
+ for (DiskDef disk : disks) {
+ if (isClvmVolume(disk, resource, vmSpec)) {
+ String volumePath = disk.getDiskPath();
+ try {
+ modifyClvmVolumeState(volumePath, state.getLvchangeFlag(), state.getDescription(), state.getLogMessage());
+ } catch (Exception e) {
+ LOGGER.error("[CLVM Migration] Exception while setting volume [{}] to {} state: {}",
+ volumePath, state.getDescription(), e.getMessage(), e);
+ }
+ }
+ }
+ }
+
+ private static void modifyClvmVolumeState(String volumePath, String lvchangeFlag,
Review Comment:
same
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java:
##########
@@ -6820,4 +6827,242 @@ public String getHypervisorPath() {
public String getGuestCpuArch() {
return guestCpuArch;
}
+
+ /**
+ * CLVM volume state for migration operations on source host
+ */
+ public enum ClvmVolumeState {
+ /** Shared mode (-asy) - used before migration to allow both hosts to access volume */
+ SHARED("-asy", "shared", "Before migration: activating in shared mode"),
+
+ /** Deactivate (-an) - used after successful migration to release volume on source */
+ DEACTIVATE("-an", "deactivated", "After successful migration: deactivating volume"),
+
+ /** Exclusive mode (-aey) - used after failed migration to revert to original exclusive state */
+ EXCLUSIVE("-aey", "exclusive", "After failed migration: reverting to exclusive mode");
+
+ private final String lvchangeFlag;
+ private final String description;
+ private final String logMessage;
+
+ ClvmVolumeState(String lvchangeFlag, String description, String logMessage) {
+ this.lvchangeFlag = lvchangeFlag;
+ this.description = description;
+ this.logMessage = logMessage;
+ }
+
+ public String getLvchangeFlag() {
+ return lvchangeFlag;
+ }
+
+ public String getDescription() {
+ return description;
+ }
+
+ public String getLogMessage() {
+ return logMessage;
+ }
+ }
+
+ public static void modifyClvmVolumesStateForMigration(List<DiskDef> disks, LibvirtComputingResource resource,
+ VirtualMachineTO vmSpec, ClvmVolumeState state) {
+ for (DiskDef disk : disks) {
+ if (isClvmVolume(disk, resource, vmSpec)) {
+ String volumePath = disk.getDiskPath();
+ try {
+ modifyClvmVolumeState(volumePath, state.getLvchangeFlag(), state.getDescription(), state.getLogMessage());
+ } catch (Exception e) {
+ LOGGER.error("[CLVM Migration] Exception while setting volume [{}] to {} state: {}",
+ volumePath, state.getDescription(), e.getMessage(), e);
+ }
+ }
+ }
+ }
+
+ private static void modifyClvmVolumeState(String volumePath, String lvchangeFlag,
+ String stateDescription, String logMessage) {
+ try {
+ LOGGER.info("{} for volume [{}]", logMessage, volumePath);
+
+ Script cmd = new Script("lvchange", Duration.standardSeconds(300), LOGGER);
+ cmd.add(lvchangeFlag);
+ cmd.add(volumePath);
+
+ String result = cmd.execute();
+ if (result != null) {
+ String errorMsg = String.format(
+ "Failed to set volume [%s] to %s state. Command result: %s",
+ volumePath, stateDescription, result);
+ LOGGER.error(errorMsg);
+ throw new CloudRuntimeException(errorMsg);
+ } else {
+ LOGGER.info("Successfully set volume [{}] to {} state.",
+ volumePath, stateDescription);
+ }
+ } catch (CloudRuntimeException e) {
+ throw e;
+ } catch (Exception e) {
+ String errorMsg = String.format(
+ "Exception while setting volume [%s] to %s state: %s",
+ volumePath, stateDescription, e.getMessage());
+ LOGGER.error(errorMsg, e);
+ throw new CloudRuntimeException(errorMsg, e);
+ }
+ }
+
+ public static void activateClvmVolumeExclusive(String volumePath) {
+ modifyClvmVolumeState(volumePath, ClvmVolumeState.EXCLUSIVE.getLvchangeFlag(),
+ ClvmVolumeState.EXCLUSIVE.getDescription(),
+ "Activating CLVM volume in exclusive mode");
+ }
+
+ public static void deactivateClvmVolume(String volumePath) {
+ try {
+ modifyClvmVolumeState(volumePath, ClvmVolumeState.DEACTIVATE.getLvchangeFlag(),
+ ClvmVolumeState.DEACTIVATE.getDescription(),
+ "Deactivating CLVM volume");
+ } catch (Exception e) {
+ LOGGER.warn("Failed to deactivate CLVM volume {}: {}", volumePath, e.getMessage());
+ }
+ }
+
+ public static void setClvmVolumeToSharedMode(String volumePath) {
+ try {
+ modifyClvmVolumeState(volumePath, ClvmVolumeState.SHARED.getLvchangeFlag(),
+ ClvmVolumeState.SHARED.getDescription(),
+ "Setting CLVM volume to shared mode");
+ } catch (Exception e) {
+ LOGGER.warn("Failed to set CLVM volume {} to shared mode: {}", volumePath, e.getMessage());
+ }
+ }
+
+ /**
+ * Determines if a disk is on a CLVM storage pool by checking the actual pool type from VirtualMachineTO.
+ * This is the most reliable method as it uses CloudStack's own storage pool information.
+ *
+ * @param disk The disk definition to check
+ * @param resource The LibvirtComputingResource instance (unused but kept for compatibility)
+ * @param vmSpec The VirtualMachineTO specification containing disk and pool information
+ * @return true if the disk is on a CLVM storage pool, false otherwise
+ */
+ private static boolean isClvmVolume(DiskDef disk, LibvirtComputingResource resource, VirtualMachineTO vmSpec) {
+ String diskPath = disk.getDiskPath();
+ if (diskPath == null || vmSpec == null) {
+ return false;
+ }
+
+ try {
+ if (vmSpec.getDisks() != null) {
+ for (DiskTO diskTO : vmSpec.getDisks()) {
+ if (diskTO.getData() instanceof VolumeObjectTO) {
+ VolumeObjectTO volumeTO = (VolumeObjectTO) diskTO.getData();
+ if (diskPath.equals(volumeTO.getPath()) || diskPath.equals(diskTO.getPath())) {
+ DataStoreTO dataStore = volumeTO.getDataStore();
+ if (dataStore instanceof PrimaryDataStoreTO) {
+ PrimaryDataStoreTO primaryStore = (PrimaryDataStoreTO) dataStore;
+ boolean isClvm = StoragePoolType.CLVM == primaryStore.getPoolType() ||
+ StoragePoolType.CLVM_NG == primaryStore.getPoolType();
+ LOGGER.debug("Disk {} identified as CLVM/CLVM_NG={} via VirtualMachineTO pool type: {}",
+ diskPath, isClvm, primaryStore.getPoolType());
+ return isClvm;
+ }
+ }
+ }
Review Comment:
We could refactor this into a new method. And invert the logic to reduce indentation.
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/ClvmStorageAdaptor.java:
##########
@@ -0,0 +1,1072 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.hypervisor.kvm.storage;
+
+import static com.cloud.utils.NumbersUtil.toHumanReadableSize;
+
+import java.io.File;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.cloud.hypervisor.kvm.resource.LibvirtComputingResource;
+import com.cloud.hypervisor.kvm.resource.LibvirtConnection;
+import com.cloud.hypervisor.kvm.resource.LibvirtStoragePoolDef;
+import com.cloud.hypervisor.kvm.resource.LibvirtStorageVolumeDef;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Storage.StoragePoolType;
+import com.cloud.storage.StorageLayer;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.script.OutputInterpreter;
+import com.cloud.utils.script.Script;
+import com.google.gson.JsonObject;
+import com.google.gson.JsonParser;
+import org.apache.cloudstack.utils.qemu.QemuImg.PhysicalDiskFormat;
+import org.joda.time.Duration;
+import org.libvirt.Connect;
+import org.libvirt.LibvirtException;
+import org.libvirt.StoragePool;
+import org.libvirt.StorageVol;
+
+/**
+ * Storage adaptor for CLVM and CLVM_NG pool types.
+ * Extends LibvirtStorageAdaptor and overrides methods with CLVM-specific logic,
+ * using direct LVM commands instead of libvirt for volume operations.
+ */
+public class ClvmStorageAdaptor extends LibvirtStorageAdaptor {
+
+ public ClvmStorageAdaptor(StorageLayer storage) {
+ super(storage);
+ }
+
+ @Override
+ public StoragePoolType getStoragePoolType() {
+ // Registered manually for both CLVM and CLVM_NG in KVMStoragePoolManager
+ return null;
+ }
+
+ @Override
+ public KVMStoragePool createStoragePool(String name, String host, int port, String path,
+ String userInfo, StoragePoolType type, Map<String, String> details, boolean isPrimaryStorage) {
+ logger.info("Attempting to create CLVM/CLVM_NG storage pool {} in libvirt", name);
+
+ Connect conn;
+ try {
+ conn = LibvirtConnection.getConnection();
+ } catch (LibvirtException e) {
+ throw new CloudRuntimeException(e.toString());
+ }
+
+ StoragePool sp = createCLVMStoragePool(conn, name, host, path);
+ if (sp == null) {
+ logger.info("Falling back to virtual CLVM/CLVM_NG pool without libvirt for: {}", name);
+ return createVirtualClvmPool(name, host, path, type, details);
+ }
+
+ try {
+ if (!isPrimaryStorage) {
+ incStoragePoolRefCount(name);
+ }
+ // CLVM/CLVM_NG pools are kept inactive in libvirt; we use direct LVM commands
+ return getStoragePool(name);
+ } catch (Exception e) {
+ decStoragePoolRefCount(name);
+ throw new CloudRuntimeException("Failed to create CLVM storage pool: " + name, e);
+ }
+ }
+
+ @Override
+ public KVMStoragePool getStoragePool(String uuid, boolean refreshInfo) {
+ logger.info("Fetching CLVM/CLVM_NG storage pool {} ", uuid);
+ try {
+ Connect conn = LibvirtConnection.getConnection();
+ StoragePool storage = conn.storagePoolLookupByUUIDString(uuid);
+
+ LibvirtStoragePoolDef spd = getStoragePoolDef(conn, storage);
+ if (spd == null) {
+ throw new CloudRuntimeException("Unable to parse storage pool definition for pool " + uuid);
+ }
+
+ // CLVM pools in libvirt are always LOGICAL type
+ StoragePoolType type = StoragePoolType.CLVM;
+
+ // Do NOT activate the pool — CLVM/CLVM_NG pools stay inactive in libvirt
+ LibvirtStoragePool pool = new LibvirtStoragePool(uuid, storage.getName(), type, this, storage);
+ pool.setLocalPath(spd.getTargetPath());
+
+ // Always read capacity from LVM directly
+ String vgName = storage.getName();
+ try {
+ long[] vgStats = getVgStats(vgName);
+ setPoolCapacityFromVgStats(pool, vgStats, vgName);
+ } catch (CloudRuntimeException e) {
+ logger.warn("Failed to get VG stats for CLVM/CLVM_NG pool {}: {}. Using libvirt values (may be 0)", vgName, e.getMessage());
+ pool.setCapacity(storage.getInfo().capacity);
+ pool.setUsed(storage.getInfo().allocation);
+ pool.setAvailable(storage.getInfo().available);
+ }
+
+ return pool;
+ } catch (LibvirtException e) {
+ // Pool not in libvirt - return virtual pool backed only by LVM
+ logger.debug("CLVM/CLVM_NG pool {} not found in libvirt, creating virtual pool", uuid);
+ throw new CloudRuntimeException(e.toString(), e);
+ }
+ }
+
+ @Override
+ public KVMPhysicalDisk getPhysicalDisk(String volumeUuid, KVMStoragePool pool) {
+ LibvirtStoragePool libvirtPool = (LibvirtStoragePool) pool;
+
+ // Pool has no libvirt backing - go directly to block device
+ if (libvirtPool.getPool() == null) {
+ logger.debug("CLVM/CLVM_NG pool has no libvirt backing, using direct block device access for volume: {}", volumeUuid);
+ return getPhysicalDiskViaDirectBlockDevice(volumeUuid, pool);
+ }
+
+ try {
+ StorageVol vol = getVolume(libvirtPool.getPool(), volumeUuid);
+ if (vol == null) {
+ logger.debug("Volume {} not found in libvirt, falling back to CLVM direct access", volumeUuid);
+ return getPhysicalDiskWithClvmFallback(volumeUuid, pool, libvirtPool);
+ }
+
+ LibvirtStorageVolumeDef voldef = getStorageVolumeDef(libvirtPool.getPool().getConnect(), vol);
+ KVMPhysicalDisk disk = new KVMPhysicalDisk(vol.getPath(), vol.getName(), pool);
+ disk.setSize(vol.getInfo().allocation);
+ disk.setVirtualSize(vol.getInfo().capacity);
+ disk.setFormat(voldef.getFormat() == LibvirtStorageVolumeDef.VolumeFormat.QCOW2
+ ? PhysicalDiskFormat.QCOW2 : PhysicalDiskFormat.RAW);
+ return disk;
+ } catch (LibvirtException e) {
+ logger.warn("LibvirtException looking up volume {}: {}", volumeUuid, e.getMessage());
+ return getPhysicalDiskWithClvmFallback(volumeUuid, pool, libvirtPool);
+ }
+ }
+
+ @Override
+ public KVMPhysicalDisk createPhysicalDisk(String name, KVMStoragePool pool,
+ PhysicalDiskFormat format, Storage.ProvisioningType provisioningType, long size, byte[] passphrase) {
+ logger.info("Creating CLVM/CLVM_NG volume {} in pool {} with size {}", name, pool.getUuid(), toHumanReadableSize(size));
+
+ if (StoragePoolType.CLVM_NG.equals(pool.getType())) {
+ return createClvmNgDiskWithBacking(name, 0, size, null, pool, provisioningType);
+ } else {
+ return createClvmVolume(name, size, pool);
+ }
+ }
+
+ @Override
+ public boolean connectPhysicalDisk(String name, KVMStoragePool pool, Map<String, String> details, boolean isVMMigrate) {
+ if (isVMMigrate) {
+ logger.info("Activating CLVM/CLVM_NG volume {} in shared mode for VM migration", name);
+ Script activateVol = new Script("lvchange", 10000, logger);
+ activateVol.add("-asy");
+ activateVol.add(pool.getLocalPath() + File.separator + name);
+ String result = activateVol.execute();
+ if (result != null) {
+ logger.error("Failed to activate CLVM/CLVM_NG volume {} in shared mode. Output: {}", name, result);
+ return false;
+ }
+ }
+
+ if (StoragePoolType.CLVM_NG.equals(pool.getType())) {
+ ensureClvmNgBackingFileAccessible(name, pool);
+ }
+
+ return true;
+ }
+
+ @Override
+ public KVMPhysicalDisk createDiskFromTemplate(KVMPhysicalDisk template,
+ String name, PhysicalDiskFormat format, Storage.ProvisioningType provisioningType,
+ long size, KVMStoragePool destPool, int timeout, byte[] passphrase) {
+
+ if (StoragePoolType.CLVM_NG.equals(destPool.getType()) && format == PhysicalDiskFormat.QCOW2) {
+ logger.info("Creating CLVM_NG volume {} with backing file from template {}", name, template.getName());
+ String backingFile = getClvmBackingFile(template, destPool);
+ return createClvmNgDiskWithBacking(name, timeout, size, backingFile, destPool, provisioningType);
+ }
+
+ return super.createDiskFromTemplate(template, name, format, provisioningType, size, destPool, timeout, passphrase);
+ }
+
+ @Override
+ public void createTemplate(String templatePath, String templateUuid, int timeout, KVMStoragePool pool) {
+ String vgName = getVgName(pool.getLocalPath());
+ String lvName = "template-" + templateUuid;
+ String lvPath = "/dev/" + vgName + "/" + lvName;
+
+ if (lvExists(lvPath)) {
+ logger.info("Template LV {} already exists in VG {}. Skipping creation.", lvName, vgName);
+ return;
+ }
+
+ logger.info("Creating new template LV {} in VG {} for template {}", lvName, vgName, templateUuid);
+
+ long virtualSize = getQcow2VirtualSize(templatePath);
+ long physicalSize = getQcow2PhysicalSize(templatePath);
+ long lvSize = virtualSize;
+
+ logger.info("Template source - Physical: {} bytes, Virtual: {} bytes, LV will be: {} bytes", physicalSize, virtualSize, lvSize);
+
+ Script lvcreate = new Script("lvcreate", Duration.millis(timeout), logger);
+ lvcreate.add("-n", lvName);
+ lvcreate.add("-L", lvSize + "B");
+ lvcreate.add("--yes");
+ lvcreate.add(vgName);
+ String result = lvcreate.execute();
+ if (result != null) {
+ throw new CloudRuntimeException("Failed to create LV for CLVM_NG template: " + result);
+ }
+
+ Script qemuImgConvert = new Script("qemu-img", Duration.millis(timeout), logger);
+ qemuImgConvert.add("convert");
+ qemuImgConvert.add(templatePath);
+ qemuImgConvert.add("-O", "qcow2");
+ qemuImgConvert.add("-o", "cluster_size=64k,extended_l2=off,preallocation=off");
+ qemuImgConvert.add(lvPath);
+ result = qemuImgConvert.execute();
+
+ if (result != null) {
+ removeLvOnFailure(lvPath, timeout);
+ throw new CloudRuntimeException("Failed to convert template to CLVM_NG volume: " + result);
+ }
+
+ long actualVirtualSize = getQcow2VirtualSize(lvPath);
+
+ try {
+ ensureTemplateLvInSharedMode(lvPath, true);
+ } catch (CloudRuntimeException e) {
+ logger.error("Failed to activate template LV {} in shared mode. Cleaning up.", lvPath);
+ removeLvOnFailure(lvPath, timeout);
+ throw e;
+ }
+
+ KVMPhysicalDisk templateDisk = new KVMPhysicalDisk(lvPath, lvName, pool);
+ templateDisk.setFormat(PhysicalDiskFormat.QCOW2);
+ templateDisk.setVirtualSize(actualVirtualSize);
+ templateDisk.setSize(lvSize);
+ }
+
+ private StoragePool createCLVMStoragePool(Connect conn, String uuid, String host, String path) {
+ String volgroupPath = "/dev/" + path;
+ String volgroupName = path;
+ volgroupName = volgroupName.replaceFirst("^/", "");
+
+ Script checkVgExists = new Script("vgs", 10000, logger);
+ checkVgExists.add("--noheadings");
+ checkVgExists.add("-o", "vg_name");
+ checkVgExists.add(volgroupName);
+ String vgCheckResult = checkVgExists.execute();
+
+ if (vgCheckResult != null) {
+ logger.error("Volume group {} does not exist or is not accessible", volgroupName);
+ return null;
+ }
+
+ logger.info("Volume group {} verified, creating libvirt pool definition for CLVM/CLVM_NG", volgroupName);
+ LibvirtStoragePoolDef poolDef = new LibvirtStoragePoolDef(
+ LibvirtStoragePoolDef.PoolType.LOGICAL,
+ volgroupName,
+ uuid,
+ null,
+ volgroupName,
+ volgroupPath
+ );
+
+ try {
+ StoragePool pool = conn.storagePoolDefineXML(poolDef.toString(), 0);
+ logger.info("Created libvirt pool definition for CLVM/CLVM_NG VG: {} (pool will remain inactive)", volgroupName);
+ pool.setAutostart(1);
+ return pool;
+ } catch (LibvirtException e) {
+ logger.warn("Failed to define CLVM/CLVM_NG pool in libvirt: {}", e.getMessage());
+ return null;
+ }
+ }
+
+ private void setPoolCapacityFromVgStats(LibvirtStoragePool pool, long[] vgStats, String vgName) {
+ long capacity = vgStats[0];
+ long available = vgStats[1];
+ long used = capacity - available;
+
+ pool.setCapacity(capacity);
+ pool.setAvailable(available);
+ pool.setUsed(used);
+
+ logger.debug("CLVM/CLVM_NG pool {} - Capacity: {}, Used: {}, Available: {}",
+ vgName, toHumanReadableSize(capacity), toHumanReadableSize(used), toHumanReadableSize(available));
+ }
+
+ private long[] getVgStats(String vgName) {
+ Script getVgStats = new Script("vgs", 10000, logger);
+ getVgStats.add("--noheadings");
+ getVgStats.add("--units", "b");
+ getVgStats.add("--nosuffix");
+ getVgStats.add("-o", "vg_size,vg_free");
+ getVgStats.add(vgName);
+
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ String result = getVgStats.execute(parser);
+
+ if (result != null) {
+ String errorMsg = "Failed to get statistics for volume group " + vgName + ": " + result;
+ logger.error(errorMsg);
+ throw new CloudRuntimeException(errorMsg);
+ }
+
+ String output = parser.getLines().trim();
+ String[] lines = output.split("\\n");
+ String dataLine = null;
+
+ for (String line : lines) {
+ line = line.trim();
+ if (!line.isEmpty() && Character.isDigit(line.charAt(0))) {
+ dataLine = line;
+ break;
+ }
+ }
+
+ if (dataLine == null) {
+ String errorMsg = "No valid data line found in vgs output for " + vgName + ": " + output;
+ logger.error(errorMsg);
+ throw new CloudRuntimeException(errorMsg);
+ }
+
+ String[] stats = dataLine.split("\\s+");
+
+ if (stats.length < 2) {
+ String errorMsg = "Unexpected output from vgs command for " + vgName + ": " + dataLine;
+ logger.error(errorMsg);
+ throw new CloudRuntimeException(errorMsg);
+ }
+
+ try {
+ long capacity = Long.parseLong(stats[0].trim());
+ long available = Long.parseLong(stats[1].trim());
+ return new long[]{capacity, available};
+ } catch (NumberFormatException e) {
+ String errorMsg = "Failed to parse VG statistics for " + vgName + ": " + e.getMessage();
+ logger.error(errorMsg);
+ throw new CloudRuntimeException(errorMsg, e);
+ }
+ }
+
+ private KVMStoragePool createVirtualClvmPool(String uuid, String host, String path, StoragePoolType type, Map<String, String> details) {
+ String volgroupName = path.replaceFirst("^/", "");
+ String volgroupPath = "/dev/" + volgroupName;
+
+ logger.info("Creating virtual CLVM/CLVM_NG pool {} without libvirt using direct LVM access", volgroupName);
+
+ long[] vgStats = getVgStats(volgroupName);
+
+ LibvirtStoragePool pool = new LibvirtStoragePool(uuid, volgroupName, type, this, null);
+ pool.setLocalPath(volgroupPath);
+ setPoolCapacityFromVgStats(pool, vgStats, volgroupName);
+
+ if (details != null) {
+ pool.setDetails(details);
+ }
+
+ return pool;
+ }
+
+ /**
+ * CLVM fallback: First tries to refresh libvirt pool to make volume visible,
+ * if that fails, accesses volume directly via block device path.
+ */
+ private KVMPhysicalDisk getPhysicalDiskWithClvmFallback(String volumeUuid, KVMStoragePool pool, LibvirtStoragePool libvirtPool) {
+ logger.info("CLVM volume not visible to libvirt, attempting pool refresh for volume: {}", volumeUuid);
+
+ try {
+ logger.debug("Refreshing libvirt storage pool: {}", pool.getUuid());
+ libvirtPool.getPool().refresh(0);
+
+ StorageVol vol = getVolume(libvirtPool.getPool(), volumeUuid);
+ if (vol != null) {
+ logger.info("Volume found after pool refresh: {}", volumeUuid);
+ LibvirtStorageVolumeDef voldef = getStorageVolumeDef(libvirtPool.getPool().getConnect(), vol);
+ KVMPhysicalDisk disk = new KVMPhysicalDisk(vol.getPath(), vol.getName(), pool);
+ disk.setSize(vol.getInfo().allocation);
+ disk.setVirtualSize(vol.getInfo().capacity);
+ disk.setFormat(voldef.getFormat() == LibvirtStorageVolumeDef.VolumeFormat.QCOW2
+ ? PhysicalDiskFormat.QCOW2 : PhysicalDiskFormat.RAW);
+ return disk;
+ }
+ } catch (LibvirtException refreshEx) {
+ logger.debug("Pool refresh failed or volume still not found: {}", refreshEx.getMessage());
+ }
+
+ logger.info("Falling back to direct block device access for volume: {}", volumeUuid);
+ return getPhysicalDiskViaDirectBlockDevice(volumeUuid, pool);
+ }
+
+ private String getVgName(String sourceDir) {
+ String vgName = sourceDir;
+ if (vgName.startsWith("/")) {
+ String[] parts = vgName.split("/");
+ List<String> tokens = Arrays.stream(parts)
+ .filter(s -> !s.isEmpty()).collect(Collectors.toList());
+
+ vgName = tokens.size() > 1 ? tokens.get(1)
+ : tokens.size() == 1 ? tokens.get(0)
+ : "";
+ }
+ return vgName;
+ }
+
+ private String extractVgNameFromPool(KVMStoragePool pool) {
+ String sourceDir = pool.getLocalPath();
+ if (sourceDir == null || sourceDir.isEmpty()) {
+ throw new CloudRuntimeException("CLVM pool sourceDir is not set, cannot determine VG name");
+ }
+ String vgName = getVgName(sourceDir);
+ logger.debug("Using VG name: {} (from sourceDir: {})", vgName, sourceDir);
+ return vgName;
+ }
+
+ /**
+ * For CLVM volumes that exist in LVM but are not visible to libvirt,
+ * access them directly via block device path.
+ */
+ private KVMPhysicalDisk getPhysicalDiskViaDirectBlockDevice(String volumeUuid, KVMStoragePool pool) {
+ try {
+ String vgName = extractVgNameFromPool(pool);
+
+ verifyLvExistsInVg(volumeUuid, vgName);
+
+ logger.info("Volume {} exists in LVM but not visible to libvirt, accessing directly", volumeUuid);
+
+ String lvPath = findAccessibleDeviceNode(volumeUuid, vgName, pool);
+ long size = getClvmVolumeSize(lvPath);
+
+ KVMPhysicalDisk disk = createPhysicalDiskFromClvmLv(lvPath, volumeUuid, pool, size);
+ ensureTemplateAccessibility(volumeUuid, lvPath, pool);
+
+ return disk;
+ } catch (CloudRuntimeException ex) {
+ throw ex;
+ } catch (Exception ex) {
+ logger.error("Failed to access CLVM volume via direct block device: {}", volumeUuid, ex);
+ throw new CloudRuntimeException(String.format("Could not find volume %s: %s", volumeUuid, ex.getMessage()));
+ }
+ }
+
+ private void verifyLvExistsInVg(String volumeUuid, String vgName) {
+ logger.debug("Checking if volume {} exists in VG {}", volumeUuid, vgName);
+ Script checkLvCmd = new Script("/usr/sbin/lvs", 10000, logger);
+ checkLvCmd.add("--noheadings");
+ checkLvCmd.add("--unbuffered");
+ checkLvCmd.add(vgName + "/" + volumeUuid);
+ String checkResult = checkLvCmd.execute();
+ if (checkResult != null) {
+ throw new CloudRuntimeException(String.format("Storage volume not found: no storage vol with matching name '%s'", volumeUuid));
+ }
+ }
+
+ private String findAccessibleDeviceNode(String volumeUuid, String vgName, KVMStoragePool pool) {
+ String lvPath = "/dev/" + vgName + "/" + volumeUuid;
+ File lvDevice = new File(lvPath);
+
+ if (!lvDevice.exists()) {
+ lvPath = tryDeviceMapperPath(volumeUuid, vgName);
+ if (!new File(lvPath).exists()) {
+ lvPath = handleMissingDeviceNode(volumeUuid, vgName, pool);
+ }
+ }
+
+ return lvPath;
+ }
+
+ private String tryDeviceMapperPath(String volumeUuid, String vgName) {
+ String vgNameEscaped = vgName.replace("-", "--");
+ String volumeUuidEscaped = volumeUuid.replace("-", "--");
+ return "/dev/mapper/" + vgNameEscaped + "-" + volumeUuidEscaped;
+ }
+
+ private String handleMissingDeviceNode(String volumeUuid, String vgName, KVMStoragePool pool) {
+ if (StoragePoolType.CLVM_NG.equals(pool.getType()) && volumeUuid.startsWith("template-")) {
+ return activateTemplateAndGetPath(volumeUuid, vgName);
+ }
+ throw new CloudRuntimeException(String.format("Could not find volume %s in VG %s - volume exists in LVM but device node not accessible", volumeUuid, vgName));
+ }
+
+ private String activateTemplateAndGetPath(String volumeUuid, String vgName) {
+ logger.info("Template volume {} device node not found. Attempting to activate in shared mode.", volumeUuid);
+ String templateLvPath = "/dev/" + vgName + "/" + volumeUuid;
+
+ try {
+ ensureTemplateLvInSharedMode(templateLvPath, false);
+
+ String lvPath = findDeviceNodeAfterActivation(templateLvPath, volumeUuid, vgName);
+
+ logger.info("Successfully activated template volume {} at {}", volumeUuid, lvPath);
+ return lvPath;
+ } catch (CloudRuntimeException e) {
+ throw new CloudRuntimeException(String.format("Failed to activate template volume %s in VG %s: %s", volumeUuid, vgName, e.getMessage()), e);
+ }
+ }
+
+ private String findDeviceNodeAfterActivation(String templateLvPath, String volumeUuid, String vgName) {
+ File lvDevice = new File(templateLvPath);
+ String lvPath = templateLvPath;
+
+ if (!lvDevice.exists()) {
+ String vgNameEscaped = vgName.replace("-", "--");
+ String volumeUuidEscaped = volumeUuid.replace("-", "--");
+ lvPath = "/dev/mapper/" + vgNameEscaped + "-" + volumeUuidEscaped;
+ lvDevice = new File(lvPath);
+ }
+
+ if (!lvDevice.exists()) {
+ logger.error("Template volume {} still not accessible after activation attempt", volumeUuid);
+ throw new CloudRuntimeException(String.format("Could not activate template volume %s in VG %s - device node not accessible after activation", volumeUuid, vgName));
+ }
+
+ return lvPath;
+ }
+
+ private void ensureTemplateAccessibility(String volumeUuid, String lvPath, KVMStoragePool pool) {
+ if (StoragePoolType.CLVM_NG.equals(pool.getType()) && volumeUuid.startsWith("template-")) {
+ logger.info("Detected template volume {}. Ensuring it's activated in shared mode.", volumeUuid);
+ ensureTemplateLvInSharedMode(lvPath, false);
+ }
+ }
+
+ private long getClvmVolumeSize(String lvPath) {
+ try {
+ Script lvsCmd = new Script("/usr/sbin/lvs", 10000, logger);
+ lvsCmd.add("--noheadings");
+ lvsCmd.add("--units");
+ lvsCmd.add("b");
+ lvsCmd.add("-o");
+ lvsCmd.add("lv_size");
+ lvsCmd.add(lvPath);
+
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ String result = lvsCmd.execute(parser);
+
+ String output = (result == null) ? parser.getLines() : result;
+
+ if (output != null && !output.isEmpty()) {
+ String sizeStr = output.trim().replaceAll("[^0-9]", "");
+ if (!sizeStr.isEmpty()) {
+ return Long.parseLong(sizeStr);
+ }
+ }
+ } catch (Exception sizeEx) {
+ logger.warn("Failed to get size for CLVM volume via lvs: {}", sizeEx.getMessage());
+ File lvDevice = new File(lvPath);
+ if (lvDevice.isFile()) {
+ return lvDevice.length();
+ }
+ }
+ return 0;
+ }
+
+ private KVMPhysicalDisk createPhysicalDiskFromClvmLv(String lvPath, String volumeUuid, KVMStoragePool pool, long size) {
+ PhysicalDiskFormat diskFormat = StoragePoolType.CLVM_NG.equals(pool.getType())
+ ? PhysicalDiskFormat.QCOW2 : PhysicalDiskFormat.RAW;
+
+ logger.debug("{} pool detected, setting disk format to {} for volume {}", pool.getType(), diskFormat, volumeUuid);
+
+ KVMPhysicalDisk disk = new KVMPhysicalDisk(lvPath, volumeUuid, pool);
+ disk.setFormat(diskFormat);
+ disk.setSize(size);
+ disk.setVirtualSize(size);
+
+ logger.info("Successfully accessed CLVM/CLVM_NG volume via direct block device: {} with format: {} and size: {} bytes",
+ lvPath, diskFormat, size);
+ return disk;
+ }
+
+ /**
+ * Checks if a CLVM_NG QCOW2 volume has a backing file (template) and ensures it's activated in shared mode.
+ */
+ private void ensureClvmNgBackingFileAccessible(String volumeName, KVMStoragePool pool) {
+ try {
+ String vgName = getVgName(pool.getLocalPath());
+ String volumePath = "/dev/" + vgName + "/" + volumeName;
+
+ logger.debug("Checking if CLVM_NG volume {} has a backing file that needs activation", volumePath);
+
+ Script qemuImgInfo = new Script("qemu-img", Duration.millis(10000), logger);
+ qemuImgInfo.add("info");
+ qemuImgInfo.add("--output=json");
+ qemuImgInfo.add(volumePath);
+
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ String result = qemuImgInfo.execute(parser);
+
+ if (result == null && parser.getLines() != null && !parser.getLines().isEmpty()) {
+ String jsonOutput = parser.getLines();
+
+ if (jsonOutput.contains("\"backing-filename\"")) {
+ int backingStart = jsonOutput.indexOf("\"backing-filename\"");
+ if (backingStart > 0) {
+ int valueStart = jsonOutput.indexOf(":", backingStart);
+ if (valueStart > 0) {
+ valueStart = jsonOutput.indexOf("\"", valueStart) + 1;
+ int valueEnd = jsonOutput.indexOf("\"", valueStart);
+
+ if (valueEnd > valueStart) {
+ String backingFile = jsonOutput.substring(valueStart, valueEnd).trim();
+ if (!backingFile.isEmpty() && backingFile.startsWith("/dev/")) {
+ logger.info("Volume {} has backing file: {}. Ensuring backing file is in shared mode.", volumePath, backingFile);
+ ensureTemplateLvInSharedMode(backingFile, false);
+ }
+ }
+ }
+ }
Review Comment:
There must be a better way to parse this...
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/ClvmStorageAdaptor.java:
##########
@@ -0,0 +1,1072 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.hypervisor.kvm.storage;
+
+import static com.cloud.utils.NumbersUtil.toHumanReadableSize;
+
+import java.io.File;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.cloud.hypervisor.kvm.resource.LibvirtComputingResource;
+import com.cloud.hypervisor.kvm.resource.LibvirtConnection;
+import com.cloud.hypervisor.kvm.resource.LibvirtStoragePoolDef;
+import com.cloud.hypervisor.kvm.resource.LibvirtStorageVolumeDef;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Storage.StoragePoolType;
+import com.cloud.storage.StorageLayer;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.script.OutputInterpreter;
+import com.cloud.utils.script.Script;
+import com.google.gson.JsonObject;
+import com.google.gson.JsonParser;
+import org.apache.cloudstack.utils.qemu.QemuImg.PhysicalDiskFormat;
+import org.joda.time.Duration;
+import org.libvirt.Connect;
+import org.libvirt.LibvirtException;
+import org.libvirt.StoragePool;
+import org.libvirt.StorageVol;
+
+/**
+ * Storage adaptor for CLVM and CLVM_NG pool types.
+ * Extends LibvirtStorageAdaptor and overrides methods with CLVM-specific logic,
+ * using direct LVM commands instead of libvirt for volume operations.
+ */
+public class ClvmStorageAdaptor extends LibvirtStorageAdaptor {
+
+ public ClvmStorageAdaptor(StorageLayer storage) {
+ super(storage);
+ }
+
+ @Override
+ public StoragePoolType getStoragePoolType() {
+ // Registered manually for both CLVM and CLVM_NG in KVMStoragePoolManager
+ return null;
+ }
+
+ @Override
+ public KVMStoragePool createStoragePool(String name, String host, int port, String path,
+ String userInfo, StoragePoolType type, Map<String, String> details, boolean isPrimaryStorage) {
+ logger.info("Attempting to create CLVM/CLVM_NG storage pool {} in libvirt", name);
+
+ Connect conn;
+ try {
+ conn = LibvirtConnection.getConnection();
+ } catch (LibvirtException e) {
+ throw new CloudRuntimeException(e.toString());
+ }
+
+ StoragePool sp = createCLVMStoragePool(conn, name, host, path);
+ if (sp == null) {
+ logger.info("Falling back to virtual CLVM/CLVM_NG pool without libvirt for: {}", name);
+ return createVirtualClvmPool(name, host, path, type, details);
+ }
+
+ try {
+ if (!isPrimaryStorage) {
+ incStoragePoolRefCount(name);
+ }
+ // CLVM/CLVM_NG pools are kept inactive in libvirt; we use direct LVM commands
+ return getStoragePool(name);
+ } catch (Exception e) {
+ decStoragePoolRefCount(name);
+ throw new CloudRuntimeException("Failed to create CLVM storage pool: " + name, e);
+ }
+ }
+
+ @Override
+ public KVMStoragePool getStoragePool(String uuid, boolean refreshInfo) {
+ logger.info("Fetching CLVM/CLVM_NG storage pool {} ", uuid);
+ try {
+ Connect conn = LibvirtConnection.getConnection();
+ StoragePool storage = conn.storagePoolLookupByUUIDString(uuid);
+
+ LibvirtStoragePoolDef spd = getStoragePoolDef(conn, storage);
+ if (spd == null) {
+ throw new CloudRuntimeException("Unable to parse storage pool definition for pool " + uuid);
+ }
+
+ // CLVM pools in libvirt are always LOGICAL type
+ StoragePoolType type = StoragePoolType.CLVM;
+
+ // Do NOT activate the pool — CLVM/CLVM_NG pools stay inactive in libvirt
+ LibvirtStoragePool pool = new LibvirtStoragePool(uuid, storage.getName(), type, this, storage);
+ pool.setLocalPath(spd.getTargetPath());
+
+ // Always read capacity from LVM directly
+ String vgName = storage.getName();
+ try {
+ long[] vgStats = getVgStats(vgName);
+ setPoolCapacityFromVgStats(pool, vgStats, vgName);
+ } catch (CloudRuntimeException e) {
+ logger.warn("Failed to get VG stats for CLVM/CLVM_NG pool {}: {}. Using libvirt values (may be 0)", vgName, e.getMessage());
+ pool.setCapacity(storage.getInfo().capacity);
+ pool.setUsed(storage.getInfo().allocation);
+ pool.setAvailable(storage.getInfo().available);
+ }
+
+ return pool;
+ } catch (LibvirtException e) {
+ // Pool not in libvirt - return virtual pool backed only by LVM
+ logger.debug("CLVM/CLVM_NG pool {} not found in libvirt, creating virtual pool", uuid);
+ throw new CloudRuntimeException(e.toString(), e);
+ }
+ }
+
+ @Override
+ public KVMPhysicalDisk getPhysicalDisk(String volumeUuid, KVMStoragePool pool) {
+ LibvirtStoragePool libvirtPool = (LibvirtStoragePool) pool;
+
+ // Pool has no libvirt backing - go directly to block device
+ if (libvirtPool.getPool() == null) {
+ logger.debug("CLVM/CLVM_NG pool has no libvirt backing, using direct block device access for volume: {}", volumeUuid);
+ return getPhysicalDiskViaDirectBlockDevice(volumeUuid, pool);
+ }
+
+ try {
+ StorageVol vol = getVolume(libvirtPool.getPool(), volumeUuid);
+ if (vol == null) {
+ logger.debug("Volume {} not found in libvirt, falling back to CLVM direct access", volumeUuid);
+ return getPhysicalDiskWithClvmFallback(volumeUuid, pool, libvirtPool);
+ }
+
+ LibvirtStorageVolumeDef voldef = getStorageVolumeDef(libvirtPool.getPool().getConnect(), vol);
+ KVMPhysicalDisk disk = new KVMPhysicalDisk(vol.getPath(), vol.getName(), pool);
+ disk.setSize(vol.getInfo().allocation);
+ disk.setVirtualSize(vol.getInfo().capacity);
+ disk.setFormat(voldef.getFormat() == LibvirtStorageVolumeDef.VolumeFormat.QCOW2
+ ? PhysicalDiskFormat.QCOW2 : PhysicalDiskFormat.RAW);
+ return disk;
+ } catch (LibvirtException e) {
+ logger.warn("LibvirtException looking up volume {}: {}", volumeUuid, e.getMessage());
+ return getPhysicalDiskWithClvmFallback(volumeUuid, pool, libvirtPool);
+ }
+ }
+
+ @Override
+ public KVMPhysicalDisk createPhysicalDisk(String name, KVMStoragePool pool,
+ PhysicalDiskFormat format, Storage.ProvisioningType provisioningType, long size, byte[] passphrase) {
+ logger.info("Creating CLVM/CLVM_NG volume {} in pool {} with size {}", name, pool.getUuid(), toHumanReadableSize(size));
+
+ if (StoragePoolType.CLVM_NG.equals(pool.getType())) {
+ return createClvmNgDiskWithBacking(name, 0, size, null, pool, provisioningType);
+ } else {
+ return createClvmVolume(name, size, pool);
+ }
+ }
+
+ @Override
+ public boolean connectPhysicalDisk(String name, KVMStoragePool pool, Map<String, String> details, boolean isVMMigrate) {
+ if (isVMMigrate) {
+ logger.info("Activating CLVM/CLVM_NG volume {} in shared mode for VM migration", name);
+ Script activateVol = new Script("lvchange", 10000, logger);
+ activateVol.add("-asy");
+ activateVol.add(pool.getLocalPath() + File.separator + name);
+ String result = activateVol.execute();
+ if (result != null) {
+ logger.error("Failed to activate CLVM/CLVM_NG volume {} in shared mode. Output: {}", name, result);
+ return false;
+ }
+ }
+
+ if (StoragePoolType.CLVM_NG.equals(pool.getType())) {
+ ensureClvmNgBackingFileAccessible(name, pool);
+ }
+
+ return true;
+ }
+
+ @Override
+ public KVMPhysicalDisk createDiskFromTemplate(KVMPhysicalDisk template,
+ String name, PhysicalDiskFormat format, Storage.ProvisioningType provisioningType,
+ long size, KVMStoragePool destPool, int timeout, byte[] passphrase) {
+
+ if (StoragePoolType.CLVM_NG.equals(destPool.getType()) && format == PhysicalDiskFormat.QCOW2) {
+ logger.info("Creating CLVM_NG volume {} with backing file from template {}", name, template.getName());
+ String backingFile = getClvmBackingFile(template, destPool);
+ return createClvmNgDiskWithBacking(name, timeout, size, backingFile, destPool, provisioningType);
+ }
+
+ return super.createDiskFromTemplate(template, name, format, provisioningType, size, destPool, timeout, passphrase);
+ }
+
+ @Override
+ public void createTemplate(String templatePath, String templateUuid, int timeout, KVMStoragePool pool) {
+ String vgName = getVgName(pool.getLocalPath());
+ String lvName = "template-" + templateUuid;
+ String lvPath = "/dev/" + vgName + "/" + lvName;
+
+ if (lvExists(lvPath)) {
+ logger.info("Template LV {} already exists in VG {}. Skipping creation.", lvName, vgName);
+ return;
+ }
+
+ logger.info("Creating new template LV {} in VG {} for template {}", lvName, vgName, templateUuid);
+
+ long virtualSize = getQcow2VirtualSize(templatePath);
+ long physicalSize = getQcow2PhysicalSize(templatePath);
+ long lvSize = virtualSize;
+
+ logger.info("Template source - Physical: {} bytes, Virtual: {} bytes, LV will be: {} bytes", physicalSize, virtualSize, lvSize);
+
+ Script lvcreate = new Script("lvcreate", Duration.millis(timeout), logger);
+ lvcreate.add("-n", lvName);
+ lvcreate.add("-L", lvSize + "B");
+ lvcreate.add("--yes");
+ lvcreate.add(vgName);
+ String result = lvcreate.execute();
+ if (result != null) {
+ throw new CloudRuntimeException("Failed to create LV for CLVM_NG template: " + result);
+ }
+
+ Script qemuImgConvert = new Script("qemu-img", Duration.millis(timeout), logger);
+ qemuImgConvert.add("convert");
+ qemuImgConvert.add(templatePath);
+ qemuImgConvert.add("-O", "qcow2");
+ qemuImgConvert.add("-o", "cluster_size=64k,extended_l2=off,preallocation=off");
+ qemuImgConvert.add(lvPath);
+ result = qemuImgConvert.execute();
+
+ if (result != null) {
+ removeLvOnFailure(lvPath, timeout);
+ throw new CloudRuntimeException("Failed to convert template to CLVM_NG volume: " + result);
+ }
+
+ long actualVirtualSize = getQcow2VirtualSize(lvPath);
+
+ try {
+ ensureTemplateLvInSharedMode(lvPath, true);
+ } catch (CloudRuntimeException e) {
+ logger.error("Failed to activate template LV {} in shared mode. Cleaning up.", lvPath);
+ removeLvOnFailure(lvPath, timeout);
+ throw e;
+ }
+
+ KVMPhysicalDisk templateDisk = new KVMPhysicalDisk(lvPath, lvName, pool);
+ templateDisk.setFormat(PhysicalDiskFormat.QCOW2);
+ templateDisk.setVirtualSize(actualVirtualSize);
+ templateDisk.setSize(lvSize);
+ }
+
+ private StoragePool createCLVMStoragePool(Connect conn, String uuid, String host, String path) {
+ String volgroupPath = "/dev/" + path;
+ String volgroupName = path;
+ volgroupName = volgroupName.replaceFirst("^/", "");
+
+ Script checkVgExists = new Script("vgs", 10000, logger);
+ checkVgExists.add("--noheadings");
+ checkVgExists.add("-o", "vg_name");
+ checkVgExists.add(volgroupName);
+ String vgCheckResult = checkVgExists.execute();
+
+ if (vgCheckResult != null) {
+ logger.error("Volume group {} does not exist or is not accessible", volgroupName);
+ return null;
+ }
+
+ logger.info("Volume group {} verified, creating libvirt pool definition for CLVM/CLVM_NG", volgroupName);
+ LibvirtStoragePoolDef poolDef = new LibvirtStoragePoolDef(
+ LibvirtStoragePoolDef.PoolType.LOGICAL,
+ volgroupName,
+ uuid,
+ null,
+ volgroupName,
+ volgroupPath
+ );
+
+ try {
+ StoragePool pool = conn.storagePoolDefineXML(poolDef.toString(), 0);
+ logger.info("Created libvirt pool definition for CLVM/CLVM_NG VG: {} (pool will remain inactive)", volgroupName);
+ pool.setAutostart(1);
+ return pool;
+ } catch (LibvirtException e) {
+ logger.warn("Failed to define CLVM/CLVM_NG pool in libvirt: {}", e.getMessage());
+ return null;
+ }
+ }
+
+ private void setPoolCapacityFromVgStats(LibvirtStoragePool pool, long[] vgStats, String vgName) {
+ long capacity = vgStats[0];
+ long available = vgStats[1];
+ long used = capacity - available;
+
+ pool.setCapacity(capacity);
+ pool.setAvailable(available);
+ pool.setUsed(used);
+
+ logger.debug("CLVM/CLVM_NG pool {} - Capacity: {}, Used: {}, Available: {}",
+ vgName, toHumanReadableSize(capacity), toHumanReadableSize(used), toHumanReadableSize(available));
+ }
+
+ private long[] getVgStats(String vgName) {
+ Script getVgStats = new Script("vgs", 10000, logger);
+ getVgStats.add("--noheadings");
+ getVgStats.add("--units", "b");
+ getVgStats.add("--nosuffix");
+ getVgStats.add("-o", "vg_size,vg_free");
+ getVgStats.add(vgName);
+
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ String result = getVgStats.execute(parser);
+
+ if (result != null) {
+ String errorMsg = "Failed to get statistics for volume group " + vgName + ": " + result;
+ logger.error(errorMsg);
+ throw new CloudRuntimeException(errorMsg);
+ }
+
+ String output = parser.getLines().trim();
+ String[] lines = output.split("\\n");
+ String dataLine = null;
+
+ for (String line : lines) {
+ line = line.trim();
+ if (!line.isEmpty() && Character.isDigit(line.charAt(0))) {
+ dataLine = line;
+ break;
+ }
+ }
+
+ if (dataLine == null) {
+ String errorMsg = "No valid data line found in vgs output for " + vgName + ": " + output;
+ logger.error(errorMsg);
+ throw new CloudRuntimeException(errorMsg);
+ }
+
+ String[] stats = dataLine.split("\\s+");
+
+ if (stats.length < 2) {
+ String errorMsg = "Unexpected output from vgs command for " + vgName + ": " + dataLine;
+ logger.error(errorMsg);
+ throw new CloudRuntimeException(errorMsg);
+ }
+
+ try {
+ long capacity = Long.parseLong(stats[0].trim());
+ long available = Long.parseLong(stats[1].trim());
+ return new long[]{capacity, available};
+ } catch (NumberFormatException e) {
+ String errorMsg = "Failed to parse VG statistics for " + vgName + ": " + e.getMessage();
+ logger.error(errorMsg);
+ throw new CloudRuntimeException(errorMsg, e);
+ }
+ }
+
+ private KVMStoragePool createVirtualClvmPool(String uuid, String host, String path, StoragePoolType type, Map<String, String> details) {
+ String volgroupName = path.replaceFirst("^/", "");
+ String volgroupPath = "/dev/" + volgroupName;
+
+ logger.info("Creating virtual CLVM/CLVM_NG pool {} without libvirt using direct LVM access", volgroupName);
+
+ long[] vgStats = getVgStats(volgroupName);
+
+ LibvirtStoragePool pool = new LibvirtStoragePool(uuid, volgroupName, type, this, null);
+ pool.setLocalPath(volgroupPath);
+ setPoolCapacityFromVgStats(pool, vgStats, volgroupName);
+
+ if (details != null) {
+ pool.setDetails(details);
+ }
+
+ return pool;
+ }
+
+ /**
+ * CLVM fallback: First tries to refresh libvirt pool to make volume visible,
+ * if that fails, accesses volume directly via block device path.
+ */
+ private KVMPhysicalDisk getPhysicalDiskWithClvmFallback(String volumeUuid, KVMStoragePool pool, LibvirtStoragePool libvirtPool) {
+ logger.info("CLVM volume not visible to libvirt, attempting pool refresh for volume: {}", volumeUuid);
+
+ try {
+ logger.debug("Refreshing libvirt storage pool: {}", pool.getUuid());
+ libvirtPool.getPool().refresh(0);
+
+ StorageVol vol = getVolume(libvirtPool.getPool(), volumeUuid);
+ if (vol != null) {
+ logger.info("Volume found after pool refresh: {}", volumeUuid);
+ LibvirtStorageVolumeDef voldef = getStorageVolumeDef(libvirtPool.getPool().getConnect(), vol);
+ KVMPhysicalDisk disk = new KVMPhysicalDisk(vol.getPath(), vol.getName(), pool);
+ disk.setSize(vol.getInfo().allocation);
+ disk.setVirtualSize(vol.getInfo().capacity);
+ disk.setFormat(voldef.getFormat() == LibvirtStorageVolumeDef.VolumeFormat.QCOW2
+ ? PhysicalDiskFormat.QCOW2 : PhysicalDiskFormat.RAW);
+ return disk;
+ }
+ } catch (LibvirtException refreshEx) {
+ logger.debug("Pool refresh failed or volume still not found: {}", refreshEx.getMessage());
+ }
+
+ logger.info("Falling back to direct block device access for volume: {}", volumeUuid);
+ return getPhysicalDiskViaDirectBlockDevice(volumeUuid, pool);
+ }
+
+ private String getVgName(String sourceDir) {
+ String vgName = sourceDir;
+ if (vgName.startsWith("/")) {
+ String[] parts = vgName.split("/");
+ List<String> tokens = Arrays.stream(parts)
+ .filter(s -> !s.isEmpty()).collect(Collectors.toList());
+
+ vgName = tokens.size() > 1 ? tokens.get(1)
+ : tokens.size() == 1 ? tokens.get(0)
+ : "";
+ }
+ return vgName;
+ }
+
+ private String extractVgNameFromPool(KVMStoragePool pool) {
+ String sourceDir = pool.getLocalPath();
+ if (sourceDir == null || sourceDir.isEmpty()) {
+ throw new CloudRuntimeException("CLVM pool sourceDir is not set, cannot determine VG name");
+ }
+ String vgName = getVgName(sourceDir);
+ logger.debug("Using VG name: {} (from sourceDir: {})", vgName, sourceDir);
+ return vgName;
+ }
+
+ /**
+ * For CLVM volumes that exist in LVM but are not visible to libvirt,
+ * access them directly via block device path.
+ */
+ private KVMPhysicalDisk getPhysicalDiskViaDirectBlockDevice(String volumeUuid, KVMStoragePool pool) {
+ try {
+ String vgName = extractVgNameFromPool(pool);
+
+ verifyLvExistsInVg(volumeUuid, vgName);
+
+ logger.info("Volume {} exists in LVM but not visible to libvirt, accessing directly", volumeUuid);
+
+ String lvPath = findAccessibleDeviceNode(volumeUuid, vgName, pool);
+ long size = getClvmVolumeSize(lvPath);
+
+ KVMPhysicalDisk disk = createPhysicalDiskFromClvmLv(lvPath, volumeUuid, pool, size);
+ ensureTemplateAccessibility(volumeUuid, lvPath, pool);
+
+ return disk;
+ } catch (CloudRuntimeException ex) {
+ throw ex;
+ } catch (Exception ex) {
+ logger.error("Failed to access CLVM volume via direct block device: {}", volumeUuid, ex);
+ throw new CloudRuntimeException(String.format("Could not find volume %s: %s", volumeUuid, ex.getMessage()));
+ }
+ }
+
+ private void verifyLvExistsInVg(String volumeUuid, String vgName) {
+ logger.debug("Checking if volume {} exists in VG {}", volumeUuid, vgName);
+ Script checkLvCmd = new Script("/usr/sbin/lvs", 10000, logger);
+ checkLvCmd.add("--noheadings");
+ checkLvCmd.add("--unbuffered");
+ checkLvCmd.add(vgName + "/" + volumeUuid);
+ String checkResult = checkLvCmd.execute();
+ if (checkResult != null) {
+ throw new CloudRuntimeException(String.format("Storage volume not found: no storage vol with matching name '%s'", volumeUuid));
+ }
+ }
+
+ private String findAccessibleDeviceNode(String volumeUuid, String vgName, KVMStoragePool pool) {
+ String lvPath = "/dev/" + vgName + "/" + volumeUuid;
+ File lvDevice = new File(lvPath);
+
+ if (!lvDevice.exists()) {
+ lvPath = tryDeviceMapperPath(volumeUuid, vgName);
+ if (!new File(lvPath).exists()) {
+ lvPath = handleMissingDeviceNode(volumeUuid, vgName, pool);
+ }
+ }
+
+ return lvPath;
+ }
+
+ private String tryDeviceMapperPath(String volumeUuid, String vgName) {
+ String vgNameEscaped = vgName.replace("-", "--");
+ String volumeUuidEscaped = volumeUuid.replace("-", "--");
+ return "/dev/mapper/" + vgNameEscaped + "-" + volumeUuidEscaped;
+ }
+
+ private String handleMissingDeviceNode(String volumeUuid, String vgName, KVMStoragePool pool) {
+ if (StoragePoolType.CLVM_NG.equals(pool.getType()) && volumeUuid.startsWith("template-")) {
+ return activateTemplateAndGetPath(volumeUuid, vgName);
+ }
+ throw new CloudRuntimeException(String.format("Could not find volume %s in VG %s - volume exists in LVM but device node not accessible", volumeUuid, vgName));
+ }
+
+ private String activateTemplateAndGetPath(String volumeUuid, String vgName) {
+ logger.info("Template volume {} device node not found. Attempting to activate in shared mode.", volumeUuid);
+ String templateLvPath = "/dev/" + vgName + "/" + volumeUuid;
+
+ try {
+ ensureTemplateLvInSharedMode(templateLvPath, false);
+
+ String lvPath = findDeviceNodeAfterActivation(templateLvPath, volumeUuid, vgName);
+
+ logger.info("Successfully activated template volume {} at {}", volumeUuid, lvPath);
+ return lvPath;
+ } catch (CloudRuntimeException e) {
+ throw new CloudRuntimeException(String.format("Failed to activate template volume %s in VG %s: %s", volumeUuid, vgName, e.getMessage()), e);
+ }
+ }
+
+ private String findDeviceNodeAfterActivation(String templateLvPath, String volumeUuid, String vgName) {
+ File lvDevice = new File(templateLvPath);
+ String lvPath = templateLvPath;
+
+ if (!lvDevice.exists()) {
+ String vgNameEscaped = vgName.replace("-", "--");
+ String volumeUuidEscaped = volumeUuid.replace("-", "--");
+ lvPath = "/dev/mapper/" + vgNameEscaped + "-" + volumeUuidEscaped;
+ lvDevice = new File(lvPath);
+ }
+
+ if (!lvDevice.exists()) {
+ logger.error("Template volume {} still not accessible after activation attempt", volumeUuid);
+ throw new CloudRuntimeException(String.format("Could not activate template volume %s in VG %s - device node not accessible after activation", volumeUuid, vgName));
+ }
+
+ return lvPath;
+ }
+
+ private void ensureTemplateAccessibility(String volumeUuid, String lvPath, KVMStoragePool pool) {
+ if (StoragePoolType.CLVM_NG.equals(pool.getType()) && volumeUuid.startsWith("template-")) {
+ logger.info("Detected template volume {}. Ensuring it's activated in shared mode.", volumeUuid);
+ ensureTemplateLvInSharedMode(lvPath, false);
+ }
+ }
+
+ private long getClvmVolumeSize(String lvPath) {
+ try {
+ Script lvsCmd = new Script("/usr/sbin/lvs", 10000, logger);
+ lvsCmd.add("--noheadings");
+ lvsCmd.add("--units");
+ lvsCmd.add("b");
+ lvsCmd.add("-o");
+ lvsCmd.add("lv_size");
+ lvsCmd.add(lvPath);
+
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ String result = lvsCmd.execute(parser);
+
+ String output = (result == null) ? parser.getLines() : result;
+
+ if (output != null && !output.isEmpty()) {
+ String sizeStr = output.trim().replaceAll("[^0-9]", "");
+ if (!sizeStr.isEmpty()) {
+ return Long.parseLong(sizeStr);
+ }
+ }
+ } catch (Exception sizeEx) {
+ logger.warn("Failed to get size for CLVM volume via lvs: {}", sizeEx.getMessage());
+ File lvDevice = new File(lvPath);
+ if (lvDevice.isFile()) {
+ return lvDevice.length();
+ }
+ }
+ return 0;
+ }
+
+ private KVMPhysicalDisk createPhysicalDiskFromClvmLv(String lvPath, String volumeUuid, KVMStoragePool pool, long size) {
+ PhysicalDiskFormat diskFormat = StoragePoolType.CLVM_NG.equals(pool.getType())
+ ? PhysicalDiskFormat.QCOW2 : PhysicalDiskFormat.RAW;
+
+ logger.debug("{} pool detected, setting disk format to {} for volume {}", pool.getType(), diskFormat, volumeUuid);
+
+ KVMPhysicalDisk disk = new KVMPhysicalDisk(lvPath, volumeUuid, pool);
+ disk.setFormat(diskFormat);
+ disk.setSize(size);
+ disk.setVirtualSize(size);
+
+ logger.info("Successfully accessed CLVM/CLVM_NG volume via direct block device: {} with format: {} and size: {} bytes",
+ lvPath, diskFormat, size);
+ return disk;
+ }
+
+ /**
+ * Checks if a CLVM_NG QCOW2 volume has a backing file (template) and ensures it's activated in shared mode.
+ */
+ private void ensureClvmNgBackingFileAccessible(String volumeName, KVMStoragePool pool) {
+ try {
+ String vgName = getVgName(pool.getLocalPath());
+ String volumePath = "/dev/" + vgName + "/" + volumeName;
+
+ logger.debug("Checking if CLVM_NG volume {} has a backing file that needs activation", volumePath);
+
+ Script qemuImgInfo = new Script("qemu-img", Duration.millis(10000), logger);
+ qemuImgInfo.add("info");
+ qemuImgInfo.add("--output=json");
+ qemuImgInfo.add(volumePath);
+
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ String result = qemuImgInfo.execute(parser);
+
+ if (result == null && parser.getLines() != null && !parser.getLines().isEmpty()) {
+ String jsonOutput = parser.getLines();
+
+ if (jsonOutput.contains("\"backing-filename\"")) {
+ int backingStart = jsonOutput.indexOf("\"backing-filename\"");
+ if (backingStart > 0) {
+ int valueStart = jsonOutput.indexOf(":", backingStart);
+ if (valueStart > 0) {
+ valueStart = jsonOutput.indexOf("\"", valueStart) + 1;
+ int valueEnd = jsonOutput.indexOf("\"", valueStart);
+
+ if (valueEnd > valueStart) {
+ String backingFile = jsonOutput.substring(valueStart, valueEnd).trim();
+ if (!backingFile.isEmpty() && backingFile.startsWith("/dev/")) {
+ logger.info("Volume {} has backing file: {}. Ensuring backing file is in shared mode.", volumePath, backingFile);
+ ensureTemplateLvInSharedMode(backingFile, false);
+ }
+ }
+ }
+ }
+ } else {
+ logger.debug("Volume {} does not have a backing file (full clone)", volumePath);
+ }
+ }
+ } catch (Exception e) {
+ logger.warn("Failed to check/activate backing file for volume {}: {}. VM deployment may fail if template is not accessible.",
+ volumeName, e.getMessage());
+ }
+ }
+
+ private String getClvmBackingFile(KVMPhysicalDisk template, KVMStoragePool destPool) {
+ String templateLvName = template.getName();
+ KVMPhysicalDisk templateOnPrimary = null;
+
+ try {
+ templateOnPrimary = destPool.getPhysicalDisk(templateLvName);
+ } catch (CloudRuntimeException e) {
+ logger.warn("Template {} not found on CLVM_NG pool {}.", templateLvName, destPool.getUuid());
+ }
+
+ if (templateOnPrimary != null) {
+ String backingFile = templateOnPrimary.getPath();
+ logger.info("Using template on primary storage as backing file: {}", backingFile);
+ ensureTemplateLvInSharedMode(backingFile);
+ return backingFile;
+ }
+
+ logger.error("Template {} should be on primary storage before creating volumes from it", templateLvName);
+ throw new CloudRuntimeException(String.format("Template not found on CLVM_NG primary storage: %s. Template must be copied to primary storage first.", templateLvName));
+ }
+
+ /**
+ * Ensures a template LV is activated in shared mode so multiple VMs can use it as a backing file.
+ */
+ private void ensureTemplateLvInSharedMode(String templatePath, boolean throwOnFailure) {
+ try {
+ Script checkLvs = new Script("lvs", Duration.millis(10000), logger);
+ checkLvs.add("--noheadings");
+ checkLvs.add("-o", "lv_attr");
+ checkLvs.add(templatePath);
+
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ String result = checkLvs.execute(parser);
+
+ if (result == null && parser.getLines() != null && !parser.getLines().isEmpty()) {
+ String lvAttr = parser.getLines().trim();
+ if (lvAttr.length() >= 6) {
+ boolean isActive = (lvAttr.indexOf('a') >= 0);
+ boolean isShared = (lvAttr.indexOf('s') >= 0);
+
+ if (!isShared || !isActive) {
+ logger.info("Template LV {} is not in shared mode (attr: {}). Activating in shared mode.", templatePath, lvAttr);
+ LibvirtComputingResource.setClvmVolumeToSharedMode(templatePath);
+ } else {
+ logger.debug("Template LV {} is already in shared mode (attr: {})", templatePath, lvAttr);
+ }
+ }
+ }
+ } catch (CloudRuntimeException e) {
+ throw e;
+ } catch (Exception e) {
+ String errorMsg = "Failed to check/ensure template LV shared mode for " + templatePath + ": " + e.getMessage();
+ if (throwOnFailure) {
+ throw new CloudRuntimeException(errorMsg, e);
+ } else {
+ logger.warn(errorMsg, e);
+ }
+ }
+ }
+
+ private void ensureTemplateLvInSharedMode(String templatePath) {
+ ensureTemplateLvInSharedMode(templatePath, false);
+ }
+
+ private long getVgPhysicalExtentSize(String vgName) {
+ final long DEFAULT_PE_SIZE = 4 * 1024 * 1024L;
+ String warningMessage = String.format("Failed to get PE size for VG %s, defaulting to 4MiB", vgName);
+
+ try {
+ Script vgDisplay = new Script("vgdisplay", 300000, logger);
+ vgDisplay.add("--units", "b");
+ vgDisplay.add("-C");
+ vgDisplay.add("--noheadings");
+ vgDisplay.add("-o", "vg_extent_size");
+ vgDisplay.add(vgName);
+
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ String result = vgDisplay.execute(parser);
+
+ if (result != null) {
+ logger.warn("{}: {}", warningMessage, result);
+ return DEFAULT_PE_SIZE;
+ }
+
+ String output = parser.getLines();
+ if (output == null || output.trim().isEmpty()) {
+ logger.warn("{}: empty output", warningMessage);
+ return DEFAULT_PE_SIZE;
+ }
+
+ output = output.trim();
+ if (output.endsWith("B")) {
+ output = output.substring(0, output.length() - 1).trim();
+ }
+
+ long peSize = Long.parseLong(output);
+ logger.debug("Physical Extent size for VG {} is {} bytes", vgName, peSize);
+ return peSize;
+ } catch (NumberFormatException e) {
+ logger.warn("{}: failed to parse PE size", warningMessage, e);
+ } catch (Exception e) {
+ logger.warn("{}: {}", warningMessage, e.getMessage());
+ }
+
+ logger.info("Using default PE size for VG {}: {} bytes (4 MiB)", vgName, DEFAULT_PE_SIZE);
+ return DEFAULT_PE_SIZE;
+ }
+
+ /**
+ * Calculate LVM LV size for CLVM_NG volume allocation.
+ */
+ private long calculateClvmNgLvSize(long virtualSize, String vgName) {
+ long peSize = getVgPhysicalExtentSize(vgName);
+
+ long clusterSize = 64 * 1024L;
+ long l2Multiplier = 4096L;
+
+ long numDataClusters = (virtualSize + clusterSize - 1) / clusterSize;
+ long numL2Clusters = (numDataClusters + l2Multiplier - 1) / l2Multiplier;
+ long l2TableSize = numL2Clusters * clusterSize;
+ long refcountTableSize = l2TableSize;
+
+ long headerOverhead = 2 * 1024 * 1024L;
+ long metadataOverhead = l2TableSize + refcountTableSize + headerOverhead;
+ long targetSize = virtualSize + metadataOverhead;
+ long roundedSize = ((targetSize + peSize - 1) / peSize) * peSize;
+ long virtualSizeGiB = virtualSize / (1024 * 1024 * 1024L);
+ long overheadMiB = metadataOverhead / (1024 * 1024L);
+
+ logger.info("Calculated volume LV size: {} bytes (virtual: {} GiB, QCOW2 metadata overhead: {} MiB, rounded to {} PEs, PE size = {} bytes)",
+ roundedSize, virtualSizeGiB, overheadMiB, roundedSize / peSize, peSize);
+
+ return roundedSize;
+ }
+
+ private long getQcow2VirtualSize(String imagePath) {
+ Script qemuImg = new Script("qemu-img", 300000, logger);
+ qemuImg.add("info");
+ qemuImg.add("--output=json");
+ qemuImg.add(imagePath);
+
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ String result = qemuImg.execute(parser);
+
+ if (result != null) {
+ throw new CloudRuntimeException("Failed to get QCOW2 virtual size for " + imagePath + ": " + result);
+ }
+
+ String output = parser.getLines();
+ if (output == null || output.trim().isEmpty()) {
+ throw new CloudRuntimeException("qemu-img info returned empty output for " + imagePath);
+ }
+
+ JsonObject info = JsonParser.parseString(output).getAsJsonObject();
+ return info.get("virtual-size").getAsLong();
+ }
+
+ private long getQcow2PhysicalSize(String imagePath) {
+ Script qemuImg = new Script("qemu-img", Duration.millis(300000), logger);
+ qemuImg.add("info");
+ qemuImg.add("--output=json");
+ qemuImg.add(imagePath);
Review Comment:
QemuImg class
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/ClvmStorageAdaptor.java:
##########
@@ -0,0 +1,1072 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.hypervisor.kvm.storage;
+
+import static com.cloud.utils.NumbersUtil.toHumanReadableSize;
+
+import java.io.File;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.cloud.hypervisor.kvm.resource.LibvirtComputingResource;
+import com.cloud.hypervisor.kvm.resource.LibvirtConnection;
+import com.cloud.hypervisor.kvm.resource.LibvirtStoragePoolDef;
+import com.cloud.hypervisor.kvm.resource.LibvirtStorageVolumeDef;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Storage.StoragePoolType;
+import com.cloud.storage.StorageLayer;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.script.OutputInterpreter;
+import com.cloud.utils.script.Script;
+import com.google.gson.JsonObject;
+import com.google.gson.JsonParser;
+import org.apache.cloudstack.utils.qemu.QemuImg.PhysicalDiskFormat;
+import org.joda.time.Duration;
+import org.libvirt.Connect;
+import org.libvirt.LibvirtException;
+import org.libvirt.StoragePool;
+import org.libvirt.StorageVol;
+
+/**
+ * Storage adaptor for CLVM and CLVM_NG pool types.
+ * Extends LibvirtStorageAdaptor and overrides methods with CLVM-specific logic,
+ * using direct LVM commands instead of libvirt for volume operations.
+ */
+public class ClvmStorageAdaptor extends LibvirtStorageAdaptor {
+
+ public ClvmStorageAdaptor(StorageLayer storage) {
+ super(storage);
+ }
+
+ @Override
+ public StoragePoolType getStoragePoolType() {
+ // Registered manually for both CLVM and CLVM_NG in KVMStoragePoolManager
+ return null;
+ }
+
+ @Override
+ public KVMStoragePool createStoragePool(String name, String host, int port, String path,
+ String userInfo, StoragePoolType type, Map<String, String> details, boolean isPrimaryStorage) {
+ logger.info("Attempting to create CLVM/CLVM_NG storage pool {} in libvirt", name);
+
+ Connect conn;
+ try {
+ conn = LibvirtConnection.getConnection();
+ } catch (LibvirtException e) {
+ throw new CloudRuntimeException(e.toString());
+ }
+
+ StoragePool sp = createCLVMStoragePool(conn, name, host, path);
+ if (sp == null) {
+ logger.info("Falling back to virtual CLVM/CLVM_NG pool without libvirt for: {}", name);
+ return createVirtualClvmPool(name, host, path, type, details);
+ }
+
+ try {
+ if (!isPrimaryStorage) {
+ incStoragePoolRefCount(name);
+ }
+ // CLVM/CLVM_NG pools are kept inactive in libvirt; we use direct LVM commands
+ return getStoragePool(name);
+ } catch (Exception e) {
+ decStoragePoolRefCount(name);
+ throw new CloudRuntimeException("Failed to create CLVM storage pool: " + name, e);
+ }
+ }
+
+ @Override
+ public KVMStoragePool getStoragePool(String uuid, boolean refreshInfo) {
+ logger.info("Fetching CLVM/CLVM_NG storage pool {} ", uuid);
+ try {
+ Connect conn = LibvirtConnection.getConnection();
+ StoragePool storage = conn.storagePoolLookupByUUIDString(uuid);
+
+ LibvirtStoragePoolDef spd = getStoragePoolDef(conn, storage);
+ if (spd == null) {
+ throw new CloudRuntimeException("Unable to parse storage pool definition for pool " + uuid);
+ }
+
+ // CLVM pools in libvirt are always LOGICAL type
+ StoragePoolType type = StoragePoolType.CLVM;
+
+ // Do NOT activate the pool — CLVM/CLVM_NG pools stay inactive in libvirt
+ LibvirtStoragePool pool = new LibvirtStoragePool(uuid, storage.getName(), type, this, storage);
+ pool.setLocalPath(spd.getTargetPath());
+
+ // Always read capacity from LVM directly
+ String vgName = storage.getName();
+ try {
+ long[] vgStats = getVgStats(vgName);
+ setPoolCapacityFromVgStats(pool, vgStats, vgName);
+ } catch (CloudRuntimeException e) {
+ logger.warn("Failed to get VG stats for CLVM/CLVM_NG pool {}: {}. Using libvirt values (may be 0)", vgName, e.getMessage());
+ pool.setCapacity(storage.getInfo().capacity);
+ pool.setUsed(storage.getInfo().allocation);
+ pool.setAvailable(storage.getInfo().available);
+ }
+
+ return pool;
+ } catch (LibvirtException e) {
+ // Pool not in libvirt - return virtual pool backed only by LVM
+ logger.debug("CLVM/CLVM_NG pool {} not found in libvirt, creating virtual pool", uuid);
+ throw new CloudRuntimeException(e.toString(), e);
+ }
+ }
+
+ @Override
+ public KVMPhysicalDisk getPhysicalDisk(String volumeUuid, KVMStoragePool pool) {
+ LibvirtStoragePool libvirtPool = (LibvirtStoragePool) pool;
+
+ // Pool has no libvirt backing - go directly to block device
+ if (libvirtPool.getPool() == null) {
+ logger.debug("CLVM/CLVM_NG pool has no libvirt backing, using direct block device access for volume: {}", volumeUuid);
+ return getPhysicalDiskViaDirectBlockDevice(volumeUuid, pool);
+ }
+
+ try {
+ StorageVol vol = getVolume(libvirtPool.getPool(), volumeUuid);
+ if (vol == null) {
+ logger.debug("Volume {} not found in libvirt, falling back to CLVM direct access", volumeUuid);
+ return getPhysicalDiskWithClvmFallback(volumeUuid, pool, libvirtPool);
+ }
+
+ LibvirtStorageVolumeDef voldef = getStorageVolumeDef(libvirtPool.getPool().getConnect(), vol);
+ KVMPhysicalDisk disk = new KVMPhysicalDisk(vol.getPath(), vol.getName(), pool);
+ disk.setSize(vol.getInfo().allocation);
+ disk.setVirtualSize(vol.getInfo().capacity);
+ disk.setFormat(voldef.getFormat() == LibvirtStorageVolumeDef.VolumeFormat.QCOW2
+ ? PhysicalDiskFormat.QCOW2 : PhysicalDiskFormat.RAW);
+ return disk;
+ } catch (LibvirtException e) {
+ logger.warn("LibvirtException looking up volume {}: {}", volumeUuid, e.getMessage());
+ return getPhysicalDiskWithClvmFallback(volumeUuid, pool, libvirtPool);
+ }
+ }
+
+ @Override
+ public KVMPhysicalDisk createPhysicalDisk(String name, KVMStoragePool pool,
+ PhysicalDiskFormat format, Storage.ProvisioningType provisioningType, long size, byte[] passphrase) {
+ logger.info("Creating CLVM/CLVM_NG volume {} in pool {} with size {}", name, pool.getUuid(), toHumanReadableSize(size));
+
+ if (StoragePoolType.CLVM_NG.equals(pool.getType())) {
+ return createClvmNgDiskWithBacking(name, 0, size, null, pool, provisioningType);
+ } else {
+ return createClvmVolume(name, size, pool);
+ }
+ }
+
+ @Override
+ public boolean connectPhysicalDisk(String name, KVMStoragePool pool, Map<String, String> details, boolean isVMMigrate) {
+ if (isVMMigrate) {
+ logger.info("Activating CLVM/CLVM_NG volume {} in shared mode for VM migration", name);
+ Script activateVol = new Script("lvchange", 10000, logger);
+ activateVol.add("-asy");
+ activateVol.add(pool.getLocalPath() + File.separator + name);
+ String result = activateVol.execute();
+ if (result != null) {
+ logger.error("Failed to activate CLVM/CLVM_NG volume {} in shared mode. Output: {}", name, result);
+ return false;
+ }
+ }
+
+ if (StoragePoolType.CLVM_NG.equals(pool.getType())) {
+ ensureClvmNgBackingFileAccessible(name, pool);
+ }
+
+ return true;
+ }
+
+ @Override
+ public KVMPhysicalDisk createDiskFromTemplate(KVMPhysicalDisk template,
+ String name, PhysicalDiskFormat format, Storage.ProvisioningType provisioningType,
+ long size, KVMStoragePool destPool, int timeout, byte[] passphrase) {
+
+ if (StoragePoolType.CLVM_NG.equals(destPool.getType()) && format == PhysicalDiskFormat.QCOW2) {
+ logger.info("Creating CLVM_NG volume {} with backing file from template {}", name, template.getName());
+ String backingFile = getClvmBackingFile(template, destPool);
+ return createClvmNgDiskWithBacking(name, timeout, size, backingFile, destPool, provisioningType);
+ }
+
+ return super.createDiskFromTemplate(template, name, format, provisioningType, size, destPool, timeout, passphrase);
+ }
+
+ @Override
+ public void createTemplate(String templatePath, String templateUuid, int timeout, KVMStoragePool pool) {
+ String vgName = getVgName(pool.getLocalPath());
+ String lvName = "template-" + templateUuid;
+ String lvPath = "/dev/" + vgName + "/" + lvName;
+
+ if (lvExists(lvPath)) {
+ logger.info("Template LV {} already exists in VG {}. Skipping creation.", lvName, vgName);
+ return;
+ }
+
+ logger.info("Creating new template LV {} in VG {} for template {}", lvName, vgName, templateUuid);
+
+ long virtualSize = getQcow2VirtualSize(templatePath);
+ long physicalSize = getQcow2PhysicalSize(templatePath);
+ long lvSize = virtualSize;
+
+ logger.info("Template source - Physical: {} bytes, Virtual: {} bytes, LV will be: {} bytes", physicalSize, virtualSize, lvSize);
+
+ Script lvcreate = new Script("lvcreate", Duration.millis(timeout), logger);
+ lvcreate.add("-n", lvName);
+ lvcreate.add("-L", lvSize + "B");
+ lvcreate.add("--yes");
+ lvcreate.add(vgName);
+ String result = lvcreate.execute();
+ if (result != null) {
+ throw new CloudRuntimeException("Failed to create LV for CLVM_NG template: " + result);
+ }
+
+ Script qemuImgConvert = new Script("qemu-img", Duration.millis(timeout), logger);
+ qemuImgConvert.add("convert");
+ qemuImgConvert.add(templatePath);
+ qemuImgConvert.add("-O", "qcow2");
+ qemuImgConvert.add("-o", "cluster_size=64k,extended_l2=off,preallocation=off");
+ qemuImgConvert.add(lvPath);
+ result = qemuImgConvert.execute();
+
+ if (result != null) {
+ removeLvOnFailure(lvPath, timeout);
+ throw new CloudRuntimeException("Failed to convert template to CLVM_NG volume: " + result);
+ }
+
+ long actualVirtualSize = getQcow2VirtualSize(lvPath);
+
+ try {
+ ensureTemplateLvInSharedMode(lvPath, true);
+ } catch (CloudRuntimeException e) {
+ logger.error("Failed to activate template LV {} in shared mode. Cleaning up.", lvPath);
+ removeLvOnFailure(lvPath, timeout);
+ throw e;
+ }
+
+ KVMPhysicalDisk templateDisk = new KVMPhysicalDisk(lvPath, lvName, pool);
+ templateDisk.setFormat(PhysicalDiskFormat.QCOW2);
+ templateDisk.setVirtualSize(actualVirtualSize);
+ templateDisk.setSize(lvSize);
+ }
+
+ private StoragePool createCLVMStoragePool(Connect conn, String uuid, String host, String path) {
+ String volgroupPath = "/dev/" + path;
+ String volgroupName = path;
+ volgroupName = volgroupName.replaceFirst("^/", "");
+
+ Script checkVgExists = new Script("vgs", 10000, logger);
+ checkVgExists.add("--noheadings");
+ checkVgExists.add("-o", "vg_name");
+ checkVgExists.add(volgroupName);
+ String vgCheckResult = checkVgExists.execute();
+
+ if (vgCheckResult != null) {
+ logger.error("Volume group {} does not exist or is not accessible", volgroupName);
+ return null;
+ }
+
+ logger.info("Volume group {} verified, creating libvirt pool definition for CLVM/CLVM_NG", volgroupName);
+ LibvirtStoragePoolDef poolDef = new LibvirtStoragePoolDef(
+ LibvirtStoragePoolDef.PoolType.LOGICAL,
+ volgroupName,
+ uuid,
+ null,
+ volgroupName,
+ volgroupPath
+ );
+
+ try {
+ StoragePool pool = conn.storagePoolDefineXML(poolDef.toString(), 0);
+ logger.info("Created libvirt pool definition for CLVM/CLVM_NG VG: {} (pool will remain inactive)", volgroupName);
+ pool.setAutostart(1);
+ return pool;
+ } catch (LibvirtException e) {
+ logger.warn("Failed to define CLVM/CLVM_NG pool in libvirt: {}", e.getMessage());
+ return null;
+ }
+ }
+
+ private void setPoolCapacityFromVgStats(LibvirtStoragePool pool, long[] vgStats, String vgName) {
+ long capacity = vgStats[0];
+ long available = vgStats[1];
+ long used = capacity - available;
+
+ pool.setCapacity(capacity);
+ pool.setAvailable(available);
+ pool.setUsed(used);
+
+ logger.debug("CLVM/CLVM_NG pool {} - Capacity: {}, Used: {}, Available: {}",
+ vgName, toHumanReadableSize(capacity), toHumanReadableSize(used), toHumanReadableSize(available));
+ }
+
+ private long[] getVgStats(String vgName) {
+ Script getVgStats = new Script("vgs", 10000, logger);
+ getVgStats.add("--noheadings");
+ getVgStats.add("--units", "b");
+ getVgStats.add("--nosuffix");
+ getVgStats.add("-o", "vg_size,vg_free");
+ getVgStats.add(vgName);
+
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ String result = getVgStats.execute(parser);
+
+ if (result != null) {
+ String errorMsg = "Failed to get statistics for volume group " + vgName + ": " + result;
+ logger.error(errorMsg);
+ throw new CloudRuntimeException(errorMsg);
+ }
+
+ String output = parser.getLines().trim();
+ String[] lines = output.split("\\n");
+ String dataLine = null;
+
+ for (String line : lines) {
+ line = line.trim();
+ if (!line.isEmpty() && Character.isDigit(line.charAt(0))) {
+ dataLine = line;
+ break;
+ }
+ }
+
+ if (dataLine == null) {
+ String errorMsg = "No valid data line found in vgs output for " + vgName + ": " + output;
+ logger.error(errorMsg);
+ throw new CloudRuntimeException(errorMsg);
+ }
+
+ String[] stats = dataLine.split("\\s+");
+
+ if (stats.length < 2) {
+ String errorMsg = "Unexpected output from vgs command for " + vgName + ": " + dataLine;
+ logger.error(errorMsg);
+ throw new CloudRuntimeException(errorMsg);
+ }
+
+ try {
+ long capacity = Long.parseLong(stats[0].trim());
+ long available = Long.parseLong(stats[1].trim());
+ return new long[]{capacity, available};
+ } catch (NumberFormatException e) {
+ String errorMsg = "Failed to parse VG statistics for " + vgName + ": " + e.getMessage();
+ logger.error(errorMsg);
+ throw new CloudRuntimeException(errorMsg, e);
+ }
+ }
+
+ private KVMStoragePool createVirtualClvmPool(String uuid, String host, String path, StoragePoolType type, Map<String, String> details) {
+ String volgroupName = path.replaceFirst("^/", "");
+ String volgroupPath = "/dev/" + volgroupName;
+
+ logger.info("Creating virtual CLVM/CLVM_NG pool {} without libvirt using direct LVM access", volgroupName);
+
+ long[] vgStats = getVgStats(volgroupName);
+
+ LibvirtStoragePool pool = new LibvirtStoragePool(uuid, volgroupName, type, this, null);
+ pool.setLocalPath(volgroupPath);
+ setPoolCapacityFromVgStats(pool, vgStats, volgroupName);
+
+ if (details != null) {
+ pool.setDetails(details);
+ }
+
+ return pool;
+ }
+
+ /**
+ * CLVM fallback: First tries to refresh libvirt pool to make volume visible,
+ * if that fails, accesses volume directly via block device path.
+ */
+ private KVMPhysicalDisk getPhysicalDiskWithClvmFallback(String volumeUuid, KVMStoragePool pool, LibvirtStoragePool libvirtPool) {
+ logger.info("CLVM volume not visible to libvirt, attempting pool refresh for volume: {}", volumeUuid);
+
+ try {
+ logger.debug("Refreshing libvirt storage pool: {}", pool.getUuid());
+ libvirtPool.getPool().refresh(0);
+
+ StorageVol vol = getVolume(libvirtPool.getPool(), volumeUuid);
+ if (vol != null) {
+ logger.info("Volume found after pool refresh: {}", volumeUuid);
+ LibvirtStorageVolumeDef voldef = getStorageVolumeDef(libvirtPool.getPool().getConnect(), vol);
+ KVMPhysicalDisk disk = new KVMPhysicalDisk(vol.getPath(), vol.getName(), pool);
+ disk.setSize(vol.getInfo().allocation);
+ disk.setVirtualSize(vol.getInfo().capacity);
+ disk.setFormat(voldef.getFormat() == LibvirtStorageVolumeDef.VolumeFormat.QCOW2
+ ? PhysicalDiskFormat.QCOW2 : PhysicalDiskFormat.RAW);
+ return disk;
+ }
+ } catch (LibvirtException refreshEx) {
+ logger.debug("Pool refresh failed or volume still not found: {}", refreshEx.getMessage());
+ }
+
+ logger.info("Falling back to direct block device access for volume: {}", volumeUuid);
+ return getPhysicalDiskViaDirectBlockDevice(volumeUuid, pool);
+ }
+
+ private String getVgName(String sourceDir) {
+ String vgName = sourceDir;
+ if (vgName.startsWith("/")) {
+ String[] parts = vgName.split("/");
+ List<String> tokens = Arrays.stream(parts)
+ .filter(s -> !s.isEmpty()).collect(Collectors.toList());
+
+ vgName = tokens.size() > 1 ? tokens.get(1)
+ : tokens.size() == 1 ? tokens.get(0)
+ : "";
+ }
+ return vgName;
+ }
+
+ private String extractVgNameFromPool(KVMStoragePool pool) {
+ String sourceDir = pool.getLocalPath();
+ if (sourceDir == null || sourceDir.isEmpty()) {
+ throw new CloudRuntimeException("CLVM pool sourceDir is not set, cannot determine VG name");
+ }
+ String vgName = getVgName(sourceDir);
+ logger.debug("Using VG name: {} (from sourceDir: {})", vgName, sourceDir);
+ return vgName;
+ }
+
+ /**
+ * For CLVM volumes that exist in LVM but are not visible to libvirt,
+ * access them directly via block device path.
+ */
+ private KVMPhysicalDisk getPhysicalDiskViaDirectBlockDevice(String volumeUuid, KVMStoragePool pool) {
+ try {
+ String vgName = extractVgNameFromPool(pool);
+
+ verifyLvExistsInVg(volumeUuid, vgName);
+
+ logger.info("Volume {} exists in LVM but not visible to libvirt, accessing directly", volumeUuid);
+
+ String lvPath = findAccessibleDeviceNode(volumeUuid, vgName, pool);
+ long size = getClvmVolumeSize(lvPath);
+
+ KVMPhysicalDisk disk = createPhysicalDiskFromClvmLv(lvPath, volumeUuid, pool, size);
+ ensureTemplateAccessibility(volumeUuid, lvPath, pool);
+
+ return disk;
+ } catch (CloudRuntimeException ex) {
+ throw ex;
+ } catch (Exception ex) {
+ logger.error("Failed to access CLVM volume via direct block device: {}", volumeUuid, ex);
+ throw new CloudRuntimeException(String.format("Could not find volume %s: %s", volumeUuid, ex.getMessage()));
+ }
+ }
+
+ private void verifyLvExistsInVg(String volumeUuid, String vgName) {
+ logger.debug("Checking if volume {} exists in VG {}", volumeUuid, vgName);
+ Script checkLvCmd = new Script("/usr/sbin/lvs", 10000, logger);
+ checkLvCmd.add("--noheadings");
+ checkLvCmd.add("--unbuffered");
+ checkLvCmd.add(vgName + "/" + volumeUuid);
+ String checkResult = checkLvCmd.execute();
+ if (checkResult != null) {
+ throw new CloudRuntimeException(String.format("Storage volume not found: no storage vol with matching name '%s'", volumeUuid));
+ }
+ }
+
+ private String findAccessibleDeviceNode(String volumeUuid, String vgName, KVMStoragePool pool) {
+ String lvPath = "/dev/" + vgName + "/" + volumeUuid;
+ File lvDevice = new File(lvPath);
+
+ if (!lvDevice.exists()) {
+ lvPath = tryDeviceMapperPath(volumeUuid, vgName);
+ if (!new File(lvPath).exists()) {
+ lvPath = handleMissingDeviceNode(volumeUuid, vgName, pool);
+ }
+ }
+
+ return lvPath;
+ }
+
+ private String tryDeviceMapperPath(String volumeUuid, String vgName) {
+ String vgNameEscaped = vgName.replace("-", "--");
+ String volumeUuidEscaped = volumeUuid.replace("-", "--");
+ return "/dev/mapper/" + vgNameEscaped + "-" + volumeUuidEscaped;
+ }
+
+ private String handleMissingDeviceNode(String volumeUuid, String vgName, KVMStoragePool pool) {
+ if (StoragePoolType.CLVM_NG.equals(pool.getType()) && volumeUuid.startsWith("template-")) {
+ return activateTemplateAndGetPath(volumeUuid, vgName);
+ }
+ throw new CloudRuntimeException(String.format("Could not find volume %s in VG %s - volume exists in LVM but device node not accessible", volumeUuid, vgName));
+ }
+
+ private String activateTemplateAndGetPath(String volumeUuid, String vgName) {
+ logger.info("Template volume {} device node not found. Attempting to activate in shared mode.", volumeUuid);
+ String templateLvPath = "/dev/" + vgName + "/" + volumeUuid;
+
+ try {
+ ensureTemplateLvInSharedMode(templateLvPath, false);
+
+ String lvPath = findDeviceNodeAfterActivation(templateLvPath, volumeUuid, vgName);
+
+ logger.info("Successfully activated template volume {} at {}", volumeUuid, lvPath);
+ return lvPath;
+ } catch (CloudRuntimeException e) {
+ throw new CloudRuntimeException(String.format("Failed to activate template volume %s in VG %s: %s", volumeUuid, vgName, e.getMessage()), e);
+ }
+ }
+
+ private String findDeviceNodeAfterActivation(String templateLvPath, String volumeUuid, String vgName) {
+ File lvDevice = new File(templateLvPath);
+ String lvPath = templateLvPath;
+
+ if (!lvDevice.exists()) {
+ String vgNameEscaped = vgName.replace("-", "--");
+ String volumeUuidEscaped = volumeUuid.replace("-", "--");
+ lvPath = "/dev/mapper/" + vgNameEscaped + "-" + volumeUuidEscaped;
+ lvDevice = new File(lvPath);
+ }
+
+ if (!lvDevice.exists()) {
+ logger.error("Template volume {} still not accessible after activation attempt", volumeUuid);
+ throw new CloudRuntimeException(String.format("Could not activate template volume %s in VG %s - device node not accessible after activation", volumeUuid, vgName));
+ }
+
+ return lvPath;
+ }
+
+ private void ensureTemplateAccessibility(String volumeUuid, String lvPath, KVMStoragePool pool) {
+ if (StoragePoolType.CLVM_NG.equals(pool.getType()) && volumeUuid.startsWith("template-")) {
+ logger.info("Detected template volume {}. Ensuring it's activated in shared mode.", volumeUuid);
+ ensureTemplateLvInSharedMode(lvPath, false);
+ }
+ }
+
+ private long getClvmVolumeSize(String lvPath) {
+ try {
+ Script lvsCmd = new Script("/usr/sbin/lvs", 10000, logger);
+ lvsCmd.add("--noheadings");
+ lvsCmd.add("--units");
+ lvsCmd.add("b");
+ lvsCmd.add("-o");
+ lvsCmd.add("lv_size");
+ lvsCmd.add(lvPath);
+
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ String result = lvsCmd.execute(parser);
+
+ String output = (result == null) ? parser.getLines() : result;
+
+ if (output != null && !output.isEmpty()) {
+ String sizeStr = output.trim().replaceAll("[^0-9]", "");
+ if (!sizeStr.isEmpty()) {
+ return Long.parseLong(sizeStr);
+ }
+ }
+ } catch (Exception sizeEx) {
+ logger.warn("Failed to get size for CLVM volume via lvs: {}", sizeEx.getMessage());
+ File lvDevice = new File(lvPath);
+ if (lvDevice.isFile()) {
+ return lvDevice.length();
+ }
+ }
+ return 0;
+ }
+
+ private KVMPhysicalDisk createPhysicalDiskFromClvmLv(String lvPath, String volumeUuid, KVMStoragePool pool, long size) {
+ PhysicalDiskFormat diskFormat = StoragePoolType.CLVM_NG.equals(pool.getType())
+ ? PhysicalDiskFormat.QCOW2 : PhysicalDiskFormat.RAW;
+
+ logger.debug("{} pool detected, setting disk format to {} for volume {}", pool.getType(), diskFormat, volumeUuid);
+
+ KVMPhysicalDisk disk = new KVMPhysicalDisk(lvPath, volumeUuid, pool);
+ disk.setFormat(diskFormat);
+ disk.setSize(size);
+ disk.setVirtualSize(size);
+
+ logger.info("Successfully accessed CLVM/CLVM_NG volume via direct block device: {} with format: {} and size: {} bytes",
+ lvPath, diskFormat, size);
+ return disk;
+ }
+
+ /**
+ * Checks if a CLVM_NG QCOW2 volume has a backing file (template) and ensures it's activated in shared mode.
+ */
+ private void ensureClvmNgBackingFileAccessible(String volumeName, KVMStoragePool pool) {
+ try {
+ String vgName = getVgName(pool.getLocalPath());
+ String volumePath = "/dev/" + vgName + "/" + volumeName;
+
+ logger.debug("Checking if CLVM_NG volume {} has a backing file that needs activation", volumePath);
+
+ Script qemuImgInfo = new Script("qemu-img", Duration.millis(10000), logger);
+ qemuImgInfo.add("info");
+ qemuImgInfo.add("--output=json");
+ qemuImgInfo.add(volumePath);
Review Comment:
Please use the QemuImg class
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java:
##########
@@ -2123,6 +2347,209 @@ protected String getParentCheckpointName(String[] parents) {
return immediateParentPath.substring(immediateParentPath.lastIndexOf(File.separator) + 1);
}
+ /**
+ * Checks if a QEMU bitmap exists in the volume and is usable (not in-use or corrupted).
+ * This is important after lock migration where bitmaps may be left in "in-use" state.
+ *
+ * @param pool The storage pool containing the volume
+ * @param volume The volume to check
+ * @param bitmapName The name of the bitmap to check
+ * @return true if bitmap exists and is usable, false if missing, in-use, or corrupted
+ */
+ protected boolean isBitmapUsable(KVMStoragePool pool, VolumeObjectTO volume, String bitmapName) {
+ try {
+ String volumePath = pool.getLocalPathFor(volume.getPath());
+
+ String command = String.format("qemu-img info --output=json -U %s", volumePath);
+ String jsonOutput = Script.runSimpleBashScriptWithFullResult(command, 30);
+
+ if (jsonOutput == null || jsonOutput.trim().isEmpty()) {
+ logger.warn("Failed to get qemu-img info for volume [{}]", volumePath);
+ return false;
+ }
+
+ logger.debug("qemu-img info output for volume [{}]: {}", volumePath, jsonOutput);
+
+ try {
+ ObjectMapper mapper = new ObjectMapper();
+ JsonNode root = mapper.readTree(jsonOutput);
+
+ JsonNode formatSpecific = root.path("format-specific");
+ if (formatSpecific.isMissingNode()) {
+ logger.debug("No format-specific data found for volume [{}], bitmap check skipped", volumePath);
+ return false;
+ }
+
+ JsonNode data = formatSpecific.path("data");
+ JsonNode bitmaps = data.path("bitmaps");
+
+ if (bitmaps.isMissingNode() || !bitmaps.isArray()) {
+ logger.debug("No bitmaps found in volume [{}]", volumePath);
+ return false;
+ }
+
+ for (JsonNode bitmap : bitmaps) {
+ String name = bitmap.path("name").asText();
+ if (bitmapName.equals(name)) {
+ JsonNode flags = bitmap.path("flags");
+ if (flags.isArray()) {
+ for (JsonNode flag : flags) {
+ String flagValue = flag.asText();
+ if ("in-use".equals(flagValue)) {
+ logger.warn("Bitmap [{}] in volume [{}] is marked as 'in-use' and cannot be used for incremental snapshot",
+ bitmapName, volumePath);
+ return false;
+ }
+ }
+ }
+ logger.debug("Bitmap [{}] found in volume [{}] and is usable", bitmapName, volumePath);
+ return true;
+ }
+ }
+
+ logger.warn("Bitmap [{}] not found in volume [{}]", bitmapName, volumePath);
+ return false;
+
+ } catch (JsonProcessingException e) {
+ logger.error("Failed to parse qemu-img JSON output for volume [{}]: {}", volumePath, e.getMessage(), e);
+ return false;
+ }
+
+ } catch (Exception e) {
+ logger.error("Error checking bitmap [{}] for volume [{}]: {}", bitmapName, volume, e.getMessage(), e);
+ return false;
+ }
+ }
+
+ /**
+ * Removes a broken or unusable bitmap from a volume.
+ * Called before falling back to full snapshot to keep volume metadata clean.
+ *
+ * @param pool Storage pool containing the volume
+ * @param volume Volume with broken bitmap
+ * @param bitmapName Name of bitmap to remove
+ */
+ private void cleanupBrokenBitmap(KVMStoragePool pool, VolumeObjectTO volume, String bitmapName) {
+ try {
+ String volumePath = pool.getLocalPathFor(volume.getPath());
+
+ logger.info("Removing broken bitmap [{}] from volume [{}] before taking full snapshot",
+ bitmapName, volumePath);
+
+ QemuImgFile volumeFile = new QemuImgFile(volumePath, PhysicalDiskFormat.QCOW2);
+ QemuImg qemuImg = new QemuImg(0);
+
+ try {
+ qemuImg.bitmap(QemuImg.BitmapOperation.Remove, volumeFile, bitmapName);
+ logger.info("Successfully removed broken bitmap [{}] from volume [{}]",
+ bitmapName, volume.getPath());
+ } catch (QemuImgException e) {
+ logger.warn("Failed to remove broken bitmap [{}] from volume [{}]: {}. " +
+ "Proceeding with fallback anyway.",
+ bitmapName, volume.getPath(), e.getMessage());
+ }
+
+ } catch (Exception e) {
+ logger.warn("Exception while cleaning up broken bitmap [{}] for volume [{}]: {}. " +
+ "Proceeding with fallback anyway.",
+ bitmapName, volume.getPath(), e.getMessage());
+ }
+ }
+
+ /**
+ * Validates QEMU bitmap for CLVM_NG incremental snapshots and falls back to full snapshot if needed.
+ * This method checks if the bitmap from the parent checkpoint is usable. If the bitmap is corrupted,
+ * in-use, or missing, it cleans up the broken bitmap and falls back to taking a full snapshot with
+ * a new checkpoint to restart the incremental chain.
+ *
+ * @param snapshotObjectTO Snapshot being created
+ * @param primaryPool Primary storage pool
+ * @param secondaryPool Secondary storage pool for backup
+ * @param secondaryPoolUrl Secondary pool URL
+ * @param snapshotName Name of the snapshot
+ * @param volumeObjectTo Volume being snapshotted
+ * @param conn Libvirt connection
+ * @param wait Timeout for operations
+ * @return SnapshotObjectTO if fallback to full snapshot occurred, null if validation passed
+ * @throws LibvirtException if libvirt operations fail
+ */
+ protected SnapshotObjectTO validateClvmNgBitmapAndFallbackIfNeeded(SnapshotObjectTO snapshotObjectTO,
+ KVMStoragePool primaryPool,
+ KVMStoragePool secondaryPool,
+ String secondaryPoolUrl,
+ String snapshotName,
+ VolumeObjectTO volumeObjectTo,
+ Connect conn,
+ int wait) throws LibvirtException {
+ if (primaryPool.getType() != StoragePoolType.CLVM_NG || snapshotObjectTO.getParentSnapshotPath() == null) {
+ return null;
+ }
+
+ String[] parents = snapshotObjectTO.getParents();
+ if (parents == null || parents.length == 0) {
+ return null;
+ }
+
+ String parentCheckpointName = getParentCheckpointName(parents);
+ logger.debug("Validating bitmap [{}] for CLVM_NG volume [{}] before taking incremental snapshot",
+ parentCheckpointName, volumeObjectTo.getPath());
+
+ if (!isBitmapUsable(primaryPool, volumeObjectTo, parentCheckpointName)) {
+ logger.warn("Bitmap [{}] is not usable for volume [{}]. Falling back to full snapshot with new checkpoint.",
+ parentCheckpointName, volumeObjectTo.getPath());
+ cleanupBrokenBitmap(primaryPool, volumeObjectTo, parentCheckpointName);
+ return takeFullVolumeSnapshotOfStoppedVmForIncremental(snapshotObjectTO, primaryPool, secondaryPool,
+ secondaryPoolUrl, snapshotName, volumeObjectTo, conn, wait);
+ }
+
+ logger.debug("Bitmap [{}] is valid and usable for incremental snapshot", parentCheckpointName);
+ return null;
+ }
+
+ /**
+ * Takes a full snapshot of a stopped VM and creates a new checkpoint to restart the incremental chain.
+ * This is used as a fallback when incremental snapshot fails due to bitmap issues.
+ */
+ private SnapshotObjectTO takeFullVolumeSnapshotOfStoppedVmForIncremental(SnapshotObjectTO snapshotObjectTO,
Review Comment:
same
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java:
##########
@@ -6820,4 +6827,242 @@ public String getHypervisorPath() {
public String getGuestCpuArch() {
return guestCpuArch;
}
+
+ /**
+ * CLVM volume state for migration operations on source host
+ */
+ public enum ClvmVolumeState {
+ /** Shared mode (-asy) - used before migration to allow both hosts to access volume */
+ SHARED("-asy", "shared", "Before migration: activating in shared mode"),
+
+ /** Deactivate (-an) - used after successful migration to release volume on source */
+ DEACTIVATE("-an", "deactivated", "After successful migration: deactivating volume"),
+
+ /** Exclusive mode (-aey) - used after failed migration to revert to original exclusive state */
+ EXCLUSIVE("-aey", "exclusive", "After failed migration: reverting to exclusive mode");
+
+ private final String lvchangeFlag;
+ private final String description;
+ private final String logMessage;
+
+ ClvmVolumeState(String lvchangeFlag, String description, String logMessage) {
+ this.lvchangeFlag = lvchangeFlag;
+ this.description = description;
+ this.logMessage = logMessage;
+ }
+
+ public String getLvchangeFlag() {
+ return lvchangeFlag;
+ }
+
+ public String getDescription() {
+ return description;
+ }
+
+ public String getLogMessage() {
+ return logMessage;
+ }
+ }
+
+ public static void modifyClvmVolumesStateForMigration(List<DiskDef> disks, LibvirtComputingResource resource,
Review Comment:
Why is this method static?
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java:
##########
@@ -6820,4 +6827,242 @@ public String getHypervisorPath() {
public String getGuestCpuArch() {
return guestCpuArch;
}
+
+ /**
+ * CLVM volume state for migration operations on source host
+ */
+ public enum ClvmVolumeState {
+ /** Shared mode (-asy) - used before migration to allow both hosts to access volume */
+ SHARED("-asy", "shared", "Before migration: activating in shared mode"),
+
+ /** Deactivate (-an) - used after successful migration to release volume on source */
+ DEACTIVATE("-an", "deactivated", "After successful migration: deactivating volume"),
+
+ /** Exclusive mode (-aey) - used after failed migration to revert to original exclusive state */
+ EXCLUSIVE("-aey", "exclusive", "After failed migration: reverting to exclusive mode");
+
+ private final String lvchangeFlag;
+ private final String description;
+ private final String logMessage;
+
+ ClvmVolumeState(String lvchangeFlag, String description, String logMessage) {
+ this.lvchangeFlag = lvchangeFlag;
+ this.description = description;
+ this.logMessage = logMessage;
+ }
+
+ public String getLvchangeFlag() {
+ return lvchangeFlag;
+ }
+
+ public String getDescription() {
+ return description;
+ }
+
+ public String getLogMessage() {
+ return logMessage;
+ }
+ }
+
+ public static void modifyClvmVolumesStateForMigration(List<DiskDef> disks, LibvirtComputingResource resource,
+ VirtualMachineTO vmSpec, ClvmVolumeState state) {
+ for (DiskDef disk : disks) {
+ if (isClvmVolume(disk, resource, vmSpec)) {
+ String volumePath = disk.getDiskPath();
+ try {
+ modifyClvmVolumeState(volumePath, state.getLvchangeFlag(), state.getDescription(), state.getLogMessage());
+ } catch (Exception e) {
+ LOGGER.error("[CLVM Migration] Exception while setting volume [{}] to {} state: {}",
+ volumePath, state.getDescription(), e.getMessage(), e);
+ }
+ }
+ }
+ }
+
+ private static void modifyClvmVolumeState(String volumePath, String lvchangeFlag,
+ String stateDescription, String logMessage) {
+ try {
+ LOGGER.info("{} for volume [{}]", logMessage, volumePath);
+
+ Script cmd = new Script("lvchange", Duration.standardSeconds(300), LOGGER);
+ cmd.add(lvchangeFlag);
+ cmd.add(volumePath);
+
+ String result = cmd.execute();
+ if (result != null) {
+ String errorMsg = String.format(
+ "Failed to set volume [%s] to %s state. Command result: %s",
+ volumePath, stateDescription, result);
+ LOGGER.error(errorMsg);
+ throw new CloudRuntimeException(errorMsg);
+ } else {
+ LOGGER.info("Successfully set volume [{}] to {} state.",
+ volumePath, stateDescription);
+ }
+ } catch (CloudRuntimeException e) {
+ throw e;
+ } catch (Exception e) {
+ String errorMsg = String.format(
+ "Exception while setting volume [%s] to %s state: %s",
+ volumePath, stateDescription, e.getMessage());
+ LOGGER.error(errorMsg, e);
+ throw new CloudRuntimeException(errorMsg, e);
+ }
+ }
+
+ public static void activateClvmVolumeExclusive(String volumePath) {
+ modifyClvmVolumeState(volumePath, ClvmVolumeState.EXCLUSIVE.getLvchangeFlag(),
+ ClvmVolumeState.EXCLUSIVE.getDescription(),
+ "Activating CLVM volume in exclusive mode");
+ }
+
+ public static void deactivateClvmVolume(String volumePath) {
+ try {
+ modifyClvmVolumeState(volumePath, ClvmVolumeState.DEACTIVATE.getLvchangeFlag(),
+ ClvmVolumeState.DEACTIVATE.getDescription(),
+ "Deactivating CLVM volume");
+ } catch (Exception e) {
+ LOGGER.warn("Failed to deactivate CLVM volume {}: {}", volumePath, e.getMessage());
+ }
+ }
+
+ public static void setClvmVolumeToSharedMode(String volumePath) {
+ try {
+ modifyClvmVolumeState(volumePath, ClvmVolumeState.SHARED.getLvchangeFlag(),
+ ClvmVolumeState.SHARED.getDescription(),
+ "Setting CLVM volume to shared mode");
+ } catch (Exception e) {
+ LOGGER.warn("Failed to set CLVM volume {} to shared mode: {}", volumePath, e.getMessage());
+ }
+ }
+
+ /**
+ * Determines if a disk is on a CLVM storage pool by checking the actual pool type from VirtualMachineTO.
+ * This is the most reliable method as it uses CloudStack's own storage pool information.
+ *
+ * @param disk The disk definition to check
+ * @param resource The LibvirtComputingResource instance (unused but kept for compatibility)
+ * @param vmSpec The VirtualMachineTO specification containing disk and pool information
+ * @return true if the disk is on a CLVM storage pool, false otherwise
+ */
+ private static boolean isClvmVolume(DiskDef disk, LibvirtComputingResource resource, VirtualMachineTO vmSpec) {
Review Comment:
same
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java:
##########
@@ -6820,4 +6827,242 @@ public String getHypervisorPath() {
public String getGuestCpuArch() {
return guestCpuArch;
}
+
+ /**
+ * CLVM volume state for migration operations on source host
+ */
+ public enum ClvmVolumeState {
+ /** Shared mode (-asy) - used before migration to allow both hosts to access volume */
+ SHARED("-asy", "shared", "Before migration: activating in shared mode"),
+
+ /** Deactivate (-an) - used after successful migration to release volume on source */
+ DEACTIVATE("-an", "deactivated", "After successful migration: deactivating volume"),
+
+ /** Exclusive mode (-aey) - used after failed migration to revert to original exclusive state */
+ EXCLUSIVE("-aey", "exclusive", "After failed migration: reverting to exclusive mode");
+
+ private final String lvchangeFlag;
+ private final String description;
+ private final String logMessage;
+
+ ClvmVolumeState(String lvchangeFlag, String description, String logMessage) {
+ this.lvchangeFlag = lvchangeFlag;
+ this.description = description;
+ this.logMessage = logMessage;
+ }
+
+ public String getLvchangeFlag() {
+ return lvchangeFlag;
+ }
+
+ public String getDescription() {
+ return description;
+ }
+
+ public String getLogMessage() {
+ return logMessage;
+ }
+ }
+
+ public static void modifyClvmVolumesStateForMigration(List<DiskDef> disks, LibvirtComputingResource resource,
+ VirtualMachineTO vmSpec, ClvmVolumeState state) {
+ for (DiskDef disk : disks) {
+ if (isClvmVolume(disk, resource, vmSpec)) {
+ String volumePath = disk.getDiskPath();
+ try {
+ modifyClvmVolumeState(volumePath, state.getLvchangeFlag(), state.getDescription(), state.getLogMessage());
+ } catch (Exception e) {
+ LOGGER.error("[CLVM Migration] Exception while setting volume [{}] to {} state: {}",
+ volumePath, state.getDescription(), e.getMessage(), e);
+ }
+ }
+ }
+ }
+
+ private static void modifyClvmVolumeState(String volumePath, String lvchangeFlag,
+ String stateDescription, String logMessage) {
+ try {
+ LOGGER.info("{} for volume [{}]", logMessage, volumePath);
+
+ Script cmd = new Script("lvchange", Duration.standardSeconds(300), LOGGER);
+ cmd.add(lvchangeFlag);
+ cmd.add(volumePath);
+
+ String result = cmd.execute();
+ if (result != null) {
+ String errorMsg = String.format(
+ "Failed to set volume [%s] to %s state. Command result: %s",
+ volumePath, stateDescription, result);
+ LOGGER.error(errorMsg);
+ throw new CloudRuntimeException(errorMsg);
+ } else {
+ LOGGER.info("Successfully set volume [{}] to {} state.",
+ volumePath, stateDescription);
+ }
+ } catch (CloudRuntimeException e) {
+ throw e;
+ } catch (Exception e) {
+ String errorMsg = String.format(
+ "Exception while setting volume [%s] to %s state: %s",
+ volumePath, stateDescription, e.getMessage());
+ LOGGER.error(errorMsg, e);
+ throw new CloudRuntimeException(errorMsg, e);
+ }
+ }
+
+ public static void activateClvmVolumeExclusive(String volumePath) {
+ modifyClvmVolumeState(volumePath, ClvmVolumeState.EXCLUSIVE.getLvchangeFlag(),
+ ClvmVolumeState.EXCLUSIVE.getDescription(),
+ "Activating CLVM volume in exclusive mode");
+ }
+
+ public static void deactivateClvmVolume(String volumePath) {
+ try {
+ modifyClvmVolumeState(volumePath, ClvmVolumeState.DEACTIVATE.getLvchangeFlag(),
+ ClvmVolumeState.DEACTIVATE.getDescription(),
+ "Deactivating CLVM volume");
+ } catch (Exception e) {
+ LOGGER.warn("Failed to deactivate CLVM volume {}: {}", volumePath, e.getMessage());
+ }
+ }
+
+ public static void setClvmVolumeToSharedMode(String volumePath) {
Review Comment:
same
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/ClvmStorageAdaptor.java:
##########
@@ -0,0 +1,1072 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.hypervisor.kvm.storage;
+
+import static com.cloud.utils.NumbersUtil.toHumanReadableSize;
+
+import java.io.File;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.cloud.hypervisor.kvm.resource.LibvirtComputingResource;
+import com.cloud.hypervisor.kvm.resource.LibvirtConnection;
+import com.cloud.hypervisor.kvm.resource.LibvirtStoragePoolDef;
+import com.cloud.hypervisor.kvm.resource.LibvirtStorageVolumeDef;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Storage.StoragePoolType;
+import com.cloud.storage.StorageLayer;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.script.OutputInterpreter;
+import com.cloud.utils.script.Script;
+import com.google.gson.JsonObject;
+import com.google.gson.JsonParser;
+import org.apache.cloudstack.utils.qemu.QemuImg.PhysicalDiskFormat;
+import org.joda.time.Duration;
+import org.libvirt.Connect;
+import org.libvirt.LibvirtException;
+import org.libvirt.StoragePool;
+import org.libvirt.StorageVol;
+
+/**
+ * Storage adaptor for CLVM and CLVM_NG pool types.
+ * Extends LibvirtStorageAdaptor and overrides methods with CLVM-specific logic,
+ * using direct LVM commands instead of libvirt for volume operations.
+ */
+public class ClvmStorageAdaptor extends LibvirtStorageAdaptor {
+
+ public ClvmStorageAdaptor(StorageLayer storage) {
+ super(storage);
+ }
+
+ @Override
+ public StoragePoolType getStoragePoolType() {
+ // Registered manually for both CLVM and CLVM_NG in KVMStoragePoolManager
+ return null;
+ }
+
+ @Override
+ public KVMStoragePool createStoragePool(String name, String host, int port, String path,
+ String userInfo, StoragePoolType type, Map<String, String> details, boolean isPrimaryStorage) {
+ logger.info("Attempting to create CLVM/CLVM_NG storage pool {} in libvirt", name);
+
+ Connect conn;
+ try {
+ conn = LibvirtConnection.getConnection();
+ } catch (LibvirtException e) {
+ throw new CloudRuntimeException(e.toString());
+ }
+
+ StoragePool sp = createCLVMStoragePool(conn, name, host, path);
+ if (sp == null) {
+ logger.info("Falling back to virtual CLVM/CLVM_NG pool without libvirt for: {}", name);
+ return createVirtualClvmPool(name, host, path, type, details);
+ }
+
+ try {
+ if (!isPrimaryStorage) {
+ incStoragePoolRefCount(name);
+ }
+ // CLVM/CLVM_NG pools are kept inactive in libvirt; we use direct LVM commands
+ return getStoragePool(name);
+ } catch (Exception e) {
+ decStoragePoolRefCount(name);
+ throw new CloudRuntimeException("Failed to create CLVM storage pool: " + name, e);
+ }
+ }
+
+ @Override
+ public KVMStoragePool getStoragePool(String uuid, boolean refreshInfo) {
+ logger.info("Fetching CLVM/CLVM_NG storage pool {} ", uuid);
+ try {
+ Connect conn = LibvirtConnection.getConnection();
+ StoragePool storage = conn.storagePoolLookupByUUIDString(uuid);
+
+ LibvirtStoragePoolDef spd = getStoragePoolDef(conn, storage);
+ if (spd == null) {
+ throw new CloudRuntimeException("Unable to parse storage pool definition for pool " + uuid);
+ }
+
+ // CLVM pools in libvirt are always LOGICAL type
+ StoragePoolType type = StoragePoolType.CLVM;
+
+ // Do NOT activate the pool — CLVM/CLVM_NG pools stay inactive in libvirt
+ LibvirtStoragePool pool = new LibvirtStoragePool(uuid, storage.getName(), type, this, storage);
+ pool.setLocalPath(spd.getTargetPath());
+
+ // Always read capacity from LVM directly
+ String vgName = storage.getName();
+ try {
+ long[] vgStats = getVgStats(vgName);
+ setPoolCapacityFromVgStats(pool, vgStats, vgName);
+ } catch (CloudRuntimeException e) {
+ logger.warn("Failed to get VG stats for CLVM/CLVM_NG pool {}: {}. Using libvirt values (may be 0)", vgName, e.getMessage());
+ pool.setCapacity(storage.getInfo().capacity);
+ pool.setUsed(storage.getInfo().allocation);
+ pool.setAvailable(storage.getInfo().available);
+ }
+
+ return pool;
+ } catch (LibvirtException e) {
+ // Pool not in libvirt - return virtual pool backed only by LVM
+ logger.debug("CLVM/CLVM_NG pool {} not found in libvirt, creating virtual pool", uuid);
+ throw new CloudRuntimeException(e.toString(), e);
Review Comment:
The comment and log make it seem like we are trying something else and returning something, but we are throwing an exception.
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java:
##########
@@ -6820,4 +6827,242 @@ public String getHypervisorPath() {
public String getGuestCpuArch() {
return guestCpuArch;
}
+
+ /**
+ * CLVM volume state for migration operations on source host
+ */
+ public enum ClvmVolumeState {
+ /** Shared mode (-asy) - used before migration to allow both hosts to access volume */
+ SHARED("-asy", "shared", "Before migration: activating in shared mode"),
+
+ /** Deactivate (-an) - used after successful migration to release volume on source */
+ DEACTIVATE("-an", "deactivated", "After successful migration: deactivating volume"),
+
+ /** Exclusive mode (-aey) - used after failed migration to revert to original exclusive state */
+ EXCLUSIVE("-aey", "exclusive", "After failed migration: reverting to exclusive mode");
+
+ private final String lvchangeFlag;
+ private final String description;
+ private final String logMessage;
+
+ ClvmVolumeState(String lvchangeFlag, String description, String logMessage) {
+ this.lvchangeFlag = lvchangeFlag;
+ this.description = description;
+ this.logMessage = logMessage;
+ }
+
+ public String getLvchangeFlag() {
+ return lvchangeFlag;
+ }
+
+ public String getDescription() {
+ return description;
+ }
+
+ public String getLogMessage() {
+ return logMessage;
+ }
+ }
+
+ public static void modifyClvmVolumesStateForMigration(List<DiskDef> disks, LibvirtComputingResource resource,
+ VirtualMachineTO vmSpec, ClvmVolumeState state) {
+ for (DiskDef disk : disks) {
+ if (isClvmVolume(disk, resource, vmSpec)) {
+ String volumePath = disk.getDiskPath();
+ try {
+ modifyClvmVolumeState(volumePath, state.getLvchangeFlag(), state.getDescription(), state.getLogMessage());
+ } catch (Exception e) {
+ LOGGER.error("[CLVM Migration] Exception while setting volume [{}] to {} state: {}",
+ volumePath, state.getDescription(), e.getMessage(), e);
+ }
+ }
+ }
+ }
+
+ private static void modifyClvmVolumeState(String volumePath, String lvchangeFlag,
+ String stateDescription, String logMessage) {
+ try {
+ LOGGER.info("{} for volume [{}]", logMessage, volumePath);
+
+ Script cmd = new Script("lvchange", Duration.standardSeconds(300), LOGGER);
+ cmd.add(lvchangeFlag);
+ cmd.add(volumePath);
+
+ String result = cmd.execute();
+ if (result != null) {
+ String errorMsg = String.format(
+ "Failed to set volume [%s] to %s state. Command result: %s",
+ volumePath, stateDescription, result);
+ LOGGER.error(errorMsg);
+ throw new CloudRuntimeException(errorMsg);
+ } else {
+ LOGGER.info("Successfully set volume [{}] to {} state.",
+ volumePath, stateDescription);
+ }
+ } catch (CloudRuntimeException e) {
+ throw e;
+ } catch (Exception e) {
+ String errorMsg = String.format(
+ "Exception while setting volume [%s] to %s state: %s",
+ volumePath, stateDescription, e.getMessage());
+ LOGGER.error(errorMsg, e);
+ throw new CloudRuntimeException(errorMsg, e);
+ }
+ }
+
+ public static void activateClvmVolumeExclusive(String volumePath) {
Review Comment:
same
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java:
##########
@@ -1845,8 +2042,10 @@ public Answer createSnapshot(final CreateObjectCommand cmd) {
}
}
- if (DomainInfo.DomainState.VIR_DOMAIN_RUNNING.equals(state) && volume.requiresEncryption()) {
- throw new CloudRuntimeException("VM is running, encrypted volume snapshots aren't supported");
+ if (DomainInfo.DomainState.VIR_DOMAIN_RUNNING.equals(state)) {
+ if (volume.requiresEncryption()) {
+ throw new CloudRuntimeException("VM is running, encrypted volume snapshots aren't supported");
+ }
Review Comment:
???
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java:
##########
@@ -2123,6 +2347,209 @@ protected String getParentCheckpointName(String[] parents) {
return immediateParentPath.substring(immediateParentPath.lastIndexOf(File.separator) + 1);
}
+ /**
+ * Checks if a QEMU bitmap exists in the volume and is usable (not in-use or corrupted).
+ * This is important after lock migration where bitmaps may be left in "in-use" state.
+ *
+ * @param pool The storage pool containing the volume
+ * @param volume The volume to check
+ * @param bitmapName The name of the bitmap to check
+ * @return true if bitmap exists and is usable, false if missing, in-use, or corrupted
+ */
+ protected boolean isBitmapUsable(KVMStoragePool pool, VolumeObjectTO volume, String bitmapName) {
+ try {
+ String volumePath = pool.getLocalPathFor(volume.getPath());
+
+ String command = String.format("qemu-img info --output=json -U %s", volumePath);
+ String jsonOutput = Script.runSimpleBashScriptWithFullResult(command, 30);
+
+ if (jsonOutput == null || jsonOutput.trim().isEmpty()) {
+ logger.warn("Failed to get qemu-img info for volume [{}]", volumePath);
+ return false;
+ }
+
+ logger.debug("qemu-img info output for volume [{}]: {}", volumePath, jsonOutput);
+
+ try {
+ ObjectMapper mapper = new ObjectMapper();
+ JsonNode root = mapper.readTree(jsonOutput);
+
+ JsonNode formatSpecific = root.path("format-specific");
+ if (formatSpecific.isMissingNode()) {
+ logger.debug("No format-specific data found for volume [{}], bitmap check skipped", volumePath);
+ return false;
+ }
+
+ JsonNode data = formatSpecific.path("data");
+ JsonNode bitmaps = data.path("bitmaps");
+
+ if (bitmaps.isMissingNode() || !bitmaps.isArray()) {
+ logger.debug("No bitmaps found in volume [{}]", volumePath);
+ return false;
+ }
+
+ for (JsonNode bitmap : bitmaps) {
+ String name = bitmap.path("name").asText();
+ if (bitmapName.equals(name)) {
+ JsonNode flags = bitmap.path("flags");
+ if (flags.isArray()) {
+ for (JsonNode flag : flags) {
+ String flagValue = flag.asText();
+ if ("in-use".equals(flagValue)) {
+ logger.warn("Bitmap [{}] in volume [{}] is marked as 'in-use' and cannot be used for incremental snapshot",
+ bitmapName, volumePath);
+ return false;
+ }
+ }
+ }
+ logger.debug("Bitmap [{}] found in volume [{}] and is usable", bitmapName, volumePath);
+ return true;
+ }
+ }
+
+ logger.warn("Bitmap [{}] not found in volume [{}]", bitmapName, volumePath);
+ return false;
+
+ } catch (JsonProcessingException e) {
+ logger.error("Failed to parse qemu-img JSON output for volume [{}]: {}", volumePath, e.getMessage(), e);
+ return false;
+ }
+
+ } catch (Exception e) {
+ logger.error("Error checking bitmap [{}] for volume [{}]: {}", bitmapName, volume, e.getMessage(), e);
+ return false;
+ }
+ }
+
+ /**
+ * Removes a broken or unusable bitmap from a volume.
+ * Called before falling back to full snapshot to keep volume metadata clean.
+ *
+ * @param pool Storage pool containing the volume
+ * @param volume Volume with broken bitmap
+ * @param bitmapName Name of bitmap to remove
+ */
+ private void cleanupBrokenBitmap(KVMStoragePool pool, VolumeObjectTO volume, String bitmapName) {
Review Comment:
this is used in a method that is not used anywhere, we should remove it.
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java:
##########
@@ -2123,6 +2347,209 @@ protected String getParentCheckpointName(String[] parents) {
return immediateParentPath.substring(immediateParentPath.lastIndexOf(File.separator) + 1);
}
+ /**
+ * Checks if a QEMU bitmap exists in the volume and is usable (not in-use or corrupted).
+ * This is important after lock migration where bitmaps may be left in "in-use" state.
+ *
+ * @param pool The storage pool containing the volume
+ * @param volume The volume to check
+ * @param bitmapName The name of the bitmap to check
+ * @return true if bitmap exists and is usable, false if missing, in-use, or corrupted
+ */
+ protected boolean isBitmapUsable(KVMStoragePool pool, VolumeObjectTO volume, String bitmapName) {
Review Comment:
this is used in a method that is not used anywhere, we should remove it.
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/ClvmStorageAdaptor.java:
##########
@@ -0,0 +1,1072 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.hypervisor.kvm.storage;
+
+import static com.cloud.utils.NumbersUtil.toHumanReadableSize;
+
+import java.io.File;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.cloud.hypervisor.kvm.resource.LibvirtComputingResource;
+import com.cloud.hypervisor.kvm.resource.LibvirtConnection;
+import com.cloud.hypervisor.kvm.resource.LibvirtStoragePoolDef;
+import com.cloud.hypervisor.kvm.resource.LibvirtStorageVolumeDef;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Storage.StoragePoolType;
+import com.cloud.storage.StorageLayer;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.script.OutputInterpreter;
+import com.cloud.utils.script.Script;
+import com.google.gson.JsonObject;
+import com.google.gson.JsonParser;
+import org.apache.cloudstack.utils.qemu.QemuImg.PhysicalDiskFormat;
+import org.joda.time.Duration;
+import org.libvirt.Connect;
+import org.libvirt.LibvirtException;
+import org.libvirt.StoragePool;
+import org.libvirt.StorageVol;
+
+/**
+ * Storage adaptor for CLVM and CLVM_NG pool types.
+ * Extends LibvirtStorageAdaptor and overrides methods with CLVM-specific logic,
+ * using direct LVM commands instead of libvirt for volume operations.
+ */
+public class ClvmStorageAdaptor extends LibvirtStorageAdaptor {
+
+ public ClvmStorageAdaptor(StorageLayer storage) {
+ super(storage);
+ }
+
+ @Override
+ public StoragePoolType getStoragePoolType() {
+ // Registered manually for both CLVM and CLVM_NG in KVMStoragePoolManager
+ return null;
+ }
+
+ @Override
+ public KVMStoragePool createStoragePool(String name, String host, int port, String path,
+ String userInfo, StoragePoolType type, Map<String, String> details, boolean isPrimaryStorage) {
+ logger.info("Attempting to create CLVM/CLVM_NG storage pool {} in libvirt", name);
+
+ Connect conn;
+ try {
+ conn = LibvirtConnection.getConnection();
+ } catch (LibvirtException e) {
+ throw new CloudRuntimeException(e.toString());
+ }
+
+ StoragePool sp = createCLVMStoragePool(conn, name, host, path);
+ if (sp == null) {
+ logger.info("Falling back to virtual CLVM/CLVM_NG pool without libvirt for: {}", name);
+ return createVirtualClvmPool(name, host, path, type, details);
+ }
+
+ try {
+ if (!isPrimaryStorage) {
+ incStoragePoolRefCount(name);
+ }
+ // CLVM/CLVM_NG pools are kept inactive in libvirt; we use direct LVM commands
+ return getStoragePool(name);
+ } catch (Exception e) {
+ decStoragePoolRefCount(name);
+ throw new CloudRuntimeException("Failed to create CLVM storage pool: " + name, e);
+ }
+ }
+
+ @Override
+ public KVMStoragePool getStoragePool(String uuid, boolean refreshInfo) {
+ logger.info("Fetching CLVM/CLVM_NG storage pool {} ", uuid);
+ try {
+ Connect conn = LibvirtConnection.getConnection();
+ StoragePool storage = conn.storagePoolLookupByUUIDString(uuid);
+
+ LibvirtStoragePoolDef spd = getStoragePoolDef(conn, storage);
+ if (spd == null) {
+ throw new CloudRuntimeException("Unable to parse storage pool definition for pool " + uuid);
+ }
+
+ // CLVM pools in libvirt are always LOGICAL type
+ StoragePoolType type = StoragePoolType.CLVM;
+
+ // Do NOT activate the pool — CLVM/CLVM_NG pools stay inactive in libvirt
+ LibvirtStoragePool pool = new LibvirtStoragePool(uuid, storage.getName(), type, this, storage);
+ pool.setLocalPath(spd.getTargetPath());
+
+ // Always read capacity from LVM directly
+ String vgName = storage.getName();
+ try {
+ long[] vgStats = getVgStats(vgName);
+ setPoolCapacityFromVgStats(pool, vgStats, vgName);
+ } catch (CloudRuntimeException e) {
+ logger.warn("Failed to get VG stats for CLVM/CLVM_NG pool {}: {}. Using libvirt values (may be 0)", vgName, e.getMessage());
+ pool.setCapacity(storage.getInfo().capacity);
+ pool.setUsed(storage.getInfo().allocation);
+ pool.setAvailable(storage.getInfo().available);
+ }
+
+ return pool;
+ } catch (LibvirtException e) {
+ // Pool not in libvirt - return virtual pool backed only by LVM
+ logger.debug("CLVM/CLVM_NG pool {} not found in libvirt, creating virtual pool", uuid);
+ throw new CloudRuntimeException(e.toString(), e);
+ }
+ }
+
+ @Override
+ public KVMPhysicalDisk getPhysicalDisk(String volumeUuid, KVMStoragePool pool) {
+ LibvirtStoragePool libvirtPool = (LibvirtStoragePool) pool;
+
+ // Pool has no libvirt backing - go directly to block device
+ if (libvirtPool.getPool() == null) {
+ logger.debug("CLVM/CLVM_NG pool has no libvirt backing, using direct block device access for volume: {}", volumeUuid);
+ return getPhysicalDiskViaDirectBlockDevice(volumeUuid, pool);
+ }
+
+ try {
+ StorageVol vol = getVolume(libvirtPool.getPool(), volumeUuid);
+ if (vol == null) {
+ logger.debug("Volume {} not found in libvirt, falling back to CLVM direct access", volumeUuid);
+ return getPhysicalDiskWithClvmFallback(volumeUuid, pool, libvirtPool);
+ }
+
+ LibvirtStorageVolumeDef voldef = getStorageVolumeDef(libvirtPool.getPool().getConnect(), vol);
+ KVMPhysicalDisk disk = new KVMPhysicalDisk(vol.getPath(), vol.getName(), pool);
+ disk.setSize(vol.getInfo().allocation);
+ disk.setVirtualSize(vol.getInfo().capacity);
+ disk.setFormat(voldef.getFormat() == LibvirtStorageVolumeDef.VolumeFormat.QCOW2
+ ? PhysicalDiskFormat.QCOW2 : PhysicalDiskFormat.RAW);
+ return disk;
+ } catch (LibvirtException e) {
+ logger.warn("LibvirtException looking up volume {}: {}", volumeUuid, e.getMessage());
+ return getPhysicalDiskWithClvmFallback(volumeUuid, pool, libvirtPool);
+ }
+ }
+
+ @Override
+ public KVMPhysicalDisk createPhysicalDisk(String name, KVMStoragePool pool,
+ PhysicalDiskFormat format, Storage.ProvisioningType provisioningType, long size, byte[] passphrase) {
+ logger.info("Creating CLVM/CLVM_NG volume {} in pool {} with size {}", name, pool.getUuid(), toHumanReadableSize(size));
+
+ if (StoragePoolType.CLVM_NG.equals(pool.getType())) {
+ return createClvmNgDiskWithBacking(name, 0, size, null, pool, provisioningType);
+ } else {
+ return createClvmVolume(name, size, pool);
+ }
+ }
+
+ @Override
+ public boolean connectPhysicalDisk(String name, KVMStoragePool pool, Map<String, String> details, boolean isVMMigrate) {
+ if (isVMMigrate) {
+ logger.info("Activating CLVM/CLVM_NG volume {} in shared mode for VM migration", name);
+ Script activateVol = new Script("lvchange", 10000, logger);
+ activateVol.add("-asy");
+ activateVol.add(pool.getLocalPath() + File.separator + name);
+ String result = activateVol.execute();
+ if (result != null) {
+ logger.error("Failed to activate CLVM/CLVM_NG volume {} in shared mode. Output: {}", name, result);
+ return false;
+ }
+ }
+
+ if (StoragePoolType.CLVM_NG.equals(pool.getType())) {
+ ensureClvmNgBackingFileAccessible(name, pool);
+ }
+
+ return true;
+ }
+
+ @Override
+ public KVMPhysicalDisk createDiskFromTemplate(KVMPhysicalDisk template,
+ String name, PhysicalDiskFormat format, Storage.ProvisioningType provisioningType,
+ long size, KVMStoragePool destPool, int timeout, byte[] passphrase) {
+
+ if (StoragePoolType.CLVM_NG.equals(destPool.getType()) && format == PhysicalDiskFormat.QCOW2) {
+ logger.info("Creating CLVM_NG volume {} with backing file from template {}", name, template.getName());
+ String backingFile = getClvmBackingFile(template, destPool);
+ return createClvmNgDiskWithBacking(name, timeout, size, backingFile, destPool, provisioningType);
+ }
+
+ return super.createDiskFromTemplate(template, name, format, provisioningType, size, destPool, timeout, passphrase);
+ }
+
+ @Override
+ public void createTemplate(String templatePath, String templateUuid, int timeout, KVMStoragePool pool) {
+ String vgName = getVgName(pool.getLocalPath());
+ String lvName = "template-" + templateUuid;
+ String lvPath = "/dev/" + vgName + "/" + lvName;
+
+ if (lvExists(lvPath)) {
+ logger.info("Template LV {} already exists in VG {}. Skipping creation.", lvName, vgName);
+ return;
+ }
+
+ logger.info("Creating new template LV {} in VG {} for template {}", lvName, vgName, templateUuid);
+
+ long virtualSize = getQcow2VirtualSize(templatePath);
+ long physicalSize = getQcow2PhysicalSize(templatePath);
+ long lvSize = virtualSize;
+
+ logger.info("Template source - Physical: {} bytes, Virtual: {} bytes, LV will be: {} bytes", physicalSize, virtualSize, lvSize);
+
+ Script lvcreate = new Script("lvcreate", Duration.millis(timeout), logger);
+ lvcreate.add("-n", lvName);
+ lvcreate.add("-L", lvSize + "B");
+ lvcreate.add("--yes");
+ lvcreate.add(vgName);
+ String result = lvcreate.execute();
+ if (result != null) {
+ throw new CloudRuntimeException("Failed to create LV for CLVM_NG template: " + result);
+ }
+
+ Script qemuImgConvert = new Script("qemu-img", Duration.millis(timeout), logger);
+ qemuImgConvert.add("convert");
+ qemuImgConvert.add(templatePath);
+ qemuImgConvert.add("-O", "qcow2");
+ qemuImgConvert.add("-o", "cluster_size=64k,extended_l2=off,preallocation=off");
+ qemuImgConvert.add(lvPath);
+ result = qemuImgConvert.execute();
Review Comment:
Please use the QemuImg class.
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/ClvmStorageAdaptor.java:
##########
@@ -0,0 +1,1072 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.hypervisor.kvm.storage;
+
+import static com.cloud.utils.NumbersUtil.toHumanReadableSize;
+
+import java.io.File;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.cloud.hypervisor.kvm.resource.LibvirtComputingResource;
+import com.cloud.hypervisor.kvm.resource.LibvirtConnection;
+import com.cloud.hypervisor.kvm.resource.LibvirtStoragePoolDef;
+import com.cloud.hypervisor.kvm.resource.LibvirtStorageVolumeDef;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Storage.StoragePoolType;
+import com.cloud.storage.StorageLayer;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.script.OutputInterpreter;
+import com.cloud.utils.script.Script;
+import com.google.gson.JsonObject;
+import com.google.gson.JsonParser;
+import org.apache.cloudstack.utils.qemu.QemuImg.PhysicalDiskFormat;
+import org.joda.time.Duration;
+import org.libvirt.Connect;
+import org.libvirt.LibvirtException;
+import org.libvirt.StoragePool;
+import org.libvirt.StorageVol;
+
+/**
+ * Storage adaptor for CLVM and CLVM_NG pool types.
+ * Extends LibvirtStorageAdaptor and overrides methods with CLVM-specific logic,
+ * using direct LVM commands instead of libvirt for volume operations.
+ */
+public class ClvmStorageAdaptor extends LibvirtStorageAdaptor {
+
+ public ClvmStorageAdaptor(StorageLayer storage) {
+ super(storage);
+ }
+
+ @Override
+ public StoragePoolType getStoragePoolType() {
+ // Registered manually for both CLVM and CLVM_NG in KVMStoragePoolManager
+ return null;
+ }
+
+ @Override
+ public KVMStoragePool createStoragePool(String name, String host, int port, String path,
+ String userInfo, StoragePoolType type, Map<String, String> details, boolean isPrimaryStorage) {
+ logger.info("Attempting to create CLVM/CLVM_NG storage pool {} in libvirt", name);
+
+ Connect conn;
+ try {
+ conn = LibvirtConnection.getConnection();
+ } catch (LibvirtException e) {
+ throw new CloudRuntimeException(e.toString());
+ }
+
+ StoragePool sp = createCLVMStoragePool(conn, name, host, path);
+ if (sp == null) {
+ logger.info("Falling back to virtual CLVM/CLVM_NG pool without libvirt for: {}", name);
+ return createVirtualClvmPool(name, host, path, type, details);
+ }
+
+ try {
+ if (!isPrimaryStorage) {
+ incStoragePoolRefCount(name);
+ }
+ // CLVM/CLVM_NG pools are kept inactive in libvirt; we use direct LVM commands
+ return getStoragePool(name);
+ } catch (Exception e) {
+ decStoragePoolRefCount(name);
+ throw new CloudRuntimeException("Failed to create CLVM storage pool: " + name, e);
+ }
+ }
+
+ @Override
+ public KVMStoragePool getStoragePool(String uuid, boolean refreshInfo) {
+ logger.info("Fetching CLVM/CLVM_NG storage pool {} ", uuid);
+ try {
+ Connect conn = LibvirtConnection.getConnection();
+ StoragePool storage = conn.storagePoolLookupByUUIDString(uuid);
+
+ LibvirtStoragePoolDef spd = getStoragePoolDef(conn, storage);
+ if (spd == null) {
+ throw new CloudRuntimeException("Unable to parse storage pool definition for pool " + uuid);
+ }
+
+ // CLVM pools in libvirt are always LOGICAL type
+ StoragePoolType type = StoragePoolType.CLVM;
+
+ // Do NOT activate the pool — CLVM/CLVM_NG pools stay inactive in libvirt
+ LibvirtStoragePool pool = new LibvirtStoragePool(uuid, storage.getName(), type, this, storage);
+ pool.setLocalPath(spd.getTargetPath());
+
+ // Always read capacity from LVM directly
+ String vgName = storage.getName();
+ try {
+ long[] vgStats = getVgStats(vgName);
+ setPoolCapacityFromVgStats(pool, vgStats, vgName);
+ } catch (CloudRuntimeException e) {
+ logger.warn("Failed to get VG stats for CLVM/CLVM_NG pool {}: {}. Using libvirt values (may be 0)", vgName, e.getMessage());
+ pool.setCapacity(storage.getInfo().capacity);
+ pool.setUsed(storage.getInfo().allocation);
+ pool.setAvailable(storage.getInfo().available);
+ }
+
+ return pool;
+ } catch (LibvirtException e) {
+ // Pool not in libvirt - return virtual pool backed only by LVM
+ logger.debug("CLVM/CLVM_NG pool {} not found in libvirt, creating virtual pool", uuid);
+ throw new CloudRuntimeException(e.toString(), e);
+ }
+ }
+
+ @Override
+ public KVMPhysicalDisk getPhysicalDisk(String volumeUuid, KVMStoragePool pool) {
+ LibvirtStoragePool libvirtPool = (LibvirtStoragePool) pool;
+
+ // Pool has no libvirt backing - go directly to block device
+ if (libvirtPool.getPool() == null) {
+ logger.debug("CLVM/CLVM_NG pool has no libvirt backing, using direct block device access for volume: {}", volumeUuid);
+ return getPhysicalDiskViaDirectBlockDevice(volumeUuid, pool);
+ }
+
+ try {
+ StorageVol vol = getVolume(libvirtPool.getPool(), volumeUuid);
+ if (vol == null) {
+ logger.debug("Volume {} not found in libvirt, falling back to CLVM direct access", volumeUuid);
+ return getPhysicalDiskWithClvmFallback(volumeUuid, pool, libvirtPool);
+ }
+
+ LibvirtStorageVolumeDef voldef = getStorageVolumeDef(libvirtPool.getPool().getConnect(), vol);
+ KVMPhysicalDisk disk = new KVMPhysicalDisk(vol.getPath(), vol.getName(), pool);
+ disk.setSize(vol.getInfo().allocation);
+ disk.setVirtualSize(vol.getInfo().capacity);
+ disk.setFormat(voldef.getFormat() == LibvirtStorageVolumeDef.VolumeFormat.QCOW2
+ ? PhysicalDiskFormat.QCOW2 : PhysicalDiskFormat.RAW);
+ return disk;
+ } catch (LibvirtException e) {
+ logger.warn("LibvirtException looking up volume {}: {}", volumeUuid, e.getMessage());
+ return getPhysicalDiskWithClvmFallback(volumeUuid, pool, libvirtPool);
+ }
+ }
+
+ @Override
+ public KVMPhysicalDisk createPhysicalDisk(String name, KVMStoragePool pool,
+ PhysicalDiskFormat format, Storage.ProvisioningType provisioningType, long size, byte[] passphrase) {
+ logger.info("Creating CLVM/CLVM_NG volume {} in pool {} with size {}", name, pool.getUuid(), toHumanReadableSize(size));
+
+ if (StoragePoolType.CLVM_NG.equals(pool.getType())) {
+ return createClvmNgDiskWithBacking(name, 0, size, null, pool, provisioningType);
+ } else {
+ return createClvmVolume(name, size, pool);
+ }
+ }
+
+ @Override
+ public boolean connectPhysicalDisk(String name, KVMStoragePool pool, Map<String, String> details, boolean isVMMigrate) {
+ if (isVMMigrate) {
+ logger.info("Activating CLVM/CLVM_NG volume {} in shared mode for VM migration", name);
+ Script activateVol = new Script("lvchange", 10000, logger);
+ activateVol.add("-asy");
+ activateVol.add(pool.getLocalPath() + File.separator + name);
+ String result = activateVol.execute();
+ if (result != null) {
+ logger.error("Failed to activate CLVM/CLVM_NG volume {} in shared mode. Output: {}", name, result);
+ return false;
+ }
+ }
+
+ if (StoragePoolType.CLVM_NG.equals(pool.getType())) {
+ ensureClvmNgBackingFileAccessible(name, pool);
+ }
+
+ return true;
+ }
+
+ @Override
+ public KVMPhysicalDisk createDiskFromTemplate(KVMPhysicalDisk template,
+ String name, PhysicalDiskFormat format, Storage.ProvisioningType provisioningType,
+ long size, KVMStoragePool destPool, int timeout, byte[] passphrase) {
+
+ if (StoragePoolType.CLVM_NG.equals(destPool.getType()) && format == PhysicalDiskFormat.QCOW2) {
+ logger.info("Creating CLVM_NG volume {} with backing file from template {}", name, template.getName());
+ String backingFile = getClvmBackingFile(template, destPool);
+ return createClvmNgDiskWithBacking(name, timeout, size, backingFile, destPool, provisioningType);
+ }
+
+ return super.createDiskFromTemplate(template, name, format, provisioningType, size, destPool, timeout, passphrase);
+ }
+
+ @Override
+ public void createTemplate(String templatePath, String templateUuid, int timeout, KVMStoragePool pool) {
+ String vgName = getVgName(pool.getLocalPath());
+ String lvName = "template-" + templateUuid;
+ String lvPath = "/dev/" + vgName + "/" + lvName;
+
+ if (lvExists(lvPath)) {
+ logger.info("Template LV {} already exists in VG {}. Skipping creation.", lvName, vgName);
+ return;
+ }
+
+ logger.info("Creating new template LV {} in VG {} for template {}", lvName, vgName, templateUuid);
+
+ long virtualSize = getQcow2VirtualSize(templatePath);
+ long physicalSize = getQcow2PhysicalSize(templatePath);
+ long lvSize = virtualSize;
+
+ logger.info("Template source - Physical: {} bytes, Virtual: {} bytes, LV will be: {} bytes", physicalSize, virtualSize, lvSize);
+
+ Script lvcreate = new Script("lvcreate", Duration.millis(timeout), logger);
+ lvcreate.add("-n", lvName);
+ lvcreate.add("-L", lvSize + "B");
+ lvcreate.add("--yes");
+ lvcreate.add(vgName);
+ String result = lvcreate.execute();
+ if (result != null) {
+ throw new CloudRuntimeException("Failed to create LV for CLVM_NG template: " + result);
+ }
+
+ Script qemuImgConvert = new Script("qemu-img", Duration.millis(timeout), logger);
+ qemuImgConvert.add("convert");
+ qemuImgConvert.add(templatePath);
+ qemuImgConvert.add("-O", "qcow2");
+ qemuImgConvert.add("-o", "cluster_size=64k,extended_l2=off,preallocation=off");
+ qemuImgConvert.add(lvPath);
+ result = qemuImgConvert.execute();
+
+ if (result != null) {
+ removeLvOnFailure(lvPath, timeout);
+ throw new CloudRuntimeException("Failed to convert template to CLVM_NG volume: " + result);
+ }
+
+ long actualVirtualSize = getQcow2VirtualSize(lvPath);
+
+ try {
+ ensureTemplateLvInSharedMode(lvPath, true);
+ } catch (CloudRuntimeException e) {
+ logger.error("Failed to activate template LV {} in shared mode. Cleaning up.", lvPath);
+ removeLvOnFailure(lvPath, timeout);
+ throw e;
+ }
+
+ KVMPhysicalDisk templateDisk = new KVMPhysicalDisk(lvPath, lvName, pool);
+ templateDisk.setFormat(PhysicalDiskFormat.QCOW2);
+ templateDisk.setVirtualSize(actualVirtualSize);
+ templateDisk.setSize(lvSize);
+ }
+
+ private StoragePool createCLVMStoragePool(Connect conn, String uuid, String host, String path) {
+ String volgroupPath = "/dev/" + path;
+ String volgroupName = path;
+ volgroupName = volgroupName.replaceFirst("^/", "");
+
+ Script checkVgExists = new Script("vgs", 10000, logger);
+ checkVgExists.add("--noheadings");
+ checkVgExists.add("-o", "vg_name");
+ checkVgExists.add(volgroupName);
+ String vgCheckResult = checkVgExists.execute();
+
+ if (vgCheckResult != null) {
+ logger.error("Volume group {} does not exist or is not accessible", volgroupName);
+ return null;
+ }
+
+ logger.info("Volume group {} verified, creating libvirt pool definition for CLVM/CLVM_NG", volgroupName);
+ LibvirtStoragePoolDef poolDef = new LibvirtStoragePoolDef(
+ LibvirtStoragePoolDef.PoolType.LOGICAL,
+ volgroupName,
+ uuid,
+ null,
+ volgroupName,
+ volgroupPath
+ );
+
+ try {
+ StoragePool pool = conn.storagePoolDefineXML(poolDef.toString(), 0);
+ logger.info("Created libvirt pool definition for CLVM/CLVM_NG VG: {} (pool will remain inactive)", volgroupName);
+ pool.setAutostart(1);
+ return pool;
+ } catch (LibvirtException e) {
+ logger.warn("Failed to define CLVM/CLVM_NG pool in libvirt: {}", e.getMessage());
+ return null;
+ }
+ }
+
+ private void setPoolCapacityFromVgStats(LibvirtStoragePool pool, long[] vgStats, String vgName) {
+ long capacity = vgStats[0];
+ long available = vgStats[1];
+ long used = capacity - available;
+
+ pool.setCapacity(capacity);
+ pool.setAvailable(available);
+ pool.setUsed(used);
+
+ logger.debug("CLVM/CLVM_NG pool {} - Capacity: {}, Used: {}, Available: {}",
+ vgName, toHumanReadableSize(capacity), toHumanReadableSize(used), toHumanReadableSize(available));
+ }
+
+ private long[] getVgStats(String vgName) {
+ Script getVgStats = new Script("vgs", 10000, logger);
+ getVgStats.add("--noheadings");
+ getVgStats.add("--units", "b");
+ getVgStats.add("--nosuffix");
+ getVgStats.add("-o", "vg_size,vg_free");
+ getVgStats.add(vgName);
+
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ String result = getVgStats.execute(parser);
+
+ if (result != null) {
+ String errorMsg = "Failed to get statistics for volume group " + vgName + ": " + result;
+ logger.error(errorMsg);
+ throw new CloudRuntimeException(errorMsg);
+ }
+
+ String output = parser.getLines().trim();
+ String[] lines = output.split("\\n");
+ String dataLine = null;
+
+ for (String line : lines) {
+ line = line.trim();
+ if (!line.isEmpty() && Character.isDigit(line.charAt(0))) {
+ dataLine = line;
+ break;
+ }
+ }
+
+ if (dataLine == null) {
+ String errorMsg = "No valid data line found in vgs output for " + vgName + ": " + output;
+ logger.error(errorMsg);
+ throw new CloudRuntimeException(errorMsg);
+ }
+
+ String[] stats = dataLine.split("\\s+");
+
+ if (stats.length < 2) {
+ String errorMsg = "Unexpected output from vgs command for " + vgName + ": " + dataLine;
+ logger.error(errorMsg);
+ throw new CloudRuntimeException(errorMsg);
+ }
+
+ try {
+ long capacity = Long.parseLong(stats[0].trim());
+ long available = Long.parseLong(stats[1].trim());
+ return new long[]{capacity, available};
+ } catch (NumberFormatException e) {
+ String errorMsg = "Failed to parse VG statistics for " + vgName + ": " + e.getMessage();
+ logger.error(errorMsg);
+ throw new CloudRuntimeException(errorMsg, e);
+ }
+ }
+
+ private KVMStoragePool createVirtualClvmPool(String uuid, String host, String path, StoragePoolType type, Map<String, String> details) {
+ String volgroupName = path.replaceFirst("^/", "");
+ String volgroupPath = "/dev/" + volgroupName;
+
+ logger.info("Creating virtual CLVM/CLVM_NG pool {} without libvirt using direct LVM access", volgroupName);
+
+ long[] vgStats = getVgStats(volgroupName);
+
+ LibvirtStoragePool pool = new LibvirtStoragePool(uuid, volgroupName, type, this, null);
+ pool.setLocalPath(volgroupPath);
+ setPoolCapacityFromVgStats(pool, vgStats, volgroupName);
+
+ if (details != null) {
+ pool.setDetails(details);
+ }
+
+ return pool;
+ }
+
+ /**
+ * CLVM fallback: First tries to refresh libvirt pool to make volume visible,
+ * if that fails, accesses volume directly via block device path.
+ */
+ private KVMPhysicalDisk getPhysicalDiskWithClvmFallback(String volumeUuid, KVMStoragePool pool, LibvirtStoragePool libvirtPool) {
+ logger.info("CLVM volume not visible to libvirt, attempting pool refresh for volume: {}", volumeUuid);
+
+ try {
+ logger.debug("Refreshing libvirt storage pool: {}", pool.getUuid());
+ libvirtPool.getPool().refresh(0);
+
+ StorageVol vol = getVolume(libvirtPool.getPool(), volumeUuid);
+ if (vol != null) {
+ logger.info("Volume found after pool refresh: {}", volumeUuid);
+ LibvirtStorageVolumeDef voldef = getStorageVolumeDef(libvirtPool.getPool().getConnect(), vol);
+ KVMPhysicalDisk disk = new KVMPhysicalDisk(vol.getPath(), vol.getName(), pool);
+ disk.setSize(vol.getInfo().allocation);
+ disk.setVirtualSize(vol.getInfo().capacity);
+ disk.setFormat(voldef.getFormat() == LibvirtStorageVolumeDef.VolumeFormat.QCOW2
+ ? PhysicalDiskFormat.QCOW2 : PhysicalDiskFormat.RAW);
+ return disk;
+ }
+ } catch (LibvirtException refreshEx) {
+ logger.debug("Pool refresh failed or volume still not found: {}", refreshEx.getMessage());
+ }
+
+ logger.info("Falling back to direct block device access for volume: {}", volumeUuid);
+ return getPhysicalDiskViaDirectBlockDevice(volumeUuid, pool);
+ }
+
+ private String getVgName(String sourceDir) {
+ String vgName = sourceDir;
+ if (vgName.startsWith("/")) {
+ String[] parts = vgName.split("/");
+ List<String> tokens = Arrays.stream(parts)
+ .filter(s -> !s.isEmpty()).collect(Collectors.toList());
+
+ vgName = tokens.size() > 1 ? tokens.get(1)
+ : tokens.size() == 1 ? tokens.get(0)
+ : "";
+ }
+ return vgName;
+ }
+
+ private String extractVgNameFromPool(KVMStoragePool pool) {
+ String sourceDir = pool.getLocalPath();
+ if (sourceDir == null || sourceDir.isEmpty()) {
+ throw new CloudRuntimeException("CLVM pool sourceDir is not set, cannot determine VG name");
+ }
+ String vgName = getVgName(sourceDir);
+ logger.debug("Using VG name: {} (from sourceDir: {})", vgName, sourceDir);
+ return vgName;
+ }
+
+ /**
+ * For CLVM volumes that exist in LVM but are not visible to libvirt,
+ * access them directly via block device path.
+ */
+ private KVMPhysicalDisk getPhysicalDiskViaDirectBlockDevice(String volumeUuid, KVMStoragePool pool) {
+ try {
+ String vgName = extractVgNameFromPool(pool);
+
+ verifyLvExistsInVg(volumeUuid, vgName);
+
+ logger.info("Volume {} exists in LVM but not visible to libvirt, accessing directly", volumeUuid);
+
+ String lvPath = findAccessibleDeviceNode(volumeUuid, vgName, pool);
+ long size = getClvmVolumeSize(lvPath);
+
+ KVMPhysicalDisk disk = createPhysicalDiskFromClvmLv(lvPath, volumeUuid, pool, size);
+ ensureTemplateAccessibility(volumeUuid, lvPath, pool);
+
+ return disk;
+ } catch (CloudRuntimeException ex) {
+ throw ex;
+ } catch (Exception ex) {
+ logger.error("Failed to access CLVM volume via direct block device: {}", volumeUuid, ex);
+ throw new CloudRuntimeException(String.format("Could not find volume %s: %s", volumeUuid, ex.getMessage()));
+ }
+ }
+
+ private void verifyLvExistsInVg(String volumeUuid, String vgName) {
+ logger.debug("Checking if volume {} exists in VG {}", volumeUuid, vgName);
+ Script checkLvCmd = new Script("/usr/sbin/lvs", 10000, logger);
+ checkLvCmd.add("--noheadings");
+ checkLvCmd.add("--unbuffered");
+ checkLvCmd.add(vgName + "/" + volumeUuid);
+ String checkResult = checkLvCmd.execute();
+ if (checkResult != null) {
+ throw new CloudRuntimeException(String.format("Storage volume not found: no storage vol with matching name '%s'", volumeUuid));
+ }
+ }
+
+ private String findAccessibleDeviceNode(String volumeUuid, String vgName, KVMStoragePool pool) {
+ String lvPath = "/dev/" + vgName + "/" + volumeUuid;
+ File lvDevice = new File(lvPath);
+
+ if (!lvDevice.exists()) {
+ lvPath = tryDeviceMapperPath(volumeUuid, vgName);
+ if (!new File(lvPath).exists()) {
+ lvPath = handleMissingDeviceNode(volumeUuid, vgName, pool);
+ }
+ }
+
+ return lvPath;
+ }
+
+ private String tryDeviceMapperPath(String volumeUuid, String vgName) {
+ String vgNameEscaped = vgName.replace("-", "--");
+ String volumeUuidEscaped = volumeUuid.replace("-", "--");
+ return "/dev/mapper/" + vgNameEscaped + "-" + volumeUuidEscaped;
+ }
+
+ private String handleMissingDeviceNode(String volumeUuid, String vgName, KVMStoragePool pool) {
+ if (StoragePoolType.CLVM_NG.equals(pool.getType()) && volumeUuid.startsWith("template-")) {
+ return activateTemplateAndGetPath(volumeUuid, vgName);
+ }
+ throw new CloudRuntimeException(String.format("Could not find volume %s in VG %s - volume exists in LVM but device node not accessible", volumeUuid, vgName));
+ }
+
+ private String activateTemplateAndGetPath(String volumeUuid, String vgName) {
+ logger.info("Template volume {} device node not found. Attempting to activate in shared mode.", volumeUuid);
+ String templateLvPath = "/dev/" + vgName + "/" + volumeUuid;
+
+ try {
+ ensureTemplateLvInSharedMode(templateLvPath, false);
+
+ String lvPath = findDeviceNodeAfterActivation(templateLvPath, volumeUuid, vgName);
+
+ logger.info("Successfully activated template volume {} at {}", volumeUuid, lvPath);
+ return lvPath;
+ } catch (CloudRuntimeException e) {
+ throw new CloudRuntimeException(String.format("Failed to activate template volume %s in VG %s: %s", volumeUuid, vgName, e.getMessage()), e);
+ }
+ }
+
+ private String findDeviceNodeAfterActivation(String templateLvPath, String volumeUuid, String vgName) {
+ File lvDevice = new File(templateLvPath);
+ String lvPath = templateLvPath;
+
+ if (!lvDevice.exists()) {
+ String vgNameEscaped = vgName.replace("-", "--");
+ String volumeUuidEscaped = volumeUuid.replace("-", "--");
+ lvPath = "/dev/mapper/" + vgNameEscaped + "-" + volumeUuidEscaped;
+ lvDevice = new File(lvPath);
+ }
+
+ if (!lvDevice.exists()) {
+ logger.error("Template volume {} still not accessible after activation attempt", volumeUuid);
+ throw new CloudRuntimeException(String.format("Could not activate template volume %s in VG %s - device node not accessible after activation", volumeUuid, vgName));
+ }
+
+ return lvPath;
+ }
+
+ private void ensureTemplateAccessibility(String volumeUuid, String lvPath, KVMStoragePool pool) {
+ if (StoragePoolType.CLVM_NG.equals(pool.getType()) && volumeUuid.startsWith("template-")) {
+ logger.info("Detected template volume {}. Ensuring it's activated in shared mode.", volumeUuid);
+ ensureTemplateLvInSharedMode(lvPath, false);
+ }
+ }
+
+ private long getClvmVolumeSize(String lvPath) {
+ try {
+ Script lvsCmd = new Script("/usr/sbin/lvs", 10000, logger);
+ lvsCmd.add("--noheadings");
+ lvsCmd.add("--units");
+ lvsCmd.add("b");
+ lvsCmd.add("-o");
+ lvsCmd.add("lv_size");
+ lvsCmd.add(lvPath);
+
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ String result = lvsCmd.execute(parser);
+
+ String output = (result == null) ? parser.getLines() : result;
+
+ if (output != null && !output.isEmpty()) {
+ String sizeStr = output.trim().replaceAll("[^0-9]", "");
+ if (!sizeStr.isEmpty()) {
+ return Long.parseLong(sizeStr);
+ }
+ }
+ } catch (Exception sizeEx) {
+ logger.warn("Failed to get size for CLVM volume via lvs: {}", sizeEx.getMessage());
+ File lvDevice = new File(lvPath);
+ if (lvDevice.isFile()) {
+ return lvDevice.length();
+ }
+ }
+ return 0;
+ }
+
+ private KVMPhysicalDisk createPhysicalDiskFromClvmLv(String lvPath, String volumeUuid, KVMStoragePool pool, long size) {
+ PhysicalDiskFormat diskFormat = StoragePoolType.CLVM_NG.equals(pool.getType())
+ ? PhysicalDiskFormat.QCOW2 : PhysicalDiskFormat.RAW;
+
+ logger.debug("{} pool detected, setting disk format to {} for volume {}", pool.getType(), diskFormat, volumeUuid);
+
+ KVMPhysicalDisk disk = new KVMPhysicalDisk(lvPath, volumeUuid, pool);
+ disk.setFormat(diskFormat);
+ disk.setSize(size);
+ disk.setVirtualSize(size);
+
+ logger.info("Successfully accessed CLVM/CLVM_NG volume via direct block device: {} with format: {} and size: {} bytes",
+ lvPath, diskFormat, size);
+ return disk;
+ }
+
+ /**
+ * Checks if a CLVM_NG QCOW2 volume has a backing file (template) and ensures it's activated in shared mode.
+ */
+ private void ensureClvmNgBackingFileAccessible(String volumeName, KVMStoragePool pool) {
+ try {
+ String vgName = getVgName(pool.getLocalPath());
+ String volumePath = "/dev/" + vgName + "/" + volumeName;
+
+ logger.debug("Checking if CLVM_NG volume {} has a backing file that needs activation", volumePath);
+
+ Script qemuImgInfo = new Script("qemu-img", Duration.millis(10000), logger);
+ qemuImgInfo.add("info");
+ qemuImgInfo.add("--output=json");
+ qemuImgInfo.add(volumePath);
+
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ String result = qemuImgInfo.execute(parser);
+
+ if (result == null && parser.getLines() != null && !parser.getLines().isEmpty()) {
+ String jsonOutput = parser.getLines();
+
+ if (jsonOutput.contains("\"backing-filename\"")) {
+ int backingStart = jsonOutput.indexOf("\"backing-filename\"");
+ if (backingStart > 0) {
+ int valueStart = jsonOutput.indexOf(":", backingStart);
+ if (valueStart > 0) {
+ valueStart = jsonOutput.indexOf("\"", valueStart) + 1;
+ int valueEnd = jsonOutput.indexOf("\"", valueStart);
+
+ if (valueEnd > valueStart) {
+ String backingFile = jsonOutput.substring(valueStart, valueEnd).trim();
+ if (!backingFile.isEmpty() && backingFile.startsWith("/dev/")) {
+ logger.info("Volume {} has backing file: {}. Ensuring backing file is in shared mode.", volumePath, backingFile);
+ ensureTemplateLvInSharedMode(backingFile, false);
+ }
+ }
+ }
+ }
+ } else {
+ logger.debug("Volume {} does not have a backing file (full clone)", volumePath);
+ }
+ }
+ } catch (Exception e) {
+ logger.warn("Failed to check/activate backing file for volume {}: {}. VM deployment may fail if template is not accessible.",
+ volumeName, e.getMessage());
+ }
+ }
+
+ private String getClvmBackingFile(KVMPhysicalDisk template, KVMStoragePool destPool) {
+ String templateLvName = template.getName();
+ KVMPhysicalDisk templateOnPrimary = null;
+
+ try {
+ templateOnPrimary = destPool.getPhysicalDisk(templateLvName);
+ } catch (CloudRuntimeException e) {
+ logger.warn("Template {} not found on CLVM_NG pool {}.", templateLvName, destPool.getUuid());
+ }
+
+ if (templateOnPrimary != null) {
+ String backingFile = templateOnPrimary.getPath();
+ logger.info("Using template on primary storage as backing file: {}", backingFile);
+ ensureTemplateLvInSharedMode(backingFile);
+ return backingFile;
+ }
+
+ logger.error("Template {} should be on primary storage before creating volumes from it", templateLvName);
+ throw new CloudRuntimeException(String.format("Template not found on CLVM_NG primary storage: %s. Template must be copied to primary storage first.", templateLvName));
+ }
+
+ /**
+ * Ensures a template LV is activated in shared mode so multiple VMs can use it as a backing file.
+ */
+ private void ensureTemplateLvInSharedMode(String templatePath, boolean throwOnFailure) {
+ try {
+ Script checkLvs = new Script("lvs", Duration.millis(10000), logger);
+ checkLvs.add("--noheadings");
+ checkLvs.add("-o", "lv_attr");
+ checkLvs.add(templatePath);
+
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ String result = checkLvs.execute(parser);
+
+ if (result == null && parser.getLines() != null && !parser.getLines().isEmpty()) {
+ String lvAttr = parser.getLines().trim();
+ if (lvAttr.length() >= 6) {
+ boolean isActive = (lvAttr.indexOf('a') >= 0);
+ boolean isShared = (lvAttr.indexOf('s') >= 0);
+
+ if (!isShared || !isActive) {
+ logger.info("Template LV {} is not in shared mode (attr: {}). Activating in shared mode.", templatePath, lvAttr);
+ LibvirtComputingResource.setClvmVolumeToSharedMode(templatePath);
+ } else {
+ logger.debug("Template LV {} is already in shared mode (attr: {})", templatePath, lvAttr);
+ }
+ }
+ }
+ } catch (CloudRuntimeException e) {
+ throw e;
+ } catch (Exception e) {
+ String errorMsg = "Failed to check/ensure template LV shared mode for " + templatePath + ": " + e.getMessage();
+ if (throwOnFailure) {
+ throw new CloudRuntimeException(errorMsg, e);
+ } else {
+ logger.warn(errorMsg, e);
+ }
+ }
+ }
+
+ private void ensureTemplateLvInSharedMode(String templatePath) {
+ ensureTemplateLvInSharedMode(templatePath, false);
+ }
+
+ private long getVgPhysicalExtentSize(String vgName) {
+ final long DEFAULT_PE_SIZE = 4 * 1024 * 1024L;
+ String warningMessage = String.format("Failed to get PE size for VG %s, defaulting to 4MiB", vgName);
+
+ try {
+ Script vgDisplay = new Script("vgdisplay", 300000, logger);
+ vgDisplay.add("--units", "b");
+ vgDisplay.add("-C");
+ vgDisplay.add("--noheadings");
+ vgDisplay.add("-o", "vg_extent_size");
+ vgDisplay.add(vgName);
+
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ String result = vgDisplay.execute(parser);
+
+ if (result != null) {
+ logger.warn("{}: {}", warningMessage, result);
+ return DEFAULT_PE_SIZE;
+ }
+
+ String output = parser.getLines();
+ if (output == null || output.trim().isEmpty()) {
+ logger.warn("{}: empty output", warningMessage);
+ return DEFAULT_PE_SIZE;
+ }
+
+ output = output.trim();
+ if (output.endsWith("B")) {
+ output = output.substring(0, output.length() - 1).trim();
+ }
+
+ long peSize = Long.parseLong(output);
+ logger.debug("Physical Extent size for VG {} is {} bytes", vgName, peSize);
+ return peSize;
+ } catch (NumberFormatException e) {
+ logger.warn("{}: failed to parse PE size", warningMessage, e);
+ } catch (Exception e) {
+ logger.warn("{}: {}", warningMessage, e.getMessage());
+ }
+
+ logger.info("Using default PE size for VG {}: {} bytes (4 MiB)", vgName, DEFAULT_PE_SIZE);
+ return DEFAULT_PE_SIZE;
+ }
+
+ /**
+ * Calculate LVM LV size for CLVM_NG volume allocation.
+ */
+ private long calculateClvmNgLvSize(long virtualSize, String vgName) {
+ long peSize = getVgPhysicalExtentSize(vgName);
+
+ long clusterSize = 64 * 1024L;
+ long l2Multiplier = 4096L;
+
+ long numDataClusters = (virtualSize + clusterSize - 1) / clusterSize;
+ long numL2Clusters = (numDataClusters + l2Multiplier - 1) / l2Multiplier;
+ long l2TableSize = numL2Clusters * clusterSize;
+ long refcountTableSize = l2TableSize;
+
+ long headerOverhead = 2 * 1024 * 1024L;
+ long metadataOverhead = l2TableSize + refcountTableSize + headerOverhead;
+ long targetSize = virtualSize + metadataOverhead;
+ long roundedSize = ((targetSize + peSize - 1) / peSize) * peSize;
+ long virtualSizeGiB = virtualSize / (1024 * 1024 * 1024L);
+ long overheadMiB = metadataOverhead / (1024 * 1024L);
+
+ logger.info("Calculated volume LV size: {} bytes (virtual: {} GiB, QCOW2 metadata overhead: {} MiB, rounded to {} PEs, PE size = {} bytes)",
+ roundedSize, virtualSizeGiB, overheadMiB, roundedSize / peSize, peSize);
+
+ return roundedSize;
+ }
+
+ private long getQcow2VirtualSize(String imagePath) {
+ Script qemuImg = new Script("qemu-img", 300000, logger);
+ qemuImg.add("info");
+ qemuImg.add("--output=json");
+ qemuImg.add(imagePath);
Review Comment:
QemuImg class
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java:
##########
@@ -1920,6 +2131,15 @@ private SnapshotObjectTO takeIncrementalVolumeSnapshotOfStoppedVm(SnapshotObject
logger.debug("Taking incremental volume snapshot of volume [{}]. Snapshot will be copied to [{}].", volumeObjectTo,
ObjectUtils.defaultIfNull(secondaryPool, primaryPool));
try {
+ // For CLVM_NG incremental snapshots, validate bitmap before proceeding
+ /*
+ SnapshotObjectTO bitmapValidationResult = validateClvmNgBitmapAndFallbackIfNeeded(snapshotObjectTO, primaryPool,
+ secondaryPool, secondaryPoolUrl, snapshotName, volumeObjectTo, conn, wait);
+ if (bitmapValidationResult != null) {
+ return bitmapValidationResult;
+ }
+ */
Review Comment:
please remove if it will not be implemented
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/LibvirtStorageAdaptor.java:
##########
@@ -1240,6 +1241,8 @@ public boolean deletePhysicalDisk(String uuid, KVMStoragePool pool, Storage.Imag
}
}
+
Review Comment:
same
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/LibvirtStorageAdaptor.java:
##########
@@ -1338,6 +1341,9 @@ public KVMPhysicalDisk createDiskFromTemplate(KVMPhysicalDisk template,
return disk;
}
+
+
Review Comment:
same
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java:
##########
@@ -2123,6 +2347,209 @@ protected String getParentCheckpointName(String[] parents) {
return immediateParentPath.substring(immediateParentPath.lastIndexOf(File.separator) + 1);
}
+ /**
+ * Checks if a QEMU bitmap exists in the volume and is usable (not in-use or corrupted).
+ * This is important after lock migration where bitmaps may be left in "in-use" state.
+ *
+ * @param pool The storage pool containing the volume
+ * @param volume The volume to check
+ * @param bitmapName The name of the bitmap to check
+ * @return true if bitmap exists and is usable, false if missing, in-use, or corrupted
+ */
+ protected boolean isBitmapUsable(KVMStoragePool pool, VolumeObjectTO volume, String bitmapName) {
+ try {
+ String volumePath = pool.getLocalPathFor(volume.getPath());
+
+ String command = String.format("qemu-img info --output=json -U %s", volumePath);
+ String jsonOutput = Script.runSimpleBashScriptWithFullResult(command, 30);
+
+ if (jsonOutput == null || jsonOutput.trim().isEmpty()) {
+ logger.warn("Failed to get qemu-img info for volume [{}]", volumePath);
+ return false;
+ }
+
+ logger.debug("qemu-img info output for volume [{}]: {}", volumePath, jsonOutput);
+
+ try {
+ ObjectMapper mapper = new ObjectMapper();
+ JsonNode root = mapper.readTree(jsonOutput);
+
+ JsonNode formatSpecific = root.path("format-specific");
+ if (formatSpecific.isMissingNode()) {
+ logger.debug("No format-specific data found for volume [{}], bitmap check skipped", volumePath);
+ return false;
+ }
+
+ JsonNode data = formatSpecific.path("data");
+ JsonNode bitmaps = data.path("bitmaps");
+
+ if (bitmaps.isMissingNode() || !bitmaps.isArray()) {
+ logger.debug("No bitmaps found in volume [{}]", volumePath);
+ return false;
+ }
+
+ for (JsonNode bitmap : bitmaps) {
+ String name = bitmap.path("name").asText();
+ if (bitmapName.equals(name)) {
+ JsonNode flags = bitmap.path("flags");
+ if (flags.isArray()) {
+ for (JsonNode flag : flags) {
+ String flagValue = flag.asText();
+ if ("in-use".equals(flagValue)) {
+ logger.warn("Bitmap [{}] in volume [{}] is marked as 'in-use' and cannot be used for incremental snapshot",
+ bitmapName, volumePath);
+ return false;
+ }
+ }
+ }
+ logger.debug("Bitmap [{}] found in volume [{}] and is usable", bitmapName, volumePath);
+ return true;
+ }
+ }
+
+ logger.warn("Bitmap [{}] not found in volume [{}]", bitmapName, volumePath);
+ return false;
+
+ } catch (JsonProcessingException e) {
+ logger.error("Failed to parse qemu-img JSON output for volume [{}]: {}", volumePath, e.getMessage(), e);
+ return false;
+ }
+
+ } catch (Exception e) {
+ logger.error("Error checking bitmap [{}] for volume [{}]: {}", bitmapName, volume, e.getMessage(), e);
+ return false;
+ }
+ }
+
+ /**
+ * Removes a broken or unusable bitmap from a volume.
+ * Called before falling back to full snapshot to keep volume metadata clean.
+ *
+ * @param pool Storage pool containing the volume
+ * @param volume Volume with broken bitmap
+ * @param bitmapName Name of bitmap to remove
+ */
+ private void cleanupBrokenBitmap(KVMStoragePool pool, VolumeObjectTO volume, String bitmapName) {
+ try {
+ String volumePath = pool.getLocalPathFor(volume.getPath());
+
+ logger.info("Removing broken bitmap [{}] from volume [{}] before taking full snapshot",
+ bitmapName, volumePath);
+
+ QemuImgFile volumeFile = new QemuImgFile(volumePath, PhysicalDiskFormat.QCOW2);
+ QemuImg qemuImg = new QemuImg(0);
+
+ try {
+ qemuImg.bitmap(QemuImg.BitmapOperation.Remove, volumeFile, bitmapName);
+ logger.info("Successfully removed broken bitmap [{}] from volume [{}]",
+ bitmapName, volume.getPath());
+ } catch (QemuImgException e) {
+ logger.warn("Failed to remove broken bitmap [{}] from volume [{}]: {}. " +
+ "Proceeding with fallback anyway.",
+ bitmapName, volume.getPath(), e.getMessage());
+ }
+
+ } catch (Exception e) {
+ logger.warn("Exception while cleaning up broken bitmap [{}] for volume [{}]: {}. " +
+ "Proceeding with fallback anyway.",
+ bitmapName, volume.getPath(), e.getMessage());
+ }
+ }
+
+ /**
+ * Validates QEMU bitmap for CLVM_NG incremental snapshots and falls back to full snapshot if needed.
+ * This method checks if the bitmap from the parent checkpoint is usable. If the bitmap is corrupted,
+ * in-use, or missing, it cleans up the broken bitmap and falls back to taking a full snapshot with
+ * a new checkpoint to restart the incremental chain.
+ *
+ * @param snapshotObjectTO Snapshot being created
+ * @param primaryPool Primary storage pool
+ * @param secondaryPool Secondary storage pool for backup
+ * @param secondaryPoolUrl Secondary pool URL
+ * @param snapshotName Name of the snapshot
+ * @param volumeObjectTo Volume being snapshotted
+ * @param conn Libvirt connection
+ * @param wait Timeout for operations
+ * @return SnapshotObjectTO if fallback to full snapshot occurred, null if validation passed
+ * @throws LibvirtException if libvirt operations fail
+ */
+ protected SnapshotObjectTO validateClvmNgBitmapAndFallbackIfNeeded(SnapshotObjectTO snapshotObjectTO,
Review Comment:
this is not called anywhere, we should remove it.
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/ClvmStorageAdaptor.java:
##########
@@ -0,0 +1,1072 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.hypervisor.kvm.storage;
+
+import static com.cloud.utils.NumbersUtil.toHumanReadableSize;
+
+import java.io.File;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.cloud.hypervisor.kvm.resource.LibvirtComputingResource;
+import com.cloud.hypervisor.kvm.resource.LibvirtConnection;
+import com.cloud.hypervisor.kvm.resource.LibvirtStoragePoolDef;
+import com.cloud.hypervisor.kvm.resource.LibvirtStorageVolumeDef;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Storage.StoragePoolType;
+import com.cloud.storage.StorageLayer;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.script.OutputInterpreter;
+import com.cloud.utils.script.Script;
+import com.google.gson.JsonObject;
+import com.google.gson.JsonParser;
+import org.apache.cloudstack.utils.qemu.QemuImg.PhysicalDiskFormat;
+import org.joda.time.Duration;
+import org.libvirt.Connect;
+import org.libvirt.LibvirtException;
+import org.libvirt.StoragePool;
+import org.libvirt.StorageVol;
+
+/**
+ * Storage adaptor for CLVM and CLVM_NG pool types.
+ * Extends LibvirtStorageAdaptor and overrides methods with CLVM-specific logic,
+ * using direct LVM commands instead of libvirt for volume operations.
+ */
+public class ClvmStorageAdaptor extends LibvirtStorageAdaptor {
+
+ public ClvmStorageAdaptor(StorageLayer storage) {
+ super(storage);
+ }
+
+ @Override
+ public StoragePoolType getStoragePoolType() {
+ // Registered manually for both CLVM and CLVM_NG in KVMStoragePoolManager
+ return null;
+ }
+
+ @Override
+ public KVMStoragePool createStoragePool(String name, String host, int port, String path,
+ String userInfo, StoragePoolType type, Map<String, String> details, boolean isPrimaryStorage) {
+ logger.info("Attempting to create CLVM/CLVM_NG storage pool {} in libvirt", name);
+
+ Connect conn;
+ try {
+ conn = LibvirtConnection.getConnection();
+ } catch (LibvirtException e) {
+ throw new CloudRuntimeException(e.toString());
+ }
+
+ StoragePool sp = createCLVMStoragePool(conn, name, host, path);
+ if (sp == null) {
+ logger.info("Falling back to virtual CLVM/CLVM_NG pool without libvirt for: {}", name);
+ return createVirtualClvmPool(name, host, path, type, details);
+ }
+
+ try {
+ if (!isPrimaryStorage) {
+ incStoragePoolRefCount(name);
+ }
+ // CLVM/CLVM_NG pools are kept inactive in libvirt; we use direct LVM commands
+ return getStoragePool(name);
+ } catch (Exception e) {
+ decStoragePoolRefCount(name);
+ throw new CloudRuntimeException("Failed to create CLVM storage pool: " + name, e);
+ }
+ }
+
+ @Override
+ public KVMStoragePool getStoragePool(String uuid, boolean refreshInfo) {
+ logger.info("Fetching CLVM/CLVM_NG storage pool {} ", uuid);
+ try {
+ Connect conn = LibvirtConnection.getConnection();
+ StoragePool storage = conn.storagePoolLookupByUUIDString(uuid);
+
+ LibvirtStoragePoolDef spd = getStoragePoolDef(conn, storage);
+ if (spd == null) {
+ throw new CloudRuntimeException("Unable to parse storage pool definition for pool " + uuid);
+ }
+
+ // CLVM pools in libvirt are always LOGICAL type
+ StoragePoolType type = StoragePoolType.CLVM;
+
+ // Do NOT activate the pool — CLVM/CLVM_NG pools stay inactive in libvirt
+ LibvirtStoragePool pool = new LibvirtStoragePool(uuid, storage.getName(), type, this, storage);
+ pool.setLocalPath(spd.getTargetPath());
+
+ // Always read capacity from LVM directly
+ String vgName = storage.getName();
+ try {
+ long[] vgStats = getVgStats(vgName);
+ setPoolCapacityFromVgStats(pool, vgStats, vgName);
+ } catch (CloudRuntimeException e) {
+ logger.warn("Failed to get VG stats for CLVM/CLVM_NG pool {}: {}. Using libvirt values (may be 0)", vgName, e.getMessage());
+ pool.setCapacity(storage.getInfo().capacity);
+ pool.setUsed(storage.getInfo().allocation);
+ pool.setAvailable(storage.getInfo().available);
+ }
+
+ return pool;
+ } catch (LibvirtException e) {
+ // Pool not in libvirt - return virtual pool backed only by LVM
+ logger.debug("CLVM/CLVM_NG pool {} not found in libvirt, creating virtual pool", uuid);
+ throw new CloudRuntimeException(e.toString(), e);
+ }
+ }
+
+ @Override
+ public KVMPhysicalDisk getPhysicalDisk(String volumeUuid, KVMStoragePool pool) {
+ LibvirtStoragePool libvirtPool = (LibvirtStoragePool) pool;
+
+ // Pool has no libvirt backing - go directly to block device
+ if (libvirtPool.getPool() == null) {
+ logger.debug("CLVM/CLVM_NG pool has no libvirt backing, using direct block device access for volume: {}", volumeUuid);
+ return getPhysicalDiskViaDirectBlockDevice(volumeUuid, pool);
+ }
+
+ try {
+ StorageVol vol = getVolume(libvirtPool.getPool(), volumeUuid);
+ if (vol == null) {
+ logger.debug("Volume {} not found in libvirt, falling back to CLVM direct access", volumeUuid);
+ return getPhysicalDiskWithClvmFallback(volumeUuid, pool, libvirtPool);
+ }
+
+ LibvirtStorageVolumeDef voldef = getStorageVolumeDef(libvirtPool.getPool().getConnect(), vol);
+ KVMPhysicalDisk disk = new KVMPhysicalDisk(vol.getPath(), vol.getName(), pool);
+ disk.setSize(vol.getInfo().allocation);
+ disk.setVirtualSize(vol.getInfo().capacity);
+ disk.setFormat(voldef.getFormat() == LibvirtStorageVolumeDef.VolumeFormat.QCOW2
+ ? PhysicalDiskFormat.QCOW2 : PhysicalDiskFormat.RAW);
+ return disk;
+ } catch (LibvirtException e) {
+ logger.warn("LibvirtException looking up volume {}: {}", volumeUuid, e.getMessage());
+ return getPhysicalDiskWithClvmFallback(volumeUuid, pool, libvirtPool);
+ }
+ }
+
+ @Override
+ public KVMPhysicalDisk createPhysicalDisk(String name, KVMStoragePool pool,
+ PhysicalDiskFormat format, Storage.ProvisioningType provisioningType, long size, byte[] passphrase) {
+ logger.info("Creating CLVM/CLVM_NG volume {} in pool {} with size {}", name, pool.getUuid(), toHumanReadableSize(size));
+
+ if (StoragePoolType.CLVM_NG.equals(pool.getType())) {
+ return createClvmNgDiskWithBacking(name, 0, size, null, pool, provisioningType);
+ } else {
+ return createClvmVolume(name, size, pool);
+ }
+ }
+
+ @Override
+ public boolean connectPhysicalDisk(String name, KVMStoragePool pool, Map<String, String> details, boolean isVMMigrate) {
+ if (isVMMigrate) {
+ logger.info("Activating CLVM/CLVM_NG volume {} in shared mode for VM migration", name);
+ Script activateVol = new Script("lvchange", 10000, logger);
+ activateVol.add("-asy");
+ activateVol.add(pool.getLocalPath() + File.separator + name);
+ String result = activateVol.execute();
+ if (result != null) {
+ logger.error("Failed to activate CLVM/CLVM_NG volume {} in shared mode. Output: {}", name, result);
+ return false;
+ }
+ }
+
+ if (StoragePoolType.CLVM_NG.equals(pool.getType())) {
+ ensureClvmNgBackingFileAccessible(name, pool);
+ }
+
+ return true;
+ }
+
+ @Override
+ public KVMPhysicalDisk createDiskFromTemplate(KVMPhysicalDisk template,
+ String name, PhysicalDiskFormat format, Storage.ProvisioningType provisioningType,
+ long size, KVMStoragePool destPool, int timeout, byte[] passphrase) {
+
+ if (StoragePoolType.CLVM_NG.equals(destPool.getType()) && format == PhysicalDiskFormat.QCOW2) {
+ logger.info("Creating CLVM_NG volume {} with backing file from template {}", name, template.getName());
+ String backingFile = getClvmBackingFile(template, destPool);
+ return createClvmNgDiskWithBacking(name, timeout, size, backingFile, destPool, provisioningType);
+ }
+
+ return super.createDiskFromTemplate(template, name, format, provisioningType, size, destPool, timeout, passphrase);
+ }
+
+ @Override
+ public void createTemplate(String templatePath, String templateUuid, int timeout, KVMStoragePool pool) {
+ String vgName = getVgName(pool.getLocalPath());
+ String lvName = "template-" + templateUuid;
+ String lvPath = "/dev/" + vgName + "/" + lvName;
+
+ if (lvExists(lvPath)) {
+ logger.info("Template LV {} already exists in VG {}. Skipping creation.", lvName, vgName);
+ return;
+ }
+
+ logger.info("Creating new template LV {} in VG {} for template {}", lvName, vgName, templateUuid);
+
+ long virtualSize = getQcow2VirtualSize(templatePath);
+ long physicalSize = getQcow2PhysicalSize(templatePath);
+ long lvSize = virtualSize;
+
+ logger.info("Template source - Physical: {} bytes, Virtual: {} bytes, LV will be: {} bytes", physicalSize, virtualSize, lvSize);
+
+ Script lvcreate = new Script("lvcreate", Duration.millis(timeout), logger);
+ lvcreate.add("-n", lvName);
+ lvcreate.add("-L", lvSize + "B");
+ lvcreate.add("--yes");
+ lvcreate.add(vgName);
+ String result = lvcreate.execute();
+ if (result != null) {
+ throw new CloudRuntimeException("Failed to create LV for CLVM_NG template: " + result);
+ }
+
+ Script qemuImgConvert = new Script("qemu-img", Duration.millis(timeout), logger);
+ qemuImgConvert.add("convert");
+ qemuImgConvert.add(templatePath);
+ qemuImgConvert.add("-O", "qcow2");
+ qemuImgConvert.add("-o", "cluster_size=64k,extended_l2=off,preallocation=off");
+ qemuImgConvert.add(lvPath);
+ result = qemuImgConvert.execute();
+
+ if (result != null) {
+ removeLvOnFailure(lvPath, timeout);
+ throw new CloudRuntimeException("Failed to convert template to CLVM_NG volume: " + result);
+ }
+
+ long actualVirtualSize = getQcow2VirtualSize(lvPath);
+
+ try {
+ ensureTemplateLvInSharedMode(lvPath, true);
+ } catch (CloudRuntimeException e) {
+ logger.error("Failed to activate template LV {} in shared mode. Cleaning up.", lvPath);
+ removeLvOnFailure(lvPath, timeout);
+ throw e;
+ }
+
+ KVMPhysicalDisk templateDisk = new KVMPhysicalDisk(lvPath, lvName, pool);
+ templateDisk.setFormat(PhysicalDiskFormat.QCOW2);
+ templateDisk.setVirtualSize(actualVirtualSize);
+ templateDisk.setSize(lvSize);
+ }
+
+ private StoragePool createCLVMStoragePool(Connect conn, String uuid, String host, String path) {
+ String volgroupPath = "/dev/" + path;
+ String volgroupName = path;
+ volgroupName = volgroupName.replaceFirst("^/", "");
+
+ Script checkVgExists = new Script("vgs", 10000, logger);
+ checkVgExists.add("--noheadings");
+ checkVgExists.add("-o", "vg_name");
+ checkVgExists.add(volgroupName);
+ String vgCheckResult = checkVgExists.execute();
+
+ if (vgCheckResult != null) {
+ logger.error("Volume group {} does not exist or is not accessible", volgroupName);
+ return null;
+ }
+
+ logger.info("Volume group {} verified, creating libvirt pool definition for CLVM/CLVM_NG", volgroupName);
+ LibvirtStoragePoolDef poolDef = new LibvirtStoragePoolDef(
+ LibvirtStoragePoolDef.PoolType.LOGICAL,
+ volgroupName,
+ uuid,
+ null,
+ volgroupName,
+ volgroupPath
+ );
+
+ try {
+ StoragePool pool = conn.storagePoolDefineXML(poolDef.toString(), 0);
+ logger.info("Created libvirt pool definition for CLVM/CLVM_NG VG: {} (pool will remain inactive)", volgroupName);
+ pool.setAutostart(1);
+ return pool;
+ } catch (LibvirtException e) {
+ logger.warn("Failed to define CLVM/CLVM_NG pool in libvirt: {}", e.getMessage());
+ return null;
+ }
+ }
+
+ private void setPoolCapacityFromVgStats(LibvirtStoragePool pool, long[] vgStats, String vgName) {
+ long capacity = vgStats[0];
+ long available = vgStats[1];
+ long used = capacity - available;
+
+ pool.setCapacity(capacity);
+ pool.setAvailable(available);
+ pool.setUsed(used);
+
+ logger.debug("CLVM/CLVM_NG pool {} - Capacity: {}, Used: {}, Available: {}",
+ vgName, toHumanReadableSize(capacity), toHumanReadableSize(used), toHumanReadableSize(available));
+ }
+
+ private long[] getVgStats(String vgName) {
+ Script getVgStats = new Script("vgs", 10000, logger);
+ getVgStats.add("--noheadings");
+ getVgStats.add("--units", "b");
+ getVgStats.add("--nosuffix");
+ getVgStats.add("-o", "vg_size,vg_free");
+ getVgStats.add(vgName);
+
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ String result = getVgStats.execute(parser);
+
+ if (result != null) {
+ String errorMsg = "Failed to get statistics for volume group " + vgName + ": " + result;
+ logger.error(errorMsg);
+ throw new CloudRuntimeException(errorMsg);
+ }
+
+ String output = parser.getLines().trim();
+ String[] lines = output.split("\\n");
+ String dataLine = null;
+
+ for (String line : lines) {
+ line = line.trim();
+ if (!line.isEmpty() && Character.isDigit(line.charAt(0))) {
+ dataLine = line;
+ break;
+ }
+ }
+
+ if (dataLine == null) {
+ String errorMsg = "No valid data line found in vgs output for " + vgName + ": " + output;
+ logger.error(errorMsg);
+ throw new CloudRuntimeException(errorMsg);
+ }
+
+ String[] stats = dataLine.split("\\s+");
+
+ if (stats.length < 2) {
+ String errorMsg = "Unexpected output from vgs command for " + vgName + ": " + dataLine;
+ logger.error(errorMsg);
+ throw new CloudRuntimeException(errorMsg);
+ }
+
+ try {
+ long capacity = Long.parseLong(stats[0].trim());
+ long available = Long.parseLong(stats[1].trim());
+ return new long[]{capacity, available};
+ } catch (NumberFormatException e) {
+ String errorMsg = "Failed to parse VG statistics for " + vgName + ": " + e.getMessage();
+ logger.error(errorMsg);
+ throw new CloudRuntimeException(errorMsg, e);
+ }
+ }
+
+ private KVMStoragePool createVirtualClvmPool(String uuid, String host, String path, StoragePoolType type, Map<String, String> details) {
+ String volgroupName = path.replaceFirst("^/", "");
+ String volgroupPath = "/dev/" + volgroupName;
+
+ logger.info("Creating virtual CLVM/CLVM_NG pool {} without libvirt using direct LVM access", volgroupName);
+
+ long[] vgStats = getVgStats(volgroupName);
+
+ LibvirtStoragePool pool = new LibvirtStoragePool(uuid, volgroupName, type, this, null);
+ pool.setLocalPath(volgroupPath);
+ setPoolCapacityFromVgStats(pool, vgStats, volgroupName);
+
+ if (details != null) {
+ pool.setDetails(details);
+ }
+
+ return pool;
+ }
+
+ /**
+ * CLVM fallback: First tries to refresh libvirt pool to make volume visible,
+ * if that fails, accesses volume directly via block device path.
+ */
+ private KVMPhysicalDisk getPhysicalDiskWithClvmFallback(String volumeUuid, KVMStoragePool pool, LibvirtStoragePool libvirtPool) {
+ logger.info("CLVM volume not visible to libvirt, attempting pool refresh for volume: {}", volumeUuid);
+
+ try {
+ logger.debug("Refreshing libvirt storage pool: {}", pool.getUuid());
+ libvirtPool.getPool().refresh(0);
+
+ StorageVol vol = getVolume(libvirtPool.getPool(), volumeUuid);
+ if (vol != null) {
+ logger.info("Volume found after pool refresh: {}", volumeUuid);
+ LibvirtStorageVolumeDef voldef = getStorageVolumeDef(libvirtPool.getPool().getConnect(), vol);
+ KVMPhysicalDisk disk = new KVMPhysicalDisk(vol.getPath(), vol.getName(), pool);
+ disk.setSize(vol.getInfo().allocation);
+ disk.setVirtualSize(vol.getInfo().capacity);
+ disk.setFormat(voldef.getFormat() == LibvirtStorageVolumeDef.VolumeFormat.QCOW2
+ ? PhysicalDiskFormat.QCOW2 : PhysicalDiskFormat.RAW);
+ return disk;
+ }
+ } catch (LibvirtException refreshEx) {
+ logger.debug("Pool refresh failed or volume still not found: {}", refreshEx.getMessage());
+ }
+
+ logger.info("Falling back to direct block device access for volume: {}", volumeUuid);
+ return getPhysicalDiskViaDirectBlockDevice(volumeUuid, pool);
+ }
+
+ private String getVgName(String sourceDir) {
+ String vgName = sourceDir;
+ if (vgName.startsWith("/")) {
+ String[] parts = vgName.split("/");
+ List<String> tokens = Arrays.stream(parts)
+ .filter(s -> !s.isEmpty()).collect(Collectors.toList());
+
+ vgName = tokens.size() > 1 ? tokens.get(1)
+ : tokens.size() == 1 ? tokens.get(0)
+ : "";
+ }
+ return vgName;
+ }
+
+ private String extractVgNameFromPool(KVMStoragePool pool) {
+ String sourceDir = pool.getLocalPath();
+ if (sourceDir == null || sourceDir.isEmpty()) {
+ throw new CloudRuntimeException("CLVM pool sourceDir is not set, cannot determine VG name");
+ }
+ String vgName = getVgName(sourceDir);
+ logger.debug("Using VG name: {} (from sourceDir: {})", vgName, sourceDir);
+ return vgName;
+ }
+
+ /**
+ * For CLVM volumes that exist in LVM but are not visible to libvirt,
+ * access them directly via block device path.
+ */
+ private KVMPhysicalDisk getPhysicalDiskViaDirectBlockDevice(String volumeUuid, KVMStoragePool pool) {
+ try {
+ String vgName = extractVgNameFromPool(pool);
+
+ verifyLvExistsInVg(volumeUuid, vgName);
+
+ logger.info("Volume {} exists in LVM but not visible to libvirt, accessing directly", volumeUuid);
+
+ String lvPath = findAccessibleDeviceNode(volumeUuid, vgName, pool);
+ long size = getClvmVolumeSize(lvPath);
+
+ KVMPhysicalDisk disk = createPhysicalDiskFromClvmLv(lvPath, volumeUuid, pool, size);
+ ensureTemplateAccessibility(volumeUuid, lvPath, pool);
+
+ return disk;
+ } catch (CloudRuntimeException ex) {
+ throw ex;
+ } catch (Exception ex) {
+ logger.error("Failed to access CLVM volume via direct block device: {}", volumeUuid, ex);
+ throw new CloudRuntimeException(String.format("Could not find volume %s: %s", volumeUuid, ex.getMessage()));
+ }
+ }
+
+ private void verifyLvExistsInVg(String volumeUuid, String vgName) {
+ logger.debug("Checking if volume {} exists in VG {}", volumeUuid, vgName);
+ Script checkLvCmd = new Script("/usr/sbin/lvs", 10000, logger);
+ checkLvCmd.add("--noheadings");
+ checkLvCmd.add("--unbuffered");
+ checkLvCmd.add(vgName + "/" + volumeUuid);
+ String checkResult = checkLvCmd.execute();
+ if (checkResult != null) {
+ throw new CloudRuntimeException(String.format("Storage volume not found: no storage vol with matching name '%s'", volumeUuid));
+ }
+ }
+
+ private String findAccessibleDeviceNode(String volumeUuid, String vgName, KVMStoragePool pool) {
+ String lvPath = "/dev/" + vgName + "/" + volumeUuid;
+ File lvDevice = new File(lvPath);
+
+ if (!lvDevice.exists()) {
+ lvPath = tryDeviceMapperPath(volumeUuid, vgName);
+ if (!new File(lvPath).exists()) {
+ lvPath = handleMissingDeviceNode(volumeUuid, vgName, pool);
+ }
+ }
+
+ return lvPath;
+ }
+
+ private String tryDeviceMapperPath(String volumeUuid, String vgName) {
+ String vgNameEscaped = vgName.replace("-", "--");
+ String volumeUuidEscaped = volumeUuid.replace("-", "--");
+ return "/dev/mapper/" + vgNameEscaped + "-" + volumeUuidEscaped;
+ }
+
+ private String handleMissingDeviceNode(String volumeUuid, String vgName, KVMStoragePool pool) {
+ if (StoragePoolType.CLVM_NG.equals(pool.getType()) && volumeUuid.startsWith("template-")) {
+ return activateTemplateAndGetPath(volumeUuid, vgName);
+ }
+ throw new CloudRuntimeException(String.format("Could not find volume %s in VG %s - volume exists in LVM but device node not accessible", volumeUuid, vgName));
+ }
+
+ private String activateTemplateAndGetPath(String volumeUuid, String vgName) {
+ logger.info("Template volume {} device node not found. Attempting to activate in shared mode.", volumeUuid);
+ String templateLvPath = "/dev/" + vgName + "/" + volumeUuid;
+
+ try {
+ ensureTemplateLvInSharedMode(templateLvPath, false);
+
+ String lvPath = findDeviceNodeAfterActivation(templateLvPath, volumeUuid, vgName);
+
+ logger.info("Successfully activated template volume {} at {}", volumeUuid, lvPath);
+ return lvPath;
+ } catch (CloudRuntimeException e) {
+ throw new CloudRuntimeException(String.format("Failed to activate template volume %s in VG %s: %s", volumeUuid, vgName, e.getMessage()), e);
+ }
+ }
+
+ private String findDeviceNodeAfterActivation(String templateLvPath, String volumeUuid, String vgName) {
+ File lvDevice = new File(templateLvPath);
+ String lvPath = templateLvPath;
+
+ if (!lvDevice.exists()) {
+ String vgNameEscaped = vgName.replace("-", "--");
+ String volumeUuidEscaped = volumeUuid.replace("-", "--");
+ lvPath = "/dev/mapper/" + vgNameEscaped + "-" + volumeUuidEscaped;
+ lvDevice = new File(lvPath);
+ }
+
+ if (!lvDevice.exists()) {
+ logger.error("Template volume {} still not accessible after activation attempt", volumeUuid);
+ throw new CloudRuntimeException(String.format("Could not activate template volume %s in VG %s - device node not accessible after activation", volumeUuid, vgName));
+ }
+
+ return lvPath;
+ }
+
+ private void ensureTemplateAccessibility(String volumeUuid, String lvPath, KVMStoragePool pool) {
+ if (StoragePoolType.CLVM_NG.equals(pool.getType()) && volumeUuid.startsWith("template-")) {
+ logger.info("Detected template volume {}. Ensuring it's activated in shared mode.", volumeUuid);
+ ensureTemplateLvInSharedMode(lvPath, false);
+ }
+ }
+
+ private long getClvmVolumeSize(String lvPath) {
+ try {
+ Script lvsCmd = new Script("/usr/sbin/lvs", 10000, logger);
+ lvsCmd.add("--noheadings");
+ lvsCmd.add("--units");
+ lvsCmd.add("b");
+ lvsCmd.add("-o");
+ lvsCmd.add("lv_size");
+ lvsCmd.add(lvPath);
+
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ String result = lvsCmd.execute(parser);
+
+ String output = (result == null) ? parser.getLines() : result;
+
+ if (output != null && !output.isEmpty()) {
+ String sizeStr = output.trim().replaceAll("[^0-9]", "");
+ if (!sizeStr.isEmpty()) {
+ return Long.parseLong(sizeStr);
+ }
+ }
+ } catch (Exception sizeEx) {
+ logger.warn("Failed to get size for CLVM volume via lvs: {}", sizeEx.getMessage());
+ File lvDevice = new File(lvPath);
+ if (lvDevice.isFile()) {
+ return lvDevice.length();
+ }
+ }
+ return 0;
+ }
+
+ private KVMPhysicalDisk createPhysicalDiskFromClvmLv(String lvPath, String volumeUuid, KVMStoragePool pool, long size) {
+ PhysicalDiskFormat diskFormat = StoragePoolType.CLVM_NG.equals(pool.getType())
+ ? PhysicalDiskFormat.QCOW2 : PhysicalDiskFormat.RAW;
+
+ logger.debug("{} pool detected, setting disk format to {} for volume {}", pool.getType(), diskFormat, volumeUuid);
+
+ KVMPhysicalDisk disk = new KVMPhysicalDisk(lvPath, volumeUuid, pool);
+ disk.setFormat(diskFormat);
+ disk.setSize(size);
+ disk.setVirtualSize(size);
+
+ logger.info("Successfully accessed CLVM/CLVM_NG volume via direct block device: {} with format: {} and size: {} bytes",
+ lvPath, diskFormat, size);
+ return disk;
+ }
+
+ /**
+ * Checks if a CLVM_NG QCOW2 volume has a backing file (template) and ensures it's activated in shared mode.
+ */
+ private void ensureClvmNgBackingFileAccessible(String volumeName, KVMStoragePool pool) {
+ try {
+ String vgName = getVgName(pool.getLocalPath());
+ String volumePath = "/dev/" + vgName + "/" + volumeName;
+
+ logger.debug("Checking if CLVM_NG volume {} has a backing file that needs activation", volumePath);
+
+ Script qemuImgInfo = new Script("qemu-img", Duration.millis(10000), logger);
+ qemuImgInfo.add("info");
+ qemuImgInfo.add("--output=json");
+ qemuImgInfo.add(volumePath);
+
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ String result = qemuImgInfo.execute(parser);
+
+ if (result == null && parser.getLines() != null && !parser.getLines().isEmpty()) {
+ String jsonOutput = parser.getLines();
+
+ if (jsonOutput.contains("\"backing-filename\"")) {
+ int backingStart = jsonOutput.indexOf("\"backing-filename\"");
+ if (backingStart > 0) {
+ int valueStart = jsonOutput.indexOf(":", backingStart);
+ if (valueStart > 0) {
+ valueStart = jsonOutput.indexOf("\"", valueStart) + 1;
+ int valueEnd = jsonOutput.indexOf("\"", valueStart);
+
+ if (valueEnd > valueStart) {
+ String backingFile = jsonOutput.substring(valueStart, valueEnd).trim();
+ if (!backingFile.isEmpty() && backingFile.startsWith("/dev/")) {
+ logger.info("Volume {} has backing file: {}. Ensuring backing file is in shared mode.", volumePath, backingFile);
+ ensureTemplateLvInSharedMode(backingFile, false);
+ }
+ }
+ }
+ }
+ } else {
+ logger.debug("Volume {} does not have a backing file (full clone)", volumePath);
+ }
+ }
+ } catch (Exception e) {
+ logger.warn("Failed to check/activate backing file for volume {}: {}. VM deployment may fail if template is not accessible.",
+ volumeName, e.getMessage());
+ }
+ }
+
+ private String getClvmBackingFile(KVMPhysicalDisk template, KVMStoragePool destPool) {
+ String templateLvName = template.getName();
+ KVMPhysicalDisk templateOnPrimary = null;
+
+ try {
+ templateOnPrimary = destPool.getPhysicalDisk(templateLvName);
+ } catch (CloudRuntimeException e) {
+ logger.warn("Template {} not found on CLVM_NG pool {}.", templateLvName, destPool.getUuid());
+ }
+
+ if (templateOnPrimary != null) {
+ String backingFile = templateOnPrimary.getPath();
+ logger.info("Using template on primary storage as backing file: {}", backingFile);
+ ensureTemplateLvInSharedMode(backingFile);
+ return backingFile;
+ }
+
+ logger.error("Template {} should be on primary storage before creating volumes from it", templateLvName);
+ throw new CloudRuntimeException(String.format("Template not found on CLVM_NG primary storage: %s. Template must be copied to primary storage first.", templateLvName));
+ }
+
+ /**
+ * Ensures a template LV is activated in shared mode so multiple VMs can use it as a backing file.
+ */
+ private void ensureTemplateLvInSharedMode(String templatePath, boolean throwOnFailure) {
+ try {
+ Script checkLvs = new Script("lvs", Duration.millis(10000), logger);
+ checkLvs.add("--noheadings");
+ checkLvs.add("-o", "lv_attr");
+ checkLvs.add(templatePath);
+
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ String result = checkLvs.execute(parser);
+
+ if (result == null && parser.getLines() != null && !parser.getLines().isEmpty()) {
+ String lvAttr = parser.getLines().trim();
+ if (lvAttr.length() >= 6) {
+ boolean isActive = (lvAttr.indexOf('a') >= 0);
+ boolean isShared = (lvAttr.indexOf('s') >= 0);
+
+ if (!isShared || !isActive) {
+ logger.info("Template LV {} is not in shared mode (attr: {}). Activating in shared mode.", templatePath, lvAttr);
+ LibvirtComputingResource.setClvmVolumeToSharedMode(templatePath);
+ } else {
+ logger.debug("Template LV {} is already in shared mode (attr: {})", templatePath, lvAttr);
+ }
+ }
+ }
+ } catch (CloudRuntimeException e) {
+ throw e;
+ } catch (Exception e) {
+ String errorMsg = "Failed to check/ensure template LV shared mode for " + templatePath + ": " + e.getMessage();
+ if (throwOnFailure) {
+ throw new CloudRuntimeException(errorMsg, e);
+ } else {
+ logger.warn(errorMsg, e);
+ }
+ }
+ }
+
+ private void ensureTemplateLvInSharedMode(String templatePath) {
+ ensureTemplateLvInSharedMode(templatePath, false);
+ }
+
+ private long getVgPhysicalExtentSize(String vgName) {
+ final long DEFAULT_PE_SIZE = 4 * 1024 * 1024L;
+ String warningMessage = String.format("Failed to get PE size for VG %s, defaulting to 4MiB", vgName);
+
+ try {
+ Script vgDisplay = new Script("vgdisplay", 300000, logger);
+ vgDisplay.add("--units", "b");
+ vgDisplay.add("-C");
+ vgDisplay.add("--noheadings");
+ vgDisplay.add("-o", "vg_extent_size");
+ vgDisplay.add(vgName);
+
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ String result = vgDisplay.execute(parser);
+
+ if (result != null) {
+ logger.warn("{}: {}", warningMessage, result);
+ return DEFAULT_PE_SIZE;
+ }
+
+ String output = parser.getLines();
+ if (output == null || output.trim().isEmpty()) {
+ logger.warn("{}: empty output", warningMessage);
+ return DEFAULT_PE_SIZE;
+ }
+
+ output = output.trim();
+ if (output.endsWith("B")) {
+ output = output.substring(0, output.length() - 1).trim();
+ }
+
+ long peSize = Long.parseLong(output);
+ logger.debug("Physical Extent size for VG {} is {} bytes", vgName, peSize);
+ return peSize;
+ } catch (NumberFormatException e) {
+ logger.warn("{}: failed to parse PE size", warningMessage, e);
+ } catch (Exception e) {
+ logger.warn("{}: {}", warningMessage, e.getMessage());
+ }
+
+ logger.info("Using default PE size for VG {}: {} bytes (4 MiB)", vgName, DEFAULT_PE_SIZE);
+ return DEFAULT_PE_SIZE;
+ }
+
+ /**
+ * Calculate LVM LV size for CLVM_NG volume allocation.
+ */
+ private long calculateClvmNgLvSize(long virtualSize, String vgName) {
+ long peSize = getVgPhysicalExtentSize(vgName);
+
+ long clusterSize = 64 * 1024L;
+ long l2Multiplier = 4096L;
+
+ long numDataClusters = (virtualSize + clusterSize - 1) / clusterSize;
+ long numL2Clusters = (numDataClusters + l2Multiplier - 1) / l2Multiplier;
+ long l2TableSize = numL2Clusters * clusterSize;
+ long refcountTableSize = l2TableSize;
+
+ long headerOverhead = 2 * 1024 * 1024L;
+ long metadataOverhead = l2TableSize + refcountTableSize + headerOverhead;
+ long targetSize = virtualSize + metadataOverhead;
+ long roundedSize = ((targetSize + peSize - 1) / peSize) * peSize;
+ long virtualSizeGiB = virtualSize / (1024 * 1024 * 1024L);
+ long overheadMiB = metadataOverhead / (1024 * 1024L);
+
+ logger.info("Calculated volume LV size: {} bytes (virtual: {} GiB, QCOW2 metadata overhead: {} MiB, rounded to {} PEs, PE size = {} bytes)",
+ roundedSize, virtualSizeGiB, overheadMiB, roundedSize / peSize, peSize);
+
+ return roundedSize;
+ }
+
+ private long getQcow2VirtualSize(String imagePath) {
+ Script qemuImg = new Script("qemu-img", 300000, logger);
+ qemuImg.add("info");
+ qemuImg.add("--output=json");
+ qemuImg.add(imagePath);
+
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ String result = qemuImg.execute(parser);
+
+ if (result != null) {
+ throw new CloudRuntimeException("Failed to get QCOW2 virtual size for " + imagePath + ": " + result);
+ }
+
+ String output = parser.getLines();
+ if (output == null || output.trim().isEmpty()) {
+ throw new CloudRuntimeException("qemu-img info returned empty output for " + imagePath);
+ }
+
+ JsonObject info = JsonParser.parseString(output).getAsJsonObject();
+ return info.get("virtual-size").getAsLong();
+ }
+
+ private long getQcow2PhysicalSize(String imagePath) {
+ Script qemuImg = new Script("qemu-img", Duration.millis(300000), logger);
+ qemuImg.add("info");
+ qemuImg.add("--output=json");
+ qemuImg.add(imagePath);
+
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ String result = qemuImg.execute(parser);
+
+ if (result != null) {
+ throw new CloudRuntimeException("Failed to get QCOW2 physical size for " + imagePath + ": " + result);
+ }
+
+ String output = parser.getLines();
+ if (output == null || output.trim().isEmpty()) {
+ throw new CloudRuntimeException("qemu-img info returned empty output for " + imagePath);
+ }
+
+ JsonObject info = JsonParser.parseString(output).getAsJsonObject();
+ return info.get("actual-size").getAsLong();
+ }
+
+ private KVMPhysicalDisk createClvmNgDiskWithBacking(String volumeUuid, int timeout, long virtualSize, String backingFile,
+ KVMStoragePool pool, Storage.ProvisioningType provisioningType) {
+ String vgName = getVgName(pool.getLocalPath());
+ long lvSize = calculateClvmNgLvSize(virtualSize, vgName);
+ String volumePath = "/dev/" + vgName + "/" + volumeUuid;
+
+ logger.debug("Creating CLVM_NG volume {} with LV size {} bytes (virtual size: {} bytes, provisioning: {})",
+ volumeUuid, lvSize, virtualSize, provisioningType);
+
+ Script lvcreate = new Script("lvcreate", Duration.millis(timeout), logger);
+ lvcreate.add("-n", volumeUuid);
+ lvcreate.add("-L", lvSize + "B");
+ lvcreate.add("--yes");
+ lvcreate.add(vgName);
+
+ String result = lvcreate.execute();
+ if (result != null) {
+ throw new CloudRuntimeException("Failed to create LV for CLVM_NG volume: " + result);
+ }
+
+ Script qemuImg = new Script("qemu-img", Duration.millis(timeout), logger);
+ qemuImg.add("create");
+ qemuImg.add("-f", "qcow2");
Review Comment:
QemuImg class
##########
server/src/main/java/com/cloud/storage/snapshot/SnapshotManagerImpl.java:
##########
@@ -1854,6 +1874,7 @@ private void updateSnapshotPayload(long storagePoolId, CreateSnapshotPayload pay
payload.setLocationType(null);
}
+ // CLVM_NG excluded: incremental snapshots not supported in this release (handled in takeSnapshot).
Review Comment:
Is this comment needed?
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/LibvirtStorageAdaptor.java:
##########
@@ -693,11 +678,24 @@ public KVMPhysicalDisk getPhysicalDisk(String volumeUuid, KVMStoragePool pool) {
}
return disk;
} catch (LibvirtException e) {
- logger.debug("Failed to get physical disk:", e);
+ logger.debug("Failed to get volume from libvirt: " + e.getMessage());
throw new CloudRuntimeException(e.toString());
}
}
+
Review Comment:
Please remove these newlines
##########
server/src/main/java/com/cloud/storage/snapshot/SnapshotManagerImpl.java:
##########
@@ -1633,15 +1634,17 @@ public SnapshotInfo takeSnapshot(VolumeInfo volume) throws ResourceAllocationExc
boolean isKvmAndFileBasedStorage = isHypervisorKvmAndFileBasedStorage(volume, storagePool);
boolean backupSnapToSecondary = isBackupSnapshotToSecondaryForZone(volume.getDataCenterId());
- if (isKvmAndFileBasedStorage && backupSnapToSecondary) {
+ StoragePoolType poolType = volume.getStoragePoolType();
+
+ if ((isKvmAndFileBasedStorage) && backupSnapToSecondary) {
Review Comment:
```suggestion
if (isKvmAndFileBasedStorage && backupSnapToSecondary) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4692900751
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4693525587
@Pearl1594 a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r3154477766
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/LibvirtStorageAdaptor.java:
##########
@@ -693,11 +848,248 @@ public KVMPhysicalDisk getPhysicalDisk(String volumeUuid, KVMStoragePool pool) {
}
return disk;
} catch (LibvirtException e) {
- logger.debug("Failed to get physical disk:", e);
+ logger.debug("Failed to get volume from libvirt: " + e.getMessage());
+ if (isClvmPool) {
+ return getPhysicalDiskWithClvmFallback(volumeUuid, pool, libvirtPool);
+ }
+
throw new CloudRuntimeException(e.toString());
}
}
+ /**
+ * CLVM fallback: First tries to refresh libvirt pool to make volume visible,
+ * if that fails, accesses volume directly via block device path.
+ */
+ private KVMPhysicalDisk getPhysicalDiskWithClvmFallback(String volumeUuid, KVMStoragePool pool, LibvirtStoragePool libvirtPool) {
+ logger.info("CLVM volume not visible to libvirt, attempting pool refresh for volume: {}", volumeUuid);
+
+ try {
+ logger.debug("Refreshing libvirt storage pool: {}", pool.getUuid());
+ libvirtPool.getPool().refresh(0);
+
+ StorageVol vol = getVolume(libvirtPool.getPool(), volumeUuid);
+ if (vol != null) {
+ logger.info("Volume found after pool refresh: {}", volumeUuid);
+ KVMPhysicalDisk disk;
+ LibvirtStorageVolumeDef voldef = getStorageVolumeDef(libvirtPool.getPool().getConnect(), vol);
+ disk = new KVMPhysicalDisk(vol.getPath(), vol.getName(), pool);
+ disk.setSize(vol.getInfo().allocation);
+ disk.setVirtualSize(vol.getInfo().capacity);
+ disk.setFormat(voldef.getFormat() == LibvirtStorageVolumeDef.VolumeFormat.QCOW2 ?
+ PhysicalDiskFormat.QCOW2 : PhysicalDiskFormat.RAW);
+ return disk;
+ }
+ } catch (LibvirtException refreshEx) {
+ logger.debug("Pool refresh failed or volume still not found: {}", refreshEx.getMessage());
+ }
+
+ logger.info("Falling back to direct block device access for volume: {}", volumeUuid);
+ return getPhysicalDiskViaDirectBlockDevice(volumeUuid, pool);
+ }
+
+ private String getVgName(String sourceDir) {
+ String vgName = sourceDir;
+ if (vgName.startsWith("/")) {
+ String[] parts = vgName.split("/");
+ List<String> tokens = Arrays.stream(parts)
+ .filter(s -> !s.isEmpty()).collect(Collectors.toList());
+
+ vgName = tokens.size() > 1 ? tokens.get(1)
+ : tokens.size() == 1 ? tokens.get(0)
+ : "";
+ }
+ return vgName;
+ }
+
+ /**
+ * For CLVM volumes that exist in LVM but are not visible to libvirt,
+ * access them directly via block device path.
+ */
+ private KVMPhysicalDisk getPhysicalDiskViaDirectBlockDevice(String volumeUuid, KVMStoragePool pool) {
+ try {
+ String vgName = extractVgNameFromPool(pool);
+ verifyLvExistsInVg(volumeUuid, vgName);
+
+ logger.info("Volume {} exists in LVM but not visible to libvirt, accessing directly", volumeUuid);
+
+ String lvPath = findAccessibleDeviceNode(volumeUuid, vgName, pool);
+ long size = getClvmVolumeSize(lvPath);
+
+ KVMPhysicalDisk disk = createPhysicalDiskFromClvmLv(lvPath, volumeUuid, pool, size);
+
+ ensureTemplateAccessibility(volumeUuid, lvPath, pool);
+
+ return disk;
+
+ } catch (CloudRuntimeException ex) {
+ throw ex;
+ } catch (Exception ex) {
+ logger.error("Failed to access CLVM volume via direct block device: {}", volumeUuid, ex);
+ throw new CloudRuntimeException(String.format("Could not find volume %s: %s ", volumeUuid, ex.getMessage()));
+ }
+ }
+
+ private String extractVgNameFromPool(KVMStoragePool pool) {
+ String sourceDir = pool.getLocalPath();
+ if (sourceDir == null || sourceDir.isEmpty()) {
+ throw new CloudRuntimeException("CLVM pool sourceDir is not set, cannot determine VG name");
+ }
+ String vgName = getVgName(sourceDir);
+ logger.debug("Using VG name: {} (from sourceDir: {})", vgName, sourceDir);
+ return vgName;
+ }
+
+ private void verifyLvExistsInVg(String volumeUuid, String vgName) {
+ logger.debug("Checking if volume {} exists in VG {}", volumeUuid, vgName);
+ Script checkLvCmd = new Script("/usr/sbin/lvs", 5000, logger);
+ checkLvCmd.add("--noheadings");
+ checkLvCmd.add("--unbuffered");
+ checkLvCmd.add(vgName + "/" + volumeUuid);
+
+ String checkResult = checkLvCmd.execute();
+ if (checkResult != null) {
+ logger.debug("Volume {} does not exist in VG {}: {}", volumeUuid, vgName, checkResult);
+ throw new CloudRuntimeException(String.format("Storage volume not found: no storage vol with matching name '%s'", volumeUuid));
+ }
+ }
+
+ private String findAccessibleDeviceNode(String volumeUuid, String vgName, KVMStoragePool pool) {
+ String lvPath = "/dev/" + vgName + "/" + volumeUuid;
+ File lvDevice = new File(lvPath);
+
+ if (!lvDevice.exists()) {
+ lvPath = tryDeviceMapperPath(volumeUuid, vgName, lvDevice);
+
+ if (!lvDevice.exists()) {
+ lvPath = handleMissingDeviceNode(volumeUuid, vgName, pool);
+ }
+ }
+
+ return lvPath;
+ }
+
+ private String tryDeviceMapperPath(String volumeUuid, String vgName, File lvDevice) {
+ String vgNameEscaped = vgName.replace("-", "--");
+ String volumeUuidEscaped = volumeUuid.replace("-", "--");
+ String mapperPath = "/dev/mapper/" + vgNameEscaped + "-" + volumeUuidEscaped;
+ File mapperDevice = new File(mapperPath);
+
+ if (!mapperDevice.exists()) {
+ lvDevice = mapperDevice;
+ }
+
+ return mapperPath;
+ }
+
+ private String handleMissingDeviceNode(String volumeUuid, String vgName, KVMStoragePool pool) {
+ if (pool.getType() == StoragePoolType.CLVM_NG && volumeUuid.startsWith("template-")) {
+ return activateTemplateAndGetPath(volumeUuid, vgName);
+ } else {
+ logger.warn("Volume exists in LVM but device node not found: {}", volumeUuid);
+ throw new CloudRuntimeException(String.format("Could not find volume %s " +
+ "in VG %s - volume exists in LVM but device node not accessible", volumeUuid, vgName));
+ }
+ }
+
+ private String activateTemplateAndGetPath(String volumeUuid, String vgName) {
+ logger.info("Template volume {} device node not found. Attempting to activate in shared mode.", volumeUuid);
+ String templateLvPath = "/dev/" + vgName + "/" + volumeUuid;
+
+ try {
+ ensureTemplateLvInSharedMode(templateLvPath, false);
+
+ String lvPath = findDeviceNodeAfterActivation(templateLvPath, volumeUuid, vgName);
+
+ logger.info("Successfully activated template volume {} at {}", volumeUuid, lvPath);
+ return lvPath;
+
+ } catch (CloudRuntimeException e) {
+ throw new CloudRuntimeException(String.format("Failed to activate template volume %s " +
+ "in VG %s: %s", volumeUuid, vgName, e.getMessage()), e);
+ }
+ }
+
+ private String findDeviceNodeAfterActivation(String templateLvPath, String volumeUuid, String vgName) {
+ File lvDevice = new File(templateLvPath);
+ String lvPath = templateLvPath;
+
+ if (!lvDevice.exists()) {
+ String vgNameEscaped = vgName.replace("-", "--");
+ String volumeUuidEscaped = volumeUuid.replace("-", "--");
+ lvPath = "/dev/mapper/" + vgNameEscaped + "-" + volumeUuidEscaped;
+ lvDevice = new File(lvPath);
+ }
+
+ if (!lvDevice.exists()) {
+ logger.error("Template volume {} still not accessible after activation attempt", volumeUuid);
+ throw new CloudRuntimeException(String.format("Could not activate template volume %s " +
+ "in VG %s - device node not accessible even after activation", volumeUuid, vgName));
+ }
+
+ return lvPath;
+ }
+
+ private long getClvmVolumeSize(String lvPath) {
+ try {
+ Script lvsCmd = new Script("/usr/sbin/lvs", 5000, logger);
+ lvsCmd.add("--noheadings");
+ lvsCmd.add("--units");
+ lvsCmd.add("b");
+ lvsCmd.add("-o");
+ lvsCmd.add("lv_size");
+ lvsCmd.add(lvPath);
+
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ String result = lvsCmd.execute(parser);
+
+ String output = (result == null) ? parser.getLines() : result;
+
+ if (output != null && !output.isEmpty()) {
+ String sizeStr = output.trim().replaceAll("[^0-9]", "");
+ if (!sizeStr.isEmpty()) {
+ return Long.parseLong(sizeStr);
+ }
+ }
+ } catch (Exception sizeEx) {
+ logger.warn("Failed to get size for CLVM volume via lvs: {}", sizeEx.getMessage());
+ File lvDevice = new File(lvPath);
+ if (lvDevice.isFile()) {
+ return lvDevice.length();
+ }
+ }
+ return 0;
+ }
+
+ private KVMPhysicalDisk createPhysicalDiskFromClvmLv(String lvPath, String volumeUuid,
+ KVMStoragePool pool, long size) {
+ KVMPhysicalDisk disk = new KVMPhysicalDisk(lvPath, volumeUuid, pool);
+
+ PhysicalDiskFormat diskFormat = (pool.getType() == StoragePoolType.CLVM_NG)
+ ? PhysicalDiskFormat.QCOW2
+ : PhysicalDiskFormat.RAW;
+
+ logger.debug("{} pool detected, setting disk format to {} for volume {}",
+ pool.getType(), diskFormat, volumeUuid);
+
+ disk.setFormat(diskFormat);
+ disk.setSize(size);
+ disk.setVirtualSize(size);
+
+ logger.info("Successfully accessed CLVM/CLVM_NG volume via direct block device: {} " +
+ "with format: {} and size: {} bytes", lvPath, diskFormat, size);
+
+ return disk;
+ }
+
+ private void ensureTemplateAccessibility(String volumeUuid, String lvPath, KVMStoragePool pool) {
+ if (pool.getType() == StoragePoolType.CLVM_NG && volumeUuid.startsWith("template-")) {
+ logger.info("Detected template volume {}. Ensuring it's activated in shared mode for backing file access.",
+ volumeUuid);
+ ensureTemplateLvInSharedMode(lvPath, false);
+ }
+ }
+
Review Comment:
can have separate ClvmStorageAdaptor with all these func ^^^ for CLVM/CLVM_NG extending this class?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4351013892
<b>[SF] Trillian test result (tid-15988)</b>
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 49309 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12617-t15988-kvm-ol8.zip
Smoke tests completed. 151 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4456484075
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17865
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4498266584
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4070584755
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-3955975910
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-3931837202
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [2.4% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-3931234365
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [2.3% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-3923212352
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [2.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4168140926
@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4632521069
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4700316627
@Pearl1594 a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r3376567340
##########
engine/storage/src/main/java/org/apache/cloudstack/storage/endpoint/DefaultEndPointSelector.java:
##########
@@ -493,6 +645,31 @@ public EndPoint select(DataObject object, StorageAction action, boolean encrypti
}
case DELETEVOLUME: {
VolumeInfo volume = (VolumeInfo) object;
+
+ // For CLVM volumes, route to the host holding the exclusive lock
+ if (volume.getHypervisorType() == Hypervisor.HypervisorType.KVM) {
+ DataStore store = volume.getDataStore();
+ if (store.getRole() == DataStoreRole.Primary) {
+ StoragePoolVO pool = _storagePoolDao.findById(store.getId());
+ if (pool != null && ClvmPoolManager.isClvmPoolType(pool.getPoolType())) {
+ Long lockHostId = getClvmLockHostId(volume);
+ if (lockHostId != null) {
+ logger.info("Routing CLVM volume {} deletion to lock holder host {}",
+ volume.getUuid(), lockHostId);
+ EndPoint ep = getEndPointFromHostId(lockHostId);
+ if (ep != null) {
+ return ep;
+ }
+ logger.warn("Could not get endpoint for CLVM lock host {}, falling back to default selection",
+ lockHostId);
+ } else {
+ logger.debug("No CLVM lock host tracked for volume {}, using default endpoint selection",
+ volume.getUuid());
+ }
+ }
+ }
+ }
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4653893205
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4674385289
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4558035606
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [33.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
<!-- sqra:qg_revision=a4a94a903772b41c572eeff2d53c58419eb106eef00633953feaab5f8e65ec43 -->
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4665900309
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4665893341
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "JoaoJandre (via GitHub)" <gi...@apache.org>.
JoaoJandre commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r3383432189
##########
engine/storage/datamotion/src/main/java/org/apache/cloudstack/storage/motion/StorageSystemDataMotionStrategy.java:
##########
@@ -2132,6 +2157,23 @@ public void copyAsync(Map<VolumeInfo, DataStore> volumeDataStoreMap, VirtualMach
throw new AgentUnavailableException("Operation timed out", destHost.getId());
}
+ for (VolumeInfo vol : samePoolClvmVolumes) {
Review Comment:
I see, thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4409291173
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [33.1% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
<!-- sqra:qg_revision=2810404f9bdbb08abd7243adeba83ed47ec8a40367ecfd74dc4787d0c336544b -->
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4407860915
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4416566114
Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 17805
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4416539156
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4389936826
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4400144687
<b>[SF] Trillian test result (tid-16033)</b>
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 89277 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12617-t16033-kvm-ol8.zip
Smoke tests completed. 136 look OK, 15 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_CRUD_operations_userdata | `Error` | 1522.70 | test_register_userdata.py
test_deploy_vm_with_registered_userdata | `Error` | 6.30 | test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_allow | `Error` | 6.86 | test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_append | `Error` | 6.50 | test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_deny | `Error` | 6.19 | test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_params | `Error` | 6.23 | test_register_userdata.py
test_link_and_unlink_userdata_to_template | `Error` | 6.36 | test_register_userdata.py
test_user_userdata_crud | `Error` | 11.52 | test_register_userdata.py
ContextSuite context=TestResetVmOnReboot>:setup | `Error` | 0.00 | test_reset_vm_on_reboot.py
ContextSuite context=TestRAMCPUResourceAccounting>:setup | `Error` | 0.00 | test_resource_accounting.py
test_03_register_template | `Error` | 1.15 | test_resource_names.py
ContextSuite context=TestRestoreVM>:setup | `Error` | 0.00 | test_restore_vm.py
ContextSuite context=TestRouterDHCPHosts>:setup | `Error` | 0.00 | test_router_dhcphosts.py
ContextSuite context=TestRouterDHCPOpts>:setup | `Error` | 0.00 | test_router_dhcphosts.py
ContextSuite context=TestRouterDns>:setup | `Error` | 0.00 | test_router_dns.py
ContextSuite context=TestRouterDnsService>:setup | `Error` | 0.00 | test_router_dnsservice.py
ContextSuite context=TestRouterIpTablesPolicies>:setup | `Error` | 0.00 | test_routers_iptables_default_policy.py
ContextSuite context=TestVPCIpTablesPolicies>:setup | `Error` | 0.00 | test_routers_iptables_default_policy.py
ContextSuite context=TestIsolatedNetworks>:setup | `Error` | 0.00 | test_routers_network_ops.py
ContextSuite context=TestRedundantIsolateNetworks>:setup | `Error` | 0.00 | test_routers_network_ops.py
ContextSuite context=TestRouterServices>:setup | `Error` | 0.00 | test_routers.py
ContextSuite context=TestCpuCapServiceOfferings>:setup | `Error` | 0.00 | test_service_offerings.py
ContextSuite context=TestServiceOfferings>:setup | `Error` | 0.32 | test_service_offerings.py
ContextSuite context=TestSetSourceNatIp>:setup | `Error` | 0.00 | test_set_sourcenat.py
ContextSuite context=TestSnapshotRootDisk>:setup | `Error` | 0.00 | test_snapshots.py
ContextSuite context=TestSnapshotStandaloneBackup>:setup | `Error` | 0.00 | test_snapshots.py
ContextSuite context=TestSslOffloading>:setup | `Error` | 0.00 | test_ssl_offloading.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4685654727
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4691619006
<b>[SF] Trillian test result (tid-16292)</b>
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 50205 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12617-t16292-kvm-ol8.zip
Smoke tests completed. 151 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-3873544859
## [Codecov](https://app.codecov.io/gh/apache/cloudstack/pull/12617?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 3.67%. Comparing base ([`d3e1976`](https://app.codecov.io/gh/apache/cloudstack/commit/d3e1976912561e5919385b1995fbbdd2b0066416?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5fc1f12`](https://app.codecov.io/gh/apache/cloudstack/commit/5fc1f12965c0aed7c5fb66150f1fd0ab6f5ae495?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
> :exclamation: There is a different number of reports uploaded between BASE (d3e1976) and HEAD (5fc1f12). Click for more details.
>
> <details><summary>HEAD has 1 upload less than BASE</summary>
>
>| Flag | BASE (d3e1976) | HEAD (5fc1f12) |
>|------|------|------|
>|unittests|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #12617 +/- ##
=============================================
- Coverage 17.90% 3.67% -14.23%
=============================================
Files 5938 453 -5485
Lines 532864 38447 -494417
Branches 65192 7115 -58077
=============================================
- Hits 95392 1414 -93978
+ Misses 426793 36846 -389947
+ Partials 10679 187 -10492
```
| [Flag](https://app.codecov.io/gh/apache/cloudstack/pull/12617/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [uitests](https://app.codecov.io/gh/apache/cloudstack/pull/12617/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `3.67% <ø> (ø)` | |
| [unittests](https://app.codecov.io/gh/apache/cloudstack/pull/12617/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/cloudstack/pull/12617?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
weizhouapache commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r2784253161
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/LibvirtStorageAdaptor.java:
##########
@@ -663,6 +667,17 @@ public KVMPhysicalDisk getPhysicalDisk(String volumeUuid, KVMStoragePool pool) {
try {
StorageVol vol = getVolume(libvirtPool.getPool(), volumeUuid);
+
+ // Check if volume was found - if null, treat as not found and trigger fallback for CLVM
Review Comment:
maybe better
```
if (pool.getType() == StoragePoolType.CLVM) {
return getPhysicalDisk(volumeUuid, pool, libvirtPool);
}
StorageVol vol = getVolume(libvirtPool.getPool(), volumeUuid);
...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-3873760898
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-3923016253
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-3934848201
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [2.3% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4069723852
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [8.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4200992176
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4158399568
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [25.1% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4159339724
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [25.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4145758204
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [23.4% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4680520112
<b>[SF] Trillian test result (tid-16284)</b>
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 52854 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12617-t16284-kvm-ol8.zip
Smoke tests completed. 149 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_03_ping_in_ssvm_success | `Failure` | 15.53 | test_diagnostics.py
test_05_ping_in_cpvm_success | `Failure` | 15.53 | test_diagnostics.py
test_isolate_network_password_server | `Failure` | 12.93 | test_password_server.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4673702739
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18218
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4673272959
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4673964991
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4700744106
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4685593924
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [33.6% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
<!-- sqra:qg_revision=53c006beaba57171b18fd2630c9417d904ce5d787964659b3d2365024115ed98 -->
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4555452518
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4564598786
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [32.9% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
<!-- sqra:qg_revision=732ccec38704e2bea45d255b79300e3ecf42986f41b74cc983e83252208a070c -->
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4653908265
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4654202221
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4652053593
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "winterhazel (via GitHub)" <gi...@apache.org>.
winterhazel commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r3375913541
##########
engine/storage/src/main/java/org/apache/cloudstack/storage/endpoint/DefaultEndPointSelector.java:
##########
@@ -493,6 +645,31 @@ public EndPoint select(DataObject object, StorageAction action, boolean encrypti
}
case DELETEVOLUME: {
VolumeInfo volume = (VolumeInfo) object;
+
+ // For CLVM volumes, route to the host holding the exclusive lock
+ if (volume.getHypervisorType() == Hypervisor.HypervisorType.KVM) {
+ DataStore store = volume.getDataStore();
+ if (store.getRole() == DataStoreRole.Primary) {
+ StoragePoolVO pool = _storagePoolDao.findById(store.getId());
+ if (pool != null && ClvmPoolManager.isClvmPoolType(pool.getPoolType())) {
+ Long lockHostId = getClvmLockHostId(volume);
+ if (lockHostId != null) {
+ logger.info("Routing CLVM volume {} deletion to lock holder host {}",
+ volume.getUuid(), lockHostId);
+ EndPoint ep = getEndPointFromHostId(lockHostId);
+ if (ep != null) {
+ return ep;
+ }
+ logger.warn("Could not get endpoint for CLVM lock host {}, falling back to default selection",
+ lockHostId);
+ } else {
+ logger.debug("No CLVM lock host tracked for volume {}, using default endpoint selection",
+ volume.getUuid());
+ }
+ }
+ }
+ }
Review Comment:
A similar logic is executed many times throughout this file. We could use a single method instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r3382965170
##########
engine/storage/datamotion/src/main/java/org/apache/cloudstack/storage/motion/KvmNonManagedStorageDataMotionStrategy.java:
##########
@@ -144,12 +144,16 @@ protected boolean isDestinationNfsPrimaryStorageClusterWide(Map<VolumeInfo, Data
}
/**
- * Configures a {@link MigrateDiskInfo} object configured for migrating a File System volume and calls rootImageProvisioning.
+ * Configures a {@link MigrateDiskInfo} object configured for migrating a File System volume.
*/
@Override
protected MigrateCommand.MigrateDiskInfo configureMigrateDiskInfo(VolumeInfo srcVolumeInfo, String destPath, String backingPath) {
- return new MigrateCommand.MigrateDiskInfo(srcVolumeInfo.getPath(), MigrateCommand.MigrateDiskInfo.DiskType.FILE, MigrateCommand.MigrateDiskInfo.DriverType.QCOW2,
- MigrateCommand.MigrateDiskInfo.Source.FILE, destPath, backingPath);
+ return new MigrateCommand.MigrateDiskInfo(srcVolumeInfo.getPath(),
+ MigrateCommand.MigrateDiskInfo.DiskType.FILE,
+ MigrateCommand.MigrateDiskInfo.DriverType.QCOW2,
+ MigrateCommand.MigrateDiskInfo.Source.FILE,
+ destPath,
+ backingPath);
Review Comment:
this was just cosmetic for better readability.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4666130850
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18209
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r3389085487
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java:
##########
@@ -1920,6 +2131,15 @@ private SnapshotObjectTO takeIncrementalVolumeSnapshotOfStoppedVm(SnapshotObject
logger.debug("Taking incremental volume snapshot of volume [{}]. Snapshot will be copied to [{}].", volumeObjectTo,
ObjectUtils.defaultIfNull(secondaryPool, primaryPool));
try {
+ // For CLVM_NG incremental snapshots, validate bitmap before proceeding
+ /*
+ SnapshotObjectTO bitmapValidationResult = validateClvmNgBitmapAndFallbackIfNeeded(snapshotObjectTO, primaryPool,
+ secondaryPool, secondaryPoolUrl, snapshotName, volumeObjectTo, conn, wait);
+ if (bitmapValidationResult != null) {
+ return bitmapValidationResult;
+ }
+ */
Review Comment:
Thanks, will do - I was working on a separate PR to support incremental snaps for CLVM pool type, but the bitmap solution may not be suitable. So yes, I'll clean it up. Thanks for the review, much appreciated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4239101361
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [33.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4128734705
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [12.5% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4130342311
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [12.2% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r3154330059
##########
core/src/main/java/org/apache/cloudstack/storage/command/ClvmLockTransferAnswer.java:
##########
@@ -0,0 +1,95 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.cloudstack.storage.command;
Review Comment:
```suggestion
package org.apache.cloudstack.storage.clvm.command;
```
move to clvm specific pkg if possible - ClvmLockTransferCommand , ClvmLockTransferAnswer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4327234114
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [33.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4346340525
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17659
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4346352468
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r3166018899
##########
server/src/main/java/com/cloud/storage/VolumeApiServiceImpl.java:
##########
@@ -414,6 +417,8 @@ public class VolumeApiServiceImpl extends ManagerBase implements VolumeApiServic
public static final ConfigKey<Boolean> AllowCheckAndRepairVolume = new ConfigKey<>("Advanced", Boolean.class, "volume.check.and.repair.leaks.before.use", "false",
"To check and repair the volume if it has any leaks before performing volume attach or VM start operations", true, ConfigKey.Scope.StoragePool);
+
Review Comment:
```suggestion
```
##########
server/src/main/java/com/cloud/storage/VolumeApiServiceImpl.java:
##########
@@ -414,6 +417,8 @@ public class VolumeApiServiceImpl extends ManagerBase implements VolumeApiServic
public static final ConfigKey<Boolean> AllowCheckAndRepairVolume = new ConfigKey<>("Advanced", Boolean.class, "volume.check.and.repair.leaks.before.use", "false",
"To check and repair the volume if it has any leaks before performing volume attach or VM start operations", true, ConfigKey.Scope.StoragePool);
+
+
Review Comment:
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4327049729
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4201714629
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [33.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4168421107
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17322
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4163378850
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [29.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4169805917
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
weizhouapache commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r2784253161
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/LibvirtStorageAdaptor.java:
##########
@@ -663,6 +667,17 @@ public KVMPhysicalDisk getPhysicalDisk(String volumeUuid, KVMStoragePool pool) {
try {
StorageVol vol = getVolume(libvirtPool.getPool(), volumeUuid);
+
+ // Check if volume was found - if null, treat as not found and trigger fallback for CLVM
Review Comment:
maybe better
```
if (pool.getType() == StoragePoolType.CLVM) {
return getPhysicalDisk(volumeUuid, pool, libvirtPool);
}
StorageVol vol = getVolume(libvirtPool.getPool(), volumeUuid);
...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4070544001
Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 17161
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4070860490
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [8.5% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r2962444971
##########
engine/storage/volume/src/main/java/org/apache/cloudstack/storage/datastore/provider/DefaultHostListener.java:
##########
@@ -139,6 +141,18 @@ public boolean hostConnect(long hostId, long poolId) throws StorageConflictExcep
Map<String, String> nfsMountOpts = storageManager.getStoragePoolNFSMountOpts(pool, null).first();
Optional.ofNullable(nfsMountOpts).ifPresent(detailsMap::putAll);
+
+ // Propagate CLVM secure zero-fill setting to the host
+ // Note: This is done during host connection (agent start, MS restart, host reconnection)
+ // so the setting is non-dynamic. Changes require host reconnection to take effect.
+ if (ClvmLockManager.isClvmPoolType(pool.getPoolType())) {
+ Boolean clvmSecureZeroFill = VolumeApiServiceImpl.CLVMSecureZeroFill.valueIn(poolId);
+ if (clvmSecureZeroFill != null) {
+ detailsMap.put("clvmsecurezerofill", String.valueOf(clvmSecureZeroFill));
+ logger.debug("Added CLVM secure zero-fill setting: {} for storage pool: {}", clvmSecureZeroFill, pool);
+ }
Review Comment:
This code hard-codes the detail key string `"clvmsecurezerofill"`. The same key is also defined as `KVMStoragePool.CLVM_SECURE_ZERO_FILL`; having two sources of truth risks drift/typos between management server and agent. Consider moving this key to a shared constant accessible from both modules (or otherwise centralizing it) and reference that constant here.
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/LibvirtStorageAdaptor.java:
##########
@@ -693,11 +848,248 @@ public KVMPhysicalDisk getPhysicalDisk(String volumeUuid, KVMStoragePool pool) {
}
return disk;
} catch (LibvirtException e) {
- logger.debug("Failed to get physical disk:", e);
+ logger.debug("Failed to get volume from libvirt: " + e.getMessage());
+ if (isClvmPool) {
+ return getPhysicalDiskWithClvmFallback(volumeUuid, pool, libvirtPool);
+ }
+
throw new CloudRuntimeException(e.toString());
}
}
+ /**
+ * CLVM fallback: First tries to refresh libvirt pool to make volume visible,
+ * if that fails, accesses volume directly via block device path.
+ */
+ private KVMPhysicalDisk getPhysicalDiskWithClvmFallback(String volumeUuid, KVMStoragePool pool, LibvirtStoragePool libvirtPool) {
+ logger.info("CLVM volume not visible to libvirt, attempting pool refresh for volume: {}", volumeUuid);
+
+ try {
+ logger.debug("Refreshing libvirt storage pool: {}", pool.getUuid());
+ libvirtPool.getPool().refresh(0);
+
+ StorageVol vol = getVolume(libvirtPool.getPool(), volumeUuid);
+ if (vol != null) {
+ logger.info("Volume found after pool refresh: {}", volumeUuid);
+ KVMPhysicalDisk disk;
+ LibvirtStorageVolumeDef voldef = getStorageVolumeDef(libvirtPool.getPool().getConnect(), vol);
+ disk = new KVMPhysicalDisk(vol.getPath(), vol.getName(), pool);
+ disk.setSize(vol.getInfo().allocation);
+ disk.setVirtualSize(vol.getInfo().capacity);
+ disk.setFormat(voldef.getFormat() == LibvirtStorageVolumeDef.VolumeFormat.QCOW2 ?
+ PhysicalDiskFormat.QCOW2 : PhysicalDiskFormat.RAW);
+ return disk;
+ }
+ } catch (LibvirtException refreshEx) {
+ logger.debug("Pool refresh failed or volume still not found: {}", refreshEx.getMessage());
+ }
+
+ logger.info("Falling back to direct block device access for volume: {}", volumeUuid);
+ return getPhysicalDiskViaDirectBlockDevice(volumeUuid, pool);
+ }
+
+ private String getVgName(String sourceDir) {
+ String vgName = sourceDir;
+ if (vgName.startsWith("/")) {
+ String[] parts = vgName.split("/");
+ List<String> tokens = Arrays.stream(parts)
+ .filter(s -> !s.isEmpty()).collect(Collectors.toList());
+
+ vgName = tokens.size() > 1 ? tokens.get(1)
+ : tokens.size() == 1 ? tokens.get(0)
+ : "";
+ }
+ return vgName;
+ }
+
+ /**
+ * For CLVM volumes that exist in LVM but are not visible to libvirt,
+ * access them directly via block device path.
+ */
+ private KVMPhysicalDisk getPhysicalDiskViaDirectBlockDevice(String volumeUuid, KVMStoragePool pool) {
+ try {
+ String vgName = extractVgNameFromPool(pool);
+ verifyLvExistsInVg(volumeUuid, vgName);
+
+ logger.info("Volume {} exists in LVM but not visible to libvirt, accessing directly", volumeUuid);
+
+ String lvPath = findAccessibleDeviceNode(volumeUuid, vgName, pool);
+ long size = getClvmVolumeSize(lvPath);
+
+ KVMPhysicalDisk disk = createPhysicalDiskFromClvmLv(lvPath, volumeUuid, pool, size);
+
+ ensureTemplateAccessibility(volumeUuid, lvPath, pool);
+
+ return disk;
+
+ } catch (CloudRuntimeException ex) {
+ throw ex;
+ } catch (Exception ex) {
+ logger.error("Failed to access CLVM volume via direct block device: {}", volumeUuid, ex);
+ throw new CloudRuntimeException(String.format("Could not find volume %s: %s ", volumeUuid, ex.getMessage()));
+ }
+ }
+
+ private String extractVgNameFromPool(KVMStoragePool pool) {
+ String sourceDir = pool.getLocalPath();
+ if (sourceDir == null || sourceDir.isEmpty()) {
+ throw new CloudRuntimeException("CLVM pool sourceDir is not set, cannot determine VG name");
+ }
+ String vgName = getVgName(sourceDir);
+ logger.debug("Using VG name: {} (from sourceDir: {})", vgName, sourceDir);
+ return vgName;
+ }
+
+ private void verifyLvExistsInVg(String volumeUuid, String vgName) {
+ logger.debug("Checking if volume {} exists in VG {}", volumeUuid, vgName);
+ Script checkLvCmd = new Script("/usr/sbin/lvs", 5000, logger);
+ checkLvCmd.add("--noheadings");
+ checkLvCmd.add("--unbuffered");
+ checkLvCmd.add(vgName + "/" + volumeUuid);
+
+ String checkResult = checkLvCmd.execute();
+ if (checkResult != null) {
+ logger.debug("Volume {} does not exist in VG {}: {}", volumeUuid, vgName, checkResult);
+ throw new CloudRuntimeException(String.format("Storage volume not found: no storage vol with matching name '%s'", volumeUuid));
+ }
+ }
+
+ private String findAccessibleDeviceNode(String volumeUuid, String vgName, KVMStoragePool pool) {
+ String lvPath = "/dev/" + vgName + "/" + volumeUuid;
+ File lvDevice = new File(lvPath);
+
+ if (!lvDevice.exists()) {
+ lvPath = tryDeviceMapperPath(volumeUuid, vgName, lvDevice);
+
+ if (!lvDevice.exists()) {
+ lvPath = handleMissingDeviceNode(volumeUuid, vgName, pool);
+ }
+ }
+
+ return lvPath;
+ }
+
+ private String tryDeviceMapperPath(String volumeUuid, String vgName, File lvDevice) {
+ String vgNameEscaped = vgName.replace("-", "--");
+ String volumeUuidEscaped = volumeUuid.replace("-", "--");
+ String mapperPath = "/dev/mapper/" + vgNameEscaped + "-" + volumeUuidEscaped;
+ File mapperDevice = new File(mapperPath);
+
+ if (!mapperDevice.exists()) {
+ lvDevice = mapperDevice;
+ }
+
+ return mapperPath;
+ }
Review Comment:
`findAccessibleDeviceNode` attempts to fall back to `/dev/mapper/...` if `/dev/<vg>/<lv>` doesn’t exist, but the logic in `tryDeviceMapperPath` is broken: it checks `if (!mapperDevice.exists())` (reversed) and assigning to `lvDevice` has no effect on the caller. As a result, even if the mapper node exists, the method will still think it’s missing and may throw incorrectly. Rework this to check the mapper path existence and return the correct path (or return a `File`/boolean) so the fallback actually works.
##########
server/src/main/java/com/cloud/storage/VolumeApiServiceImpl.java:
##########
@@ -2631,6 +2662,351 @@ private Volume orchestrateAttachVolumeToVM(Long vmId, Long volumeId, Long device
return newVol;
}
+ /**
+ * Helper method to get storage pools for volume and VM.
+ *
+ * @param volumeToAttach The volume being attached
+ * @param vmExistingVolume The VM's existing volume
+ * @return Pair of StoragePoolVO objects (volumePool, vmPool), or null if either pool is missing
+ */
+ private Pair<StoragePoolVO, StoragePoolVO> getStoragePoolsForVolumeAttachment(VolumeInfo volumeToAttach, VolumeVO vmExistingVolume) {
+ if (volumeToAttach == null || vmExistingVolume == null) {
+ return null;
+ }
+
+ StoragePoolVO volumePool = _storagePoolDao.findById(volumeToAttach.getPoolId());
+ StoragePoolVO vmPool = _storagePoolDao.findById(vmExistingVolume.getPoolId());
+
+ if (volumePool == null || vmPool == null) {
+ return null;
+ }
+
+ return new Pair<>(volumePool, vmPool);
+ }
+
+ /**
+ * Checks if both storage pools are CLVM type.
+ *
+ * @param volumePool Storage pool for the volume
+ * @param vmPool Storage pool for the VM
+ * @return true if both pools are CLVM type
+ */
+ private boolean areBothPoolsClvmType(StoragePoolVO volumePool, StoragePoolVO vmPool) {
+ return volumePool.getPoolType() == StoragePoolType.CLVM &&
+ vmPool.getPoolType() == StoragePoolType.CLVM;
+ }
+
+ /**
+ * Checks if a storage pool is CLVM type.
+ *
+ * @param pool Storage pool to check
+ * @return true if pool is CLVM type
+ */
+ private boolean isClvmPool(StoragePoolVO pool) {
+ return pool != null && pool.getPoolType() == StoragePoolType.CLVM;
+ }
Review Comment:
The new CLVM attachment/migration helpers only recognize `StoragePoolType.CLVM` (e.g., `areBothPoolsClvmType` and `isClvmPool`). Since this PR introduces `CLVM_NG` and `ClvmLockManager.isClvmPoolType` already treats both as CLVM-like, these helpers will silently skip the lightweight lock-transfer path for CLVM_NG volumes. Update the helper predicates to include `CLVM_NG` (or reuse a shared helper) so behavior is consistent for both pool types.
##########
pom.xml:
##########
@@ -55,6 +55,10 @@
<sonar.host.url>https://sonarcloud.io</sonar.host.url>
<sonar.exclusions>engine/schema/src/main/java/org/apache/cloudstack/backup/BackupOfferingDetailsVO.java</sonar.exclusions>
<sonar.exclusions>api/src/main/java/org/apache/cloudstack/api/response/BackupOfferingResponse.java</sonar.exclusions>
+ <sonar.exclusions>core/src/main/java/com/cloud/agent/api/PostMigrationAnswer.java</sonar.exclusions>
+ <sonar.exclusions>core/src/main/java/com/cloud/agent/api/PostMigrationCommand.java</sonar.exclusions>
+ <sonar.exclusions>core/src/main/java/com/cloud/agent/api/PreMigrationCommand.java</sonar.exclusions>
+ <sonar.exclusions>core/src/main/java/org/apache/cloudstack/storage/command/ClvmLockTransferCommand.java</sonar.exclusions>
Review Comment:
`sonar.exclusions` is declared multiple times. In Maven, later property declarations override earlier ones, so only the last exclusion will actually be used. Combine all exclusions into a single `sonar.exclusions` property (comma-separated) so none of the earlier exclusions are lost.
##########
engine/storage/src/main/java/org/apache/cloudstack/storage/endpoint/DefaultEndPointSelector.java:
##########
@@ -32,8 +32,13 @@
import com.cloud.dc.DedicatedResourceVO;
import com.cloud.dc.dao.DedicatedResourceDao;
+import com.cloud.storage.ClvmLockManager;
+import com.cloud.storage.Storage;
+import com.cloud.storage.VolumeDetailVO;
+import com.cloud.storage.dao.VolumeDetailsDao;
Review Comment:
This module (`cloud-engine-storage`) does not depend on `cloud-server` (see the commented-out dependency in `engine/storage/pom.xml`), but this file imports `com.cloud.storage.ClvmLockManager` which is defined under `server/`. That will break compilation for this module. Consider removing this dependency (e.g., inline the pool-type check here), or move `ClvmLockManager` (or at least its `isClvmPoolType` helper) into a module that `cloud-engine-storage` depends on (api/core/engine).
##########
engine/storage/src/main/java/org/apache/cloudstack/storage/endpoint/DefaultEndPointSelector.java:
##########
@@ -388,18 +406,91 @@ private List<HostVO> listUpAndConnectingSecondaryStorageVmHost(Long dcId) {
return sc.list();
}
+ /**
+ * Selects endpoint for CLVM volumes with destination host hint.
+ * This ensures volumes are created on the correct host with exclusive locks.
+ *
+ * @param volume The volume to check for CLVM routing
+ * @param operation Description of the operation (for logging)
+ * @return EndPoint for the destination host if CLVM routing applies, null otherwise
+ */
+ private EndPoint selectClvmEndpointIfApplicable(VolumeInfo volume, String operation) {
+ DataStore store = volume.getDataStore();
+
+ if (store.getRole() != DataStoreRole.Primary) {
+ return null;
+ }
+
+ // Check if this is a CLVM pool
+ StoragePoolVO pool = _storagePoolDao.findById(store.getId());
+ if (pool == null ||
+ (pool.getPoolType() != Storage.StoragePoolType.CLVM ||
Review Comment:
The CLVM pool type check is logically incorrect: `(poolType != CLVM || poolType != CLVM_NG)` is always true, so this method will always return `null` and CLVM routing will never be applied. Change this to an AND check (or use a helper like `isClvmPoolType`) so CLVM/CLVM_NG volumes can actually select the intended endpoint.
```suggestion
(pool.getPoolType() != Storage.StoragePoolType.CLVM &&
```
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/wrapper/LibvirtClvmLockTransferCommandWrapper.java:
##########
@@ -0,0 +1,90 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package com.cloud.hypervisor.kvm.resource.wrapper;
+
+import org.apache.logging.log4j.Logger;
+import org.apache.logging.log4j.LogManager;
+
+import com.cloud.agent.api.Answer;
+import org.apache.cloudstack.storage.command.ClvmLockTransferCommand;
+import com.cloud.hypervisor.kvm.resource.LibvirtComputingResource;
+import com.cloud.resource.CommandWrapper;
+import com.cloud.resource.ResourceWrapper;
+import com.cloud.utils.script.Script;
+
+@ResourceWrapper(handles = ClvmLockTransferCommand.class)
+public class LibvirtClvmLockTransferCommandWrapper
+ extends CommandWrapper<ClvmLockTransferCommand, Answer, LibvirtComputingResource> {
+
+ protected Logger logger = LogManager.getLogger(getClass());
+
+ @Override
+ public Answer execute(ClvmLockTransferCommand cmd, LibvirtComputingResource serverResource) {
+ String lvPath = cmd.getLvPath();
+ ClvmLockTransferCommand.Operation operation = cmd.getOperation();
+ String volumeUuid = cmd.getVolumeUuid();
+
+ logger.info(String.format("Executing CLVM lock transfer: operation=%s, lv=%s, volume=%s",
+ operation, lvPath, volumeUuid));
+
+ try {
+ String lvchangeOpt;
+ String operationDesc;
+ switch (operation) {
+ case DEACTIVATE:
+ lvchangeOpt = "-an";
+ operationDesc = "deactivated";
+ break;
+ case ACTIVATE_EXCLUSIVE:
+ lvchangeOpt = "-aey";
+ operationDesc = "activated exclusively";
+ break;
+ case ACTIVATE_SHARED:
+ lvchangeOpt = "-asy";
+ operationDesc = "activated in shared mode";
+ break;
+ default:
+ return new Answer(cmd, false, "Unknown operation: " + operation);
+ }
+
+ Script script = new Script("/usr/sbin/lvchange", 30000, logger);
+ script.add(lvchangeOpt);
+ script.add(lvPath);
+
+ String result = script.execute();
+
+ if (result != null) {
+ logger.error("CLVM lock transfer failed for volume {}: {}}",
+ volumeUuid, result);
+ return new Answer(cmd, false,
+ String.format("lvchange %s %s failed: %s", lvchangeOpt, lvPath, result));
+ }
+
+ logger.info("Successfully executed CLVM lock transfer: {} {}} for volume {}}",
+ lvchangeOpt, lvPath, volumeUuid);
+
+ return new Answer(cmd, true,
+ String.format("Successfully %s CLVM volume %s", operationDesc, volumeUuid));
+
+ } catch (Exception e) {
+ logger.error("Exception during CLVM lock transfer for volume {}: {}}",
Review Comment:
Log message placeholder has an extra `}` (`{}}`), resulting in malformed output. Replace `{}}` with `{}` (and similarly fix the other affected log lines in this class).
```suggestion
logger.error("CLVM lock transfer failed for volume {}: {}",
volumeUuid, result);
return new Answer(cmd, false,
String.format("lvchange %s %s failed: %s", lvchangeOpt, lvPath, result));
}
logger.info("Successfully executed CLVM lock transfer: {} {} for volume {}",
lvchangeOpt, lvPath, volumeUuid);
return new Answer(cmd, true,
String.format("Successfully %s CLVM volume %s", operationDesc, volumeUuid));
} catch (Exception e) {
logger.error("Exception during CLVM lock transfer for volume {}: {}",
```
##########
engine/storage/volume/src/main/java/org/apache/cloudstack/storage/datastore/provider/DefaultHostListener.java:
##########
@@ -37,12 +37,14 @@
import com.cloud.network.dao.NetworkVO;
import com.cloud.offerings.NetworkOfferingVO;
import com.cloud.offerings.dao.NetworkOfferingDao;
+import com.cloud.storage.ClvmLockManager;
import com.cloud.storage.DataStoreRole;
import com.cloud.storage.Storage;
import com.cloud.storage.StorageManager;
import com.cloud.storage.StoragePool;
import com.cloud.storage.StoragePoolHostVO;
import com.cloud.storage.StorageService;
+import com.cloud.storage.VolumeApiServiceImpl;
import com.cloud.storage.dao.StoragePoolHostDao;
Review Comment:
`cloud-engine-storage-volume` does not have a dependency on `cloud-server`, but this class imports `com.cloud.storage.ClvmLockManager` and `com.cloud.storage.VolumeApiServiceImpl` (both live under `server/`). This will fail compilation for the engine module. To fix, avoid referencing server-layer classes here (e.g., move `CLVMSecureZeroFill` ConfigKey and the CLVM pool-type helper into an api/core module, or query the config via configuration DAOs available in this module).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4074520824
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [8.5% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4400168734
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4393633696
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4399296933
<b>[SF] Trillian test result (tid-16034)</b>
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 51129 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12617-t16034-kvm-ol8.zip
Smoke tests completed. 151 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4391434445
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4390413928
@Pearl1594 a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4327053668
@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-3945593477
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [2.6% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4407654599
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4408177334
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17793
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4416681750
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4426014591
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4245905589
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4341557881
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17648
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4346190525
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [33.2% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4336379063
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [33.5% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4443803297
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4652038881
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "winterhazel (via GitHub)" <gi...@apache.org>.
winterhazel commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r3383434984
##########
server/src/main/java/com/cloud/storage/VolumeApiServiceImpl.java:
##########
@@ -2748,6 +2777,204 @@ private Volume orchestrateAttachVolumeToVM(Long vmId, Long volumeId, Long device
return newVol;
}
+ /**
+ * Helper method to get storage pools for volume and VM.
+ *
+ * @param volumeToAttach The volume being attached
+ * @param vmExistingVolume The VM's existing volume
+ * @return Pair of StoragePoolVO objects (volumePool, vmPool), or null if either pool is missing
+ */
+ private Pair<StoragePoolVO, StoragePoolVO> getStoragePoolsForVolumeAttachment(VolumeInfo volumeToAttach, VolumeVO vmExistingVolume) {
+ if (volumeToAttach == null || vmExistingVolume == null) {
+ return null;
+ }
+
+ StoragePoolVO volumePool = _storagePoolDao.findById(volumeToAttach.getPoolId());
+ StoragePoolVO vmPool = _storagePoolDao.findById(vmExistingVolume.getPoolId());
+
+ if (volumePool == null || vmPool == null) {
+ return null;
+ }
+
+ return new Pair<>(volumePool, vmPool);
+ }
+
+ /**
+ * Checks if both storage pools are CLVM type (CLVM or CLVM_NG).
+ * Delegates to VolumeService for the actual check.
+ *
+ * @param volumePool Storage pool for the volume
+ * @param vmPool Storage pool for the VM
+ * @return true if both pools are CLVM type (CLVM or CLVM_NG)
+ */
+ private boolean areBothPoolsClvmType(StoragePoolVO volumePool, StoragePoolVO vmPool) {
+ return volService.areBothPoolsClvmType(volumePool.getPoolType(), vmPool.getPoolType());
+ }
+
+ /**
+ * Determines if a CLVM volume needs lightweight lock migration instead of full data copy.
+ *
+ * Lightweight migration is needed when:
+ * 1. Volume is on CLVM storage
+ * 2. Source and destination are in the same Volume Group
+ * 3. Only the host/lock needs to change (not the storage pool)
+ *
+ * @param volumeToAttach The volume being attached
+ * @param vmExistingVolume The VM's existing volume (typically root volume)
+ * @param vm The VM to attach the volume to
+ * @return true if lightweight CLVM lock migration should be used
+ */
+ private boolean isClvmLightweightMigrationNeeded(VolumeInfo volumeToAttach, VolumeVO vmExistingVolume, UserVmVO vm) {
+ Pair<StoragePoolVO, StoragePoolVO> pools = getStoragePoolsForVolumeAttachment(volumeToAttach, vmExistingVolume);
+ if (pools == null) {
+ return false;
+ }
+
+ StoragePoolVO volumePool = pools.first();
+ StoragePoolVO vmPool = pools.second();
+
+ return volService.isLightweightMigrationNeeded(volumePool.getPoolType(), vmPool.getPoolType(),
+ volumePool.getPath(), vmPool.getPath());
+ }
+
+ /**
+ * Determines if a CLVM volume requires lock transfer when already on the correct storage pool.
+ *
+ * Lock transfer is needed when:
+ * 1. Volume is already on the same CLVM storage pool as VM's volumes
+ * 2. But the volume lock is held by a different host than where the VM is running
+ * 3. Only the lock needs to change (no pool change, no data copy)
+ *
+ * @param volumeToAttach The volume being attached
+ * @param vmExistingVolume The VM's existing volume (typically root volume)
+ * @param vm The VM to attach the volume to
+ * @return true if CLVM lock transfer is needed (but not full migration)
+ */
+ private boolean isClvmLockTransferRequired(VolumeInfo volumeToAttach, VolumeVO vmExistingVolume, UserVmVO vm) {
+ if (vm == null) {
+ return false;
+ }
+
+ Pair<StoragePoolVO, StoragePoolVO> pools = getStoragePoolsForVolumeAttachment(volumeToAttach, vmExistingVolume);
+ if (pools == null) {
+ return false;
+ }
+
+ StoragePoolVO volumePool = pools.first();
+ StoragePoolVO vmPool = pools.second();
+
+ Long vmHostId = vm.getHostId();
+ if (vmHostId == null) {
+ vmHostId = vm.getLastHostId();
+ }
+
+ return volService.isLockTransferRequired(volumeToAttach, volumePool.getPoolType(), vmPool.getPoolType(),
+ volumePool.getId(), vmPool.getId(), vmHostId);
+ }
+
+ /**
+ * Determines the destination host for CLVM lock migration.
+ *
+ * If VM is running, uses the VM's current host.
+ * If VM is stopped, picks an available UP host from the storage pool's cluster.
+ *
+ * @param vm The VM
+ * @param vmExistingVolume The VM's existing volume (to determine cluster)
+ * @return Host ID, or null if cannot be determined
+ */
+ private Long determineClvmLockDestinationHost(UserVmVO vm, VolumeVO vmExistingVolume) {
+ Long destHostId = vm.getHostId();
+ if (destHostId != null) {
+ return destHostId;
+ }
+
+ if (vmExistingVolume != null && vmExistingVolume.getPoolId() != null) {
+ StoragePoolVO pool = _storagePoolDao.findById(vmExistingVolume.getPoolId());
+ if (pool != null && pool.getClusterId() != null) {
+ List<HostVO> hosts = _hostDao.findByClusterId(pool.getClusterId());
+ if (hosts != null && !hosts.isEmpty()) {
+ // Pick first available UP host
+ for (HostVO host : hosts) {
+ if (host.getStatus() == Status.Up) {
+ destHostId = host.getId();
+ logger.debug("VM {} is stopped, selected host {} from cluster {} for CLVM lock migration",
+ vm.getUuid(), destHostId, pool.getClusterId());
+ return destHostId;
+ }
+ }
+ }
+ }
+ }
+
+ return null;
+ }
+
+ /**
+ * Executes CLVM lightweight migration with consistent logging and error handling.
+ *
+ * This helper method wraps the actual migration logic to eliminate code duplication
+ * between different CLVM migration scenarios (lock transfer vs. lightweight migration).
+ *
+ * @param volume The volume to migrate locks for
+ * @param vm The VM to attach the volume to
+ * @param vmExistingVolume The VM's existing volume (to determine target host)
+ * @param operationType Description of the operation type for logging (e.g., "CLVM lock transfer")
+ * @param scenarioDescription Description of the scenario for logging (e.g., "same pool to different host")
+ * @return Updated VolumeInfo after lock migration
+ * @throws CloudRuntimeException if migration fails
+ */
+ private VolumeInfo executeLightweightLockMigration(VolumeInfo volume, UserVmVO vm, VolumeVO vmExistingVolume,
+ String operationType, String scenarioDescription) {
+ logger.info("Performing {} for volume {} to VM {} ({})",
+ operationType, volume.getUuid(), vm.getUuid(), scenarioDescription);
+
+ try {
+ return performLightweightLockMigration(volume, vm, vmExistingVolume);
+ } catch (Exception e) {
+ logger.error("{} failed for volume {}: {}",
+ operationType, volume.getUuid(), e.getMessage(), e);
+ throw new CloudRuntimeException(operationType + " failed", e);
+ }
+ }
+
+ /**
+ * Performs lightweight CLVM lock migration for volume attachment.
+ * Delegates to VolumeService for the actual lock migration.
+ *
+ * @param volume The volume to migrate locks for
+ * @param vm The VM to attach the volume to
+ * @param vmExistingVolume The VM's existing volume (to determine target host)
+ * @return Updated VolumeInfo after lock migration
+ * @throws Exception if lock migration fails
+ */
+ private VolumeInfo performLightweightLockMigration(VolumeInfo volume, UserVmVO vm, VolumeVO vmExistingVolume) throws CloudRuntimeException {
+ Long destHostId = determineClvmLockDestinationHost(vm, vmExistingVolume);
+
+ if (destHostId == null) {
+ throw new CloudRuntimeException(
+ "Cannot determine destination host for CLVM lock migration - VM has no host and no available cluster hosts");
+ }
+
+ try {
+ return volService.performLockMigration(volume, destHostId);
+ } catch (CloudRuntimeException e) {
+ logger.error("CLVM lock migration failed for volume {}: {}", volume.getUuid(), e.getMessage(), e);
+ throw e;
+ }
+ }
+
+ /**
+ * Finds which host currently has the exclusive lock on a CLVM volume.
+ * Delegates to VolumeService for the actual lookup.
+ *
+ * @param volume The CLVM volume
+ * @return Host ID that has the exclusive lock, or null if cannot be determined
+ */
+ private Long findVolumeLockHost(VolumeInfo volume) {
Review Comment:
Ping @Pearl1594, could you have a look? This method is not used
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4664392780
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [34.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
<!-- sqra:qg_revision=d3af8bffd7d31ca4478e55db4f10c80bb2804d3e37486e2931e72f8e4b860038 -->
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4632967182
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [33.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
<!-- sqra:qg_revision=54698d2cf11f65d11c8f16d5a691c8488d3eab5f8d3a73ef129cff49d21ff92d -->
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r3389667999
##########
engine/storage/datamotion/src/main/java/org/apache/cloudstack/storage/motion/StorageSystemDataMotionStrategy.java:
##########
@@ -2320,7 +2434,26 @@ String getVolumeBackingFile(VolumeInfo srcVolumeInfo) {
return null;
}
- private void handlePostMigration(boolean success, Map<VolumeInfo, VolumeInfo> srcVolumeInfoToDestVolumeInfo, VirtualMachineTO vmTO, Host destHost) {
+ private void sendClvmLockCommand(long hostId, StoragePoolVO pool, VolumeInfo volumeInfo,
+ ClvmLockTransferCommand.Operation operation) {
+ String vgName = pool.getPath();
+ if (vgName.startsWith("/")) {
+ vgName = vgName.substring(1);
+ }
+ String lvPath = String.format("/dev/%s/%s", vgName, volumeInfo.getPath());
+ try {
+ Answer answer = agentManager.send(hostId,
+ new ClvmLockTransferCommand(operation, lvPath, volumeInfo.getUuid()));
+ if (answer == null || !answer.getResult()) {
+ String details = answer != null ? answer.getDetails() : "null answer";
+ logger.warn("CLVM lock command [{}] failed for LV [{}] on host [{}]: {}", operation, lvPath, hostId, details);
Review Comment:
The locking here is not about making the device accessible for the migration itself, that's handled earlier (pre-migration ACTIVATE_SHARED, which hard-fails). These post-migration ACTIVATE_EXCLUSIVE calls are state normalization.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "winterhazel (via GitHub)" <gi...@apache.org>.
winterhazel commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r3383645679
##########
server/src/main/java/com/cloud/storage/snapshot/SnapshotManagerImpl.java:
##########
@@ -1659,6 +1662,9 @@ public SnapshotInfo takeSnapshot(VolumeInfo volume) throws ResourceAllocationExc
if (backupSnapToSecondary) {
if (!isKvmAndFileBasedStorage) {
backupSnapshotToSecondary(payload.getAsyncBackup(), snapshotStrategy, snapshotOnPrimary, payload.getZoneIds(), payload.getStoragePoolIds());
+ if (!payload.getAsyncBackup() && (storagePool.getPoolType() == StoragePoolType.CLVM || storagePool.getPoolType() == StoragePoolType.CLVM_NG)) {
Review Comment:
Use `ClvmPoolManager.isClvmPoolType` here
##########
server/src/main/java/com/cloud/storage/VolumeApiServiceImpl.java:
##########
@@ -2738,21 +2746,42 @@ private Volume orchestrateAttachVolumeToVM(Long vmId, Long volumeId, Long device
logger.trace(String.format("is it needed to move the volume: %b?", moveVolumeNeeded));
}
- if (moveVolumeNeeded) {
+ // Check if CLVM lock transfer is needed (even if moveVolumeNeeded is false)
+ // This handles the case where the volume is already on the correct storage pool
+ // but the VM is running on a different host, requiring only a lock transfer
+ boolean isClvmLockTransferNeeded = !moveVolumeNeeded &&
+ isClvmLockTransferRequired(newVolumeOnPrimaryStorage, existingVolumeOfVm, vm);
+
+ if (isClvmLockTransferNeeded) {
+ // CLVM lock transfer - no data copy, no pool change needed
+ newVolumeOnPrimaryStorage = executeLightweightLockMigration(
+ newVolumeOnPrimaryStorage, vm, existingVolumeOfVm,
+ "CLVM lock transfer", "same pool to different host");
+ } else if (moveVolumeNeeded) {
PrimaryDataStoreInfo primaryStore = (PrimaryDataStoreInfo)newVolumeOnPrimaryStorage.getDataStore();
if (primaryStore.isLocal()) {
throw new CloudRuntimeException(
"Failed to attach local data volume " + volumeToAttach.getName() + " to VM " + vm.getDisplayName() + " as migration of local data volume is not allowed");
}
- StoragePoolVO vmRootVolumePool = _storagePoolDao.findById(existingVolumeOfVm.getPoolId());
- try {
- HypervisorType volumeToAttachHyperType = _volsDao.getHypervisorType(volumeToAttach.getId());
- newVolumeOnPrimaryStorage = _volumeMgr.moveVolume(newVolumeOnPrimaryStorage, vmRootVolumePool.getDataCenterId(), vmRootVolumePool.getPodId(), vmRootVolumePool.getClusterId(),
- volumeToAttachHyperType);
- } catch (ConcurrentOperationException | StorageUnavailableException e) {
- logger.debug("move volume failed", e);
- throw new CloudRuntimeException("move volume failed", e);
+ boolean isClvmLightweightMigration = isClvmLightweightMigrationNeeded(
+ newVolumeOnPrimaryStorage, existingVolumeOfVm, vm);
+
+ if (isClvmLightweightMigration) {
+ newVolumeOnPrimaryStorage = executeLightweightLockMigration(
+ newVolumeOnPrimaryStorage, vm, existingVolumeOfVm,
+ "CLVM lightweight migration", "different pools, same VG");
+ } else {
+ StoragePoolVO vmRootVolumePool = _storagePoolDao.findById(existingVolumeOfVm.getPoolId());
+
+ try {
+ HypervisorType volumeToAttachHyperType = _volsDao.getHypervisorType(volumeToAttach.getId());
+ newVolumeOnPrimaryStorage = _volumeMgr.moveVolume(newVolumeOnPrimaryStorage, vmRootVolumePool.getDataCenterId(), vmRootVolumePool.getPodId(), vmRootVolumePool.getClusterId(),
+ volumeToAttachHyperType);
+ } catch (ConcurrentOperationException | StorageUnavailableException e) {
+ logger.debug("move volume failed", e);
+ throw new CloudRuntimeException("move volume failed", e);
+ }
}
Review Comment:
This should be a straightfoward refactor. I would prefer having it on this one already, but its up to you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4673923666
<b>[SF] Trillian test result (tid-16274)</b>
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 56161 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12617-t16274-kvm-ol8.zip
Smoke tests completed. 151 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4674421208
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [33.4% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
<!-- sqra:qg_revision=472cdf7db613205e6d2c4744c7816053eea17f61ac6e3c845dbb5942e4ac4394 -->
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4700314301
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4070594702
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4091801868
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [8.2% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-3955782470
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [2.5% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4238729448
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4498691130
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17936
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r3154354092
##########
engine/api/src/main/java/org/apache/cloudstack/engine/subsystem/api/storage/VolumeInfo.java:
##########
@@ -31,6 +31,12 @@
public interface VolumeInfo extends DownloadableDataInfo, Volume {
+ /**
+ * Constant for the volume detail key that stores the host ID currently holding the CLVM exclusive lock.
+ * This is used during lightweight lock migration to determine the source host for lock transfer.
+ */
+ String CLVM_LOCK_HOST_ID = "clvmLockHostId";
Review Comment:
move to ClvmLockManager
```suggestion
public static final String CLVM_LOCK_HOST_ID = "clvmLockHostId";
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r3166043834
##########
server/src/main/java/com/cloud/storage/VolumeApiServiceImpl.java:
##########
@@ -2748,6 +2777,204 @@ private Volume orchestrateAttachVolumeToVM(Long vmId, Long volumeId, Long device
return newVol;
}
+ /**
+ * Helper method to get storage pools for volume and VM.
+ *
+ * @param volumeToAttach The volume being attached
+ * @param vmExistingVolume The VM's existing volume
+ * @return Pair of StoragePoolVO objects (volumePool, vmPool), or null if either pool is missing
+ */
+ private Pair<StoragePoolVO, StoragePoolVO> getStoragePoolsForVolumeAttachment(VolumeInfo volumeToAttach, VolumeVO vmExistingVolume) {
+ if (volumeToAttach == null || vmExistingVolume == null) {
+ return null;
+ }
+
+ StoragePoolVO volumePool = _storagePoolDao.findById(volumeToAttach.getPoolId());
+ StoragePoolVO vmPool = _storagePoolDao.findById(vmExistingVolume.getPoolId());
+
+ if (volumePool == null || vmPool == null) {
+ return null;
+ }
+
+ return new Pair<>(volumePool, vmPool);
+ }
+
+ /**
+ * Checks if both storage pools are CLVM type (CLVM or CLVM_NG).
+ * Delegates to VolumeService for the actual check.
+ *
+ * @param volumePool Storage pool for the volume
+ * @param vmPool Storage pool for the VM
+ * @return true if both pools are CLVM type (CLVM or CLVM_NG)
+ */
+ private boolean areBothPoolsClvmType(StoragePoolVO volumePool, StoragePoolVO vmPool) {
+ return volService.areBothPoolsClvmType(volumePool.getPoolType(), vmPool.getPoolType());
+ }
+
+ /**
+ * Determines if a CLVM volume needs lightweight lock migration instead of full data copy.
+ *
+ * Lightweight migration is needed when:
+ * 1. Volume is on CLVM storage
+ * 2. Source and destination are in the same Volume Group
+ * 3. Only the host/lock needs to change (not the storage pool)
+ *
+ * @param volumeToAttach The volume being attached
+ * @param vmExistingVolume The VM's existing volume (typically root volume)
+ * @param vm The VM to attach the volume to
+ * @return true if lightweight CLVM lock migration should be used
+ */
+ private boolean isClvmLightweightMigrationNeeded(VolumeInfo volumeToAttach, VolumeVO vmExistingVolume, UserVmVO vm) {
+ Pair<StoragePoolVO, StoragePoolVO> pools = getStoragePoolsForVolumeAttachment(volumeToAttach, vmExistingVolume);
+ if (pools == null) {
+ return false;
+ }
+
+ StoragePoolVO volumePool = pools.first();
+ StoragePoolVO vmPool = pools.second();
+
+ return volService.isLightweightMigrationNeeded(volumePool.getPoolType(), vmPool.getPoolType(),
+ volumePool.getPath(), vmPool.getPath());
+ }
+
+ /**
+ * Determines if a CLVM volume requires lock transfer when already on the correct storage pool.
+ *
+ * Lock transfer is needed when:
+ * 1. Volume is already on the same CLVM storage pool as VM's volumes
+ * 2. But the volume lock is held by a different host than where the VM is running
+ * 3. Only the lock needs to change (no pool change, no data copy)
+ *
+ * @param volumeToAttach The volume being attached
+ * @param vmExistingVolume The VM's existing volume (typically root volume)
+ * @param vm The VM to attach the volume to
+ * @return true if CLVM lock transfer is needed (but not full migration)
+ */
+ private boolean isClvmLockTransferRequired(VolumeInfo volumeToAttach, VolumeVO vmExistingVolume, UserVmVO vm) {
+ if (vm == null) {
+ return false;
+ }
+
+ Pair<StoragePoolVO, StoragePoolVO> pools = getStoragePoolsForVolumeAttachment(volumeToAttach, vmExistingVolume);
+ if (pools == null) {
+ return false;
+ }
+
+ StoragePoolVO volumePool = pools.first();
+ StoragePoolVO vmPool = pools.second();
+
+ Long vmHostId = vm.getHostId();
+ if (vmHostId == null) {
+ vmHostId = vm.getLastHostId();
+ }
+
+ return volService.isLockTransferRequired(volumeToAttach, volumePool.getPoolType(), vmPool.getPoolType(),
+ volumePool.getId(), vmPool.getId(), vmHostId);
+ }
+
+ /**
+ * Determines the destination host for CLVM lock migration.
+ *
+ * If VM is running, uses the VM's current host.
+ * If VM is stopped, picks an available UP host from the storage pool's cluster.
+ *
+ * @param vm The VM
+ * @param vmExistingVolume The VM's existing volume (to determine cluster)
+ * @return Host ID, or null if cannot be determined
+ */
+ private Long determineClvmLockDestinationHost(UserVmVO vm, VolumeVO vmExistingVolume) {
+ Long destHostId = vm.getHostId();
+ if (destHostId != null) {
+ return destHostId;
+ }
+
+ if (vmExistingVolume != null && vmExistingVolume.getPoolId() != null) {
+ StoragePoolVO pool = _storagePoolDao.findById(vmExistingVolume.getPoolId());
+ if (pool != null && pool.getClusterId() != null) {
+ List<HostVO> hosts = _hostDao.findByClusterId(pool.getClusterId());
+ if (hosts != null && !hosts.isEmpty()) {
+ // Pick first available UP host
+ for (HostVO host : hosts) {
+ if (host.getStatus() == Status.Up) {
+ destHostId = host.getId();
+ logger.debug("VM {} is stopped, selected host {} from cluster {} for CLVM lock migration",
+ vm.getUuid(), destHostId, pool.getClusterId());
+ return destHostId;
+ }
+ }
+ }
+ }
+ }
+
+ return null;
+ }
+
+ /**
+ * Executes CLVM lightweight migration with consistent logging and error handling.
+ *
+ * This helper method wraps the actual migration logic to eliminate code duplication
+ * between different CLVM migration scenarios (lock transfer vs. lightweight migration).
+ *
+ * @param volume The volume to migrate locks for
+ * @param vm The VM to attach the volume to
+ * @param vmExistingVolume The VM's existing volume (to determine target host)
+ * @param operationType Description of the operation type for logging (e.g., "CLVM lock transfer")
+ * @param scenarioDescription Description of the scenario for logging (e.g., "same pool to different host")
+ * @return Updated VolumeInfo after lock migration
+ * @throws CloudRuntimeException if migration fails
+ */
+ private VolumeInfo executeLightweightLockMigration(VolumeInfo volume, UserVmVO vm, VolumeVO vmExistingVolume,
+ String operationType, String scenarioDescription) {
+ logger.info("Performing {} for volume {} to VM {} ({})",
+ operationType, volume.getUuid(), vm.getUuid(), scenarioDescription);
+
+ try {
+ return performLightweightLockMigration(volume, vm, vmExistingVolume);
+ } catch (Exception e) {
+ logger.error("{} failed for volume {}: {}",
+ operationType, volume.getUuid(), e.getMessage(), e);
+ throw new CloudRuntimeException(operationType + " failed", e);
+ }
+ }
+
+ /**
+ * Performs lightweight CLVM lock migration for volume attachment.
+ * Delegates to VolumeService for the actual lock migration.
+ *
+ * @param volume The volume to migrate locks for
+ * @param vm The VM to attach the volume to
+ * @param vmExistingVolume The VM's existing volume (to determine target host)
+ * @return Updated VolumeInfo after lock migration
+ * @throws Exception if lock migration fails
+ */
+ private VolumeInfo performLightweightLockMigration(VolumeInfo volume, UserVmVO vm, VolumeVO vmExistingVolume) throws CloudRuntimeException {
+ Long destHostId = determineClvmLockDestinationHost(vm, vmExistingVolume);
+
+ if (destHostId == null) {
+ throw new CloudRuntimeException(
+ "Cannot determine destination host for CLVM lock migration - VM has no host and no available cluster hosts");
+ }
+
+ try {
+ return volService.performLockMigration(volume, destHostId);
+ } catch (CloudRuntimeException e) {
+ logger.error("CLVM lock migration failed for volume {}: {}", volume.getUuid(), e.getMessage(), e);
+ throw e;
+ }
+ }
+
+ /**
+ * Finds which host currently has the exclusive lock on a CLVM volume.
+ * Delegates to VolumeService for the actual lookup.
+ *
+ * @param volume The CLVM volume
+ * @return Host ID that has the exclusive lock, or null if cannot be determined
+ */
+ private Long findVolumeLockHost(VolumeInfo volume) {
Review Comment:
@Pearl1594 this method seems unused, please check.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4518329709
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4554753372
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [33.5% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
<!-- sqra:qg_revision=fbd5e701df238503f6cd6ca48d7a6fb8c4655e56a6e9aa7cb3bc9424049ebeee -->
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "harikrishna-patnala (via GitHub)" <gi...@apache.org>.
harikrishna-patnala merged PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4702862999
<b>[SF] Trillian test result (tid-16306)</b>
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 50471 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12617-t16306-kvm-ol8.zip
Smoke tests completed. 151 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4632908100
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18165
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4632980499
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4632533444
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r3382789400
##########
engine/orchestration/src/main/java/com/cloud/vm/VirtualMachineManagerImpl.java:
##########
@@ -3320,6 +3334,29 @@ protected void migrate(final VMInstanceVO vm, final long srcHostId, final Deploy
}
}
+ private void executePostMigrationCommand(VMInstanceVO vm, VirtualMachineTO to, long dstHostId) {
+ if (vm.getHypervisorType() == HypervisorType.KVM && hasClvmVolumes(vm.getId())) {
+ try {
+ logger.info("Executing post-migration tasks for VM {} with CLVM volumes on destination host {}", vm.getInstanceName(), dstHostId);
+ final PostMigrationCommand postMigrationCommand = new PostMigrationCommand(to, vm.getInstanceName());
+ final Answer postMigrationAnswer = _agentMgr.send(dstHostId, postMigrationCommand);
+
+ if (postMigrationAnswer == null || !postMigrationAnswer.getResult()) {
+ final String details = postMigrationAnswer != null ? postMigrationAnswer.getDetails() : "null answer returned";
+ logger.warn("Post-migration tasks failed for VM {} on destination host {}: {}. Migration completed but some cleanup may be needed.",
+ vm.getInstanceName(), dstHostId, details);
+ } else {
+ logger.info("Successfully completed post-migration tasks for VM {} on destination host {}", vm.getInstanceName(), dstHostId);
+ }
+ } catch (Exception e) {
+ logger.warn("Exception during post-migration tasks for VM {} on destination host {}: {}. Migration completed but some cleanup may be needed.",
+ vm.getInstanceName(), dstHostId, e.getMessage(), e);
+ }
+ }
+
+ updateClvmLockHostForVmVolumes(vm.getId(), dstHostId);
Review Comment:
if it does, in the next re-try it will get corrected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4664728859
<b>[SF] Trillian test result (tid-16265)</b>
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 94885 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12617-t16265-kvm-ol8.zip
Smoke tests completed. 136 look OK, 15 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_CRUD_operations_userdata | `Error` | 1522.99 | test_register_userdata.py
test_deploy_vm_with_registered_userdata | `Error` | 5.58 | test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_allow | `Error` | 5.55 | test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_append | `Error` | 6.48 | test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_deny | `Error` | 5.56 | test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_params | `Error` | 6.65 | test_register_userdata.py
test_link_and_unlink_userdata_to_template | `Error` | 5.73 | test_register_userdata.py
test_user_userdata_crud | `Error` | 9.72 | test_register_userdata.py
ContextSuite context=TestResetVmOnReboot>:setup | `Error` | 0.00 | test_reset_vm_on_reboot.py
ContextSuite context=TestRAMCPUResourceAccounting>:setup | `Error` | 0.00 | test_resource_accounting.py
test_03_register_template | `Error` | 1.13 | test_resource_names.py
ContextSuite context=TestRestoreVM>:setup | `Error` | 0.00 | test_restore_vm.py
ContextSuite context=TestRouterDHCPHosts>:setup | `Error` | 0.00 | test_router_dhcphosts.py
ContextSuite context=TestRouterDHCPOpts>:setup | `Error` | 0.00 | test_router_dhcphosts.py
ContextSuite context=TestRouterDns>:setup | `Error` | 0.00 | test_router_dns.py
ContextSuite context=TestRouterDnsService>:setup | `Error` | 0.00 | test_router_dnsservice.py
ContextSuite context=TestRouterIpTablesPolicies>:setup | `Error` | 0.00 | test_routers_iptables_default_policy.py
ContextSuite context=TestVPCIpTablesPolicies>:setup | `Error` | 0.00 | test_routers_iptables_default_policy.py
ContextSuite context=TestIsolatedNetworks>:setup | `Error` | 0.00 | test_routers_network_ops.py
ContextSuite context=TestRedundantIsolateNetworks>:setup | `Error` | 0.00 | test_routers_network_ops.py
ContextSuite context=TestRouterServices>:setup | `Error` | 0.00 | test_routers.py
ContextSuite context=TestCpuCapServiceOfferings>:setup | `Error` | 0.00 | test_service_offerings.py
ContextSuite context=TestServiceOfferings>:setup | `Error` | 0.30 | test_service_offerings.py
ContextSuite context=TestSetSourceNatIp>:setup | `Error` | 0.00 | test_set_sourcenat.py
ContextSuite context=TestSnapshotRootDisk>:setup | `Error` | 0.00 | test_snapshots.py
ContextSuite context=TestSnapshotStandaloneBackup>:setup | `Error` | 0.00 | test_snapshots.py
ContextSuite context=TestSslOffloading>:setup | `Error` | 0.00 | test_ssl_offloading.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4435766603
<b>[SF] Trillian test result (tid-16072)</b>
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 84618 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12617-t16072-kvm-ol8.zip
Smoke tests completed. 146 look OK, 5 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_vm_backup_create_vm_from_backup | `Failure` | 607.91 | test_backup_recovery_nas.py
test_vm_backup_lifecycle | `Error` | 0.09 | test_backup_recovery_nas.py
ContextSuite context=TestSharedFSLifecycle>:setup | `Error` | 0.00 | test_sharedfs_lifecycle.py
test_10_attachAndDetach_iso | `Failure` | 607.04 | test_vm_life_cycle.py
test_01_create_vm_snapshots | `Failure` | 606.78 | test_vm_snapshots.py
test_02_revert_vm_snapshots | `Failure` | 600.71 | test_vm_snapshots.py
test_03_delete_vm_snapshots | `Failure` | 0.02 | test_vm_snapshots.py
test_01_create_volume | `Failure` | 611.34 | test_volumes.py
test_01_root_volume_encryption | `Failure` | 700.08 | test_volumes.py
test_02_data_volume_encryption | `Failure` | 640.27 | test_volumes.py
test_03_root_and_data_volume_encryption | `Failure` | 664.37 | test_volumes.py
test_02_attach_volume | `Failure` | 1270.96 | test_volumes.py
test_02_attach_volume | `Failure` | 1270.98 | test_volumes.py
test_03_download_attached_volume | `Failure` | 662.27 | test_volumes.py
test_04_delete_attached_volume | `Failure` | 664.38 | test_volumes.py
test_05_detach_volume | `Failure` | 756.41 | test_volumes.py
test_06_download_detached_volume | `Failure` | 846.50 | test_volumes.py
test_07_resize_fail | `Failure` | 661.00 | test_volumes.py
test_08_resize_volume | `Failure` | 670.58 | test_volumes.py
test_09_delete_detached_volume | `Failure` | 662.29 | test_volumes.py
test_10_list_volumes | `Failure` | 658.02 | test_volumes.py
test_11_attach_volume_with_unstarted_vm | `Failure` | 759.75 | test_volumes.py
test_12_resize_volume_with_only_size_parameter | `Failure` | 670.50 | test_volumes.py
test_13_migrate_volume_and_change_offering | `Failure` | 795.75 | test_volumes.py
test_14_delete_volume_delete_protection | `Failure` | 666.67 | test_volumes.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4443811789
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4350569296
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [33.2% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4456288893
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4456414519
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [33.5% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
<!-- sqra:qg_revision=2a37d2b5e7f9615b9fd051509ea3b98a8af2150f049b6a848e7cc8ec203726ba -->
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4456189947
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4203780530
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4666164097
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4652659318
@Pearl1594 a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4697952186
<b>[SF] Trillian test result (tid-16300)</b>
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 50741 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12617-t16300-kvm-ol8.zip
Smoke tests completed. 151 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4693307426
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18241
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4693507929
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4700659012
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4700747816
@Pearl1594 a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Luskan777 (via GitHub)" <gi...@apache.org>.
Luskan777 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4393512566
Hi @Pearl1594 ,
I'm a little confused. Is the image type for CLVM-NG QCOW2 or RAW? Or can the user choose the image format?
I was reading the link from the proposal designer, but I didn't find this information. The PR title confused me; as far as I know, LVM defaults to RAW.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4400173073
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r3154337015
##########
server/src/main/java/com/cloud/storage/ClvmLockManager.java:
##########
@@ -0,0 +1,334 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package com.cloud.storage;
Review Comment:
```suggestion
package com.cloud.storage.clvm;
```
?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r3162734534
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/LibvirtStorageAdaptor.java:
##########
@@ -693,11 +848,248 @@ public KVMPhysicalDisk getPhysicalDisk(String volumeUuid, KVMStoragePool pool) {
}
return disk;
} catch (LibvirtException e) {
- logger.debug("Failed to get physical disk:", e);
+ logger.debug("Failed to get volume from libvirt: " + e.getMessage());
+ if (isClvmPool) {
+ return getPhysicalDiskWithClvmFallback(volumeUuid, pool, libvirtPool);
+ }
+
throw new CloudRuntimeException(e.toString());
}
}
+ /**
+ * CLVM fallback: First tries to refresh libvirt pool to make volume visible,
+ * if that fails, accesses volume directly via block device path.
+ */
+ private KVMPhysicalDisk getPhysicalDiskWithClvmFallback(String volumeUuid, KVMStoragePool pool, LibvirtStoragePool libvirtPool) {
+ logger.info("CLVM volume not visible to libvirt, attempting pool refresh for volume: {}", volumeUuid);
+
+ try {
+ logger.debug("Refreshing libvirt storage pool: {}", pool.getUuid());
+ libvirtPool.getPool().refresh(0);
+
+ StorageVol vol = getVolume(libvirtPool.getPool(), volumeUuid);
+ if (vol != null) {
+ logger.info("Volume found after pool refresh: {}", volumeUuid);
+ KVMPhysicalDisk disk;
+ LibvirtStorageVolumeDef voldef = getStorageVolumeDef(libvirtPool.getPool().getConnect(), vol);
+ disk = new KVMPhysicalDisk(vol.getPath(), vol.getName(), pool);
+ disk.setSize(vol.getInfo().allocation);
+ disk.setVirtualSize(vol.getInfo().capacity);
+ disk.setFormat(voldef.getFormat() == LibvirtStorageVolumeDef.VolumeFormat.QCOW2 ?
+ PhysicalDiskFormat.QCOW2 : PhysicalDiskFormat.RAW);
+ return disk;
+ }
+ } catch (LibvirtException refreshEx) {
+ logger.debug("Pool refresh failed or volume still not found: {}", refreshEx.getMessage());
+ }
+
+ logger.info("Falling back to direct block device access for volume: {}", volumeUuid);
+ return getPhysicalDiskViaDirectBlockDevice(volumeUuid, pool);
+ }
+
+ private String getVgName(String sourceDir) {
+ String vgName = sourceDir;
+ if (vgName.startsWith("/")) {
+ String[] parts = vgName.split("/");
+ List<String> tokens = Arrays.stream(parts)
+ .filter(s -> !s.isEmpty()).collect(Collectors.toList());
+
+ vgName = tokens.size() > 1 ? tokens.get(1)
+ : tokens.size() == 1 ? tokens.get(0)
+ : "";
+ }
+ return vgName;
+ }
+
+ /**
+ * For CLVM volumes that exist in LVM but are not visible to libvirt,
+ * access them directly via block device path.
+ */
+ private KVMPhysicalDisk getPhysicalDiskViaDirectBlockDevice(String volumeUuid, KVMStoragePool pool) {
+ try {
+ String vgName = extractVgNameFromPool(pool);
+ verifyLvExistsInVg(volumeUuid, vgName);
+
+ logger.info("Volume {} exists in LVM but not visible to libvirt, accessing directly", volumeUuid);
+
+ String lvPath = findAccessibleDeviceNode(volumeUuid, vgName, pool);
+ long size = getClvmVolumeSize(lvPath);
+
+ KVMPhysicalDisk disk = createPhysicalDiskFromClvmLv(lvPath, volumeUuid, pool, size);
+
+ ensureTemplateAccessibility(volumeUuid, lvPath, pool);
+
+ return disk;
+
+ } catch (CloudRuntimeException ex) {
+ throw ex;
+ } catch (Exception ex) {
+ logger.error("Failed to access CLVM volume via direct block device: {}", volumeUuid, ex);
+ throw new CloudRuntimeException(String.format("Could not find volume %s: %s ", volumeUuid, ex.getMessage()));
+ }
+ }
+
+ private String extractVgNameFromPool(KVMStoragePool pool) {
+ String sourceDir = pool.getLocalPath();
+ if (sourceDir == null || sourceDir.isEmpty()) {
+ throw new CloudRuntimeException("CLVM pool sourceDir is not set, cannot determine VG name");
+ }
+ String vgName = getVgName(sourceDir);
+ logger.debug("Using VG name: {} (from sourceDir: {})", vgName, sourceDir);
+ return vgName;
+ }
+
+ private void verifyLvExistsInVg(String volumeUuid, String vgName) {
+ logger.debug("Checking if volume {} exists in VG {}", volumeUuid, vgName);
+ Script checkLvCmd = new Script("/usr/sbin/lvs", 5000, logger);
+ checkLvCmd.add("--noheadings");
+ checkLvCmd.add("--unbuffered");
+ checkLvCmd.add(vgName + "/" + volumeUuid);
+
+ String checkResult = checkLvCmd.execute();
+ if (checkResult != null) {
+ logger.debug("Volume {} does not exist in VG {}: {}", volumeUuid, vgName, checkResult);
+ throw new CloudRuntimeException(String.format("Storage volume not found: no storage vol with matching name '%s'", volumeUuid));
+ }
+ }
+
+ private String findAccessibleDeviceNode(String volumeUuid, String vgName, KVMStoragePool pool) {
+ String lvPath = "/dev/" + vgName + "/" + volumeUuid;
+ File lvDevice = new File(lvPath);
+
+ if (!lvDevice.exists()) {
+ lvPath = tryDeviceMapperPath(volumeUuid, vgName, lvDevice);
+
+ if (!lvDevice.exists()) {
+ lvPath = handleMissingDeviceNode(volumeUuid, vgName, pool);
+ }
+ }
+
+ return lvPath;
+ }
+
+ private String tryDeviceMapperPath(String volumeUuid, String vgName, File lvDevice) {
+ String vgNameEscaped = vgName.replace("-", "--");
+ String volumeUuidEscaped = volumeUuid.replace("-", "--");
+ String mapperPath = "/dev/mapper/" + vgNameEscaped + "-" + volumeUuidEscaped;
+ File mapperDevice = new File(mapperPath);
+
+ if (!mapperDevice.exists()) {
+ lvDevice = mapperDevice;
+ }
+
+ return mapperPath;
+ }
+
+ private String handleMissingDeviceNode(String volumeUuid, String vgName, KVMStoragePool pool) {
+ if (pool.getType() == StoragePoolType.CLVM_NG && volumeUuid.startsWith("template-")) {
+ return activateTemplateAndGetPath(volumeUuid, vgName);
+ } else {
+ logger.warn("Volume exists in LVM but device node not found: {}", volumeUuid);
+ throw new CloudRuntimeException(String.format("Could not find volume %s " +
+ "in VG %s - volume exists in LVM but device node not accessible", volumeUuid, vgName));
+ }
+ }
+
+ private String activateTemplateAndGetPath(String volumeUuid, String vgName) {
+ logger.info("Template volume {} device node not found. Attempting to activate in shared mode.", volumeUuid);
+ String templateLvPath = "/dev/" + vgName + "/" + volumeUuid;
+
+ try {
+ ensureTemplateLvInSharedMode(templateLvPath, false);
+
+ String lvPath = findDeviceNodeAfterActivation(templateLvPath, volumeUuid, vgName);
+
+ logger.info("Successfully activated template volume {} at {}", volumeUuid, lvPath);
+ return lvPath;
+
+ } catch (CloudRuntimeException e) {
+ throw new CloudRuntimeException(String.format("Failed to activate template volume %s " +
+ "in VG %s: %s", volumeUuid, vgName, e.getMessage()), e);
+ }
+ }
+
+ private String findDeviceNodeAfterActivation(String templateLvPath, String volumeUuid, String vgName) {
+ File lvDevice = new File(templateLvPath);
+ String lvPath = templateLvPath;
+
+ if (!lvDevice.exists()) {
+ String vgNameEscaped = vgName.replace("-", "--");
+ String volumeUuidEscaped = volumeUuid.replace("-", "--");
+ lvPath = "/dev/mapper/" + vgNameEscaped + "-" + volumeUuidEscaped;
+ lvDevice = new File(lvPath);
+ }
+
+ if (!lvDevice.exists()) {
+ logger.error("Template volume {} still not accessible after activation attempt", volumeUuid);
+ throw new CloudRuntimeException(String.format("Could not activate template volume %s " +
+ "in VG %s - device node not accessible even after activation", volumeUuid, vgName));
+ }
+
+ return lvPath;
+ }
+
+ private long getClvmVolumeSize(String lvPath) {
+ try {
+ Script lvsCmd = new Script("/usr/sbin/lvs", 5000, logger);
+ lvsCmd.add("--noheadings");
+ lvsCmd.add("--units");
+ lvsCmd.add("b");
+ lvsCmd.add("-o");
+ lvsCmd.add("lv_size");
+ lvsCmd.add(lvPath);
+
+ OutputInterpreter.AllLinesParser parser = new OutputInterpreter.AllLinesParser();
+ String result = lvsCmd.execute(parser);
+
+ String output = (result == null) ? parser.getLines() : result;
+
+ if (output != null && !output.isEmpty()) {
+ String sizeStr = output.trim().replaceAll("[^0-9]", "");
+ if (!sizeStr.isEmpty()) {
+ return Long.parseLong(sizeStr);
+ }
+ }
+ } catch (Exception sizeEx) {
+ logger.warn("Failed to get size for CLVM volume via lvs: {}", sizeEx.getMessage());
+ File lvDevice = new File(lvPath);
+ if (lvDevice.isFile()) {
+ return lvDevice.length();
+ }
+ }
+ return 0;
+ }
+
+ private KVMPhysicalDisk createPhysicalDiskFromClvmLv(String lvPath, String volumeUuid,
+ KVMStoragePool pool, long size) {
+ KVMPhysicalDisk disk = new KVMPhysicalDisk(lvPath, volumeUuid, pool);
+
+ PhysicalDiskFormat diskFormat = (pool.getType() == StoragePoolType.CLVM_NG)
+ ? PhysicalDiskFormat.QCOW2
+ : PhysicalDiskFormat.RAW;
+
+ logger.debug("{} pool detected, setting disk format to {} for volume {}",
+ pool.getType(), diskFormat, volumeUuid);
+
+ disk.setFormat(diskFormat);
+ disk.setSize(size);
+ disk.setVirtualSize(size);
+
+ logger.info("Successfully accessed CLVM/CLVM_NG volume via direct block device: {} " +
+ "with format: {} and size: {} bytes", lvPath, diskFormat, size);
+
+ return disk;
+ }
+
+ private void ensureTemplateAccessibility(String volumeUuid, String lvPath, KVMStoragePool pool) {
+ if (pool.getType() == StoragePoolType.CLVM_NG && volumeUuid.startsWith("template-")) {
+ logger.info("Detected template volume {}. Ensuring it's activated in shared mode for backing file access.",
+ volumeUuid);
+ ensureTemplateLvInSharedMode(lvPath, false);
+ }
+ }
+
Review Comment:
Moved to ClvmStorageAdaptor class
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4338249133
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [33.3% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4456178525
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4590449504
@RosiKyu a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4444538331
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [33.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
<!-- sqra:qg_revision=9e94a2c806a38af0923753d50ce1dc1a3a0b34d2eb90ca0979950a0e11440bba -->
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4632986204
@Pearl1594 a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4652643360
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4652516607
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [33.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
<!-- sqra:qg_revision=042272e0503041d9e87f676f55f0a7efbb807975dfa31d476f88cd0bfdd0d8d4 -->
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-3921839745
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-3923007701
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-3931554148
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-3917686220
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [3.2% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4416589287
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4416686691
@Pearl1594 a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4409016627
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 closed pull request #12617: CLVM enhancements and fixes
URL: https://github.com/apache/cloudstack/pull/12617
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4548689353
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4557849569
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4498722296
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [33.4% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
<!-- sqra:qg_revision=75c4a9078e075062690f78089478af78613bd46dd56bfc9fac5f5405b739b23b -->
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4426025566
@Pearl1594 a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4519114715
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [33.4% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
<!-- sqra:qg_revision=75c4a9078e075062690f78089478af78613bd46dd56bfc9fac5f5405b739b23b -->
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4138505051
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [15.6% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4203918700
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17393
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4238742625
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4157882259
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [24.6% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-3960050209
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4663328168
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "winterhazel (via GitHub)" <gi...@apache.org>.
winterhazel commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r3383498299
##########
server/src/main/java/com/cloud/storage/VolumeApiServiceImpl.java:
##########
@@ -2767,6 +2796,203 @@ private Volume orchestrateAttachVolumeToVM(Long vmId, Long volumeId, Long device
return newVol;
}
+ /**
+ * Helper method to get storage pools for volume and VM.
+ *
+ * @param volumeToAttach The volume being attached
+ * @param vmExistingVolume The VM's existing volume
+ * @return Pair of StoragePoolVO objects (volumePool, vmPool), or null if either pool is missing
+ */
+ private Pair<StoragePoolVO, StoragePoolVO> getStoragePoolsForVolumeAttachment(VolumeInfo volumeToAttach, VolumeVO vmExistingVolume) {
+ if (volumeToAttach == null || vmExistingVolume == null) {
+ return null;
+ }
+
+ StoragePoolVO volumePool = _storagePoolDao.findById(volumeToAttach.getPoolId());
+ StoragePoolVO vmPool = _storagePoolDao.findById(vmExistingVolume.getPoolId());
+
+ if (volumePool == null || vmPool == null) {
+ return null;
+ }
+
+ return new Pair<>(volumePool, vmPool);
+ }
+
+ /**
+ * Checks if both storage pools are CLVM type (CLVM or CLVM_NG).
+ * Delegates to VolumeService for the actual check.
+ *
+ * @param volumePool Storage pool for the volume
+ * @param vmPool Storage pool for the VM
+ * @return true if both pools are CLVM type (CLVM or CLVM_NG)
+ */
+ private boolean areBothPoolsClvmType(StoragePoolVO volumePool, StoragePoolVO vmPool) {
Review Comment:
This one is also unused
##########
server/src/main/java/com/cloud/storage/VolumeApiServiceImpl.java:
##########
@@ -2767,6 +2796,203 @@ private Volume orchestrateAttachVolumeToVM(Long vmId, Long volumeId, Long device
return newVol;
}
+ /**
+ * Helper method to get storage pools for volume and VM.
+ *
+ * @param volumeToAttach The volume being attached
+ * @param vmExistingVolume The VM's existing volume
+ * @return Pair of StoragePoolVO objects (volumePool, vmPool), or null if either pool is missing
+ */
+ private Pair<StoragePoolVO, StoragePoolVO> getStoragePoolsForVolumeAttachment(VolumeInfo volumeToAttach, VolumeVO vmExistingVolume) {
+ if (volumeToAttach == null || vmExistingVolume == null) {
+ return null;
+ }
+
+ StoragePoolVO volumePool = _storagePoolDao.findById(volumeToAttach.getPoolId());
+ StoragePoolVO vmPool = _storagePoolDao.findById(vmExistingVolume.getPoolId());
+
+ if (volumePool == null || vmPool == null) {
+ return null;
+ }
+
+ return new Pair<>(volumePool, vmPool);
+ }
+
+ /**
+ * Checks if both storage pools are CLVM type (CLVM or CLVM_NG).
+ * Delegates to VolumeService for the actual check.
+ *
+ * @param volumePool Storage pool for the volume
+ * @param vmPool Storage pool for the VM
+ * @return true if both pools are CLVM type (CLVM or CLVM_NG)
+ */
+ private boolean areBothPoolsClvmType(StoragePoolVO volumePool, StoragePoolVO vmPool) {
+ return volService.areBothPoolsClvmType(volumePool.getPoolType(), vmPool.getPoolType());
+ }
+
+ /**
+ * Determines if a CLVM volume needs lightweight lock migration instead of full data copy.
+ *
+ * Lightweight migration is needed when:
+ * 1. Volume is on CLVM storage
+ * 2. Source and destination are in the same Volume Group
+ * 3. Only the host/lock needs to change (not the storage pool)
+ *
+ * @param volumeToAttach The volume being attached
+ * @param vmExistingVolume The VM's existing volume (typically root volume)
+ * @param vm The VM to attach the volume to
+ * @return true if lightweight CLVM lock migration should be used
+ */
+ private boolean isClvmLightweightMigrationNeeded(VolumeInfo volumeToAttach, VolumeVO vmExistingVolume, UserVmVO vm) {
Review Comment:
Parameter `vm` is unused here
##########
server/src/main/java/com/cloud/storage/VolumeApiServiceImpl.java:
##########
@@ -2738,21 +2746,42 @@ private Volume orchestrateAttachVolumeToVM(Long vmId, Long volumeId, Long device
logger.trace(String.format("is it needed to move the volume: %b?", moveVolumeNeeded));
}
- if (moveVolumeNeeded) {
+ // Check if CLVM lock transfer is needed (even if moveVolumeNeeded is false)
+ // This handles the case where the volume is already on the correct storage pool
+ // but the VM is running on a different host, requiring only a lock transfer
+ boolean isClvmLockTransferNeeded = !moveVolumeNeeded &&
+ isClvmLockTransferRequired(newVolumeOnPrimaryStorage, existingVolumeOfVm, vm);
+
+ if (isClvmLockTransferNeeded) {
+ // CLVM lock transfer - no data copy, no pool change needed
+ newVolumeOnPrimaryStorage = executeLightweightLockMigration(
+ newVolumeOnPrimaryStorage, vm, existingVolumeOfVm,
+ "CLVM lock transfer", "same pool to different host");
+ } else if (moveVolumeNeeded) {
PrimaryDataStoreInfo primaryStore = (PrimaryDataStoreInfo)newVolumeOnPrimaryStorage.getDataStore();
if (primaryStore.isLocal()) {
throw new CloudRuntimeException(
"Failed to attach local data volume " + volumeToAttach.getName() + " to VM " + vm.getDisplayName() + " as migration of local data volume is not allowed");
}
- StoragePoolVO vmRootVolumePool = _storagePoolDao.findById(existingVolumeOfVm.getPoolId());
- try {
- HypervisorType volumeToAttachHyperType = _volsDao.getHypervisorType(volumeToAttach.getId());
- newVolumeOnPrimaryStorage = _volumeMgr.moveVolume(newVolumeOnPrimaryStorage, vmRootVolumePool.getDataCenterId(), vmRootVolumePool.getPodId(), vmRootVolumePool.getClusterId(),
- volumeToAttachHyperType);
- } catch (ConcurrentOperationException | StorageUnavailableException e) {
- logger.debug("move volume failed", e);
- throw new CloudRuntimeException("move volume failed", e);
+ boolean isClvmLightweightMigration = isClvmLightweightMigrationNeeded(
+ newVolumeOnPrimaryStorage, existingVolumeOfVm, vm);
+
+ if (isClvmLightweightMigration) {
+ newVolumeOnPrimaryStorage = executeLightweightLockMigration(
+ newVolumeOnPrimaryStorage, vm, existingVolumeOfVm,
+ "CLVM lightweight migration", "different pools, same VG");
+ } else {
+ StoragePoolVO vmRootVolumePool = _storagePoolDao.findById(existingVolumeOfVm.getPoolId());
+
+ try {
+ HypervisorType volumeToAttachHyperType = _volsDao.getHypervisorType(volumeToAttach.getId());
+ newVolumeOnPrimaryStorage = _volumeMgr.moveVolume(newVolumeOnPrimaryStorage, vmRootVolumePool.getDataCenterId(), vmRootVolumePool.getPodId(), vmRootVolumePool.getClusterId(),
+ volumeToAttachHyperType);
+ } catch (ConcurrentOperationException | StorageUnavailableException e) {
+ logger.debug("move volume failed", e);
+ throw new CloudRuntimeException("move volume failed", e);
+ }
}
Review Comment:
Can this volume move section be extracted to a separate method to reduce indentation?
##########
server/src/main/java/com/cloud/storage/VolumeApiServiceImpl.java:
##########
@@ -2767,6 +2796,203 @@ private Volume orchestrateAttachVolumeToVM(Long vmId, Long volumeId, Long device
return newVol;
}
+ /**
+ * Helper method to get storage pools for volume and VM.
+ *
+ * @param volumeToAttach The volume being attached
+ * @param vmExistingVolume The VM's existing volume
+ * @return Pair of StoragePoolVO objects (volumePool, vmPool), or null if either pool is missing
+ */
+ private Pair<StoragePoolVO, StoragePoolVO> getStoragePoolsForVolumeAttachment(VolumeInfo volumeToAttach, VolumeVO vmExistingVolume) {
+ if (volumeToAttach == null || vmExistingVolume == null) {
+ return null;
+ }
+
+ StoragePoolVO volumePool = _storagePoolDao.findById(volumeToAttach.getPoolId());
+ StoragePoolVO vmPool = _storagePoolDao.findById(vmExistingVolume.getPoolId());
+
+ if (volumePool == null || vmPool == null) {
+ return null;
+ }
+
+ return new Pair<>(volumePool, vmPool);
+ }
+
+ /**
+ * Checks if both storage pools are CLVM type (CLVM or CLVM_NG).
+ * Delegates to VolumeService for the actual check.
+ *
+ * @param volumePool Storage pool for the volume
+ * @param vmPool Storage pool for the VM
+ * @return true if both pools are CLVM type (CLVM or CLVM_NG)
+ */
+ private boolean areBothPoolsClvmType(StoragePoolVO volumePool, StoragePoolVO vmPool) {
+ return volService.areBothPoolsClvmType(volumePool.getPoolType(), vmPool.getPoolType());
+ }
+
+ /**
+ * Determines if a CLVM volume needs lightweight lock migration instead of full data copy.
+ *
+ * Lightweight migration is needed when:
+ * 1. Volume is on CLVM storage
+ * 2. Source and destination are in the same Volume Group
+ * 3. Only the host/lock needs to change (not the storage pool)
+ *
+ * @param volumeToAttach The volume being attached
+ * @param vmExistingVolume The VM's existing volume (typically root volume)
+ * @param vm The VM to attach the volume to
+ * @return true if lightweight CLVM lock migration should be used
+ */
+ private boolean isClvmLightweightMigrationNeeded(VolumeInfo volumeToAttach, VolumeVO vmExistingVolume, UserVmVO vm) {
+ Pair<StoragePoolVO, StoragePoolVO> pools = getStoragePoolsForVolumeAttachment(volumeToAttach, vmExistingVolume);
+ if (pools == null) {
+ return false;
+ }
+
+ StoragePoolVO volumePool = pools.first();
+ StoragePoolVO vmPool = pools.second();
+
+ return volService.isLightweightMigrationNeeded(volumePool.getPoolType(), vmPool.getPoolType(),
+ volumePool.getPath(), vmPool.getPath());
+ }
+
+ /**
+ * Determines if a CLVM volume requires lock transfer when already on the correct storage pool.
+ *
+ * Lock transfer is needed when:
+ * 1. Volume is already on the same CLVM storage pool as VM's volumes
+ * 2. But the volume lock is held by a different host than where the VM is running
+ * 3. Only the lock needs to change (no pool change, no data copy)
+ *
+ * @param volumeToAttach The volume being attached
+ * @param vmExistingVolume The VM's existing volume (typically root volume)
+ * @param vm The VM to attach the volume to
+ * @return true if CLVM lock transfer is needed (but not full migration)
+ */
+ private boolean isClvmLockTransferRequired(VolumeInfo volumeToAttach, VolumeVO vmExistingVolume, UserVmVO vm) {
+ if (vm == null) {
+ return false;
+ }
+
+ Pair<StoragePoolVO, StoragePoolVO> pools = getStoragePoolsForVolumeAttachment(volumeToAttach, vmExistingVolume);
+ if (pools == null) {
+ return false;
+ }
+
+ StoragePoolVO volumePool = pools.first();
+ StoragePoolVO vmPool = pools.second();
+
+ Long vmHostId = vm.getHostId();
+ if (vmHostId == null) {
+ vmHostId = vm.getLastHostId();
+ }
+
+ return volService.isLockTransferRequired(volumeToAttach, volumePool.getPoolType(), vmPool.getPoolType(),
+ volumePool.getId(), vmPool.getId(), vmHostId);
+ }
+
+ /**
+ * Determines the destination host for CLVM lock migration.
+ *
+ * If VM is running, uses the VM's current host.
+ * If VM is stopped, picks an available UP host from the storage pool's cluster.
+ *
+ * @param vm The VM
+ * @param vmExistingVolume The VM's existing volume (to determine cluster)
+ * @return Host ID, or null if cannot be determined
+ */
+ private Long determineClvmLockDestinationHost(UserVmVO vm, VolumeVO vmExistingVolume) {
+ Long destHostId = vm.getHostId();
+ if (destHostId != null) {
+ return destHostId;
+ }
+
+ if (vmExistingVolume != null && vmExistingVolume.getPoolId() != null) {
+ StoragePoolVO pool = _storagePoolDao.findById(vmExistingVolume.getPoolId());
+ if (pool != null && pool.getClusterId() != null) {
+ List<HostVO> hosts = _hostDao.findByClusterId(pool.getClusterId());
+ if (hosts != null && !hosts.isEmpty()) {
+ // Pick first available UP host
+ for (HostVO host : hosts) {
+ if (host.getStatus() == Status.Up) {
+ destHostId = host.getId();
+ logger.debug("VM {} is stopped, selected host {} from cluster {} for CLVM lock migration",
+ vm.getUuid(), destHostId, pool.getClusterId());
+ return destHostId;
+ }
+ }
+ }
+ }
+ }
Review Comment:
The conditions can be inverted to reduce indentation here
##########
server/src/main/java/com/cloud/storage/VolumeApiServiceImpl.java:
##########
@@ -2748,6 +2777,204 @@ private Volume orchestrateAttachVolumeToVM(Long vmId, Long volumeId, Long device
return newVol;
}
+ /**
+ * Helper method to get storage pools for volume and VM.
+ *
+ * @param volumeToAttach The volume being attached
+ * @param vmExistingVolume The VM's existing volume
+ * @return Pair of StoragePoolVO objects (volumePool, vmPool), or null if either pool is missing
+ */
+ private Pair<StoragePoolVO, StoragePoolVO> getStoragePoolsForVolumeAttachment(VolumeInfo volumeToAttach, VolumeVO vmExistingVolume) {
+ if (volumeToAttach == null || vmExistingVolume == null) {
+ return null;
+ }
+
+ StoragePoolVO volumePool = _storagePoolDao.findById(volumeToAttach.getPoolId());
+ StoragePoolVO vmPool = _storagePoolDao.findById(vmExistingVolume.getPoolId());
+
+ if (volumePool == null || vmPool == null) {
+ return null;
+ }
+
+ return new Pair<>(volumePool, vmPool);
+ }
+
+ /**
+ * Checks if both storage pools are CLVM type (CLVM or CLVM_NG).
+ * Delegates to VolumeService for the actual check.
+ *
+ * @param volumePool Storage pool for the volume
+ * @param vmPool Storage pool for the VM
+ * @return true if both pools are CLVM type (CLVM or CLVM_NG)
+ */
+ private boolean areBothPoolsClvmType(StoragePoolVO volumePool, StoragePoolVO vmPool) {
+ return volService.areBothPoolsClvmType(volumePool.getPoolType(), vmPool.getPoolType());
+ }
+
+ /**
+ * Determines if a CLVM volume needs lightweight lock migration instead of full data copy.
+ *
+ * Lightweight migration is needed when:
+ * 1. Volume is on CLVM storage
+ * 2. Source and destination are in the same Volume Group
+ * 3. Only the host/lock needs to change (not the storage pool)
+ *
+ * @param volumeToAttach The volume being attached
+ * @param vmExistingVolume The VM's existing volume (typically root volume)
+ * @param vm The VM to attach the volume to
+ * @return true if lightweight CLVM lock migration should be used
+ */
+ private boolean isClvmLightweightMigrationNeeded(VolumeInfo volumeToAttach, VolumeVO vmExistingVolume, UserVmVO vm) {
+ Pair<StoragePoolVO, StoragePoolVO> pools = getStoragePoolsForVolumeAttachment(volumeToAttach, vmExistingVolume);
+ if (pools == null) {
+ return false;
+ }
+
+ StoragePoolVO volumePool = pools.first();
+ StoragePoolVO vmPool = pools.second();
+
+ return volService.isLightweightMigrationNeeded(volumePool.getPoolType(), vmPool.getPoolType(),
+ volumePool.getPath(), vmPool.getPath());
+ }
+
+ /**
+ * Determines if a CLVM volume requires lock transfer when already on the correct storage pool.
+ *
+ * Lock transfer is needed when:
+ * 1. Volume is already on the same CLVM storage pool as VM's volumes
+ * 2. But the volume lock is held by a different host than where the VM is running
+ * 3. Only the lock needs to change (no pool change, no data copy)
+ *
+ * @param volumeToAttach The volume being attached
+ * @param vmExistingVolume The VM's existing volume (typically root volume)
+ * @param vm The VM to attach the volume to
+ * @return true if CLVM lock transfer is needed (but not full migration)
+ */
+ private boolean isClvmLockTransferRequired(VolumeInfo volumeToAttach, VolumeVO vmExistingVolume, UserVmVO vm) {
+ if (vm == null) {
+ return false;
+ }
+
+ Pair<StoragePoolVO, StoragePoolVO> pools = getStoragePoolsForVolumeAttachment(volumeToAttach, vmExistingVolume);
+ if (pools == null) {
+ return false;
+ }
+
+ StoragePoolVO volumePool = pools.first();
+ StoragePoolVO vmPool = pools.second();
+
+ Long vmHostId = vm.getHostId();
+ if (vmHostId == null) {
+ vmHostId = vm.getLastHostId();
+ }
+
+ return volService.isLockTransferRequired(volumeToAttach, volumePool.getPoolType(), vmPool.getPoolType(),
+ volumePool.getId(), vmPool.getId(), vmHostId);
+ }
+
+ /**
+ * Determines the destination host for CLVM lock migration.
+ *
+ * If VM is running, uses the VM's current host.
+ * If VM is stopped, picks an available UP host from the storage pool's cluster.
+ *
+ * @param vm The VM
+ * @param vmExistingVolume The VM's existing volume (to determine cluster)
+ * @return Host ID, or null if cannot be determined
+ */
+ private Long determineClvmLockDestinationHost(UserVmVO vm, VolumeVO vmExistingVolume) {
+ Long destHostId = vm.getHostId();
+ if (destHostId != null) {
+ return destHostId;
+ }
+
+ if (vmExistingVolume != null && vmExistingVolume.getPoolId() != null) {
+ StoragePoolVO pool = _storagePoolDao.findById(vmExistingVolume.getPoolId());
+ if (pool != null && pool.getClusterId() != null) {
+ List<HostVO> hosts = _hostDao.findByClusterId(pool.getClusterId());
+ if (hosts != null && !hosts.isEmpty()) {
+ // Pick first available UP host
+ for (HostVO host : hosts) {
+ if (host.getStatus() == Status.Up) {
+ destHostId = host.getId();
+ logger.debug("VM {} is stopped, selected host {} from cluster {} for CLVM lock migration",
+ vm.getUuid(), destHostId, pool.getClusterId());
+ return destHostId;
+ }
+ }
+ }
+ }
+ }
+
+ return null;
+ }
+
+ /**
+ * Executes CLVM lightweight migration with consistent logging and error handling.
+ *
+ * This helper method wraps the actual migration logic to eliminate code duplication
+ * between different CLVM migration scenarios (lock transfer vs. lightweight migration).
+ *
+ * @param volume The volume to migrate locks for
+ * @param vm The VM to attach the volume to
+ * @param vmExistingVolume The VM's existing volume (to determine target host)
+ * @param operationType Description of the operation type for logging (e.g., "CLVM lock transfer")
+ * @param scenarioDescription Description of the scenario for logging (e.g., "same pool to different host")
+ * @return Updated VolumeInfo after lock migration
+ * @throws CloudRuntimeException if migration fails
+ */
+ private VolumeInfo executeLightweightLockMigration(VolumeInfo volume, UserVmVO vm, VolumeVO vmExistingVolume,
+ String operationType, String scenarioDescription) {
+ logger.info("Performing {} for volume {} to VM {} ({})",
+ operationType, volume.getUuid(), vm.getUuid(), scenarioDescription);
+
+ try {
+ return performLightweightLockMigration(volume, vm, vmExistingVolume);
+ } catch (Exception e) {
+ logger.error("{} failed for volume {}: {}",
+ operationType, volume.getUuid(), e.getMessage(), e);
+ throw new CloudRuntimeException(operationType + " failed", e);
+ }
+ }
+
+ /**
+ * Performs lightweight CLVM lock migration for volume attachment.
+ * Delegates to VolumeService for the actual lock migration.
+ *
+ * @param volume The volume to migrate locks for
+ * @param vm The VM to attach the volume to
+ * @param vmExistingVolume The VM's existing volume (to determine target host)
+ * @return Updated VolumeInfo after lock migration
+ * @throws Exception if lock migration fails
+ */
+ private VolumeInfo performLightweightLockMigration(VolumeInfo volume, UserVmVO vm, VolumeVO vmExistingVolume) throws CloudRuntimeException {
+ Long destHostId = determineClvmLockDestinationHost(vm, vmExistingVolume);
+
+ if (destHostId == null) {
+ throw new CloudRuntimeException(
+ "Cannot determine destination host for CLVM lock migration - VM has no host and no available cluster hosts");
+ }
+
+ try {
+ return volService.performLockMigration(volume, destHostId);
+ } catch (CloudRuntimeException e) {
+ logger.error("CLVM lock migration failed for volume {}: {}", volume.getUuid(), e.getMessage(), e);
+ throw e;
+ }
+ }
+
+ /**
+ * Finds which host currently has the exclusive lock on a CLVM volume.
+ * Delegates to VolumeService for the actual lookup.
+ *
+ * @param volume The CLVM volume
+ * @return Host ID that has the exclusive lock, or null if cannot be determined
+ */
+ private Long findVolumeLockHost(VolumeInfo volume) {
Review Comment:
Ping @Pearl1594, could you have a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4666171975
@Pearl1594 a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4700202568
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4700734098
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18254
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4693277145
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [32.4% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
<!-- sqra:qg_revision=2fc4c8746dbe7c65388f20be10b451cbed492ac5bae060034d61d05e4794a7ad -->
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4675555106
<b>[SF] Trillian test result (tid-16267)</b>
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 171330 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12617-t16267-kvm-ol8.zip
Smoke tests completed. 90 look OK, 61 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_01_invalid_upgrade_kubernetes_cluster | `Failure` | 0.00 | test_kubernetes_clusters.py
test_02_upgrade_kubernetes_cluster | `Failure` | 0.00 | test_kubernetes_clusters.py
test_03_deploy_and_scale_kubernetes_cluster | `Failure` | 0.00 | test_kubernetes_clusters.py
test_04_autoscale_kubernetes_cluster | `Failure` | 0.00 | test_kubernetes_clusters.py
test_05_basic_lifecycle_kubernetes_cluster | `Failure` | 0.00 | test_kubernetes_clusters.py
test_06_delete_kubernetes_cluster | `Failure` | 0.00 | test_kubernetes_clusters.py
test_08_upgrade_kubernetes_ha_cluster | `Failure` | 0.00 | test_kubernetes_clusters.py
test_10_vpc_tier_kubernetes_cluster | `Failure` | 0.00 | test_kubernetes_clusters.py
test_11_test_unmanaged_cluster_lifecycle | `Error` | 0.00 | test_kubernetes_clusters.py
test_12_test_deploy_cluster_different_offerings_per_node_type | `Failure` | 0.00 | test_kubernetes_clusters.py
test_01_add_delete_kubernetes_supported_version | `Error` | 1802.26 | test_kubernetes_supported_versions.py
ContextSuite context=TestListIdsParams>:setup | `Error` | 0.00 | test_list_ids_parameter.py
ContextSuite context=TestListVolumes>:setup | `Error` | 0.00 | test_list_volumes.py
ContextSuite context=TestLoadBalance>:setup | `Error` | 0.00 | test_loadbalance.py
ContextSuite context=TestMetrics>:setup | `Error` | 0.00 | test_metrics_api.py
test_01_native_to_native_network_migration | `Error` | 81.17 | test_migration.py
test_02_native_to_native_vpc_migration | `Error` | 97.37 | test_migration.py
test_nic_secondaryip_add_remove | `Error` | 1518.76 | test_multipleips_per_nic.py
ContextSuite context=TestNestedVirtualization>:setup | `Error` | 0.00 | test_nested_virtualization.py
ContextSuite context=TestNetworkACL>:setup | `Error` | 0.00 | test_network_acl.py
ContextSuite context=TestIsolatedNetworksPasswdServer>:setup | `Error` | 0.00 | test_password_server.py
ContextSuite context=TestIpv6Network>:setup | `Error` | 0.00 | test_network_ipv6.py
test_03_network_operations_on_created_vm_of_otheruser | `Error` | 63.98 | test_network_permissions.py
test_03_network_operations_on_created_vm_of_otheruser | `Error` | 63.99 | test_network_permissions.py
test_04_deploy_vm_for_other_user_and_test_vm_operations | `Failure` | 66.03 | test_network_permissions.py
ContextSuite context=TestNetworkPermissions>:teardown | `Error` | 2.38 | test_network_permissions.py
test_delete_account | `Error` | 1517.58 | test_network.py
test_delete_network_while_vm_on_it | `Error` | 1.19 | test_network.py
test_deploy_vm_l2network | `Error` | 1.59 | test_network.py
test_l2network_restart | `Error` | 2.33 | test_network.py
ContextSuite context=TestPortForwarding>:setup | `Error` | 3.55 | test_network.py
ContextSuite context=TestPublicIP>:setup | `Error` | 6.11 | test_network.py
test_reboot_router | `Failure` | 0.08 | test_network.py
test_releaseIP | `Error` | 3.06 | test_network.py
test_releaseIP_using_IP | `Error` | 3.08 | test_network.py
ContextSuite context=TestRouterRules>:setup | `Error` | 3.15 | test_network.py
test_01_deployVMInSharedNetwork | `Failure` | 64.19 | test_network.py
test_03_destroySharedNetwork | `Failure` | 1.09 | test_network.py
ContextSuite context=TestSharedNetwork>:teardown | `Error` | 1.20 | test_network.py
ContextSuite context=TestSharedNetworkWithConfigDrive>:setup | `Error` | 1519.05 | test_network.py
ContextSuite context=TestAdapterTypeForNic>:setup | `Error` | 0.00 | test_nic_adapter_type.py
test_01_nic | `Error` | 112.95 | test_nic.py
ContextSuite context=TestNonStrictAffinityGroups>:setup | `Error` | 0.00 | test_nonstrict_affinity_group.py
test_03_deploy_and_destroy_VM_and_verify_network_resources_persist | `Failure` | 6.77 | test_persistent_network.py
test_03_deploy_and_destroy_VM_and_verify_network_resources_persist | `Error` | 6.77 | test_persistent_network.py
ContextSuite context=TestL2PersistentNetworks>:teardown | `Error` | 6.87 | test_persistent_network.py
ContextSuite context=TestPortForwardingRules>:setup | `Error` | 0.00 | test_portforwardingrules.py
test_01_add_primary_storage_disabled_host | `Error` | 41.03 | test_primary_storage.py
test_01_primary_storage_nfs | `Error` | 0.26 | test_primary_storage.py
ContextSuite context=TestStorageTags>:setup | `Error` | 0.43 | test_primary_storage.py
test_01_primary_storage_scope_change | `Error` | 0.14 | test_primary_storage_scope.py
ContextSuite context=TestPrivateGwACL>:setup | `Error` | 0.00 | test_privategw_acl.py
ContextSuite context=TestProjectSuspendActivate>:setup | `Error` | 1526.98 | test_projects.py
test_01_purge_expunged_api_vm_start_date | `Error` | 14.48 | test_purge_expunged_vms.py
test_02_purge_expunged_api_vm_end_date | `Error` | 14.27 | test_purge_expunged_vms.py
test_03_purge_expunged_api_vm_start_end_date | `Error` | 12.75 | test_purge_expunged_vms.py
test_04_purge_expunged_api_vm_no_date | `Error` | 16.01 | test_purge_expunged_vms.py
test_05_purge_expunged_vm_service_offering | `Error` | 5.25 | test_purge_expunged_vms.py
test_06_purge_expunged_vm_background_task | `Error` | 374.80 | test_purge_expunged_vms.py
test_CRUD_operations_userdata | `Error` | 1521.51 | test_register_userdata.py
test_deploy_vm_with_registered_userdata | `Error` | 5.85 | test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_allow | `Error` | 5.71 | test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_append | `Error` | 5.97 | test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_deny | `Error` | 5.89 | test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_params | `Error` | 5.87 | test_register_userdata.py
test_link_and_unlink_userdata_to_template | `Error` | 5.84 | test_register_userdata.py
test_user_userdata_crud | `Error` | 10.85 | test_register_userdata.py
ContextSuite context=TestResetVmOnReboot>:setup | `Error` | 0.00 | test_reset_vm_on_reboot.py
test_03_ssvm_internals | `Failure` | 2.86 | test_ssvm.py
test_05_stop_ssvm | `Failure` | 65.31 | test_ssvm.py
test_07_reboot_ssvm | `Failure` | 127.66 | test_ssvm.py
test_09_reboot_ssvm_forced | `Failure` | 147.66 | test_ssvm.py
test_11_destroy_ssvm | `Failure` | 58.89 | test_ssvm.py
ContextSuite context=TestRAMCPUResourceAccounting>:setup | `Error` | 0.00 | test_resource_accounting.py
ContextSuite context=TestResourceNames>:setup | `Error` | 0.00 | test_resource_names.py
ContextSuite context=TestVMWareStoragePolicies>:setup | `Error` | 0.00 | test_storage_policy.py
ContextSuite context=TestRestoreVM>:setup | `Error` | 0.00 | test_restore_vm.py
ContextSuite context=TestRouterDHCPHosts>:setup | `Error` | 0.00 | test_router_dhcphosts.py
ContextSuite context=TestRouterDHCPOpts>:setup | `Error` | 0.00 | test_router_dhcphosts.py
ContextSuite context=TestRouterDns>:setup | `Error` | 0.00 | test_router_dns.py
ContextSuite context=TestRouterDnsService>:setup | `Error` | 0.00 | test_router_dnsservice.py
ContextSuite context=TestSslOffloading>:setup | `Error` | 0.00 | test_ssl_offloading.py
ContextSuite context=TestRouterIpTablesPolicies>:setup | `Error` | 0.00 | test_routers_iptables_default_policy.py
ContextSuite context=TestVPCIpTablesPolicies>:setup | `Error` | 0.00 | test_routers_iptables_default_policy.py
ContextSuite context=TestIsolatedNetworks>:setup | `Error` | 0.00 | test_routers_network_ops.py
ContextSuite context=TestRedundantIsolateNetworks>:setup | `Error` | 0.00 | test_routers_network_ops.py
ContextSuite context=TestISOUsage>:setup | `Error` | 0.00 | test_usage.py
ContextSuite context=TestLBRuleUsage>:setup | `Error` | 0.00 | test_usage.py
ContextSuite context=TestNatRuleUsage>:setup | `Error` | 0.00 | test_usage.py
ContextSuite context=TestPublicIPUsage>:setup | `Error` | 0.00 | test_usage.py
ContextSuite context=TestSnapshotUsage>:setup | `Error` | 0.00 | test_usage.py
ContextSuite context=TestVmUsage>:setup | `Error` | 0.00 | test_usage.py
ContextSuite context=TestVolumeUsage>:setup | `Error` | 0.00 | test_usage.py
ContextSuite context=TestVpnUsage>:setup | `Error` | 0.00 | test_usage.py
ContextSuite context=TestRouterServices>:setup | `Error` | 0.00 | test_routers.py
ContextSuite context=TestCpuCapServiceOfferings>:setup | `Error` | 0.00 | test_service_offerings.py
ContextSuite context=TestServiceOfferings>:setup | `Error` | 0.35 | test_service_offerings.py
test_02_create_template_with_checksum_sha1 | `Error` | 65.72 | test_templates.py
test_03_create_template_with_checksum_sha256 | `Error` | 65.72 | test_templates.py
test_04_create_template_with_checksum_md5 | `Error` | 65.74 | test_templates.py
test_05_create_template_with_no_checksum | `Error` | 65.68 | test_templates.py
ContextSuite context=TestTemplates>:setup | `Error` | 115.19 | test_templates.py
ContextSuite context=TestSetSourceNatIp>:setup | `Error` | 0.00 | test_set_sourcenat.py
ContextSuite context=TestSharedFSLifecycle>:setup | `Error` | 0.00 | test_sharedfs_lifecycle.py
ContextSuite context=TestSnapshotRootDisk>:setup | `Error` | 0.00 | test_snapshots.py
ContextSuite context=TestSnapshotStandaloneBackup>:setup | `Error` | 0.00 | test_snapshots.py
test_1_userdata_on_ssvm | `Failure` | 203.37 | test_systemvm_userdata.py
test_2_userdata_on_cpvm | `Failure` | 206.72 | test_systemvm_userdata.py
test_3_userdata_on_vr | `Failure` | 189.20 | test_systemvm_userdata.py
test_01_scale_up_verify | `Failure` | 577.38 | test_vm_autoscaling.py
test_02_update_vmprofile_and_vmgroup | `Failure` | 248.76 | test_vm_autoscaling.py
test_03_scale_down_verify | `Failure` | 308.55 | test_vm_autoscaling.py
test_04_stop_remove_vm_in_vmgroup | `Failure` | 18.14 | test_vm_autoscaling.py
test_06_autoscaling_vmgroup_on_project_network | `Failure` | 581.02 | test_vm_autoscaling.py
test_06_autoscaling_vmgroup_on_project_network | `Error` | 581.03 | test_vm_autoscaling.py
test_07_autoscaling_vmgroup_on_vpc_network | `Failure` | 635.78 | test_vm_autoscaling.py
test_07_autoscaling_vmgroup_on_vpc_network | `Error` | 635.79 | test_vm_autoscaling.py
ContextSuite context=TestVmAutoScaling>:teardown | `Error` | 647.97 | test_vm_autoscaling.py
test_01_deploy_vm_on_specific_host | `Error` | 46.85 | test_vm_deployment_planner.py
test_02_deploy_vm_on_specific_cluster | `Error` | 7.51 | test_vm_deployment_planner.py
test_03_deploy_vm_on_specific_pod | `Error` | 3.42 | test_vm_deployment_planner.py
test_04_deploy_vm_on_host_override_pod_and_cluster | `Error` | 4.54 | test_vm_deployment_planner.py
test_05_deploy_vm_on_cluster_override_pod | `Error` | 4.41 | test_vm_deployment_planner.py
ContextSuite context=TestDeployVM>:setup | `Error` | 0.00 | test_vm_life_cycle.py
test_01_migrate_VM_and_root_volume | `Error` | 56.03 | test_vm_life_cycle.py
test_02_migrate_VM_with_two_data_disks | `Error` | 14.77 | test_vm_life_cycle.py
test_01_secure_vm_migration | `Error` | 138.16 | test_vm_life_cycle.py
test_02_unsecure_vm_migration | `Error` | 234.23 | test_vm_life_cycle.py
test_03_secured_to_nonsecured_vm_migration | `Error` | 156.56 | test_vm_life_cycle.py
test_04_nonsecured_to_secured_vm_migration | `Error` | 163.54 | test_vm_life_cycle.py
ContextSuite context=TestVMLifeCycle>:setup | `Error` | 70.66 | test_vm_life_cycle.py
test_01_unmanage_vm_cycle_persistent_domain | `Error` | 55.17 | test_vm_lifecycle_unmanage_kvm_import.py
test_01_unmanage_vm_cycle_persistent_domain | `Error` | 55.17 | test_vm_lifecycle_unmanage_kvm_import.py
test_02_unmanage_stopped_vm_cycle_persistent_domain | `Error` | 65.78 | test_vm_lifecycle_unmanage_kvm_import.py
test_02_unmanage_stopped_vm_cycle_persistent_domain | `Error` | 65.78 | test_vm_lifecycle_unmanage_kvm_import.py
ContextSuite context=TestUnmanageKvmVM>:teardown | `Error` | 65.81 | test_vm_lifecycle_unmanage_kvm_import.py
ContextSuite context=TestVMSchedule>:setup | `Error` | 0.00 | test_vm_schedule.py
ContextSuite context=TestVmSnapshot>:setup | `Error` | 58.11 | test_vm_snapshots.py
ContextSuite context=TestMigrateVMStrictTags>:setup | `Error` | 0.00 | test_vm_strict_host_tags.py
ContextSuite context=TestRestoreVMStrictTags>:setup | `Error` | 0.00 | test_vm_strict_host_tags.py
ContextSuite context=TestScaleVMStrictTags>:setup | `Error` | 0.00 | test_vm_strict_host_tags.py
ContextSuite context=TestVMDeploymentPlannerStrictTags>:setup | `Error` | 0.00 | test_vm_strict_host_tags.py
test_04_deploy_vnf_appliance | `Error` | 315.78 | test_vnf_templates.py
ContextSuite context=TestCreateVolume>:setup | `Error` | 0.00 | test_volumes.py
test_01_root_volume_encryption | `Error` | 67.21 | test_volumes.py
test_02_data_volume_encryption | `Error` | 13.64 | test_volumes.py
test_03_root_and_data_volume_encryption | `Error` | 14.73 | test_volumes.py
ContextSuite context=TestVolumes>:setup | `Error` | 107.44 | test_volumes.py
ContextSuite context=TestVPCConserveModeRules>:setup | `Error` | 0.00 | test_vpc_conserve_mode.py
ContextSuite context=TestIpv6Vpc>:setup | `Error` | 0.00 | test_vpc_ipv6.py
ContextSuite context=TestVPCRedundancy>:setup | `Error` | 0.00 | test_vpc_redundant.py
ContextSuite context=TestVPCNics>:setup | `Error` | 0.00 | test_vpc_router_nics.py
ContextSuite context=TestRVPCSite2SiteVpn>:setup | `Error` | 0.00 | test_vpc_vpn.py
ContextSuite context=TestVPCSite2SiteVPNMultipleOptions>:setup | `Error` | 0.00 | test_vpc_vpn.py
ContextSuite context=TestVpcRemoteAccessVpn>:setup | `Error` | 0.00 | test_vpc_vpn.py
ContextSuite context=TestVpcSite2SiteVpn>:setup | `Error` | 0.00 | test_vpc_vpn.py
test_02_cancel_host_maintenace_with_migration_jobs | `Error` | 4.76 | test_host_maintenance.py
test_03_cancel_host_maintenace_with_migration_jobs_failure | `Error` | 2.71 | test_host_maintenance.py
test_01_cancel_host_maintenance_ssh_enabled_agent_connected | `Failure` | 30.90 | test_host_maintenance.py
test_03_cancel_host_maintenance_ssh_disabled_agent_connected | `Failure` | 16.72 | test_host_maintenance.py
test_04_cancel_host_maintenance_ssh_disabled_agent_disconnected | `Failure` | 37.56 | test_host_maintenance.py
ContextSuite context=TestHostMaintenanceAgents>:teardown | `Error` | 38.71 | test_host_maintenance.py
test_disable_oobm_ha_state_ineligible | `Error` | 1517.64 | test_hostha_kvm.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-3922182253
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16875
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
weizhouapache commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r2817514329
##########
engine/storage/snapshot/src/main/java/org/apache/cloudstack/storage/vmsnapshot/DefaultVMSnapshotStrategy.java:
##########
@@ -493,10 +501,31 @@ public boolean deleteVMSnapshotFromDB(VMSnapshot vmSnapshot, boolean unmanage) {
return vmSnapshotDao.remove(vmSnapshot.getId());
}
+ protected boolean isRunningVMVolumeOnCLVMStorage(UserVmVO vm, String cantHandleLog) {
+ Long vmId = vm.getId();
+ if (State.Running.equals(vm.getState())) {
+ List<VolumeVO> volumes = volumeDao.findByInstance(vmId);
+ for (VolumeVO volume : volumes) {
+ StoragePool pool = primaryDataStoreDao.findById(volume.getPoolId());
+ if (pool != null && pool.getPoolType() == Storage.StoragePoolType.CLVM) {
+ logger.warn("Rejecting VM snapshot request: {} - VM is running on CLVM storage (pool: {}, poolType: CLVM)",
+ cantHandleLog, pool.getName());
+ return true;
+ }
+ }
+ }
+ return false;
+ }
+
@Override
public StrategyPriority canHandle(Long vmId, Long rootPoolId, boolean snapshotMemory) {
UserVmVO vm = userVmDao.findById(vmId);
String cantHandleLog = String.format("Default VM snapshot cannot handle VM snapshot for [%s]", vm);
+
+ if (isRunningVMVolumeOnCLVMStorage(vm, cantHandleLog)) {
Review Comment:
@Pearl1594
what's the image format on CLVM ? RAW or QCOW2 ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-3886983971
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16801
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-3931560188
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4165084473
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [32.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-3955984127
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4111141532
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [8.5% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4201128564
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [32.5% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-3874220542
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r3383633597
##########
server/src/main/java/com/cloud/storage/VolumeApiServiceImpl.java:
##########
@@ -2738,21 +2746,42 @@ private Volume orchestrateAttachVolumeToVM(Long vmId, Long volumeId, Long device
logger.trace(String.format("is it needed to move the volume: %b?", moveVolumeNeeded));
}
- if (moveVolumeNeeded) {
+ // Check if CLVM lock transfer is needed (even if moveVolumeNeeded is false)
+ // This handles the case where the volume is already on the correct storage pool
+ // but the VM is running on a different host, requiring only a lock transfer
+ boolean isClvmLockTransferNeeded = !moveVolumeNeeded &&
+ isClvmLockTransferRequired(newVolumeOnPrimaryStorage, existingVolumeOfVm, vm);
+
+ if (isClvmLockTransferNeeded) {
+ // CLVM lock transfer - no data copy, no pool change needed
+ newVolumeOnPrimaryStorage = executeLightweightLockMigration(
+ newVolumeOnPrimaryStorage, vm, existingVolumeOfVm,
+ "CLVM lock transfer", "same pool to different host");
+ } else if (moveVolumeNeeded) {
PrimaryDataStoreInfo primaryStore = (PrimaryDataStoreInfo)newVolumeOnPrimaryStorage.getDataStore();
if (primaryStore.isLocal()) {
throw new CloudRuntimeException(
"Failed to attach local data volume " + volumeToAttach.getName() + " to VM " + vm.getDisplayName() + " as migration of local data volume is not allowed");
}
- StoragePoolVO vmRootVolumePool = _storagePoolDao.findById(existingVolumeOfVm.getPoolId());
- try {
- HypervisorType volumeToAttachHyperType = _volsDao.getHypervisorType(volumeToAttach.getId());
- newVolumeOnPrimaryStorage = _volumeMgr.moveVolume(newVolumeOnPrimaryStorage, vmRootVolumePool.getDataCenterId(), vmRootVolumePool.getPodId(), vmRootVolumePool.getClusterId(),
- volumeToAttachHyperType);
- } catch (ConcurrentOperationException | StorageUnavailableException e) {
- logger.debug("move volume failed", e);
- throw new CloudRuntimeException("move volume failed", e);
+ boolean isClvmLightweightMigration = isClvmLightweightMigrationNeeded(
+ newVolumeOnPrimaryStorage, existingVolumeOfVm, vm);
+
+ if (isClvmLightweightMigration) {
+ newVolumeOnPrimaryStorage = executeLightweightLockMigration(
+ newVolumeOnPrimaryStorage, vm, existingVolumeOfVm,
+ "CLVM lightweight migration", "different pools, same VG");
+ } else {
+ StoragePoolVO vmRootVolumePool = _storagePoolDao.findById(existingVolumeOfVm.getPoolId());
+
+ try {
+ HypervisorType volumeToAttachHyperType = _volsDao.getHypervisorType(volumeToAttach.getId());
+ newVolumeOnPrimaryStorage = _volumeMgr.moveVolume(newVolumeOnPrimaryStorage, vmRootVolumePool.getDataCenterId(), vmRootVolumePool.getPodId(), vmRootVolumePool.getClusterId(),
+ volumeToAttachHyperType);
+ } catch (ConcurrentOperationException | StorageUnavailableException e) {
+ logger.debug("move volume failed", e);
+ throw new CloudRuntimeException("move volume failed", e);
+ }
}
Review Comment:
Would it be ok if I addressed this in another PR - with other enhancements that are yet to come.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4663311779
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r3383633597
##########
server/src/main/java/com/cloud/storage/VolumeApiServiceImpl.java:
##########
@@ -2738,21 +2746,42 @@ private Volume orchestrateAttachVolumeToVM(Long vmId, Long volumeId, Long device
logger.trace(String.format("is it needed to move the volume: %b?", moveVolumeNeeded));
}
- if (moveVolumeNeeded) {
+ // Check if CLVM lock transfer is needed (even if moveVolumeNeeded is false)
+ // This handles the case where the volume is already on the correct storage pool
+ // but the VM is running on a different host, requiring only a lock transfer
+ boolean isClvmLockTransferNeeded = !moveVolumeNeeded &&
+ isClvmLockTransferRequired(newVolumeOnPrimaryStorage, existingVolumeOfVm, vm);
+
+ if (isClvmLockTransferNeeded) {
+ // CLVM lock transfer - no data copy, no pool change needed
+ newVolumeOnPrimaryStorage = executeLightweightLockMigration(
+ newVolumeOnPrimaryStorage, vm, existingVolumeOfVm,
+ "CLVM lock transfer", "same pool to different host");
+ } else if (moveVolumeNeeded) {
PrimaryDataStoreInfo primaryStore = (PrimaryDataStoreInfo)newVolumeOnPrimaryStorage.getDataStore();
if (primaryStore.isLocal()) {
throw new CloudRuntimeException(
"Failed to attach local data volume " + volumeToAttach.getName() + " to VM " + vm.getDisplayName() + " as migration of local data volume is not allowed");
}
- StoragePoolVO vmRootVolumePool = _storagePoolDao.findById(existingVolumeOfVm.getPoolId());
- try {
- HypervisorType volumeToAttachHyperType = _volsDao.getHypervisorType(volumeToAttach.getId());
- newVolumeOnPrimaryStorage = _volumeMgr.moveVolume(newVolumeOnPrimaryStorage, vmRootVolumePool.getDataCenterId(), vmRootVolumePool.getPodId(), vmRootVolumePool.getClusterId(),
- volumeToAttachHyperType);
- } catch (ConcurrentOperationException | StorageUnavailableException e) {
- logger.debug("move volume failed", e);
- throw new CloudRuntimeException("move volume failed", e);
+ boolean isClvmLightweightMigration = isClvmLightweightMigrationNeeded(
+ newVolumeOnPrimaryStorage, existingVolumeOfVm, vm);
+
+ if (isClvmLightweightMigration) {
+ newVolumeOnPrimaryStorage = executeLightweightLockMigration(
+ newVolumeOnPrimaryStorage, vm, existingVolumeOfVm,
+ "CLVM lightweight migration", "different pools, same VG");
+ } else {
+ StoragePoolVO vmRootVolumePool = _storagePoolDao.findById(existingVolumeOfVm.getPoolId());
+
+ try {
+ HypervisorType volumeToAttachHyperType = _volsDao.getHypervisorType(volumeToAttach.getId());
+ newVolumeOnPrimaryStorage = _volumeMgr.moveVolume(newVolumeOnPrimaryStorage, vmRootVolumePool.getDataCenterId(), vmRootVolumePool.getPodId(), vmRootVolumePool.getClusterId(),
+ volumeToAttachHyperType);
+ } catch (ConcurrentOperationException | StorageUnavailableException e) {
+ logger.debug("move volume failed", e);
+ throw new CloudRuntimeException("move volume failed", e);
+ }
}
Review Comment:
Would it be ok if I addressed this is another PR - with other enhancements that are yet to come.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4654211612
@Pearl1594 a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4673457632
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [33.4% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
<!-- sqra:qg_revision=472cdf7db613205e6d2c4744c7816053eea17f61ac6e3c845dbb5942e4ac4394 -->
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4393637696
@Pearl1594 a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4393918494
To use RAW volumes, you could use CLVM. CLVM_NG was introduced with QCOW2 support to leverage some benefits that qcow2 offers, like incremental snapshots. As of now, it is not thin provisioning.
It isn't possible to use RAW on CLVM_NG, but if your storage pool is of CLVM type, then you can. Hope this helps.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4456243512
Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 17864
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4169467881
@Pearl1594 a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4346364789
@Pearl1594 a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4345923679
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on code in PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#discussion_r3166029252
##########
server/src/main/java/com/cloud/storage/VolumeApiServiceImpl.java:
##########
@@ -2748,6 +2777,204 @@ private Volume orchestrateAttachVolumeToVM(Long vmId, Long volumeId, Long device
return newVol;
}
+ /**
+ * Helper method to get storage pools for volume and VM.
+ *
+ * @param volumeToAttach The volume being attached
+ * @param vmExistingVolume The VM's existing volume
+ * @return Pair of StoragePoolVO objects (volumePool, vmPool), or null if either pool is missing
+ */
+ private Pair<StoragePoolVO, StoragePoolVO> getStoragePoolsForVolumeAttachment(VolumeInfo volumeToAttach, VolumeVO vmExistingVolume) {
+ if (volumeToAttach == null || vmExistingVolume == null) {
+ return null;
+ }
+
+ StoragePoolVO volumePool = _storagePoolDao.findById(volumeToAttach.getPoolId());
+ StoragePoolVO vmPool = _storagePoolDao.findById(vmExistingVolume.getPoolId());
+
+ if (volumePool == null || vmPool == null) {
+ return null;
+ }
+
+ return new Pair<>(volumePool, vmPool);
+ }
+
+ /**
+ * Checks if both storage pools are CLVM type (CLVM or CLVM_NG).
+ * Delegates to VolumeService for the actual check.
+ *
+ * @param volumePool Storage pool for the volume
+ * @param vmPool Storage pool for the VM
+ * @return true if both pools are CLVM type (CLVM or CLVM_NG)
+ */
+ private boolean areBothPoolsClvmType(StoragePoolVO volumePool, StoragePoolVO vmPool) {
+ return volService.areBothPoolsClvmType(volumePool.getPoolType(), vmPool.getPoolType());
+ }
+
+ /**
+ * Determines if a CLVM volume needs lightweight lock migration instead of full data copy.
+ *
+ * Lightweight migration is needed when:
+ * 1. Volume is on CLVM storage
+ * 2. Source and destination are in the same Volume Group
+ * 3. Only the host/lock needs to change (not the storage pool)
+ *
+ * @param volumeToAttach The volume being attached
+ * @param vmExistingVolume The VM's existing volume (typically root volume)
+ * @param vm The VM to attach the volume to
+ * @return true if lightweight CLVM lock migration should be used
+ */
+ private boolean isClvmLightweightMigrationNeeded(VolumeInfo volumeToAttach, VolumeVO vmExistingVolume, UserVmVO vm) {
+ Pair<StoragePoolVO, StoragePoolVO> pools = getStoragePoolsForVolumeAttachment(volumeToAttach, vmExistingVolume);
+ if (pools == null) {
+ return false;
+ }
+
+ StoragePoolVO volumePool = pools.first();
+ StoragePoolVO vmPool = pools.second();
+
+ return volService.isLightweightMigrationNeeded(volumePool.getPoolType(), vmPool.getPoolType(),
+ volumePool.getPath(), vmPool.getPath());
+ }
+
+ /**
+ * Determines if a CLVM volume requires lock transfer when already on the correct storage pool.
+ *
+ * Lock transfer is needed when:
+ * 1. Volume is already on the same CLVM storage pool as VM's volumes
+ * 2. But the volume lock is held by a different host than where the VM is running
+ * 3. Only the lock needs to change (no pool change, no data copy)
+ *
+ * @param volumeToAttach The volume being attached
+ * @param vmExistingVolume The VM's existing volume (typically root volume)
+ * @param vm The VM to attach the volume to
+ * @return true if CLVM lock transfer is needed (but not full migration)
+ */
+ private boolean isClvmLockTransferRequired(VolumeInfo volumeToAttach, VolumeVO vmExistingVolume, UserVmVO vm) {
+ if (vm == null) {
+ return false;
+ }
+
+ Pair<StoragePoolVO, StoragePoolVO> pools = getStoragePoolsForVolumeAttachment(volumeToAttach, vmExistingVolume);
+ if (pools == null) {
+ return false;
+ }
+
+ StoragePoolVO volumePool = pools.first();
+ StoragePoolVO vmPool = pools.second();
+
+ Long vmHostId = vm.getHostId();
+ if (vmHostId == null) {
+ vmHostId = vm.getLastHostId();
+ }
+
+ return volService.isLockTransferRequired(volumeToAttach, volumePool.getPoolType(), vmPool.getPoolType(),
+ volumePool.getId(), vmPool.getId(), vmHostId);
+ }
+
+ /**
+ * Determines the destination host for CLVM lock migration.
+ *
+ * If VM is running, uses the VM's current host.
+ * If VM is stopped, picks an available UP host from the storage pool's cluster.
+ *
+ * @param vm The VM
+ * @param vmExistingVolume The VM's existing volume (to determine cluster)
+ * @return Host ID, or null if cannot be determined
+ */
+ private Long determineClvmLockDestinationHost(UserVmVO vm, VolumeVO vmExistingVolume) {
+ Long destHostId = vm.getHostId();
+ if (destHostId != null) {
+ return destHostId;
+ }
+
+ if (vmExistingVolume != null && vmExistingVolume.getPoolId() != null) {
+ StoragePoolVO pool = _storagePoolDao.findById(vmExistingVolume.getPoolId());
+ if (pool != null && pool.getClusterId() != null) {
+ List<HostVO> hosts = _hostDao.findByClusterId(pool.getClusterId());
+ if (hosts != null && !hosts.isEmpty()) {
+ // Pick first available UP host
+ for (HostVO host : hosts) {
+ if (host.getStatus() == Status.Up) {
+ destHostId = host.getId();
+ logger.debug("VM {} is stopped, selected host {} from cluster {} for CLVM lock migration",
+ vm.getUuid(), destHostId, pool.getClusterId());
+ return destHostId;
+ }
+ }
+ }
+ }
+ }
+
+ return null;
+ }
+
+ /**
+ * Executes CLVM lightweight migration with consistent logging and error handling.
+ *
+ * This helper method wraps the actual migration logic to eliminate code duplication
+ * between different CLVM migration scenarios (lock transfer vs. lightweight migration).
+ *
+ * @param volume The volume to migrate locks for
+ * @param vm The VM to attach the volume to
+ * @param vmExistingVolume The VM's existing volume (to determine target host)
+ * @param operationType Description of the operation type for logging (e.g., "CLVM lock transfer")
+ * @param scenarioDescription Description of the scenario for logging (e.g., "same pool to different host")
+ * @return Updated VolumeInfo after lock migration
+ * @throws CloudRuntimeException if migration fails
+ */
+ private VolumeInfo executeLightweightLockMigration(VolumeInfo volume, UserVmVO vm, VolumeVO vmExistingVolume,
+ String operationType, String scenarioDescription) {
+ logger.info("Performing {} for volume {} to VM {} ({})",
+ operationType, volume.getUuid(), vm.getUuid(), scenarioDescription);
+
+ try {
+ return performLightweightLockMigration(volume, vm, vmExistingVolume);
+ } catch (Exception e) {
+ logger.error("{} failed for volume {}: {}",
+ operationType, volume.getUuid(), e.getMessage(), e);
+ throw new CloudRuntimeException(operationType + " failed", e);
+ }
+ }
+
+ /**
+ * Performs lightweight CLVM lock migration for volume attachment.
+ * Delegates to VolumeService for the actual lock migration.
+ *
+ * @param volume The volume to migrate locks for
+ * @param vm The VM to attach the volume to
+ * @param vmExistingVolume The VM's existing volume (to determine target host)
+ * @return Updated VolumeInfo after lock migration
+ * @throws Exception if lock migration fails
+ */
+ private VolumeInfo performLightweightLockMigration(VolumeInfo volume, UserVmVO vm, VolumeVO vmExistingVolume) throws CloudRuntimeException {
+ Long destHostId = determineClvmLockDestinationHost(vm, vmExistingVolume);
+
+ if (destHostId == null) {
+ throw new CloudRuntimeException(
+ "Cannot determine destination host for CLVM lock migration - VM has no host and no available cluster hosts");
+ }
+
+ try {
+ return volService.performLockMigration(volume, destHostId);
+ } catch (CloudRuntimeException e) {
+ logger.error("CLVM lock migration failed for volume {}: {}", volume.getUuid(), e.getMessage(), e);
+ throw e;
+ }
+ }
+
+ /**
+ * Finds which host currently has the exclusive lock on a CLVM volume.
+ * Delegates to VolumeService for the actual lookup.
+ *
+ * @param volume The CLVM volume
+ * @return Host ID that has the exclusive lock, or null if cannot be determined
+ */
+ private Long findVolumeLockHost(VolumeInfo volume) {
+ return volService.findVolumeLockHost(volume);
+ }
+
+
Review Comment:
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4341153755
@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4327523509
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17617
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4518653515
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4518665989
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4519046238
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17990
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4407847301
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4409323888
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17795
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4409468020
@Pearl1594 a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4409456589
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4416709702
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [33.1% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
<!-- sqra:qg_revision=2810404f9bdbb08abd7243adeba83ed47ec8a40367ecfd74dc4787d0c336544b -->
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4555871555
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18062
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-5454390956
@blueorangutan sweep
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4070931131
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17162
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4169698111
<b>[SF] Trillian Build Failed (tid-15792)<b/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4169818158
@Pearl1594 a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4203786120
@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4200986652
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4201204523
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17389
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4239045854
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17474
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Luskan777 (via GitHub)" <gi...@apache.org>.
Luskan777 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4393902971
Thank you for your reply @Pearl1594.
Sorry for my ignorance @Pearl1594 , but why did you choose qcow2 and not RAW? Is it because of the snapshots? Or because it's thin-provisioning?
And will it be possible to use RAW in CLVM-NG at some point? Does that make sense? I ask because RAW is superior to QCOW2, as it has less overhead, resulting in better performance (IOPS and latency).
PS: If I'm missing anything that's in your documentation, please correct me 😬
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4391972100
## [](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617) **Quality Gate failed**
Failed conditions
 [1 Security Hotspot](https://sonarcloud.io/project/security_hotspots?id=apache_cloudstack&pullRequest=12617&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [33.1% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_cloudstack&pullRequest=12617&metric=new_coverage&view=list) (required ≥ 40%)
<!-- sqra:qg_revision=4eebb76251cbdcaa554183ffa8eb967a993638dd08b5cdc4f272ecbe53f678e3 -->
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_cloudstack&pullRequest=12617)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4390401438
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CLVM enhancements and fixes [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #12617:
URL: https://github.com/apache/cloudstack/pull/12617#issuecomment-4416677450
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17806
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org