You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by "slavkap (via GitHub)" <gi...@apache.org> on 2026/10/12 10:15:10 UTC
[PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
slavkap opened a new pull request, #14389:
URL: https://github.com/apache/cloudstack/pull/14389
### Description
This PR extends the import of unmanaged volumes capability to **StorPool** primary storage, and adds support for `importVm` from shared to work with StorPool primary storage as well.
With this change:
- Disk volumes that exist on a StorPool pool but are not yet tracked by CloudStack can be discovered and imported into CloudStack the same way volumes on other supported storage types already are.
- `importVm` with `importSource=SHARED` now also works correctly for StorPool pools, so an existing VM's shared disk(s) on StorPool can be imported as part of VM import, not just standalone volumes.
Once imported, a volume is fully managed by CloudStack: it participates in normal lifecycle operations and StorPool is kept informed of the change in ownership, so unmanaging it later correctly hands it back.
### Types of changes
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [X] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
- [ ] Build/CI
- [ ] Test (unit or integration test code)
### Feature/Enhancement Scale or Bug Severity
#### Feature/Enhancement Scale
- [X] Major
- [ ] Minor
### How Has This Been Tested?
Unit tests, Marvin integration tests, and manually verified `importVm` with `importSource=SHARED` against a VM with a
StorPool volume; list/import and unmanage StorPool volumes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#issuecomment-6314932284
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✖️ suse15. SL-JID 19643
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on code in PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#discussion_r4269917586
##########
plugins/storage/volume/storpool/src/main/java/org/apache/cloudstack/storage/datastore/driver/StorPoolPrimaryDataStoreDriver.java:
##########
@@ -1367,6 +1377,120 @@ public Pair<Long, Long> getVolumeStats(StoragePool storagePool, String volumeId)
return null;
}
+ /**
+ * Lists StorPool volumes available for import (globalId name, no "cs" tag unless
+ * {@code forUnmanage}). When {@code volumePath} is given, looks up that one volume directly
+ * instead of browsing the whole pool; when {@code forUnmanage} is also true, an already
+ * "cs"-tagged volume is still returned, since the caller is unmanageVolume() looking for the
+ * very volume it is about to drop from CloudStack.
+ */
+ @Override
+ public List<VolumeOnStorageTO> listVolumesForImport(StoragePool pool, String volumePath, String keyword, boolean forUnmanage) {
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.listVolumesForImport: pool=%s, volumePath=%s, keyword=%s, forUnmanage=%s", pool.getName(), volumePath, keyword, forUnmanage);
+ SpConnectionDesc conn = StorPoolUtil.getSpConnection(pool.getUuid(), pool.getId(), storagePoolDetailsDao, primaryStoreDao);
+ List<VolumeOnStorageTO> volumes = new ArrayList<>();
+
+ if (StringUtils.isNotBlank(volumePath)) {
+ return getVolumeOnStorageByPath(pool, volumePath, conn, volumes, forUnmanage);
+ }
+
+ SpApiResponse resp = StorPoolUtil.volumesListResponse(conn);
+ if (resp.getError() != null) {
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.listVolumesForImport: could not list volumes for pool=%s. Error: %s", pool.getName(), resp.getError());
+ return volumes;
+ }
+ return getVolumeOnStorage(pool, keyword, resp, volumes);
+ }
+
+ @NotNull
+ private List<VolumeOnStorageTO> getVolumeOnStorage(StoragePool pool, String keyword, SpApiResponse resp, List<VolumeOnStorageTO> volumes) {
+ JsonArray volumesList = resp.fullJson.getAsJsonObject().getAsJsonArray("data");
+
+ for (int i = 0; i < volumesList.size(); i++) {
+ JsonObject volume = volumesList.get(i).getAsJsonObject();
+ String rawName = volume.get("name").getAsString();
+ if (isVolumeDeleted(volume) || !isEligibleForBulkImport(volume, rawName)) {
+ continue;
+ }
+
+ String name = rawName.substring(1);
+
+ if (StringUtils.isNotBlank(keyword) && !name.contains(keyword)) {
+ continue;
+ }
+ volumes.add(toVolumeOnStorageTO(volume, name));
+ }
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.listVolumesForImport: found %s volume(s) for pool=%s", volumes.size(), pool.getName());
+ return volumes;
+ }
+
+ @NotNull
+ private List<VolumeOnStorageTO> getVolumeOnStorageByPath(StoragePool pool, String volumePath, SpConnectionDesc conn, List<VolumeOnStorageTO> volumes, boolean forUnmanage) {
+ String name = StorPoolStorageAdaptor.getVolumeNameFromPath(volumePath, true);
+ SpApiResponse resp = StorPoolUtil.volumeDescribeResponse(name, conn);
+ if (resp.getError() != null) {
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.listVolumesForImport: could not describe volume=%s on pool=%s. Error: %s", name, pool.getName(), resp.getError());
+ return volumes;
+ }
+ JsonArray data = resp.fullJson.getAsJsonObject().getAsJsonArray("data");
+ if (data == null || data.isEmpty()) {
+ return volumes;
+ }
+ JsonObject volume = data.get(0).getAsJsonObject();
+ String rawName = volume.get("name").getAsString();
+ if (isVolumeDeleted(volume) || !isGlobalIdVolumeName(rawName) || (!forUnmanage && isTaggedCs(volume))) {
+ return volumes;
+ }
+ volumes.add(toVolumeOnStorageTO(volume, rawName.substring(1)));
+ return volumes;
+ }
+
+ private static boolean isVolumeDeleted(JsonObject volume) {
+ return volume.has("deleted") && volume.get("deleted").getAsBoolean();
+ }
+
+ private static boolean isGlobalIdVolumeName(String rawName) {
+ return !rawName.startsWith("*") && !rawName.contains("@") && rawName.startsWith("~");
+ }
+
+ private static boolean isTaggedCs(JsonObject volume) {
+ JsonObject tags = volume.has("tags") && !volume.get("tags").isJsonNull() ? volume.getAsJsonObject("tags") : null;
+ return tags != null && tags.has("cs");
+ }
+
+ private static boolean isEligibleForBulkImport(JsonObject volume, String rawName) {
+ return isGlobalIdVolumeName(rawName) && !isTaggedCs(volume);
+ }
+
+ private VolumeOnStorageTO toVolumeOnStorageTO(JsonObject volume, String name) {
+ String path = StorPoolUtil.devPath(name);
+ long size = volume.get("size").getAsLong();
+ long spaceUsed = volume.has("spaceUsed") ? volume.get("spaceUsed").getAsLong() : size;
+ StorPoolUtil.spLog("listVolumesForImport: found volume=%s, path=%s, size=%s, spaceUsed=%s", name, path, size, spaceUsed);
+ return new VolumeOnStorageTO(Hypervisor.HypervisorType.KVM, path, name, path,
+ Storage.ImageFormat.QCOW2.toString(), spaceUsed, size);
+ }
+
+ @Override
+ public void volumeManagementChanged(StoragePool pool, Volume volume, boolean imported) {
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.volumeManagementChanged: pool=%s, volume=%s, imported=%s", pool.getName(), volume.getPath(), imported);
+ SpConnectionDesc conn = StorPoolUtil.getSpConnection(pool.getUuid(), pool.getId(), storagePoolDetailsDao, primaryStoreDao);
+ String name = StorPoolStorageAdaptor.getVolumeNameFromPath(volume.getPath(), true);
+ Map<String, String> tags;
+ if (imported) {
+ String tier = volume.getDiskOfferingId() != null ? getTierFromOfferingDetail(volume.getDiskOfferingId()) : null;
+ Long instanceId = volume.getInstanceId();
+ tags = StorPoolHelper.addStorPoolTags(volume.getUuid(), getVMInstanceUUID(instanceId), "volume", getVcPolicyTag(instanceId), tier);
+ } else {
+ tags = StorPoolHelper.addStorPoolTags(null, null, "", null, null);
Review Comment:
that covers it
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/wrapper/LibvirtCheckVolumeCommandWrapper.java:
##########
@@ -63,27 +71,37 @@ public Answer execute(final CheckVolumeCommand command, final LibvirtComputingRe
try {
if (STORAGE_POOL_TYPES_SUPPORTED.contains(storageFilerTO.getType())) {
- final KVMPhysicalDisk vol = pool.getPhysicalDisk(srcFile);
- final String path = vol.getPath();
- final boolean isRbd = Storage.StoragePoolType.RBD.equals(storageFilerTO.getType());
-
- Map<VolumeOnStorageTO.Detail, String> volumeDetails = getVolumeDetails(pool, vol);
- if (MapUtils.isEmpty(volumeDetails)) {
- return new Answer(command, false, "Unable to read the volume on the storage pool");
+ final boolean shouldAttachTheVolume = POOL_NEEDS_TO_EXPOSE_THE_VOLUME.contains(storageFilerTO.getType());
+ if (shouldAttachTheVolume && !pool.connectPhysicalDisk(srcFile, new HashMap<>())) {
+ return new Answer(command, false, "Unable to attach the volume on the storage pool");
}
+ try {
+ final KVMPhysicalDisk vol = pool.getPhysicalDisk(srcFile);
+ final String path = vol.getPath();
+ final boolean isRawBlockPool = RAW_BLOCK_STORAGE_POOL_TYPES.contains(storageFilerTO.getType());
+
+ Map<VolumeOnStorageTO.Detail, String> volumeDetails = getVolumeDetails(pool, vol);
+ if (MapUtils.isEmpty(volumeDetails)) {
+ return new Answer(command, false, "Unable to read the volume on the storage pool");
+ }
- if (!isRbd) {
- try {
- KVMPhysicalDisk.checkQcow2File(path);
- } catch (final CloudRuntimeException e) {
- return new CheckVolumeAnswer(command, false, "", 0, volumeDetails);
+ if (!isRawBlockPool) {
+ try {
+ KVMPhysicalDisk.checkQcow2File(path);
+ } catch (final CloudRuntimeException e) {
+ return new CheckVolumeAnswer(command, false, "", 0, volumeDetails);
+ }
}
- }
- // Images on RBD are raw and the path is an image name that qemu-img cannot open
- // without the rbd: URI, so take the size libvirt already reported for the volume.
- long size = isRbd ? vol.getVirtualSize() : KVMPhysicalDisk.getVirtualSizeFromFile(path);
- return new CheckVolumeAnswer(command, true, "", size, volumeDetails);
+ // Images on RBD are raw and the path is an image name that qemu-img cannot open
+ // without the rbd: URI, so take the size libvirt already reported for the volume
+ long size = isRawBlockPool ? vol.getVirtualSize() : KVMPhysicalDisk.getVirtualSizeFromFile(path);
+ return new CheckVolumeAnswer(command, true, "", size, volumeDetails);
+ } finally {
+ if (shouldAttachTheVolume) {
+ pool.disconnectPhysicalDisk(srcFile);
Review Comment:
that covers it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on code in PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#discussion_r4296844998
##########
plugins/storage/volume/storpool/src/main/java/org/apache/cloudstack/storage/datastore/driver/StorPoolPrimaryDataStoreDriver.java:
##########
@@ -1367,6 +1377,120 @@ public Pair<Long, Long> getVolumeStats(StoragePool storagePool, String volumeId)
return null;
}
+ /**
+ * Lists StorPool volumes available for import (globalId name, no "cs" tag unless
+ * {@code forUnmanage}). When {@code volumePath} is given, looks up that one volume directly
+ * instead of browsing the whole pool; when {@code forUnmanage} is also true, an already
+ * "cs"-tagged volume is still returned, since the caller is unmanageVolume() looking for the
+ * very volume it is about to drop from CloudStack.
+ */
+ @Override
+ public List<VolumeOnStorageTO> listVolumesForImport(StoragePool pool, String volumePath, String keyword, boolean forUnmanage) {
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.listVolumesForImport: pool=%s, volumePath=%s, keyword=%s, forUnmanage=%s", pool.getName(), volumePath, keyword, forUnmanage);
+ SpConnectionDesc conn = StorPoolUtil.getSpConnection(pool.getUuid(), pool.getId(), storagePoolDetailsDao, primaryStoreDao);
+ List<VolumeOnStorageTO> volumes = new ArrayList<>();
+
+ if (StringUtils.isNotBlank(volumePath)) {
+ return getVolumeOnStorageByPath(pool, volumePath, conn, volumes, forUnmanage);
+ }
+
+ SpApiResponse resp = StorPoolUtil.volumesListResponse(conn);
Review Comment:
ok, makes sense. once a disk is imported it gets the cs tag and drops off every pool's list, so it cant come in twice
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "slavkap (via GitHub)" <gi...@apache.org>.
slavkap commented on code in PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#discussion_r4247547740
##########
plugins/storage/volume/storpool/src/test/java/org/apache/cloudstack/storage/datastore/driver/StorPoolPrimaryDataStoreDriverTest.java:
##########
@@ -214,6 +236,345 @@ public void testCopyVolumeNotAttachedToVmPassed() {
Assert.assertNull(response.getError());
}
+ @Test
+ public void testListVolumesForImportReturnsGlobalIdVolume() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(volumeJson("~t.t.t", null, false, 1073741824L, 524288000L));
+ when(StorPoolUtil.volumesListResponse(conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, null, null, false);
+
+ Assert.assertEquals(1, volumes.size());
+ VolumeOnStorageTO volume = volumes.get(0);
+ Assert.assertEquals("t.t.t", volume.getName());
+ Assert.assertEquals("/dev/storpool-byid/t.t.t", volume.getPath());
+ Assert.assertEquals("/dev/storpool-byid/t.t.t", volume.getFullPath());
+ Assert.assertEquals(Storage.ImageFormat.QCOW2.toString(), volume.getFormat());
+ Assert.assertEquals(524288000L, volume.getSize());
+ Assert.assertEquals(1073741824L, volume.getVirtualSize());
+ }
+
+ @Test
+ public void testListVolumesForImportSkipsVolumeTaggedCs() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ JsonObject tagged = volumeJsonObject("~t.t.t", 1073741824L, 1073741824L);
+ JsonObject tags = new JsonObject();
+ tags.addProperty("cs", "volume");
+ tagged.add("tags", tags);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(tagged);
+ when(StorPoolUtil.volumesListResponse(conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, null, null, false);
+
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportSkipsDeletedVolume() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(volumeJson("~t.t.t", null, true, 1073741824L, 1073741824L));
+ when(StorPoolUtil.volumesListResponse(conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, null, null, false);
+
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportSkipsNonGlobalIdAndCloneNames() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ JsonArray data = new JsonArray();
+ data.add(volumeJsonObject("friendlyname", 1073741824L, 1073741824L));
+ data.add(volumeJsonObject("*s.s.s", 1073741824L, 1073741824L));
+ data.add(volumeJsonObject("~s.s.s@s.s", 1073741824L, 1073741824L));
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(data);
+ when(StorPoolUtil.volumesListResponse(conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, null, null, false);
+
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportFiltersByKeyword() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ JsonArray data = new JsonArray();
+ data.add(volumeJsonObject("~t.t.t", 1073741824L, 1073741824L));
+ data.add(volumeJsonObject("~s.s.s", 1073741824L, 1073741824L));
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(data);
+ when(StorPoolUtil.volumesListResponse(conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, null, "s.s.s", false);
+
+ Assert.assertEquals(1, volumes.size());
+ Assert.assertEquals("s.s.s", volumes.get(0).getName());
+ }
+
+ @Test
+ public void testListVolumesForImportApiErrorReturnsEmptyList() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = new StorPoolUtil.SpApiResponse();
+ setResponseError(resp);
+ when(StorPoolUtil.volumesListResponse(conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, null, null, false);
+
+ Assert.assertNotNull(volumes);
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathReturnsVolume() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(volumeJson("~t.t.t", null, false, 1073741824L, 524288000L));
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, false);
+
+ Assert.assertEquals(1, volumes.size());
+ VolumeOnStorageTO volume = volumes.get(0);
+ Assert.assertEquals("t.t.t", volume.getName());
+ Assert.assertEquals("/dev/storpool-byid/t.t.t", volume.getPath());
+ Assert.assertEquals("/dev/storpool-byid/t.t.t", volume.getFullPath());
+ Assert.assertEquals(Storage.ImageFormat.QCOW2.toString(), volume.getFormat());
+ Assert.assertEquals(524288000L, volume.getSize());
+ Assert.assertEquals(1073741824L, volume.getVirtualSize());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathExcludesVolumeTaggedCs() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ JsonObject tagged = volumeJsonObject("~t.t.t", 1073741824L, 1073741824L);
+ JsonObject tags = new JsonObject();
+ tags.addProperty("cs", "volume");
+ tagged.add("tags", tags);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(tagged);
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, false);
+
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathForUnmanageReturnsVolumeEvenIfTaggedCs() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ JsonObject tagged = volumeJsonObject("~t.t.t", 1073741824L, 1073741824L);
+ JsonObject tags = new JsonObject();
+ tags.addProperty("cs", "volume");
+ tagged.add("tags", tags);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(tagged);
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, true);
+
+ Assert.assertEquals(1, volumes.size());
+ Assert.assertEquals("t.t.t", volumes.get(0).getName());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathForUnmanageSkipsNonGlobalIdName() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(volumeJsonObject("~t.t.t@s.s", 1073741824L, 1073741824L));
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, true);
+
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathSkipsDeletedVolume() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(volumeJson("~t.t.t", null, true, 1073741824L, 1073741824L));
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, false);
+
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathNotFoundReturnsEmptyList() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(new JsonArray());
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, false);
+
+ Assert.assertNotNull(volumes);
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathApiErrorReturnsEmptyList() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = new StorPoolUtil.SpApiResponse();
+ setResponseError(resp);
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, false);
+
+ Assert.assertNotNull(volumes);
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testCreateAsyncSetsVolumeFormatToQcow2() {
+ DataStore dataStore = mock(DataStore.class);
+ when(dataStore.getUuid()).thenReturn("SP_API_HTTP=1.1.1.1:81;SP_AUTH_TOKEN=token;SP_TEMPLATE=template_name");
+ when(dataStore.getId()).thenReturn(1L);
+
+ VolumeInfo vinfo = mock(VolumeInfo.class);
+ when(vinfo.getType()).thenReturn(DataObjectType.VOLUME);
+ when(vinfo.getUuid()).thenReturn(UUID.randomUUID().toString());
+ when(vinfo.getId()).thenReturn(101L);
+ when(vinfo.getSize()).thenReturn(1073741824L);
+ when(vinfo.getPassphraseId()).thenReturn(null);
+ when(vinfo.getInstanceId()).thenReturn(null);
+ when(vinfo.getDiskOfferingId()).thenReturn(null);
+
+ when(StorPoolUtil.getSpConnection(dataStore.getUuid(), dataStore.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = new StorPoolUtil.SpApiResponse();
+ when(StorPoolUtil.volumeCreate(any(StorPoolVolumeDef.class), eq(conn))).thenReturn(resp);
+ when(StorPoolUtil.getNameFromResponse(resp, false)).thenReturn("t.t.t");
+
+ VolumeVO volumeVO = mock(VolumeVO.class);
+ when(volumeVO.getId()).thenReturn(101L);
+ when(volumeDao.findById(101L)).thenReturn(volumeVO);
+
+ StoragePoolVO storagePoolVO = mock(StoragePoolVO.class);
+ when(storagePool.findById(1L)).thenReturn(storagePoolVO);
+
+ @SuppressWarnings("unchecked")
+ AsyncCompletionCallback<CreateCmdResult> createCallback = mock(AsyncCompletionCallback.class);
+
+ storPoolPrimaryDataStoreDriver.createAsync(dataStore, vinfo, createCallback);
+
+ verify(volumeVO).setFormat(Storage.ImageFormat.QCOW2);
+ verify(volumeDao).update(eq(101L), eq(volumeVO));
+ }
+
+ @Test
+ public void testVolumeImportedTagsVolumeWithCsAndUuid() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ String volumeUuid = UUID.randomUUID().toString();
+ VolumeVO volume = mock(VolumeVO.class);
+ when(volume.getPath()).thenReturn("/dev/storpool-byid/t.t.t");
+ when(volume.getUuid()).thenReturn(volumeUuid);
+ when(volume.getDiskOfferingId()).thenReturn(5L);
+ when(volume.getInstanceId()).thenReturn(null);
+ when(diskOfferingDetailsDao.findDetail(5L, StorPoolUtil.SP_TIER)).thenReturn(null);
+ StorPoolUtil.SpApiResponse resp = new StorPoolUtil.SpApiResponse();
+ when(StorPoolUtil.volumeUpdate(any(StorPoolVolumeDef.class), eq(conn))).thenReturn(resp);
+
+ storPoolPrimaryDataStoreDriver.volumeManagementChanged(importPool, volume, true);
+
+ ArgumentCaptor<StorPoolVolumeDef> captor = ArgumentCaptor.forClass(StorPoolVolumeDef.class);
+ utilities.verify(() -> StorPoolUtil.volumeUpdate(captor.capture(), eq(conn)), times(1));
+ StorPoolVolumeDef updated = captor.getValue();
+ Assert.assertEquals("~t.t.t", updated.getName());
+ Assert.assertEquals(volumeUuid, updated.getTags().get("uuid"));
+ Assert.assertEquals("volume", updated.getTags().get("cs"));
+ Assert.assertNull(updated.getTags().get("qc"));
+ Assert.assertNull(updated.getTags().get("cvm"));
+ }
+
+ @Test
+ public void testVolumeImportedSetsCvmTagWhenAttachedToVm() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ VolumeVO volume = mock(VolumeVO.class);
+ when(volume.getPath()).thenReturn("/dev/storpool-byid/t.t.t");
+ when(volume.getUuid()).thenReturn(UUID.randomUUID().toString());
+ when(volume.getInstanceId()).thenReturn(7L);
+ VMInstanceVO vm = mock(VMInstanceVO.class);
+ String vmUuid = UUID.randomUUID().toString();
+ when(vm.getUuid()).thenReturn(vmUuid);
+ when(vmInstanceDao.findById(7L)).thenReturn(vm);
+ StorPoolUtil.SpApiResponse resp = new StorPoolUtil.SpApiResponse();
+ when(StorPoolUtil.volumeUpdate(any(StorPoolVolumeDef.class), eq(conn))).thenReturn(resp);
+
+ storPoolPrimaryDataStoreDriver.volumeManagementChanged(importPool, volume, true);
+
+ ArgumentCaptor<StorPoolVolumeDef> captor = ArgumentCaptor.forClass(StorPoolVolumeDef.class);
+ utilities.verify(() -> StorPoolUtil.volumeUpdate(captor.capture(), eq(conn)), times(1));
+ Assert.assertEquals(vmUuid, captor.getValue().getTags().get("cvm"));
+ }
+
+ @Test
+ public void testVolumeImportedIncludesQosTierFromDiskOffering() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ VolumeVO volume = mock(VolumeVO.class);
+ when(volume.getPath()).thenReturn("/dev/storpool-byid/t.t.t");
+ when(volume.getUuid()).thenReturn(UUID.randomUUID().toString());
+ when(volume.getDiskOfferingId()).thenReturn(5L);
+ when(volume.getInstanceId()).thenReturn(null);
+ DiskOfferingDetailVO tierDetail = mock(DiskOfferingDetailVO.class);
+ when(tierDetail.getValue()).thenReturn("ssd-tier");
+ when(diskOfferingDetailsDao.findDetail(5L, StorPoolUtil.SP_TIER)).thenReturn(tierDetail);
+ StorPoolUtil.SpApiResponse resp = new StorPoolUtil.SpApiResponse();
+ when(StorPoolUtil.volumeUpdate(any(StorPoolVolumeDef.class), eq(conn))).thenReturn(resp);
+
+ storPoolPrimaryDataStoreDriver.volumeManagementChanged(importPool, volume, true);
+
+ ArgumentCaptor<StorPoolVolumeDef> captor = ArgumentCaptor.forClass(StorPoolVolumeDef.class);
+ utilities.verify(() -> StorPoolUtil.volumeUpdate(captor.capture(), eq(conn)), times(1));
+ Assert.assertEquals("ssd-tier", captor.getValue().getTags().get("qc"));
+ }
+
+ @Test
+ public void testVolumeUnmanagedClearsCsUuidAndQosTags() {
Review Comment:
No new unit test needed here — given the clarification on #1, a unit test asserting `cs==""` is treated as "unmanaged" would be testing the wrong thing (it assumes the JSON we send is literally what a later read-back returns, which isn't how StorPool's tag-removal semantics work).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "nvazquez (via GitHub)" <gi...@apache.org>.
nvazquez commented on PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#issuecomment-6293494564
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "slavkap (via GitHub)" <gi...@apache.org>.
slavkap commented on code in PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#discussion_r4247503994
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/wrapper/LibvirtCheckVolumeCommandWrapper.java:
##########
@@ -51,7 +51,15 @@ public final class LibvirtCheckVolumeCommandWrapper extends CommandWrapper<Check
Storage.StoragePoolType.Filesystem,
Storage.StoragePoolType.NetworkFilesystem,
Storage.StoragePoolType.SharedMountPoint,
- Storage.StoragePoolType.RBD);
+ Storage.StoragePoolType.RBD,
+ Storage.StoragePoolType.StorPool);
+
+ private static final List<Storage.StoragePoolType> RAW_BLOCK_STORAGE_POOL_TYPES = Arrays.asList(
+ Storage.StoragePoolType.RBD,
+ Storage.StoragePoolType.StorPool);
Review Comment:
This is intentional, not inconsistent. `format=QCOW2` in CloudStack's DB for StorPool volumes is a long-standing labeling convention (see `StorPoolDataMotionStrategy.setFormat(ImageFormat.QCOW2)`, which predates this PR) — it doesn't mean the bytes on the StorPool block device are actually structured as a QCOW2 container. On disk, a StorPool volume is a raw block device, same as RBD, so grouping it with RBD in `RAW_BLOCK_STORAGE_POOL_TYPES` (skip QCOW2 structural validation, read size via the libvirt-reported virtual size) is correct. Also worth noting checkQcow2File() is a no-op for any path without a .qcow2 extension, so it wouldn't have done anything differently for StorPool paths anyway. We'll leave this as is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#issuecomment-6122382804
## [Codecov](https://app.codecov.io/gh/apache/cloudstack/pull/14389?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 4.04%. Comparing base ([`0931cf8`](https://app.codecov.io/gh/apache/cloudstack/commit/0931cf818e6b1e620e158a97164bafbc8e31e0c8?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`10c7a3f`](https://app.codecov.io/gh/apache/cloudstack/commit/10c7a3f9d969eda153522cc36674df5a557338ab?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
> :exclamation: There is a different number of reports uploaded between BASE (0931cf8) and HEAD (10c7a3f). Click for more details.
>
> <details><summary>HEAD has 1 upload less than BASE</summary>
>
>| Flag | BASE (0931cf8) | HEAD (10c7a3f) |
>|------|------|------|
>|unittests|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## 4.22 #14389 +/- ##
=============================================
- Coverage 18.02% 4.04% -13.98%
=============================================
Files 5936 449 -5487
Lines 535866 38240 -497626
Branches 65614 7082 -58532
=============================================
- Hits 96574 1546 -95028
+ Misses 428291 36482 -391809
+ Partials 11001 212 -10789
```
| [Flag](https://app.codecov.io/gh/apache/cloudstack/pull/14389/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [uitests](https://app.codecov.io/gh/apache/cloudstack/pull/14389/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `4.04% <ø> (ø)` | |
| [unittests](https://app.codecov.io/gh/apache/cloudstack/pull/14389/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/cloudstack/pull/14389?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "nvazquez (via GitHub)" <gi...@apache.org>.
nvazquez merged PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "slavkap (via GitHub)" <gi...@apache.org>.
slavkap commented on code in PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#discussion_r4269860829
##########
plugins/storage/volume/storpool/src/main/java/org/apache/cloudstack/storage/datastore/driver/StorPoolPrimaryDataStoreDriver.java:
##########
@@ -1367,6 +1377,120 @@ public Pair<Long, Long> getVolumeStats(StoragePool storagePool, String volumeId)
return null;
}
+ /**
+ * Lists StorPool volumes available for import (globalId name, no "cs" tag unless
+ * {@code forUnmanage}). When {@code volumePath} is given, looks up that one volume directly
+ * instead of browsing the whole pool; when {@code forUnmanage} is also true, an already
+ * "cs"-tagged volume is still returned, since the caller is unmanageVolume() looking for the
+ * very volume it is about to drop from CloudStack.
+ */
+ @Override
+ public List<VolumeOnStorageTO> listVolumesForImport(StoragePool pool, String volumePath, String keyword, boolean forUnmanage) {
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.listVolumesForImport: pool=%s, volumePath=%s, keyword=%s, forUnmanage=%s", pool.getName(), volumePath, keyword, forUnmanage);
+ SpConnectionDesc conn = StorPoolUtil.getSpConnection(pool.getUuid(), pool.getId(), storagePoolDetailsDao, primaryStoreDao);
+ List<VolumeOnStorageTO> volumes = new ArrayList<>();
+
+ if (StringUtils.isNotBlank(volumePath)) {
+ return getVolumeOnStorageByPath(pool, volumePath, conn, volumes, forUnmanage);
+ }
+
+ SpApiResponse resp = StorPoolUtil.volumesListResponse(conn);
Review Comment:
it was internal decision to list all volumes on the StorPool cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "nvazquez (via GitHub)" <gi...@apache.org>.
nvazquez commented on PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#issuecomment-6291708235
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "slavkap (via GitHub)" <gi...@apache.org>.
slavkap commented on code in PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#discussion_r4269877892
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/wrapper/LibvirtCheckVolumeCommandWrapper.java:
##########
@@ -63,27 +71,37 @@ public Answer execute(final CheckVolumeCommand command, final LibvirtComputingRe
try {
if (STORAGE_POOL_TYPES_SUPPORTED.contains(storageFilerTO.getType())) {
- final KVMPhysicalDisk vol = pool.getPhysicalDisk(srcFile);
- final String path = vol.getPath();
- final boolean isRbd = Storage.StoragePoolType.RBD.equals(storageFilerTO.getType());
-
- Map<VolumeOnStorageTO.Detail, String> volumeDetails = getVolumeDetails(pool, vol);
- if (MapUtils.isEmpty(volumeDetails)) {
- return new Answer(command, false, "Unable to read the volume on the storage pool");
+ final boolean shouldAttachTheVolume = POOL_NEEDS_TO_EXPOSE_THE_VOLUME.contains(storageFilerTO.getType());
+ if (shouldAttachTheVolume && !pool.connectPhysicalDisk(srcFile, new HashMap<>())) {
+ return new Answer(command, false, "Unable to attach the volume on the storage pool");
}
+ try {
+ final KVMPhysicalDisk vol = pool.getPhysicalDisk(srcFile);
+ final String path = vol.getPath();
+ final boolean isRawBlockPool = RAW_BLOCK_STORAGE_POOL_TYPES.contains(storageFilerTO.getType());
+
+ Map<VolumeOnStorageTO.Detail, String> volumeDetails = getVolumeDetails(pool, vol);
+ if (MapUtils.isEmpty(volumeDetails)) {
+ return new Answer(command, false, "Unable to read the volume on the storage pool");
+ }
- if (!isRbd) {
- try {
- KVMPhysicalDisk.checkQcow2File(path);
- } catch (final CloudRuntimeException e) {
- return new CheckVolumeAnswer(command, false, "", 0, volumeDetails);
+ if (!isRawBlockPool) {
+ try {
+ KVMPhysicalDisk.checkQcow2File(path);
+ } catch (final CloudRuntimeException e) {
+ return new CheckVolumeAnswer(command, false, "", 0, volumeDetails);
+ }
}
- }
- // Images on RBD are raw and the path is an image name that qemu-img cannot open
- // without the rbd: URI, so take the size libvirt already reported for the volume.
- long size = isRbd ? vol.getVirtualSize() : KVMPhysicalDisk.getVirtualSizeFromFile(path);
- return new CheckVolumeAnswer(command, true, "", size, volumeDetails);
+ // Images on RBD are raw and the path is an image name that qemu-img cannot open
+ // without the rbd: URI, so take the size libvirt already reported for the volume
+ long size = isRawBlockPool ? vol.getVirtualSize() : KVMPhysicalDisk.getVirtualSizeFromFile(path);
+ return new CheckVolumeAnswer(command, true, "", size, volumeDetails);
+ } finally {
+ if (shouldAttachTheVolume) {
+ pool.disconnectPhysicalDisk(srcFile);
Review Comment:
thanks @Damans227!
- Removed this whole attach/detach path instead of trying to make it safe. StorPool volume info is now read directly through the driver's StorPool API call(listVolumesForImport) from the management server, so there's no need to attach/detach on a host at all anymore - LibvirtCheckVolumeCommandWrapper no longer handles.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "slavkap (via GitHub)" <gi...@apache.org>.
slavkap commented on code in PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#discussion_r4323459291
##########
server/src/main/java/org/apache/cloudstack/vm/UnmanagedVMsManagerImpl.java:
##########
@@ -2976,27 +2997,51 @@ private UserVm importKvmVirtualMachineFromDisk(final ImportSource importSource,
Map<Volume, StoragePool> storage = dest.getStorageForDisks();
Volume volume = volumeDao.findById(diskProfile.getVolumeId());
StoragePool storagePool = storage.get(volume);
- CheckVolumeCommand checkVolumeCommand = new CheckVolumeCommand();
- checkVolumeCommand.setSrcFile(diskPath);
- StorageFilerTO storageTO = new StorageFilerTO(storagePool);
- checkVolumeCommand.setStorageFilerTO(storageTO);
- Answer answer = agentManager.easySend(dest.getHost().getId(), checkVolumeCommand);
- if (!(answer instanceof CheckVolumeAnswer)) {
- cleanupFailedImportVM(userVm);
- throw new CloudRuntimeException("Disk not found or is invalid");
- }
- CheckVolumeAnswer checkVolumeAnswer = (CheckVolumeAnswer) answer;
- try {
- checkVolume(checkVolumeAnswer.getVolumeDetails());
- } catch (CloudRuntimeException e) {
- cleanupFailedImportVM(userVm);
- throw e;
- }
- if (!checkVolumeAnswer.getResult()) {
- cleanupFailedImportVM(userVm);
- throw new CloudRuntimeException("Disk not found or is invalid");
+
+ Map<VolumeOnStorageTO.Detail, String> volumeDetails;
+ long volumeSize;
+ List<VolumeOnStorageTO> volumesFromDriver = listVolumesForImportFromDriver(storagePool, diskPath);
+ if (volumesFromDriver != null) {
+ // The storage driver can enumerate its own volumes (e.g. StorPool, via its
+ // management API), so there is no need to reach the host agent just to read size.
+ if (CollectionUtils.isEmpty(volumesFromDriver)) {
+ cleanupFailedImportVM(userVm);
+ throw new CloudRuntimeException("Disk not found or is invalid");
+ }
+ VolumeOnStorageTO volumeOnStorage = volumesFromDriver.get(0);
+ volumeDetails = volumeOnStorage.getDetails();
+ try {
+ checkVolume(volumeDetails);
+ } catch (CloudRuntimeException e) {
+ cleanupFailedImportVM(userVm);
+ throw e;
+ }
+ volumeSize = volumeOnStorage.getVirtualSize();
+ } else {
+ CheckVolumeCommand checkVolumeCommand = new CheckVolumeCommand();
+ checkVolumeCommand.setSrcFile(diskPath);
+ StorageFilerTO storageTO = new StorageFilerTO(storagePool);
+ checkVolumeCommand.setStorageFilerTO(storageTO);
+ Answer answer = agentManager.easySend(dest.getHost().getId(), checkVolumeCommand);
+ if (!(answer instanceof CheckVolumeAnswer)) {
+ cleanupFailedImportVM(userVm);
+ throw new CloudRuntimeException("Disk not found or is invalid");
+ }
+ CheckVolumeAnswer checkVolumeAnswer = (CheckVolumeAnswer) answer;
+ volumeDetails = checkVolumeAnswer.getVolumeDetails();
+ try {
+ checkVolume(volumeDetails);
+ } catch (CloudRuntimeException e) {
+ cleanupFailedImportVM(userVm);
+ throw e;
+ }
+ if (!checkVolumeAnswer.getResult()) {
+ cleanupFailedImportVM(userVm);
+ throw new CloudRuntimeException("Disk not found or is invalid");
+ }
+ volumeSize = checkVolumeAnswer.getSize();
Review Comment:
thank! I extracted it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "slavkap (via GitHub)" <gi...@apache.org>.
slavkap commented on code in PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#discussion_r4323455611
##########
server/src/main/java/org/apache/cloudstack/storage/volume/VolumeImportUnmanageManagerImpl.java:
##########
@@ -232,9 +241,11 @@ public VolumeResponse importVolume(ImportVolumeCmd cmd) {
}
}
- protected VolumeOnStorageTO getVolumeOnStorageAndCheck(StoragePoolVO pool, String volumePath) {
+ protected VolumeOnStorageTO getVolumeOnStorageAndCheck(StoragePoolVO pool, String volumePath, boolean forUnmanage) {
// send a command to hypervisor to check
- List<VolumeOnStorageTO> volumes = listVolumesForImportInternal(pool, volumePath, null);
+ List<VolumeOnStorageTO> volumes = forUnmanage
+ ? listVolumesForImportInternal(pool, volumePath, null, true)
+ : listVolumesForImportInternal(pool, volumePath, null);
Review Comment:
thanks, @nvazquez! Made the change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "nvazquez (via GitHub)" <gi...@apache.org>.
nvazquez commented on code in PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#discussion_r4296748950
##########
server/src/main/java/org/apache/cloudstack/storage/volume/VolumeImportUnmanageManagerImpl.java:
##########
@@ -232,9 +241,11 @@ public VolumeResponse importVolume(ImportVolumeCmd cmd) {
}
}
- protected VolumeOnStorageTO getVolumeOnStorageAndCheck(StoragePoolVO pool, String volumePath) {
+ protected VolumeOnStorageTO getVolumeOnStorageAndCheck(StoragePoolVO pool, String volumePath, boolean forUnmanage) {
// send a command to hypervisor to check
- List<VolumeOnStorageTO> volumes = listVolumesForImportInternal(pool, volumePath, null);
+ List<VolumeOnStorageTO> volumes = forUnmanage
+ ? listVolumesForImportInternal(pool, volumePath, null, true)
+ : listVolumesForImportInternal(pool, volumePath, null);
Review Comment:
In both cases the method `listVolumesForImportInternal` will be invoked but with different values for `forUnmanage`. Would it make sense to simplify this to:
```suggestion
List<VolumeOnStorageTO> volumes = listVolumesForImportInternal(pool, volumePath, null, forUnmanage)
```
##########
server/src/main/java/org/apache/cloudstack/vm/UnmanagedVMsManagerImpl.java:
##########
@@ -2976,27 +2997,51 @@ private UserVm importKvmVirtualMachineFromDisk(final ImportSource importSource,
Map<Volume, StoragePool> storage = dest.getStorageForDisks();
Volume volume = volumeDao.findById(diskProfile.getVolumeId());
StoragePool storagePool = storage.get(volume);
- CheckVolumeCommand checkVolumeCommand = new CheckVolumeCommand();
- checkVolumeCommand.setSrcFile(diskPath);
- StorageFilerTO storageTO = new StorageFilerTO(storagePool);
- checkVolumeCommand.setStorageFilerTO(storageTO);
- Answer answer = agentManager.easySend(dest.getHost().getId(), checkVolumeCommand);
- if (!(answer instanceof CheckVolumeAnswer)) {
- cleanupFailedImportVM(userVm);
- throw new CloudRuntimeException("Disk not found or is invalid");
- }
- CheckVolumeAnswer checkVolumeAnswer = (CheckVolumeAnswer) answer;
- try {
- checkVolume(checkVolumeAnswer.getVolumeDetails());
- } catch (CloudRuntimeException e) {
- cleanupFailedImportVM(userVm);
- throw e;
- }
- if (!checkVolumeAnswer.getResult()) {
- cleanupFailedImportVM(userVm);
- throw new CloudRuntimeException("Disk not found or is invalid");
+
+ Map<VolumeOnStorageTO.Detail, String> volumeDetails;
+ long volumeSize;
+ List<VolumeOnStorageTO> volumesFromDriver = listVolumesForImportFromDriver(storagePool, diskPath);
+ if (volumesFromDriver != null) {
+ // The storage driver can enumerate its own volumes (e.g. StorPool, via its
+ // management API), so there is no need to reach the host agent just to read size.
+ if (CollectionUtils.isEmpty(volumesFromDriver)) {
+ cleanupFailedImportVM(userVm);
+ throw new CloudRuntimeException("Disk not found or is invalid");
+ }
+ VolumeOnStorageTO volumeOnStorage = volumesFromDriver.get(0);
+ volumeDetails = volumeOnStorage.getDetails();
+ try {
+ checkVolume(volumeDetails);
+ } catch (CloudRuntimeException e) {
+ cleanupFailedImportVM(userVm);
+ throw e;
+ }
+ volumeSize = volumeOnStorage.getVirtualSize();
+ } else {
+ CheckVolumeCommand checkVolumeCommand = new CheckVolumeCommand();
+ checkVolumeCommand.setSrcFile(diskPath);
+ StorageFilerTO storageTO = new StorageFilerTO(storagePool);
+ checkVolumeCommand.setStorageFilerTO(storageTO);
+ Answer answer = agentManager.easySend(dest.getHost().getId(), checkVolumeCommand);
+ if (!(answer instanceof CheckVolumeAnswer)) {
+ cleanupFailedImportVM(userVm);
+ throw new CloudRuntimeException("Disk not found or is invalid");
+ }
+ CheckVolumeAnswer checkVolumeAnswer = (CheckVolumeAnswer) answer;
+ volumeDetails = checkVolumeAnswer.getVolumeDetails();
+ try {
+ checkVolume(volumeDetails);
+ } catch (CloudRuntimeException e) {
+ cleanupFailedImportVM(userVm);
+ throw e;
+ }
+ if (!checkVolumeAnswer.getResult()) {
+ cleanupFailedImportVM(userVm);
+ throw new CloudRuntimeException("Disk not found or is invalid");
+ }
+ volumeSize = checkVolumeAnswer.getSize();
Review Comment:
Can this logic be extracted to a method that retrieves a `Pair` of volume details and volume size?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "slavkap (via GitHub)" <gi...@apache.org>.
slavkap commented on code in PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#discussion_r4247534601
##########
plugins/storage/volume/storpool/src/test/java/org/apache/cloudstack/storage/datastore/driver/StorPoolPrimaryDataStoreDriverTest.java:
##########
@@ -214,6 +236,345 @@ public void testCopyVolumeNotAttachedToVmPassed() {
Assert.assertNull(response.getError());
}
+ @Test
+ public void testListVolumesForImportReturnsGlobalIdVolume() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(volumeJson("~t.t.t", null, false, 1073741824L, 524288000L));
+ when(StorPoolUtil.volumesListResponse(conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, null, null, false);
+
+ Assert.assertEquals(1, volumes.size());
+ VolumeOnStorageTO volume = volumes.get(0);
+ Assert.assertEquals("t.t.t", volume.getName());
+ Assert.assertEquals("/dev/storpool-byid/t.t.t", volume.getPath());
+ Assert.assertEquals("/dev/storpool-byid/t.t.t", volume.getFullPath());
+ Assert.assertEquals(Storage.ImageFormat.QCOW2.toString(), volume.getFormat());
+ Assert.assertEquals(524288000L, volume.getSize());
+ Assert.assertEquals(1073741824L, volume.getVirtualSize());
+ }
+
+ @Test
+ public void testListVolumesForImportSkipsVolumeTaggedCs() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ JsonObject tagged = volumeJsonObject("~t.t.t", 1073741824L, 1073741824L);
+ JsonObject tags = new JsonObject();
+ tags.addProperty("cs", "volume");
+ tagged.add("tags", tags);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(tagged);
+ when(StorPoolUtil.volumesListResponse(conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, null, null, false);
+
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportSkipsDeletedVolume() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(volumeJson("~t.t.t", null, true, 1073741824L, 1073741824L));
+ when(StorPoolUtil.volumesListResponse(conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, null, null, false);
+
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportSkipsNonGlobalIdAndCloneNames() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ JsonArray data = new JsonArray();
+ data.add(volumeJsonObject("friendlyname", 1073741824L, 1073741824L));
+ data.add(volumeJsonObject("*s.s.s", 1073741824L, 1073741824L));
+ data.add(volumeJsonObject("~s.s.s@s.s", 1073741824L, 1073741824L));
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(data);
+ when(StorPoolUtil.volumesListResponse(conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, null, null, false);
+
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportFiltersByKeyword() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ JsonArray data = new JsonArray();
+ data.add(volumeJsonObject("~t.t.t", 1073741824L, 1073741824L));
+ data.add(volumeJsonObject("~s.s.s", 1073741824L, 1073741824L));
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(data);
+ when(StorPoolUtil.volumesListResponse(conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, null, "s.s.s", false);
+
+ Assert.assertEquals(1, volumes.size());
+ Assert.assertEquals("s.s.s", volumes.get(0).getName());
+ }
+
+ @Test
+ public void testListVolumesForImportApiErrorReturnsEmptyList() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = new StorPoolUtil.SpApiResponse();
+ setResponseError(resp);
+ when(StorPoolUtil.volumesListResponse(conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, null, null, false);
+
+ Assert.assertNotNull(volumes);
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathReturnsVolume() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(volumeJson("~t.t.t", null, false, 1073741824L, 524288000L));
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, false);
+
+ Assert.assertEquals(1, volumes.size());
+ VolumeOnStorageTO volume = volumes.get(0);
+ Assert.assertEquals("t.t.t", volume.getName());
+ Assert.assertEquals("/dev/storpool-byid/t.t.t", volume.getPath());
+ Assert.assertEquals("/dev/storpool-byid/t.t.t", volume.getFullPath());
+ Assert.assertEquals(Storage.ImageFormat.QCOW2.toString(), volume.getFormat());
+ Assert.assertEquals(524288000L, volume.getSize());
+ Assert.assertEquals(1073741824L, volume.getVirtualSize());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathExcludesVolumeTaggedCs() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ JsonObject tagged = volumeJsonObject("~t.t.t", 1073741824L, 1073741824L);
+ JsonObject tags = new JsonObject();
+ tags.addProperty("cs", "volume");
+ tagged.add("tags", tags);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(tagged);
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, false);
+
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathForUnmanageReturnsVolumeEvenIfTaggedCs() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ JsonObject tagged = volumeJsonObject("~t.t.t", 1073741824L, 1073741824L);
+ JsonObject tags = new JsonObject();
+ tags.addProperty("cs", "volume");
+ tagged.add("tags", tags);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(tagged);
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, true);
+
+ Assert.assertEquals(1, volumes.size());
+ Assert.assertEquals("t.t.t", volumes.get(0).getName());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathForUnmanageSkipsNonGlobalIdName() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(volumeJsonObject("~t.t.t@s.s", 1073741824L, 1073741824L));
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, true);
+
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathSkipsDeletedVolume() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(volumeJson("~t.t.t", null, true, 1073741824L, 1073741824L));
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, false);
+
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathNotFoundReturnsEmptyList() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(new JsonArray());
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, false);
+
+ Assert.assertNotNull(volumes);
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathApiErrorReturnsEmptyList() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = new StorPoolUtil.SpApiResponse();
+ setResponseError(resp);
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, false);
+
+ Assert.assertNotNull(volumes);
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testCreateAsyncSetsVolumeFormatToQcow2() {
+ DataStore dataStore = mock(DataStore.class);
+ when(dataStore.getUuid()).thenReturn("SP_API_HTTP=1.1.1.1:81;SP_AUTH_TOKEN=token;SP_TEMPLATE=template_name");
+ when(dataStore.getId()).thenReturn(1L);
+
+ VolumeInfo vinfo = mock(VolumeInfo.class);
+ when(vinfo.getType()).thenReturn(DataObjectType.VOLUME);
+ when(vinfo.getUuid()).thenReturn(UUID.randomUUID().toString());
+ when(vinfo.getId()).thenReturn(101L);
+ when(vinfo.getSize()).thenReturn(1073741824L);
+ when(vinfo.getPassphraseId()).thenReturn(null);
+ when(vinfo.getInstanceId()).thenReturn(null);
+ when(vinfo.getDiskOfferingId()).thenReturn(null);
+
+ when(StorPoolUtil.getSpConnection(dataStore.getUuid(), dataStore.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = new StorPoolUtil.SpApiResponse();
+ when(StorPoolUtil.volumeCreate(any(StorPoolVolumeDef.class), eq(conn))).thenReturn(resp);
+ when(StorPoolUtil.getNameFromResponse(resp, false)).thenReturn("t.t.t");
+
+ VolumeVO volumeVO = mock(VolumeVO.class);
+ when(volumeVO.getId()).thenReturn(101L);
+ when(volumeDao.findById(101L)).thenReturn(volumeVO);
+
+ StoragePoolVO storagePoolVO = mock(StoragePoolVO.class);
+ when(storagePool.findById(1L)).thenReturn(storagePoolVO);
+
+ @SuppressWarnings("unchecked")
+ AsyncCompletionCallback<CreateCmdResult> createCallback = mock(AsyncCompletionCallback.class);
+
+ storPoolPrimaryDataStoreDriver.createAsync(dataStore, vinfo, createCallback);
+
+ verify(volumeVO).setFormat(Storage.ImageFormat.QCOW2);
+ verify(volumeDao).update(eq(101L), eq(volumeVO));
+ }
+
+ @Test
+ public void testVolumeImportedTagsVolumeWithCsAndUuid() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ String volumeUuid = UUID.randomUUID().toString();
+ VolumeVO volume = mock(VolumeVO.class);
+ when(volume.getPath()).thenReturn("/dev/storpool-byid/t.t.t");
+ when(volume.getUuid()).thenReturn(volumeUuid);
+ when(volume.getDiskOfferingId()).thenReturn(5L);
+ when(volume.getInstanceId()).thenReturn(null);
+ when(diskOfferingDetailsDao.findDetail(5L, StorPoolUtil.SP_TIER)).thenReturn(null);
+ StorPoolUtil.SpApiResponse resp = new StorPoolUtil.SpApiResponse();
+ when(StorPoolUtil.volumeUpdate(any(StorPoolVolumeDef.class), eq(conn))).thenReturn(resp);
+
+ storPoolPrimaryDataStoreDriver.volumeManagementChanged(importPool, volume, true);
+
+ ArgumentCaptor<StorPoolVolumeDef> captor = ArgumentCaptor.forClass(StorPoolVolumeDef.class);
+ utilities.verify(() -> StorPoolUtil.volumeUpdate(captor.capture(), eq(conn)), times(1));
+ StorPoolVolumeDef updated = captor.getValue();
+ Assert.assertEquals("~t.t.t", updated.getName());
+ Assert.assertEquals(volumeUuid, updated.getTags().get("uuid"));
+ Assert.assertEquals("volume", updated.getTags().get("cs"));
+ Assert.assertNull(updated.getTags().get("qc"));
+ Assert.assertNull(updated.getTags().get("cvm"));
+ }
+
+ @Test
+ public void testVolumeImportedSetsCvmTagWhenAttachedToVm() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ VolumeVO volume = mock(VolumeVO.class);
+ when(volume.getPath()).thenReturn("/dev/storpool-byid/t.t.t");
+ when(volume.getUuid()).thenReturn(UUID.randomUUID().toString());
+ when(volume.getInstanceId()).thenReturn(7L);
+ VMInstanceVO vm = mock(VMInstanceVO.class);
+ String vmUuid = UUID.randomUUID().toString();
+ when(vm.getUuid()).thenReturn(vmUuid);
+ when(vmInstanceDao.findById(7L)).thenReturn(vm);
+ StorPoolUtil.SpApiResponse resp = new StorPoolUtil.SpApiResponse();
+ when(StorPoolUtil.volumeUpdate(any(StorPoolVolumeDef.class), eq(conn))).thenReturn(resp);
+
+ storPoolPrimaryDataStoreDriver.volumeManagementChanged(importPool, volume, true);
+
+ ArgumentCaptor<StorPoolVolumeDef> captor = ArgumentCaptor.forClass(StorPoolVolumeDef.class);
+ utilities.verify(() -> StorPoolUtil.volumeUpdate(captor.capture(), eq(conn)), times(1));
+ Assert.assertEquals(vmUuid, captor.getValue().getTags().get("cvm"));
+ }
+
+ @Test
+ public void testVolumeImportedIncludesQosTierFromDiskOffering() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ VolumeVO volume = mock(VolumeVO.class);
+ when(volume.getPath()).thenReturn("/dev/storpool-byid/t.t.t");
+ when(volume.getUuid()).thenReturn(UUID.randomUUID().toString());
+ when(volume.getDiskOfferingId()).thenReturn(5L);
+ when(volume.getInstanceId()).thenReturn(null);
+ DiskOfferingDetailVO tierDetail = mock(DiskOfferingDetailVO.class);
+ when(tierDetail.getValue()).thenReturn("ssd-tier");
+ when(diskOfferingDetailsDao.findDetail(5L, StorPoolUtil.SP_TIER)).thenReturn(tierDetail);
+ StorPoolUtil.SpApiResponse resp = new StorPoolUtil.SpApiResponse();
+ when(StorPoolUtil.volumeUpdate(any(StorPoolVolumeDef.class), eq(conn))).thenReturn(resp);
+
+ storPoolPrimaryDataStoreDriver.volumeManagementChanged(importPool, volume, true);
+
+ ArgumentCaptor<StorPoolVolumeDef> captor = ArgumentCaptor.forClass(StorPoolVolumeDef.class);
+ utilities.verify(() -> StorPoolUtil.volumeUpdate(captor.capture(), eq(conn)), times(1));
+ Assert.assertEquals("ssd-tier", captor.getValue().getTags().get("qc"));
+ }
+
+ @Test
+ public void testVolumeUnmanagedClearsCsUuidAndQosTags() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ VolumeVO volume = mock(VolumeVO.class);
+ when(volume.getPath()).thenReturn("/dev/storpool-byid/t.t.t");
+ StorPoolUtil.SpApiResponse resp = new StorPoolUtil.SpApiResponse();
+ when(StorPoolUtil.volumeUpdate(any(StorPoolVolumeDef.class), eq(conn))).thenReturn(resp);
+
+ storPoolPrimaryDataStoreDriver.volumeManagementChanged(importPool, volume, false);
+
+ ArgumentCaptor<StorPoolVolumeDef> captor = ArgumentCaptor.forClass(StorPoolVolumeDef.class);
+ utilities.verify(() -> StorPoolUtil.volumeUpdate(captor.capture(), eq(conn)), times(1));
+ StorPoolVolumeDef updated = captor.getValue();
+ Assert.assertEquals("~t.t.t", updated.getName());
+ Assert.assertEquals("", updated.getTags().get("cs"));
Review Comment:
No new unit test needed here — given the clarification on #1, a unit test asserting `cs==""` is treated as "unmanaged" would be testing the wrong thing (it assumes the JSON we send is literally what a later read-back returns, which isn't how StorPool's tag-removal semantics work).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "venko-storpool-com (via GitHub)" <gi...@apache.org>.
venko-storpool-com commented on PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#issuecomment-6295900974
Tested functions in UI for StorPool volumes:
- Import KVM instance / Import disk image from Shared Storage - with the provided path /dev/storpool-byid/<volume_gid> - OK
- Import KVM instance with provided path to a missing volume - OK
- Import KVM instance with provided path to a used volume - OK
- Import Data volumes / List Unmanaged Volumes - lists only volumes not used by CloudStack - OK
- Unmanage volume - OK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "slavkap (via GitHub)" <gi...@apache.org>.
slavkap commented on PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#issuecomment-6313292928
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on code in PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#discussion_r4258631538
##########
plugins/storage/volume/storpool/src/main/java/org/apache/cloudstack/storage/datastore/driver/StorPoolPrimaryDataStoreDriver.java:
##########
@@ -1367,6 +1377,120 @@ public Pair<Long, Long> getVolumeStats(StoragePool storagePool, String volumeId)
return null;
}
+ /**
+ * Lists StorPool volumes available for import (globalId name, no "cs" tag unless
+ * {@code forUnmanage}). When {@code volumePath} is given, looks up that one volume directly
+ * instead of browsing the whole pool; when {@code forUnmanage} is also true, an already
+ * "cs"-tagged volume is still returned, since the caller is unmanageVolume() looking for the
+ * very volume it is about to drop from CloudStack.
+ */
+ @Override
+ public List<VolumeOnStorageTO> listVolumesForImport(StoragePool pool, String volumePath, String keyword, boolean forUnmanage) {
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.listVolumesForImport: pool=%s, volumePath=%s, keyword=%s, forUnmanage=%s", pool.getName(), volumePath, keyword, forUnmanage);
+ SpConnectionDesc conn = StorPoolUtil.getSpConnection(pool.getUuid(), pool.getId(), storagePoolDetailsDao, primaryStoreDao);
+ List<VolumeOnStorageTO> volumes = new ArrayList<>();
+
+ if (StringUtils.isNotBlank(volumePath)) {
+ return getVolumeOnStorageByPath(pool, volumePath, conn, volumes, forUnmanage);
+ }
+
+ SpApiResponse resp = StorPoolUtil.volumesListResponse(conn);
Review Comment:
should we only list volumes on this pool's storpool template? right now the whole cluster comes back, so with two pools on one cluster the same disk shows under both
##########
plugins/storage/volume/storpool/src/main/java/org/apache/cloudstack/storage/datastore/driver/StorPoolPrimaryDataStoreDriver.java:
##########
@@ -1367,6 +1377,120 @@ public Pair<Long, Long> getVolumeStats(StoragePool storagePool, String volumeId)
return null;
}
+ /**
+ * Lists StorPool volumes available for import (globalId name, no "cs" tag unless
+ * {@code forUnmanage}). When {@code volumePath} is given, looks up that one volume directly
+ * instead of browsing the whole pool; when {@code forUnmanage} is also true, an already
+ * "cs"-tagged volume is still returned, since the caller is unmanageVolume() looking for the
+ * very volume it is about to drop from CloudStack.
+ */
+ @Override
+ public List<VolumeOnStorageTO> listVolumesForImport(StoragePool pool, String volumePath, String keyword, boolean forUnmanage) {
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.listVolumesForImport: pool=%s, volumePath=%s, keyword=%s, forUnmanage=%s", pool.getName(), volumePath, keyword, forUnmanage);
+ SpConnectionDesc conn = StorPoolUtil.getSpConnection(pool.getUuid(), pool.getId(), storagePoolDetailsDao, primaryStoreDao);
+ List<VolumeOnStorageTO> volumes = new ArrayList<>();
+
+ if (StringUtils.isNotBlank(volumePath)) {
+ return getVolumeOnStorageByPath(pool, volumePath, conn, volumes, forUnmanage);
+ }
+
+ SpApiResponse resp = StorPoolUtil.volumesListResponse(conn);
+ if (resp.getError() != null) {
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.listVolumesForImport: could not list volumes for pool=%s. Error: %s", pool.getName(), resp.getError());
+ return volumes;
+ }
+ return getVolumeOnStorage(pool, keyword, resp, volumes);
+ }
+
+ @NotNull
+ private List<VolumeOnStorageTO> getVolumeOnStorage(StoragePool pool, String keyword, SpApiResponse resp, List<VolumeOnStorageTO> volumes) {
+ JsonArray volumesList = resp.fullJson.getAsJsonObject().getAsJsonArray("data");
+
+ for (int i = 0; i < volumesList.size(); i++) {
+ JsonObject volume = volumesList.get(i).getAsJsonObject();
+ String rawName = volume.get("name").getAsString();
+ if (isVolumeDeleted(volume) || !isEligibleForBulkImport(volume, rawName)) {
+ continue;
+ }
+
+ String name = rawName.substring(1);
+
+ if (StringUtils.isNotBlank(keyword) && !name.contains(keyword)) {
+ continue;
+ }
+ volumes.add(toVolumeOnStorageTO(volume, name));
+ }
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.listVolumesForImport: found %s volume(s) for pool=%s", volumes.size(), pool.getName());
+ return volumes;
+ }
+
+ @NotNull
+ private List<VolumeOnStorageTO> getVolumeOnStorageByPath(StoragePool pool, String volumePath, SpConnectionDesc conn, List<VolumeOnStorageTO> volumes, boolean forUnmanage) {
+ String name = StorPoolStorageAdaptor.getVolumeNameFromPath(volumePath, true);
+ SpApiResponse resp = StorPoolUtil.volumeDescribeResponse(name, conn);
+ if (resp.getError() != null) {
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.listVolumesForImport: could not describe volume=%s on pool=%s. Error: %s", name, pool.getName(), resp.getError());
+ return volumes;
+ }
+ JsonArray data = resp.fullJson.getAsJsonObject().getAsJsonArray("data");
+ if (data == null || data.isEmpty()) {
+ return volumes;
+ }
+ JsonObject volume = data.get(0).getAsJsonObject();
+ String rawName = volume.get("name").getAsString();
+ if (isVolumeDeleted(volume) || !isGlobalIdVolumeName(rawName) || (!forUnmanage && isTaggedCs(volume))) {
+ return volumes;
+ }
+ volumes.add(toVolumeOnStorageTO(volume, rawName.substring(1)));
+ return volumes;
+ }
+
+ private static boolean isVolumeDeleted(JsonObject volume) {
+ return volume.has("deleted") && volume.get("deleted").getAsBoolean();
+ }
+
+ private static boolean isGlobalIdVolumeName(String rawName) {
+ return !rawName.startsWith("*") && !rawName.contains("@") && rawName.startsWith("~");
+ }
+
+ private static boolean isTaggedCs(JsonObject volume) {
+ JsonObject tags = volume.has("tags") && !volume.get("tags").isJsonNull() ? volume.getAsJsonObject("tags") : null;
+ return tags != null && tags.has("cs");
+ }
+
+ private static boolean isEligibleForBulkImport(JsonObject volume, String rawName) {
+ return isGlobalIdVolumeName(rawName) && !isTaggedCs(volume);
+ }
+
+ private VolumeOnStorageTO toVolumeOnStorageTO(JsonObject volume, String name) {
+ String path = StorPoolUtil.devPath(name);
+ long size = volume.get("size").getAsLong();
+ long spaceUsed = volume.has("spaceUsed") ? volume.get("spaceUsed").getAsLong() : size;
+ StorPoolUtil.spLog("listVolumesForImport: found volume=%s, path=%s, size=%s, spaceUsed=%s", name, path, size, spaceUsed);
+ return new VolumeOnStorageTO(Hypervisor.HypervisorType.KVM, path, name, path,
+ Storage.ImageFormat.QCOW2.toString(), spaceUsed, size);
+ }
+
+ @Override
+ public void volumeManagementChanged(StoragePool pool, Volume volume, boolean imported) {
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.volumeManagementChanged: pool=%s, volume=%s, imported=%s", pool.getName(), volume.getPath(), imported);
+ SpConnectionDesc conn = StorPoolUtil.getSpConnection(pool.getUuid(), pool.getId(), storagePoolDetailsDao, primaryStoreDao);
+ String name = StorPoolStorageAdaptor.getVolumeNameFromPath(volume.getPath(), true);
+ Map<String, String> tags;
+ if (imported) {
+ String tier = volume.getDiskOfferingId() != null ? getTierFromOfferingDetail(volume.getDiskOfferingId()) : null;
+ Long instanceId = volume.getInstanceId();
+ tags = StorPoolHelper.addStorPoolTags(volume.getUuid(), getVMInstanceUUID(instanceId), "volume", getVcPolicyTag(instanceId), tier);
+ } else {
+ tags = StorPoolHelper.addStorPoolTags(null, null, "", null, null);
Review Comment:
should unmanage also clear the vm and snapshot policy tags? only the cs tag is removed, so the disk keeps the old vm link and snapshot policy
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/wrapper/LibvirtCheckVolumeCommandWrapper.java:
##########
@@ -63,27 +71,37 @@ public Answer execute(final CheckVolumeCommand command, final LibvirtComputingRe
try {
if (STORAGE_POOL_TYPES_SUPPORTED.contains(storageFilerTO.getType())) {
- final KVMPhysicalDisk vol = pool.getPhysicalDisk(srcFile);
- final String path = vol.getPath();
- final boolean isRbd = Storage.StoragePoolType.RBD.equals(storageFilerTO.getType());
-
- Map<VolumeOnStorageTO.Detail, String> volumeDetails = getVolumeDetails(pool, vol);
- if (MapUtils.isEmpty(volumeDetails)) {
- return new Answer(command, false, "Unable to read the volume on the storage pool");
+ final boolean shouldAttachTheVolume = POOL_NEEDS_TO_EXPOSE_THE_VOLUME.contains(storageFilerTO.getType());
+ if (shouldAttachTheVolume && !pool.connectPhysicalDisk(srcFile, new HashMap<>())) {
+ return new Answer(command, false, "Unable to attach the volume on the storage pool");
}
+ try {
+ final KVMPhysicalDisk vol = pool.getPhysicalDisk(srcFile);
+ final String path = vol.getPath();
+ final boolean isRawBlockPool = RAW_BLOCK_STORAGE_POOL_TYPES.contains(storageFilerTO.getType());
+
+ Map<VolumeOnStorageTO.Detail, String> volumeDetails = getVolumeDetails(pool, vol);
+ if (MapUtils.isEmpty(volumeDetails)) {
+ return new Answer(command, false, "Unable to read the volume on the storage pool");
+ }
- if (!isRbd) {
- try {
- KVMPhysicalDisk.checkQcow2File(path);
- } catch (final CloudRuntimeException e) {
- return new CheckVolumeAnswer(command, false, "", 0, volumeDetails);
+ if (!isRawBlockPool) {
+ try {
+ KVMPhysicalDisk.checkQcow2File(path);
+ } catch (final CloudRuntimeException e) {
+ return new CheckVolumeAnswer(command, false, "", 0, volumeDetails);
+ }
}
- }
- // Images on RBD are raw and the path is an image name that qemu-img cannot open
- // without the rbd: URI, so take the size libvirt already reported for the volume.
- long size = isRbd ? vol.getVirtualSize() : KVMPhysicalDisk.getVirtualSizeFromFile(path);
- return new CheckVolumeAnswer(command, true, "", size, volumeDetails);
+ // Images on RBD are raw and the path is an image name that qemu-img cannot open
+ // without the rbd: URI, so take the size libvirt already reported for the volume
+ long size = isRawBlockPool ? vol.getVirtualSize() : KVMPhysicalDisk.getVirtualSizeFromFile(path);
+ return new CheckVolumeAnswer(command, true, "", size, volumeDetails);
+ } finally {
+ if (shouldAttachTheVolume) {
+ pool.disconnectPhysicalDisk(srcFile);
Review Comment:
what happens if the disk is already attached on this host before the check? looks like the detach at the end would pull it from there, or fail the check if its in use
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#issuecomment-6291738795
@nvazquez a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#issuecomment-6315045395
@nvazquez a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#discussion_r4247217048
##########
plugins/storage/volume/storpool/src/main/java/org/apache/cloudstack/storage/datastore/driver/StorPoolPrimaryDataStoreDriver.java:
##########
@@ -1367,6 +1377,120 @@ public Pair<Long, Long> getVolumeStats(StoragePool storagePool, String volumeId)
return null;
}
+ /**
+ * Lists StorPool volumes available for import (globalId name, no "cs" tag unless
+ * {@code forUnmanage}). When {@code volumePath} is given, looks up that one volume directly
+ * instead of browsing the whole pool; when {@code forUnmanage} is also true, an already
+ * "cs"-tagged volume is still returned, since the caller is unmanageVolume() looking for the
+ * very volume it is about to drop from CloudStack.
+ */
+ @Override
+ public List<VolumeOnStorageTO> listVolumesForImport(StoragePool pool, String volumePath, String keyword, boolean forUnmanage) {
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.listVolumesForImport: pool=%s, volumePath=%s, keyword=%s, forUnmanage=%s", pool.getName(), volumePath, keyword, forUnmanage);
+ SpConnectionDesc conn = StorPoolUtil.getSpConnection(pool.getUuid(), pool.getId(), storagePoolDetailsDao, primaryStoreDao);
+ List<VolumeOnStorageTO> volumes = new ArrayList<>();
+
+ if (StringUtils.isNotBlank(volumePath)) {
+ return getVolumeOnStorageByPath(pool, volumePath, conn, volumes, forUnmanage);
+ }
+
+ SpApiResponse resp = StorPoolUtil.volumesListResponse(conn);
+ if (resp.getError() != null) {
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.listVolumesForImport: could not list volumes for pool=%s. Error: %s", pool.getName(), resp.getError());
+ return volumes;
+ }
+ return getVolumeOnStorage(pool, keyword, resp, volumes);
+ }
+
+ @NotNull
+ private List<VolumeOnStorageTO> getVolumeOnStorage(StoragePool pool, String keyword, SpApiResponse resp, List<VolumeOnStorageTO> volumes) {
+ JsonArray volumesList = resp.fullJson.getAsJsonObject().getAsJsonArray("data");
+
+ for (int i = 0; i < volumesList.size(); i++) {
+ JsonObject volume = volumesList.get(i).getAsJsonObject();
+ String rawName = volume.get("name").getAsString();
+ if (isVolumeDeleted(volume) || !isEligibleForBulkImport(volume, rawName)) {
+ continue;
+ }
+
+ String name = rawName.split("~")[1];
+
+ if (StringUtils.isNotBlank(keyword) && !name.contains(keyword)) {
+ continue;
+ }
+ volumes.add(toVolumeOnStorageTO(volume, name));
+ }
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.listVolumesForImport: found %s volume(s) for pool=%s", volumes.size(), pool.getName());
+ return volumes;
+ }
+
+ @NotNull
+ private List<VolumeOnStorageTO> getVolumeOnStorageByPath(StoragePool pool, String volumePath, SpConnectionDesc conn, List<VolumeOnStorageTO> volumes, boolean forUnmanage) {
+ String name = StorPoolStorageAdaptor.getVolumeNameFromPath(volumePath, true);
+ SpApiResponse resp = StorPoolUtil.volumeDescribeResponse(name, conn);
+ if (resp.getError() != null) {
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.listVolumesForImport: could not describe volume=%s on pool=%s. Error: %s", name, pool.getName(), resp.getError());
+ return volumes;
+ }
+ JsonArray data = resp.fullJson.getAsJsonObject().getAsJsonArray("data");
+ if (data == null || data.isEmpty()) {
+ return volumes;
+ }
+ JsonObject volume = data.get(0).getAsJsonObject();
+ String rawName = volume.get("name").getAsString();
+ if (isVolumeDeleted(volume) || !isGlobalIdVolumeName(rawName) || (!forUnmanage && isTaggedCs(volume))) {
+ return volumes;
+ }
+ volumes.add(toVolumeOnStorageTO(volume, rawName.split("~")[1]));
+ return volumes;
+ }
+
+ private static boolean isVolumeDeleted(JsonObject volume) {
+ return volume.has("deleted") && volume.get("deleted").getAsBoolean();
+ }
+
+ private static boolean isGlobalIdVolumeName(String rawName) {
+ return !rawName.startsWith("*") && !rawName.contains("@") && rawName.startsWith("~");
+ }
+
+ private static boolean isTaggedCs(JsonObject volume) {
+ JsonObject tags = volume.has("tags") && !volume.get("tags").isJsonNull() ? volume.getAsJsonObject("tags") : null;
+ return tags != null && tags.has("cs");
+ }
+
+ private static boolean isEligibleForBulkImport(JsonObject volume, String rawName) {
+ return isGlobalIdVolumeName(rawName) && !isTaggedCs(volume);
+ }
Review Comment:
`isTaggedCs` treats any presence of the `cs` key as 'managed'. However this PR’s unmanage path sets `cs` to an empty string (see `volumeManagementChanged(..., imported=false)`), which would still be considered managed here and would prevent unmanaged volumes from reappearing in `listVolumesForImport`. Update the check to treat blank/empty `cs` values as *not managed* (e.g., require a non-blank value, or explicitly require `cs == \"volume\"`).
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/wrapper/LibvirtCheckVolumeCommandWrapper.java:
##########
@@ -51,7 +51,15 @@ public final class LibvirtCheckVolumeCommandWrapper extends CommandWrapper<Check
Storage.StoragePoolType.Filesystem,
Storage.StoragePoolType.NetworkFilesystem,
Storage.StoragePoolType.SharedMountPoint,
- Storage.StoragePoolType.RBD);
+ Storage.StoragePoolType.RBD,
+ Storage.StoragePoolType.StorPool);
+
+ private static final List<Storage.StoragePoolType> RAW_BLOCK_STORAGE_POOL_TYPES = Arrays.asList(
+ Storage.StoragePoolType.RBD,
+ Storage.StoragePoolType.StorPool);
Review Comment:
StorPool volumes are treated as `RAW_BLOCK_STORAGE_POOL_TYPES`, which skips QCOW2 validation and uses `vol.getVirtualSize()` instead of reading the virtual size from the volume path. This is inconsistent with the rest of the PR which treats StorPool imports as QCOW2 (and the volume path appears usable, e.g., `/dev/storpool-byid/...`). If StorPool volumes are QCOW2, this logic can cause incorrect backing-file detection/format checks and potentially incorrect size reporting. Consider removing `StorPool` from `RAW_BLOCK_STORAGE_POOL_TYPES` and only applying the RBD-specific behavior to RBD.
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/wrapper/LibvirtCheckVolumeCommandWrapper.java:
##########
@@ -63,27 +71,37 @@ public Answer execute(final CheckVolumeCommand command, final LibvirtComputingRe
try {
if (STORAGE_POOL_TYPES_SUPPORTED.contains(storageFilerTO.getType())) {
- final KVMPhysicalDisk vol = pool.getPhysicalDisk(srcFile);
- final String path = vol.getPath();
- final boolean isRbd = Storage.StoragePoolType.RBD.equals(storageFilerTO.getType());
-
- Map<VolumeOnStorageTO.Detail, String> volumeDetails = getVolumeDetails(pool, vol);
- if (MapUtils.isEmpty(volumeDetails)) {
- return new Answer(command, false, "Unable to read the volume on the storage pool");
+ final boolean shouldAttachTheVolume = POOL_NEEDS_TO_EXPOSE_THE_VOLUME.contains(storageFilerTO.getType());
+ if (shouldAttachTheVolume && !pool.connectPhysicalDisk(srcFile, new HashMap<>())) {
+ return new Answer(command, false, "Unable to attach the volume on the storage pool");
}
+ try {
+ final KVMPhysicalDisk vol = pool.getPhysicalDisk(srcFile);
+ final String path = vol.getPath();
+ final boolean isRawBlockPool = RAW_BLOCK_STORAGE_POOL_TYPES.contains(storageFilerTO.getType());
+
+ Map<VolumeOnStorageTO.Detail, String> volumeDetails = getVolumeDetails(pool, vol);
+ if (MapUtils.isEmpty(volumeDetails)) {
+ return new Answer(command, false, "Unable to read the volume on the storage pool");
+ }
- if (!isRbd) {
- try {
- KVMPhysicalDisk.checkQcow2File(path);
- } catch (final CloudRuntimeException e) {
- return new CheckVolumeAnswer(command, false, "", 0, volumeDetails);
+ if (!isRawBlockPool) {
+ try {
+ KVMPhysicalDisk.checkQcow2File(path);
+ } catch (final CloudRuntimeException e) {
+ return new CheckVolumeAnswer(command, false, "", 0, volumeDetails);
+ }
}
- }
- // Images on RBD are raw and the path is an image name that qemu-img cannot open
- // without the rbd: URI, so take the size libvirt already reported for the volume.
- long size = isRbd ? vol.getVirtualSize() : KVMPhysicalDisk.getVirtualSizeFromFile(path);
- return new CheckVolumeAnswer(command, true, "", size, volumeDetails);
+ // Images on RBD are raw and the path is an image name that qemu-img cannot open
+ // without the rbd: URI, so take the size libvirt already reported for the volume
+ long size = isRawBlockPool ? vol.getVirtualSize() : KVMPhysicalDisk.getVirtualSizeFromFile(path);
Review Comment:
StorPool volumes are treated as `RAW_BLOCK_STORAGE_POOL_TYPES`, which skips QCOW2 validation and uses `vol.getVirtualSize()` instead of reading the virtual size from the volume path. This is inconsistent with the rest of the PR which treats StorPool imports as QCOW2 (and the volume path appears usable, e.g., `/dev/storpool-byid/...`). If StorPool volumes are QCOW2, this logic can cause incorrect backing-file detection/format checks and potentially incorrect size reporting. Consider removing `StorPool` from `RAW_BLOCK_STORAGE_POOL_TYPES` and only applying the RBD-specific behavior to RBD.
##########
plugins/storage/volume/storpool/src/main/java/org/apache/cloudstack/storage/datastore/driver/StorPoolPrimaryDataStoreDriver.java:
##########
@@ -1367,6 +1377,120 @@ public Pair<Long, Long> getVolumeStats(StoragePool storagePool, String volumeId)
return null;
}
+ /**
+ * Lists StorPool volumes available for import (globalId name, no "cs" tag unless
+ * {@code forUnmanage}). When {@code volumePath} is given, looks up that one volume directly
+ * instead of browsing the whole pool; when {@code forUnmanage} is also true, an already
+ * "cs"-tagged volume is still returned, since the caller is unmanageVolume() looking for the
+ * very volume it is about to drop from CloudStack.
+ */
+ @Override
+ public List<VolumeOnStorageTO> listVolumesForImport(StoragePool pool, String volumePath, String keyword, boolean forUnmanage) {
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.listVolumesForImport: pool=%s, volumePath=%s, keyword=%s, forUnmanage=%s", pool.getName(), volumePath, keyword, forUnmanage);
+ SpConnectionDesc conn = StorPoolUtil.getSpConnection(pool.getUuid(), pool.getId(), storagePoolDetailsDao, primaryStoreDao);
+ List<VolumeOnStorageTO> volumes = new ArrayList<>();
+
+ if (StringUtils.isNotBlank(volumePath)) {
+ return getVolumeOnStorageByPath(pool, volumePath, conn, volumes, forUnmanage);
+ }
+
+ SpApiResponse resp = StorPoolUtil.volumesListResponse(conn);
+ if (resp.getError() != null) {
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.listVolumesForImport: could not list volumes for pool=%s. Error: %s", pool.getName(), resp.getError());
+ return volumes;
+ }
+ return getVolumeOnStorage(pool, keyword, resp, volumes);
+ }
+
+ @NotNull
+ private List<VolumeOnStorageTO> getVolumeOnStorage(StoragePool pool, String keyword, SpApiResponse resp, List<VolumeOnStorageTO> volumes) {
+ JsonArray volumesList = resp.fullJson.getAsJsonObject().getAsJsonArray("data");
+
+ for (int i = 0; i < volumesList.size(); i++) {
+ JsonObject volume = volumesList.get(i).getAsJsonObject();
+ String rawName = volume.get("name").getAsString();
+ if (isVolumeDeleted(volume) || !isEligibleForBulkImport(volume, rawName)) {
+ continue;
+ }
+
+ String name = rawName.split("~")[1];
Review Comment:
This uses `split(\"~\")` just to drop the first character. Since `split` allocates an array and uses regex semantics, it’s heavier and less explicit than needed. Prefer a simple substring (e.g., `rawName.substring(1)`) after the `isGlobalIdVolumeName` check to make the assumption clear and avoid unnecessary allocations.
##########
server/src/main/java/org/apache/cloudstack/vm/UnmanagedVMsManagerImpl.java:
##########
@@ -828,9 +833,16 @@ private Pair<DiskProfile, StoragePool> importKVMSharedDisk(VirtualMachine vm, Di
DiskProfile profile = volumeManager.updateImportedVolume(type, diskOffering, vm, template, deviceId,
poolId, storagePool.getPoolType(), diskPath, null, diskProfile, format);
+ notifyStorageDriverOfVolumeImport(storagePool, profile.getVolumeId());
+
return new Pair<>(profile, storagePool);
}
+ private void notifyStorageDriverOfVolumeImport(StoragePool pool, long volumeId) {
+ Volume volume = volumeDao.findById(volumeId);
Review Comment:
`volumeDao.findById(volumeId)` can return null (e.g., if the DB write failed/was rolled back or the ID is invalid in an unexpected path), which would lead to a `NullPointerException` when notifying the service. Add a null-guard (and ideally log at least a warning) before calling `notifyStorageDriverOfVolumeManagementChange`.
##########
plugins/storage/volume/storpool/src/test/java/org/apache/cloudstack/storage/datastore/driver/StorPoolPrimaryDataStoreDriverTest.java:
##########
@@ -214,6 +236,345 @@ public void testCopyVolumeNotAttachedToVmPassed() {
Assert.assertNull(response.getError());
}
+ @Test
+ public void testListVolumesForImportReturnsGlobalIdVolume() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(volumeJson("~t.t.t", null, false, 1073741824L, 524288000L));
+ when(StorPoolUtil.volumesListResponse(conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, null, null, false);
+
+ Assert.assertEquals(1, volumes.size());
+ VolumeOnStorageTO volume = volumes.get(0);
+ Assert.assertEquals("t.t.t", volume.getName());
+ Assert.assertEquals("/dev/storpool-byid/t.t.t", volume.getPath());
+ Assert.assertEquals("/dev/storpool-byid/t.t.t", volume.getFullPath());
+ Assert.assertEquals(Storage.ImageFormat.QCOW2.toString(), volume.getFormat());
+ Assert.assertEquals(524288000L, volume.getSize());
+ Assert.assertEquals(1073741824L, volume.getVirtualSize());
+ }
+
+ @Test
+ public void testListVolumesForImportSkipsVolumeTaggedCs() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ JsonObject tagged = volumeJsonObject("~t.t.t", 1073741824L, 1073741824L);
+ JsonObject tags = new JsonObject();
+ tags.addProperty("cs", "volume");
+ tagged.add("tags", tags);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(tagged);
+ when(StorPoolUtil.volumesListResponse(conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, null, null, false);
+
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportSkipsDeletedVolume() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(volumeJson("~t.t.t", null, true, 1073741824L, 1073741824L));
+ when(StorPoolUtil.volumesListResponse(conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, null, null, false);
+
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportSkipsNonGlobalIdAndCloneNames() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ JsonArray data = new JsonArray();
+ data.add(volumeJsonObject("friendlyname", 1073741824L, 1073741824L));
+ data.add(volumeJsonObject("*s.s.s", 1073741824L, 1073741824L));
+ data.add(volumeJsonObject("~s.s.s@s.s", 1073741824L, 1073741824L));
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(data);
+ when(StorPoolUtil.volumesListResponse(conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, null, null, false);
+
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportFiltersByKeyword() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ JsonArray data = new JsonArray();
+ data.add(volumeJsonObject("~t.t.t", 1073741824L, 1073741824L));
+ data.add(volumeJsonObject("~s.s.s", 1073741824L, 1073741824L));
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(data);
+ when(StorPoolUtil.volumesListResponse(conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, null, "s.s.s", false);
+
+ Assert.assertEquals(1, volumes.size());
+ Assert.assertEquals("s.s.s", volumes.get(0).getName());
+ }
+
+ @Test
+ public void testListVolumesForImportApiErrorReturnsEmptyList() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = new StorPoolUtil.SpApiResponse();
+ setResponseError(resp);
+ when(StorPoolUtil.volumesListResponse(conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, null, null, false);
+
+ Assert.assertNotNull(volumes);
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathReturnsVolume() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(volumeJson("~t.t.t", null, false, 1073741824L, 524288000L));
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, false);
+
+ Assert.assertEquals(1, volumes.size());
+ VolumeOnStorageTO volume = volumes.get(0);
+ Assert.assertEquals("t.t.t", volume.getName());
+ Assert.assertEquals("/dev/storpool-byid/t.t.t", volume.getPath());
+ Assert.assertEquals("/dev/storpool-byid/t.t.t", volume.getFullPath());
+ Assert.assertEquals(Storage.ImageFormat.QCOW2.toString(), volume.getFormat());
+ Assert.assertEquals(524288000L, volume.getSize());
+ Assert.assertEquals(1073741824L, volume.getVirtualSize());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathExcludesVolumeTaggedCs() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ JsonObject tagged = volumeJsonObject("~t.t.t", 1073741824L, 1073741824L);
+ JsonObject tags = new JsonObject();
+ tags.addProperty("cs", "volume");
+ tagged.add("tags", tags);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(tagged);
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, false);
+
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathForUnmanageReturnsVolumeEvenIfTaggedCs() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ JsonObject tagged = volumeJsonObject("~t.t.t", 1073741824L, 1073741824L);
+ JsonObject tags = new JsonObject();
+ tags.addProperty("cs", "volume");
+ tagged.add("tags", tags);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(tagged);
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, true);
+
+ Assert.assertEquals(1, volumes.size());
+ Assert.assertEquals("t.t.t", volumes.get(0).getName());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathForUnmanageSkipsNonGlobalIdName() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(volumeJsonObject("~t.t.t@s.s", 1073741824L, 1073741824L));
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, true);
+
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathSkipsDeletedVolume() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(volumeJson("~t.t.t", null, true, 1073741824L, 1073741824L));
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, false);
+
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathNotFoundReturnsEmptyList() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(new JsonArray());
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, false);
+
+ Assert.assertNotNull(volumes);
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathApiErrorReturnsEmptyList() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = new StorPoolUtil.SpApiResponse();
+ setResponseError(resp);
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, false);
+
+ Assert.assertNotNull(volumes);
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testCreateAsyncSetsVolumeFormatToQcow2() {
+ DataStore dataStore = mock(DataStore.class);
+ when(dataStore.getUuid()).thenReturn("SP_API_HTTP=1.1.1.1:81;SP_AUTH_TOKEN=token;SP_TEMPLATE=template_name");
+ when(dataStore.getId()).thenReturn(1L);
+
+ VolumeInfo vinfo = mock(VolumeInfo.class);
+ when(vinfo.getType()).thenReturn(DataObjectType.VOLUME);
+ when(vinfo.getUuid()).thenReturn(UUID.randomUUID().toString());
+ when(vinfo.getId()).thenReturn(101L);
+ when(vinfo.getSize()).thenReturn(1073741824L);
+ when(vinfo.getPassphraseId()).thenReturn(null);
+ when(vinfo.getInstanceId()).thenReturn(null);
+ when(vinfo.getDiskOfferingId()).thenReturn(null);
+
+ when(StorPoolUtil.getSpConnection(dataStore.getUuid(), dataStore.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = new StorPoolUtil.SpApiResponse();
+ when(StorPoolUtil.volumeCreate(any(StorPoolVolumeDef.class), eq(conn))).thenReturn(resp);
+ when(StorPoolUtil.getNameFromResponse(resp, false)).thenReturn("t.t.t");
+
+ VolumeVO volumeVO = mock(VolumeVO.class);
+ when(volumeVO.getId()).thenReturn(101L);
+ when(volumeDao.findById(101L)).thenReturn(volumeVO);
+
+ StoragePoolVO storagePoolVO = mock(StoragePoolVO.class);
+ when(storagePool.findById(1L)).thenReturn(storagePoolVO);
+
+ @SuppressWarnings("unchecked")
+ AsyncCompletionCallback<CreateCmdResult> createCallback = mock(AsyncCompletionCallback.class);
+
+ storPoolPrimaryDataStoreDriver.createAsync(dataStore, vinfo, createCallback);
+
+ verify(volumeVO).setFormat(Storage.ImageFormat.QCOW2);
+ verify(volumeDao).update(eq(101L), eq(volumeVO));
+ }
+
+ @Test
+ public void testVolumeImportedTagsVolumeWithCsAndUuid() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ String volumeUuid = UUID.randomUUID().toString();
+ VolumeVO volume = mock(VolumeVO.class);
+ when(volume.getPath()).thenReturn("/dev/storpool-byid/t.t.t");
+ when(volume.getUuid()).thenReturn(volumeUuid);
+ when(volume.getDiskOfferingId()).thenReturn(5L);
+ when(volume.getInstanceId()).thenReturn(null);
+ when(diskOfferingDetailsDao.findDetail(5L, StorPoolUtil.SP_TIER)).thenReturn(null);
+ StorPoolUtil.SpApiResponse resp = new StorPoolUtil.SpApiResponse();
+ when(StorPoolUtil.volumeUpdate(any(StorPoolVolumeDef.class), eq(conn))).thenReturn(resp);
+
+ storPoolPrimaryDataStoreDriver.volumeManagementChanged(importPool, volume, true);
+
+ ArgumentCaptor<StorPoolVolumeDef> captor = ArgumentCaptor.forClass(StorPoolVolumeDef.class);
+ utilities.verify(() -> StorPoolUtil.volumeUpdate(captor.capture(), eq(conn)), times(1));
+ StorPoolVolumeDef updated = captor.getValue();
+ Assert.assertEquals("~t.t.t", updated.getName());
+ Assert.assertEquals(volumeUuid, updated.getTags().get("uuid"));
+ Assert.assertEquals("volume", updated.getTags().get("cs"));
+ Assert.assertNull(updated.getTags().get("qc"));
+ Assert.assertNull(updated.getTags().get("cvm"));
+ }
+
+ @Test
+ public void testVolumeImportedSetsCvmTagWhenAttachedToVm() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ VolumeVO volume = mock(VolumeVO.class);
+ when(volume.getPath()).thenReturn("/dev/storpool-byid/t.t.t");
+ when(volume.getUuid()).thenReturn(UUID.randomUUID().toString());
+ when(volume.getInstanceId()).thenReturn(7L);
+ VMInstanceVO vm = mock(VMInstanceVO.class);
+ String vmUuid = UUID.randomUUID().toString();
+ when(vm.getUuid()).thenReturn(vmUuid);
+ when(vmInstanceDao.findById(7L)).thenReturn(vm);
+ StorPoolUtil.SpApiResponse resp = new StorPoolUtil.SpApiResponse();
+ when(StorPoolUtil.volumeUpdate(any(StorPoolVolumeDef.class), eq(conn))).thenReturn(resp);
+
+ storPoolPrimaryDataStoreDriver.volumeManagementChanged(importPool, volume, true);
+
+ ArgumentCaptor<StorPoolVolumeDef> captor = ArgumentCaptor.forClass(StorPoolVolumeDef.class);
+ utilities.verify(() -> StorPoolUtil.volumeUpdate(captor.capture(), eq(conn)), times(1));
+ Assert.assertEquals(vmUuid, captor.getValue().getTags().get("cvm"));
+ }
+
+ @Test
+ public void testVolumeImportedIncludesQosTierFromDiskOffering() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ VolumeVO volume = mock(VolumeVO.class);
+ when(volume.getPath()).thenReturn("/dev/storpool-byid/t.t.t");
+ when(volume.getUuid()).thenReturn(UUID.randomUUID().toString());
+ when(volume.getDiskOfferingId()).thenReturn(5L);
+ when(volume.getInstanceId()).thenReturn(null);
+ DiskOfferingDetailVO tierDetail = mock(DiskOfferingDetailVO.class);
+ when(tierDetail.getValue()).thenReturn("ssd-tier");
+ when(diskOfferingDetailsDao.findDetail(5L, StorPoolUtil.SP_TIER)).thenReturn(tierDetail);
+ StorPoolUtil.SpApiResponse resp = new StorPoolUtil.SpApiResponse();
+ when(StorPoolUtil.volumeUpdate(any(StorPoolVolumeDef.class), eq(conn))).thenReturn(resp);
+
+ storPoolPrimaryDataStoreDriver.volumeManagementChanged(importPool, volume, true);
+
+ ArgumentCaptor<StorPoolVolumeDef> captor = ArgumentCaptor.forClass(StorPoolVolumeDef.class);
+ utilities.verify(() -> StorPoolUtil.volumeUpdate(captor.capture(), eq(conn)), times(1));
+ Assert.assertEquals("ssd-tier", captor.getValue().getTags().get("qc"));
+ }
+
+ @Test
+ public void testVolumeUnmanagedClearsCsUuidAndQosTags() {
Review Comment:
Given the unmanage path leaves `cs` present but empty (`cs == \"\"`), add a unit test to ensure `listVolumesForImport(..., forUnmanage=false)` treats an empty `cs` value as *unmanaged* and lists the volume again. This directly guards the behavior around the new `isTaggedCs`/eligibility logic.
##########
server/src/main/java/org/apache/cloudstack/vm/UnmanagedVMsManagerImpl.java:
##########
@@ -2959,6 +2971,10 @@ private UserVm importKvmVirtualMachineFromDisk(final ImportSource importSource,
// check then runs against whichever pool that cluster does have. Both are null for a zone
// wide pool, which every host can see.
StoragePoolVO importStoragePool = primaryDataStoreDao.findById(poolId);
+ if (importStoragePool == null) {
+ cleanupFailedImportVM(userVm);
+ throw new ServerApiException(ApiErrorCode.INTERNAL_ERROR, String.format("Import failed for Vm: %s. Storage pool with id: %s not found", userVm.getInstanceName(), poolId));
Review Comment:
The new error message has inconsistent capitalization (`Vm`) and is slightly awkward/less searchable. Consider changing to `VM` and tightening the wording (e.g., \"Import failed for VM %s: storage pool id %s not found\").
##########
plugins/storage/volume/storpool/src/test/java/org/apache/cloudstack/storage/datastore/driver/StorPoolPrimaryDataStoreDriverTest.java:
##########
@@ -214,6 +236,345 @@ public void testCopyVolumeNotAttachedToVmPassed() {
Assert.assertNull(response.getError());
}
+ @Test
+ public void testListVolumesForImportReturnsGlobalIdVolume() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(volumeJson("~t.t.t", null, false, 1073741824L, 524288000L));
+ when(StorPoolUtil.volumesListResponse(conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, null, null, false);
+
+ Assert.assertEquals(1, volumes.size());
+ VolumeOnStorageTO volume = volumes.get(0);
+ Assert.assertEquals("t.t.t", volume.getName());
+ Assert.assertEquals("/dev/storpool-byid/t.t.t", volume.getPath());
+ Assert.assertEquals("/dev/storpool-byid/t.t.t", volume.getFullPath());
+ Assert.assertEquals(Storage.ImageFormat.QCOW2.toString(), volume.getFormat());
+ Assert.assertEquals(524288000L, volume.getSize());
+ Assert.assertEquals(1073741824L, volume.getVirtualSize());
+ }
+
+ @Test
+ public void testListVolumesForImportSkipsVolumeTaggedCs() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ JsonObject tagged = volumeJsonObject("~t.t.t", 1073741824L, 1073741824L);
+ JsonObject tags = new JsonObject();
+ tags.addProperty("cs", "volume");
+ tagged.add("tags", tags);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(tagged);
+ when(StorPoolUtil.volumesListResponse(conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, null, null, false);
+
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportSkipsDeletedVolume() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(volumeJson("~t.t.t", null, true, 1073741824L, 1073741824L));
+ when(StorPoolUtil.volumesListResponse(conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, null, null, false);
+
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportSkipsNonGlobalIdAndCloneNames() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ JsonArray data = new JsonArray();
+ data.add(volumeJsonObject("friendlyname", 1073741824L, 1073741824L));
+ data.add(volumeJsonObject("*s.s.s", 1073741824L, 1073741824L));
+ data.add(volumeJsonObject("~s.s.s@s.s", 1073741824L, 1073741824L));
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(data);
+ when(StorPoolUtil.volumesListResponse(conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, null, null, false);
+
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportFiltersByKeyword() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ JsonArray data = new JsonArray();
+ data.add(volumeJsonObject("~t.t.t", 1073741824L, 1073741824L));
+ data.add(volumeJsonObject("~s.s.s", 1073741824L, 1073741824L));
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(data);
+ when(StorPoolUtil.volumesListResponse(conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, null, "s.s.s", false);
+
+ Assert.assertEquals(1, volumes.size());
+ Assert.assertEquals("s.s.s", volumes.get(0).getName());
+ }
+
+ @Test
+ public void testListVolumesForImportApiErrorReturnsEmptyList() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = new StorPoolUtil.SpApiResponse();
+ setResponseError(resp);
+ when(StorPoolUtil.volumesListResponse(conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, null, null, false);
+
+ Assert.assertNotNull(volumes);
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathReturnsVolume() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(volumeJson("~t.t.t", null, false, 1073741824L, 524288000L));
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, false);
+
+ Assert.assertEquals(1, volumes.size());
+ VolumeOnStorageTO volume = volumes.get(0);
+ Assert.assertEquals("t.t.t", volume.getName());
+ Assert.assertEquals("/dev/storpool-byid/t.t.t", volume.getPath());
+ Assert.assertEquals("/dev/storpool-byid/t.t.t", volume.getFullPath());
+ Assert.assertEquals(Storage.ImageFormat.QCOW2.toString(), volume.getFormat());
+ Assert.assertEquals(524288000L, volume.getSize());
+ Assert.assertEquals(1073741824L, volume.getVirtualSize());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathExcludesVolumeTaggedCs() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ JsonObject tagged = volumeJsonObject("~t.t.t", 1073741824L, 1073741824L);
+ JsonObject tags = new JsonObject();
+ tags.addProperty("cs", "volume");
+ tagged.add("tags", tags);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(tagged);
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, false);
+
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathForUnmanageReturnsVolumeEvenIfTaggedCs() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ JsonObject tagged = volumeJsonObject("~t.t.t", 1073741824L, 1073741824L);
+ JsonObject tags = new JsonObject();
+ tags.addProperty("cs", "volume");
+ tagged.add("tags", tags);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(tagged);
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, true);
+
+ Assert.assertEquals(1, volumes.size());
+ Assert.assertEquals("t.t.t", volumes.get(0).getName());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathForUnmanageSkipsNonGlobalIdName() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(volumeJsonObject("~t.t.t@s.s", 1073741824L, 1073741824L));
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, true);
+
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathSkipsDeletedVolume() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(volumeJson("~t.t.t", null, true, 1073741824L, 1073741824L));
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, false);
+
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathNotFoundReturnsEmptyList() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = volumesListResponse(new JsonArray());
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, false);
+
+ Assert.assertNotNull(volumes);
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testListVolumesForImportWithPathApiErrorReturnsEmptyList() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = new StorPoolUtil.SpApiResponse();
+ setResponseError(resp);
+ when(StorPoolUtil.volumeDescribeResponse("~t.t.t", conn)).thenReturn(resp);
+
+ List<VolumeOnStorageTO> volumes = storPoolPrimaryDataStoreDriver.listVolumesForImport(importPool, "/dev/storpool-byid/t.t.t", null, false);
+
+ Assert.assertNotNull(volumes);
+ Assert.assertTrue(volumes.isEmpty());
+ }
+
+ @Test
+ public void testCreateAsyncSetsVolumeFormatToQcow2() {
+ DataStore dataStore = mock(DataStore.class);
+ when(dataStore.getUuid()).thenReturn("SP_API_HTTP=1.1.1.1:81;SP_AUTH_TOKEN=token;SP_TEMPLATE=template_name");
+ when(dataStore.getId()).thenReturn(1L);
+
+ VolumeInfo vinfo = mock(VolumeInfo.class);
+ when(vinfo.getType()).thenReturn(DataObjectType.VOLUME);
+ when(vinfo.getUuid()).thenReturn(UUID.randomUUID().toString());
+ when(vinfo.getId()).thenReturn(101L);
+ when(vinfo.getSize()).thenReturn(1073741824L);
+ when(vinfo.getPassphraseId()).thenReturn(null);
+ when(vinfo.getInstanceId()).thenReturn(null);
+ when(vinfo.getDiskOfferingId()).thenReturn(null);
+
+ when(StorPoolUtil.getSpConnection(dataStore.getUuid(), dataStore.getId(), detailsDao, storagePool)).thenReturn(conn);
+ StorPoolUtil.SpApiResponse resp = new StorPoolUtil.SpApiResponse();
+ when(StorPoolUtil.volumeCreate(any(StorPoolVolumeDef.class), eq(conn))).thenReturn(resp);
+ when(StorPoolUtil.getNameFromResponse(resp, false)).thenReturn("t.t.t");
+
+ VolumeVO volumeVO = mock(VolumeVO.class);
+ when(volumeVO.getId()).thenReturn(101L);
+ when(volumeDao.findById(101L)).thenReturn(volumeVO);
+
+ StoragePoolVO storagePoolVO = mock(StoragePoolVO.class);
+ when(storagePool.findById(1L)).thenReturn(storagePoolVO);
+
+ @SuppressWarnings("unchecked")
+ AsyncCompletionCallback<CreateCmdResult> createCallback = mock(AsyncCompletionCallback.class);
+
+ storPoolPrimaryDataStoreDriver.createAsync(dataStore, vinfo, createCallback);
+
+ verify(volumeVO).setFormat(Storage.ImageFormat.QCOW2);
+ verify(volumeDao).update(eq(101L), eq(volumeVO));
+ }
+
+ @Test
+ public void testVolumeImportedTagsVolumeWithCsAndUuid() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ String volumeUuid = UUID.randomUUID().toString();
+ VolumeVO volume = mock(VolumeVO.class);
+ when(volume.getPath()).thenReturn("/dev/storpool-byid/t.t.t");
+ when(volume.getUuid()).thenReturn(volumeUuid);
+ when(volume.getDiskOfferingId()).thenReturn(5L);
+ when(volume.getInstanceId()).thenReturn(null);
+ when(diskOfferingDetailsDao.findDetail(5L, StorPoolUtil.SP_TIER)).thenReturn(null);
+ StorPoolUtil.SpApiResponse resp = new StorPoolUtil.SpApiResponse();
+ when(StorPoolUtil.volumeUpdate(any(StorPoolVolumeDef.class), eq(conn))).thenReturn(resp);
+
+ storPoolPrimaryDataStoreDriver.volumeManagementChanged(importPool, volume, true);
+
+ ArgumentCaptor<StorPoolVolumeDef> captor = ArgumentCaptor.forClass(StorPoolVolumeDef.class);
+ utilities.verify(() -> StorPoolUtil.volumeUpdate(captor.capture(), eq(conn)), times(1));
+ StorPoolVolumeDef updated = captor.getValue();
+ Assert.assertEquals("~t.t.t", updated.getName());
+ Assert.assertEquals(volumeUuid, updated.getTags().get("uuid"));
+ Assert.assertEquals("volume", updated.getTags().get("cs"));
+ Assert.assertNull(updated.getTags().get("qc"));
+ Assert.assertNull(updated.getTags().get("cvm"));
+ }
+
+ @Test
+ public void testVolumeImportedSetsCvmTagWhenAttachedToVm() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ VolumeVO volume = mock(VolumeVO.class);
+ when(volume.getPath()).thenReturn("/dev/storpool-byid/t.t.t");
+ when(volume.getUuid()).thenReturn(UUID.randomUUID().toString());
+ when(volume.getInstanceId()).thenReturn(7L);
+ VMInstanceVO vm = mock(VMInstanceVO.class);
+ String vmUuid = UUID.randomUUID().toString();
+ when(vm.getUuid()).thenReturn(vmUuid);
+ when(vmInstanceDao.findById(7L)).thenReturn(vm);
+ StorPoolUtil.SpApiResponse resp = new StorPoolUtil.SpApiResponse();
+ when(StorPoolUtil.volumeUpdate(any(StorPoolVolumeDef.class), eq(conn))).thenReturn(resp);
+
+ storPoolPrimaryDataStoreDriver.volumeManagementChanged(importPool, volume, true);
+
+ ArgumentCaptor<StorPoolVolumeDef> captor = ArgumentCaptor.forClass(StorPoolVolumeDef.class);
+ utilities.verify(() -> StorPoolUtil.volumeUpdate(captor.capture(), eq(conn)), times(1));
+ Assert.assertEquals(vmUuid, captor.getValue().getTags().get("cvm"));
+ }
+
+ @Test
+ public void testVolumeImportedIncludesQosTierFromDiskOffering() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ VolumeVO volume = mock(VolumeVO.class);
+ when(volume.getPath()).thenReturn("/dev/storpool-byid/t.t.t");
+ when(volume.getUuid()).thenReturn(UUID.randomUUID().toString());
+ when(volume.getDiskOfferingId()).thenReturn(5L);
+ when(volume.getInstanceId()).thenReturn(null);
+ DiskOfferingDetailVO tierDetail = mock(DiskOfferingDetailVO.class);
+ when(tierDetail.getValue()).thenReturn("ssd-tier");
+ when(diskOfferingDetailsDao.findDetail(5L, StorPoolUtil.SP_TIER)).thenReturn(tierDetail);
+ StorPoolUtil.SpApiResponse resp = new StorPoolUtil.SpApiResponse();
+ when(StorPoolUtil.volumeUpdate(any(StorPoolVolumeDef.class), eq(conn))).thenReturn(resp);
+
+ storPoolPrimaryDataStoreDriver.volumeManagementChanged(importPool, volume, true);
+
+ ArgumentCaptor<StorPoolVolumeDef> captor = ArgumentCaptor.forClass(StorPoolVolumeDef.class);
+ utilities.verify(() -> StorPoolUtil.volumeUpdate(captor.capture(), eq(conn)), times(1));
+ Assert.assertEquals("ssd-tier", captor.getValue().getTags().get("qc"));
+ }
+
+ @Test
+ public void testVolumeUnmanagedClearsCsUuidAndQosTags() {
+ when(StorPoolUtil.getSpConnection(importPool.getUuid(), importPool.getId(), detailsDao, storagePool)).thenReturn(conn);
+ VolumeVO volume = mock(VolumeVO.class);
+ when(volume.getPath()).thenReturn("/dev/storpool-byid/t.t.t");
+ StorPoolUtil.SpApiResponse resp = new StorPoolUtil.SpApiResponse();
+ when(StorPoolUtil.volumeUpdate(any(StorPoolVolumeDef.class), eq(conn))).thenReturn(resp);
+
+ storPoolPrimaryDataStoreDriver.volumeManagementChanged(importPool, volume, false);
+
+ ArgumentCaptor<StorPoolVolumeDef> captor = ArgumentCaptor.forClass(StorPoolVolumeDef.class);
+ utilities.verify(() -> StorPoolUtil.volumeUpdate(captor.capture(), eq(conn)), times(1));
+ StorPoolVolumeDef updated = captor.getValue();
+ Assert.assertEquals("~t.t.t", updated.getName());
+ Assert.assertEquals("", updated.getTags().get("cs"));
Review Comment:
Given the unmanage path leaves `cs` present but empty (`cs == \"\"`), add a unit test to ensure `listVolumesForImport(..., forUnmanage=false)` treats an empty `cs` value as *unmanaged* and lists the volume again. This directly guards the behavior around the new `isTaggedCs`/eligibility logic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "Damans227 (via GitHub)" <gi...@apache.org>.
Damans227 commented on PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#issuecomment-6289446092
can you look at the build? three of the new storpool driver tests fail on every push since oct 14. the unmanage one checks vc_policy but the tag is vc-policy
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#issuecomment-6293528597
@nvazquez a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#issuecomment-6309509057
<b>[SF] Trillian test result (tid-17171)</b>
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 52210 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr14389-t17171-kvm-ol8.zip
Smoke tests completed. 148 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_03_ssvm_internals | `Failure` | 2.88 | test_ssvm.py
test_05_stop_ssvm | `Failure` | 51.60 | test_ssvm.py
test_07_reboot_ssvm | `Failure` | 137.64 | test_ssvm.py
test_09_reboot_ssvm_forced | `Failure` | 144.40 | test_ssvm.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#issuecomment-6313358730
<b>[SF] Trillian Build Failed (tid-17173)<b/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "slavkap (via GitHub)" <gi...@apache.org>.
slavkap commented on code in PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#discussion_r4269846588
##########
plugins/storage/volume/storpool/src/main/java/org/apache/cloudstack/storage/datastore/driver/StorPoolPrimaryDataStoreDriver.java:
##########
@@ -1367,6 +1377,120 @@ public Pair<Long, Long> getVolumeStats(StoragePool storagePool, String volumeId)
return null;
}
+ /**
+ * Lists StorPool volumes available for import (globalId name, no "cs" tag unless
+ * {@code forUnmanage}). When {@code volumePath} is given, looks up that one volume directly
+ * instead of browsing the whole pool; when {@code forUnmanage} is also true, an already
+ * "cs"-tagged volume is still returned, since the caller is unmanageVolume() looking for the
+ * very volume it is about to drop from CloudStack.
+ */
+ @Override
+ public List<VolumeOnStorageTO> listVolumesForImport(StoragePool pool, String volumePath, String keyword, boolean forUnmanage) {
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.listVolumesForImport: pool=%s, volumePath=%s, keyword=%s, forUnmanage=%s", pool.getName(), volumePath, keyword, forUnmanage);
+ SpConnectionDesc conn = StorPoolUtil.getSpConnection(pool.getUuid(), pool.getId(), storagePoolDetailsDao, primaryStoreDao);
+ List<VolumeOnStorageTO> volumes = new ArrayList<>();
+
+ if (StringUtils.isNotBlank(volumePath)) {
+ return getVolumeOnStorageByPath(pool, volumePath, conn, volumes, forUnmanage);
+ }
+
+ SpApiResponse resp = StorPoolUtil.volumesListResponse(conn);
+ if (resp.getError() != null) {
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.listVolumesForImport: could not list volumes for pool=%s. Error: %s", pool.getName(), resp.getError());
+ return volumes;
+ }
+ return getVolumeOnStorage(pool, keyword, resp, volumes);
+ }
+
+ @NotNull
+ private List<VolumeOnStorageTO> getVolumeOnStorage(StoragePool pool, String keyword, SpApiResponse resp, List<VolumeOnStorageTO> volumes) {
+ JsonArray volumesList = resp.fullJson.getAsJsonObject().getAsJsonArray("data");
+
+ for (int i = 0; i < volumesList.size(); i++) {
+ JsonObject volume = volumesList.get(i).getAsJsonObject();
+ String rawName = volume.get("name").getAsString();
+ if (isVolumeDeleted(volume) || !isEligibleForBulkImport(volume, rawName)) {
+ continue;
+ }
+
+ String name = rawName.substring(1);
+
+ if (StringUtils.isNotBlank(keyword) && !name.contains(keyword)) {
+ continue;
+ }
+ volumes.add(toVolumeOnStorageTO(volume, name));
+ }
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.listVolumesForImport: found %s volume(s) for pool=%s", volumes.size(), pool.getName());
+ return volumes;
+ }
+
+ @NotNull
+ private List<VolumeOnStorageTO> getVolumeOnStorageByPath(StoragePool pool, String volumePath, SpConnectionDesc conn, List<VolumeOnStorageTO> volumes, boolean forUnmanage) {
+ String name = StorPoolStorageAdaptor.getVolumeNameFromPath(volumePath, true);
+ SpApiResponse resp = StorPoolUtil.volumeDescribeResponse(name, conn);
+ if (resp.getError() != null) {
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.listVolumesForImport: could not describe volume=%s on pool=%s. Error: %s", name, pool.getName(), resp.getError());
+ return volumes;
+ }
+ JsonArray data = resp.fullJson.getAsJsonObject().getAsJsonArray("data");
+ if (data == null || data.isEmpty()) {
+ return volumes;
+ }
+ JsonObject volume = data.get(0).getAsJsonObject();
+ String rawName = volume.get("name").getAsString();
+ if (isVolumeDeleted(volume) || !isGlobalIdVolumeName(rawName) || (!forUnmanage && isTaggedCs(volume))) {
+ return volumes;
+ }
+ volumes.add(toVolumeOnStorageTO(volume, rawName.substring(1)));
+ return volumes;
+ }
+
+ private static boolean isVolumeDeleted(JsonObject volume) {
+ return volume.has("deleted") && volume.get("deleted").getAsBoolean();
+ }
+
+ private static boolean isGlobalIdVolumeName(String rawName) {
+ return !rawName.startsWith("*") && !rawName.contains("@") && rawName.startsWith("~");
+ }
+
+ private static boolean isTaggedCs(JsonObject volume) {
+ JsonObject tags = volume.has("tags") && !volume.get("tags").isJsonNull() ? volume.getAsJsonObject("tags") : null;
+ return tags != null && tags.has("cs");
+ }
+
+ private static boolean isEligibleForBulkImport(JsonObject volume, String rawName) {
+ return isGlobalIdVolumeName(rawName) && !isTaggedCs(volume);
+ }
+
+ private VolumeOnStorageTO toVolumeOnStorageTO(JsonObject volume, String name) {
+ String path = StorPoolUtil.devPath(name);
+ long size = volume.get("size").getAsLong();
+ long spaceUsed = volume.has("spaceUsed") ? volume.get("spaceUsed").getAsLong() : size;
+ StorPoolUtil.spLog("listVolumesForImport: found volume=%s, path=%s, size=%s, spaceUsed=%s", name, path, size, spaceUsed);
+ return new VolumeOnStorageTO(Hypervisor.HypervisorType.KVM, path, name, path,
+ Storage.ImageFormat.QCOW2.toString(), spaceUsed, size);
+ }
+
+ @Override
+ public void volumeManagementChanged(StoragePool pool, Volume volume, boolean imported) {
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.volumeManagementChanged: pool=%s, volume=%s, imported=%s", pool.getName(), volume.getPath(), imported);
+ SpConnectionDesc conn = StorPoolUtil.getSpConnection(pool.getUuid(), pool.getId(), storagePoolDetailsDao, primaryStoreDao);
+ String name = StorPoolStorageAdaptor.getVolumeNameFromPath(volume.getPath(), true);
+ Map<String, String> tags;
+ if (imported) {
+ String tier = volume.getDiskOfferingId() != null ? getTierFromOfferingDetail(volume.getDiskOfferingId()) : null;
+ Long instanceId = volume.getInstanceId();
+ tags = StorPoolHelper.addStorPoolTags(volume.getUuid(), getVMInstanceUUID(instanceId), "volume", getVcPolicyTag(instanceId), tier);
+ } else {
+ tags = StorPoolHelper.addStorPoolTags(null, null, "", null, null);
Review Comment:
Thanks @Damans227, for the review and good catch - passing null here left uuid/qc/cvm untouched instead of clearing them, since addStorPoolTags only sends a key when its value isn't null. Fixed to pass "" for all of them (cs/uuid/cvm/qc/vc-policy), so unmanage now clears every tag it set
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#issuecomment-6331846360
<b>[SF] Trillian test result (tid-17180)</b>
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 52541 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr14389-t17180-kvm-ol8.zip
Smoke tests completed. 149 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "slavkap (via GitHub)" <gi...@apache.org>.
slavkap commented on PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#issuecomment-6314641842
this are the test results from my dev for the failing ones from `test_ssvm.py`
```
Test List secondary storage VMs ... === TestName: test_01_list_sec_storage_vm | Status : SUCCESS ===
ok
Test List console proxy VMs ... === TestName: test_02_list_cpvm_vm | Status : SUCCESS ===
ok
Test SSVM Internals ... === TestName: test_03_ssvm_internals | Status : SUCCESS ===
ok
Test CPVM Internals ... === TestName: test_04_cpvm_internals | Status : SUCCESS ===
ok
Test stop SSVM ... === TestName: test_05_stop_ssvm | Status : SUCCESS ===
ok
Test stop CPVM ... === TestName: test_06_stop_cpvm | Status : SUCCESS ===
ok
Test reboot SSVM ... === TestName: test_07_reboot_ssvm | Status : SUCCESS ===
ok
Test reboot CPVM ... === TestName: test_08_reboot_cpvm | Status : SUCCESS ===
ok
Test force reboot SSVM ... === TestName: test_09_reboot_ssvm_forced | Status : SUCCESS ===
ok
Test force reboot CPVM ... === TestName: test_10_reboot_cpvm_forced | Status : SUCCESS ===
ok
Test destroy SSVM ... === TestName: test_11_destroy_ssvm | Status : SUCCESS ===
ok
Test destroy CPVM ... === TestName: test_12_destroy_cpvm | Status : SUCCESS ===
ok
Test NFS Version on Secondary Storage mounted properly on SSVM ... SKIP: No NFS version provided in test data
----------------------------------------------------------------------
Ran 13 tests in 736.491s
OK (SKIP=1)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#issuecomment-6313319393
@slavkap a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "nvazquez (via GitHub)" <gi...@apache.org>.
nvazquez commented on PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#issuecomment-6315030828
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#issuecomment-6293280996
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✖️ suse15. SL-JID 19632
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "slavkap (via GitHub)" <gi...@apache.org>.
slavkap commented on code in PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#discussion_r4247489530
##########
plugins/storage/volume/storpool/src/main/java/org/apache/cloudstack/storage/datastore/driver/StorPoolPrimaryDataStoreDriver.java:
##########
@@ -1367,6 +1377,120 @@ public Pair<Long, Long> getVolumeStats(StoragePool storagePool, String volumeId)
return null;
}
+ /**
+ * Lists StorPool volumes available for import (globalId name, no "cs" tag unless
+ * {@code forUnmanage}). When {@code volumePath} is given, looks up that one volume directly
+ * instead of browsing the whole pool; when {@code forUnmanage} is also true, an already
+ * "cs"-tagged volume is still returned, since the caller is unmanageVolume() looking for the
+ * very volume it is about to drop from CloudStack.
+ */
+ @Override
+ public List<VolumeOnStorageTO> listVolumesForImport(StoragePool pool, String volumePath, String keyword, boolean forUnmanage) {
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.listVolumesForImport: pool=%s, volumePath=%s, keyword=%s, forUnmanage=%s", pool.getName(), volumePath, keyword, forUnmanage);
+ SpConnectionDesc conn = StorPoolUtil.getSpConnection(pool.getUuid(), pool.getId(), storagePoolDetailsDao, primaryStoreDao);
+ List<VolumeOnStorageTO> volumes = new ArrayList<>();
+
+ if (StringUtils.isNotBlank(volumePath)) {
+ return getVolumeOnStorageByPath(pool, volumePath, conn, volumes, forUnmanage);
+ }
+
+ SpApiResponse resp = StorPoolUtil.volumesListResponse(conn);
+ if (resp.getError() != null) {
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.listVolumesForImport: could not list volumes for pool=%s. Error: %s", pool.getName(), resp.getError());
+ return volumes;
+ }
+ return getVolumeOnStorage(pool, keyword, resp, volumes);
+ }
+
+ @NotNull
+ private List<VolumeOnStorageTO> getVolumeOnStorage(StoragePool pool, String keyword, SpApiResponse resp, List<VolumeOnStorageTO> volumes) {
+ JsonArray volumesList = resp.fullJson.getAsJsonObject().getAsJsonArray("data");
+
+ for (int i = 0; i < volumesList.size(); i++) {
+ JsonObject volume = volumesList.get(i).getAsJsonObject();
+ String rawName = volume.get("name").getAsString();
+ if (isVolumeDeleted(volume) || !isEligibleForBulkImport(volume, rawName)) {
+ continue;
+ }
+
+ String name = rawName.split("~")[1];
+
+ if (StringUtils.isNotBlank(keyword) && !name.contains(keyword)) {
+ continue;
+ }
+ volumes.add(toVolumeOnStorageTO(volume, name));
+ }
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.listVolumesForImport: found %s volume(s) for pool=%s", volumes.size(), pool.getName());
+ return volumes;
+ }
+
+ @NotNull
+ private List<VolumeOnStorageTO> getVolumeOnStorageByPath(StoragePool pool, String volumePath, SpConnectionDesc conn, List<VolumeOnStorageTO> volumes, boolean forUnmanage) {
+ String name = StorPoolStorageAdaptor.getVolumeNameFromPath(volumePath, true);
+ SpApiResponse resp = StorPoolUtil.volumeDescribeResponse(name, conn);
+ if (resp.getError() != null) {
+ StorPoolUtil.spLog("StorPoolPrimaryDataStoreDriver.listVolumesForImport: could not describe volume=%s on pool=%s. Error: %s", name, pool.getName(), resp.getError());
+ return volumes;
+ }
+ JsonArray data = resp.fullJson.getAsJsonObject().getAsJsonArray("data");
+ if (data == null || data.isEmpty()) {
+ return volumes;
+ }
+ JsonObject volume = data.get(0).getAsJsonObject();
+ String rawName = volume.get("name").getAsString();
+ if (isVolumeDeleted(volume) || !isGlobalIdVolumeName(rawName) || (!forUnmanage && isTaggedCs(volume))) {
+ return volumes;
+ }
+ volumes.add(toVolumeOnStorageTO(volume, rawName.split("~")[1]));
+ return volumes;
+ }
+
+ private static boolean isVolumeDeleted(JsonObject volume) {
+ return volume.has("deleted") && volume.get("deleted").getAsBoolean();
+ }
+
+ private static boolean isGlobalIdVolumeName(String rawName) {
+ return !rawName.startsWith("*") && !rawName.contains("@") && rawName.startsWith("~");
+ }
+
+ private static boolean isTaggedCs(JsonObject volume) {
+ JsonObject tags = volume.has("tags") && !volume.get("tags").isJsonNull() ? volume.getAsJsonObject("tags") : null;
+ return tags != null && tags.has("cs");
+ }
+
+ private static boolean isEligibleForBulkImport(JsonObject volume, String rawName) {
+ return isGlobalIdVolumeName(rawName) && !isTaggedCs(volume);
+ }
Review Comment:
Not a bug — StorPool's VolumeUpdate API treats an empty-string tag value as tag removal, not as setting the tag to "". This is an existing convention in this codebase: see `StorPoolUtil.volumeRemoveTags()`, which predates this PR and explicitly sends "" for tags it wants removed. So when `volumeManagementChanged(..., imported=false)` sends `cs=""`, the StorPool side drops the cs key entirely — a subsequent list/describe call returns a volume with no cs tag at all, not `cs="". isTaggedCs()`'s presence-only check is therefore correct as written.
This exact scenario is already covered end-to-end against a real StorPool pool by t`est_01_import_volume_tags_storpool_volume_with_cs_and_uuid` in t`est/integration/plugins/storpool/TestStorPoolImportVolume.py`: it creates a volume via CloudStack, unmanages it, confirms the StorPool-side cs tag is cleared, then calls the bulk `listVolumesForImport` (no path filter) and asserts the unmanaged volume reappears in the list. That's precisely the regression this finding was concerned about, and it's green.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] StorPool support importing standalone disk volumes and importVm from SHARED [cloudstack]
Posted by "slavkap (via GitHub)" <gi...@apache.org>.
slavkap commented on code in PR #14389:
URL: https://github.com/apache/cloudstack/pull/14389#discussion_r4247499419
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/wrapper/LibvirtCheckVolumeCommandWrapper.java:
##########
@@ -63,27 +71,37 @@ public Answer execute(final CheckVolumeCommand command, final LibvirtComputingRe
try {
if (STORAGE_POOL_TYPES_SUPPORTED.contains(storageFilerTO.getType())) {
- final KVMPhysicalDisk vol = pool.getPhysicalDisk(srcFile);
- final String path = vol.getPath();
- final boolean isRbd = Storage.StoragePoolType.RBD.equals(storageFilerTO.getType());
-
- Map<VolumeOnStorageTO.Detail, String> volumeDetails = getVolumeDetails(pool, vol);
- if (MapUtils.isEmpty(volumeDetails)) {
- return new Answer(command, false, "Unable to read the volume on the storage pool");
+ final boolean shouldAttachTheVolume = POOL_NEEDS_TO_EXPOSE_THE_VOLUME.contains(storageFilerTO.getType());
+ if (shouldAttachTheVolume && !pool.connectPhysicalDisk(srcFile, new HashMap<>())) {
+ return new Answer(command, false, "Unable to attach the volume on the storage pool");
}
+ try {
+ final KVMPhysicalDisk vol = pool.getPhysicalDisk(srcFile);
+ final String path = vol.getPath();
+ final boolean isRawBlockPool = RAW_BLOCK_STORAGE_POOL_TYPES.contains(storageFilerTO.getType());
+
+ Map<VolumeOnStorageTO.Detail, String> volumeDetails = getVolumeDetails(pool, vol);
+ if (MapUtils.isEmpty(volumeDetails)) {
+ return new Answer(command, false, "Unable to read the volume on the storage pool");
+ }
- if (!isRbd) {
- try {
- KVMPhysicalDisk.checkQcow2File(path);
- } catch (final CloudRuntimeException e) {
- return new CheckVolumeAnswer(command, false, "", 0, volumeDetails);
+ if (!isRawBlockPool) {
+ try {
+ KVMPhysicalDisk.checkQcow2File(path);
+ } catch (final CloudRuntimeException e) {
+ return new CheckVolumeAnswer(command, false, "", 0, volumeDetails);
+ }
}
- }
- // Images on RBD are raw and the path is an image name that qemu-img cannot open
- // without the rbd: URI, so take the size libvirt already reported for the volume.
- long size = isRbd ? vol.getVirtualSize() : KVMPhysicalDisk.getVirtualSizeFromFile(path);
- return new CheckVolumeAnswer(command, true, "", size, volumeDetails);
+ // Images on RBD are raw and the path is an image name that qemu-img cannot open
+ // without the rbd: URI, so take the size libvirt already reported for the volume
+ long size = isRawBlockPool ? vol.getVirtualSize() : KVMPhysicalDisk.getVirtualSizeFromFile(path);
Review Comment:
This is intentional, not inconsistent. `format=QCOW2` in CloudStack's DB for StorPool volumes is a long-standing labeling convention (see `StorPoolDataMotionStrategy.setFormat(ImageFormat.QCOW2)`, which predates this PR) — it doesn't mean the bytes on the StorPool block device are actually structured as a QCOW2 container. On disk, a StorPool volume is a raw block device, same as RBD, so grouping it with RBD in `RAW_BLOCK_STORAGE_POOL_TYPES` (skip QCOW2 structural validation, read size via the libvirt-reported virtual size) is correct. Also worth noting checkQcow2File() is a no-op for any path without a .qcow2 extension, so it wouldn't have done anything differently for StorPool paths anyway. We'll leave this as is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org