You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by "abh1sar (via GitHub)" <gi...@apache.org> on 2024/12/25 05:12:52 UTC
[PR] [WIP] Create new Instance from VM backup [cloudstack]
abh1sar opened a new pull request, #10140:
URL: https://github.com/apache/cloudstack/pull/10140
### Description
This PR adds the ability to create a new Instance from a VM backup for Dummy, NAS and Veeam backup Providers.
This will still work if the original Instance used to create the backup was expunged.
<!--- Describe your changes in DETAIL - And how has behaviour functionally changed. -->
<!-- For new features, provide link to FS, dev ML discussion etc. -->
<!-- In case of bug fix, the expected and actual behaviours, steps to reproduce. -->
<!-- When "Fixes: #<id>" is specified, the issue/PR will automatically be closed when this PR gets merged -->
<!-- For addressing multiple issues/PRs, use multiple "Fixes: #<id>" -->
<!-- Fixes: # -->
<!--- ******************************************************************************* -->
<!--- NOTE: AUTOMATION USES THE DESCRIPTIONS TO SET LABELS AND PRODUCE DOCUMENTATION. -->
<!--- PLEASE PUT AN 'X' in only **ONE** box -->
<!--- ******************************************************************************* -->
### Types of changes
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
- [ ] build/CI
- [ ] test (unit or integration test code)
### Feature/Enhancement Scale or Bug Severity
#### Feature/Enhancement Scale
- [ ] Major
- [ ] Minor
#### Bug Severity
- [ ] BLOCKER
- [ ] Critical
- [ ] Major
- [ ] Minor
- [ ] Trivial
### Screenshots (if appropriate):
### How Has This Been Tested?
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
#### How did you try to break this feature and the system with this change?
<!-- see how your change affects other areas of the code, etc. -->
<!-- Please read the [CONTRIBUTING](https://github.com/apache/cloudstack/blob/main/CONTRIBUTING.md) document -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [WIP] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2585691917
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12063
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "shwstppr (via GitHub)" <gi...@apache.org>.
shwstppr commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r1922242112
##########
server/src/main/java/org/apache/cloudstack/backup/BackupManagerImpl.java:
##########
@@ -748,6 +819,151 @@ private Backup.VolumeInfo getVolumeInfo(List<Backup.VolumeInfo> backedUpVolumes,
return null;
}
+ @Override
+ public void updateDiskOfferingSizeFromBackup(List<DiskOfferingInfo> dataDiskOfferingsInfo, Backup backup) {
+ List<DiskOfferingInfo> dataDiskOfferingsInfoFromBackup = getDataDiskOfferingListFromBackup(backup);
+ int index = 0;
+ for(DiskOfferingInfo diskOfferingInfo : dataDiskOfferingsInfo) {
+ diskOfferingInfo.setSize(Math.max(diskOfferingInfo.getSize(), dataDiskOfferingsInfoFromBackup.get(index).getSize()));
+ index++;
+ }
+ }
+
+ @Override
+ public DiskOfferingInfo getRootDiskOfferingInfoFromBackup(Backup backup) {
+ String diskOfferingIdsDetail = backup.getDetail(ApiConstants.DISK_OFFERING_IDS);
+ if (diskOfferingIdsDetail == null) {
+ return null;
+ }
+ String [] diskOfferingIds = diskOfferingIdsDetail.split(",");
+ String [] deviceIds = backup.getDetail(ApiConstants.DEVICE_IDS).split(",");
+ String [] diskSizes = backup.getDetail(ApiConstants.DISK_SIZES).split(",");
+
+ for (int i = 0; i < diskOfferingIds.length; i++) {
+ if (deviceIds[i].equals("0")) {
+ DiskOfferingVO diskOffering = diskOfferingDao.findByUuid(diskOfferingIds[i]);
+ if (diskOffering == null) {
+ throw new CloudRuntimeException("Unable to find the root disk offering with uuid (" + diskOfferingIds[i] + ") stored in backup. Please specify a valid root disk offering id while creating the instance");
+ }
+ Long size = Long.parseLong(diskSizes[i]) / (1024 * 1024 * 1024);
+ return new DiskOfferingInfo(diskOffering, size, null, null, 0L);
+ }
+ }
+ return null;
+ }
+
+ @Override
+ public List<DiskOfferingInfo> getDataDiskOfferingListFromBackup(Backup backup) {
+ String diskOfferingIdsDetail = backup.getDetail(ApiConstants.DISK_OFFERING_IDS);
+ if (diskOfferingIdsDetail == null) {
+ return null;
+ }
+
+ String [] diskOfferingIds = diskOfferingIdsDetail.split(",");
+ String [] deviceIds = backup.getDetail(ApiConstants.DEVICE_IDS).split(",");
+ String [] diskSizes = backup.getDetail(ApiConstants.DISK_SIZES).split(",");
+ String [] minIopsList = backup.getDetail(ApiConstants.MIN_IOPS).split(",");
+ String [] maxIopsList = backup.getDetail(ApiConstants.MAX_IOPS).split(",");
+
+ List<DiskOfferingInfo> diskOfferingInfoList = new ArrayList<>();
+ for (int i = 0; i < diskOfferingIds.length; i++) {
+ Long deviceId = Long.parseLong(deviceIds[i]);
+ if (deviceId == 0) {
+ continue;
+ }
+ DiskOfferingVO diskOffering = diskOfferingDao.findByUuid(diskOfferingIds[i]);
+ if (diskOffering == null) {
+ throw new CloudRuntimeException("Unable to find the disk offering with uuid (" + diskOfferingIds[i] + ") stored in backup. Please specify a valid disk offering id while creating the instance");
+ }
+ Long size = Long.parseLong(diskSizes[i]) / (1024 * 1024 * 1024);
+ Long minIops = (diskOffering.isCustomizedIops() != null && diskOffering.isCustomizedIops() && !minIopsList[i].equals("null")) ?
+ Long.parseLong(minIopsList[i]) : null;
+ Long maxIops = (diskOffering.isCustomizedIops() != null && diskOffering.isCustomizedIops() && !maxIopsList[i].equals("null")) ?
+ Long.parseLong(maxIopsList[i]) : null;
+ diskOfferingInfoList.add(new DiskOfferingInfo(diskOffering, size, minIops, maxIops, deviceId));
+ }
+ return diskOfferingInfoList;
+ }
+
+ @Override
+ public boolean restoreBackupToVM(final Long backupId, final Long vmId) throws ResourceUnavailableException {
+ final BackupVO backup = backupDao.findById(backupId);
+ if (backup == null) {
+ throw new CloudRuntimeException("Backup " + backupId + " does not exist");
+ }
+ validateBackupForZone(backup.getZoneId());
+
+ VMInstanceVO vm = vmInstanceDao.findByIdIncludingRemoved(vmId);
+ if (vm == null) {
+ throw new CloudRuntimeException("VM ID " + backup.getVmId() + " couldn't be found on existing or removed VMs");
+ }
+ accountManager.checkAccess(CallContext.current().getCallingAccount(), null, true, vm);
+
+ if (vm.getRemoved() != null) {
+ throw new CloudRuntimeException("VM is removed from CloudStack");
+ }
+ if (vm.getRemoved() != null) {
+ throw new CloudRuntimeException("VM should be in the stopped state");
+ }
+ if (!vm.getState().equals(VirtualMachine.State.Stopped)) {
+ throw new CloudRuntimeException("Existing VM should be stopped before being restored from backup");
+ }
+
+ List<Backup.VolumeInfo> backupVolumes = backup.getBackedUpVolumes();
+ if (backupVolumes == null) {
+ throw new CloudRuntimeException("Backed-up volumes not found");
+ }
+
+ List<VolumeVO> vmVolumes = volumeDao.findByInstance(vmId);
+ if (vmVolumes.size() != backupVolumes.size()) {
+ throw new CloudRuntimeException("Unable to restore VM with the current backup as the backup has different number of disks as the VM");
+ }
+
+ BackupOffering offering = backupOfferingDao.findByIdIncludingRemoved(backup.getBackupOfferingId());
+ if (offering == null) {
+ String errorMessage = "Failed to find backup offering of the VM backup.";
+ throw new CloudRuntimeException("Failed to find backup offering of the VM backup.");
+ }
+ if ("networker".equals(offering.getProvider())) {
+ throw new CloudRuntimeException("Create instance from VM is not supported for Networker Backup plugin.");
+ }
+
+ String backupDetailsInMessage = ReflectionToStringBuilderUtils.reflectOnlySelectedFields(backup, "uuid", "externalId", "newVMId", "type", "status", "date");
+ try {
+ updateVmState(vm, VirtualMachine.Event.RestoringRequested, VirtualMachine.State.Restoring);
+ updateVolumeState(vm, Volume.Event.RestoreRequested, Volume.State.Restoring);
+ ActionEventUtils.onStartedActionEvent(User.UID_SYSTEM, vm.getAccountId(), EventTypes.EVENT_VM_BACKUP_RESTORE,
+ String.format("Restoring VM %s from backup %s", vm.getUuid(), backup.getUuid()),
+ vm.getId(), ApiCommandResourceType.VirtualMachine.toString(),
+ true, 0);
+
+ String host = null;
+ String dataStore = null;
+ if (!"nas".equals(offering.getProvider())) {
+ Pair<HostVO, StoragePoolVO> restoreInfo = getRestoreVolumeHostAndDatastore(vm);
+ host = restoreInfo.first().getPrivateIpAddress();
+ dataStore = restoreInfo.second().getUuid();
+ }
+ final BackupProvider backupProvider = getBackupProvider(offering.getProvider());
+ if (!backupProvider.restoreBackupToVM(vm, backup, host, dataStore)) {
+ ActionEventUtils.onCompletedActionEvent(User.UID_SYSTEM, vm.getAccountId(), EventVO.LEVEL_ERROR, EventTypes.EVENT_VM_BACKUP_RESTORE,
+ String.format("Failed to restore VM %s from backup %s", vm.getInstanceName(), backup.getUuid()),
+ vm.getId(), ApiCommandResourceType.VirtualMachine.toString(),0);
+ throw new CloudRuntimeException("Error restoring VM from backup with uuid " + backup.getUuid());
+ }
+ // The restore process is executed by a backup provider outside of ACS, I am using the catch-all (Exception) to
+ // ensure that no provider-side exception is missed. Therefore, we have a proper handling of exceptions, and rollbacks if needed.
+ } catch (Exception e) {
Review Comment:
@abh1sar up to you. Catching all will also mask any NPEs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3021251242
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3026180998
Packaging result [SF]: ✔️ el8 ✔️ el9 ✖️ debian ✔️ suse15. SL-JID 13972
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3026340188
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
weizhouapache commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2037030743
##########
plugins/backup/veeam/src/main/java/org/apache/cloudstack/backup/veeam/VeeamClient.java:
##########
@@ -659,9 +656,7 @@ public boolean setJobSchedule(final String jobName) {
public boolean deleteJobAndBackup(final String jobName) {
Pair<Boolean, String> result = executePowerShellCommands(Arrays.asList(
String.format("$job = Get-VBRJob -Name '%s'", jobName),
- "if ($job) { Remove-VBRJob -Job $job -Confirm:$false }",
- String.format("$backup = Get-VBRBackup -Name '%s'", jobName),
- "if ($backup) { Remove-VBRBackup -Backup $backup -FromDisk -Confirm:$false }"
+ "if ($job) { Remove-VBRJob -Job $job -Confirm:$false }"
Review Comment:
there was a PR #6589 . Is that included in this PR ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
weizhouapache commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2792362931
> > this PR contains 3 changes
> >
> > * create new vm from backup
> > * threshold for object storage
> > * threshold for backup
> >
> > I would prefer to see 3 PRs (at least 2 PRs) but it looks difficult to seperate them now
>
> It's not ideal, but it was done this way so test effort could be minimised.
I do not know how much QA work will be reduced. I think they will still have to test all the scenarios.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2888181229
<b>[SF] Trillian test result (tid-13333)</b>
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 51527 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10140-t13333-kvm-ol8.zip
Smoke tests completed. 139 look OK, 3 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_attach_and_distribute_multiple_volumes | `Error` | 29.44 | test_attach_multiple_volumes.py
test_attach_multiple_volumes | `Failure` | 3.50 | test_attach_multiple_volumes.py
test_attach_multiple_volumes | `Error` | 3.50 | test_attach_multiple_volumes.py
ContextSuite context=TestMultipleVolumeAttach>:teardown | `Error` | 4.64 | test_attach_multiple_volumes.py
ContextSuite context=TestNASBackupAndRecovery>:setup | `Error` | 0.00 | test_backup_recovery_nas.py
test_01_deployVMInSharedNetwork | `Failure` | 70.26 | test_network.py
ContextSuite context=TestSharedNetworkWithConfigDrive>:teardown | `Error` | 71.38 | test_network.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
weizhouapache commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2812198952
> Thanks @weizhouapache
>
> > * for vm created from ISO, the root volume uses diskofferingId , deviceId should be 0 I think.
>
> I tested this. The root volume is being assigned deviceId 0 for both template and iso.
>
> > * for vm created with multiple data disks, it uses `datadiskofferinglist`, but deviceId of volumes are not set in dataDiskOfferingInfo.
>
> deviceIds are set here
>
> ```
> public List<DiskOfferingInfo> getDataDiskOfferingListFromBackup(Backup backup) {
> ...
> diskOfferingInfoList.add(new DiskOfferingInfo(diskOffering, size, minIops, maxIops, deviceId));
> ```
>
> Please let me know if this sounds ok.
thanks @abh1sar
good to know it
About the deviceId of datadiskoffering
```
dataDiskOfferingInfo.setDeviceId(1L);
```
should it be 0 if the vm is created from ISO ?
this is not a valid question if we do not support backup actions for vm from ISO.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2893589515
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2965460869
Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 13732
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2966015217
Packaging result [SF]: ✖️ el8 ✖️ el9 ✔️ debian ✖️ suse15. SL-JID 13742
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2803448000
<b>[SF] Trillian test result (tid-12978)</b>
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 54966 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10140-t12978-kvm-ol8.zip
Smoke tests completed. 140 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
ContextSuite context=TestClusterDRS>:setup | `Error` | 0.00 | test_cluster_drs.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2788714289
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2824241860
Packaging result [SF]: ✔️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 13151
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2869346838
@abh1sar a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3072023962
<b>[SF] Trillian Build Failed (tid-13773)<b/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3067874852
Packaging result [SF]: ✔️ el8 ✔️ el9 ✖️ debian ✔️ suse15. SL-JID 14156
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3100558536
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14284
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3138784363
<b>[SF] Trillian test result (tid-13974)</b>
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 80618 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10140-t13974-kvm-ol8.zip
Smoke tests completed. 138 look OK, 8 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
ContextSuite context=TestRAMCPUResourceAccounting>:setup | `Error` | 0.00 | test_resource_accounting.py
test_01_list_sec_storage_vm | `Failure` | 0.05 | test_ssvm.py
test_03_ssvm_internals | `Failure` | 0.05 | test_ssvm.py
test_05_stop_ssvm | `Failure` | 0.04 | test_ssvm.py
test_07_reboot_ssvm | `Failure` | 0.05 | test_ssvm.py
test_11_destroy_ssvm | `Failure` | 917.32 | test_ssvm.py
ContextSuite context=TestVMWareStoragePolicies>:setup | `Error` | 0.00 | test_storage_policy.py
test_01_create_template | `Error` | 1.14 | test_templates.py
test_CreateTemplateWithDuplicateName | `Error` | 1.11 | test_templates.py
test_02_create_template_with_checksum_sha1 | `Error` | 65.65 | test_templates.py
test_03_create_template_with_checksum_sha256 | `Error` | 65.63 | test_templates.py
test_04_create_template_with_checksum_md5 | `Error` | 65.64 | test_templates.py
test_05_create_template_with_no_checksum | `Error` | 65.63 | test_templates.py
ContextSuite context=TestTemplates>:setup | `Error` | 240.85 | test_templates.py
ContextSuite context=TestISOUsage>:setup | `Error` | 0.00 | test_usage.py
test_01_snapshot_usage | `Error` | 1.19 | test_usage.py
test_01_template_usage | `Error` | 1.20 | test_usage.py
test_01_volume_usage | `Error` | 130.86 | test_usage.py
test_06_download_detached_volume | `Error` | 300.10 | test_volumes.py
test_13_migrate_volume_and_change_offering | `Error` | 126.36 | test_volumes.py
ContextSuite context=TestIpv6Vpc>:setup | `Error` | 0.00 | test_vpc_ipv6.py
ContextSuite context=TestRVPCSite2SiteVpn>:setup | `Error` | 0.00 | test_vpc_vpn.py
ContextSuite context=TestVPCSite2SiteVPNMultipleOptions>:setup | `Error` | 0.00 | test_vpc_vpn.py
ContextSuite context=TestVpcRemoteAccessVpn>:setup | `Error` | 0.00 | test_vpc_vpn.py
ContextSuite context=TestVpcSite2SiteVpn>:setup | `Error` | 0.00 | test_vpc_vpn.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3125500284
> [SF] Trillian test result (tid-13919) Environment: xcpng82 (x2), Advanced Networking with Mgmt server ol9 Total time taken: 76201 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10140-t13919-xcpng82.zip Smoke tests completed. 137 look OK, 6 have errors, 0 did not run Only failed and skipped tests results shown below:
>
> Test Result Time (s) Test File
> test_01_events_resource `Error` 233.26 test_events_resource.py
> ContextSuite context=TestClusterDRS>:setup `Error` 0.00 test_cluster_drs.py
> test_01_non_strict_host_anti_affinity `Error` 230.06 test_nonstrict_affinity_group.py
> test_02_non_strict_host_affinity `Error` 67.24 test_nonstrict_affinity_group.py
> test_01_vpn_usage `Error` 1.12 test_usage.py
> test_11_destroy_vm_and_volumes `Error` 21.05 test_vm_life_cycle.py
> test_01_migrate_vm_strict_tags_success `Error` 61.50 test_vm_strict_host_tags.py
@abh1sar can you check these test failures
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3139118850
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14488
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3048885485
Packaging result [SF]: ✔️ el8 ✔️ el9 ✖️ debian ✔️ suse15. SL-JID 14095
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "GutoVeronezi (via GitHub)" <gi...@apache.org>.
GutoVeronezi commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3048614823
@abh1sar, your PR started as a feature to create a new instance from a VM backup, and now it has several changes not related to it. The PR has more than 10k lines and misses a proper documentation/specification, which makes it difficult for anyone to review and understand what is going on, and make it more error prone. Please, separate the contexts in different PRs and add proper documentation/specification of workflows being changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3132936624
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3131844998
@JoaoJandre , we would like to have this in 4.21. Can you please point out if there are any blockers?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2800703539
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2852203581
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2965518950
@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2970845772
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13765
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3048336270
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3068415533
@GutoVeronezi Thanks for your interest. I have created a detailed spec for the feature on cwiki [Create Instance from Backup](https://shapeblue.atlassian.net/jira/software/c/projects/FRO/boards/34?selectedIssue=FRO-3835). Hope it helps.
The majority of the changes in this PR belong to the feature as described in the spec above. But, I have also included a few minor enhancements related to the Backup and Recovery framework (listed in the PR description).
While I agree that ideally these should be done as a separate PR, but doing so at this point of time would require significant bandwidth, both on my part and for reviewers who have already reviewed and tested the PR.
These enhancements along with the core feature add a lot of value to the framework as well as to the NAS backup provider, which is currently the only backup solution we have for KVM. With the community's help, I am hoping to get this included in the next release.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2795191555
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2876713013
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "JoaoJandre (via GitHub)" <gi...@apache.org>.
JoaoJandre commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3114370250
> > > @JoaoJandre are you good with the changes here?
> >
> >
> > I'm still not done reviewing. I also intend on testing this after I'm done reviewing.
>
> @JoaoJandre would you see a chance to finish tests in the next 1-2 days?
@sureshanaparti considering the huge size and scope of this PR, I will not be able to finish testing this so soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3113332850
> > @JoaoJandre are you good with the changes here?
>
> I'm still not done reviewing. I also intend on testing this after I'm done reviewing.
@JoaoJandre would you see a chance to finish tests in the next 1-2 days?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2230046913
##########
server/src/main/java/org/apache/cloudstack/backup/BackupManagerImpl.java:
##########
@@ -926,6 +1058,224 @@ private Backup.VolumeInfo getVolumeInfo(List<Backup.VolumeInfo> backedUpVolumes,
return null;
}
+ @Override
+ public void checkVmDisksSizeAgainstBackup(List<VmDiskInfo> vmDiskInfoList, Backup backup) {
+ List<VmDiskInfo> vmDiskInfoListFromBackup = getDataDiskInfoListFromBackup(backup);
+ int index = 0;
+ if (vmDiskInfoList.size() != vmDiskInfoListFromBackup.size()) {
+ throw new InvalidParameterValueException("Unable to create Instance from Backup " +
+ "as the backup has a different number of disks than the Instance.");
+ }
+ for (VmDiskInfo vmDiskInfo : vmDiskInfoList) {
+ if (index < vmDiskInfoListFromBackup.size()) {
+ if (vmDiskInfo.getSize() < vmDiskInfoListFromBackup.get(index).getSize()) {
+ throw new InvalidParameterValueException(
+ String.format("Instance volume size %d[GiB] cannot be less than the backed-up volume size %d[GiB].",
+ vmDiskInfo.getSize(), vmDiskInfoListFromBackup.get(index).getSize()));
+ }
+ }
+ index++;
+ }
+ }
+
+ @Override
+ public VmDiskInfo getRootDiskInfoFromBackup(Backup backup) {
+ List<Backup.VolumeInfo> volumes = backup.getBackedUpVolumes();
+ VmDiskInfo rootDiskOffering = null;
+ if (volumes == null || volumes.isEmpty()) {
+ throw new CloudRuntimeException("Failed to get backed-up volumes info from backup");
+ }
+ for (Backup.VolumeInfo volume : volumes) {
+ if (volume.getType() == Volume.Type.ROOT) {
+ DiskOfferingVO diskOffering = diskOfferingDao.findByUuid(volume.getDiskOfferingId());
+ if (diskOffering == null) {
+ throw new CloudRuntimeException(String.format("Unable to find the root disk offering with uuid (%s) " +
+ "stored in backup. Please specify a valid root disk offering id while creating the instance",
+ volume.getDiskOfferingId()));
+ }
+ Long size = volume.getSize() / (1024 * 1024 * 1024);
+ rootDiskOffering = new VmDiskInfo(diskOffering, size, volume.getMinIops(), volume.getMaxIops());
+ }
+ }
+ if (rootDiskOffering == null) {
+ throw new CloudRuntimeException("Failed to get the root disk in backed-up volumes info from backup");
+ }
+ return rootDiskOffering;
+ }
+
+ @Override
+ public List<VmDiskInfo> getDataDiskInfoListFromBackup(Backup backup) {
+ List<VmDiskInfo> vmDiskInfoList = new ArrayList<>();
+ List<Backup.VolumeInfo> volumes = backup.getBackedUpVolumes();
+ if (volumes == null || volumes.isEmpty()) {
+ throw new CloudRuntimeException("Failed to get backed-up Volumes info from backup");
+ }
+ for (Backup.VolumeInfo volume : volumes) {
+ if (volume.getType() == Volume.Type.DATADISK) {
+ DiskOfferingVO diskOffering = diskOfferingDao.findByUuid(volume.getDiskOfferingId());
+ if (diskOffering == null || diskOffering.getState().equals(DiskOffering.State.Inactive)) {
+ throw new CloudRuntimeException("Unable to find the disk offering with uuid (" + volume.getDiskOfferingId() + ") stored in backup. " +
+ "Please specify a valid disk offering id while creating the instance");
+ }
+ Long size = volume.getSize() / (1024 * 1024 * 1024);
+ vmDiskInfoList.add(new VmDiskInfo(diskOffering, size, volume.getMinIops(), volume.getMaxIops(), volume.getDeviceId()));
+ }
+ }
+ return vmDiskInfoList;
+ }
+
+ private List<String> parseAddressString(String input) {
+ if (input == null) return null;
+ return Arrays.stream(input.split(","))
+ .map(s -> "null".equalsIgnoreCase(s.trim()) ? null : s.trim())
+ .collect(Collectors.toList());
+ }
+
+ @Override
+ public Map<Long, Network.IpAddresses> getIpToNetworkMapFromBackup(Backup backup, boolean preserveIps, List<Long> networkIds)
+ {
+ Map<Long, Network.IpAddresses> ipToNetworkMap = new LinkedHashMap<Long, Network.IpAddresses>();
+
+ String nicsJson = backup.getDetail(ApiConstants.NICS);
+ if (nicsJson == null) {
+ throw new CloudRuntimeException("Backup doesn't contain network information. " +
+ "Please specify at least one valid network while creating instance");
+ }
+
+ Type type = new TypeToken<List<Map<String, String>>>(){}.getType();
+ List<Map<String, String>> nics = new Gson().fromJson(nicsJson, type);
+
+ for (Map<String, String> nic : nics) {
+ String networkUuid = nic.get(ApiConstants.NETWORK_ID);
+ if (networkUuid == null) {
+ throw new CloudRuntimeException("Backup doesn't contain network information. " +
+ "Please specify at least one valid network while creating instance");
+ }
+
+ Network network = networkDao.findByUuid(networkUuid);
+ if (network == null) {
+ throw new CloudRuntimeException("Unable to find network with the uuid " + networkUuid + " stored in backup. " +
+ "Please specify a valid network id while creating the instance");
+ }
+
+ Long networkId = network.getId();
+ Network.IpAddresses ipAddresses = null;
+
+ if (preserveIps) {
+ String ip = nic.get(ApiConstants.IP_ADDRESS);
+ String ipv6 = nic.get(ApiConstants.IP6_ADDRESS);
+ String mac = nic.get(ApiConstants.MAC_ADDRESS);
+ ipAddresses = networkService.getIpAddressesFromIps(ip, ipv6, mac);
+ }
+
+ ipToNetworkMap.put(networkId, ipAddresses);
+ networkIds.add(networkId);
+ }
+ return ipToNetworkMap;
+ }
+
+ @Override
+ public Boolean canCreateInstanceFromBackup(final Long backupId) {
+ final BackupVO backup = backupDao.findById(backupId);
+ BackupOffering offering = backupOfferingDao.findByIdIncludingRemoved(backup.getBackupOfferingId());
+ if (offering == null) {
+ throw new CloudRuntimeException("Failed to find backup offering");
+ }
+ final BackupProvider backupProvider = getBackupProvider(offering.getProvider());
+ return backupProvider.supportsInstanceFromBackup();
+ }
+
+ @Override
+ public boolean restoreBackupToVM(final Long backupId, final Long vmId) throws ResourceUnavailableException {
+ final BackupVO backup = backupDao.findById(backupId);
+ if (backup == null) {
+ throw new CloudRuntimeException("Backup " + backupId + " does not exist");
+ }
+ if (backup.getStatus() != Backup.Status.BackedUp) {
+ throw new CloudRuntimeException("Backup should be in BackedUp state");
+ }
+ validateBackupForZone(backup.getZoneId());
+
+ VMInstanceVO vm = vmInstanceDao.findByIdIncludingRemoved(vmId);
+ if (vm == null) {
+ throw new CloudRuntimeException("Instance with ID " + backup.getVmId() + " couldn't be found.");
+ }
+ accountManager.checkAccess(CallContext.current().getCallingAccount(), null, true, vm);
+
+ if (vm.getRemoved() != null) {
+ throw new CloudRuntimeException("Instance with ID " + backup.getVmId() + " couldn't be found.");
+ }
+ if (!vm.getState().equals(VirtualMachine.State.Stopped)) {
+ throw new CloudRuntimeException("The VM should be in stopped state");
+ }
+
+ List<Backup.VolumeInfo> backupVolumes = backup.getBackedUpVolumes();
+ if (backupVolumes == null) {
+ throw new CloudRuntimeException("Backed up volumes info not found in the backup");
+ }
+
+ List<VolumeVO> vmVolumes = volumeDao.findByInstance(vmId);
+ if (vmVolumes.size() != backupVolumes.size()) {
+ throw new CloudRuntimeException("Unable to create Instance from backup as the backup has a different number of disks than the Instance");
+ }
+
+ int index = 0;
+ for (VolumeVO vmVolume: vmVolumes) {
+ Backup.VolumeInfo backupVolume = backupVolumes.get(index);
+ if (vmVolume.getSize() < backupVolume.getSize()) {
+ throw new CloudRuntimeException(String.format(
+ "Instance volume size %d[GiB] for volume (%s) is less than the backed-up volume size %d[GiB] for backed-up volume (%s).",
+ vmVolume.getSize(), vmVolume.getUuid(), backupVolume.getSize(), backupVolume.getUuid()));
+ }
+ index++;
+ }
+
+ BackupOffering offering = backupOfferingDao.findByIdIncludingRemoved(backup.getBackupOfferingId());
+ if (offering == null) {
+ throw new CloudRuntimeException("Failed to find backup offering");
+ }
+ final BackupProvider backupProvider = getBackupProvider(offering.getProvider());
+ if (!backupProvider.supportsInstanceFromBackup()) {
+ throw new CloudRuntimeException("Create instance from backup is not supported by the " + offering.getProvider() + " provider.");
+ }
+
+ String backupDetailsInMessage = ReflectionToStringBuilderUtils.reflectOnlySelectedFields(backup, "uuid", "externalId", "newVMId", "type", "status", "date");
+ Long eventId = null;
+ try {
+ updateVmState(vm, VirtualMachine.Event.RestoringRequested, VirtualMachine.State.Restoring);
+ updateVolumeState(vm, Volume.Event.RestoreRequested, Volume.State.Restoring);
+ eventId = ActionEventUtils.onStartedActionEvent(User.UID_SYSTEM, vm.getAccountId(), EventTypes.EVENT_VM_CREATE_FROM_BACKUP,
+ String.format("Creating Instance %s from backup %s", vm.getInstanceName(), backup.getUuid()),
+ vm.getId(), ApiCommandResourceType.VirtualMachine.toString(),
+ true, 0);
+
+ String host = null;
+ String dataStore = null;
+ if (!"nas".equals(offering.getProvider())) {
+ Pair<HostVO, StoragePoolVO> restoreInfo = getRestoreVolumeHostAndDatastore(vm);
+ host = restoreInfo.first().getPrivateIpAddress();
+ dataStore = restoreInfo.second().getUuid();
+ }
+ if (!backupProvider.restoreBackupToVM(vm, backup, host, dataStore)) {
+ throw new CloudRuntimeException(String.format("Error restoring backup [%s] to VM %s.", backupDetailsInMessage, vm.getUuid()));
+ }
+ } catch (Exception e) {
+ updateVolumeState(vm, Volume.Event.RestoreFailed, Volume.State.Ready);
+ updateVmState(vm, VirtualMachine.Event.RestoringFailed, VirtualMachine.State.Stopped);
Review Comment:
yes, expunge is handled in restoreVMFromBackup
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3095936063
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2230043046
##########
engine/schema/src/main/resources/META-INF/db/schema-42010to42100.sql:
##########
@@ -241,3 +237,88 @@ ALTER TABLE `cloud`.`vm_instance_details` ADD CONSTRAINT `fk_vm_instance_details
CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backup_schedule', 'uuid', 'VARCHAR(40) NOT NULL');
UPDATE `cloud`.`backup_schedule` SET uuid = UUID();
+
+-- Add column max_backup to backup_schedule table
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backup_schedule', 'max_backups', 'int(8) default NULL COMMENT "maximum number of backups to maintain"');
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backup_schedule', 'quiescevm', 'tinyint(1) default NULL COMMENT "Quiesce VM before taking backup"');
+
+-- Add columns name, description and backup_interval_type to backup table
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backups', 'name', 'VARCHAR(255) NULL COMMENT "name of the backup"');
+UPDATE `cloud`.`backups` backup INNER JOIN `cloud`.`vm_instance` vm ON backup.vm_id = vm.id SET backup.name = vm.name;
+ALTER TABLE `cloud`.`backups` MODIFY COLUMN `name` VARCHAR(255) NOT NULL;
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backups', 'description', 'VARCHAR(1024) COMMENT "description for the backup"');
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backups', 'backup_interval_type', 'int(5) COMMENT "type of backup, e.g. manual, recurring - hourly, daily, weekly or monthly"');
+
+-- Create backup details table
+CREATE TABLE IF NOT EXISTS `cloud`.`backup_details` (
+ `id` bigint unsigned NOT NULL auto_increment,
+ `backup_id` bigint unsigned NOT NULL COMMENT 'backup id',
+ `name` varchar(255) NOT NULL,
+ `value` TEXT NOT NULL,
+ `display` tinyint(1) NOT NULL DEFAULT 1 COMMENT 'Should detail be displayed to the end user',
+ PRIMARY KEY (`id`),
+ CONSTRAINT `fk_backup_details__backup_id` FOREIGN KEY `fk_backup_details__backup_id`(`backup_id`) REFERENCES `backups`(`id`) ON DELETE CASCADE
+) ENGINE=InnoDB DEFAULT CHARSET=utf8;
+
+-- Add diskOfferingId, deviceId, minIops and maxIops to backed_volumes in backups table
+UPDATE `cloud`.`backups` b
+INNER JOIN `cloud`.`vm_instance` vm ON b.vm_id = vm.id
+SET b.backed_volumes = (
+ SELECT CONCAT("[",
+ GROUP_CONCAT(
+ CONCAT(
+ "{\"uuid\":\"", v.uuid, "\",",
+ "\"type\":\"", v.volume_type, "\",",
+ "\"size\":", v.`size`, ",",
+ "\"path\":\"", IFNULL(v.path, 'null'), "\",",
+ "\"deviceId\":", IFNULL(v.device_id, 'null'), ",",
+ "\"diskOfferingId\":\"", doff.uuid, "\",",
+ "\"minIops\":", IFNULL(v.min_iops, 'null'), ",",
+ "\"maxIops\":", IFNULL(v.max_iops, 'null'),
+ "}"
+ )
+ SEPARATOR ","
+ ),
+ "]")
+ FROM `cloud`.`volumes` v
+ LEFT JOIN `cloud`.`disk_offering` doff ON v.disk_offering_id = doff.id
+ WHERE v.instance_id = vm.id
+);
+
+-- Add diskOfferingId, deviceId, minIops and maxIops to backup_volumes in vm_instance table
+UPDATE `cloud`.`vm_instance` vm
+SET vm.backup_volumes = (
+ SELECT CONCAT("[",
Review Comment:
As far as I understand vm.backup_volumes is a legacy thing added in #3553 . backup.backed_volumes was added later in #9451 . Backups from older versions will only have vm.backup_volumes, so that needs to be handled in code.
I guess with some code changes and upgrade handling vm.backup_volumes can be done away with, but that would need some more consideration and testing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3125816950
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3121644235
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3121755984
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14395
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3123951955
<b>[SF] Trillian test result (tid-13918)</b>
Environment: vmware-70u3 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 55751 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10140-t13918-vmware-70u3.zip
Smoke tests completed. 142 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_01_vpn_usage | `Error` | 1.13 | test_usage.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3102270029
@abh1sar a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3102263076
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2225026739
##########
api/src/main/java/org/apache/cloudstack/api/command/admin/vm/CreateVMFromBackupCmdByAdmin.java:
##########
@@ -0,0 +1,55 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.api.command.admin.vm;
+
+import org.apache.cloudstack.acl.RoleType;
+import org.apache.cloudstack.api.APICommand;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.api.Parameter;
+import org.apache.cloudstack.api.ResponseObject;
+import org.apache.cloudstack.api.command.admin.AdminCmd;
+import org.apache.cloudstack.api.command.user.vm.CreateVMFromBackupCmd;
+import org.apache.cloudstack.api.response.ClusterResponse;
+import org.apache.cloudstack.api.response.PodResponse;
+import org.apache.cloudstack.api.response.UserVmResponse;
+
+import com.cloud.vm.VirtualMachine;
+
+@APICommand(name = "createVMFromBackup",
+ description = "Creates and automatically starts a VM from a backup.",
+ responseObject = UserVmResponse.class,
+ responseView = ResponseObject.ResponseView.Full,
+ entityType = {VirtualMachine.class},
+ requestHasSensitiveInfo = false, responseHasSensitiveInfo = true,
+ since = "4.21.0",
+ authorized = {RoleType.Admin, RoleType.ResourceAdmin, RoleType.DomainAdmin, RoleType.User})
+public class CreateVMFromBackupCmdByAdmin extends CreateVMFromBackupCmd implements AdminCmd {
+
+ @Parameter(name = ApiConstants.POD_ID, type = CommandType.UUID, entityType = PodResponse.class, description = "destination Pod ID to deploy the VM to - parameter available for root admin only", since = "4.13")
Review Comment:
fixed by commit https://github.com/apache/cloudstack/pull/10140/commits/a3f7bfa68d12ff290332fd110c25a1bf7506b550
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2225056032
##########
api/src/main/java/org/apache/cloudstack/backup/BackupProvider.java:
##########
@@ -110,9 +111,28 @@ public interface BackupProvider {
/**
* This method should TODO
+ *
* @param restorePoint the restore point to create a backup for
- * @param vm The machine for which to create a backup
- * @param metric the metric object to update with the new backup data
+ * @param vm The machine for which to create a backup
+ */
+ Backup createNewBackupEntryForRestorePoint(Backup.RestorePoint restorePoint, VirtualMachine vm);
+
+ /**
+ * Returns if the backup provider supports creating new instance from backup
*/
- Backup createNewBackupEntryForRestorePoint(Backup.RestorePoint restorePoint, VirtualMachine vm, Backup.Metric metric);
+ boolean supportsInstanceFromBackup();
+
+ /**
+ * Returns the backup storage usage (Used, Total) for a backup provider
+ * @param zoneId the zone for which to return metrics
+ * @return a pair of Used size and Total size for the backup storage
+ */
+ Pair<Long, Long> getBackupStorageStats(Long zoneId);
Review Comment:
@JoaoJandre This is not related to create instance from backup, but backup in general.
This PR contains lots of generic backup improvements.
In hindsight, I would have raised separate PRs, but it will be very difficult to break the PR now.
Even if the commits can be separated, they still are changing similar set of files and separating them will require lot of merging and unmerging, which is bound to introduce bugs.
I am sorry. This PR is quite big and difficult to review. I'll take this as a lesson going forward.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [WIP] Create new Instance from VM backup [cloudstack]
Posted by "leo79901 (via GitHub)" <gi...@apache.org>.
leo79901 commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2566182452
Yes, we need this !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [WIP] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2581715083
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [WIP] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2581806484
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12039
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r1918189120
##########
server/src/main/java/org/apache/cloudstack/backup/BackupManagerImpl.java:
##########
@@ -273,10 +292,62 @@ public boolean deleteBackupOffering(final Long offeringId) {
throw new CloudRuntimeException("Backup offering is assigned to VMs, remove the assignment(s) in order to remove the offering.");
}
- validateForZone(offering.getZoneId());
+ validateBackupForZone(offering.getZoneId());
return backupOfferingDao.remove(offering.getId());
}
+ @Override
+ public Map<String, String> getVmDetailsForBackup(VirtualMachine vm) {
+ HashMap<String, String> details = new HashMap<>();
+ details.put(ApiConstants.HYPERVISOR, vm.getHypervisorType().toString());
+ ServiceOffering serviceOffering = entityManager.findById(ServiceOffering.class, vm.getServiceOfferingId());
Review Comment:
No, will use the respective Daos.
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/NetworkerBackupProvider.java:
##########
@@ -117,6 +122,12 @@ public class NetworkerBackupProvider extends AdapterBase implements BackupProvid
@Inject
private VMInstanceDao vmInstanceDao;
+ @Inject
+ private EntityManager entityManager;
Review Comment:
will do
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r1988920461
##########
engine/orchestration/src/main/java/com/cloud/vm/VirtualMachineManagerImpl.java:
##########
@@ -2345,6 +2349,17 @@ public void doInTransactionWithoutResult(final TransactionStatus status) throws
throw new CloudRuntimeException("Unable to destroy " + vm);
} else {
if (expunge) {
+ if (vm.getBackupOfferingId() != null) {
+ List<Backup> backupsForVm = backupDao.listByVmId(vm.getDataCenterId(), vm.getId());
+ if (CollectionUtils.isEmpty(backupsForVm)) {
+ backupManager.removeVMFromBackupOffering(vm.getId(), true);
+ } else {
+ throw new CloudRuntimeException(String.format("This VM [uuid: %s, name: %s] has a "
+ + "Backup Offering [id: %s, external id: %s] with %s backups. Please, remove the backup offering "
+ + "before proceeding to VM exclusion!", vm.getUuid(), vm.getInstanceName(), vm.getBackupOfferingId(),
+ vm.getBackupExternalId(), backupsForVm.size()));
+ }
+ }
Review Comment:
extract method?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3028038572
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13985
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3134861462
@blueorangutan ui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3139048190
> > > * Tried to create VM from backup and delete the backup before the process ends, the deletion was allowed but the VM was not restored. I think that these processes should be either blocked from executing at the same time, or be queued.
> >
> >
> > The processes are queued once they reach the agent. i.e, if the new instance is in Restoring state then backup is deleted, the agent will process the restore first and then backup delete. If the new instance is still being created/starting/stopping before restore, and the backup is deleted, delete is not blocked and restore will throw an error that backup is deleted. IMO this is not a must have, and I'll be happy to fix this as a future improvement.
>
> Could we map it as an issue then?
Created issue https://github.com/apache/cloudstack/issues/11356
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [WIP] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2573096703
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 11996
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [WIP] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2585672274
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2824529413
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13152
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2893862478
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13461
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2915790787
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13541
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3100932555
<b>[SF] Trillian test result (tid-13834)</b>
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 48663 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10140-t13834-kvm-ol8.zip
Smoke tests completed. 142 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_attach_and_distribute_multiple_volumes | `Error` | 27.13 | test_attach_multiple_volumes.py
test_attach_multiple_volumes | `Failure` | 3.42 | test_attach_multiple_volumes.py
test_attach_multiple_volumes | `Error` | 3.42 | test_attach_multiple_volumes.py
ContextSuite context=TestMultipleVolumeAttach>:teardown | `Error` | 4.53 | test_attach_multiple_volumes.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3097012771
@sureshanaparti a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3086545472
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3086543174
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3086816446
Packaging result [SF]: ✔️ el8 ✔️ el9 ✖️ debian ✔️ suse15. SL-JID 14247
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "JoaoJandre (via GitHub)" <gi...@apache.org>.
JoaoJandre commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2225869526
##########
engine/schema/src/main/java/org/apache/cloudstack/backup/BackupVO.java:
##########
@@ -240,10 +268,32 @@ public void setBackedUpVolumes(String backedUpVolumes) {
this.backedUpVolumes = backedUpVolumes;
}
+ @Override
+ public Map<String, String> getDetails() {
+ return details;
+ }
+
+ public void setDetail(String name, String value) {
+ assert (details != null) : "Did you forget to load the details?";
Review Comment:
@abh1sar Two wrongs do not make a right. It is not advisable to use asserts in production code, these can be disabled when running the process, for example.
I also think it would be better to throw a CloudRuntimeException instead of an AssertionError if this condition is true.
##########
engine/schema/src/main/resources/META-INF/db/schema-42010to42100.sql:
##########
@@ -203,3 +199,91 @@ SET `sort_key` = CASE
ELSE `sort_key`
END;
-- End: Changes for Guest OS category cleanup
+
+-- Add column max_backup to backup_schedule table
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backup_schedule', 'max_backups', 'int(8) default NULL COMMENT "maximum number of backups to maintain"');
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backup_schedule', 'quiescevm', 'tinyint(1) default NULL COMMENT "Quiesce VM before taking backup"');
+
+-- Add columns name, description and backup_interval_type to backup table
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backups', 'name', 'VARCHAR(255) NULL COMMENT "name of the backup"');
+UPDATE `cloud`.`backups` backup INNER JOIN `cloud`.`vm_instance` vm ON backup.vm_id = vm.id SET backup.name = vm.name;
+ALTER TABLE `cloud`.`backups` MODIFY COLUMN `name` VARCHAR(255) NOT NULL;
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backups', 'description', 'VARCHAR(1024) COMMENT "description for the backup"');
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backups', 'backup_interval_type', 'int(5) COMMENT "type of backup, e.g. manual, recurring - hourly, daily, weekly or monthly"');
+
+-- Make the column vm_id in backups table nullable to handle orphan backups
+ALTER TABLE `cloud`.`backups` MODIFY COLUMN `vm_id` BIGINT UNSIGNED NULL;
Review Comment:
After a VM is expunged, its record is still on the DB, it just set as removed. I think it would be better to keep the record to the VM.
##########
engine/schema/src/main/java/org/apache/cloudstack/backup/dao/BackupDaoImpl.java:
##########
@@ -193,27 +260,61 @@ public List<BackupVO> listBackupsByVMandIntervalType(Long vmId, Backup.Type back
}
@Override
- public BackupResponse newBackupResponse(Backup backup) {
+ public void loadDetails(BackupVO backup) {
+ Map<String, String> details = backupDetailsDao.listDetailsKeyPairs(backup.getId());
+ backup.setDetails(details);
+ }
+
+ @Override
+ public void saveDetails(BackupVO backup) {
+ Map<String, String> detailsStr = backup.getDetails();
+ if (detailsStr == null) {
+ return;
+ }
+ List<BackupDetailVO> details = new ArrayList<BackupDetailVO>();
+ for (String key : detailsStr.keySet()) {
+ BackupDetailVO detail = new BackupDetailVO(backup.getId(), key, detailsStr.get(key), true);
+ details.add(detail);
+ }
+ backupDetailsDao.saveDetails(details);
+ }
+
+ @Override
+ public List<Long> listVmIdsWithBackupsInZone(Long zoneId) {
+ SearchCriteria<Long> sc = backupVmSearchInZone.create();
+ sc.setParameters("zone_id", zoneId);
+ return customSearchIncludingRemoved(sc, null);
+ }
+
+ @Override
+ public BackupResponse newBackupResponse(Backup backup, Boolean listVmDetails) {
VMInstanceVO vm = vmInstanceDao.findByIdIncludingRemoved(backup.getVmId());
- AccountVO account = accountDao.findByIdIncludingRemoved(vm.getAccountId());
- DomainVO domain = domainDao.findByIdIncludingRemoved(vm.getDomainId());
- DataCenterVO zone = dataCenterDao.findByIdIncludingRemoved(vm.getDataCenterId());
+ AccountVO account = accountDao.findByIdIncludingRemoved(backup.getAccountId());
+ DomainVO domain = domainDao.findByIdIncludingRemoved(backup.getDomainId());
+ DataCenterVO zone = dataCenterDao.findByIdIncludingRemoved(backup.getZoneId());
Long offeringId = backup.getBackupOfferingId();
- if (offeringId == null) {
- offeringId = vm.getBackupOfferingId();
- }
BackupOffering offering = backupOfferingDao.findByIdIncludingRemoved(offeringId);
BackupResponse response = new BackupResponse();
response.setId(backup.getUuid());
- response.setVmId(vm.getUuid());
+ response.setName(backup.getName());
+ response.setDescription(backup.getDescription());
response.setVmName(vm.getHostName());
+ if (vm.getState() != VirtualMachine.State.Expunging) {
Review Comment:
Why only not in expunging?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "JoaoJandre (via GitHub)" <gi...@apache.org>.
JoaoJandre commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2207205369
##########
engine/schema/src/main/resources/META-INF/db/schema-42010to42100.sql:
##########
@@ -203,3 +199,91 @@ SET `sort_key` = CASE
ELSE `sort_key`
END;
-- End: Changes for Guest OS category cleanup
+
+-- Add column max_backup to backup_schedule table
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backup_schedule', 'max_backups', 'int(8) default NULL COMMENT "maximum number of backups to maintain"');
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backup_schedule', 'quiescevm', 'tinyint(1) default NULL COMMENT "Quiesce VM before taking backup"');
+
+-- Add columns name, description and backup_interval_type to backup table
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backups', 'name', 'VARCHAR(255) NULL COMMENT "name of the backup"');
+UPDATE `cloud`.`backups` backup INNER JOIN `cloud`.`vm_instance` vm ON backup.vm_id = vm.id SET backup.name = vm.name;
+ALTER TABLE `cloud`.`backups` MODIFY COLUMN `name` VARCHAR(255) NOT NULL;
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backups', 'description', 'VARCHAR(1024) COMMENT "description for the backup"');
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backups', 'backup_interval_type', 'int(5) COMMENT "type of backup, e.g. manual, recurring - hourly, daily, weekly or monthly"');
+
+-- Make the column vm_id in backups table nullable to handle orphan backups
+ALTER TABLE `cloud`.`backups` MODIFY COLUMN `vm_id` BIGINT UNSIGNED NULL;
Review Comment:
After a VM is expunged, its record is still on the DB, it is just set as removed. I think it would be better to keep the record to the VM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2225036260
##########
api/src/main/java/com/cloud/offering/DiskOfferingInfo.java:
##########
@@ -23,6 +23,7 @@ public class DiskOfferingInfo {
private Long _size;
private Long _minIops;
private Long _maxIops;
+ private Long _deviceId;
Review Comment:
You are right. When creating instance from backup we have to replicate the deviceIds as stored in the backup. So I was using DiskOfferingInfo to pass all the information regarding the volumes to the orchestrator.
I am using a new class VmDiskInfo now, which makes more sense.
added in commit https://github.com/apache/cloudstack/pull/10140/commits/fef53cfb7520f962671ffaaf9834f512d1624c82
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "GutoVeronezi (via GitHub)" <gi...@apache.org>.
GutoVeronezi commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3070584746
> I have created a detailed spec for the feature on cwiki [Create Instance from Backup](https://shapeblue.atlassian.net/jira/software/c/projects/FRO/boards/34?selectedIssue=FRO-3835). Hope it helps.
The link is not public.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "JoaoJandre (via GitHub)" <gi...@apache.org>.
JoaoJandre commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2228507025
##########
server/src/main/java/com/cloud/server/ManagementServerImpl.java:
##########
@@ -1036,6 +1037,8 @@ public class ManagementServerImpl extends ManagerBase implements ManagementServe
@Inject
StoragePoolTagsDao storagePoolTagsDao;
@Inject
+ BackupManager backupManager;
Review Comment:
```suggestion
private BackupManager backupManager;
```
##########
server/src/main/java/com/cloud/vm/UserVmManagerImpl.java:
##########
@@ -9317,6 +9391,234 @@ public boolean unmanageUserVM(Long vmId) {
return true;
}
+ private void updateDetailsWithRootDiskAttributes(Map<String, String> details, VmDiskInfo rootVmDiskInfo) {
+ details.put(VmDetailConstants.ROOT_DISK_SIZE, rootVmDiskInfo.getSize().toString());
+ if (rootVmDiskInfo.getMinIops() != null) {
+ details.put(MIN_IOPS, rootVmDiskInfo.getMinIops().toString());
+ }
+ if (rootVmDiskInfo.getMaxIops() != null) {
+ details.put(MAX_IOPS, rootVmDiskInfo.getMaxIops().toString());
+ }
+ }
+
+ private void checkRootDiskSizeAgainstBackup(Long instanceVolumeSize,DiskOffering rootDiskOffering, Long backupVolumeSize) {
+ Long instanceRootDiskSize = rootDiskOffering.isCustomized() ? instanceVolumeSize : rootDiskOffering.getDiskSize() / GiB_TO_BYTES;
+ if (instanceRootDiskSize < backupVolumeSize) {
+ throw new InvalidParameterValueException(
+ String.format("Instance volume root disk size %d[GiB] cannot be less than the backed-up volume size %d[GiB].",
+ instanceVolumeSize, backupVolumeSize));
+ }
+ }
+
+ @Override
+ public UserVm allocateVMFromBackup(CreateVMFromBackupCmd cmd) throws InsufficientCapacityException, ResourceAllocationException, ResourceUnavailableException {
+ if (!backupManager.canCreateInstanceFromBackup(cmd.getBackupId())) {
+ throw new CloudRuntimeException("Create instance from backup is not supported for this provider.");
+ }
+ DataCenter zone = _dcDao.findById(cmd.getZoneId());
+ if (zone == null) {
+ throw new InvalidParameterValueException("Unable to find zone by id=" + cmd.getZoneId());
+ }
+
+ BackupVO backup = backupDao.findById(cmd.getBackupId());
+ if (backup == null) {
+ throw new InvalidParameterValueException("Backup " + cmd.getBackupId() + " does not exist");
+ }
+ if (backup.getZoneId() != cmd.getZoneId()) {
+ throw new InvalidParameterValueException("Instance should be created in the same zone as the backup");
+ }
+ backupManager.validateBackupForZone(backup.getZoneId());
+ backupDao.loadDetails(backup);
+
+ verifyDetails(cmd.getDetails());
+
+ UserVmVO backupVm = _vmDao.findByIdIncludingRemoved(backup.getVmId());
+ HypervisorType hypervisorType = backupVm.getHypervisorType();
+
+ Long serviceOfferingId = cmd.getServiceOfferingId();
+ ServiceOffering serviceOffering;
+ if (serviceOfferingId != null) {
+ serviceOffering = serviceOfferingDao.findById(serviceOfferingId);
+ if (serviceOffering == null) {
+ throw new InvalidParameterValueException("Unable to find service offering: " + serviceOffering.getId());
+ }
+ } else {
+ String serviceOfferingUuid = backup.getDetail(ApiConstants.SERVICE_OFFERING_ID);
+ if (serviceOfferingUuid == null) {
+ throw new CloudRuntimeException("Backup doesn't contain service offering uuid. Please specify a valid service offering id while creating the instance");
+ }
+ serviceOffering = serviceOfferingDao.findByUuid(serviceOfferingUuid);
+ if (serviceOffering == null) {
+ throw new CloudRuntimeException("Unable to find service offering with the uuid stored in backup. Please specify a valid service offering id while creating instance");
+ }
+ }
+ verifyServiceOffering(cmd, serviceOffering);
+
+ VirtualMachineTemplate template;
+ if (cmd.getTemplateId() != null) {
+ Long templateId = cmd.getTemplateId();
+ template = _templateDao.findById(templateId);
+ if (template == null) {
+ throw new InvalidParameterValueException("Unable to use template " + templateId);
+ }
+ } else {
+ String templateUuid = backup.getDetail(ApiConstants.TEMPLATE_ID);
+ if (templateUuid == null) {
+ throw new CloudRuntimeException("Backup doesn't contain Template uuid. Please specify a valid Template/ISO while creating the instance");
+ }
+ template = _templateDao.findByUuid(templateUuid);
+ if (template == null) {
+ throw new CloudRuntimeException("Unable to find template associated with the backup. Please specify a valid Template/ISO while creating instance");
+ }
+ }
+ verifyTemplate(cmd, template, serviceOffering.getId());
+
+ Long size = cmd.getSize();
+
+ Long diskOfferingId = cmd.getDiskOfferingId();
+ Boolean isIso = template.getFormat().equals(ImageFormat.ISO);
+ if (diskOfferingId != null) {
+ if (!isIso) {
+ throw new InvalidParameterValueException(ApiConstants.DISK_OFFERING_ID + " parameter is supported for creating instance from backup only for ISO. For creating VMs with templates, please use the parameter " + ApiConstants.DATADISKS_DETAILS);
+ }
+ DiskOffering diskOffering = _diskOfferingDao.findById(diskOfferingId);
+ if (diskOffering == null) {
+ throw new InvalidParameterValueException("Unable to find disk offering " + diskOfferingId);
+ }
+ if (diskOffering.isComputeOnly()) {
+ throw new InvalidParameterValueException(String.format("The disk offering %s provided is directly mapped to a service offering, please provide an individual disk offering", diskOffering));
+ }
+ }
+
+ Long overrideDiskOfferingId = cmd.getOverrideDiskOfferingId();
+
+ VmDiskInfo rootVmDiskInfoFromBackup = backupManager.getRootDiskInfoFromBackup(backup);
+
+ if (isIso) {
+ if (diskOfferingId == null) {
+ diskOfferingId = rootVmDiskInfoFromBackup.getDiskOffering().getId();
+ updateDetailsWithRootDiskAttributes(cmd.getDetails(), rootVmDiskInfoFromBackup);
+ size = rootVmDiskInfoFromBackup.getSize();
+ } else {
+ DiskOffering rootDiskOffering = _diskOfferingDao.findById(diskOfferingId);
+ checkRootDiskSizeAgainstBackup(size, rootDiskOffering, rootVmDiskInfoFromBackup.getSize());
+ }
+ } else {
+ if (overrideDiskOfferingId == null) {
+ overrideDiskOfferingId = serviceOffering.getDiskOfferingId();
+ updateDetailsWithRootDiskAttributes(cmd.getDetails(), rootVmDiskInfoFromBackup);
+ } else {
+ DiskOffering overrideDiskOffering = _diskOfferingDao.findById(overrideDiskOfferingId);
+ if (overrideDiskOffering.isComputeOnly()) {
+ updateDetailsWithRootDiskAttributes(cmd.getDetails(), rootVmDiskInfoFromBackup);
+ } else {
+ String diskSizeFromDetails = cmd.getDetails().get(VmDetailConstants.ROOT_DISK_SIZE);
+ Long rootDiskSize = diskSizeFromDetails == null ? null : Long.parseLong(diskSizeFromDetails);
+ checkRootDiskSizeAgainstBackup(rootDiskSize, overrideDiskOffering, rootVmDiskInfoFromBackup.getSize());
+ }
+ }
+ }
+
+ List<VmDiskInfo> dataDiskInfoList = cmd.getDataDiskInfoList();
+ if (dataDiskInfoList != null) {
+ backupManager.checkVmDisksSizeAgainstBackup(dataDiskInfoList, backup);
+ } else {
+ dataDiskInfoList = backupManager.getDataDiskInfoListFromBackup(backup);
+ }
+
+ List<Long> networkIds = cmd.getNetworkIds();
+ Account owner = _accountService.getActiveAccountById(cmd.getEntityOwnerId());
+ LinkedHashMap<Integer, Long> userVmNetworkMap = getVmOvfNetworkMapping(zone, owner, template, cmd.getVmNetworkMap());
+ if (MapUtils.isNotEmpty(userVmNetworkMap)) {
+ networkIds = new ArrayList<>(userVmNetworkMap.values());
+ }
+
+ Map<Long, IpAddresses> ipToNetworkMap = cmd.getIpToNetworkMap();
+ if (networkIds == null && ipToNetworkMap == null) {
+ networkIds = new ArrayList<Long>();
+ ipToNetworkMap = backupManager.getIpToNetworkMapFromBackup(backup, cmd.getPreserveIp(), networkIds);
+ }
+
+ UserVm vm = createVirtualMachine(cmd, zone, owner, serviceOffering, template, hypervisorType, diskOfferingId, size, overrideDiskOfferingId, dataDiskInfoList, networkIds, ipToNetworkMap, null, null);
+
+ String vmSettingsFromBackup = backup.getDetail(ApiConstants.VM_SETTINGS);
+ if (vm != null && vmSettingsFromBackup != null) {
+ UserVmVO vmVO = _vmDao.findById(vm.getId());
+ Map<String, String> details = vmInstanceDetailsDao.listDetailsKeyPairs(vm.getId());
+ vmVO.setDetails(details);
+
+ Type type = new TypeToken<Map<String, String>>(){}.getType();
+ Map<String, String> vmDetailsFromBackup = new Gson().fromJson(vmSettingsFromBackup, type);
+ for (Entry<String, String> entry : vmDetailsFromBackup.entrySet()) {
+ if (!details.containsKey(entry.getKey())) {
+ vmVO.setDetail(entry.getKey(), entry.getValue());
+ }
+ }
+ _vmDao.saveDetails(vmVO);
+ }
+
+ return vm;
+ }
+
+ @Override
+ public UserVm restoreVMFromBackup(CreateVMFromBackupCmd cmd) throws ResourceUnavailableException, InsufficientCapacityException, ResourceAllocationException {
+ long vmId = cmd.getEntityId();
+ Map<Long, DiskOffering> diskOfferingMap = cmd.getDataDiskTemplateToDiskOfferingMap();
+ Map<VirtualMachineProfile.Param, Object> additonalParams = new HashMap<>();
+ UserVm vm;
+
+ try {
+ vm = startVirtualMachine(vmId, null, null, null, diskOfferingMap, additonalParams, null);
+
+ boolean status = stopVirtualMachine(CallContext.current().getCallingUserId(), vm.getId()) ;
Review Comment:
Why start then stop the VM?
##########
engine/schema/src/main/java/org/apache/cloudstack/backup/dao/BackupDaoImpl.java:
##########
@@ -193,27 +272,97 @@ public List<BackupVO> listBackupsByVMandIntervalType(Long vmId, Backup.Type back
}
@Override
- public BackupResponse newBackupResponse(Backup backup) {
+ public void loadDetails(BackupVO backup) {
+ Map<String, String> details = backupDetailsDao.listDetailsKeyPairs(backup.getId());
+ backup.setDetails(details);
+ }
+
+ @Override
+ public void saveDetails(BackupVO backup) {
+ Map<String, String> detailsStr = backup.getDetails();
+ if (detailsStr == null) {
+ return;
+ }
+ List<BackupDetailVO> details = new ArrayList<BackupDetailVO>();
+ for (String key : detailsStr.keySet()) {
+ BackupDetailVO detail = new BackupDetailVO(backup.getId(), key, detailsStr.get(key), true);
+ details.add(detail);
+ }
+ backupDetailsDao.saveDetails(details);
+ }
+
+ @Override
+ public List<Long> listVmIdsWithBackupsInZone(Long zoneId) {
+ SearchCriteria<Long> sc = backupVmSearchInZone.create();
+ sc.setParameters("zone_id", zoneId);
+ return customSearchIncludingRemoved(sc, null);
+ }
+
+ Map<String, String> getDetailsFromBackupDetails(Long backupId) {
Review Comment:
Considering that this method does not do a single query in the `backups` table (instead querying multiple separate tables), don't you think it should not be somewhere else? I think that it should not be on the BackupDaoImpl.
##########
server/src/main/java/org/apache/cloudstack/backup/BackupManagerImpl.java:
##########
@@ -406,9 +501,12 @@ public boolean removeVMFromBackupOffering(final Long vmId, final boolean forced)
}
}
if ((result || forced) && vmInstanceDao.update(vm.getId(), vm)) {
- UsageEventUtils.publishUsageEvent(EventTypes.EVENT_VM_BACKUP_OFFERING_REMOVE, vm.getAccountId(), vm.getDataCenterId(), vm.getId(),
- "Backup-" + vm.getHostName() + "-" + vm.getUuid(), vm.getBackupOfferingId(), null, null,
- Backup.class.getSimpleName(), vm.getUuid());
+ final List<Backup> backups = backupDao.listByVmId(null, vm.getId());
+ if (backups.size() == 0) {
+ UsageEventUtils.publishUsageEvent(EventTypes.EVENT_VM_BACKUP_OFFERING_REMOVE, vm.getAccountId(), vm.getDataCenterId(), vm.getId(),
+ "Backup-" + vm.getHostName() + "-" + vm.getUuid(), backupOfferingId, null, null,
+ Backup.class.getSimpleName(), vm.getUuid());
+ }
Review Comment:
If this event is not published here, when will it be published?
##########
server/src/main/java/org/apache/cloudstack/backup/BackupManagerImpl.java:
##########
@@ -926,6 +1058,224 @@ private Backup.VolumeInfo getVolumeInfo(List<Backup.VolumeInfo> backedUpVolumes,
return null;
}
+ @Override
+ public void checkVmDisksSizeAgainstBackup(List<VmDiskInfo> vmDiskInfoList, Backup backup) {
+ List<VmDiskInfo> vmDiskInfoListFromBackup = getDataDiskInfoListFromBackup(backup);
+ int index = 0;
+ if (vmDiskInfoList.size() != vmDiskInfoListFromBackup.size()) {
+ throw new InvalidParameterValueException("Unable to create Instance from Backup " +
+ "as the backup has a different number of disks than the Instance.");
+ }
+ for (VmDiskInfo vmDiskInfo : vmDiskInfoList) {
+ if (index < vmDiskInfoListFromBackup.size()) {
+ if (vmDiskInfo.getSize() < vmDiskInfoListFromBackup.get(index).getSize()) {
+ throw new InvalidParameterValueException(
+ String.format("Instance volume size %d[GiB] cannot be less than the backed-up volume size %d[GiB].",
+ vmDiskInfo.getSize(), vmDiskInfoListFromBackup.get(index).getSize()));
+ }
+ }
+ index++;
+ }
+ }
+
+ @Override
+ public VmDiskInfo getRootDiskInfoFromBackup(Backup backup) {
+ List<Backup.VolumeInfo> volumes = backup.getBackedUpVolumes();
+ VmDiskInfo rootDiskOffering = null;
+ if (volumes == null || volumes.isEmpty()) {
+ throw new CloudRuntimeException("Failed to get backed-up volumes info from backup");
+ }
+ for (Backup.VolumeInfo volume : volumes) {
+ if (volume.getType() == Volume.Type.ROOT) {
+ DiskOfferingVO diskOffering = diskOfferingDao.findByUuid(volume.getDiskOfferingId());
+ if (diskOffering == null) {
+ throw new CloudRuntimeException(String.format("Unable to find the root disk offering with uuid (%s) " +
+ "stored in backup. Please specify a valid root disk offering id while creating the instance",
+ volume.getDiskOfferingId()));
+ }
+ Long size = volume.getSize() / (1024 * 1024 * 1024);
+ rootDiskOffering = new VmDiskInfo(diskOffering, size, volume.getMinIops(), volume.getMaxIops());
+ }
+ }
+ if (rootDiskOffering == null) {
+ throw new CloudRuntimeException("Failed to get the root disk in backed-up volumes info from backup");
+ }
+ return rootDiskOffering;
+ }
+
+ @Override
+ public List<VmDiskInfo> getDataDiskInfoListFromBackup(Backup backup) {
+ List<VmDiskInfo> vmDiskInfoList = new ArrayList<>();
+ List<Backup.VolumeInfo> volumes = backup.getBackedUpVolumes();
+ if (volumes == null || volumes.isEmpty()) {
+ throw new CloudRuntimeException("Failed to get backed-up Volumes info from backup");
+ }
+ for (Backup.VolumeInfo volume : volumes) {
+ if (volume.getType() == Volume.Type.DATADISK) {
+ DiskOfferingVO diskOffering = diskOfferingDao.findByUuid(volume.getDiskOfferingId());
+ if (diskOffering == null || diskOffering.getState().equals(DiskOffering.State.Inactive)) {
+ throw new CloudRuntimeException("Unable to find the disk offering with uuid (" + volume.getDiskOfferingId() + ") stored in backup. " +
+ "Please specify a valid disk offering id while creating the instance");
+ }
+ Long size = volume.getSize() / (1024 * 1024 * 1024);
+ vmDiskInfoList.add(new VmDiskInfo(diskOffering, size, volume.getMinIops(), volume.getMaxIops(), volume.getDeviceId()));
+ }
+ }
+ return vmDiskInfoList;
+ }
+
+ private List<String> parseAddressString(String input) {
+ if (input == null) return null;
+ return Arrays.stream(input.split(","))
+ .map(s -> "null".equalsIgnoreCase(s.trim()) ? null : s.trim())
+ .collect(Collectors.toList());
+ }
+
+ @Override
+ public Map<Long, Network.IpAddresses> getIpToNetworkMapFromBackup(Backup backup, boolean preserveIps, List<Long> networkIds)
+ {
+ Map<Long, Network.IpAddresses> ipToNetworkMap = new LinkedHashMap<Long, Network.IpAddresses>();
+
+ String nicsJson = backup.getDetail(ApiConstants.NICS);
+ if (nicsJson == null) {
+ throw new CloudRuntimeException("Backup doesn't contain network information. " +
+ "Please specify at least one valid network while creating instance");
+ }
+
+ Type type = new TypeToken<List<Map<String, String>>>(){}.getType();
+ List<Map<String, String>> nics = new Gson().fromJson(nicsJson, type);
+
+ for (Map<String, String> nic : nics) {
+ String networkUuid = nic.get(ApiConstants.NETWORK_ID);
+ if (networkUuid == null) {
+ throw new CloudRuntimeException("Backup doesn't contain network information. " +
+ "Please specify at least one valid network while creating instance");
+ }
+
+ Network network = networkDao.findByUuid(networkUuid);
+ if (network == null) {
+ throw new CloudRuntimeException("Unable to find network with the uuid " + networkUuid + " stored in backup. " +
+ "Please specify a valid network id while creating the instance");
+ }
+
+ Long networkId = network.getId();
+ Network.IpAddresses ipAddresses = null;
+
+ if (preserveIps) {
+ String ip = nic.get(ApiConstants.IP_ADDRESS);
+ String ipv6 = nic.get(ApiConstants.IP6_ADDRESS);
+ String mac = nic.get(ApiConstants.MAC_ADDRESS);
+ ipAddresses = networkService.getIpAddressesFromIps(ip, ipv6, mac);
+ }
+
+ ipToNetworkMap.put(networkId, ipAddresses);
+ networkIds.add(networkId);
+ }
+ return ipToNetworkMap;
+ }
+
+ @Override
+ public Boolean canCreateInstanceFromBackup(final Long backupId) {
+ final BackupVO backup = backupDao.findById(backupId);
+ BackupOffering offering = backupOfferingDao.findByIdIncludingRemoved(backup.getBackupOfferingId());
+ if (offering == null) {
+ throw new CloudRuntimeException("Failed to find backup offering");
+ }
+ final BackupProvider backupProvider = getBackupProvider(offering.getProvider());
+ return backupProvider.supportsInstanceFromBackup();
+ }
+
+ @Override
+ public boolean restoreBackupToVM(final Long backupId, final Long vmId) throws ResourceUnavailableException {
+ final BackupVO backup = backupDao.findById(backupId);
+ if (backup == null) {
+ throw new CloudRuntimeException("Backup " + backupId + " does not exist");
+ }
+ if (backup.getStatus() != Backup.Status.BackedUp) {
+ throw new CloudRuntimeException("Backup should be in BackedUp state");
+ }
+ validateBackupForZone(backup.getZoneId());
+
+ VMInstanceVO vm = vmInstanceDao.findByIdIncludingRemoved(vmId);
+ if (vm == null) {
+ throw new CloudRuntimeException("Instance with ID " + backup.getVmId() + " couldn't be found.");
+ }
+ accountManager.checkAccess(CallContext.current().getCallingAccount(), null, true, vm);
+
+ if (vm.getRemoved() != null) {
+ throw new CloudRuntimeException("Instance with ID " + backup.getVmId() + " couldn't be found.");
+ }
+ if (!vm.getState().equals(VirtualMachine.State.Stopped)) {
+ throw new CloudRuntimeException("The VM should be in stopped state");
+ }
+
+ List<Backup.VolumeInfo> backupVolumes = backup.getBackedUpVolumes();
+ if (backupVolumes == null) {
+ throw new CloudRuntimeException("Backed up volumes info not found in the backup");
+ }
+
+ List<VolumeVO> vmVolumes = volumeDao.findByInstance(vmId);
+ if (vmVolumes.size() != backupVolumes.size()) {
+ throw new CloudRuntimeException("Unable to create Instance from backup as the backup has a different number of disks than the Instance");
+ }
+
+ int index = 0;
+ for (VolumeVO vmVolume: vmVolumes) {
+ Backup.VolumeInfo backupVolume = backupVolumes.get(index);
+ if (vmVolume.getSize() < backupVolume.getSize()) {
+ throw new CloudRuntimeException(String.format(
+ "Instance volume size %d[GiB] for volume (%s) is less than the backed-up volume size %d[GiB] for backed-up volume (%s).",
+ vmVolume.getSize(), vmVolume.getUuid(), backupVolume.getSize(), backupVolume.getUuid()));
+ }
+ index++;
+ }
+
+ BackupOffering offering = backupOfferingDao.findByIdIncludingRemoved(backup.getBackupOfferingId());
+ if (offering == null) {
+ throw new CloudRuntimeException("Failed to find backup offering");
+ }
+ final BackupProvider backupProvider = getBackupProvider(offering.getProvider());
+ if (!backupProvider.supportsInstanceFromBackup()) {
+ throw new CloudRuntimeException("Create instance from backup is not supported by the " + offering.getProvider() + " provider.");
+ }
+
+ String backupDetailsInMessage = ReflectionToStringBuilderUtils.reflectOnlySelectedFields(backup, "uuid", "externalId", "newVMId", "type", "status", "date");
+ Long eventId = null;
+ try {
+ updateVmState(vm, VirtualMachine.Event.RestoringRequested, VirtualMachine.State.Restoring);
+ updateVolumeState(vm, Volume.Event.RestoreRequested, Volume.State.Restoring);
+ eventId = ActionEventUtils.onStartedActionEvent(User.UID_SYSTEM, vm.getAccountId(), EventTypes.EVENT_VM_CREATE_FROM_BACKUP,
+ String.format("Creating Instance %s from backup %s", vm.getInstanceName(), backup.getUuid()),
+ vm.getId(), ApiCommandResourceType.VirtualMachine.toString(),
+ true, 0);
+
+ String host = null;
+ String dataStore = null;
+ if (!"nas".equals(offering.getProvider())) {
+ Pair<HostVO, StoragePoolVO> restoreInfo = getRestoreVolumeHostAndDatastore(vm);
+ host = restoreInfo.first().getPrivateIpAddress();
+ dataStore = restoreInfo.second().getUuid();
+ }
+ if (!backupProvider.restoreBackupToVM(vm, backup, host, dataStore)) {
+ throw new CloudRuntimeException(String.format("Error restoring backup [%s] to VM %s.", backupDetailsInMessage, vm.getUuid()));
+ }
+ } catch (Exception e) {
+ updateVolumeState(vm, Volume.Event.RestoreFailed, Volume.State.Ready);
+ updateVmState(vm, VirtualMachine.Event.RestoringFailed, VirtualMachine.State.Stopped);
Review Comment:
If we are creating a VM from a backup, this is a new VM correct?
If the process failed, shouldn't the VM and the volumes go into an error state? If not, we might have a bunch of volumes that are marked as `Ready` but are inconsistent.
##########
server/src/main/java/org/apache/cloudstack/backup/BackupManagerImpl.java:
##########
@@ -282,18 +319,75 @@ public boolean deleteBackupOffering(final Long offeringId) {
throw new CloudRuntimeException("Could not find a backup offering with id: " + offeringId);
}
- if (vmInstanceDao.listByZoneWithBackups(offering.getZoneId(), offering.getId()).size() > 0) {
+ if (backupDao.listByOfferingId(offering.getId()).size() > 0) {
+ throw new CloudRuntimeException("Backup Offering cannot be removed as it has backups associated with it.");
+ }
+
+ if (vmInstanceDao.listByZoneAndBackupOffering(offering.getZoneId(), offering.getId()).size() > 0) {
throw new CloudRuntimeException("Backup offering is assigned to VMs, remove the assignment(s) in order to remove the offering.");
}
- validateForZone(offering.getZoneId());
+ validateBackupForZone(offering.getZoneId());
return backupOfferingDao.remove(offering.getId());
}
- public static String createVolumeInfoFromVolumes(List<VolumeVO> vmVolumes) {
+ @Override
+ public Map<String, String> getBackupDetailsFromVM(VirtualMachine vm) {
+ HashMap<String, String> details = new HashMap<>();
+
+ ServiceOffering serviceOffering = serviceOfferingDao.findById(vm.getServiceOfferingId());
+ details.put(ApiConstants.SERVICE_OFFERING_ID, serviceOffering.getUuid());
+ VirtualMachineTemplate template = vmTemplateDao.findById(vm.getTemplateId());
+ details.put(ApiConstants.TEMPLATE_ID, template.getUuid());
+
+ List<VMInstanceDetailVO> vmDetails = vmInstanceDetailsDao.listDetails(vm.getId());
+ HashMap<String, String> settings = new HashMap<>();
+ for (VMInstanceDetailVO detail : vmDetails) {
+ settings.put(detail.getName(), detail.getValue());
+ }
+ if (!settings.isEmpty()) {
+ details.put(ApiConstants.VM_SETTINGS, new Gson().toJson(settings));
+ }
+
+ List<UserVmJoinVO> userVmJoinVOs = userVmJoinDao.searchByIds(vm.getId());
+ if (userVmJoinVOs != null && !userVmJoinVOs.isEmpty()) {
+ List<Map<String, String>> nics = new ArrayList<>();
+ Set<String> seen = new HashSet<>();
+
+ for (UserVmJoinVO userVmJoinVO : userVmJoinVOs) {
+ Map<String, String> nicInfo = new HashMap<>();
+ String key = userVmJoinVO.getNetworkUuid();
+ if (seen.add(key)) {
+ nicInfo.put(ApiConstants.NETWORK_ID, userVmJoinVO.getNetworkUuid());
+ nicInfo.put(ApiConstants.IP_ADDRESS, userVmJoinVO.getIpAddress());
+ nicInfo.put(ApiConstants.IP6_ADDRESS, userVmJoinVO.getIp6Address());
+ nicInfo.put(ApiConstants.MAC_ADDRESS, userVmJoinVO.getMacAddress());
+ nics.add(nicInfo);
+ }
+ }
+
+ if (!nics.isEmpty()) {
+ String nicJson = new Gson().toJson(nics);
+ details.put("nics", nicJson);
+ }
Review Comment:
We could extract this to a method
##########
engine/schema/src/main/java/org/apache/cloudstack/backup/BackupVO.java:
##########
@@ -240,10 +268,27 @@ public void setBackedUpVolumes(String backedUpVolumes) {
this.backedUpVolumes = backedUpVolumes;
}
+ @Override
+ public Map<String, String> getDetails() {
+ return details;
+ }
+
+ @Override
+ public String getDetail(String name) {
+ return this.details.get(name);
+ }
+
+ public void setDetails(Map<String, String> details) {
+ this.details = details;
+ }
+
+ public void addDetails(Map<String, String> details) {
+ this.details.putAll(details);
+ }
Review Comment:
It seems like this is never used in the code
##########
server/src/main/java/org/apache/cloudstack/backup/BackupManagerImpl.java:
##########
@@ -599,10 +705,19 @@ public boolean createBackup(final Long vmId, final Long scheduleId) throws Resou
throw new CloudRuntimeException("VM backup offering not found");
}
+ final BackupProvider backupProvider = getBackupProvider(offering.getProvider());
+ if (offering == null) {
+ throw new CloudRuntimeException("VM backup provider not found for the offering");
+ }
Review Comment:
If offering is null, a NPE will be thrown in the previous line.
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/NetworkerBackupProvider.java:
##########
@@ -116,6 +121,15 @@ public class NetworkerBackupProvider extends AdapterBase implements BackupProvid
@Inject
private VMInstanceDao vmInstanceDao;
+ @Inject
+ private EntityManager entityManager;
+
+ @Inject
+ BackupManager backupManager;
Review Comment:
```suggestion
private BackupManager backupManager;
```
##########
engine/schema/src/main/resources/META-INF/db/schema-42010to42100.sql:
##########
@@ -241,3 +237,88 @@ ALTER TABLE `cloud`.`vm_instance_details` ADD CONSTRAINT `fk_vm_instance_details
CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backup_schedule', 'uuid', 'VARCHAR(40) NOT NULL');
UPDATE `cloud`.`backup_schedule` SET uuid = UUID();
+
+-- Add column max_backup to backup_schedule table
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backup_schedule', 'max_backups', 'int(8) default NULL COMMENT "maximum number of backups to maintain"');
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backup_schedule', 'quiescevm', 'tinyint(1) default NULL COMMENT "Quiesce VM before taking backup"');
+
+-- Add columns name, description and backup_interval_type to backup table
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backups', 'name', 'VARCHAR(255) NULL COMMENT "name of the backup"');
+UPDATE `cloud`.`backups` backup INNER JOIN `cloud`.`vm_instance` vm ON backup.vm_id = vm.id SET backup.name = vm.name;
+ALTER TABLE `cloud`.`backups` MODIFY COLUMN `name` VARCHAR(255) NOT NULL;
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backups', 'description', 'VARCHAR(1024) COMMENT "description for the backup"');
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backups', 'backup_interval_type', 'int(5) COMMENT "type of backup, e.g. manual, recurring - hourly, daily, weekly or monthly"');
+
+-- Create backup details table
+CREATE TABLE IF NOT EXISTS `cloud`.`backup_details` (
+ `id` bigint unsigned NOT NULL auto_increment,
+ `backup_id` bigint unsigned NOT NULL COMMENT 'backup id',
+ `name` varchar(255) NOT NULL,
+ `value` TEXT NOT NULL,
+ `display` tinyint(1) NOT NULL DEFAULT 1 COMMENT 'Should detail be displayed to the end user',
+ PRIMARY KEY (`id`),
+ CONSTRAINT `fk_backup_details__backup_id` FOREIGN KEY `fk_backup_details__backup_id`(`backup_id`) REFERENCES `backups`(`id`) ON DELETE CASCADE
+) ENGINE=InnoDB DEFAULT CHARSET=utf8;
+
+-- Add diskOfferingId, deviceId, minIops and maxIops to backed_volumes in backups table
+UPDATE `cloud`.`backups` b
+INNER JOIN `cloud`.`vm_instance` vm ON b.vm_id = vm.id
+SET b.backed_volumes = (
+ SELECT CONCAT("[",
+ GROUP_CONCAT(
+ CONCAT(
+ "{\"uuid\":\"", v.uuid, "\",",
+ "\"type\":\"", v.volume_type, "\",",
+ "\"size\":", v.`size`, ",",
+ "\"path\":\"", IFNULL(v.path, 'null'), "\",",
+ "\"deviceId\":", IFNULL(v.device_id, 'null'), ",",
+ "\"diskOfferingId\":\"", doff.uuid, "\",",
+ "\"minIops\":", IFNULL(v.min_iops, 'null'), ",",
+ "\"maxIops\":", IFNULL(v.max_iops, 'null'),
+ "}"
+ )
+ SEPARATOR ","
+ ),
+ "]")
+ FROM `cloud`.`volumes` v
+ LEFT JOIN `cloud`.`disk_offering` doff ON v.disk_offering_id = doff.id
+ WHERE v.instance_id = vm.id
+);
+
+-- Add diskOfferingId, deviceId, minIops and maxIops to backup_volumes in vm_instance table
+UPDATE `cloud`.`vm_instance` vm
+SET vm.backup_volumes = (
+ SELECT CONCAT("[",
Review Comment:
Why do we have the same info in two tables?
##########
server/src/main/java/org/apache/cloudstack/backup/BackupManagerImpl.java:
##########
@@ -1470,7 +1910,22 @@ private Backup checkAndUpdateIfBackupEntryExistsForRestorePoint(Backup.RestorePo
return null;
}
- private void syncBackups(BackupProvider backupProvider, VirtualMachine vm, Backup.Metric metric) {
+ private void processRemoveList(List<Long> removeList, VirtualMachine vm) {
+ for (final Long backupIdToRemove : removeList) {
+ logger.warn(String.format("Removing backup with ID: [%s].", backupIdToRemove));
+ Backup backup = backupDao.findById(backupIdToRemove);
+ resourceLimitMgr.decrementResourceCount(backup.getAccountId(), Resource.ResourceType.backup);
+ resourceLimitMgr.decrementResourceCount(backup.getAccountId(), Resource.ResourceType.backup_storage, backup.getSize());
+ boolean result = backupDao.remove(backupIdToRemove);
+ if (result) {
+ checkAndGenerateUsageForLastBackupDeletedAfterOfferingRemove(vm, backup);
+ } else {
+ logger.error("Failed to remove backup db wentry ith ID: {} during sync backups", backupIdToRemove);
Review Comment:
```suggestion
logger.error("Failed to remove backup db entry with ID: {} during sync backups", backupIdToRemove);
```
##########
server/src/main/java/org/apache/cloudstack/backup/BackupManagerImpl.java:
##########
@@ -926,6 +1058,224 @@ private Backup.VolumeInfo getVolumeInfo(List<Backup.VolumeInfo> backedUpVolumes,
return null;
}
+ @Override
+ public void checkVmDisksSizeAgainstBackup(List<VmDiskInfo> vmDiskInfoList, Backup backup) {
+ List<VmDiskInfo> vmDiskInfoListFromBackup = getDataDiskInfoListFromBackup(backup);
+ int index = 0;
+ if (vmDiskInfoList.size() != vmDiskInfoListFromBackup.size()) {
+ throw new InvalidParameterValueException("Unable to create Instance from Backup " +
+ "as the backup has a different number of disks than the Instance.");
+ }
+ for (VmDiskInfo vmDiskInfo : vmDiskInfoList) {
+ if (index < vmDiskInfoListFromBackup.size()) {
+ if (vmDiskInfo.getSize() < vmDiskInfoListFromBackup.get(index).getSize()) {
+ throw new InvalidParameterValueException(
+ String.format("Instance volume size %d[GiB] cannot be less than the backed-up volume size %d[GiB].",
+ vmDiskInfo.getSize(), vmDiskInfoListFromBackup.get(index).getSize()));
+ }
+ }
+ index++;
+ }
+ }
+
+ @Override
+ public VmDiskInfo getRootDiskInfoFromBackup(Backup backup) {
+ List<Backup.VolumeInfo> volumes = backup.getBackedUpVolumes();
+ VmDiskInfo rootDiskOffering = null;
+ if (volumes == null || volumes.isEmpty()) {
+ throw new CloudRuntimeException("Failed to get backed-up volumes info from backup");
+ }
+ for (Backup.VolumeInfo volume : volumes) {
+ if (volume.getType() == Volume.Type.ROOT) {
+ DiskOfferingVO diskOffering = diskOfferingDao.findByUuid(volume.getDiskOfferingId());
+ if (diskOffering == null) {
+ throw new CloudRuntimeException(String.format("Unable to find the root disk offering with uuid (%s) " +
+ "stored in backup. Please specify a valid root disk offering id while creating the instance",
+ volume.getDiskOfferingId()));
+ }
+ Long size = volume.getSize() / (1024 * 1024 * 1024);
+ rootDiskOffering = new VmDiskInfo(diskOffering, size, volume.getMinIops(), volume.getMaxIops());
+ }
+ }
+ if (rootDiskOffering == null) {
+ throw new CloudRuntimeException("Failed to get the root disk in backed-up volumes info from backup");
+ }
+ return rootDiskOffering;
+ }
+
+ @Override
+ public List<VmDiskInfo> getDataDiskInfoListFromBackup(Backup backup) {
+ List<VmDiskInfo> vmDiskInfoList = new ArrayList<>();
+ List<Backup.VolumeInfo> volumes = backup.getBackedUpVolumes();
+ if (volumes == null || volumes.isEmpty()) {
+ throw new CloudRuntimeException("Failed to get backed-up Volumes info from backup");
+ }
+ for (Backup.VolumeInfo volume : volumes) {
+ if (volume.getType() == Volume.Type.DATADISK) {
+ DiskOfferingVO diskOffering = diskOfferingDao.findByUuid(volume.getDiskOfferingId());
+ if (diskOffering == null || diskOffering.getState().equals(DiskOffering.State.Inactive)) {
+ throw new CloudRuntimeException("Unable to find the disk offering with uuid (" + volume.getDiskOfferingId() + ") stored in backup. " +
+ "Please specify a valid disk offering id while creating the instance");
+ }
+ Long size = volume.getSize() / (1024 * 1024 * 1024);
+ vmDiskInfoList.add(new VmDiskInfo(diskOffering, size, volume.getMinIops(), volume.getMaxIops(), volume.getDeviceId()));
+ }
+ }
+ return vmDiskInfoList;
+ }
+
+ private List<String> parseAddressString(String input) {
+ if (input == null) return null;
+ return Arrays.stream(input.split(","))
+ .map(s -> "null".equalsIgnoreCase(s.trim()) ? null : s.trim())
+ .collect(Collectors.toList());
+ }
Review Comment:
Is this used anywhere? Also, what is its purpose?
##########
server/src/main/java/com/cloud/storage/VolumeApiServiceImpl.java:
##########
@@ -1959,8 +1959,7 @@ private Pair<String, String> handleCheckAndRepairVolumeJob(Long vmId, Long volum
} else if (jobResult instanceof ResourceAllocationException) {
throw (ResourceAllocationException)jobResult;
} else if (jobResult instanceof Throwable) {
- Throwable throwable = (Throwable) jobResult;
- throw new RuntimeException(String.format("Unexpected exception: %s", throwable.getMessage()), throwable);
+ throw new RuntimeException("Unexpected exception", (Throwable) jobResult);
Review Comment:
CloudRuntimeException
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3121225102
<b>[SF] Trillian test result (tid-13896)</b>
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 57544 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10140-t13896-kvm-ol8.zip
Smoke tests completed. 141 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_attach_and_distribute_multiple_volumes | `Error` | 34.32 | test_attach_multiple_volumes.py
test_attach_multiple_volumes | `Failure` | 3.52 | test_attach_multiple_volumes.py
test_attach_multiple_volumes | `Error` | 3.52 | test_attach_multiple_volumes.py
ContextSuite context=TestMultipleVolumeAttach>:teardown | `Error` | 3.61 | test_attach_multiple_volumes.py
test_list_zones_metrics | `Error` | 0.22 | test_metrics_api.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3126197844
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14411
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "JoaoJandre (via GitHub)" <gi...@apache.org>.
JoaoJandre commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3132137375
> @JoaoJandre , we would like to have this in 4.21. Can you please point out if there are any blockers?
In my limited testing, I did not find any blockers.
However, I would like to have a warning for users that they cannot change de ISO or template when creating a VM from backup unless the ISO/Template from the original is removed; or maybe just hide this part of the form. But I would not consider this a blocker, just bad UX.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3070678483
> > I have created a detailed spec for the feature on cwiki [Create Instance from Backup](https://shapeblue.atlassian.net/jira/software/c/projects/FRO/boards/34?selectedIssue=FRO-3835). Hope it helps.
>
> The link is not public.
Sorry, pasted the wrong link.
here it is : https://cwiki.apache.org/confluence/display/CLOUDSTACK/Create+Instance+from+Backup
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3007103515
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3021249417
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
weizhouapache commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2808502266
overall lgtm
note about the `setDeviceId`
- for vm created from template, the data volume uses diskofferingId , so deviceId=1 is correct
- for vm created from ISO, the root volume uses diskofferingId , deviceId should be 0 I think.
- for vm created with multiple data disks, it uses `datadiskofferinglist`, but deviceId of volumes are not set in dataDiskOfferingInfo.
is `deviceId` important for this feature ? cc @abh1sar
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r1918117662
##########
api/src/main/java/org/apache/cloudstack/api/command/user/vm/DeployVMCmd.java:
##########
@@ -147,6 +148,13 @@ public class DeployVMCmd extends BaseAsyncCreateCustomIdCmd implements SecurityG
since = "4.4")
private Long rootdisksize;
+ @Parameter(name = ApiConstants.DATADISKS_DETAILS,
Review Comment:
@shwstppr I am not sure how we can input a list of datadiskdetails using the details Map.
Anyway, It seems better to me to keep the parameter separate like ipToNetworkList for easier usage and understanding (this could be used for deployvm in general to create instances with multiple volumes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2716411432
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r1988656990
##########
engine/schema/src/test/java/org/apache/cloudstack/backup/dao/BackupDaoImplTest.java:
##########
@@ -0,0 +1,150 @@
+package org.apache.cloudstack.backup.dao;
+
+import java.util.HashMap;
+import java.util.Map;
+
+import org.apache.cloudstack.api.response.BackupResponse;
+import org.apache.cloudstack.backup.Backup;
+import org.apache.cloudstack.backup.BackupOfferingVO;
+import org.apache.cloudstack.backup.BackupVO;
+import org.junit.Assert;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.mockito.InjectMocks;
+import org.mockito.Mock;
+import org.mockito.Mockito;
+import org.mockito.Spy;
+import org.mockito.junit.MockitoJUnitRunner;
+import org.springframework.test.util.ReflectionTestUtils;
+
+import com.cloud.dc.DataCenter;
+import com.cloud.dc.DataCenterVO;
+import com.cloud.dc.dao.DataCenterDao;
+import com.cloud.domain.DomainVO;
+import com.cloud.domain.dao.DomainDao;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.user.AccountVO;
+import com.cloud.user.dao.AccountDao;
+import com.cloud.utils.db.SearchBuilder;
+import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.dao.VMInstanceDao;
+
+@RunWith(MockitoJUnitRunner.class)
+public class BackupDaoImplTest {
+ @Spy
+ @InjectMocks
+ private BackupDaoImpl backupDao;
+
+ @Mock
+ BackupDetailsDao backupDetailsDao;
+
+ @Mock
+ SearchBuilder<BackupVO> backupSearch;
+
+ @Mock
+ VMInstanceDao vmInstanceDao;
+
+ @Mock
+ AccountDao accountDao;
+
+ @Mock
+ DomainDao domainDao;
+
+ @Mock
+ DataCenterDao dataCenterDao;
+
+ @Mock
+ BackupOfferingDao backupOfferingDao;
+
+ @Test
+ public void testLoadDetails() {
+ Long backupId = 1L;
+ BackupVO backup = new BackupVO();
+ ReflectionTestUtils.setField(backup, "id", backupId);
+ Map<String, String> details = new HashMap<>();
+ details.put("key1", "value1");
+ details.put("key2", "value2");
+
+ Mockito.when(backupDetailsDao.listDetailsKeyPairs(backupId)).thenReturn(details);
+
+ backupDao.loadDetails(backup);
+
+ Assert.assertEquals(details, backup.getDetails());
+ Mockito.verify(backupDetailsDao).listDetailsKeyPairs(backupId);
+ }
+
+ @Test
+ public void testSaveDetails() {
+ Long backupId = 1L;
+ BackupVO backup = new BackupVO();
+ ReflectionTestUtils.setField(backup, "id", backupId);
+ Map<String, String> details = new HashMap<>();
+ details.put("key1", "value1");
+ details.put("key2", "value2");
+ backup.setDetails(details);
+
+ backupDao.saveDetails(backup);
+
+ Mockito.verify(backupDetailsDao).saveDetails(Mockito.anyList());
+ }
+
+ @Test
+ public void testNewBackupResponse() {
+ Long vmId = 1L;
+ Long accountId = 2L;
+ Long domainId = 3L;
+ Long zoneId = 4L;
+ Long offeringId = 5L;
+ Long backupId = 6L;
+ BackupVO backup = new BackupVO();
+ ReflectionTestUtils.setField(backup, "id", backupId);
+ ReflectionTestUtils.setField(backup, "uuid", "backup-uuid");
+ backup.setVmId(vmId);
+ backup.setAccountId(accountId);
+ backup.setDomainId(domainId);
+ backup.setZoneId(zoneId);
+ backup.setBackupOfferingId(offeringId);
+ backup.setType("Full");
+ backup.setBackupIntervalType((short) Backup.Type.MANUAL.ordinal());
+
+ VMInstanceVO vm = new VMInstanceVO(vmId, 0L, "test-vm", "test-vm", VirtualMachine.Type.User,
+ 0L, Hypervisor.HypervisorType.Simulator, 0L, domainId, accountId, 0L, false);
+ vm.setDataCenterId(zoneId);
+
+ AccountVO account = new AccountVO();
+ account.setUuid("account-uuid");
+ account.setAccountName("test-account");
+
+ DomainVO domain = new DomainVO();
+ domain.setUuid("domain-uuid");
+ domain.setName("test-domain");
+
+ DataCenterVO zone = new DataCenterVO(1L, "test-zone", null, null, null, null, null, null, null, null, DataCenter.NetworkType.Advanced, null, null);
+ zone.setUuid("zone-uuid");
+
+ BackupOfferingVO offering = Mockito.mock(BackupOfferingVO.class);
+ Mockito.when(offering.getUuid()).thenReturn("offering-uuid");
+ Mockito.when(offering.getName()).thenReturn("test-offering");
+
+ Mockito.when(vmInstanceDao.findByIdIncludingRemoved(vmId)).thenReturn(vm);
+ Mockito.when(accountDao.findByIdIncludingRemoved(accountId)).thenReturn(account);
+ Mockito.when(domainDao.findByIdIncludingRemoved(domainId)).thenReturn(domain);
+ Mockito.when(dataCenterDao.findByIdIncludingRemoved(zoneId)).thenReturn(zone);
+ Mockito.when(backupOfferingDao.findByIdIncludingRemoved(offeringId)).thenReturn(offering);
+
+ BackupResponse response = backupDao.newBackupResponse(backup, false);
+
+ Assert.assertEquals("backup-uuid", response.getId());
+ Assert.assertEquals("test-vm", response.getVmName());
+ Assert.assertEquals("account-uuid", response.getAccountId());
+ Assert.assertEquals("test-account", response.getAccount());
+ Assert.assertEquals("domain-uuid", response.getDomainId());
+ Assert.assertEquals("test-domain", response.getDomain());
+ Assert.assertEquals("zone-uuid", response.getZoneId());
+ Assert.assertEquals("test-zone", response.getZone());
+ Assert.assertEquals("offering-uuid", response.getBackupOfferingId());
+ Assert.assertEquals("test-offering", response.getBackupOffering());
+ Assert.assertEquals("MANUAL", response.getIntervalType());
+ }
+}
Review Comment:
eol before eof
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2714384779
Packaging result [SF]: ✖️ el8 ✖️ el9 ✔️ debian ✖️ suse15. SL-JID 12737
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2792406015
> > > this PR contains 3 changes
> > >
> > > * create new vm from backup
> > > * threshold for object storage
> > > * threshold for backup
> > >
> > > I would prefer to see 3 PRs (at least 2 PRs) but it looks difficult to seperate them now
> >
> >
> > It's not ideal, but it was done this way so test effort could be minimised.
>
> I do not know how much QA work will be reduced. I think they will still have to test all the scenarios.
yes, just in terms of deploying multiple envs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2036958857
##########
engine/schema/src/main/java/org/apache/cloudstack/backup/BackupDetailVO.java:
##########
@@ -0,0 +1,98 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.backup;
+
+import javax.persistence.Column;
+import javax.persistence.Entity;
+import javax.persistence.GeneratedValue;
+import javax.persistence.GenerationType;
+import javax.persistence.Id;
+import javax.persistence.Table;
+
+import org.apache.cloudstack.api.ResourceDetail;
+
+@Entity
+@Table(name = "backup_details")
+public class BackupDetailVO implements ResourceDetail {
+ @Id
+ @GeneratedValue(strategy = GenerationType.IDENTITY)
+ @Column(name = "id")
+ private long id;
+
+ @Column(name = "backup_id")
+ private long resourceId;
+
+ @Column(name = "name")
+ private String name;
+
+ @Column(name = "value", length = 1024)
Review Comment:
no, I'll change it to 65536 which is what BackupVO.backedUpVolumes also uses.
thanks for catching.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2036923812
##########
api/src/main/java/org/apache/cloudstack/api/response/BackupResponse.java:
##########
@@ -102,6 +111,18 @@ public class BackupResponse extends BaseResponse {
@Param(description = "zone name")
private String zone;
+ @SerializedName(ApiConstants.VM_DETAILS)
+ @Param(description = "Lists the vm specific details for the backup", since = "4.21.0")
+ private Map<String, String> vmDetails;
+
+ @SerializedName(ApiConstants.INTERVAL_TYPE)
+ @Param(description = "the interval type of the backup schedule", since = "4.21.0")
+ private String intervalType;
+
+ @SerializedName(ApiConstants.BACKUP_VM_OFFERING_REMOVED)
Review Comment:
just in response, storage.js references it to decide whether to show removeOffering button in backup list view.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "shwstppr (via GitHub)" <gi...@apache.org>.
shwstppr commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2785499344
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r1988656592
##########
engine/schema/src/test/java/org/apache/cloudstack/backup/dao/BackupDaoImplTest.java:
##########
@@ -0,0 +1,150 @@
+package org.apache.cloudstack.backup.dao;
Review Comment:
license
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2716408965
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2824239036
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2824236902
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2961126685
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2961172147
Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 13698
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3047905357
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2964926161
Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 13726
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3021480604
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3017945519
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13951
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3026050285
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3026048415
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3026338999
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2852201179
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13264
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2896747394
<b>[SF] Trillian test result (tid-13365)</b>
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 52480 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10140-t13365-kvm-ol8.zip
Smoke tests completed. 141 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_attach_and_distribute_multiple_volumes | `Error` | 27.56 | test_attach_multiple_volumes.py
test_attach_multiple_volumes | `Failure` | 4.54 | test_attach_multiple_volumes.py
test_attach_multiple_volumes | `Error` | 4.55 | test_attach_multiple_volumes.py
ContextSuite context=TestMultipleVolumeAttach>:teardown | `Error` | 5.66 | test_attach_multiple_volumes.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2943474170
@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2943889301
Packaging result [SF]: ✔️ el8 ✔️ el9 ✖️ debian ✔️ suse15. SL-JID 13619
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2785503701
@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2037113772
##########
server/src/main/java/com/cloud/vm/UserVmManagerImpl.java:
##########
@@ -8987,6 +9024,215 @@ public boolean unmanageUserVM(Long vmId) {
return true;
}
+ @Override
+ public UserVm allocateVMFromBackup(CreateVMFromBackupCmd cmd) throws InsufficientCapacityException, ResourceAllocationException, ResourceUnavailableException {
Review Comment:
@weizhouapache Thanks for the comments. I have addressed all of them apart from this one and the one about iso. I'll get back to you on them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
weizhouapache commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2037143721
##########
plugins/backup/veeam/src/main/java/org/apache/cloudstack/backup/veeam/VeeamClient.java:
##########
@@ -855,55 +816,61 @@ public List<Backup.RestorePoint> listRestorePointsLegacy(String backupName, Stri
}
logger.debug(String.format("Found restore points from [backupName: %s, vmInternalName: %s] which is: [%s].", backupName, vmInternalName, block));
final String[] parts = block.split("\r\n");
- restorePoints.add(getRestorePointFromBlock(parts));
+ restorePoints.add(getRestorePointFromBlock(parts, metricsMap));
}
return restorePoints;
}
- public List<Backup.RestorePoint> listRestorePoints(String backupName, String vmInternalName) {
+ public List<Backup.RestorePoint> listRestorePoints(String backupName, String vmwareDcName, String vmInternalName, Map<String, Backup.Metric> metricsMap) {
if (isLegacyServer()) {
- return listRestorePointsLegacy(backupName, vmInternalName);
+ return listRestorePointsLegacy(backupName, vmInternalName, metricsMap);
} else {
- return listVmRestorePointsViaVeeamAPI(vmInternalName);
+ return listVmRestorePointsViaVeeamAPI(vmwareDcName, vmInternalName, metricsMap);
}
}
- public List<Backup.RestorePoint> listVmRestorePointsViaVeeamAPI(String vmInternalName) {
+ public List<Backup.RestorePoint> listVmRestorePointsViaVeeamAPI(String vmwareDcName, String vmInternalName, Map<String, Backup.Metric> metricsMap) {
logger.debug(String.format("Trying to list VM restore points via Veeam B&R API for VM %s: ", vmInternalName));
try {
final HttpResponse response = get(String.format("/vmRestorePoints?format=Entity"));
checkResponseOK(response);
- return processHttpResponseForVmRestorePoints(response.getEntity().getContent(), vmInternalName);
+ return processHttpResponseForVmRestorePoints(response.getEntity().getContent(), vmwareDcName, vmInternalName, metricsMap);
} catch (final IOException e) {
logger.error("Failed to list VM restore points via Veeam B&R API due to:", e);
checkResponseTimeOut(e);
}
return new ArrayList<>();
}
- public List<Backup.RestorePoint> processHttpResponseForVmRestorePoints(InputStream content, String vmInternalName) {
+ public List<Backup.RestorePoint> processHttpResponseForVmRestorePoints(InputStream content, String vmwareDcName, String vmInternalName, Map<String, Backup.Metric> metricsMap) {
List<Backup.RestorePoint> vmRestorePointList = new ArrayList<>();
try {
final ObjectMapper objectMapper = new XmlMapper();
final VmRestorePoints vmRestorePoints = objectMapper.readValue(content, VmRestorePoints.class);
+ final String hierarchyId = findDCHierarchy(vmwareDcName);
+ final String hierarchyUuid = hierarchyId.substring(hierarchyId.lastIndexOf(":") + 1);
if (vmRestorePoints == null) {
throw new CloudRuntimeException("Could not get VM restore points via Veeam B&R API");
}
for (final VmRestorePoint vmRestorePoint : vmRestorePoints.getVmRestorePoints()) {
logger.debug(String.format("Processing VM restore point Name=%s, VmDisplayName=%s for vm name=%s",
vmRestorePoint.getName(), vmRestorePoint.getVmDisplayName(), vmInternalName));
- if (!vmInternalName.equals(vmRestorePoint.getVmDisplayName())) {
+ if (!vmInternalName.equals(vmRestorePoint.getVmDisplayName()) || !vmRestorePoint.getHierarchyObjRef().contains(hierarchyUuid)) {
continue;
}
boolean isReady = true;
+ String backupFileId = null;
List<Link> links = vmRestorePoint.getLink();
for (Link link : links) {
if (Arrays.asList(BACKUP_FILE_REFERENCE, RESTORE_POINT_REFERENCE).contains(link.getType()) && !link.getRel().equals("Up")) {
logger.info(String.format("The VM restore point is not ready. Reference: %s, state: %s", link.getType(), link.getRel()));
isReady = false;
break;
}
+ if (link.getType() != null && link.getType().equals(BACKUP_FILE_REFERENCE)) {
+ backupFileId = link.getHref().substring(link.getHref().lastIndexOf('/') + 1);
Review Comment:
ah ok, there is already a method `StringUtils.substringAfterLast`
good to know it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2794489574
<b>[SF] Trillian test result (tid-12919)</b>
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 114554 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10140-t12919-kvm-ol8.zip
Smoke tests completed. 114 look OK, 27 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
ContextSuite context=TestInternalLb>:setup | `Error` | 0.00 | test_internal_lb.py
ContextSuite context=TestIpv4Routing>:setup | `Error` | 0.00 | test_ipv4_routing.py
test_01_1_create_iso_with_checksum_sha1_negative | `Error` | 66.60 | test_iso.py
test_01_create_iso_with_checksum_sha1 | `Error` | 66.58 | test_iso.py
test_01_create_iso_with_checksum_sha1 | `Error` | 66.58 | test_iso.py
test_02_1_create_iso_with_checksum_sha256_negative | `Error` | 66.59 | test_iso.py
test_02_create_iso_with_checksum_sha256 | `Error` | 66.56 | test_iso.py
test_02_create_iso_with_checksum_sha256 | `Error` | 66.56 | test_iso.py
test_03_1_create_iso_with_checksum_md5_negative | `Error` | 66.52 | test_iso.py
test_03_create_iso_with_checksum_md5 | `Error` | 66.56 | test_iso.py
test_03_create_iso_with_checksum_md5 | `Error` | 66.56 | test_iso.py
test_04_create_iso_with_no_checksum | `Error` | 66.55 | test_iso.py
test_04_create_iso_with_no_checksum | `Error` | 66.55 | test_iso.py
test_01_create_iso | `Failure` | 1520.27 | test_iso.py
ContextSuite context=TestISO>:setup | `Error` | 3038.15 | test_iso.py
test_01_browser_migrate_template | `Error` | 65.78 | test_image_store_object_migration.py
test_01_invalid_upgrade_kubernetes_cluster | `Failure` | 0.01 | test_kubernetes_clusters.py
test_02_upgrade_kubernetes_cluster | `Failure` | 0.01 | test_kubernetes_clusters.py
test_03_deploy_and_scale_kubernetes_cluster | `Failure` | 0.01 | test_kubernetes_clusters.py
test_04_autoscale_kubernetes_cluster | `Failure` | 0.00 | test_kubernetes_clusters.py
test_05_basic_lifecycle_kubernetes_cluster | `Failure` | 0.01 | test_kubernetes_clusters.py
test_06_delete_kubernetes_cluster | `Failure` | 0.01 | test_kubernetes_clusters.py
test_08_upgrade_kubernetes_ha_cluster | `Failure` | 0.01 | test_kubernetes_clusters.py
test_10_vpc_tier_kubernetes_cluster | `Failure` | 0.01 | test_kubernetes_clusters.py
test_11_test_unmanaged_cluster_lifecycle | `Error` | 0.01 | test_kubernetes_clusters.py
test_01_add_delete_kubernetes_supported_version | `Error` | 1801.81 | test_kubernetes_supported_versions.py
ContextSuite context=TestListIdsParams>:setup | `Error` | 0.00 | test_list_ids_parameter.py
ContextSuite context=TestLoadBalance>:setup | `Error` | 0.00 | test_loadbalance.py
ContextSuite context=TestMetrics>:setup | `Error` | 0.00 | test_metrics_api.py
test_01_native_to_native_network_migration | `Error` | 13.95 | test_migration.py
test_02_native_to_native_vpc_migration | `Error` | 11.57 | test_migration.py
test_nic_secondaryip_add_remove | `Error` | 1519.07 | test_multipleips_per_nic.py
ContextSuite context=TestNestedVirtualization>:setup | `Error` | 0.00 | test_nested_virtualization.py
ContextSuite context=TestNetworkACL>:setup | `Error` | 0.00 | test_network_acl.py
ContextSuite context=TestPrivateGwACL>:setup | `Error` | 0.00 | test_privategw_acl.py
ContextSuite context=TestIpv6Network>:setup | `Error` | 0.00 | test_network_ipv6.py
test_03_network_operations_on_created_vm_of_otheruser | `Error` | 2.55 | test_network_permissions.py
test_03_network_operations_on_created_vm_of_otheruser | `Error` | 2.55 | test_network_permissions.py
test_04_deploy_vm_for_other_user_and_test_vm_operations | `Failure` | 1.39 | test_network_permissions.py
ContextSuite context=TestNetworkPermissions>:teardown | `Error` | 1.43 | test_network_permissions.py
test_delete_account | `Error` | 1515.93 | test_network.py
test_delete_network_while_vm_on_it | `Error` | 1.17 | test_network.py
test_deploy_vm_l2network | `Error` | 1.17 | test_network.py
test_l2network_restart | `Error` | 2.29 | test_network.py
ContextSuite context=TestPortForwarding>:setup | `Error` | 3.51 | test_network.py
ContextSuite context=TestPublicIP>:setup | `Error` | 9.13 | test_network.py
test_reboot_router | `Failure` | 0.08 | test_network.py
test_releaseIP | `Error` | 4.72 | test_network.py
test_releaseIP_using_IP | `Error` | 4.57 | test_network.py
ContextSuite context=TestRouterRules>:setup | `Error` | 4.64 | test_network.py
test_01_deployVMInSharedNetwork | `Failure` | 1.26 | test_network.py
test_02_verifyRouterIpAfterNetworkRestart | `Failure` | 1.09 | test_network.py
test_03_destroySharedNetwork | `Failure` | 1.09 | test_network.py
ContextSuite context=TestSharedNetwork>:teardown | `Error` | 2.22 | test_network.py
ContextSuite context=TestSharedNetworkWithConfigDrive>:setup | `Error` | 1518.40 | test_network.py
test_01_add_primary_storage_disabled_host | `Error` | 38.27 | test_primary_storage.py
test_01_primary_storage_nfs | `Error` | 0.19 | test_primary_storage.py
ContextSuite context=TestStorageTags>:setup | `Error` | 0.32 | test_primary_storage.py
ContextSuite context=TestAdapterTypeForNic>:setup | `Error` | 0.00 | test_nic_adapter_type.py
test_01_nic | `Error` | 55.46 | test_nic.py
test_01_non_strict_host_anti_affinity | `Error` | 2.46 | test_nonstrict_affinity_group.py
test_02_non_strict_host_affinity | `Error` | 2.53 | test_nonstrict_affinity_group.py
ContextSuite context=TestIsolatedNetworksPasswdServer>:setup | `Error` | 0.00 | test_password_server.py
test_01_isolated_persistent_network | `Error` | 3.58 | test_persistent_network.py
test_03_deploy_and_destroy_VM_and_verify_network_resources_persist | `Failure` | 2.68 | test_persistent_network.py
test_03_deploy_and_destroy_VM_and_verify_network_resources_persist | `Error` | 2.68 | test_persistent_network.py
ContextSuite context=TestL2PersistentNetworks>:teardown | `Error` | 2.74 | test_persistent_network.py
ContextSuite context=TestPortForwardingRules>:setup | `Error` | 0.00 | test_portforwardingrules.py
test_01_primary_storage_scope_change | `Error` | 0.12 | test_primary_storage_scope.py
ContextSuite context=TestProjectSuspendActivate>:setup | `Error` | 1527.89 | test_projects.py
test_01_purge_expunged_api_vm_start_date | `Error` | 3.51 | test_purge_expunged_vms.py
test_02_purge_expunged_api_vm_end_date | `Error` | 2.83 | test_purge_expunged_vms.py
test_03_purge_expunged_api_vm_start_end_date | `Error` | 2.09 | test_purge_expunged_vms.py
test_04_purge_expunged_api_vm_no_date | `Error` | 1.93 | test_purge_expunged_vms.py
test_05_purge_expunged_vm_service_offering | `Error` | 1.48 | test_purge_expunged_vms.py
test_06_purge_expunged_vm_background_task | `Error` | 356.07 | test_purge_expunged_vms.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2987259543
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3121643746
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3113100933
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "JoaoJandre (via GitHub)" <gi...@apache.org>.
JoaoJandre commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2230938954
##########
engine/schema/src/main/resources/META-INF/db/schema-42010to42100.sql:
##########
@@ -241,3 +237,88 @@ ALTER TABLE `cloud`.`vm_instance_details` ADD CONSTRAINT `fk_vm_instance_details
CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backup_schedule', 'uuid', 'VARCHAR(40) NOT NULL');
UPDATE `cloud`.`backup_schedule` SET uuid = UUID();
+
+-- Add column max_backup to backup_schedule table
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backup_schedule', 'max_backups', 'int(8) default NULL COMMENT "maximum number of backups to maintain"');
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backup_schedule', 'quiescevm', 'tinyint(1) default NULL COMMENT "Quiesce VM before taking backup"');
+
+-- Add columns name, description and backup_interval_type to backup table
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backups', 'name', 'VARCHAR(255) NULL COMMENT "name of the backup"');
+UPDATE `cloud`.`backups` backup INNER JOIN `cloud`.`vm_instance` vm ON backup.vm_id = vm.id SET backup.name = vm.name;
+ALTER TABLE `cloud`.`backups` MODIFY COLUMN `name` VARCHAR(255) NOT NULL;
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backups', 'description', 'VARCHAR(1024) COMMENT "description for the backup"');
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backups', 'backup_interval_type', 'int(5) COMMENT "type of backup, e.g. manual, recurring - hourly, daily, weekly or monthly"');
+
+-- Create backup details table
+CREATE TABLE IF NOT EXISTS `cloud`.`backup_details` (
+ `id` bigint unsigned NOT NULL auto_increment,
+ `backup_id` bigint unsigned NOT NULL COMMENT 'backup id',
+ `name` varchar(255) NOT NULL,
+ `value` TEXT NOT NULL,
+ `display` tinyint(1) NOT NULL DEFAULT 1 COMMENT 'Should detail be displayed to the end user',
+ PRIMARY KEY (`id`),
+ CONSTRAINT `fk_backup_details__backup_id` FOREIGN KEY `fk_backup_details__backup_id`(`backup_id`) REFERENCES `backups`(`id`) ON DELETE CASCADE
+) ENGINE=InnoDB DEFAULT CHARSET=utf8;
+
+-- Add diskOfferingId, deviceId, minIops and maxIops to backed_volumes in backups table
+UPDATE `cloud`.`backups` b
+INNER JOIN `cloud`.`vm_instance` vm ON b.vm_id = vm.id
+SET b.backed_volumes = (
+ SELECT CONCAT("[",
+ GROUP_CONCAT(
+ CONCAT(
+ "{\"uuid\":\"", v.uuid, "\",",
+ "\"type\":\"", v.volume_type, "\",",
+ "\"size\":", v.`size`, ",",
+ "\"path\":\"", IFNULL(v.path, 'null'), "\",",
+ "\"deviceId\":", IFNULL(v.device_id, 'null'), ",",
+ "\"diskOfferingId\":\"", doff.uuid, "\",",
+ "\"minIops\":", IFNULL(v.min_iops, 'null'), ",",
+ "\"maxIops\":", IFNULL(v.max_iops, 'null'),
+ "}"
+ )
+ SEPARATOR ","
+ ),
+ "]")
+ FROM `cloud`.`volumes` v
+ LEFT JOIN `cloud`.`disk_offering` doff ON v.disk_offering_id = doff.id
+ WHERE v.instance_id = vm.id
+);
+
+-- Add diskOfferingId, deviceId, minIops and maxIops to backup_volumes in vm_instance table
+UPDATE `cloud`.`vm_instance` vm
+SET vm.backup_volumes = (
+ SELECT CONCAT("[",
Review Comment:
Yes some upgrade path should have been done when backup.backed_volumes was added for sure. But I understand it is out of scope for this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2225032181
##########
engine/schema/src/main/java/org/apache/cloudstack/backup/BackupVO.java:
##########
@@ -240,10 +268,32 @@ public void setBackedUpVolumes(String backedUpVolumes) {
this.backedUpVolumes = backedUpVolumes;
}
+ @Override
+ public Map<String, String> getDetails() {
+ return details;
+ }
+
+ public void setDetail(String name, String value) {
+ assert (details != null) : "Did you forget to load the details?";
Review Comment:
Other detailsVOs follow the same pattern. I guess it is for developers to know that details are not being used correctly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3107561694
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "JoaoJandre (via GitHub)" <gi...@apache.org>.
JoaoJandre commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2225869526
##########
engine/schema/src/main/java/org/apache/cloudstack/backup/BackupVO.java:
##########
@@ -240,10 +268,32 @@ public void setBackedUpVolumes(String backedUpVolumes) {
this.backedUpVolumes = backedUpVolumes;
}
+ @Override
+ public Map<String, String> getDetails() {
+ return details;
+ }
+
+ public void setDetail(String name, String value) {
+ assert (details != null) : "Did you forget to load the details?";
Review Comment:
@abh1sar It is not advisable to use asserts in production code, these can be disabled when running the process, for example.
I also think it would be better to throw a CloudRuntimeException instead of an AssertionError if this condition is true.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3124064334
<b>[SF] Trillian test result (tid-13917)</b>
Environment: kvm-ubuntu22 (x2), Advanced Networking with Mgmt server u22
Total time taken: 59792 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10140-t13917-kvm-ubuntu22.zip
Smoke tests completed. 143 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2619796120
> one diskoffering - zone link edge case seems to be remaining for testing/changes
> Please see if we need to add some documentation for the feature
thanks @shwstppr
I have fixed the disk-offering edge case and catch all exceptions plus some minor UI fixes. Also linked doc PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [WIP] Create new Instance from VM backup [cloudstack]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2561620639
## [Codecov](https://app.codecov.io/gh/apache/cloudstack/pull/10140?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 4.01%. Comparing base [(`54bc150`)](https://app.codecov.io/gh/apache/cloudstack/commit/54bc1501402ceea4a3c96b1868da4e2b673268c1?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`e094e14`)](https://app.codecov.io/gh/apache/cloudstack/commit/e094e1456c315f368005107852e0182486d2646b?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
> Report is 22 commits behind head on main.
> :exclamation: There is a different number of reports uploaded between BASE (54bc150) and HEAD (e094e14). Click for more details.
>
> <details><summary>HEAD has 1 upload less than BASE</summary>
>
>| Flag | BASE (54bc150) | HEAD (e094e14) |
>|------|------|------|
>|unittests|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #10140 +/- ##
=============================================
- Coverage 16.06% 4.01% -12.05%
=============================================
Files 5642 394 -5248
Lines 493856 32406 -461450
Branches 59860 5725 -54135
=============================================
- Hits 79324 1301 -78023
+ Misses 405749 30957 -374792
+ Partials 8783 148 -8635
```
| [Flag](https://app.codecov.io/gh/apache/cloudstack/pull/10140/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [uitests](https://app.codecov.io/gh/apache/cloudstack/pull/10140/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `4.01% <ø> (-0.01%)` | :arrow_down: |
| [unittests](https://app.codecov.io/gh/apache/cloudstack/pull/10140/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/cloudstack/pull/10140?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3068792774
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "rohityadavcloud (via GitHub)" <gi...@apache.org>.
rohityadavcloud commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3067760754
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3131736805
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14437
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "JoaoJandre (via GitHub)" <gi...@apache.org>.
JoaoJandre commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3132114108
> > * Tried to configure create VM from backup in the GUI, was unable to change template.
> > * Tried to configure create VM from backup in the FUI, was unable to change ISO.
>
> Template/ISO can only be changed if the original template/iso was deleted. It is possible to do it from API though for advanced users. Added a note in the doc [apache/cloudstack-documentation@01c0bc2](https://github.com/apache/cloudstack-documentation/commit/01c0bc287f405b338f8b80c63e7551fc6f0175c1).
>
Could we add either a warning or not show that part of the form when this is the case? Otherwise this would be a bad UX.
> > * The form to configure VM from backup does not allow to create network if an option already exists.
>
> I didn't see this in my testing. https://github.com/user-attachments/assets/fd97e79a-7b07-4c1a-9f0f-fee6465edff3
Maybe it was only present in the commit I was testing. I tested it again on the latest commit and was able to create a network this time.
> > * Tried to create VM from backup and delete the backup before the process ends, the deletion was allowed but the VM was not restored. I think that these processes should be either blocked from executing at the same time, or be queued.
>
> The processes are queued once they reach the agent. i.e, if the new instance is in Restoring state then backup is deleted, the agent will process the restore first and then backup delete. If the new instance is still being created/starting/stopping before restore, and the backup is deleted, delete is not blocked and restore will throw an error that backup is deleted. IMO this is not a must have, and I'll be happy to fix this as a future improvement.
Could we map it as an issue then?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3135194381
@sureshanaparti a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2987231690
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2987227115
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2978963421
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2915524851
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2814485921
> About the deviceId of datadiskoffering
>
> ```
> dataDiskOfferingInfo.setDeviceId(1L);
> ```
>
> should it be 0 if the vm is created from ISO ? this is not a valid question if we do not support backup actions for vm from ISO.
For VM from ISOs createVirtualMachineFromScratch is called where deviceId remains null in rootDiskOfferingInfo and in allocateRawVolume the deviceId for volume is set to 0L.
```
if (deviceId != null) {
vol.setDeviceId(deviceId);
} else if (type.equals(Type.ROOT)) {
vol.setDeviceId(0l);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2893593423
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2887182095
@abh1sar a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2890220456
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2871254769
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2642759729
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2594610247
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2594426112
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2596333025
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [WIP] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2572967130
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2679535756
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2716520837
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12746
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti merged PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2795192381
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2850750841
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13261
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3096543304
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14281
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3086665985
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2230376567
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/NetworkerBackupProvider.java:
##########
@@ -116,6 +121,15 @@ public class NetworkerBackupProvider extends AdapterBase implements BackupProvid
@Inject
private VMInstanceDao vmInstanceDao;
+ @Inject
+ private EntityManager entityManager;
+
+ @Inject
+ BackupManager backupManager;
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2228995688
##########
server/src/main/java/org/apache/cloudstack/backup/BackupManagerImpl.java:
##########
@@ -926,6 +1058,224 @@ private Backup.VolumeInfo getVolumeInfo(List<Backup.VolumeInfo> backedUpVolumes,
return null;
}
+ @Override
+ public void checkVmDisksSizeAgainstBackup(List<VmDiskInfo> vmDiskInfoList, Backup backup) {
+ List<VmDiskInfo> vmDiskInfoListFromBackup = getDataDiskInfoListFromBackup(backup);
+ int index = 0;
+ if (vmDiskInfoList.size() != vmDiskInfoListFromBackup.size()) {
+ throw new InvalidParameterValueException("Unable to create Instance from Backup " +
+ "as the backup has a different number of disks than the Instance.");
+ }
+ for (VmDiskInfo vmDiskInfo : vmDiskInfoList) {
+ if (index < vmDiskInfoListFromBackup.size()) {
+ if (vmDiskInfo.getSize() < vmDiskInfoListFromBackup.get(index).getSize()) {
+ throw new InvalidParameterValueException(
+ String.format("Instance volume size %d[GiB] cannot be less than the backed-up volume size %d[GiB].",
+ vmDiskInfo.getSize(), vmDiskInfoListFromBackup.get(index).getSize()));
+ }
+ }
+ index++;
+ }
+ }
+
+ @Override
+ public VmDiskInfo getRootDiskInfoFromBackup(Backup backup) {
+ List<Backup.VolumeInfo> volumes = backup.getBackedUpVolumes();
+ VmDiskInfo rootDiskOffering = null;
+ if (volumes == null || volumes.isEmpty()) {
+ throw new CloudRuntimeException("Failed to get backed-up volumes info from backup");
+ }
+ for (Backup.VolumeInfo volume : volumes) {
+ if (volume.getType() == Volume.Type.ROOT) {
+ DiskOfferingVO diskOffering = diskOfferingDao.findByUuid(volume.getDiskOfferingId());
+ if (diskOffering == null) {
+ throw new CloudRuntimeException(String.format("Unable to find the root disk offering with uuid (%s) " +
+ "stored in backup. Please specify a valid root disk offering id while creating the instance",
+ volume.getDiskOfferingId()));
+ }
+ Long size = volume.getSize() / (1024 * 1024 * 1024);
+ rootDiskOffering = new VmDiskInfo(diskOffering, size, volume.getMinIops(), volume.getMaxIops());
+ }
+ }
+ if (rootDiskOffering == null) {
+ throw new CloudRuntimeException("Failed to get the root disk in backed-up volumes info from backup");
+ }
+ return rootDiskOffering;
+ }
+
+ @Override
+ public List<VmDiskInfo> getDataDiskInfoListFromBackup(Backup backup) {
+ List<VmDiskInfo> vmDiskInfoList = new ArrayList<>();
+ List<Backup.VolumeInfo> volumes = backup.getBackedUpVolumes();
+ if (volumes == null || volumes.isEmpty()) {
+ throw new CloudRuntimeException("Failed to get backed-up Volumes info from backup");
+ }
+ for (Backup.VolumeInfo volume : volumes) {
+ if (volume.getType() == Volume.Type.DATADISK) {
+ DiskOfferingVO diskOffering = diskOfferingDao.findByUuid(volume.getDiskOfferingId());
+ if (diskOffering == null || diskOffering.getState().equals(DiskOffering.State.Inactive)) {
+ throw new CloudRuntimeException("Unable to find the disk offering with uuid (" + volume.getDiskOfferingId() + ") stored in backup. " +
+ "Please specify a valid disk offering id while creating the instance");
+ }
+ Long size = volume.getSize() / (1024 * 1024 * 1024);
+ vmDiskInfoList.add(new VmDiskInfo(diskOffering, size, volume.getMinIops(), volume.getMaxIops(), volume.getDeviceId()));
+ }
+ }
+ return vmDiskInfoList;
+ }
+
+ private List<String> parseAddressString(String input) {
+ if (input == null) return null;
+ return Arrays.stream(input.split(","))
+ .map(s -> "null".equalsIgnoreCase(s.trim()) ? null : s.trim())
+ .collect(Collectors.toList());
+ }
Review Comment:
no, will remove
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2228996628
##########
server/src/main/java/org/apache/cloudstack/backup/BackupManagerImpl.java:
##########
@@ -599,10 +705,19 @@ public boolean createBackup(final Long vmId, final Long scheduleId) throws Resou
throw new CloudRuntimeException("VM backup offering not found");
}
+ final BackupProvider backupProvider = getBackupProvider(offering.getProvider());
+ if (offering == null) {
+ throw new CloudRuntimeException("VM backup provider not found for the offering");
+ }
Review Comment:
it should be `if (backup_provider == null) {` here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2225027278
##########
api/src/main/java/org/apache/cloudstack/api/command/admin/vm/CreateVMFromBackupCmdByAdmin.java:
##########
@@ -0,0 +1,55 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.api.command.admin.vm;
+
+import org.apache.cloudstack.acl.RoleType;
+import org.apache.cloudstack.api.APICommand;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.api.Parameter;
+import org.apache.cloudstack.api.ResponseObject;
+import org.apache.cloudstack.api.command.admin.AdminCmd;
+import org.apache.cloudstack.api.command.user.vm.CreateVMFromBackupCmd;
+import org.apache.cloudstack.api.response.ClusterResponse;
+import org.apache.cloudstack.api.response.PodResponse;
+import org.apache.cloudstack.api.response.UserVmResponse;
+
+import com.cloud.vm.VirtualMachine;
+
+@APICommand(name = "createVMFromBackup",
+ description = "Creates and automatically starts a VM from a backup.",
+ responseObject = UserVmResponse.class,
+ responseView = ResponseObject.ResponseView.Full,
+ entityType = {VirtualMachine.class},
+ requestHasSensitiveInfo = false, responseHasSensitiveInfo = true,
+ since = "4.21.0",
+ authorized = {RoleType.Admin, RoleType.ResourceAdmin, RoleType.DomainAdmin, RoleType.User})
+public class CreateVMFromBackupCmdByAdmin extends CreateVMFromBackupCmd implements AdminCmd {
+
+ @Parameter(name = ApiConstants.POD_ID, type = CommandType.UUID, entityType = PodResponse.class, description = "destination Pod ID to deploy the VM to - parameter available for root admin only", since = "4.13")
+ private Long podId;
+
+ @Parameter(name = ApiConstants.CLUSTER_ID, type = CommandType.UUID, entityType = ClusterResponse.class, description = "destination Cluster ID to deploy the VM to - parameter available for root admin only", since = "4.13")
Review Comment:
fixed by commit https://github.com/apache/cloudstack/pull/10140/commits/a3f7bfa68d12ff290332fd110c25a1bf7506b550
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3107482718
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14312
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2230360081
##########
server/src/main/java/org/apache/cloudstack/backup/BackupManagerImpl.java:
##########
@@ -1470,7 +1910,22 @@ private Backup checkAndUpdateIfBackupEntryExistsForRestorePoint(Backup.RestorePo
return null;
}
- private void syncBackups(BackupProvider backupProvider, VirtualMachine vm, Backup.Metric metric) {
+ private void processRemoveList(List<Long> removeList, VirtualMachine vm) {
+ for (final Long backupIdToRemove : removeList) {
+ logger.warn(String.format("Removing backup with ID: [%s].", backupIdToRemove));
+ Backup backup = backupDao.findById(backupIdToRemove);
+ resourceLimitMgr.decrementResourceCount(backup.getAccountId(), Resource.ResourceType.backup);
+ resourceLimitMgr.decrementResourceCount(backup.getAccountId(), Resource.ResourceType.backup_storage, backup.getSize());
+ boolean result = backupDao.remove(backupIdToRemove);
+ if (result) {
+ checkAndGenerateUsageForLastBackupDeletedAfterOfferingRemove(vm, backup);
+ } else {
+ logger.error("Failed to remove backup db wentry ith ID: {} during sync backups", backupIdToRemove);
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3116829194
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3130183427
<b>[SF] Trillian test result (tid-13932)</b>
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 51888 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10140-t13932-kvm-ol8.zip
Smoke tests completed. 145 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_01_deployVMInSharedNetwork | `Failure` | 426.95 | test_network.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3117439453
@sureshanaparti a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "JoaoJandre (via GitHub)" <gi...@apache.org>.
JoaoJandre commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2237556196
##########
server/src/main/java/org/apache/cloudstack/resource/ResourceCleanupServiceImpl.java:
##########
@@ -320,20 +324,33 @@ protected Pair<List<Long>, List<Long>> getFilteredVmIdsForSnapshots(List<Long> v
}
currentSkippedVmIds.addAll(vmIdsWithActiveVolumeSnapshots);
}
+
+ List<BackupVO> backups = backupDao.searchByVmIds(vmIds);
+ if (CollectionUtils.isNotEmpty(backups)) {
+ HashSet<Long> vmIdsWithBackups = backups.stream().map(BackupVO::getVmId)
+ .collect(Collectors.toCollection(HashSet::new));
+ if (logger.isDebugEnabled()) {
+ logger.debug(String.format("Skipping purging VMs with IDs %s as they have backups",
+ StringUtils.join(vmIdsWithBackups)));
+ }
+ currentSkippedVmIds.addAll(vmIdsWithBackups);
+ }
Review Comment:
We could make this behavior configurable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3113393162
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14347
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3121572207
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3095892945
Packaging result [SF]: ✖️ el8 ✖️ el9 ✔️ debian ✖️ suse15. SL-JID 14278
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3134893744
UI build: :heavy_check_mark:
Live QA URL: https://qa.cloudstack.cloud/simulator/pr/10140 (QA-JID-703)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3135175259
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14457
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3136255909
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2852019970
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2852208412
@abh1sar a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2800938046
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2037150147
##########
engine/schema/src/main/resources/META-INF/db/schema-42010to42100.sql:
##########
@@ -37,3 +33,38 @@ WHERE rp.rule = 'quotaStatement'
AND NOT EXISTS(SELECT 1 FROM cloud.role_permissions rp_ WHERE rp.role_id = rp_.role_id AND rp_.rule = 'quotaCreditsList');
CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.host', 'last_mgmt_server_id', 'bigint unsigned DEFAULT NULL COMMENT "last management server this host is connected to" AFTER `mgmt_server_id`');
+
+-- Add column max_backup to backup_schedule table
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backup_schedule', 'max_backups', 'int(8) default NULL COMMENT "maximum number of backups to maintain"');
+
+-- Add columns name, description and backup_interval_type to backup table
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backups', 'name', 'VARCHAR(255) NOT NULL COMMENT "name of the backup"');
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backups', 'description', 'VARCHAR(1024) COMMENT "description for the backup"');
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backups', 'backup_interval_type', 'int(5) COMMENT "type of backup, e.g. manual, recurring - hourly, daily, weekly or monthly"');
+
+-- Make the column vm_id in backups table nullable to handle orphan backups
+ALTER TABLE `cloud`.`backups` MODIFY COLUMN `vm_id` BIGINT UNSIGNED NULL;
+
+-- Create backup details table
+CREATE TABLE `cloud`.`backup_details` (
+ `id` bigint unsigned NOT NULL auto_increment,
+ `backup_id` bigint unsigned NOT NULL COMMENT 'backup id',
+ `name` varchar(255) NOT NULL,
+ `value` varchar(65536) NOT NULL,
Review Comment:
Yes, fixing it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2810177222
Thanks @weizhouapache
> * for vm created from ISO, the root volume uses diskofferingId , deviceId should be 0 I think.
I tested this. The root volume is being assigned deviceId 0 for both template and iso.
> * for vm created with multiple data disks, it uses `datadiskofferinglist`, but deviceId of volumes are not set in dataDiskOfferingInfo.
deviceIds are set here
```
public List<DiskOfferingInfo> getDataDiskOfferingListFromBackup(Backup backup) {
...
diskOfferingInfoList.add(new DiskOfferingInfo(diskOffering, size, minIops, maxIops, deviceId));
```
Please let me know if this sounds ok.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2854066507
<b>[SF] Trillian test result (tid-13204)</b>
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 49079 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10140-t13204-kvm-ol8.zip
Smoke tests completed. 141 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_attach_and_distribute_multiple_volumes | `Error` | 32.21 | test_attach_multiple_volumes.py
test_attach_multiple_volumes | `Failure` | 2.42 | test_attach_multiple_volumes.py
test_attach_multiple_volumes | `Error` | 2.42 | test_attach_multiple_volumes.py
ContextSuite context=TestMultipleVolumeAttach>:teardown | `Error` | 3.51 | test_attach_multiple_volumes.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2709730225
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2036958857
##########
engine/schema/src/main/java/org/apache/cloudstack/backup/BackupDetailVO.java:
##########
@@ -0,0 +1,98 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.backup;
+
+import javax.persistence.Column;
+import javax.persistence.Entity;
+import javax.persistence.GeneratedValue;
+import javax.persistence.GenerationType;
+import javax.persistence.Id;
+import javax.persistence.Table;
+
+import org.apache.cloudstack.api.ResourceDetail;
+
+@Entity
+@Table(name = "backup_details")
+public class BackupDetailVO implements ResourceDetail {
+ @Id
+ @GeneratedValue(strategy = GenerationType.IDENTITY)
+ @Column(name = "id")
+ private long id;
+
+ @Column(name = "backup_id")
+ private long resourceId;
+
+ @Column(name = "name")
+ private String name;
+
+ @Column(name = "value", length = 1024)
Review Comment:
no, I'll change it to 65536 which is what BackupVO.backedUpVolumes also uses
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2037008932
##########
plugins/backup/veeam/src/main/java/org/apache/cloudstack/backup/veeam/VeeamClient.java:
##########
@@ -659,9 +656,7 @@ public boolean setJobSchedule(final String jobName) {
public boolean deleteJobAndBackup(final String jobName) {
Pair<Boolean, String> result = executePowerShellCommands(Arrays.asList(
String.format("$job = Get-VBRJob -Name '%s'", jobName),
- "if ($job) { Remove-VBRJob -Job $job -Confirm:$false }",
- String.format("$backup = Get-VBRBackup -Name '%s'", jobName),
- "if ($backup) { Remove-VBRBackup -Backup $backup -FromDisk -Confirm:$false }"
+ "if ($job) { Remove-VBRJob -Job $job -Confirm:$false }"
Review Comment:
Yes.
new instance can be created from the old (and now orphan) backups.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
weizhouapache commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2035455775
##########
api/src/main/java/org/apache/cloudstack/backup/BackupManager.java:
##########
@@ -138,6 +144,14 @@ public interface BackupManager extends BackupService, Configurable, PluggableSer
ConfigKey.Scope.Global,
null);
+ ConfigKey<Float> BackupStorageCapacityThreshold = new ConfigKey<>("Alert", Float.class,
Review Comment:
do we need to a setting to disable threshold ?
##########
api/src/main/java/com/cloud/vm/VirtualMachine.java:
##########
@@ -128,7 +128,6 @@ public static StateMachine2<State, VirtualMachine.Event, VirtualMachine> getStat
s_fsm.addTransition(new Transition<State, Event>(State.Error, VirtualMachine.Event.DestroyRequested, State.Expunging, null));
s_fsm.addTransition(new Transition<State, Event>(State.Error, VirtualMachine.Event.ExpungeOperation, State.Expunging, null));
s_fsm.addTransition(new Transition<State, Event>(State.Stopped, Event.RestoringRequested, State.Restoring, null));
- s_fsm.addTransition(new Transition<State, Event>(State.Expunging, Event.RestoringRequested, State.Restoring, null));
Review Comment:
makes sense
is it needed for 4.20 ? (a trivial issue as I can see)
##########
engine/schema/src/main/java/com/cloud/vm/dao/VMInstanceDaoImpl.java:
##########
@@ -1224,4 +1224,14 @@ public Map<String, Long> getNameIdMapForVmIds(Collection<Long> ids) {
return vms.stream()
.collect(Collectors.toMap(VMInstanceVO::getInstanceName, VMInstanceVO::getId));
}
+
+ @Override
+ public List<VMInstanceVO> listByIds(List<Long> ids) {
Review Comment:
1. `UserVmDaoImpl` has the same method
2. in case of `listIncludingRemovedBy` (not `listBy`), would it better to rename the method ?
##########
engine/schema/src/main/java/org/apache/cloudstack/backup/BackupDetailVO.java:
##########
@@ -0,0 +1,98 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.backup;
+
+import javax.persistence.Column;
+import javax.persistence.Entity;
+import javax.persistence.GeneratedValue;
+import javax.persistence.GenerationType;
+import javax.persistence.Id;
+import javax.persistence.Table;
+
+import org.apache.cloudstack.api.ResourceDetail;
+
+@Entity
+@Table(name = "backup_details")
+public class BackupDetailVO implements ResourceDetail {
+ @Id
+ @GeneratedValue(strategy = GenerationType.IDENTITY)
+ @Column(name = "id")
+ private long id;
+
+ @Column(name = "backup_id")
+ private long resourceId;
+
+ @Column(name = "name")
+ private String name;
+
+ @Column(name = "value", length = 1024)
Review Comment:
is 1024 big enough ?
##########
api/src/main/java/org/apache/cloudstack/api/response/BackupResponse.java:
##########
@@ -102,6 +111,18 @@ public class BackupResponse extends BaseResponse {
@Param(description = "zone name")
private String zone;
+ @SerializedName(ApiConstants.VM_DETAILS)
+ @Param(description = "Lists the vm specific details for the backup", since = "4.21.0")
+ private Map<String, String> vmDetails;
+
+ @SerializedName(ApiConstants.INTERVAL_TYPE)
+ @Param(description = "the interval type of the backup schedule", since = "4.21.0")
+ private String intervalType;
+
+ @SerializedName(ApiConstants.BACKUP_VM_OFFERING_REMOVED)
Review Comment:
is it used somewhere, except in the response ?
##########
api/src/main/java/org/apache/cloudstack/api/command/user/vm/DeployVMCmd.java:
##########
@@ -147,6 +148,13 @@ public class DeployVMCmd extends BaseAsyncCreateCustomIdCmd implements SecurityG
since = "4.4")
private Long rootdisksize;
+ @Parameter(name = ApiConstants.DATADISKS_DETAILS,
+ type = CommandType.MAP,
+ since = "4.21.0",
+ description = "Disk offering details for creating multiple data volumes. Mutually exclusibe with diskOfferingId." +
+ " Example: datadisksdetails[0].diskofferingid=1&datadisksdetails[0].size=10&datadisksdetails[0].miniops=100&datadisksdetails[0].maxiops=200")
Review Comment:
is `diskofferingid` a id (long) or uuid (string)?
in case of uuid, it would be good to change the value `1` to a uuid string in this example
##########
engine/orchestration/src/main/java/org/apache/cloudstack/engine/orchestration/CloudOrchestrator.java:
##########
@@ -225,6 +226,7 @@ public VirtualMachineEntity createVirtualMachine(String id, String owner, String
DiskOfferingInfo dataDiskOfferingInfo = new DiskOfferingInfo();
dataDiskOfferingInfo.setDiskOffering(diskOffering);
+ dataDiskOfferingInfo.setDeviceId(1L);
Review Comment:
if the vm is created from an ISO (not template), is the data disk offering used for root volume ?
did not test yet
##########
api/src/main/java/org/apache/cloudstack/api/response/ObjectStoreResponse.java:
##########
@@ -44,6 +46,10 @@ public class ObjectStoreResponse extends BaseResponseWithAnnotations {
@Param(description = "the total size of the object store")
private Long storageTotal;
+ @SerializedName("storageallocated")
+ @Param(description = "the object store currently allocated size")
Review Comment:
`the allocated size of the oject store` ?
##########
engine/components-api/src/main/java/com/cloud/storage/StorageManager.java:
##########
@@ -220,6 +220,14 @@ public interface StorageManager extends StorageService {
"storage.pool.host.connect.workers", "1",
"Number of worker threads to be used to connect hosts to a primary storage", true);
+ ConfigKey<Float> ObjectStorageCapacityThreshold = new ConfigKey<>("Alert", Float.class,
+ "zone.objectStorage.capacity.notificationthreshold",
+ "0.75",
+ "Percentage (as a value between 0 and 1) of object storage utilization above which alerts will be sent about low storage available.",
+ true,
+ ConfigKey.Scope.Global,
Review Comment:
is it a global setting or zone setting ?
the key has `zone`, but the scope is `Global`
##########
api/src/main/java/org/apache/cloudstack/api/command/user/vm/DeployVMCmd.java:
##########
@@ -147,6 +148,13 @@ public class DeployVMCmd extends BaseAsyncCreateCustomIdCmd implements SecurityG
since = "4.4")
private Long rootdisksize;
+ @Parameter(name = ApiConstants.DATADISKS_DETAILS,
+ type = CommandType.MAP,
+ since = "4.21.0",
+ description = "Disk offering details for creating multiple data volumes. Mutually exclusibe with diskOfferingId." +
Review Comment:
exclusibe -> exclusive
##########
engine/schema/src/main/resources/META-INF/db/schema-42010to42100.sql:
##########
@@ -37,3 +33,38 @@ WHERE rp.rule = 'quotaStatement'
AND NOT EXISTS(SELECT 1 FROM cloud.role_permissions rp_ WHERE rp.role_id = rp_.role_id AND rp_.rule = 'quotaCreditsList');
CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.host', 'last_mgmt_server_id', 'bigint unsigned DEFAULT NULL COMMENT "last management server this host is connected to" AFTER `mgmt_server_id`');
+
+-- Add column max_backup to backup_schedule table
+ALTER TABLE `cloud`.`backup_schedule` ADD COLUMN `max_backups` int(8) default NULL COMMENT 'maximum number of backups to maintain';
Review Comment:
call `IDEMPOTENT_ADD_COLUMN` here ?
##########
engine/schema/src/main/java/org/apache/cloudstack/backup/BackupDetailVO.java:
##########
@@ -0,0 +1,98 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.backup;
+
+import javax.persistence.Column;
+import javax.persistence.Entity;
+import javax.persistence.GeneratedValue;
+import javax.persistence.GenerationType;
+import javax.persistence.Id;
+import javax.persistence.Table;
+
+import org.apache.cloudstack.api.ResourceDetail;
+
+@Entity
+@Table(name = "backup_details")
+public class BackupDetailVO implements ResourceDetail {
+ @Id
+ @GeneratedValue(strategy = GenerationType.IDENTITY)
+ @Column(name = "id")
+ private long id;
+
+ @Column(name = "backup_id")
+ private long resourceId;
+
+ @Column(name = "name")
+ private String name;
+
+ @Column(name = "value", length = 1024)
+ private String value;
+
+ @Column(name = "display")
+ private boolean display = true;
+
+ public BackupDetailVO() {
+ }
+
+ public BackupDetailVO(long templateId, String name, String value, boolean display) {
Review Comment:
templateId or backupId ?
##########
api/src/main/java/org/apache/cloudstack/api/command/admin/resource/ListCapacityCmd.java:
##########
@@ -134,6 +134,8 @@ public int compare(CapacityResponse resp1, CapacityResponse resp2) {
int res = resp1.getZoneName().compareTo(resp2.getZoneName());
if (res != 0) {
return res;
+ } else if (getSortBy() != null) {
Review Comment:
makes sense
will you create a separated pr for 4.19/4.20 ? @abh1sar
##########
api/src/main/java/org/apache/cloudstack/api/command/admin/storage/AddObjectStoragePoolCmd.java:
##########
@@ -56,6 +56,9 @@ public class AddObjectStoragePoolCmd extends BaseCmd {
@Parameter(name = ApiConstants.TAGS, type = CommandType.STRING, description = "the tags for the storage pool")
private String tags;
+ @Parameter(name = ApiConstants.SIZE, type = CommandType.LONG, description = "the total size of the object store in GiB. Used for tracking capacity and sending alerts", since = "4.21")
Review Comment:
@abh1sar
does it mean that CloudStack does not get the actual capacity ?
It relies on the admins to pass correct value, right ?
##########
api/src/main/java/org/apache/cloudstack/api/command/user/vm/CreateVMFromBackupCmd.java:
##########
@@ -0,0 +1,127 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.api.command.user.vm;
+
+import javax.inject.Inject;
+
+import org.apache.cloudstack.acl.RoleType;
+import org.apache.cloudstack.api.APICommand;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.api.ApiErrorCode;
+import org.apache.cloudstack.api.Parameter;
+import org.apache.cloudstack.api.ResponseObject;
+import org.apache.cloudstack.api.ServerApiException;
+import org.apache.cloudstack.api.command.user.UserCmd;
+import org.apache.cloudstack.api.response.BackupResponse;
+import org.apache.cloudstack.api.response.UserVmResponse;
+import org.apache.cloudstack.backup.BackupManager;
+
+import com.cloud.exception.ConcurrentOperationException;
+import com.cloud.exception.InsufficientCapacityException;
+import com.cloud.exception.InsufficientServerCapacityException;
+import com.cloud.exception.ResourceAllocationException;
+import com.cloud.exception.ResourceUnavailableException;
+import com.cloud.uservm.UserVm;
+import com.cloud.vm.VirtualMachine;
+
+@APICommand(name = "createVMFromBackup",
+ description = "Creates and automatically starts a VM from a backup.",
Review Comment:
DeployVMCmd has a parameter `start`, is it honored by this command ?
##########
engine/schema/src/main/java/org/apache/cloudstack/backup/dao/BackupDaoImpl.java:
##########
@@ -158,6 +196,27 @@ public void removeExistingBackups(Long zoneId, Long vmId) {
expunge(sc);
}
+ @Override
+ public BackupVO persist(BackupVO backup) {
+ return Transaction.execute((TransactionCallback<BackupVO>) status -> {
+ BackupVO backupDb = super.persist(backup);
+ saveDetails(backup);
+ loadDetails(backupDb);
+ return backupDb;
+ });
+ }
+
+ @Override
+ public boolean update(Long id, BackupVO backup) {
+ return Transaction.execute((TransactionCallback<Boolean>) status -> {
+ boolean result = super.update(id, backup);
+ if (result == true) {
Review Comment:
`== true` is not needed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2036932484
##########
api/src/main/java/org/apache/cloudstack/backup/BackupManager.java:
##########
@@ -138,6 +144,14 @@ public interface BackupManager extends BackupService, Configurable, PluggableSer
ConfigKey.Scope.Global,
null);
+ ConfigKey<Float> BackupStorageCapacityThreshold = new ConfigKey<>("Alert", Float.class,
Review Comment:
such setting doesn't exist for other any resource as of now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2915531461
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2916389315
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3017825642
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3021922828
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2964622404
Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 13725
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2964805570
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2958546597
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3027767257
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "shwstppr (via GitHub)" <gi...@apache.org>.
shwstppr commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r1991041222
##########
api/src/main/java/org/apache/cloudstack/api/response/ObjectStoreResponse.java:
##########
@@ -17,34 +17,40 @@
package org.apache.cloudstack.api.response;
import com.cloud.serializer.Param;
+
+import org.apache.cloudstack.api.ApiConstants;
import org.apache.cloudstack.storage.object.ObjectStore;
import com.google.gson.annotations.SerializedName;
import org.apache.cloudstack.api.BaseResponseWithAnnotations;
import org.apache.cloudstack.api.EntityReference;
@EntityReference(value = ObjectStore.class)
public class ObjectStoreResponse extends BaseResponseWithAnnotations {
- @SerializedName("id")
+ @SerializedName(ApiConstants.ID)
@Param(description = "the ID of the object store")
private String id;
- @SerializedName("name")
+ @SerializedName(ApiConstants.NAME)
@Param(description = "the name of the object store")
private String name;
- @SerializedName("url")
+ @SerializedName(ApiConstants.URL)
@Param(description = "the url of the object store")
private String url;
- @SerializedName("providername")
- @Param(description = "the provider name of the object store")
+ @SerializedName(ApiConstants.PROVIDER)
+ @Param(description = "the name of the object store provider")
private String providerName;
- @SerializedName("storagetotal")
+ @SerializedName(ApiConstants.SIZE)
Review Comment:
This changes the response param itself. Do we need some kind of deprecation instead?
Similarly for other changed params. Could this cause any issues with automation/SDKs?
##########
api/src/main/java/org/apache/cloudstack/api/response/BackupResponse.java:
##########
@@ -35,6 +35,14 @@ public class BackupResponse extends BaseResponse {
@Param(description = "ID of the VM backup")
private String id;
+ @SerializedName(ApiConstants.NAME)
+ @Param(description = "name of the backup")
Review Comment:
Do we need to add since key here and other new response params?
##########
engine/orchestration/src/main/java/com/cloud/vm/VirtualMachineManagerImpl.java:
##########
@@ -2340,6 +2349,17 @@ public void doInTransactionWithoutResult(final TransactionStatus status) throws
throw new CloudRuntimeException("Unable to destroy " + vm);
} else {
if (expunge) {
+ if (vm.getBackupOfferingId() != null) {
Review Comment:
Possible to refactor to this a method?
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/NetworkerBackupProvider.java:
##########
@@ -615,6 +611,7 @@ public Backup createNewBackupEntryForRestorePoint(Backup.RestorePoint restorePoi
backup.setAccountId(vm.getAccountId());
backup.setDomainId(vm.getDomainId());
backup.setZoneId(vm.getDataCenterId());
+ backup.setName(vm.getHostName() + '-' + new SimpleDateFormat("yyyy-MM-dd'T'HH:mmX").format(new Date()));
Review Comment:
This is a repeating code. Would it make sense to add method in BackupProvider interface or some other common class to get the backup name?
##########
server/src/main/java/com/cloud/server/ManagementServerImpl.java:
##########
@@ -3428,11 +3438,19 @@ protected List<CapacityVO> listCapacitiesWithDetails(final Long zoneId, final Lo
for (final Long zId : dcList) {
// op_host_Capacity contains only allocated stats and the real time
// stats are stored "in memory".
- // List secondary storage capacity only when the api is invoked for the zone layer.
- if ((capacityType == null || capacityType == Capacity.CAPACITY_TYPE_SECONDARY_STORAGE) &&
- podId == null && clusterId == null &&
- StringUtils.isEmpty(t)) {
- taggedCapacities.add(_storageMgr.getSecondaryStorageUsedStats(null, zId));
+ // List secondary and object storage capacities only when the api is invoked for the zone layer.
+ if (podId == null && clusterId == null && StringUtils.isEmpty(t)) {
+ if (capacityType == null) {
Review Comment:
possible to move to a separate method?
##########
server/src/main/java/com/cloud/configuration/Config.java:
##########
@@ -134,6 +134,22 @@ public enum Config {
"0.75",
"Percentage (as a value between 0 and 1) of local storage utilization above which alerts will be sent about low local storage available.",
null),
+ BackupStorageCapacityThreshold(
Review Comment:
Would it be better to define them as ConfigKey in the respective manager/service?
##########
ui/src/views/compute/wizard/TemplateIsoSelection.vue:
##########
@@ -114,6 +114,7 @@ export default {
}
}
if (this.preFillContent.templateid) {
+ let i = 0
for (i < this.filterOpts.length; i++;) {
const filter = this.filterOpts[i]
Review Comment:
Maybe `for...of` would read better
##########
plugins/backup/nas/src/main/java/org/apache/cloudstack/backup/NASBackupProvider.java:
##########
@@ -208,6 +219,7 @@ private BackupVO createBackupObject(VirtualMachine vm, String backupPath) {
backup.setAccountId(vm.getAccountId());
backup.setDomainId(vm.getDomainId());
backup.setZoneId(vm.getDataCenterId());
+ backup.setName(vm.getHostName() + '-' + new SimpleDateFormat("yyyy-MM-dd'T'HH:mmX").format(new Date()));
Review Comment:
There is a DateUtil class. It can be used to format date if possible
##########
engine/schema/src/main/resources/META-INF/db/schema-42010to42100.sql:
##########
@@ -19,16 +19,34 @@
-- Schema upgrade from 4.20.1.0 to 4.21.0.0
--;
--- Add columns max_backup and backup_interval_type to backup table
-ALTER TABLE `cloud`.`backup_schedule` ADD COLUMN `max_backups` int(8) default NULL COMMENT 'maximum number of backups to maintain';
-ALTER TABLE `cloud`.`backups` ADD COLUMN `backup_interval_type` int(5) COMMENT 'type of backup, e.g. manual, recurring - hourly, daily, weekly or monthly';
-
-- Add console_endpoint_creator_address column to cloud.console_session table
CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.console_session', 'console_endpoint_creator_address', 'VARCHAR(45)');
-- Add client_address column to cloud.console_session table
CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.console_session', 'client_address', 'VARCHAR(45)');
+-- Allow default roles to use quotaCreditsList
+INSERT INTO `cloud`.`role_permissions` (uuid, role_id, rule, permission, sort_order)
+SELECT uuid(), role_id, 'quotaCreditsList', permission, sort_order
+FROM `cloud`.`role_permissions` rp
+WHERE rp.rule = 'quotaStatement'
+AND NOT EXISTS(SELECT 1 FROM cloud.role_permissions rp_ WHERE rp.role_id = rp_.role_id AND rp_.rule = 'quotaCreditsList');
+
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.host', 'last_mgmt_server_id', 'bigint unsigned DEFAULT NULL COMMENT "last management server this host is connected to" AFTER `mgmt_server_id`');
+
+-- Add column max_backup to backup_schedule table
+ALTER TABLE `cloud`.`backup_schedule` ADD COLUMN `max_backups` int(8) default NULL COMMENT 'maximum number of backups to maintain';
+
+-- Add columns name, description and backup_interval_type to backup table
+ALTER TABLE `cloud`.`backups` ADD COLUMN `backup_interval_type` int(5) COMMENT 'type of backup, e.g. manual, recurring - hourly, daily, weekly or monthly';
+ALTER TABLE `cloud`.`backups` ADD COLUMN `name` varchar(255) NOT NULL COMMENT 'name of the backup';
+ALTER TABLE `cloud`.`backups` ADD COLUMN `vm_name` varchar(255) COMMENT 'name of the vm for which backup is taken, only set for orphaned backups';
+ALTER TABLE `cloud`.`backups` ADD COLUMN `description` varchar(1024) COMMENT 'description for the backup';
Review Comment:
Maybe use `IDEMPOTENT_ADD_COLUMN` instead?
##########
plugins/backup/nas/src/main/java/org/apache/cloudstack/backup/NASBackupProvider.java:
##########
@@ -90,6 +94,19 @@ public class NASBackupProvider extends AdapterBase implements BackupProvider, Co
@Inject
BackupManager backupManager;
+ private Host getUpHostInZone(Long zoneId) {
Review Comment:
Not sure but we may already have a similar method in ResourceManager or somewhere to find a random Up host in zone
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2712763171
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2716354954
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2800699997
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2800941364
@abh1sar a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3112898152
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "JoaoJandre (via GitHub)" <gi...@apache.org>.
JoaoJandre commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2229202350
##########
server/src/main/java/org/apache/cloudstack/backup/BackupManagerImpl.java:
##########
@@ -926,6 +1058,224 @@ private Backup.VolumeInfo getVolumeInfo(List<Backup.VolumeInfo> backedUpVolumes,
return null;
}
+ @Override
+ public void checkVmDisksSizeAgainstBackup(List<VmDiskInfo> vmDiskInfoList, Backup backup) {
+ List<VmDiskInfo> vmDiskInfoListFromBackup = getDataDiskInfoListFromBackup(backup);
+ int index = 0;
+ if (vmDiskInfoList.size() != vmDiskInfoListFromBackup.size()) {
+ throw new InvalidParameterValueException("Unable to create Instance from Backup " +
+ "as the backup has a different number of disks than the Instance.");
+ }
+ for (VmDiskInfo vmDiskInfo : vmDiskInfoList) {
+ if (index < vmDiskInfoListFromBackup.size()) {
+ if (vmDiskInfo.getSize() < vmDiskInfoListFromBackup.get(index).getSize()) {
+ throw new InvalidParameterValueException(
+ String.format("Instance volume size %d[GiB] cannot be less than the backed-up volume size %d[GiB].",
+ vmDiskInfo.getSize(), vmDiskInfoListFromBackup.get(index).getSize()));
+ }
+ }
+ index++;
+ }
+ }
+
+ @Override
+ public VmDiskInfo getRootDiskInfoFromBackup(Backup backup) {
+ List<Backup.VolumeInfo> volumes = backup.getBackedUpVolumes();
+ VmDiskInfo rootDiskOffering = null;
+ if (volumes == null || volumes.isEmpty()) {
+ throw new CloudRuntimeException("Failed to get backed-up volumes info from backup");
+ }
+ for (Backup.VolumeInfo volume : volumes) {
+ if (volume.getType() == Volume.Type.ROOT) {
+ DiskOfferingVO diskOffering = diskOfferingDao.findByUuid(volume.getDiskOfferingId());
+ if (diskOffering == null) {
+ throw new CloudRuntimeException(String.format("Unable to find the root disk offering with uuid (%s) " +
+ "stored in backup. Please specify a valid root disk offering id while creating the instance",
+ volume.getDiskOfferingId()));
+ }
+ Long size = volume.getSize() / (1024 * 1024 * 1024);
+ rootDiskOffering = new VmDiskInfo(diskOffering, size, volume.getMinIops(), volume.getMaxIops());
+ }
+ }
+ if (rootDiskOffering == null) {
+ throw new CloudRuntimeException("Failed to get the root disk in backed-up volumes info from backup");
+ }
+ return rootDiskOffering;
+ }
+
+ @Override
+ public List<VmDiskInfo> getDataDiskInfoListFromBackup(Backup backup) {
+ List<VmDiskInfo> vmDiskInfoList = new ArrayList<>();
+ List<Backup.VolumeInfo> volumes = backup.getBackedUpVolumes();
+ if (volumes == null || volumes.isEmpty()) {
+ throw new CloudRuntimeException("Failed to get backed-up Volumes info from backup");
+ }
+ for (Backup.VolumeInfo volume : volumes) {
+ if (volume.getType() == Volume.Type.DATADISK) {
+ DiskOfferingVO diskOffering = diskOfferingDao.findByUuid(volume.getDiskOfferingId());
+ if (diskOffering == null || diskOffering.getState().equals(DiskOffering.State.Inactive)) {
+ throw new CloudRuntimeException("Unable to find the disk offering with uuid (" + volume.getDiskOfferingId() + ") stored in backup. " +
+ "Please specify a valid disk offering id while creating the instance");
+ }
+ Long size = volume.getSize() / (1024 * 1024 * 1024);
+ vmDiskInfoList.add(new VmDiskInfo(diskOffering, size, volume.getMinIops(), volume.getMaxIops(), volume.getDeviceId()));
+ }
+ }
+ return vmDiskInfoList;
+ }
+
+ private List<String> parseAddressString(String input) {
+ if (input == null) return null;
+ return Arrays.stream(input.split(","))
+ .map(s -> "null".equalsIgnoreCase(s.trim()) ? null : s.trim())
+ .collect(Collectors.toList());
+ }
+
+ @Override
+ public Map<Long, Network.IpAddresses> getIpToNetworkMapFromBackup(Backup backup, boolean preserveIps, List<Long> networkIds)
+ {
+ Map<Long, Network.IpAddresses> ipToNetworkMap = new LinkedHashMap<Long, Network.IpAddresses>();
+
+ String nicsJson = backup.getDetail(ApiConstants.NICS);
+ if (nicsJson == null) {
+ throw new CloudRuntimeException("Backup doesn't contain network information. " +
+ "Please specify at least one valid network while creating instance");
+ }
+
+ Type type = new TypeToken<List<Map<String, String>>>(){}.getType();
+ List<Map<String, String>> nics = new Gson().fromJson(nicsJson, type);
+
+ for (Map<String, String> nic : nics) {
+ String networkUuid = nic.get(ApiConstants.NETWORK_ID);
+ if (networkUuid == null) {
+ throw new CloudRuntimeException("Backup doesn't contain network information. " +
+ "Please specify at least one valid network while creating instance");
+ }
+
+ Network network = networkDao.findByUuid(networkUuid);
+ if (network == null) {
+ throw new CloudRuntimeException("Unable to find network with the uuid " + networkUuid + " stored in backup. " +
+ "Please specify a valid network id while creating the instance");
+ }
+
+ Long networkId = network.getId();
+ Network.IpAddresses ipAddresses = null;
+
+ if (preserveIps) {
+ String ip = nic.get(ApiConstants.IP_ADDRESS);
+ String ipv6 = nic.get(ApiConstants.IP6_ADDRESS);
+ String mac = nic.get(ApiConstants.MAC_ADDRESS);
+ ipAddresses = networkService.getIpAddressesFromIps(ip, ipv6, mac);
+ }
+
+ ipToNetworkMap.put(networkId, ipAddresses);
+ networkIds.add(networkId);
+ }
+ return ipToNetworkMap;
+ }
+
+ @Override
+ public Boolean canCreateInstanceFromBackup(final Long backupId) {
+ final BackupVO backup = backupDao.findById(backupId);
+ BackupOffering offering = backupOfferingDao.findByIdIncludingRemoved(backup.getBackupOfferingId());
+ if (offering == null) {
+ throw new CloudRuntimeException("Failed to find backup offering");
+ }
+ final BackupProvider backupProvider = getBackupProvider(offering.getProvider());
+ return backupProvider.supportsInstanceFromBackup();
+ }
+
+ @Override
+ public boolean restoreBackupToVM(final Long backupId, final Long vmId) throws ResourceUnavailableException {
+ final BackupVO backup = backupDao.findById(backupId);
+ if (backup == null) {
+ throw new CloudRuntimeException("Backup " + backupId + " does not exist");
+ }
+ if (backup.getStatus() != Backup.Status.BackedUp) {
+ throw new CloudRuntimeException("Backup should be in BackedUp state");
+ }
+ validateBackupForZone(backup.getZoneId());
+
+ VMInstanceVO vm = vmInstanceDao.findByIdIncludingRemoved(vmId);
+ if (vm == null) {
+ throw new CloudRuntimeException("Instance with ID " + backup.getVmId() + " couldn't be found.");
+ }
+ accountManager.checkAccess(CallContext.current().getCallingAccount(), null, true, vm);
+
+ if (vm.getRemoved() != null) {
+ throw new CloudRuntimeException("Instance with ID " + backup.getVmId() + " couldn't be found.");
+ }
+ if (!vm.getState().equals(VirtualMachine.State.Stopped)) {
+ throw new CloudRuntimeException("The VM should be in stopped state");
+ }
+
+ List<Backup.VolumeInfo> backupVolumes = backup.getBackedUpVolumes();
+ if (backupVolumes == null) {
+ throw new CloudRuntimeException("Backed up volumes info not found in the backup");
+ }
+
+ List<VolumeVO> vmVolumes = volumeDao.findByInstance(vmId);
+ if (vmVolumes.size() != backupVolumes.size()) {
+ throw new CloudRuntimeException("Unable to create Instance from backup as the backup has a different number of disks than the Instance");
+ }
+
+ int index = 0;
+ for (VolumeVO vmVolume: vmVolumes) {
+ Backup.VolumeInfo backupVolume = backupVolumes.get(index);
+ if (vmVolume.getSize() < backupVolume.getSize()) {
+ throw new CloudRuntimeException(String.format(
+ "Instance volume size %d[GiB] for volume (%s) is less than the backed-up volume size %d[GiB] for backed-up volume (%s).",
+ vmVolume.getSize(), vmVolume.getUuid(), backupVolume.getSize(), backupVolume.getUuid()));
+ }
+ index++;
+ }
+
+ BackupOffering offering = backupOfferingDao.findByIdIncludingRemoved(backup.getBackupOfferingId());
+ if (offering == null) {
+ throw new CloudRuntimeException("Failed to find backup offering");
+ }
+ final BackupProvider backupProvider = getBackupProvider(offering.getProvider());
+ if (!backupProvider.supportsInstanceFromBackup()) {
+ throw new CloudRuntimeException("Create instance from backup is not supported by the " + offering.getProvider() + " provider.");
+ }
+
+ String backupDetailsInMessage = ReflectionToStringBuilderUtils.reflectOnlySelectedFields(backup, "uuid", "externalId", "newVMId", "type", "status", "date");
+ Long eventId = null;
+ try {
+ updateVmState(vm, VirtualMachine.Event.RestoringRequested, VirtualMachine.State.Restoring);
+ updateVolumeState(vm, Volume.Event.RestoreRequested, Volume.State.Restoring);
+ eventId = ActionEventUtils.onStartedActionEvent(User.UID_SYSTEM, vm.getAccountId(), EventTypes.EVENT_VM_CREATE_FROM_BACKUP,
+ String.format("Creating Instance %s from backup %s", vm.getInstanceName(), backup.getUuid()),
+ vm.getId(), ApiCommandResourceType.VirtualMachine.toString(),
+ true, 0);
+
+ String host = null;
+ String dataStore = null;
+ if (!"nas".equals(offering.getProvider())) {
+ Pair<HostVO, StoragePoolVO> restoreInfo = getRestoreVolumeHostAndDatastore(vm);
+ host = restoreInfo.first().getPrivateIpAddress();
+ dataStore = restoreInfo.second().getUuid();
+ }
+ if (!backupProvider.restoreBackupToVM(vm, backup, host, dataStore)) {
+ throw new CloudRuntimeException(String.format("Error restoring backup [%s] to VM %s.", backupDetailsInMessage, vm.getUuid()));
+ }
+ } catch (Exception e) {
+ updateVolumeState(vm, Volume.Event.RestoreFailed, Volume.State.Ready);
+ updateVmState(vm, VirtualMachine.Event.RestoringFailed, VirtualMachine.State.Stopped);
Review Comment:
So if an exception happens the new VM is automatically expunged?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3121573497
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3125818747
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2230365467
##########
server/src/main/java/org/apache/cloudstack/backup/BackupManagerImpl.java:
##########
@@ -282,18 +319,75 @@ public boolean deleteBackupOffering(final Long offeringId) {
throw new CloudRuntimeException("Could not find a backup offering with id: " + offeringId);
}
- if (vmInstanceDao.listByZoneWithBackups(offering.getZoneId(), offering.getId()).size() > 0) {
+ if (backupDao.listByOfferingId(offering.getId()).size() > 0) {
+ throw new CloudRuntimeException("Backup Offering cannot be removed as it has backups associated with it.");
+ }
+
+ if (vmInstanceDao.listByZoneAndBackupOffering(offering.getZoneId(), offering.getId()).size() > 0) {
throw new CloudRuntimeException("Backup offering is assigned to VMs, remove the assignment(s) in order to remove the offering.");
}
- validateForZone(offering.getZoneId());
+ validateBackupForZone(offering.getZoneId());
return backupOfferingDao.remove(offering.getId());
}
- public static String createVolumeInfoFromVolumes(List<VolumeVO> vmVolumes) {
+ @Override
+ public Map<String, String> getBackupDetailsFromVM(VirtualMachine vm) {
+ HashMap<String, String> details = new HashMap<>();
+
+ ServiceOffering serviceOffering = serviceOfferingDao.findById(vm.getServiceOfferingId());
+ details.put(ApiConstants.SERVICE_OFFERING_ID, serviceOffering.getUuid());
+ VirtualMachineTemplate template = vmTemplateDao.findById(vm.getTemplateId());
+ details.put(ApiConstants.TEMPLATE_ID, template.getUuid());
+
+ List<VMInstanceDetailVO> vmDetails = vmInstanceDetailsDao.listDetails(vm.getId());
+ HashMap<String, String> settings = new HashMap<>();
+ for (VMInstanceDetailVO detail : vmDetails) {
+ settings.put(detail.getName(), detail.getValue());
+ }
+ if (!settings.isEmpty()) {
+ details.put(ApiConstants.VM_SETTINGS, new Gson().toJson(settings));
+ }
+
+ List<UserVmJoinVO> userVmJoinVOs = userVmJoinDao.searchByIds(vm.getId());
+ if (userVmJoinVOs != null && !userVmJoinVOs.isEmpty()) {
+ List<Map<String, String>> nics = new ArrayList<>();
+ Set<String> seen = new HashSet<>();
+
+ for (UserVmJoinVO userVmJoinVO : userVmJoinVOs) {
+ Map<String, String> nicInfo = new HashMap<>();
+ String key = userVmJoinVO.getNetworkUuid();
+ if (seen.add(key)) {
+ nicInfo.put(ApiConstants.NETWORK_ID, userVmJoinVO.getNetworkUuid());
+ nicInfo.put(ApiConstants.IP_ADDRESS, userVmJoinVO.getIpAddress());
+ nicInfo.put(ApiConstants.IP6_ADDRESS, userVmJoinVO.getIp6Address());
+ nicInfo.put(ApiConstants.MAC_ADDRESS, userVmJoinVO.getMacAddress());
+ nics.add(nicInfo);
+ }
+ }
+
+ if (!nics.isEmpty()) {
+ String nicJson = new Gson().toJson(nics);
+ details.put("nics", nicJson);
+ }
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3121611427
Packaging result [SF]: ✔️ el8 ✔️ el9 ✖️ debian ✔️ suse15. SL-JID 14394
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3123836025
<b>[SF] Trillian test result (tid-13916)</b>
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 50230 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10140-t13916-kvm-ol8.zip
Smoke tests completed. 143 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3096260951
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2964804277
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2964556019
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2987496152
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13840
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3111894250
@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3134859804
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3096265056
@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3113404496
@sureshanaparti a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3116825012
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3105342474
<b>[SF] Trillian test result (tid-13840)</b>
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 48825 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10140-t13840-kvm-ol8.zip
Smoke tests completed. 142 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_attach_and_distribute_multiple_volumes | `Error` | 32.16 | test_attach_multiple_volumes.py
test_attach_multiple_volumes | `Failure` | 2.51 | test_attach_multiple_volumes.py
test_attach_multiple_volumes | `Error` | 2.51 | test_attach_multiple_volumes.py
ContextSuite context=TestMultipleVolumeAttach>:teardown | `Error` | 2.60 | test_attach_multiple_volumes.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "sureshanaparti (via GitHub)" <gi...@apache.org>.
sureshanaparti commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3138800157
@abh1sar can you check the conflicts
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3131875883
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "JoaoJandre (via GitHub)" <gi...@apache.org>.
JoaoJandre commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2205604506
##########
engine/schema/src/main/java/org/apache/cloudstack/backup/BackupVO.java:
##########
@@ -240,10 +268,32 @@ public void setBackedUpVolumes(String backedUpVolumes) {
this.backedUpVolumes = backedUpVolumes;
}
+ @Override
+ public Map<String, String> getDetails() {
+ return details;
+ }
+
+ public void setDetail(String name, String value) {
+ assert (details != null) : "Did you forget to load the details?";
Review Comment:
Using an if is better here.
##########
api/src/main/java/org/apache/cloudstack/backup/BackupProvider.java:
##########
@@ -110,9 +111,28 @@ public interface BackupProvider {
/**
* This method should TODO
Review Comment:
Please describe what this method should do.
##########
api/src/main/java/org/apache/cloudstack/api/command/admin/vm/CreateVMFromBackupCmdByAdmin.java:
##########
@@ -0,0 +1,55 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.api.command.admin.vm;
+
+import org.apache.cloudstack.acl.RoleType;
+import org.apache.cloudstack.api.APICommand;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.api.Parameter;
+import org.apache.cloudstack.api.ResponseObject;
+import org.apache.cloudstack.api.command.admin.AdminCmd;
+import org.apache.cloudstack.api.command.user.vm.CreateVMFromBackupCmd;
+import org.apache.cloudstack.api.response.ClusterResponse;
+import org.apache.cloudstack.api.response.PodResponse;
+import org.apache.cloudstack.api.response.UserVmResponse;
+
+import com.cloud.vm.VirtualMachine;
+
+@APICommand(name = "createVMFromBackup",
+ description = "Creates and automatically starts a VM from a backup.",
+ responseObject = UserVmResponse.class,
+ responseView = ResponseObject.ResponseView.Full,
+ entityType = {VirtualMachine.class},
+ requestHasSensitiveInfo = false, responseHasSensitiveInfo = true,
+ since = "4.21.0",
+ authorized = {RoleType.Admin, RoleType.ResourceAdmin, RoleType.DomainAdmin, RoleType.User})
+public class CreateVMFromBackupCmdByAdmin extends CreateVMFromBackupCmd implements AdminCmd {
+
+ @Parameter(name = ApiConstants.POD_ID, type = CommandType.UUID, entityType = PodResponse.class, description = "destination Pod ID to deploy the VM to - parameter available for root admin only", since = "4.13")
+ private Long podId;
+
+ @Parameter(name = ApiConstants.CLUSTER_ID, type = CommandType.UUID, entityType = ClusterResponse.class, description = "destination Cluster ID to deploy the VM to - parameter available for root admin only", since = "4.13")
Review Comment:
```suggestion
@Parameter(name = ApiConstants.CLUSTER_ID, type = CommandType.UUID, entityType = ClusterResponse.class, description = "destination Cluster ID to deploy the VM to - parameter available for root admin only", since = "4.21")
```
##########
api/src/main/java/org/apache/cloudstack/api/command/user/vm/CreateVMFromBackupCmd.java:
##########
@@ -0,0 +1,159 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.api.command.user.vm;
+
+import javax.inject.Inject;
+
+import org.apache.cloudstack.acl.RoleType;
+import org.apache.cloudstack.api.ACL;
+import org.apache.cloudstack.api.APICommand;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.api.ApiErrorCode;
+import org.apache.cloudstack.api.Parameter;
+import org.apache.cloudstack.api.ResponseObject;
+import org.apache.cloudstack.api.ServerApiException;
+import org.apache.cloudstack.api.response.BackupResponse;
+import org.apache.cloudstack.api.response.ServiceOfferingResponse;
+import org.apache.cloudstack.api.response.TemplateResponse;
+import org.apache.cloudstack.api.response.UserVmResponse;
+import org.apache.cloudstack.backup.BackupManager;
+
+import com.cloud.exception.ConcurrentOperationException;
+import com.cloud.exception.InsufficientCapacityException;
+import com.cloud.exception.InsufficientServerCapacityException;
+import com.cloud.exception.ResourceAllocationException;
+import com.cloud.exception.ResourceUnavailableException;
+import com.cloud.uservm.UserVm;
+import com.cloud.vm.VirtualMachine;
+
+@APICommand(name = "createVMFromBackup",
+ description = "Creates and automatically starts a VM from a backup.",
+ responseObject = UserVmResponse.class,
+ responseView = ResponseObject.ResponseView.Restricted,
+ entityType = {VirtualMachine.class},
+ requestHasSensitiveInfo = false, responseHasSensitiveInfo = true,
+ since = "4.21.0",
+ authorized = {RoleType.Admin, RoleType.ResourceAdmin, RoleType.DomainAdmin, RoleType.User})
+public class CreateVMFromBackupCmd extends BaseDeployVMCmd {
+
+ @Inject
+ BackupManager backupManager;
Review Comment:
Is this used in this cmd?
##########
api/src/main/java/org/apache/cloudstack/api/command/user/backup/ListBackupsCmd.java:
##########
@@ -87,10 +107,22 @@ public Long getVmId() {
return vmId;
}
+ public String getName() {
+ return name;
+ }
+
+ public Long getBackupOfferingId() {
+ return backupOfferingId;
Review Comment:
are these methods used anywhere?
##########
api/src/main/java/com/cloud/offering/DiskOfferingInfo.java:
##########
@@ -23,6 +23,7 @@ public class DiskOfferingInfo {
private Long _size;
private Long _minIops;
private Long _maxIops;
+ private Long _deviceId;
Review Comment:
A disk offering should not have a deviceId tied to it, don't you agree?
##########
api/src/main/java/org/apache/cloudstack/backup/BackupProvider.java:
##########
@@ -110,9 +111,28 @@ public interface BackupProvider {
/**
* This method should TODO
+ *
* @param restorePoint the restore point to create a backup for
- * @param vm The machine for which to create a backup
- * @param metric the metric object to update with the new backup data
+ * @param vm The machine for which to create a backup
+ */
+ Backup createNewBackupEntryForRestorePoint(Backup.RestorePoint restorePoint, VirtualMachine vm);
+
+ /**
+ * Returns if the backup provider supports creating new instance from backup
*/
- Backup createNewBackupEntryForRestorePoint(Backup.RestorePoint restorePoint, VirtualMachine vm, Backup.Metric metric);
+ boolean supportsInstanceFromBackup();
+
+ /**
+ * Returns the backup storage usage (Used, Total) for a backup provider
+ * @param zoneId the zone for which to return metrics
+ * @return a pair of Used size and Total size for the backup storage
+ */
+ Pair<Long, Long> getBackupStorageStats(Long zoneId);
Review Comment:
How is this related to VM creation from backup?
##########
api/src/main/java/org/apache/cloudstack/backup/BackupProvider.java:
##########
@@ -110,9 +111,28 @@ public interface BackupProvider {
/**
* This method should TODO
+ *
* @param restorePoint the restore point to create a backup for
- * @param vm The machine for which to create a backup
- * @param metric the metric object to update with the new backup data
+ * @param vm The machine for which to create a backup
+ */
+ Backup createNewBackupEntryForRestorePoint(Backup.RestorePoint restorePoint, VirtualMachine vm);
+
+ /**
+ * Returns if the backup provider supports creating new instance from backup
*/
- Backup createNewBackupEntryForRestorePoint(Backup.RestorePoint restorePoint, VirtualMachine vm, Backup.Metric metric);
+ boolean supportsInstanceFromBackup();
+
+ /**
+ * Returns the backup storage usage (Used, Total) for a backup provider
+ * @param zoneId the zone for which to return metrics
+ * @return a pair of Used size and Total size for the backup storage
+ */
+ Pair<Long, Long> getBackupStorageStats(Long zoneId);
+
+ /**
+ * Gets the backup storage usage (Used, Total) from the plugin and stores it in db
+ * @param zoneId the zone for which to return metrics
+ */
+ void syncBackupStorageStats(Long zoneId);
Review Comment:
How is this related to VM creation from backup?
##########
api/src/main/java/org/apache/cloudstack/api/command/user/backup/CreateBackupCmd.java:
##########
@@ -62,13 +62,34 @@ public class CreateBackupCmd extends BaseAsyncCreateCmd {
description = "ID of the VM")
private Long vmId;
+ @Parameter(name = ApiConstants.NAME,
+ type = CommandType.STRING,
+ description = "the name of the backup",
+ since = "4.21.0")
+ private String name;
+
+ @Parameter(name = ApiConstants.DESCRIPTION,
+ type = CommandType.STRING,
+ description = "the description for the backup",
+ since = "4.21.0")
+ private String description;
+
@Parameter(name = ApiConstants.SCHEDULE_ID,
type = CommandType.LONG,
entityType = BackupScheduleResponse.class,
description = "backup schedule ID of the VM, if this is null, it indicates that it is a manual backup.",
since = "4.21.0")
private Long scheduleId;
+ @Parameter(name = ApiConstants.QUIESCE_VM,
+ type = CommandType.BOOLEAN,
+ required = false,
+ description = "Quiesce the instance before checkpointing the disks for backup. Applicable only to NAS backup provider. " +
+ "The filesystem is frozen before the backup starts and thawed immediately after. " +
+ "Requires the instance to have the QEMU Guest Agent installed and running.",
+ since = "4.21.0")
+ private Boolean quiesceVM;
Review Comment:
Shouldn't this change be in a different PR?
##########
api/src/main/java/org/apache/cloudstack/api/command/admin/vm/CreateVMFromBackupCmdByAdmin.java:
##########
@@ -0,0 +1,55 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.api.command.admin.vm;
+
+import org.apache.cloudstack.acl.RoleType;
+import org.apache.cloudstack.api.APICommand;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.api.Parameter;
+import org.apache.cloudstack.api.ResponseObject;
+import org.apache.cloudstack.api.command.admin.AdminCmd;
+import org.apache.cloudstack.api.command.user.vm.CreateVMFromBackupCmd;
+import org.apache.cloudstack.api.response.ClusterResponse;
+import org.apache.cloudstack.api.response.PodResponse;
+import org.apache.cloudstack.api.response.UserVmResponse;
+
+import com.cloud.vm.VirtualMachine;
+
+@APICommand(name = "createVMFromBackup",
+ description = "Creates and automatically starts a VM from a backup.",
+ responseObject = UserVmResponse.class,
+ responseView = ResponseObject.ResponseView.Full,
+ entityType = {VirtualMachine.class},
+ requestHasSensitiveInfo = false, responseHasSensitiveInfo = true,
+ since = "4.21.0",
+ authorized = {RoleType.Admin, RoleType.ResourceAdmin, RoleType.DomainAdmin, RoleType.User})
+public class CreateVMFromBackupCmdByAdmin extends CreateVMFromBackupCmd implements AdminCmd {
+
+ @Parameter(name = ApiConstants.POD_ID, type = CommandType.UUID, entityType = PodResponse.class, description = "destination Pod ID to deploy the VM to - parameter available for root admin only", since = "4.13")
Review Comment:
```suggestion
@Parameter(name = ApiConstants.POD_ID, type = CommandType.UUID, entityType = PodResponse.class, description = "destination Pod ID to deploy the VM to - parameter available for root admin only", since = "4.21")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3072561219
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3082241103
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2837998478
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2850586622
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2876329932
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2036791427
##########
api/src/main/java/org/apache/cloudstack/api/command/admin/storage/AddObjectStoragePoolCmd.java:
##########
@@ -56,6 +56,9 @@ public class AddObjectStoragePoolCmd extends BaseCmd {
@Parameter(name = ApiConstants.TAGS, type = CommandType.STRING, description = "the tags for the storage pool")
private String tags;
+ @Parameter(name = ApiConstants.SIZE, type = CommandType.LONG, description = "the total size of the object store in GiB. Used for tracking capacity and sending alerts", since = "4.21")
Review Comment:
Yes, Object storage plugins don't have a proper way to return total capacity. It's for admin to set and track.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2036948340
##########
engine/schema/src/main/java/com/cloud/vm/dao/VMInstanceDaoImpl.java:
##########
@@ -1224,4 +1224,14 @@ public Map<String, Long> getNameIdMapForVmIds(Collection<Long> ids) {
return vms.stream()
.collect(Collectors.toMap(VMInstanceVO::getInstanceName, VMInstanceVO::getId));
}
+
+ @Override
+ public List<VMInstanceVO> listByIds(List<Long> ids) {
Review Comment:
Right, I'll rename this one to listByIdsIncludingRemoved
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2792328923
> this PR contains 3 changes
>
> * create new vm from backup
> * threshold for object storage
> * threshold for backup
>
> I would prefer to see 3 PRs (at least 2 PRs) but it looks difficult to seperate them now
It's not ideal, but it was done this way so test effort could be minimised.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
weizhouapache commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2037141264
##########
plugins/backup/veeam/src/main/java/org/apache/cloudstack/backup/VeeamBackupProvider.java:
##########
@@ -213,7 +214,7 @@ public boolean removeVMFromBackupOffering(final VirtualMachine vm) {
@Override
public boolean willDeleteBackupsOnOfferingRemoval() {
- return true;
+ return false;
Review Comment:
good, thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [WIP] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2585671789
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r1918196798
##########
server/src/main/java/org/apache/cloudstack/backup/BackupManagerImpl.java:
##########
@@ -748,6 +819,151 @@ private Backup.VolumeInfo getVolumeInfo(List<Backup.VolumeInfo> backedUpVolumes,
return null;
}
+ @Override
+ public void updateDiskOfferingSizeFromBackup(List<DiskOfferingInfo> dataDiskOfferingsInfo, Backup backup) {
+ List<DiskOfferingInfo> dataDiskOfferingsInfoFromBackup = getDataDiskOfferingListFromBackup(backup);
+ int index = 0;
+ for(DiskOfferingInfo diskOfferingInfo : dataDiskOfferingsInfo) {
+ diskOfferingInfo.setSize(Math.max(diskOfferingInfo.getSize(), dataDiskOfferingsInfoFromBackup.get(index).getSize()));
+ index++;
+ }
+ }
+
+ @Override
+ public DiskOfferingInfo getRootDiskOfferingInfoFromBackup(Backup backup) {
+ String diskOfferingIdsDetail = backup.getDetail(ApiConstants.DISK_OFFERING_IDS);
+ if (diskOfferingIdsDetail == null) {
+ return null;
+ }
+ String [] diskOfferingIds = diskOfferingIdsDetail.split(",");
+ String [] deviceIds = backup.getDetail(ApiConstants.DEVICE_IDS).split(",");
+ String [] diskSizes = backup.getDetail(ApiConstants.DISK_SIZES).split(",");
+
+ for (int i = 0; i < diskOfferingIds.length; i++) {
+ if (deviceIds[i].equals("0")) {
+ DiskOfferingVO diskOffering = diskOfferingDao.findByUuid(diskOfferingIds[i]);
+ if (diskOffering == null) {
+ throw new CloudRuntimeException("Unable to find the root disk offering with uuid (" + diskOfferingIds[i] + ") stored in backup. Please specify a valid root disk offering id while creating the instance");
+ }
+ Long size = Long.parseLong(diskSizes[i]) / (1024 * 1024 * 1024);
+ return new DiskOfferingInfo(diskOffering, size, null, null, 0L);
+ }
+ }
+ return null;
+ }
+
+ @Override
+ public List<DiskOfferingInfo> getDataDiskOfferingListFromBackup(Backup backup) {
+ String diskOfferingIdsDetail = backup.getDetail(ApiConstants.DISK_OFFERING_IDS);
+ if (diskOfferingIdsDetail == null) {
+ return null;
+ }
+
+ String [] diskOfferingIds = diskOfferingIdsDetail.split(",");
+ String [] deviceIds = backup.getDetail(ApiConstants.DEVICE_IDS).split(",");
+ String [] diskSizes = backup.getDetail(ApiConstants.DISK_SIZES).split(",");
+ String [] minIopsList = backup.getDetail(ApiConstants.MIN_IOPS).split(",");
+ String [] maxIopsList = backup.getDetail(ApiConstants.MAX_IOPS).split(",");
+
+ List<DiskOfferingInfo> diskOfferingInfoList = new ArrayList<>();
+ for (int i = 0; i < diskOfferingIds.length; i++) {
+ Long deviceId = Long.parseLong(deviceIds[i]);
+ if (deviceId == 0) {
+ continue;
+ }
+ DiskOfferingVO diskOffering = diskOfferingDao.findByUuid(diskOfferingIds[i]);
Review Comment:
Will test it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r1918175515
##########
ui/src/views/compute/DeployVMFromBackup.vue:
##########
@@ -0,0 +1,2211 @@
+// Licensed to the Apache Software Foundation (ASF) under one
Review Comment:
@shwstppr I had put this here as it is similar to DeployVM.vue but I guess components make more sense as it is being used as a component
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2760831529
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [WIP] Create new Instance from VM backup [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2577831769
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3073036199
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3125871618
> > [SF] Trillian test result (tid-13919) Environment: xcpng82 (x2), Advanced Networking with Mgmt server ol9 Total time taken: 76201 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10140-t13919-xcpng82.zip Smoke tests completed. 137 look OK, 6 have errors, 0 did not run Only failed and skipped tests results shown below:
> > Test Result Time (s) Test File
> > test_01_events_resource `Error` 233.26 test_events_resource.py
> > ContextSuite context=TestClusterDRS>:setup `Error` 0.00 test_cluster_drs.py
> > test_01_non_strict_host_anti_affinity `Error` 230.06 test_nonstrict_affinity_group.py
> > test_02_non_strict_host_affinity `Error` 67.24 test_nonstrict_affinity_group.py
> > test_01_vpn_usage `Error` 1.12 test_usage.py
> > test_11_destroy_vm_and_volumes `Error` 21.05 test_vm_life_cycle.py
> > test_01_migrate_vm_strict_tags_success `Error` 61.50 test_vm_strict_host_tags.py
>
> @abh1sar can you check these test failures
@sureshanaparti
The test failure in xcpng seem unrelated to this PR.
test-events-resource.py and test-vm-lifecycle.py - `Failed to attach volume for uuid: c63954f2-d739-4c68-bd6c-83c40eaae802 due to You attempted an operation on a VM which requires PV drivers to be installed but the drivers were not detected.`
test_nonstrinct_affinity_group.py and test_vm_Strict_host_tags.py - `'org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd', jobprocstatus : 0, jobresultcode : 530, jobresulttype : 'object', jobresult : {errorcode : 530, errortext : 'Unable to migrate VM(i-2-446-VM) from host(9cbb2d75-41f3-4a72-94bd-09ece8740e16)'}`
test_usage.py is failing on #11143 also with the same error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3131359283
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2978960856
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2227283797
##########
engine/schema/src/main/java/org/apache/cloudstack/backup/dao/BackupDaoImpl.java:
##########
@@ -193,27 +260,61 @@ public List<BackupVO> listBackupsByVMandIntervalType(Long vmId, Backup.Type back
}
@Override
- public BackupResponse newBackupResponse(Backup backup) {
+ public void loadDetails(BackupVO backup) {
+ Map<String, String> details = backupDetailsDao.listDetailsKeyPairs(backup.getId());
+ backup.setDetails(details);
+ }
+
+ @Override
+ public void saveDetails(BackupVO backup) {
+ Map<String, String> detailsStr = backup.getDetails();
+ if (detailsStr == null) {
+ return;
+ }
+ List<BackupDetailVO> details = new ArrayList<BackupDetailVO>();
+ for (String key : detailsStr.keySet()) {
+ BackupDetailVO detail = new BackupDetailVO(backup.getId(), key, detailsStr.get(key), true);
+ details.add(detail);
+ }
+ backupDetailsDao.saveDetails(details);
+ }
+
+ @Override
+ public List<Long> listVmIdsWithBackupsInZone(Long zoneId) {
+ SearchCriteria<Long> sc = backupVmSearchInZone.create();
+ sc.setParameters("zone_id", zoneId);
+ return customSearchIncludingRemoved(sc, null);
+ }
+
+ @Override
+ public BackupResponse newBackupResponse(Backup backup, Boolean listVmDetails) {
VMInstanceVO vm = vmInstanceDao.findByIdIncludingRemoved(backup.getVmId());
- AccountVO account = accountDao.findByIdIncludingRemoved(vm.getAccountId());
- DomainVO domain = domainDao.findByIdIncludingRemoved(vm.getDomainId());
- DataCenterVO zone = dataCenterDao.findByIdIncludingRemoved(vm.getDataCenterId());
+ AccountVO account = accountDao.findByIdIncludingRemoved(backup.getAccountId());
+ DomainVO domain = domainDao.findByIdIncludingRemoved(backup.getDomainId());
+ DataCenterVO zone = dataCenterDao.findByIdIncludingRemoved(backup.getZoneId());
Long offeringId = backup.getBackupOfferingId();
- if (offeringId == null) {
- offeringId = vm.getBackupOfferingId();
- }
BackupOffering offering = backupOfferingDao.findByIdIncludingRemoved(offeringId);
BackupResponse response = new BackupResponse();
response.setId(backup.getUuid());
- response.setVmId(vm.getUuid());
+ response.setName(backup.getName());
+ response.setDescription(backup.getDescription());
response.setVmName(vm.getHostName());
+ if (vm.getState() != VirtualMachine.State.Expunging) {
Review Comment:
not required. have reverted this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2227283153
##########
engine/schema/src/main/resources/META-INF/db/schema-42010to42100.sql:
##########
@@ -203,3 +199,91 @@ SET `sort_key` = CASE
ELSE `sort_key`
END;
-- End: Changes for Guest OS category cleanup
+
+-- Add column max_backup to backup_schedule table
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backup_schedule', 'max_backups', 'int(8) default NULL COMMENT "maximum number of backups to maintain"');
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backup_schedule', 'quiescevm', 'tinyint(1) default NULL COMMENT "Quiesce VM before taking backup"');
+
+-- Add columns name, description and backup_interval_type to backup table
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backups', 'name', 'VARCHAR(255) NULL COMMENT "name of the backup"');
+UPDATE `cloud`.`backups` backup INNER JOIN `cloud`.`vm_instance` vm ON backup.vm_id = vm.id SET backup.name = vm.name;
+ALTER TABLE `cloud`.`backups` MODIFY COLUMN `name` VARCHAR(255) NOT NULL;
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backups', 'description', 'VARCHAR(1024) COMMENT "description for the backup"');
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backups', 'backup_interval_type', 'int(5) COMMENT "type of backup, e.g. manual, recurring - hourly, daily, weekly or monthly"');
+
+-- Make the column vm_id in backups table nullable to handle orphan backups
+ALTER TABLE `cloud`.`backups` MODIFY COLUMN `vm_id` BIGINT UNSIGNED NULL;
Review Comment:
Yeah. I was earlier trying to handle when vm record is purged, but dropped the idea soon after. I think I missed to revert this code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2227284661
##########
engine/schema/src/main/java/org/apache/cloudstack/backup/BackupVO.java:
##########
@@ -240,10 +268,32 @@ public void setBackedUpVolumes(String backedUpVolumes) {
this.backedUpVolumes = backedUpVolumes;
}
+ @Override
+ public Map<String, String> getDetails() {
+ return details;
+ }
+
+ public void setDetail(String name, String value) {
+ assert (details != null) : "Did you forget to load the details?";
Review Comment:
Point taken. the setDetail method was not being used anywhere, so have removed it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3112252672
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14336
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3107577547
@abh1sar a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2230377579
##########
engine/schema/src/main/java/org/apache/cloudstack/backup/dao/BackupDaoImpl.java:
##########
@@ -193,27 +272,97 @@ public List<BackupVO> listBackupsByVMandIntervalType(Long vmId, Backup.Type back
}
@Override
- public BackupResponse newBackupResponse(Backup backup) {
+ public void loadDetails(BackupVO backup) {
+ Map<String, String> details = backupDetailsDao.listDetailsKeyPairs(backup.getId());
+ backup.setDetails(details);
+ }
+
+ @Override
+ public void saveDetails(BackupVO backup) {
+ Map<String, String> detailsStr = backup.getDetails();
+ if (detailsStr == null) {
+ return;
+ }
+ List<BackupDetailVO> details = new ArrayList<BackupDetailVO>();
+ for (String key : detailsStr.keySet()) {
+ BackupDetailVO detail = new BackupDetailVO(backup.getId(), key, detailsStr.get(key), true);
+ details.add(detail);
+ }
+ backupDetailsDao.saveDetails(details);
+ }
+
+ @Override
+ public List<Long> listVmIdsWithBackupsInZone(Long zoneId) {
+ SearchCriteria<Long> sc = backupVmSearchInZone.create();
+ sc.setParameters("zone_id", zoneId);
+ return customSearchIncludingRemoved(sc, null);
+ }
+
+ Map<String, String> getDetailsFromBackupDetails(Long backupId) {
Review Comment:
This does look weird.
I have moved the createBackupResponse method to backupManager.
##########
engine/schema/src/main/java/org/apache/cloudstack/backup/BackupVO.java:
##########
@@ -240,10 +268,27 @@ public void setBackedUpVolumes(String backedUpVolumes) {
this.backedUpVolumes = backedUpVolumes;
}
+ @Override
+ public Map<String, String> getDetails() {
+ return details;
+ }
+
+ @Override
+ public String getDetail(String name) {
+ return this.details.get(name);
+ }
+
+ public void setDetails(Map<String, String> details) {
+ this.details = details;
+ }
+
+ public void addDetails(Map<String, String> details) {
+ this.details.putAll(details);
+ }
Review Comment:
removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3117140111
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14362
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3138900790
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3126255792
@sureshanaparti a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3131367713
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3021389223
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13959
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3026528194
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13977
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3021633706
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13960
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3027770836
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3038944866
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3048340575
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3095579886
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3095669645
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3095940764
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "GutoVeronezi (via GitHub)" <gi...@apache.org>.
GutoVeronezi commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3070548858
> While I agree that ideally these should be done as a separate PR, but doing so at this point of time would require significant bandwidth, both on my part and for reviewers who have already reviewed and tested the PR.
It demands efforts to keep the project's health, and introducing changes this way is unhealthy for the project. This approach is harmful, and I am against it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3082401411
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14220
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3106340690
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2225061978
##########
api/src/main/java/org/apache/cloudstack/backup/BackupProvider.java:
##########
@@ -110,9 +111,28 @@ public interface BackupProvider {
/**
* This method should TODO
+ *
* @param restorePoint the restore point to create a backup for
- * @param vm The machine for which to create a backup
- * @param metric the metric object to update with the new backup data
+ * @param vm The machine for which to create a backup
+ */
+ Backup createNewBackupEntryForRestorePoint(Backup.RestorePoint restorePoint, VirtualMachine vm);
+
+ /**
+ * Returns if the backup provider supports creating new instance from backup
*/
- Backup createNewBackupEntryForRestorePoint(Backup.RestorePoint restorePoint, VirtualMachine vm, Backup.Metric metric);
+ boolean supportsInstanceFromBackup();
+
+ /**
+ * Returns the backup storage usage (Used, Total) for a backup provider
+ * @param zoneId the zone for which to return metrics
+ * @return a pair of Used size and Total size for the backup storage
+ */
+ Pair<Long, Long> getBackupStorageStats(Long zoneId);
+
+ /**
+ * Gets the backup storage usage (Used, Total) from the plugin and stores it in db
+ * @param zoneId the zone for which to return metrics
+ */
+ void syncBackupStorageStats(Long zoneId);
Review Comment:
Please see the reply above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-3106847743
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2036812654
##########
api/src/main/java/org/apache/cloudstack/api/command/user/vm/CreateVMFromBackupCmd.java:
##########
@@ -0,0 +1,127 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.api.command.user.vm;
+
+import javax.inject.Inject;
+
+import org.apache.cloudstack.acl.RoleType;
+import org.apache.cloudstack.api.APICommand;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.api.ApiErrorCode;
+import org.apache.cloudstack.api.Parameter;
+import org.apache.cloudstack.api.ResponseObject;
+import org.apache.cloudstack.api.ServerApiException;
+import org.apache.cloudstack.api.command.user.UserCmd;
+import org.apache.cloudstack.api.response.BackupResponse;
+import org.apache.cloudstack.api.response.UserVmResponse;
+import org.apache.cloudstack.backup.BackupManager;
+
+import com.cloud.exception.ConcurrentOperationException;
+import com.cloud.exception.InsufficientCapacityException;
+import com.cloud.exception.InsufficientServerCapacityException;
+import com.cloud.exception.ResourceAllocationException;
+import com.cloud.exception.ResourceUnavailableException;
+import com.cloud.uservm.UserVm;
+import com.cloud.vm.VirtualMachine;
+
+@APICommand(name = "createVMFromBackup",
+ description = "Creates and automatically starts a VM from a backup.",
Review Comment:
yes, it does.
But it won't be like the new Instance is never started.
This command will :
1. Create the new VM
2. Start it, so that volumes are created
3. Stop it
4. Restore the backed up data
5. Start the VM (or not start it if `start` parameter is set to false.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2036813586
##########
api/src/main/java/org/apache/cloudstack/api/command/user/vm/DeployVMCmd.java:
##########
@@ -147,6 +148,13 @@ public class DeployVMCmd extends BaseAsyncCreateCustomIdCmd implements SecurityG
since = "4.4")
private Long rootdisksize;
+ @Parameter(name = ApiConstants.DATADISKS_DETAILS,
+ type = CommandType.MAP,
+ since = "4.21.0",
+ description = "Disk offering details for creating multiple data volumes. Mutually exclusibe with diskOfferingId." +
+ " Example: datadisksdetails[0].diskofferingid=1&datadisksdetails[0].size=10&datadisksdetails[0].miniops=100&datadisksdetails[0].maxiops=200")
Review Comment:
yes, its uuid. will make the change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2037010182
##########
plugins/backup/dummy/src/main/java/org/apache/cloudstack/backup/DummyBackupProvider.java:
##########
@@ -35,12 +37,14 @@
import com.cloud.vm.VirtualMachine;
public class DummyBackupProvider extends AdapterBase implements BackupProvider {
-
+ private static final Logger LOG = LogManager.getLogger(DummyBackupProvider.class);
Review Comment:
naming convention says `private static final` means all capitals, @weizhouapache . I think this is ok, unless we remove the `static` part.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
weizhouapache commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2037024761
##########
plugins/backup/dummy/src/main/java/org/apache/cloudstack/backup/DummyBackupProvider.java:
##########
@@ -118,7 +110,7 @@ public boolean removeVMFromBackupOffering(VirtualMachine vm) {
@Override
public boolean willDeleteBackupsOnOfferingRemoval() {
- return true;
+ return false;
Review Comment:
thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "abh1sar (via GitHub)" <gi...@apache.org>.
abh1sar commented on code in PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#discussion_r2037335914
##########
engine/schema/src/main/resources/META-INF/db/schema-42010to42100.sql:
##########
@@ -37,3 +33,38 @@ WHERE rp.rule = 'quotaStatement'
AND NOT EXISTS(SELECT 1 FROM cloud.role_permissions rp_ WHERE rp.role_id = rp_.role_id AND rp_.rule = 'quotaCreditsList');
CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.host', 'last_mgmt_server_id', 'bigint unsigned DEFAULT NULL COMMENT "last management server this host is connected to" AFTER `mgmt_server_id`');
+
+-- Add column max_backup to backup_schedule table
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backup_schedule', 'max_backups', 'int(8) default NULL COMMENT "maximum number of backups to maintain"');
+
+-- Add columns name, description and backup_interval_type to backup table
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backups', 'name', 'VARCHAR(255) NOT NULL COMMENT "name of the backup"');
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backups', 'description', 'VARCHAR(1024) COMMENT "description for the backup"');
+CALL `cloud`.`IDEMPOTENT_ADD_COLUMN`('cloud.backups', 'backup_interval_type', 'int(5) COMMENT "type of backup, e.g. manual, recurring - hourly, daily, weekly or monthly"');
+
+-- Make the column vm_id in backups table nullable to handle orphan backups
+ALTER TABLE `cloud`.`backups` MODIFY COLUMN `vm_id` BIGINT UNSIGNED NULL;
+
+-- Create backup details table
+CREATE TABLE `cloud`.`backup_details` (
+ `id` bigint unsigned NOT NULL auto_increment,
+ `backup_id` bigint unsigned NOT NULL COMMENT 'backup id',
+ `name` varchar(255) NOT NULL,
+ `value` varchar(65536) NOT NULL,
Review Comment:
fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Create new Instance from VM backup [cloudstack]
Posted by "blueorangutan (via GitHub)" <gi...@apache.org>.
blueorangutan commented on PR #10140:
URL: https://github.com/apache/cloudstack/pull/10140#issuecomment-2785677880
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12976
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org