You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by GitBox <gi...@apache.org> on 2022/07/12 09:55:47 UTC
[GitHub] [cloudstack] fermosan opened a new pull request, #6550: Emc networker b&r
fermosan opened a new pull request, #6550:
URL: https://github.com/apache/cloudstack/pull/6550
### Description
This PR...
<!--- Describe your changes in DETAIL - And how has behaviour functionally changed. -->
<!-- For new features, provide link to FS, dev ML discussion etc. -->
<!-- In case of bug fix, the expected and actual behaviours, steps to reproduce. -->
<!-- When "Fixes: #<id>" is specified, the issue/PR will automatically be closed when this PR gets merged -->
<!-- For addressing multiple issues/PRs, use multiple "Fixes: #<id>" -->
<!-- Fixes: # -->
<!--- ********************************************************************************* -->
<!--- NOTE: AUTOMATATION USES THE DESCRIPTIONS TO SET LABELS AND PRODUCE DOCUMENTATION. -->
<!--- PLEASE PUT AN 'X' in only **ONE** box -->
<!--- ********************************************************************************* -->
### Types of changes
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [X ] New feature (non-breaking change which adds functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
### Feature/Enhancement Scale or Bug Severity
#### Feature/Enhancement Scale
- [ ] Major
- [ ] Minor
#### Bug Severity
- [ ] BLOCKER
- [ ] Critical
- [ ] Major
- [ ] Minor
- [ ] Trivial
### Screenshots (if appropriate):
### How Has This Been Tested?
<!-- Please describe in detail how you tested your changes. --> A small scale lab environment with two Hypervisors, One Management server, one zone, one cluster. Three different accounts with VMs. Multiple users performing common scenarios for seven days now.
<!-- Include details of your testing environment, and the tests you ran to --> All the B&R cases with the exception of SLA Based backups that is not supported by the Backup Vendor have been tested. Backup and Restore of whole VMs and single volumes. Import of backup offerings from the external provider. Assignment of backup offerings to VMs.
<!-- see how your change affects other areas of the code, etc. --> There is a basic implementation of the B&R functionality on the KVM Hypervisor Guru.
<!-- Please read the [CONTRIBUTING](https://github.com/apache/cloudstack/blob/main/CONTRIBUTING.md) document -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r955954435
##########
scripts/vm/hypervisor/kvm/nsrkvmbackup.sh:
##########
@@ -0,0 +1,270 @@
+#!/bin/bash
+
+## Licensed to the Apache Software Foundation (ASF) under one
+## or more contributor license agreements. See the NOTICE file
+## distributed with this work for additional information
+## regarding copyright ownership. The ASF licenses this file
+## to you under the Apache License, Version 2.0 (the
+## "License"); you may not use this file except in compliance
+## with the License. You may obtain a copy of the License at
+##
+## http://www.apache.org/licenses/LICENSE-2.0
+##
+## Unless required by applicable law or agreed to in writing,
+## software distributed under the License is distributed on an
+## "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+## KIND, either express or implied. See the License for the
+## specific language governing permissions and limitations
+## under the License.
+
+
+version=0.9.1
Review Comment:
This is version of the script. To keep track of changes during development.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1228299646
Hello, replies inline.
> Did a round of review, in addition to the requested changes or questions, pl also:
>
> * rebase your branch against last main branch to allow sonar and other checks to re-kick
done.
> * look at any code smells reported by the sonarcloud bot and see coverage report by codecov if it helps to improve the code
will do
> * add any marvin/functional test
There are some tests already implemented.
>
> Misc:
>
> * Does this plugin ship as enabled by default, is there a way to disable it?
It is enabled by selecting Networker as B&R Plugin from general options.
> * Does it require any UI or DB/schema changes etc?
No UI or DB/schema changes required.
For more details you can check: https://github.com/apache/cloudstack-documentation/pull/281
Since it's vacation period for us we will revert with the changes your requested when we get back.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1185957282
> you have a small conflict @fermosan . can you resolve it? (will happen more often the coming time, I'm sure)
It is resolved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1185409402
you have a small conflict @fermosan . can you resolve it? (will happen more often the coming time, I'm sure)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1376824261
@fermosan you can delete the branch, we can create new PRs or issues if we need to and we also have dev@cloudstack.apache.org
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r967636017
##########
server/src/main/java/com/cloud/hypervisor/KVMGuru.java:
##########
@@ -297,4 +343,64 @@ public Map<String, String> getClusterSettings(long vmId) {
return null;
}
-}
+ @Override
+ public VirtualMachine importVirtualMachineFromBackup(long zoneId, long domainId, long accountId, long userId, String vmInternalName, Backup backup) {
+ s_logger.debug(String.format("Trying to import VM [vmInternalName: %s] from Backup [%s].", vmInternalName,
+ ReflectionToStringBuilderUtils.reflectOnlySelectedFields(backup, "id", "uuid", "vmId", "externalId", "backupType")));
+
+ VMInstanceVO vm = _instanceDao.findVMByInstanceNameIncludingRemoved(vmInternalName);
+
+ if (vm == null) {
+ throw new CloudRuntimeException("Cannot find VM: " + vmInternalName);
+ }
+ try {
+ if (vm.getRemoved() == null) {
+ vm.setState(VirtualMachine.State.Stopped);
+ vm.setPowerState(VirtualMachine.PowerState.PowerOff);
+ _instanceDao.update(vm.getId(), vm);
+ }
+ for ( Backup.VolumeInfo VMVolToRestore : vm.getBackupVolumeList()) {
+ VolumeVO volume = _volumeDao.findByUuidIncludingRemoved(VMVolToRestore.getUuid());
+ volume.setState(Volume.State.Ready);
+ _volumeDao.update(volume.getId(), volume);
+ if (VMVolToRestore.getType() == Volume.Type.ROOT) {
+ _volumeDao.update(volume.getId(), volume);
+ _volumeDao.attachVolume(volume.getId(), vm.getId(), 0L);
+ }
+ else if ( VMVolToRestore.getType() == Volume.Type.DATADISK) {
+ List<VolumeVO> vmVolumes = _volumeDao.findByInstance(vm.getId());
+ _volumeDao.update(volume.getId(), volume);
+ _volumeDao.attachVolume(volume.getId(), vm.getId(), getNextAvailableDeviceId(vmVolumes));
+ }
+ }
+ } catch (Exception e) {
+ throw new RuntimeException("Could not restore VM " + vm.getName() + " due to : " + e.getMessage());
+ }
+ return vm;
+ }
+
+
Review Comment:
fixed in latest commit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1262597872
@rohityadavcloud a Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r979830937
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/NetworkerBackupProvider.java:
##########
@@ -0,0 +1,640 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.backup;
+
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.host.HostVO;
+import com.cloud.host.Status;
+import com.cloud.host.dao.HostDao;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.StoragePoolHostVO;
+import com.cloud.storage.Volume;
+import com.cloud.storage.VolumeVO;
+import com.cloud.storage.dao.StoragePoolHostDao;
+import com.cloud.storage.dao.VolumeDao;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.component.AdapterBase;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallbackNoReturn;
+import com.cloud.utils.db.TransactionStatus;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.ssh.SSHCmdHelper;
+import com.cloud.utils.ssh.SshException;
+import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.dao.VMInstanceDao;
+import org.apache.cloudstack.api.InternalIdentity;
+import org.apache.cloudstack.backup.dao.BackupDao;
+import org.apache.cloudstack.backup.dao.BackupOfferingDaoImpl;
+import org.apache.cloudstack.backup.networker.NetworkerClient;
+import org.apache.cloudstack.framework.config.ConfigKey;
+import org.apache.cloudstack.framework.config.Configurable;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.log4j.Logger;
+import org.apache.xml.utils.URI;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+
+
+import javax.inject.Inject;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.HashMap;
+import java.util.Date;
+import java.util.Objects;
+import java.util.UUID;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+
+public class NetworkerBackupProvider extends AdapterBase implements BackupProvider, Configurable {
+
+ public static final String BACKUP_IDENTIFIER = "-CSBKP-";
+ private static final Logger LOG = Logger.getLogger(NetworkerBackupProvider.class);
+ public ConfigKey<String> NetworkerUrl = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.url", "https://localhost:9090/nwrestapi/v3",
+ "The EMC Networker API URL.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerUsername = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.username", "administrator",
+ "The EMC Networker API username.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerPassword = new ConfigKey<>("Secure", String.class,
+ "backup.plugin.networker.password", "password",
+ "The EMC Networker API password.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerMediaPool = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.pool", "Default",
+ "The EMC Networker Media Pool", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerValidateSSLSecurity = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.validate.ssl", "false",
+ "Validate the SSL certificate when connecting to EMC Networker API service.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Integer> NetworkerApiRequestTimeout = new ConfigKey<>("Advanced", Integer.class,
+ "backup.plugin.networker.request.timeout", "300",
+ "The EMC Networker API request timeout in seconds.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerClientVerboseLogs = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.client.verbosity", "false",
+ "Produce Verbose logs in Hypervisor", true, ConfigKey.Scope.Zone);
+
+ @Inject
+ private BackupDao backupDao;
+
+ @Inject
+ private HostDao hostDao;
+
+ @Inject
+ private ConfigurationDao configDao;
+
+ @Inject
+ private ClusterDao clusterDao;
+
+ @Inject
+ private VolumeDao volumeDao;
+
+ @Inject
+ private StoragePoolHostDao storagePoolHostDao;
+
+ @Inject
+ private VMInstanceDao vmInstanceDao;
+
+ private static String getUrlDomain(String url) throws URISyntaxException {
+ URI uri;
+ try {
+ uri = new URI(url);
+ } catch (URI.MalformedURIException e) {
+ throw new CloudRuntimeException("Failed to cast URI");
+ }
+
+ return uri.getHost();
+ }
+
+ @Override
+ public ConfigKey<?>[] getConfigKeys() {
+ return new ConfigKey[]{
+ NetworkerUrl,
+ NetworkerUsername,
+ NetworkerPassword,
+ NetworkerValidateSSLSecurity,
+ NetworkerApiRequestTimeout,
+ NetworkerMediaPool,
+ NetworkerClientVerboseLogs
+ };
+ }
+
+ @Override
+ public String getName() {
+ return "networker";
+ }
+
+ @Override
+ public String getDescription() {
+ return "EMC Networker Backup Plugin";
+ }
+
+ @Override
+ public String getConfigComponentName() {
+ return BackupService.class.getSimpleName();
+ }
+
+ protected HostVO getLastVMHypervisorHost(VirtualMachine vm) {
+ HostVO host;
+ Long hostId = vm.getLastHostId();
+
+ if (hostId == null) {
+ LOG.debug("Cannot find last host for vm. This should never happen, please check your database.");
+ return null;
+ }
+ host = hostDao.findById(hostId);
+
+ if ( host.getStatus() == Status.Up ) {
+ return host;
+ } else {
+ // Try to find a host in the same cluster
+ List<HostVO> altClusterHosts = hostDao.findHypervisorHostInCluster(host.getClusterId());
+ for (final HostVO candidateClusterHost : altClusterHosts) {
+ if ( candidateClusterHost.getStatus() == Status.Up ) {
+ LOG.debug("Found Host " + candidateClusterHost.getName());
+ return candidateClusterHost;
+ }
+ }
+ }
+ // Try to find a Host in the zone
+ List<HostVO> altZoneHosts = hostDao.findByDataCenterId(host.getDataCenterId());
+ for (final HostVO candidateZoneHost : altZoneHosts) {
+ if ( candidateZoneHost.getStatus() == Status.Up && candidateZoneHost.getHypervisorType() == Hypervisor.HypervisorType.KVM ) {
+ LOG.debug("Found Host " + candidateZoneHost.getName());
+ return candidateZoneHost;
+ }
+ }
+ return null;
+ }
+
+ protected HostVO getRunningVMHypervisorHost(VirtualMachine vm) {
+
+ HostVO host;
+ Long hostId = vm.getHostId();
+
+ if (hostId == null) {
+ throw new CloudRuntimeException("Unable to find the HYPERVISOR for " + vm.getName() + ". Make sure the virtual machine is running");
+ }
+
+ host = hostDao.findById(hostId);
+
+ return host;
+ }
+
+ protected String getVMHypervisorCluster(HostVO host) {
+
+ return clusterDao.findById(host.getClusterId()).getName();
+ }
+
+ protected Ternary<String, String, String> getKVMHyperisorCredentials(HostVO host) {
+
+ String username = null;
+ String password = null;
+ String privateKey = null;
+
+ if (host != null && host.getHypervisorType() == Hypervisor.HypervisorType.KVM) {
+ hostDao.loadDetails(host);
+ password = host.getDetail("password");
+ username = host.getDetail("username");
+ privateKey = configDao.getValue("ssh.privatekey");
+ }
+ if ((password == null && privateKey == null) || username == null) {
+ throw new CloudRuntimeException("Cannot find login credentials for HYPERVISOR " + Objects.requireNonNull(host).getUuid());
+ }
+
+ return new Ternary<>(username, password, privateKey);
+ }
+
+ private String executeBackupCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+ String nstRegex = "\\bcompleted savetime=([0-9]{10})";
+ Pattern saveTimePattern = Pattern.compile(nstRegex);
+
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ Matcher saveTimeMatcher = saveTimePattern.matcher(result.getStdOut());
+ if (saveTimeMatcher.find()) {
+ return saveTimeMatcher.group(1);
+ }
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+
+ return null;
+ }
+ private boolean executeRestoreCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ return true;
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+ }
+
+ private NetworkerClient getClient(final Long zoneId) {
+ try {
+ return new NetworkerClient(NetworkerUrl.valueIn(zoneId), NetworkerUsername.valueIn(zoneId), NetworkerPassword.valueIn(zoneId),
+ NetworkerValidateSSLSecurity.valueIn(zoneId), NetworkerApiRequestTimeout.valueIn(zoneId));
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException("Failed to parse EMC Networker API URL: " + e.getMessage());
+ } catch (NoSuchAlgorithmException | KeyManagementException e) {
+ LOG.error("Failed to build EMC Networker API client due to: ", e);
+ }
+ throw new CloudRuntimeException("Failed to build EMC Networker API client");
+ }
+
+ @Override
+ public List<BackupOffering> listBackupOfferings(Long zoneId) {
+ List<BackupOffering> policies = new ArrayList<>();
+ for (final BackupOffering policy : getClient(zoneId).listPolicies()) {
+ if (!policy.getName().contains(BACKUP_IDENTIFIER)) {
+ policies.add(policy);
+ }
+ }
+
+ return policies;
+ }
+
+ @Override
+ public boolean isValidProviderOffering(Long zoneId, String uuid) {
+ List<BackupOffering> policies = listBackupOfferings(zoneId);
+ if (CollectionUtils.isEmpty(policies)) {
+ return false;
+ }
+ for (final BackupOffering policy : policies) {
+ if (Objects.equals(policy.getExternalId(), uuid)) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ @Override
+ public boolean assignVMToBackupOffering(VirtualMachine vm, BackupOffering backupOffering) { return true; }
+
+ @Override
+ public boolean removeVMFromBackupOffering(VirtualMachine vm) {
+ LOG.debug("Removing VirtualMachine from Backup offering and Deleting any existing backups");
+
+ List<String> backupsTaken = getClient(vm.getDataCenterId()).getBackupsForVm(vm);
+
+ for (String backupId : backupsTaken) {
+ LOG.debug("Trying to remove backup with id" + backupId);
+ getClient(vm.getDataCenterId()).deleteBackupForVM(backupId);
+ }
+
+ return true;
+ }
+
+ @Override
+ public boolean restoreVMFromBackup(VirtualMachine vm, Backup backup) {
+ LOG.debug("Restoring vm " + vm.getUuid() + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+ String networkerServer;
+ HostVO hostVO;
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return false;
+ }
+
+ // Find where the VM was last running
+ hostVO = getLastVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -S " + SSID;
+
+ if (NetworkerClientVerboseLogs.value())
+ command = command + " -v ";
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for VM ID " + vm.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return true;
+ } else {
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return false;
+ }
+ }
+
+ @Override
+ public Pair<Boolean, String> restoreBackedUpVolume(Backup backup, String volumeUuid, String hostIp, String dataStoreUuid) {
+ LOG.debug("Restoring volume " + volumeUuid + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+
+ String networkerServer;
+ VolumeVO volume = volumeDao.findByUuid(volumeUuid);
+ VMInstanceVO backupSourceVm = vmInstanceDao.findById(backup.getVmId());
+ StoragePoolHostVO dataStore = storagePoolHostDao.findByUuid(dataStoreUuid);
+ HostVO hostVO = hostDao.findByIp(hostIp);
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+ final String clusterName = networkerBackup.getClientHostname();
+ final String destinationNetworkerClient = hostVO.getName().split("\\.")[0];
+ Long restoredVolumeDiskSize = 0L;
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return null;
+ }
+
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ // Find volume size from backup vols
+ for ( Backup.VolumeInfo VMVolToRestore : backupSourceVm.getBackupVolumeList()) {
+ if (VMVolToRestore.getUuid().equals(volumeUuid))
+ restoredVolumeDiskSize = (VMVolToRestore.getSize());
+ }
+
+ VolumeVO restoredVolume = new VolumeVO(Volume.Type.DATADISK, null, backup.getZoneId(),
+ backup.getDomainId(), backup.getAccountId(), 0, null,
+ backup.getSize(), null, null, null);
+
+ restoredVolume.setName("RV-"+volume.getName());
+ restoredVolume.setProvisioningType(volume.getProvisioningType());
+ restoredVolume.setUpdated(new Date());
+ restoredVolume.setUuid(UUID.randomUUID().toString());
+ restoredVolume.setRemoved(null);
+ restoredVolume.setDisplayVolume(true);
+ restoredVolume.setPoolId(volume.getPoolId());
+ restoredVolume.setPath(restoredVolume.getUuid());
+ restoredVolume.setState(Volume.State.Copying);
+ restoredVolume.setSize(restoredVolumeDiskSize);
+ restoredVolume.setDiskOfferingId(volume.getDiskOfferingId());
+
+ try {
+ volumeDao.persist(restoredVolume);
+ } catch (Exception e) {
+ throw new CloudRuntimeException("Unable to craft restored volume due to: "+e);
+ }
+
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
Review Comment:
After a first look, i am not sure this can be used on the KVM Hypervisor side. Am i missing something ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r979468150
##########
plugins/backup/networker/pom.xml:
##########
@@ -0,0 +1,56 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <artifactId>cloud-plugin-backup-networker</artifactId>
+ <name>Apache CloudStack Plugin - EMC Networker Backup and Recovery Plugin</name>
+ <parent>
+ <artifactId>cloudstack-plugins</artifactId>
+ <groupId>org.apache.cloudstack</groupId>
+ <version>4.18.0.0-SNAPSHOT</version>
+ <relativePath>../../pom.xml</relativePath>
+ </parent>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.cloudstack</groupId>
+ <artifactId>cloud-plugin-hypervisor-kvm</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.commons</groupId>
+ <artifactId>commons-lang3</artifactId>
+ <version>${cs.commons-lang3.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>com.fasterxml.jackson.core</groupId>
+ <artifactId>jackson-databind</artifactId>
+ <version>2.13.0</version>
+ </dependency>
+ <dependency>
+ <groupId>com.github.tomakehurst</groupId>
+ <artifactId>wiremock-standalone</artifactId>
+ <version>2.27.2</version>
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1260931170
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 4303
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1260864804
@DaanHoogland a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1260863914
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1357488321
@blueorangutan test matrix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1358755207
<b>Trillian test result (tid-5567)</b>
Environment: vmware-65u2 (x2), Advanced Networking with Mgmt server 7
Total time taken: 53426 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6550-t5567-vmware-65u2.zip
Smoke tests completed. 103 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_02_create_template_with_checksum_sha1 | `Error` | 5.16 | test_templates.py
test_03_create_template_with_checksum_sha256 | `Error` | 5.17 | test_templates.py
test_04_create_template_with_checksum_md5 | `Error` | 5.15 | test_templates.py
test_02_upgrade_kubernetes_cluster | `Failure` | 537.80 | test_kubernetes_clusters.py
test_08_upgrade_kubernetes_ha_cluster | `Error` | 4093.32 | test_kubernetes_clusters.py
test_09_delete_kubernetes_ha_cluster | `Failure` | 0.05 | test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown | `Error` | 45.96 | test_kubernetes_clusters.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1370765913
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1362626346
@blueorangutan test matrix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1362635623
@rohityadavcloud a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1362682613
@rohityadavcloud @fermosan do we reset the package-lock.json on this? I think we can merge as soon as we have.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1363956730
@fermosan sorry to be bugging you. I don´t think the package-lock is intended to be deleted, just to remain unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1279789895
UI build: :heavy_check_mark:
Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6550 (SL-JID-2512)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r991117001
##########
plugins/pom.xml:
##########
@@ -59,6 +59,7 @@
<module>api/solidfire-intg-test</module>
<module>backup/dummy</module>
+ <module>backup/networker</module>
Review Comment:
@fermosan Does this only support KVM or all hypervisors?
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/NetworkerBackupProvider.java:
##########
@@ -0,0 +1,637 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.backup;
+
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.host.HostVO;
+import com.cloud.host.Status;
+import com.cloud.host.dao.HostDao;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.StoragePoolHostVO;
+import com.cloud.storage.Volume;
+import com.cloud.storage.VolumeVO;
+import com.cloud.storage.dao.StoragePoolHostDao;
+import com.cloud.storage.dao.VolumeDao;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.component.AdapterBase;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallbackNoReturn;
+import com.cloud.utils.db.TransactionStatus;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.ssh.SSHCmdHelper;
+import com.cloud.utils.ssh.SshException;
+import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.dao.VMInstanceDao;
+import org.apache.cloudstack.api.InternalIdentity;
+import org.apache.cloudstack.backup.dao.BackupDao;
+import org.apache.cloudstack.backup.dao.BackupOfferingDaoImpl;
+import org.apache.cloudstack.backup.networker.NetworkerClient;
+import org.apache.cloudstack.framework.config.ConfigKey;
+import org.apache.cloudstack.framework.config.Configurable;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.log4j.Logger;
+import org.apache.xml.utils.URI;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+import javax.inject.Inject;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.HashMap;
+import java.util.Date;
+import java.util.Objects;
+import java.util.UUID;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+public class NetworkerBackupProvider extends AdapterBase implements BackupProvider, Configurable {
+
+ public static final String BACKUP_IDENTIFIER = "-CSBKP-";
+ private static final Logger LOG = Logger.getLogger(NetworkerBackupProvider.class);
+ public ConfigKey<String> NetworkerUrl = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.url", "https://localhost:9090/nwrestapi/v3",
+ "The EMC Networker API URL.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerUsername = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.username", "administrator",
+ "The EMC Networker API username.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerPassword = new ConfigKey<>("Secure", String.class,
+ "backup.plugin.networker.password", "password",
+ "The EMC Networker API password.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerMediaPool = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.pool", "Default",
+ "The EMC Networker Media Pool", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerValidateSSLSecurity = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.validate.ssl", "false",
+ "Validate the SSL certificate when connecting to EMC Networker API service.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Integer> NetworkerApiRequestTimeout = new ConfigKey<>("Advanced", Integer.class,
+ "backup.plugin.networker.request.timeout", "300",
+ "The EMC Networker API request timeout in seconds.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerClientVerboseLogs = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.client.verbosity", "false",
+ "Produce Verbose logs in Hypervisor", true, ConfigKey.Scope.Zone);
+
+ @Inject
+ private BackupDao backupDao;
+
+ @Inject
+ private HostDao hostDao;
+
+ @Inject
+ private ConfigurationDao configDao;
+
+ @Inject
+ private ClusterDao clusterDao;
+
+ @Inject
+ private VolumeDao volumeDao;
+
+ @Inject
+ private StoragePoolHostDao storagePoolHostDao;
+
+ @Inject
+ private VMInstanceDao vmInstanceDao;
+
+ private static String getUrlDomain(String url) throws URISyntaxException {
+ URI uri;
+ try {
+ uri = new URI(url);
+ } catch (URI.MalformedURIException e) {
+ throw new CloudRuntimeException("Failed to cast URI");
+ }
+
+ return uri.getHost();
+ }
+
+ @Override
+ public ConfigKey<?>[] getConfigKeys() {
+ return new ConfigKey[]{
+ NetworkerUrl,
+ NetworkerUsername,
+ NetworkerPassword,
+ NetworkerValidateSSLSecurity,
+ NetworkerApiRequestTimeout,
+ NetworkerMediaPool,
+ NetworkerClientVerboseLogs
+ };
+ }
+
+ @Override
+ public String getName() {
+ return "networker";
+ }
+
+ @Override
+ public String getDescription() {
+ return "EMC Networker Backup Plugin";
+ }
+
+ @Override
+ public String getConfigComponentName() {
+ return BackupService.class.getSimpleName();
+ }
+
+ protected HostVO getLastVMHypervisorHost(VirtualMachine vm) {
+ HostVO host;
+ Long hostId = vm.getLastHostId();
+
+ if (hostId == null) {
+ LOG.debug("Cannot find last host for vm. This should never happen, please check your database.");
+ return null;
+ }
+ host = hostDao.findById(hostId);
+
+ if ( host.getStatus() == Status.Up ) {
+ return host;
+ } else {
+ // Try to find a host in the same cluster
+ List<HostVO> altClusterHosts = hostDao.findHypervisorHostInCluster(host.getClusterId());
+ for (final HostVO candidateClusterHost : altClusterHosts) {
+ if ( candidateClusterHost.getStatus() == Status.Up ) {
+ LOG.debug("Found Host " + candidateClusterHost.getName());
+ return candidateClusterHost;
+ }
+ }
+ }
+ // Try to find a Host in the zone
+ List<HostVO> altZoneHosts = hostDao.findByDataCenterId(host.getDataCenterId());
+ for (final HostVO candidateZoneHost : altZoneHosts) {
+ if ( candidateZoneHost.getStatus() == Status.Up && candidateZoneHost.getHypervisorType() == Hypervisor.HypervisorType.KVM ) {
+ LOG.debug("Found Host " + candidateZoneHost.getName());
+ return candidateZoneHost;
+ }
+ }
+ return null;
+ }
+
+ protected HostVO getRunningVMHypervisorHost(VirtualMachine vm) {
+
+ HostVO host;
+ Long hostId = vm.getHostId();
+
+ if (hostId == null) {
+ throw new CloudRuntimeException("Unable to find the HYPERVISOR for " + vm.getName() + ". Make sure the virtual machine is running");
+ }
+
+ host = hostDao.findById(hostId);
+
+ return host;
+ }
+
+ protected String getVMHypervisorCluster(HostVO host) {
+
+ return clusterDao.findById(host.getClusterId()).getName();
+ }
+
+ protected Ternary<String, String, String> getKVMHyperisorCredentials(HostVO host) {
+
+ String username = null;
+ String password = null;
+ String privateKey = null;
+
+ if (host != null && host.getHypervisorType() == Hypervisor.HypervisorType.KVM) {
+ hostDao.loadDetails(host);
+ password = host.getDetail("password");
+ username = host.getDetail("username");
+ privateKey = configDao.getValue("ssh.privatekey");
+ }
+ if ((password == null && privateKey == null) || username == null) {
+ throw new CloudRuntimeException("Cannot find login credentials for HYPERVISOR " + Objects.requireNonNull(host).getUuid());
+ }
+
+ return new Ternary<>(username, password, privateKey);
+ }
+
+ private String executeBackupCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+ String nstRegex = "\\bcompleted savetime=([0-9]{10})";
+ Pattern saveTimePattern = Pattern.compile(nstRegex);
+
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ Matcher saveTimeMatcher = saveTimePattern.matcher(result.getStdOut());
+ if (saveTimeMatcher.find()) {
+ return saveTimeMatcher.group(1);
+ }
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+
+ return null;
+ }
+ private boolean executeRestoreCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ return true;
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+ }
+
+ private NetworkerClient getClient(final Long zoneId) {
+ try {
+ return new NetworkerClient(NetworkerUrl.valueIn(zoneId), NetworkerUsername.valueIn(zoneId), NetworkerPassword.valueIn(zoneId),
+ NetworkerValidateSSLSecurity.valueIn(zoneId), NetworkerApiRequestTimeout.valueIn(zoneId));
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException("Failed to parse EMC Networker API URL: " + e.getMessage());
+ } catch (NoSuchAlgorithmException | KeyManagementException e) {
+ LOG.error("Failed to build EMC Networker API client due to: ", e);
+ }
+ throw new CloudRuntimeException("Failed to build EMC Networker API client");
+ }
+
+ @Override
+ public List<BackupOffering> listBackupOfferings(Long zoneId) {
+ List<BackupOffering> policies = new ArrayList<>();
+ for (final BackupOffering policy : getClient(zoneId).listPolicies()) {
+ if (!policy.getName().contains(BACKUP_IDENTIFIER)) {
+ policies.add(policy);
+ }
+ }
+
+ return policies;
+ }
+
+ @Override
+ public boolean isValidProviderOffering(Long zoneId, String uuid) {
+ List<BackupOffering> policies = listBackupOfferings(zoneId);
+ if (CollectionUtils.isEmpty(policies)) {
+ return false;
+ }
+ for (final BackupOffering policy : policies) {
+ if (Objects.equals(policy.getExternalId(), uuid)) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ @Override
+ public boolean assignVMToBackupOffering(VirtualMachine vm, BackupOffering backupOffering) { return true; }
+
+ @Override
+ public boolean removeVMFromBackupOffering(VirtualMachine vm) {
+ LOG.debug("Removing VirtualMachine from Backup offering and Deleting any existing backups");
+
+ List<String> backupsTaken = getClient(vm.getDataCenterId()).getBackupsForVm(vm);
+
+ for (String backupId : backupsTaken) {
+ LOG.debug("Trying to remove backup with id" + backupId);
+ getClient(vm.getDataCenterId()).deleteBackupForVM(backupId);
+ }
+
+ return true;
+ }
+
+ @Override
+ public boolean restoreVMFromBackup(VirtualMachine vm, Backup backup) {
+ LOG.debug("Restoring vm " + vm.getUuid() + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+ String networkerServer;
+ HostVO hostVO;
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return false;
+ }
+
+ // Find where the VM was last running
+ hostVO = getLastVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -S " + SSID;
+
+ if ( Boolean.TRUE.equals(NetworkerClientVerboseLogs.value()) )
+ command = command + " -v ";
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for VM ID " + vm.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return true;
+ } else {
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return false;
+ }
+ }
+
+ @Override
+ public Pair<Boolean, String> restoreBackedUpVolume(Backup backup, String volumeUuid, String hostIp, String dataStoreUuid) {
+ LOG.debug("Restoring volume " + volumeUuid + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+
+ String networkerServer;
+ VolumeVO volume = volumeDao.findByUuid(volumeUuid);
+ VMInstanceVO backupSourceVm = vmInstanceDao.findById(backup.getVmId());
+ StoragePoolHostVO dataStore = storagePoolHostDao.findByUuid(dataStoreUuid);
+ HostVO hostVO = hostDao.findByIp(hostIp);
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+ final String clusterName = networkerBackup.getClientHostname();
+ final String destinationNetworkerClient = hostVO.getName().split("\\.")[0];
+ Long restoredVolumeDiskSize = 0L;
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return null;
+ }
+
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ // Find volume size from backup vols
+ for ( Backup.VolumeInfo VMVolToRestore : backupSourceVm.getBackupVolumeList()) {
+ if (VMVolToRestore.getUuid().equals(volumeUuid))
+ restoredVolumeDiskSize = (VMVolToRestore.getSize());
+ }
+
+ VolumeVO restoredVolume = new VolumeVO(Volume.Type.DATADISK, null, backup.getZoneId(),
+ backup.getDomainId(), backup.getAccountId(), 0, null,
+ backup.getSize(), null, null, null);
+
+ restoredVolume.setName("RV-"+volume.getName());
+ restoredVolume.setProvisioningType(volume.getProvisioningType());
+ restoredVolume.setUpdated(new Date());
+ restoredVolume.setUuid(UUID.randomUUID().toString());
+ restoredVolume.setRemoved(null);
+ restoredVolume.setDisplayVolume(true);
+ restoredVolume.setPoolId(volume.getPoolId());
+ restoredVolume.setPath(restoredVolume.getUuid());
+ restoredVolume.setState(Volume.State.Copying);
+ restoredVolume.setSize(restoredVolumeDiskSize);
+ restoredVolume.setDiskOfferingId(volume.getDiskOfferingId());
+
+ try {
+ volumeDao.persist(restoredVolume);
+ } catch (Exception e) {
+ throw new CloudRuntimeException("Unable to craft restored volume due to: "+e);
+ }
+
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -c " + clusterName +
+ " -d " + destinationNetworkerClient +
+ " -n " + restoredVolume.getUuid() +
+ " -p " + dataStore.getLocalPath() +
+ " -a " + volume.getUuid();
+
+ if ( Boolean.TRUE.equals(NetworkerClientVerboseLogs.value()) )
+ command = command + " -v ";
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for Volume UUID " + volume.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return new Pair<>(true,restoredVolume.getUuid());
+ } else {
+ volumeDao.expunge(restoredVolume.getId());
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return null;
+ }
+ }
+
+ @Override
+ public boolean takeBackup(VirtualMachine vm) {
+
+ String networkerServer;
+ String clusterName;
+
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ // Find where the VM is currently running
+ HostVO hostVO = getRunningVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ // Get retention Period for our Backup
+ BackupOfferingVO vmBackupOffering = new BackupOfferingDaoImpl().findById(vm.getBackupOfferingId());
+ final String backupProviderPolicyId = vmBackupOffering.getExternalId();
+ String backupRentionPeriod = getClient(vm.getDataCenterId()).getBackupPolicyRetentionInterval(backupProviderPolicyId);
+
+ if ( backupRentionPeriod == null ) {
+ LOG.warn("There is no retention setting for Emc Networker Policy, setting default for 1 day");
+ backupRentionPeriod="1 Day";
Review Comment:
nit - space around variable assignment `=`
##########
server/src/main/java/com/cloud/hypervisor/KVMGuru.java:
##########
@@ -167,9 +213,9 @@ protected void configureVmMemoryAndCpuCores(VirtualMachineTO virtualMachineTo, H
Long maxHostMemory = max.first();
Integer maxHostCpuCore = max.second();
- Long minMemory = virtualMachineTo.getMinRam();
+ long minMemory = virtualMachineTo.getMinRam();
Review Comment:
Concern - this can cause NPE if the TO doesn't have the value set.
##########
server/src/main/java/com/cloud/hypervisor/KVMGuru.java:
##########
@@ -204,7 +250,7 @@ protected Pair<Long, Integer> getHostMaxMemoryAndCpuCores(HostVO host, VirtualMa
if (lastHost != null) {
maxHostMemory = lastHost.getTotalMemory();
maxHostCpuCore = lastHost.getCpus();
- s_logger.debug(String.format("Retrieved memory and cpu max values {\"memory\": %s, \"cpu\": %s} from %s last %s.", maxHostMemory, maxHostCpuCore, vmDescription, lastHost.toString()));
+ s_logger.debug(String.format("Retrieved memory and cpu max values {\"memory\": %s, \"cpu\": %s} from %s last %s.", maxHostMemory, maxHostCpuCore, vmDescription, lastHost));
Review Comment:
would printing lastHost, also print its ID (and other metadata)?
##########
server/src/main/java/com/cloud/hypervisor/KVMGuru.java:
##########
@@ -68,6 +83,37 @@ protected KVMGuru() {
super();
}
+ /**
+ * Get next free DeviceId for a KVM Guest
+ */
+
+ protected Long getNextAvailableDeviceId(List<VolumeVO> vmVolumes) {
+
+ int maxDataVolumesSupported;
+ int maxDeviceId;
+ List<String> devIds = new ArrayList<>();
+
+ try {
+ maxDataVolumesSupported = _hypervisorCapabilitiesDao.getMaxDataVolumesLimit(HypervisorType.KVM,"default");
+ int maxDevices = maxDataVolumesSupported + 2; // add 2 to consider devices root volume and cdrom
Review Comment:
Thanks @fermosan makes sense.
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/NetworkerBackupProvider.java:
##########
@@ -0,0 +1,640 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.backup;
+
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.host.HostVO;
+import com.cloud.host.Status;
+import com.cloud.host.dao.HostDao;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.StoragePoolHostVO;
+import com.cloud.storage.Volume;
+import com.cloud.storage.VolumeVO;
+import com.cloud.storage.dao.StoragePoolHostDao;
+import com.cloud.storage.dao.VolumeDao;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.component.AdapterBase;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallbackNoReturn;
+import com.cloud.utils.db.TransactionStatus;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.ssh.SSHCmdHelper;
+import com.cloud.utils.ssh.SshException;
+import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.dao.VMInstanceDao;
+import org.apache.cloudstack.api.InternalIdentity;
+import org.apache.cloudstack.backup.dao.BackupDao;
+import org.apache.cloudstack.backup.dao.BackupOfferingDaoImpl;
+import org.apache.cloudstack.backup.networker.NetworkerClient;
+import org.apache.cloudstack.framework.config.ConfigKey;
+import org.apache.cloudstack.framework.config.Configurable;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.log4j.Logger;
+import org.apache.xml.utils.URI;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+
+
+import javax.inject.Inject;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.HashMap;
+import java.util.Date;
+import java.util.Objects;
+import java.util.UUID;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+
+public class NetworkerBackupProvider extends AdapterBase implements BackupProvider, Configurable {
+
+ public static final String BACKUP_IDENTIFIER = "-CSBKP-";
+ private static final Logger LOG = Logger.getLogger(NetworkerBackupProvider.class);
+ public ConfigKey<String> NetworkerUrl = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.url", "https://localhost:9090/nwrestapi/v3",
+ "The EMC Networker API URL.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerUsername = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.username", "administrator",
+ "The EMC Networker API username.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerPassword = new ConfigKey<>("Secure", String.class,
+ "backup.plugin.networker.password", "password",
+ "The EMC Networker API password.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerMediaPool = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.pool", "Default",
+ "The EMC Networker Media Pool", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerValidateSSLSecurity = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.validate.ssl", "false",
+ "Validate the SSL certificate when connecting to EMC Networker API service.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Integer> NetworkerApiRequestTimeout = new ConfigKey<>("Advanced", Integer.class,
+ "backup.plugin.networker.request.timeout", "300",
+ "The EMC Networker API request timeout in seconds.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerClientVerboseLogs = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.client.verbosity", "false",
+ "Produce Verbose logs in Hypervisor", true, ConfigKey.Scope.Zone);
+
+ @Inject
+ private BackupDao backupDao;
+
+ @Inject
+ private HostDao hostDao;
+
+ @Inject
+ private ConfigurationDao configDao;
+
+ @Inject
+ private ClusterDao clusterDao;
+
+ @Inject
+ private VolumeDao volumeDao;
+
+ @Inject
+ private StoragePoolHostDao storagePoolHostDao;
+
+ @Inject
+ private VMInstanceDao vmInstanceDao;
+
+ private static String getUrlDomain(String url) throws URISyntaxException {
+ URI uri;
+ try {
+ uri = new URI(url);
+ } catch (URI.MalformedURIException e) {
+ throw new CloudRuntimeException("Failed to cast URI");
+ }
+
+ return uri.getHost();
+ }
+
+ @Override
+ public ConfigKey<?>[] getConfigKeys() {
+ return new ConfigKey[]{
+ NetworkerUrl,
+ NetworkerUsername,
+ NetworkerPassword,
+ NetworkerValidateSSLSecurity,
+ NetworkerApiRequestTimeout,
+ NetworkerMediaPool,
+ NetworkerClientVerboseLogs
+ };
+ }
+
+ @Override
+ public String getName() {
+ return "networker";
+ }
+
+ @Override
+ public String getDescription() {
+ return "EMC Networker Backup Plugin";
+ }
+
+ @Override
+ public String getConfigComponentName() {
+ return BackupService.class.getSimpleName();
+ }
+
+ protected HostVO getLastVMHypervisorHost(VirtualMachine vm) {
+ HostVO host;
+ Long hostId = vm.getLastHostId();
+
+ if (hostId == null) {
+ LOG.debug("Cannot find last host for vm. This should never happen, please check your database.");
+ return null;
+ }
+ host = hostDao.findById(hostId);
+
+ if ( host.getStatus() == Status.Up ) {
+ return host;
+ } else {
+ // Try to find a host in the same cluster
+ List<HostVO> altClusterHosts = hostDao.findHypervisorHostInCluster(host.getClusterId());
+ for (final HostVO candidateClusterHost : altClusterHosts) {
+ if ( candidateClusterHost.getStatus() == Status.Up ) {
+ LOG.debug("Found Host " + candidateClusterHost.getName());
+ return candidateClusterHost;
+ }
+ }
+ }
+ // Try to find a Host in the zone
+ List<HostVO> altZoneHosts = hostDao.findByDataCenterId(host.getDataCenterId());
+ for (final HostVO candidateZoneHost : altZoneHosts) {
+ if ( candidateZoneHost.getStatus() == Status.Up && candidateZoneHost.getHypervisorType() == Hypervisor.HypervisorType.KVM ) {
+ LOG.debug("Found Host " + candidateZoneHost.getName());
+ return candidateZoneHost;
+ }
+ }
+ return null;
+ }
+
+ protected HostVO getRunningVMHypervisorHost(VirtualMachine vm) {
+
+ HostVO host;
+ Long hostId = vm.getHostId();
+
+ if (hostId == null) {
+ throw new CloudRuntimeException("Unable to find the HYPERVISOR for " + vm.getName() + ". Make sure the virtual machine is running");
+ }
+
+ host = hostDao.findById(hostId);
+
+ return host;
+ }
+
+ protected String getVMHypervisorCluster(HostVO host) {
+
+ return clusterDao.findById(host.getClusterId()).getName();
+ }
+
+ protected Ternary<String, String, String> getKVMHyperisorCredentials(HostVO host) {
+
+ String username = null;
+ String password = null;
+ String privateKey = null;
+
+ if (host != null && host.getHypervisorType() == Hypervisor.HypervisorType.KVM) {
+ hostDao.loadDetails(host);
+ password = host.getDetail("password");
+ username = host.getDetail("username");
+ privateKey = configDao.getValue("ssh.privatekey");
+ }
+ if ((password == null && privateKey == null) || username == null) {
+ throw new CloudRuntimeException("Cannot find login credentials for HYPERVISOR " + Objects.requireNonNull(host).getUuid());
+ }
+
+ return new Ternary<>(username, password, privateKey);
+ }
+
+ private String executeBackupCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+ String nstRegex = "\\bcompleted savetime=([0-9]{10})";
+ Pattern saveTimePattern = Pattern.compile(nstRegex);
+
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ Matcher saveTimeMatcher = saveTimePattern.matcher(result.getStdOut());
+ if (saveTimeMatcher.find()) {
+ return saveTimeMatcher.group(1);
+ }
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+
+ return null;
+ }
+ private boolean executeRestoreCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ return true;
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+ }
+
+ private NetworkerClient getClient(final Long zoneId) {
+ try {
+ return new NetworkerClient(NetworkerUrl.valueIn(zoneId), NetworkerUsername.valueIn(zoneId), NetworkerPassword.valueIn(zoneId),
+ NetworkerValidateSSLSecurity.valueIn(zoneId), NetworkerApiRequestTimeout.valueIn(zoneId));
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException("Failed to parse EMC Networker API URL: " + e.getMessage());
+ } catch (NoSuchAlgorithmException | KeyManagementException e) {
+ LOG.error("Failed to build EMC Networker API client due to: ", e);
+ }
+ throw new CloudRuntimeException("Failed to build EMC Networker API client");
+ }
+
+ @Override
+ public List<BackupOffering> listBackupOfferings(Long zoneId) {
+ List<BackupOffering> policies = new ArrayList<>();
+ for (final BackupOffering policy : getClient(zoneId).listPolicies()) {
+ if (!policy.getName().contains(BACKUP_IDENTIFIER)) {
+ policies.add(policy);
+ }
+ }
+
+ return policies;
+ }
+
+ @Override
+ public boolean isValidProviderOffering(Long zoneId, String uuid) {
+ List<BackupOffering> policies = listBackupOfferings(zoneId);
+ if (CollectionUtils.isEmpty(policies)) {
+ return false;
+ }
+ for (final BackupOffering policy : policies) {
+ if (Objects.equals(policy.getExternalId(), uuid)) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ @Override
+ public boolean assignVMToBackupOffering(VirtualMachine vm, BackupOffering backupOffering) { return true; }
+
+ @Override
+ public boolean removeVMFromBackupOffering(VirtualMachine vm) {
+ LOG.debug("Removing VirtualMachine from Backup offering and Deleting any existing backups");
+
+ List<String> backupsTaken = getClient(vm.getDataCenterId()).getBackupsForVm(vm);
+
+ for (String backupId : backupsTaken) {
+ LOG.debug("Trying to remove backup with id" + backupId);
+ getClient(vm.getDataCenterId()).deleteBackupForVM(backupId);
+ }
+
+ return true;
+ }
+
+ @Override
+ public boolean restoreVMFromBackup(VirtualMachine vm, Backup backup) {
+ LOG.debug("Restoring vm " + vm.getUuid() + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+ String networkerServer;
+ HostVO hostVO;
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return false;
+ }
+
+ // Find where the VM was last running
+ hostVO = getLastVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -S " + SSID;
+
+ if (NetworkerClientVerboseLogs.value())
+ command = command + " -v ";
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for VM ID " + vm.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return true;
+ } else {
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return false;
+ }
+ }
+
+ @Override
+ public Pair<Boolean, String> restoreBackedUpVolume(Backup backup, String volumeUuid, String hostIp, String dataStoreUuid) {
+ LOG.debug("Restoring volume " + volumeUuid + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+
+ String networkerServer;
+ VolumeVO volume = volumeDao.findByUuid(volumeUuid);
+ VMInstanceVO backupSourceVm = vmInstanceDao.findById(backup.getVmId());
+ StoragePoolHostVO dataStore = storagePoolHostDao.findByUuid(dataStoreUuid);
+ HostVO hostVO = hostDao.findByIp(hostIp);
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+ final String clusterName = networkerBackup.getClientHostname();
+ final String destinationNetworkerClient = hostVO.getName().split("\\.")[0];
+ Long restoredVolumeDiskSize = 0L;
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return null;
+ }
+
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ // Find volume size from backup vols
+ for ( Backup.VolumeInfo VMVolToRestore : backupSourceVm.getBackupVolumeList()) {
+ if (VMVolToRestore.getUuid().equals(volumeUuid))
+ restoredVolumeDiskSize = (VMVolToRestore.getSize());
+ }
+
+ VolumeVO restoredVolume = new VolumeVO(Volume.Type.DATADISK, null, backup.getZoneId(),
+ backup.getDomainId(), backup.getAccountId(), 0, null,
+ backup.getSize(), null, null, null);
+
+ restoredVolume.setName("RV-"+volume.getName());
+ restoredVolume.setProvisioningType(volume.getProvisioningType());
+ restoredVolume.setUpdated(new Date());
+ restoredVolume.setUuid(UUID.randomUUID().toString());
+ restoredVolume.setRemoved(null);
+ restoredVolume.setDisplayVolume(true);
+ restoredVolume.setPoolId(volume.getPoolId());
+ restoredVolume.setPath(restoredVolume.getUuid());
+ restoredVolume.setState(Volume.State.Copying);
+ restoredVolume.setSize(restoredVolumeDiskSize);
+ restoredVolume.setDiskOfferingId(volume.getDiskOfferingId());
+
+ try {
+ volumeDao.persist(restoredVolume);
+ } catch (Exception e) {
+ throw new CloudRuntimeException("Unable to craft restored volume due to: "+e);
+ }
+
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -c " + clusterName +
+ " -d " + destinationNetworkerClient +
+ " -n " + restoredVolume.getUuid() +
+ " -p " + dataStore.getLocalPath() +
+ " -a " + volume.getUuid();
+
+ if (NetworkerClientVerboseLogs.value())
+ command = command + " -v ";
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for Volume UUID " + volume.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return new Pair<>(true,restoredVolume.getUuid());
+ } else {
+ volumeDao.expunge(restoredVolume.getId());
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return null;
+ }
+ }
+
+ @Override
+ public boolean takeBackup(VirtualMachine vm) {
+
+ String networkerServer;
+ String clusterName;
+
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ // Find where the VM is currently running
+ HostVO hostVO = getRunningVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ // Get retention Period for our Backup
+ BackupOfferingVO vmBackupOffering = new BackupOfferingDaoImpl().findById(vm.getBackupOfferingId());
+ final String backupProviderPolicyId = vmBackupOffering.getExternalId();
+ String backupRentionPeriod = getClient(vm.getDataCenterId()).getBackupPolicyRetentionInterval(backupProviderPolicyId);
+
+ if ( backupRentionPeriod == null ) {
+ LOG.warn("There is no retention setting for Emc Networker Policy, setting default for 1 day");
+ backupRentionPeriod="1 Day";
+ }
+
+ // Get Cluster
+ clusterName = getVMHypervisorCluster(hostVO);
+
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmbackup.sh" +
+ " -s " + networkerServer +
Review Comment:
Any update on this?
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/NetworkerBackupProvider.java:
##########
@@ -0,0 +1,637 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.backup;
+
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.host.HostVO;
+import com.cloud.host.Status;
+import com.cloud.host.dao.HostDao;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.StoragePoolHostVO;
+import com.cloud.storage.Volume;
+import com.cloud.storage.VolumeVO;
+import com.cloud.storage.dao.StoragePoolHostDao;
+import com.cloud.storage.dao.VolumeDao;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.component.AdapterBase;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallbackNoReturn;
+import com.cloud.utils.db.TransactionStatus;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.ssh.SSHCmdHelper;
+import com.cloud.utils.ssh.SshException;
+import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.dao.VMInstanceDao;
+import org.apache.cloudstack.api.InternalIdentity;
+import org.apache.cloudstack.backup.dao.BackupDao;
+import org.apache.cloudstack.backup.dao.BackupOfferingDaoImpl;
+import org.apache.cloudstack.backup.networker.NetworkerClient;
+import org.apache.cloudstack.framework.config.ConfigKey;
+import org.apache.cloudstack.framework.config.Configurable;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.log4j.Logger;
+import org.apache.xml.utils.URI;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+import javax.inject.Inject;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.HashMap;
+import java.util.Date;
+import java.util.Objects;
+import java.util.UUID;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+public class NetworkerBackupProvider extends AdapterBase implements BackupProvider, Configurable {
Review Comment:
nit - newline
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/NetworkerBackupOffering.java:
##########
@@ -0,0 +1,75 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.backup.networker;
+
+import org.apache.cloudstack.backup.BackupOffering;
+import java.util.Date;
+public class NetworkerBackupOffering implements BackupOffering {
Review Comment:
nit - newline
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/NetworkerBackupProvider.java:
##########
@@ -0,0 +1,640 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.backup;
+
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.host.HostVO;
+import com.cloud.host.Status;
+import com.cloud.host.dao.HostDao;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.StoragePoolHostVO;
+import com.cloud.storage.Volume;
+import com.cloud.storage.VolumeVO;
+import com.cloud.storage.dao.StoragePoolHostDao;
+import com.cloud.storage.dao.VolumeDao;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.component.AdapterBase;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallbackNoReturn;
+import com.cloud.utils.db.TransactionStatus;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.ssh.SSHCmdHelper;
+import com.cloud.utils.ssh.SshException;
+import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.dao.VMInstanceDao;
+import org.apache.cloudstack.api.InternalIdentity;
+import org.apache.cloudstack.backup.dao.BackupDao;
+import org.apache.cloudstack.backup.dao.BackupOfferingDaoImpl;
+import org.apache.cloudstack.backup.networker.NetworkerClient;
+import org.apache.cloudstack.framework.config.ConfigKey;
+import org.apache.cloudstack.framework.config.Configurable;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.log4j.Logger;
+import org.apache.xml.utils.URI;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+
+
+import javax.inject.Inject;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.HashMap;
+import java.util.Date;
+import java.util.Objects;
+import java.util.UUID;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+
+public class NetworkerBackupProvider extends AdapterBase implements BackupProvider, Configurable {
+
+ public static final String BACKUP_IDENTIFIER = "-CSBKP-";
+ private static final Logger LOG = Logger.getLogger(NetworkerBackupProvider.class);
+ public ConfigKey<String> NetworkerUrl = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.url", "https://localhost:9090/nwrestapi/v3",
+ "The EMC Networker API URL.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerUsername = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.username", "administrator",
+ "The EMC Networker API username.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerPassword = new ConfigKey<>("Secure", String.class,
+ "backup.plugin.networker.password", "password",
+ "The EMC Networker API password.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerMediaPool = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.pool", "Default",
+ "The EMC Networker Media Pool", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerValidateSSLSecurity = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.validate.ssl", "false",
+ "Validate the SSL certificate when connecting to EMC Networker API service.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Integer> NetworkerApiRequestTimeout = new ConfigKey<>("Advanced", Integer.class,
+ "backup.plugin.networker.request.timeout", "300",
+ "The EMC Networker API request timeout in seconds.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerClientVerboseLogs = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.client.verbosity", "false",
+ "Produce Verbose logs in Hypervisor", true, ConfigKey.Scope.Zone);
+
+ @Inject
+ private BackupDao backupDao;
+
+ @Inject
+ private HostDao hostDao;
+
+ @Inject
+ private ConfigurationDao configDao;
+
+ @Inject
+ private ClusterDao clusterDao;
+
+ @Inject
+ private VolumeDao volumeDao;
+
+ @Inject
+ private StoragePoolHostDao storagePoolHostDao;
+
+ @Inject
+ private VMInstanceDao vmInstanceDao;
+
+ private static String getUrlDomain(String url) throws URISyntaxException {
+ URI uri;
+ try {
+ uri = new URI(url);
+ } catch (URI.MalformedURIException e) {
+ throw new CloudRuntimeException("Failed to cast URI");
+ }
+
+ return uri.getHost();
+ }
+
+ @Override
+ public ConfigKey<?>[] getConfigKeys() {
+ return new ConfigKey[]{
+ NetworkerUrl,
+ NetworkerUsername,
+ NetworkerPassword,
+ NetworkerValidateSSLSecurity,
+ NetworkerApiRequestTimeout,
+ NetworkerMediaPool,
+ NetworkerClientVerboseLogs
+ };
+ }
+
+ @Override
+ public String getName() {
+ return "networker";
+ }
+
+ @Override
+ public String getDescription() {
+ return "EMC Networker Backup Plugin";
+ }
+
+ @Override
+ public String getConfigComponentName() {
+ return BackupService.class.getSimpleName();
+ }
+
+ protected HostVO getLastVMHypervisorHost(VirtualMachine vm) {
+ HostVO host;
+ Long hostId = vm.getLastHostId();
+
+ if (hostId == null) {
+ LOG.debug("Cannot find last host for vm. This should never happen, please check your database.");
+ return null;
+ }
+ host = hostDao.findById(hostId);
+
+ if ( host.getStatus() == Status.Up ) {
+ return host;
+ } else {
+ // Try to find a host in the same cluster
+ List<HostVO> altClusterHosts = hostDao.findHypervisorHostInCluster(host.getClusterId());
+ for (final HostVO candidateClusterHost : altClusterHosts) {
+ if ( candidateClusterHost.getStatus() == Status.Up ) {
+ LOG.debug("Found Host " + candidateClusterHost.getName());
+ return candidateClusterHost;
+ }
+ }
+ }
+ // Try to find a Host in the zone
+ List<HostVO> altZoneHosts = hostDao.findByDataCenterId(host.getDataCenterId());
+ for (final HostVO candidateZoneHost : altZoneHosts) {
+ if ( candidateZoneHost.getStatus() == Status.Up && candidateZoneHost.getHypervisorType() == Hypervisor.HypervisorType.KVM ) {
+ LOG.debug("Found Host " + candidateZoneHost.getName());
+ return candidateZoneHost;
+ }
+ }
+ return null;
+ }
+
+ protected HostVO getRunningVMHypervisorHost(VirtualMachine vm) {
+
+ HostVO host;
+ Long hostId = vm.getHostId();
+
+ if (hostId == null) {
+ throw new CloudRuntimeException("Unable to find the HYPERVISOR for " + vm.getName() + ". Make sure the virtual machine is running");
+ }
+
+ host = hostDao.findById(hostId);
+
+ return host;
+ }
+
+ protected String getVMHypervisorCluster(HostVO host) {
+
+ return clusterDao.findById(host.getClusterId()).getName();
+ }
+
+ protected Ternary<String, String, String> getKVMHyperisorCredentials(HostVO host) {
+
+ String username = null;
+ String password = null;
+ String privateKey = null;
+
+ if (host != null && host.getHypervisorType() == Hypervisor.HypervisorType.KVM) {
+ hostDao.loadDetails(host);
+ password = host.getDetail("password");
+ username = host.getDetail("username");
+ privateKey = configDao.getValue("ssh.privatekey");
+ }
+ if ((password == null && privateKey == null) || username == null) {
+ throw new CloudRuntimeException("Cannot find login credentials for HYPERVISOR " + Objects.requireNonNull(host).getUuid());
+ }
+
+ return new Ternary<>(username, password, privateKey);
+ }
+
+ private String executeBackupCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+ String nstRegex = "\\bcompleted savetime=([0-9]{10})";
+ Pattern saveTimePattern = Pattern.compile(nstRegex);
+
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ Matcher saveTimeMatcher = saveTimePattern.matcher(result.getStdOut());
+ if (saveTimeMatcher.find()) {
+ return saveTimeMatcher.group(1);
+ }
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+
+ return null;
+ }
+ private boolean executeRestoreCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ return true;
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+ }
+
+ private NetworkerClient getClient(final Long zoneId) {
+ try {
+ return new NetworkerClient(NetworkerUrl.valueIn(zoneId), NetworkerUsername.valueIn(zoneId), NetworkerPassword.valueIn(zoneId),
+ NetworkerValidateSSLSecurity.valueIn(zoneId), NetworkerApiRequestTimeout.valueIn(zoneId));
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException("Failed to parse EMC Networker API URL: " + e.getMessage());
+ } catch (NoSuchAlgorithmException | KeyManagementException e) {
+ LOG.error("Failed to build EMC Networker API client due to: ", e);
+ }
+ throw new CloudRuntimeException("Failed to build EMC Networker API client");
+ }
+
+ @Override
+ public List<BackupOffering> listBackupOfferings(Long zoneId) {
+ List<BackupOffering> policies = new ArrayList<>();
+ for (final BackupOffering policy : getClient(zoneId).listPolicies()) {
+ if (!policy.getName().contains(BACKUP_IDENTIFIER)) {
+ policies.add(policy);
+ }
+ }
+
+ return policies;
+ }
+
+ @Override
+ public boolean isValidProviderOffering(Long zoneId, String uuid) {
+ List<BackupOffering> policies = listBackupOfferings(zoneId);
+ if (CollectionUtils.isEmpty(policies)) {
+ return false;
+ }
+ for (final BackupOffering policy : policies) {
+ if (Objects.equals(policy.getExternalId(), uuid)) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ @Override
+ public boolean assignVMToBackupOffering(VirtualMachine vm, BackupOffering backupOffering) { return true; }
+
+ @Override
+ public boolean removeVMFromBackupOffering(VirtualMachine vm) {
+ LOG.debug("Removing VirtualMachine from Backup offering and Deleting any existing backups");
+
+ List<String> backupsTaken = getClient(vm.getDataCenterId()).getBackupsForVm(vm);
+
+ for (String backupId : backupsTaken) {
+ LOG.debug("Trying to remove backup with id" + backupId);
+ getClient(vm.getDataCenterId()).deleteBackupForVM(backupId);
+ }
+
+ return true;
+ }
+
+ @Override
+ public boolean restoreVMFromBackup(VirtualMachine vm, Backup backup) {
+ LOG.debug("Restoring vm " + vm.getUuid() + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+ String networkerServer;
+ HostVO hostVO;
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return false;
+ }
+
+ // Find where the VM was last running
+ hostVO = getLastVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -S " + SSID;
+
+ if (NetworkerClientVerboseLogs.value())
+ command = command + " -v ";
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for VM ID " + vm.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return true;
+ } else {
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return false;
+ }
+ }
+
+ @Override
+ public Pair<Boolean, String> restoreBackedUpVolume(Backup backup, String volumeUuid, String hostIp, String dataStoreUuid) {
+ LOG.debug("Restoring volume " + volumeUuid + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+
+ String networkerServer;
+ VolumeVO volume = volumeDao.findByUuid(volumeUuid);
+ VMInstanceVO backupSourceVm = vmInstanceDao.findById(backup.getVmId());
+ StoragePoolHostVO dataStore = storagePoolHostDao.findByUuid(dataStoreUuid);
+ HostVO hostVO = hostDao.findByIp(hostIp);
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+ final String clusterName = networkerBackup.getClientHostname();
+ final String destinationNetworkerClient = hostVO.getName().split("\\.")[0];
+ Long restoredVolumeDiskSize = 0L;
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return null;
+ }
+
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ // Find volume size from backup vols
+ for ( Backup.VolumeInfo VMVolToRestore : backupSourceVm.getBackupVolumeList()) {
+ if (VMVolToRestore.getUuid().equals(volumeUuid))
+ restoredVolumeDiskSize = (VMVolToRestore.getSize());
+ }
+
+ VolumeVO restoredVolume = new VolumeVO(Volume.Type.DATADISK, null, backup.getZoneId(),
+ backup.getDomainId(), backup.getAccountId(), 0, null,
+ backup.getSize(), null, null, null);
+
+ restoredVolume.setName("RV-"+volume.getName());
+ restoredVolume.setProvisioningType(volume.getProvisioningType());
+ restoredVolume.setUpdated(new Date());
+ restoredVolume.setUuid(UUID.randomUUID().toString());
+ restoredVolume.setRemoved(null);
+ restoredVolume.setDisplayVolume(true);
+ restoredVolume.setPoolId(volume.getPoolId());
+ restoredVolume.setPath(restoredVolume.getUuid());
+ restoredVolume.setState(Volume.State.Copying);
+ restoredVolume.setSize(restoredVolumeDiskSize);
+ restoredVolume.setDiskOfferingId(volume.getDiskOfferingId());
+
+ try {
+ volumeDao.persist(restoredVolume);
+ } catch (Exception e) {
+ throw new CloudRuntimeException("Unable to craft restored volume due to: "+e);
+ }
+
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -c " + clusterName +
+ " -d " + destinationNetworkerClient +
+ " -n " + restoredVolume.getUuid() +
Review Comment:
Any update on this?
##########
server/src/main/java/com/cloud/hypervisor/KVMGuru.java:
##########
@@ -167,9 +213,9 @@ protected void configureVmMemoryAndCpuCores(VirtualMachineTO virtualMachineTo, H
Long maxHostMemory = max.first();
Integer maxHostCpuCore = max.second();
- Long minMemory = virtualMachineTo.getMinRam();
+ long minMemory = virtualMachineTo.getMinRam();
Long maxMemory = minMemory;
- Integer minCpuCores = virtualMachineTo.getCpus();
+ int minCpuCores = virtualMachineTo.getCpus();
Review Comment:
Concern - this can cause NPE if the TO doesn't have the value set.
##########
server/src/main/java/com/cloud/hypervisor/KVMGuru.java:
##########
@@ -139,7 +185,7 @@ protected void configureVmOsDescription(VirtualMachine virtualMachine, VirtualMa
guestOsMapping = _guestOsHypervisorDao.findByOsIdAndHypervisor(guestOS.getId(), getHypervisorType().toString(), hostVo.getHypervisorVersion());
}
- if (guestOsMapping == null || hostVo == null) {
+ if (guestOsMapping == null) {
Review Comment:
Could this have regression of some kind? cc @fermosan @DaanHoogland
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/NetworkerBackupProvider.java:
##########
@@ -0,0 +1,637 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.backup;
+
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.host.HostVO;
+import com.cloud.host.Status;
+import com.cloud.host.dao.HostDao;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.StoragePoolHostVO;
+import com.cloud.storage.Volume;
+import com.cloud.storage.VolumeVO;
+import com.cloud.storage.dao.StoragePoolHostDao;
+import com.cloud.storage.dao.VolumeDao;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.component.AdapterBase;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallbackNoReturn;
+import com.cloud.utils.db.TransactionStatus;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.ssh.SSHCmdHelper;
+import com.cloud.utils.ssh.SshException;
+import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.dao.VMInstanceDao;
+import org.apache.cloudstack.api.InternalIdentity;
+import org.apache.cloudstack.backup.dao.BackupDao;
+import org.apache.cloudstack.backup.dao.BackupOfferingDaoImpl;
+import org.apache.cloudstack.backup.networker.NetworkerClient;
+import org.apache.cloudstack.framework.config.ConfigKey;
+import org.apache.cloudstack.framework.config.Configurable;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.log4j.Logger;
+import org.apache.xml.utils.URI;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+import javax.inject.Inject;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.HashMap;
+import java.util.Date;
+import java.util.Objects;
+import java.util.UUID;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+public class NetworkerBackupProvider extends AdapterBase implements BackupProvider, Configurable {
+
+ public static final String BACKUP_IDENTIFIER = "-CSBKP-";
+ private static final Logger LOG = Logger.getLogger(NetworkerBackupProvider.class);
+ public ConfigKey<String> NetworkerUrl = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.url", "https://localhost:9090/nwrestapi/v3",
+ "The EMC Networker API URL.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerUsername = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.username", "administrator",
+ "The EMC Networker API username.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerPassword = new ConfigKey<>("Secure", String.class,
+ "backup.plugin.networker.password", "password",
+ "The EMC Networker API password.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerMediaPool = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.pool", "Default",
+ "The EMC Networker Media Pool", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerValidateSSLSecurity = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.validate.ssl", "false",
+ "Validate the SSL certificate when connecting to EMC Networker API service.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Integer> NetworkerApiRequestTimeout = new ConfigKey<>("Advanced", Integer.class,
+ "backup.plugin.networker.request.timeout", "300",
+ "The EMC Networker API request timeout in seconds.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerClientVerboseLogs = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.client.verbosity", "false",
+ "Produce Verbose logs in Hypervisor", true, ConfigKey.Scope.Zone);
+
+ @Inject
+ private BackupDao backupDao;
+
+ @Inject
+ private HostDao hostDao;
+
+ @Inject
+ private ConfigurationDao configDao;
+
+ @Inject
+ private ClusterDao clusterDao;
+
+ @Inject
+ private VolumeDao volumeDao;
+
+ @Inject
+ private StoragePoolHostDao storagePoolHostDao;
+
+ @Inject
+ private VMInstanceDao vmInstanceDao;
+
+ private static String getUrlDomain(String url) throws URISyntaxException {
+ URI uri;
+ try {
+ uri = new URI(url);
+ } catch (URI.MalformedURIException e) {
+ throw new CloudRuntimeException("Failed to cast URI");
+ }
+
+ return uri.getHost();
+ }
+
+ @Override
+ public ConfigKey<?>[] getConfigKeys() {
+ return new ConfigKey[]{
+ NetworkerUrl,
+ NetworkerUsername,
+ NetworkerPassword,
+ NetworkerValidateSSLSecurity,
+ NetworkerApiRequestTimeout,
+ NetworkerMediaPool,
+ NetworkerClientVerboseLogs
+ };
+ }
+
+ @Override
+ public String getName() {
+ return "networker";
+ }
+
+ @Override
+ public String getDescription() {
+ return "EMC Networker Backup Plugin";
+ }
+
+ @Override
+ public String getConfigComponentName() {
+ return BackupService.class.getSimpleName();
+ }
+
+ protected HostVO getLastVMHypervisorHost(VirtualMachine vm) {
+ HostVO host;
+ Long hostId = vm.getLastHostId();
+
+ if (hostId == null) {
+ LOG.debug("Cannot find last host for vm. This should never happen, please check your database.");
+ return null;
+ }
+ host = hostDao.findById(hostId);
+
+ if ( host.getStatus() == Status.Up ) {
+ return host;
+ } else {
+ // Try to find a host in the same cluster
+ List<HostVO> altClusterHosts = hostDao.findHypervisorHostInCluster(host.getClusterId());
+ for (final HostVO candidateClusterHost : altClusterHosts) {
+ if ( candidateClusterHost.getStatus() == Status.Up ) {
+ LOG.debug("Found Host " + candidateClusterHost.getName());
+ return candidateClusterHost;
+ }
+ }
+ }
+ // Try to find a Host in the zone
+ List<HostVO> altZoneHosts = hostDao.findByDataCenterId(host.getDataCenterId());
+ for (final HostVO candidateZoneHost : altZoneHosts) {
+ if ( candidateZoneHost.getStatus() == Status.Up && candidateZoneHost.getHypervisorType() == Hypervisor.HypervisorType.KVM ) {
+ LOG.debug("Found Host " + candidateZoneHost.getName());
+ return candidateZoneHost;
+ }
+ }
+ return null;
+ }
+
+ protected HostVO getRunningVMHypervisorHost(VirtualMachine vm) {
+
+ HostVO host;
+ Long hostId = vm.getHostId();
+
+ if (hostId == null) {
+ throw new CloudRuntimeException("Unable to find the HYPERVISOR for " + vm.getName() + ". Make sure the virtual machine is running");
+ }
+
+ host = hostDao.findById(hostId);
+
+ return host;
+ }
+
+ protected String getVMHypervisorCluster(HostVO host) {
+
+ return clusterDao.findById(host.getClusterId()).getName();
+ }
+
+ protected Ternary<String, String, String> getKVMHyperisorCredentials(HostVO host) {
+
+ String username = null;
+ String password = null;
+ String privateKey = null;
+
+ if (host != null && host.getHypervisorType() == Hypervisor.HypervisorType.KVM) {
+ hostDao.loadDetails(host);
+ password = host.getDetail("password");
+ username = host.getDetail("username");
+ privateKey = configDao.getValue("ssh.privatekey");
+ }
+ if ((password == null && privateKey == null) || username == null) {
+ throw new CloudRuntimeException("Cannot find login credentials for HYPERVISOR " + Objects.requireNonNull(host).getUuid());
+ }
+
+ return new Ternary<>(username, password, privateKey);
+ }
+
+ private String executeBackupCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+ String nstRegex = "\\bcompleted savetime=([0-9]{10})";
+ Pattern saveTimePattern = Pattern.compile(nstRegex);
+
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ Matcher saveTimeMatcher = saveTimePattern.matcher(result.getStdOut());
+ if (saveTimeMatcher.find()) {
+ return saveTimeMatcher.group(1);
+ }
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+
+ return null;
+ }
+ private boolean executeRestoreCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ return true;
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+ }
+
+ private NetworkerClient getClient(final Long zoneId) {
+ try {
+ return new NetworkerClient(NetworkerUrl.valueIn(zoneId), NetworkerUsername.valueIn(zoneId), NetworkerPassword.valueIn(zoneId),
+ NetworkerValidateSSLSecurity.valueIn(zoneId), NetworkerApiRequestTimeout.valueIn(zoneId));
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException("Failed to parse EMC Networker API URL: " + e.getMessage());
+ } catch (NoSuchAlgorithmException | KeyManagementException e) {
+ LOG.error("Failed to build EMC Networker API client due to: ", e);
+ }
+ throw new CloudRuntimeException("Failed to build EMC Networker API client");
+ }
+
+ @Override
+ public List<BackupOffering> listBackupOfferings(Long zoneId) {
+ List<BackupOffering> policies = new ArrayList<>();
+ for (final BackupOffering policy : getClient(zoneId).listPolicies()) {
+ if (!policy.getName().contains(BACKUP_IDENTIFIER)) {
+ policies.add(policy);
+ }
+ }
+
+ return policies;
+ }
+
+ @Override
+ public boolean isValidProviderOffering(Long zoneId, String uuid) {
+ List<BackupOffering> policies = listBackupOfferings(zoneId);
+ if (CollectionUtils.isEmpty(policies)) {
+ return false;
+ }
+ for (final BackupOffering policy : policies) {
+ if (Objects.equals(policy.getExternalId(), uuid)) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ @Override
+ public boolean assignVMToBackupOffering(VirtualMachine vm, BackupOffering backupOffering) { return true; }
+
+ @Override
+ public boolean removeVMFromBackupOffering(VirtualMachine vm) {
+ LOG.debug("Removing VirtualMachine from Backup offering and Deleting any existing backups");
+
+ List<String> backupsTaken = getClient(vm.getDataCenterId()).getBackupsForVm(vm);
+
+ for (String backupId : backupsTaken) {
+ LOG.debug("Trying to remove backup with id" + backupId);
+ getClient(vm.getDataCenterId()).deleteBackupForVM(backupId);
+ }
+
+ return true;
+ }
+
+ @Override
+ public boolean restoreVMFromBackup(VirtualMachine vm, Backup backup) {
+ LOG.debug("Restoring vm " + vm.getUuid() + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+ String networkerServer;
+ HostVO hostVO;
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return false;
+ }
+
+ // Find where the VM was last running
+ hostVO = getLastVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -S " + SSID;
+
+ if ( Boolean.TRUE.equals(NetworkerClientVerboseLogs.value()) )
+ command = command + " -v ";
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for VM ID " + vm.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return true;
+ } else {
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return false;
+ }
+ }
+
+ @Override
+ public Pair<Boolean, String> restoreBackedUpVolume(Backup backup, String volumeUuid, String hostIp, String dataStoreUuid) {
+ LOG.debug("Restoring volume " + volumeUuid + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+
+ String networkerServer;
+ VolumeVO volume = volumeDao.findByUuid(volumeUuid);
+ VMInstanceVO backupSourceVm = vmInstanceDao.findById(backup.getVmId());
+ StoragePoolHostVO dataStore = storagePoolHostDao.findByUuid(dataStoreUuid);
+ HostVO hostVO = hostDao.findByIp(hostIp);
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+ final String clusterName = networkerBackup.getClientHostname();
+ final String destinationNetworkerClient = hostVO.getName().split("\\.")[0];
+ Long restoredVolumeDiskSize = 0L;
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return null;
+ }
+
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ // Find volume size from backup vols
+ for ( Backup.VolumeInfo VMVolToRestore : backupSourceVm.getBackupVolumeList()) {
+ if (VMVolToRestore.getUuid().equals(volumeUuid))
+ restoredVolumeDiskSize = (VMVolToRestore.getSize());
+ }
+
+ VolumeVO restoredVolume = new VolumeVO(Volume.Type.DATADISK, null, backup.getZoneId(),
+ backup.getDomainId(), backup.getAccountId(), 0, null,
+ backup.getSize(), null, null, null);
+
+ restoredVolume.setName("RV-"+volume.getName());
+ restoredVolume.setProvisioningType(volume.getProvisioningType());
+ restoredVolume.setUpdated(new Date());
+ restoredVolume.setUuid(UUID.randomUUID().toString());
+ restoredVolume.setRemoved(null);
+ restoredVolume.setDisplayVolume(true);
+ restoredVolume.setPoolId(volume.getPoolId());
+ restoredVolume.setPath(restoredVolume.getUuid());
+ restoredVolume.setState(Volume.State.Copying);
+ restoredVolume.setSize(restoredVolumeDiskSize);
+ restoredVolume.setDiskOfferingId(volume.getDiskOfferingId());
+
+ try {
+ volumeDao.persist(restoredVolume);
+ } catch (Exception e) {
+ throw new CloudRuntimeException("Unable to craft restored volume due to: "+e);
+ }
+
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -c " + clusterName +
+ " -d " + destinationNetworkerClient +
+ " -n " + restoredVolume.getUuid() +
+ " -p " + dataStore.getLocalPath() +
+ " -a " + volume.getUuid();
+
+ if ( Boolean.TRUE.equals(NetworkerClientVerboseLogs.value()) )
+ command = command + " -v ";
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for Volume UUID " + volume.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return new Pair<>(true,restoredVolume.getUuid());
+ } else {
+ volumeDao.expunge(restoredVolume.getId());
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return null;
+ }
+ }
+
+ @Override
+ public boolean takeBackup(VirtualMachine vm) {
+
+ String networkerServer;
+ String clusterName;
+
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ // Find where the VM is currently running
+ HostVO hostVO = getRunningVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ // Get retention Period for our Backup
+ BackupOfferingVO vmBackupOffering = new BackupOfferingDaoImpl().findById(vm.getBackupOfferingId());
+ final String backupProviderPolicyId = vmBackupOffering.getExternalId();
+ String backupRentionPeriod = getClient(vm.getDataCenterId()).getBackupPolicyRetentionInterval(backupProviderPolicyId);
+
+ if ( backupRentionPeriod == null ) {
+ LOG.warn("There is no retention setting for Emc Networker Policy, setting default for 1 day");
+ backupRentionPeriod="1 Day";
+ }
+
+ // Get Cluster
+ clusterName = getVMHypervisorCluster(hostVO);
+
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmbackup.sh" +
+ " -s " + networkerServer +
+ " -R '" + backupRentionPeriod + "'" +
+ " -P " + NetworkerMediaPool.valueIn(vm.getDataCenterId()) +
+ " -c " + clusterName +
+ " -u " + vm.getUuid() +
+ " -t " + vm.getName();
+ if ( Boolean.TRUE.equals(NetworkerClientVerboseLogs.value()) )
+ command = command + " -v ";
+
+ LOG.debug("Starting backup for VM ID " + vm.getUuid() + " on Networker provider");
+ Date backupJobStart = new Date();
+
+ String saveTime = executeBackupCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command);
+ BackupVO backup = getClient(vm.getDataCenterId()).registerBackupForVm(vm, backupJobStart, saveTime);
+ if (backup != null) {
+ backupDao.persist(backup);
+ return true;
+ } else {
+ // We need to handle this rare situation where backup is successful but can't be registered properly.
+ return false;
+ }
+ }
+
+ @Override
+ public boolean deleteBackup(Backup backup) {
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+
+ if (getClient(zoneId).deleteBackupForVM(externalBackupId)) {
+ LOG.debug("EMC Networker successfully deleted backup with id " + externalBackupId);
+ return true;
+ } else {
+ LOG.debug("There was an error removing the backup with id " + externalBackupId + " from EMC NEtworker");
+ }
+ return false;
+ }
+
+ @Override
+ public Map<VirtualMachine, Backup.Metric> getBackupMetrics(Long zoneId, List<VirtualMachine> vms) {
+ final Map<VirtualMachine, Backup.Metric> metrics = new HashMap<>();
+ Long vmBackupSize=0L;
+ Long vmBackupProtectedSize=0L;
+
+ if (CollectionUtils.isEmpty(vms)) {
+ LOG.warn("Unable to get VM Backup Metrics because the list of VMs is empty.");
+ return metrics;
+ }
+
+ for (final VirtualMachine vm : vms) {
+ for ( Backup.VolumeInfo thisVMVol : vm.getBackupVolumeList()) {
+ vmBackupSize += (thisVMVol.getSize() / 1024L / 1024L);
+ }
+ final ArrayList<String> vmBackups = getClient(zoneId).getBackupsForVm(vm);
+ for ( String vmBackup : vmBackups ) {
+ NetworkerBackup vmNwBackup = getClient(zoneId).getNetworkerBackupInfo(vmBackup);
+ vmBackupProtectedSize+= vmNwBackup.getSize().getValue() / 1024L;
+ }
+ Backup.Metric vmBackupMetric = new Backup.Metric(vmBackupSize,vmBackupProtectedSize);
+ LOG.debug(String.format("Metrics for VM [uuid: %s, name: %s] is [backup size: %s, data size: %s].", vm.getUuid(),
+ vm.getInstanceName(), vmBackupMetric.getBackupSize(), vmBackupMetric.getDataSize()));
+ metrics.put(vm, vmBackupMetric);
+ }
+ return metrics;
+ }
+
+ @Override
+ public void syncBackups(VirtualMachine vm, Backup.Metric metric) {
Review Comment:
@fermosan this is only necessary, if the backup system does out of band backups that should be sync'd to CloudStack. Does EMC networker do out of band backups for VMs?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1287117203
@rohityadavcloud awaiting your final words ;)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1199294152
UI build: :heavy_check_mark:
Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6550 (SL-JID-2032)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1181987261
@rohityadavcloud a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r1009286233
##########
ui/public/config.json:
##########
@@ -63,5 +63,121 @@
"basicZoneEnabled": true,
"multipleServer": false,
"allowSettingTheme": true,
- "docHelpMappings": {}
+ "docHelpMappings": {
Review Comment:
Maybe this change in config.json was after the rebase ? NPM is at 12.13.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1218239924
@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1222043866
@harikrishna-patnala I'm going to close/open to see if we can get some sonarcube output. Other than that I am counting two LGTM and testing performed and will merge. Do you want to have a second look, as well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r951215713
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/NetworkerBackupProvider.java:
##########
@@ -0,0 +1,640 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.backup;
+
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.host.HostVO;
+import com.cloud.host.Status;
+import com.cloud.host.dao.HostDao;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.StoragePoolHostVO;
+import com.cloud.storage.Volume;
+import com.cloud.storage.VolumeVO;
+import com.cloud.storage.dao.StoragePoolHostDao;
+import com.cloud.storage.dao.VolumeDao;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.component.AdapterBase;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallbackNoReturn;
+import com.cloud.utils.db.TransactionStatus;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.ssh.SSHCmdHelper;
+import com.cloud.utils.ssh.SshException;
+import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.dao.VMInstanceDao;
+import org.apache.cloudstack.api.InternalIdentity;
+import org.apache.cloudstack.backup.dao.BackupDao;
+import org.apache.cloudstack.backup.dao.BackupOfferingDaoImpl;
+import org.apache.cloudstack.backup.networker.NetworkerClient;
+import org.apache.cloudstack.framework.config.ConfigKey;
+import org.apache.cloudstack.framework.config.Configurable;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.log4j.Logger;
+import org.apache.xml.utils.URI;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+
+
+import javax.inject.Inject;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.HashMap;
+import java.util.Date;
+import java.util.Objects;
+import java.util.UUID;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+
+public class NetworkerBackupProvider extends AdapterBase implements BackupProvider, Configurable {
+
+ public static final String BACKUP_IDENTIFIER = "-CSBKP-";
+ private static final Logger LOG = Logger.getLogger(NetworkerBackupProvider.class);
+ public ConfigKey<String> NetworkerUrl = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.url", "https://localhost:9090/nwrestapi/v3",
+ "The EMC Networker API URL.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerUsername = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.username", "administrator",
+ "The EMC Networker API username.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerPassword = new ConfigKey<>("Secure", String.class,
+ "backup.plugin.networker.password", "password",
+ "The EMC Networker API password.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerMediaPool = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.pool", "Default",
+ "The EMC Networker Media Pool", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerValidateSSLSecurity = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.validate.ssl", "false",
+ "Validate the SSL certificate when connecting to EMC Networker API service.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Integer> NetworkerApiRequestTimeout = new ConfigKey<>("Advanced", Integer.class,
+ "backup.plugin.networker.request.timeout", "300",
+ "The EMC Networker API request timeout in seconds.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerClientVerboseLogs = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.client.verbosity", "false",
+ "Produce Verbose logs in Hypervisor", true, ConfigKey.Scope.Zone);
+
+ @Inject
+ private BackupDao backupDao;
+
+ @Inject
+ private HostDao hostDao;
+
+ @Inject
+ private ConfigurationDao configDao;
+
+ @Inject
+ private ClusterDao clusterDao;
+
+ @Inject
+ private VolumeDao volumeDao;
+
+ @Inject
+ private StoragePoolHostDao storagePoolHostDao;
+
+ @Inject
+ private VMInstanceDao vmInstanceDao;
+
+ private static String getUrlDomain(String url) throws URISyntaxException {
+ URI uri;
+ try {
+ uri = new URI(url);
+ } catch (URI.MalformedURIException e) {
+ throw new CloudRuntimeException("Failed to cast URI");
+ }
+
+ return uri.getHost();
+ }
+
+ @Override
+ public ConfigKey<?>[] getConfigKeys() {
+ return new ConfigKey[]{
+ NetworkerUrl,
+ NetworkerUsername,
+ NetworkerPassword,
+ NetworkerValidateSSLSecurity,
+ NetworkerApiRequestTimeout,
+ NetworkerMediaPool,
+ NetworkerClientVerboseLogs
+ };
+ }
+
+ @Override
+ public String getName() {
+ return "networker";
+ }
+
+ @Override
+ public String getDescription() {
+ return "EMC Networker Backup Plugin";
+ }
+
+ @Override
+ public String getConfigComponentName() {
+ return BackupService.class.getSimpleName();
+ }
+
+ protected HostVO getLastVMHypervisorHost(VirtualMachine vm) {
+ HostVO host;
+ Long hostId = vm.getLastHostId();
+
+ if (hostId == null) {
+ LOG.debug("Cannot find last host for vm. This should never happen, please check your database.");
+ return null;
+ }
+ host = hostDao.findById(hostId);
+
+ if ( host.getStatus() == Status.Up ) {
+ return host;
+ } else {
+ // Try to find a host in the same cluster
+ List<HostVO> altClusterHosts = hostDao.findHypervisorHostInCluster(host.getClusterId());
+ for (final HostVO candidateClusterHost : altClusterHosts) {
+ if ( candidateClusterHost.getStatus() == Status.Up ) {
+ LOG.debug("Found Host " + candidateClusterHost.getName());
+ return candidateClusterHost;
+ }
+ }
+ }
+ // Try to find a Host in the zone
+ List<HostVO> altZoneHosts = hostDao.findByDataCenterId(host.getDataCenterId());
+ for (final HostVO candidateZoneHost : altZoneHosts) {
+ if ( candidateZoneHost.getStatus() == Status.Up && candidateZoneHost.getHypervisorType() == Hypervisor.HypervisorType.KVM ) {
+ LOG.debug("Found Host " + candidateZoneHost.getName());
+ return candidateZoneHost;
+ }
+ }
+ return null;
+ }
+
+ protected HostVO getRunningVMHypervisorHost(VirtualMachine vm) {
+
+ HostVO host;
+ Long hostId = vm.getHostId();
+
+ if (hostId == null) {
+ throw new CloudRuntimeException("Unable to find the HYPERVISOR for " + vm.getName() + ". Make sure the virtual machine is running");
+ }
+
+ host = hostDao.findById(hostId);
+
+ return host;
+ }
+
+ protected String getVMHypervisorCluster(HostVO host) {
+
+ return clusterDao.findById(host.getClusterId()).getName();
+ }
+
+ protected Ternary<String, String, String> getKVMHyperisorCredentials(HostVO host) {
+
+ String username = null;
+ String password = null;
+ String privateKey = null;
+
+ if (host != null && host.getHypervisorType() == Hypervisor.HypervisorType.KVM) {
+ hostDao.loadDetails(host);
+ password = host.getDetail("password");
+ username = host.getDetail("username");
+ privateKey = configDao.getValue("ssh.privatekey");
+ }
+ if ((password == null && privateKey == null) || username == null) {
+ throw new CloudRuntimeException("Cannot find login credentials for HYPERVISOR " + Objects.requireNonNull(host).getUuid());
+ }
+
+ return new Ternary<>(username, password, privateKey);
+ }
+
+ private String executeBackupCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+ String nstRegex = "\\bcompleted savetime=([0-9]{10})";
+ Pattern saveTimePattern = Pattern.compile(nstRegex);
+
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ Matcher saveTimeMatcher = saveTimePattern.matcher(result.getStdOut());
+ if (saveTimeMatcher.find()) {
+ return saveTimeMatcher.group(1);
+ }
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+
+ return null;
+ }
+ private boolean executeRestoreCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ return true;
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+ }
+
+ private NetworkerClient getClient(final Long zoneId) {
+ try {
+ return new NetworkerClient(NetworkerUrl.valueIn(zoneId), NetworkerUsername.valueIn(zoneId), NetworkerPassword.valueIn(zoneId),
+ NetworkerValidateSSLSecurity.valueIn(zoneId), NetworkerApiRequestTimeout.valueIn(zoneId));
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException("Failed to parse EMC Networker API URL: " + e.getMessage());
+ } catch (NoSuchAlgorithmException | KeyManagementException e) {
+ LOG.error("Failed to build EMC Networker API client due to: ", e);
+ }
+ throw new CloudRuntimeException("Failed to build EMC Networker API client");
+ }
+
+ @Override
+ public List<BackupOffering> listBackupOfferings(Long zoneId) {
+ List<BackupOffering> policies = new ArrayList<>();
+ for (final BackupOffering policy : getClient(zoneId).listPolicies()) {
+ if (!policy.getName().contains(BACKUP_IDENTIFIER)) {
+ policies.add(policy);
+ }
+ }
+
+ return policies;
+ }
+
+ @Override
+ public boolean isValidProviderOffering(Long zoneId, String uuid) {
+ List<BackupOffering> policies = listBackupOfferings(zoneId);
+ if (CollectionUtils.isEmpty(policies)) {
+ return false;
+ }
+ for (final BackupOffering policy : policies) {
+ if (Objects.equals(policy.getExternalId(), uuid)) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ @Override
+ public boolean assignVMToBackupOffering(VirtualMachine vm, BackupOffering backupOffering) { return true; }
+
+ @Override
+ public boolean removeVMFromBackupOffering(VirtualMachine vm) {
+ LOG.debug("Removing VirtualMachine from Backup offering and Deleting any existing backups");
+
+ List<String> backupsTaken = getClient(vm.getDataCenterId()).getBackupsForVm(vm);
+
+ for (String backupId : backupsTaken) {
+ LOG.debug("Trying to remove backup with id" + backupId);
+ getClient(vm.getDataCenterId()).deleteBackupForVM(backupId);
+ }
+
+ return true;
+ }
+
+ @Override
+ public boolean restoreVMFromBackup(VirtualMachine vm, Backup backup) {
+ LOG.debug("Restoring vm " + vm.getUuid() + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+ String networkerServer;
+ HostVO hostVO;
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return false;
+ }
+
+ // Find where the VM was last running
+ hostVO = getLastVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -S " + SSID;
+
+ if (NetworkerClientVerboseLogs.value())
+ command = command + " -v ";
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for VM ID " + vm.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return true;
+ } else {
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return false;
+ }
+ }
+
+ @Override
+ public Pair<Boolean, String> restoreBackedUpVolume(Backup backup, String volumeUuid, String hostIp, String dataStoreUuid) {
+ LOG.debug("Restoring volume " + volumeUuid + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+
+ String networkerServer;
+ VolumeVO volume = volumeDao.findByUuid(volumeUuid);
+ VMInstanceVO backupSourceVm = vmInstanceDao.findById(backup.getVmId());
+ StoragePoolHostVO dataStore = storagePoolHostDao.findByUuid(dataStoreUuid);
+ HostVO hostVO = hostDao.findByIp(hostIp);
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+ final String clusterName = networkerBackup.getClientHostname();
+ final String destinationNetworkerClient = hostVO.getName().split("\\.")[0];
+ Long restoredVolumeDiskSize = 0L;
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return null;
+ }
+
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ // Find volume size from backup vols
+ for ( Backup.VolumeInfo VMVolToRestore : backupSourceVm.getBackupVolumeList()) {
+ if (VMVolToRestore.getUuid().equals(volumeUuid))
+ restoredVolumeDiskSize = (VMVolToRestore.getSize());
+ }
+
+ VolumeVO restoredVolume = new VolumeVO(Volume.Type.DATADISK, null, backup.getZoneId(),
+ backup.getDomainId(), backup.getAccountId(), 0, null,
+ backup.getSize(), null, null, null);
+
+ restoredVolume.setName("RV-"+volume.getName());
+ restoredVolume.setProvisioningType(volume.getProvisioningType());
+ restoredVolume.setUpdated(new Date());
+ restoredVolume.setUuid(UUID.randomUUID().toString());
+ restoredVolume.setRemoved(null);
+ restoredVolume.setDisplayVolume(true);
+ restoredVolume.setPoolId(volume.getPoolId());
+ restoredVolume.setPath(restoredVolume.getUuid());
+ restoredVolume.setState(Volume.State.Copying);
+ restoredVolume.setSize(restoredVolumeDiskSize);
+ restoredVolume.setDiskOfferingId(volume.getDiskOfferingId());
+
+ try {
+ volumeDao.persist(restoredVolume);
+ } catch (Exception e) {
+ throw new CloudRuntimeException("Unable to craft restored volume due to: "+e);
+ }
+
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -c " + clusterName +
+ " -d " + destinationNetworkerClient +
+ " -n " + restoredVolume.getUuid() +
+ " -p " + dataStore.getLocalPath() +
+ " -a " + volume.getUuid();
+
+ if (NetworkerClientVerboseLogs.value())
+ command = command + " -v ";
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for Volume UUID " + volume.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return new Pair<>(true,restoredVolume.getUuid());
+ } else {
+ volumeDao.expunge(restoredVolume.getId());
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return null;
+ }
+ }
+
+ @Override
+ public boolean takeBackup(VirtualMachine vm) {
+
+ String networkerServer;
+ String clusterName;
+
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ // Find where the VM is currently running
+ HostVO hostVO = getRunningVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ // Get retention Period for our Backup
+ BackupOfferingVO vmBackupOffering = new BackupOfferingDaoImpl().findById(vm.getBackupOfferingId());
+ final String backupProviderPolicyId = vmBackupOffering.getExternalId();
+ String backupRentionPeriod = getClient(vm.getDataCenterId()).getBackupPolicyRetentionInterval(backupProviderPolicyId);
+
+ if ( backupRentionPeriod == null ) {
+ LOG.warn("There is no retention setting for Emc Networker Policy, setting default for 1 day");
+ backupRentionPeriod="1 Day";
+ }
+
+ // Get Cluster
+ clusterName = getVMHypervisorCluster(hostVO);
+
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmbackup.sh" +
+ " -s " + networkerServer +
+ " -R '" + backupRentionPeriod + "'" +
+ " -P " + NetworkerMediaPool.valueIn(vm.getDataCenterId()) +
+ " -c " + clusterName +
+ " -u " + vm.getUuid() +
+ " -t " + vm.getName();
+ if (NetworkerClientVerboseLogs.value())
+ command = command + " -v ";
+
+ LOG.debug("Starting backup for VM ID " + vm.getUuid() + " on Networker provider");
+ Date backupJobStart = new Date();
+
+ String saveTime = executeBackupCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command);
+ BackupVO backup = getClient(vm.getDataCenterId()).registerBackupForVm(vm, backupJobStart, saveTime);
+ if (backup != null) {
+ backupDao.persist(backup);
+ return true;
+ } else {
+ // We need to handle this rare situation where backup is successful but can't be registered properly.
+ return false;
+ }
+ }
+
+ @Override
+ public boolean deleteBackup(Backup backup) {
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+
+ if (getClient(zoneId).deleteBackupForVM(externalBackupId)) {
+ LOG.debug("EMC Networker successfully deleted backup with id " + externalBackupId);
+ return true;
+ } else {
+ LOG.debug("There was an error removing the backup with id " + externalBackupId + " from EMC NEtworker");
+ }
+ return false;
+ }
+
+ @Override
+ public Map<VirtualMachine, Backup.Metric> getBackupMetrics(Long zoneId, List<VirtualMachine> vms) {
+ final Map<VirtualMachine, Backup.Metric> metrics = new HashMap<>();
+ Long vmBackupSize=0L;
+ Long vmBackupProtectedSize=0L;
+
+ if (CollectionUtils.isEmpty(vms)) {
+ LOG.warn("Unable to get VM Backup Metrics because the list of VMs is empty.");
+ return metrics;
+ }
+
+ for (final VirtualMachine vm : vms) {
+ for ( Backup.VolumeInfo thisVMVol : vm.getBackupVolumeList()) {
+ vmBackupSize += (thisVMVol.getSize() / 1024L / 1024L);
Review Comment:
should this be divided by 1024.0 then convert to Long?
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/NetworkerBackupProvider.java:
##########
@@ -0,0 +1,640 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.backup;
+
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.host.HostVO;
+import com.cloud.host.Status;
+import com.cloud.host.dao.HostDao;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.StoragePoolHostVO;
+import com.cloud.storage.Volume;
+import com.cloud.storage.VolumeVO;
+import com.cloud.storage.dao.StoragePoolHostDao;
+import com.cloud.storage.dao.VolumeDao;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.component.AdapterBase;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallbackNoReturn;
+import com.cloud.utils.db.TransactionStatus;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.ssh.SSHCmdHelper;
+import com.cloud.utils.ssh.SshException;
+import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.dao.VMInstanceDao;
+import org.apache.cloudstack.api.InternalIdentity;
+import org.apache.cloudstack.backup.dao.BackupDao;
+import org.apache.cloudstack.backup.dao.BackupOfferingDaoImpl;
+import org.apache.cloudstack.backup.networker.NetworkerClient;
+import org.apache.cloudstack.framework.config.ConfigKey;
+import org.apache.cloudstack.framework.config.Configurable;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.log4j.Logger;
+import org.apache.xml.utils.URI;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+
+
+import javax.inject.Inject;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.HashMap;
+import java.util.Date;
+import java.util.Objects;
+import java.util.UUID;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+
+public class NetworkerBackupProvider extends AdapterBase implements BackupProvider, Configurable {
+
+ public static final String BACKUP_IDENTIFIER = "-CSBKP-";
+ private static final Logger LOG = Logger.getLogger(NetworkerBackupProvider.class);
+ public ConfigKey<String> NetworkerUrl = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.url", "https://localhost:9090/nwrestapi/v3",
+ "The EMC Networker API URL.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerUsername = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.username", "administrator",
+ "The EMC Networker API username.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerPassword = new ConfigKey<>("Secure", String.class,
+ "backup.plugin.networker.password", "password",
+ "The EMC Networker API password.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerMediaPool = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.pool", "Default",
+ "The EMC Networker Media Pool", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerValidateSSLSecurity = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.validate.ssl", "false",
+ "Validate the SSL certificate when connecting to EMC Networker API service.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Integer> NetworkerApiRequestTimeout = new ConfigKey<>("Advanced", Integer.class,
+ "backup.plugin.networker.request.timeout", "300",
+ "The EMC Networker API request timeout in seconds.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerClientVerboseLogs = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.client.verbosity", "false",
+ "Produce Verbose logs in Hypervisor", true, ConfigKey.Scope.Zone);
+
+ @Inject
+ private BackupDao backupDao;
+
+ @Inject
+ private HostDao hostDao;
+
+ @Inject
+ private ConfigurationDao configDao;
+
+ @Inject
+ private ClusterDao clusterDao;
+
+ @Inject
+ private VolumeDao volumeDao;
+
+ @Inject
+ private StoragePoolHostDao storagePoolHostDao;
+
+ @Inject
+ private VMInstanceDao vmInstanceDao;
+
+ private static String getUrlDomain(String url) throws URISyntaxException {
+ URI uri;
+ try {
+ uri = new URI(url);
+ } catch (URI.MalformedURIException e) {
+ throw new CloudRuntimeException("Failed to cast URI");
+ }
+
+ return uri.getHost();
+ }
+
+ @Override
+ public ConfigKey<?>[] getConfigKeys() {
+ return new ConfigKey[]{
+ NetworkerUrl,
+ NetworkerUsername,
+ NetworkerPassword,
+ NetworkerValidateSSLSecurity,
+ NetworkerApiRequestTimeout,
+ NetworkerMediaPool,
+ NetworkerClientVerboseLogs
+ };
+ }
+
+ @Override
+ public String getName() {
+ return "networker";
+ }
+
+ @Override
+ public String getDescription() {
+ return "EMC Networker Backup Plugin";
+ }
+
+ @Override
+ public String getConfigComponentName() {
+ return BackupService.class.getSimpleName();
+ }
+
+ protected HostVO getLastVMHypervisorHost(VirtualMachine vm) {
+ HostVO host;
+ Long hostId = vm.getLastHostId();
+
+ if (hostId == null) {
+ LOG.debug("Cannot find last host for vm. This should never happen, please check your database.");
+ return null;
+ }
+ host = hostDao.findById(hostId);
+
+ if ( host.getStatus() == Status.Up ) {
+ return host;
+ } else {
+ // Try to find a host in the same cluster
+ List<HostVO> altClusterHosts = hostDao.findHypervisorHostInCluster(host.getClusterId());
+ for (final HostVO candidateClusterHost : altClusterHosts) {
+ if ( candidateClusterHost.getStatus() == Status.Up ) {
+ LOG.debug("Found Host " + candidateClusterHost.getName());
+ return candidateClusterHost;
+ }
+ }
+ }
+ // Try to find a Host in the zone
+ List<HostVO> altZoneHosts = hostDao.findByDataCenterId(host.getDataCenterId());
+ for (final HostVO candidateZoneHost : altZoneHosts) {
+ if ( candidateZoneHost.getStatus() == Status.Up && candidateZoneHost.getHypervisorType() == Hypervisor.HypervisorType.KVM ) {
+ LOG.debug("Found Host " + candidateZoneHost.getName());
+ return candidateZoneHost;
+ }
+ }
+ return null;
+ }
+
+ protected HostVO getRunningVMHypervisorHost(VirtualMachine vm) {
+
+ HostVO host;
+ Long hostId = vm.getHostId();
+
+ if (hostId == null) {
+ throw new CloudRuntimeException("Unable to find the HYPERVISOR for " + vm.getName() + ". Make sure the virtual machine is running");
+ }
+
+ host = hostDao.findById(hostId);
+
+ return host;
+ }
+
+ protected String getVMHypervisorCluster(HostVO host) {
+
+ return clusterDao.findById(host.getClusterId()).getName();
+ }
+
+ protected Ternary<String, String, String> getKVMHyperisorCredentials(HostVO host) {
+
+ String username = null;
+ String password = null;
+ String privateKey = null;
+
+ if (host != null && host.getHypervisorType() == Hypervisor.HypervisorType.KVM) {
+ hostDao.loadDetails(host);
+ password = host.getDetail("password");
+ username = host.getDetail("username");
+ privateKey = configDao.getValue("ssh.privatekey");
+ }
+ if ((password == null && privateKey == null) || username == null) {
+ throw new CloudRuntimeException("Cannot find login credentials for HYPERVISOR " + Objects.requireNonNull(host).getUuid());
+ }
+
+ return new Ternary<>(username, password, privateKey);
+ }
+
+ private String executeBackupCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+ String nstRegex = "\\bcompleted savetime=([0-9]{10})";
+ Pattern saveTimePattern = Pattern.compile(nstRegex);
+
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ Matcher saveTimeMatcher = saveTimePattern.matcher(result.getStdOut());
+ if (saveTimeMatcher.find()) {
+ return saveTimeMatcher.group(1);
+ }
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+
+ return null;
+ }
+ private boolean executeRestoreCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ return true;
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+ }
+
+ private NetworkerClient getClient(final Long zoneId) {
+ try {
+ return new NetworkerClient(NetworkerUrl.valueIn(zoneId), NetworkerUsername.valueIn(zoneId), NetworkerPassword.valueIn(zoneId),
+ NetworkerValidateSSLSecurity.valueIn(zoneId), NetworkerApiRequestTimeout.valueIn(zoneId));
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException("Failed to parse EMC Networker API URL: " + e.getMessage());
+ } catch (NoSuchAlgorithmException | KeyManagementException e) {
+ LOG.error("Failed to build EMC Networker API client due to: ", e);
+ }
+ throw new CloudRuntimeException("Failed to build EMC Networker API client");
+ }
+
+ @Override
+ public List<BackupOffering> listBackupOfferings(Long zoneId) {
+ List<BackupOffering> policies = new ArrayList<>();
+ for (final BackupOffering policy : getClient(zoneId).listPolicies()) {
+ if (!policy.getName().contains(BACKUP_IDENTIFIER)) {
+ policies.add(policy);
+ }
+ }
+
+ return policies;
+ }
+
+ @Override
+ public boolean isValidProviderOffering(Long zoneId, String uuid) {
+ List<BackupOffering> policies = listBackupOfferings(zoneId);
+ if (CollectionUtils.isEmpty(policies)) {
+ return false;
+ }
+ for (final BackupOffering policy : policies) {
+ if (Objects.equals(policy.getExternalId(), uuid)) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ @Override
+ public boolean assignVMToBackupOffering(VirtualMachine vm, BackupOffering backupOffering) { return true; }
+
+ @Override
+ public boolean removeVMFromBackupOffering(VirtualMachine vm) {
+ LOG.debug("Removing VirtualMachine from Backup offering and Deleting any existing backups");
+
+ List<String> backupsTaken = getClient(vm.getDataCenterId()).getBackupsForVm(vm);
+
+ for (String backupId : backupsTaken) {
+ LOG.debug("Trying to remove backup with id" + backupId);
+ getClient(vm.getDataCenterId()).deleteBackupForVM(backupId);
+ }
+
+ return true;
+ }
+
+ @Override
+ public boolean restoreVMFromBackup(VirtualMachine vm, Backup backup) {
+ LOG.debug("Restoring vm " + vm.getUuid() + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+ String networkerServer;
+ HostVO hostVO;
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return false;
+ }
+
+ // Find where the VM was last running
+ hostVO = getLastVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -S " + SSID;
+
+ if (NetworkerClientVerboseLogs.value())
+ command = command + " -v ";
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for VM ID " + vm.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return true;
+ } else {
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return false;
+ }
+ }
+
+ @Override
+ public Pair<Boolean, String> restoreBackedUpVolume(Backup backup, String volumeUuid, String hostIp, String dataStoreUuid) {
+ LOG.debug("Restoring volume " + volumeUuid + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+
+ String networkerServer;
+ VolumeVO volume = volumeDao.findByUuid(volumeUuid);
+ VMInstanceVO backupSourceVm = vmInstanceDao.findById(backup.getVmId());
+ StoragePoolHostVO dataStore = storagePoolHostDao.findByUuid(dataStoreUuid);
+ HostVO hostVO = hostDao.findByIp(hostIp);
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+ final String clusterName = networkerBackup.getClientHostname();
+ final String destinationNetworkerClient = hostVO.getName().split("\\.")[0];
+ Long restoredVolumeDiskSize = 0L;
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return null;
+ }
+
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ // Find volume size from backup vols
+ for ( Backup.VolumeInfo VMVolToRestore : backupSourceVm.getBackupVolumeList()) {
+ if (VMVolToRestore.getUuid().equals(volumeUuid))
+ restoredVolumeDiskSize = (VMVolToRestore.getSize());
+ }
+
+ VolumeVO restoredVolume = new VolumeVO(Volume.Type.DATADISK, null, backup.getZoneId(),
+ backup.getDomainId(), backup.getAccountId(), 0, null,
+ backup.getSize(), null, null, null);
+
+ restoredVolume.setName("RV-"+volume.getName());
+ restoredVolume.setProvisioningType(volume.getProvisioningType());
+ restoredVolume.setUpdated(new Date());
+ restoredVolume.setUuid(UUID.randomUUID().toString());
+ restoredVolume.setRemoved(null);
+ restoredVolume.setDisplayVolume(true);
+ restoredVolume.setPoolId(volume.getPoolId());
+ restoredVolume.setPath(restoredVolume.getUuid());
+ restoredVolume.setState(Volume.State.Copying);
+ restoredVolume.setSize(restoredVolumeDiskSize);
+ restoredVolume.setDiskOfferingId(volume.getDiskOfferingId());
+
+ try {
+ volumeDao.persist(restoredVolume);
+ } catch (Exception e) {
+ throw new CloudRuntimeException("Unable to craft restored volume due to: "+e);
+ }
+
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -c " + clusterName +
+ " -d " + destinationNetworkerClient +
+ " -n " + restoredVolume.getUuid() +
+ " -p " + dataStore.getLocalPath() +
+ " -a " + volume.getUuid();
+
+ if (NetworkerClientVerboseLogs.value())
+ command = command + " -v ";
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for Volume UUID " + volume.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return new Pair<>(true,restoredVolume.getUuid());
+ } else {
+ volumeDao.expunge(restoredVolume.getId());
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return null;
+ }
+ }
+
+ @Override
+ public boolean takeBackup(VirtualMachine vm) {
+
+ String networkerServer;
+ String clusterName;
+
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ // Find where the VM is currently running
+ HostVO hostVO = getRunningVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ // Get retention Period for our Backup
+ BackupOfferingVO vmBackupOffering = new BackupOfferingDaoImpl().findById(vm.getBackupOfferingId());
+ final String backupProviderPolicyId = vmBackupOffering.getExternalId();
+ String backupRentionPeriod = getClient(vm.getDataCenterId()).getBackupPolicyRetentionInterval(backupProviderPolicyId);
+
+ if ( backupRentionPeriod == null ) {
+ LOG.warn("There is no retention setting for Emc Networker Policy, setting default for 1 day");
+ backupRentionPeriod="1 Day";
+ }
+
+ // Get Cluster
+ clusterName = getVMHypervisorCluster(hostVO);
+
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmbackup.sh" +
+ " -s " + networkerServer +
+ " -R '" + backupRentionPeriod + "'" +
+ " -P " + NetworkerMediaPool.valueIn(vm.getDataCenterId()) +
+ " -c " + clusterName +
+ " -u " + vm.getUuid() +
+ " -t " + vm.getName();
+ if (NetworkerClientVerboseLogs.value())
+ command = command + " -v ";
+
+ LOG.debug("Starting backup for VM ID " + vm.getUuid() + " on Networker provider");
+ Date backupJobStart = new Date();
+
+ String saveTime = executeBackupCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command);
+ BackupVO backup = getClient(vm.getDataCenterId()).registerBackupForVm(vm, backupJobStart, saveTime);
+ if (backup != null) {
+ backupDao.persist(backup);
+ return true;
+ } else {
+ // We need to handle this rare situation where backup is successful but can't be registered properly.
+ return false;
+ }
+ }
+
+ @Override
+ public boolean deleteBackup(Backup backup) {
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+
+ if (getClient(zoneId).deleteBackupForVM(externalBackupId)) {
+ LOG.debug("EMC Networker successfully deleted backup with id " + externalBackupId);
+ return true;
+ } else {
+ LOG.debug("There was an error removing the backup with id " + externalBackupId + " from EMC NEtworker");
+ }
+ return false;
+ }
+
+ @Override
+ public Map<VirtualMachine, Backup.Metric> getBackupMetrics(Long zoneId, List<VirtualMachine> vms) {
+ final Map<VirtualMachine, Backup.Metric> metrics = new HashMap<>();
+ Long vmBackupSize=0L;
+ Long vmBackupProtectedSize=0L;
+
+ if (CollectionUtils.isEmpty(vms)) {
+ LOG.warn("Unable to get VM Backup Metrics because the list of VMs is empty.");
+ return metrics;
+ }
+
+ for (final VirtualMachine vm : vms) {
+ for ( Backup.VolumeInfo thisVMVol : vm.getBackupVolumeList()) {
+ vmBackupSize += (thisVMVol.getSize() / 1024L / 1024L);
+ }
+ final ArrayList<String> vmBackups = getClient(zoneId).getBackupsForVm(vm);
+ for ( String vmBackup : vmBackups ) {
+ NetworkerBackup vmNwBackup = getClient(zoneId).getNetworkerBackupInfo(vmBackup);
+ vmBackupProtectedSize+= vmNwBackup.getSize().getValue() / 1024L;
+ }
+ Backup.Metric vmBackupMetric = new Backup.Metric(vmBackupSize,vmBackupProtectedSize);
+ LOG.debug(String.format("Metrics for VM [uuid: %s, name: %s] is [backup size: %s, data size: %s].", vm.getUuid(),
+ vm.getInstanceName(), vmBackupMetric.getBackupSize(), vmBackupMetric.getDataSize()));
+ metrics.put(vm, vmBackupMetric);
+ }
+ return metrics;
+ }
Review Comment:
nit - add new line
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r951215008
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/NetworkerBackupProvider.java:
##########
@@ -0,0 +1,640 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.backup;
+
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.host.HostVO;
+import com.cloud.host.Status;
+import com.cloud.host.dao.HostDao;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.StoragePoolHostVO;
+import com.cloud.storage.Volume;
+import com.cloud.storage.VolumeVO;
+import com.cloud.storage.dao.StoragePoolHostDao;
+import com.cloud.storage.dao.VolumeDao;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.component.AdapterBase;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallbackNoReturn;
+import com.cloud.utils.db.TransactionStatus;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.ssh.SSHCmdHelper;
+import com.cloud.utils.ssh.SshException;
+import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.dao.VMInstanceDao;
+import org.apache.cloudstack.api.InternalIdentity;
+import org.apache.cloudstack.backup.dao.BackupDao;
+import org.apache.cloudstack.backup.dao.BackupOfferingDaoImpl;
+import org.apache.cloudstack.backup.networker.NetworkerClient;
+import org.apache.cloudstack.framework.config.ConfigKey;
+import org.apache.cloudstack.framework.config.Configurable;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.log4j.Logger;
+import org.apache.xml.utils.URI;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+
+
+import javax.inject.Inject;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.HashMap;
+import java.util.Date;
+import java.util.Objects;
+import java.util.UUID;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+
+public class NetworkerBackupProvider extends AdapterBase implements BackupProvider, Configurable {
+
+ public static final String BACKUP_IDENTIFIER = "-CSBKP-";
+ private static final Logger LOG = Logger.getLogger(NetworkerBackupProvider.class);
+ public ConfigKey<String> NetworkerUrl = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.url", "https://localhost:9090/nwrestapi/v3",
+ "The EMC Networker API URL.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerUsername = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.username", "administrator",
+ "The EMC Networker API username.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerPassword = new ConfigKey<>("Secure", String.class,
+ "backup.plugin.networker.password", "password",
+ "The EMC Networker API password.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerMediaPool = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.pool", "Default",
+ "The EMC Networker Media Pool", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerValidateSSLSecurity = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.validate.ssl", "false",
+ "Validate the SSL certificate when connecting to EMC Networker API service.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Integer> NetworkerApiRequestTimeout = new ConfigKey<>("Advanced", Integer.class,
+ "backup.plugin.networker.request.timeout", "300",
+ "The EMC Networker API request timeout in seconds.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerClientVerboseLogs = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.client.verbosity", "false",
+ "Produce Verbose logs in Hypervisor", true, ConfigKey.Scope.Zone);
+
+ @Inject
+ private BackupDao backupDao;
+
+ @Inject
+ private HostDao hostDao;
+
+ @Inject
+ private ConfigurationDao configDao;
+
+ @Inject
+ private ClusterDao clusterDao;
+
+ @Inject
+ private VolumeDao volumeDao;
+
+ @Inject
+ private StoragePoolHostDao storagePoolHostDao;
+
+ @Inject
+ private VMInstanceDao vmInstanceDao;
+
+ private static String getUrlDomain(String url) throws URISyntaxException {
+ URI uri;
+ try {
+ uri = new URI(url);
+ } catch (URI.MalformedURIException e) {
+ throw new CloudRuntimeException("Failed to cast URI");
+ }
+
+ return uri.getHost();
+ }
+
+ @Override
+ public ConfigKey<?>[] getConfigKeys() {
+ return new ConfigKey[]{
+ NetworkerUrl,
+ NetworkerUsername,
+ NetworkerPassword,
+ NetworkerValidateSSLSecurity,
+ NetworkerApiRequestTimeout,
+ NetworkerMediaPool,
+ NetworkerClientVerboseLogs
+ };
+ }
+
+ @Override
+ public String getName() {
+ return "networker";
+ }
+
+ @Override
+ public String getDescription() {
+ return "EMC Networker Backup Plugin";
+ }
+
+ @Override
+ public String getConfigComponentName() {
+ return BackupService.class.getSimpleName();
+ }
+
+ protected HostVO getLastVMHypervisorHost(VirtualMachine vm) {
+ HostVO host;
+ Long hostId = vm.getLastHostId();
+
+ if (hostId == null) {
+ LOG.debug("Cannot find last host for vm. This should never happen, please check your database.");
+ return null;
+ }
+ host = hostDao.findById(hostId);
+
+ if ( host.getStatus() == Status.Up ) {
+ return host;
+ } else {
+ // Try to find a host in the same cluster
+ List<HostVO> altClusterHosts = hostDao.findHypervisorHostInCluster(host.getClusterId());
+ for (final HostVO candidateClusterHost : altClusterHosts) {
+ if ( candidateClusterHost.getStatus() == Status.Up ) {
+ LOG.debug("Found Host " + candidateClusterHost.getName());
+ return candidateClusterHost;
+ }
+ }
+ }
+ // Try to find a Host in the zone
+ List<HostVO> altZoneHosts = hostDao.findByDataCenterId(host.getDataCenterId());
+ for (final HostVO candidateZoneHost : altZoneHosts) {
+ if ( candidateZoneHost.getStatus() == Status.Up && candidateZoneHost.getHypervisorType() == Hypervisor.HypervisorType.KVM ) {
+ LOG.debug("Found Host " + candidateZoneHost.getName());
+ return candidateZoneHost;
+ }
+ }
+ return null;
+ }
+
+ protected HostVO getRunningVMHypervisorHost(VirtualMachine vm) {
+
+ HostVO host;
+ Long hostId = vm.getHostId();
+
+ if (hostId == null) {
+ throw new CloudRuntimeException("Unable to find the HYPERVISOR for " + vm.getName() + ". Make sure the virtual machine is running");
+ }
+
+ host = hostDao.findById(hostId);
+
+ return host;
+ }
+
+ protected String getVMHypervisorCluster(HostVO host) {
+
+ return clusterDao.findById(host.getClusterId()).getName();
+ }
+
+ protected Ternary<String, String, String> getKVMHyperisorCredentials(HostVO host) {
+
+ String username = null;
+ String password = null;
+ String privateKey = null;
+
+ if (host != null && host.getHypervisorType() == Hypervisor.HypervisorType.KVM) {
+ hostDao.loadDetails(host);
+ password = host.getDetail("password");
+ username = host.getDetail("username");
+ privateKey = configDao.getValue("ssh.privatekey");
+ }
+ if ((password == null && privateKey == null) || username == null) {
+ throw new CloudRuntimeException("Cannot find login credentials for HYPERVISOR " + Objects.requireNonNull(host).getUuid());
+ }
+
+ return new Ternary<>(username, password, privateKey);
+ }
+
+ private String executeBackupCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+ String nstRegex = "\\bcompleted savetime=([0-9]{10})";
+ Pattern saveTimePattern = Pattern.compile(nstRegex);
+
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ Matcher saveTimeMatcher = saveTimePattern.matcher(result.getStdOut());
+ if (saveTimeMatcher.find()) {
+ return saveTimeMatcher.group(1);
+ }
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+
+ return null;
+ }
+ private boolean executeRestoreCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ return true;
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+ }
+
+ private NetworkerClient getClient(final Long zoneId) {
+ try {
+ return new NetworkerClient(NetworkerUrl.valueIn(zoneId), NetworkerUsername.valueIn(zoneId), NetworkerPassword.valueIn(zoneId),
+ NetworkerValidateSSLSecurity.valueIn(zoneId), NetworkerApiRequestTimeout.valueIn(zoneId));
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException("Failed to parse EMC Networker API URL: " + e.getMessage());
+ } catch (NoSuchAlgorithmException | KeyManagementException e) {
+ LOG.error("Failed to build EMC Networker API client due to: ", e);
+ }
+ throw new CloudRuntimeException("Failed to build EMC Networker API client");
+ }
+
+ @Override
+ public List<BackupOffering> listBackupOfferings(Long zoneId) {
+ List<BackupOffering> policies = new ArrayList<>();
+ for (final BackupOffering policy : getClient(zoneId).listPolicies()) {
+ if (!policy.getName().contains(BACKUP_IDENTIFIER)) {
+ policies.add(policy);
+ }
+ }
+
+ return policies;
+ }
+
+ @Override
+ public boolean isValidProviderOffering(Long zoneId, String uuid) {
+ List<BackupOffering> policies = listBackupOfferings(zoneId);
+ if (CollectionUtils.isEmpty(policies)) {
+ return false;
+ }
+ for (final BackupOffering policy : policies) {
+ if (Objects.equals(policy.getExternalId(), uuid)) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ @Override
+ public boolean assignVMToBackupOffering(VirtualMachine vm, BackupOffering backupOffering) { return true; }
+
+ @Override
+ public boolean removeVMFromBackupOffering(VirtualMachine vm) {
+ LOG.debug("Removing VirtualMachine from Backup offering and Deleting any existing backups");
+
+ List<String> backupsTaken = getClient(vm.getDataCenterId()).getBackupsForVm(vm);
+
+ for (String backupId : backupsTaken) {
+ LOG.debug("Trying to remove backup with id" + backupId);
+ getClient(vm.getDataCenterId()).deleteBackupForVM(backupId);
+ }
+
+ return true;
+ }
+
+ @Override
+ public boolean restoreVMFromBackup(VirtualMachine vm, Backup backup) {
+ LOG.debug("Restoring vm " + vm.getUuid() + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+ String networkerServer;
+ HostVO hostVO;
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return false;
+ }
+
+ // Find where the VM was last running
+ hostVO = getLastVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -S " + SSID;
+
+ if (NetworkerClientVerboseLogs.value())
+ command = command + " -v ";
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for VM ID " + vm.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return true;
+ } else {
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return false;
+ }
+ }
+
+ @Override
+ public Pair<Boolean, String> restoreBackedUpVolume(Backup backup, String volumeUuid, String hostIp, String dataStoreUuid) {
+ LOG.debug("Restoring volume " + volumeUuid + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+
+ String networkerServer;
+ VolumeVO volume = volumeDao.findByUuid(volumeUuid);
+ VMInstanceVO backupSourceVm = vmInstanceDao.findById(backup.getVmId());
+ StoragePoolHostVO dataStore = storagePoolHostDao.findByUuid(dataStoreUuid);
+ HostVO hostVO = hostDao.findByIp(hostIp);
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+ final String clusterName = networkerBackup.getClientHostname();
+ final String destinationNetworkerClient = hostVO.getName().split("\\.")[0];
+ Long restoredVolumeDiskSize = 0L;
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return null;
+ }
+
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ // Find volume size from backup vols
+ for ( Backup.VolumeInfo VMVolToRestore : backupSourceVm.getBackupVolumeList()) {
+ if (VMVolToRestore.getUuid().equals(volumeUuid))
+ restoredVolumeDiskSize = (VMVolToRestore.getSize());
+ }
+
+ VolumeVO restoredVolume = new VolumeVO(Volume.Type.DATADISK, null, backup.getZoneId(),
+ backup.getDomainId(), backup.getAccountId(), 0, null,
+ backup.getSize(), null, null, null);
+
+ restoredVolume.setName("RV-"+volume.getName());
+ restoredVolume.setProvisioningType(volume.getProvisioningType());
+ restoredVolume.setUpdated(new Date());
+ restoredVolume.setUuid(UUID.randomUUID().toString());
+ restoredVolume.setRemoved(null);
+ restoredVolume.setDisplayVolume(true);
+ restoredVolume.setPoolId(volume.getPoolId());
+ restoredVolume.setPath(restoredVolume.getUuid());
+ restoredVolume.setState(Volume.State.Copying);
+ restoredVolume.setSize(restoredVolumeDiskSize);
+ restoredVolume.setDiskOfferingId(volume.getDiskOfferingId());
+
+ try {
+ volumeDao.persist(restoredVolume);
+ } catch (Exception e) {
+ throw new CloudRuntimeException("Unable to craft restored volume due to: "+e);
+ }
+
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -c " + clusterName +
+ " -d " + destinationNetworkerClient +
+ " -n " + restoredVolume.getUuid() +
+ " -p " + dataStore.getLocalPath() +
+ " -a " + volume.getUuid();
+
+ if (NetworkerClientVerboseLogs.value())
+ command = command + " -v ";
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for Volume UUID " + volume.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return new Pair<>(true,restoredVolume.getUuid());
+ } else {
+ volumeDao.expunge(restoredVolume.getId());
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return null;
+ }
+ }
+
+ @Override
+ public boolean takeBackup(VirtualMachine vm) {
+
+ String networkerServer;
+ String clusterName;
+
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ // Find where the VM is currently running
+ HostVO hostVO = getRunningVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ // Get retention Period for our Backup
+ BackupOfferingVO vmBackupOffering = new BackupOfferingDaoImpl().findById(vm.getBackupOfferingId());
+ final String backupProviderPolicyId = vmBackupOffering.getExternalId();
+ String backupRentionPeriod = getClient(vm.getDataCenterId()).getBackupPolicyRetentionInterval(backupProviderPolicyId);
+
+ if ( backupRentionPeriod == null ) {
+ LOG.warn("There is no retention setting for Emc Networker Policy, setting default for 1 day");
+ backupRentionPeriod="1 Day";
+ }
+
+ // Get Cluster
+ clusterName = getVMHypervisorCluster(hostVO);
+
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmbackup.sh" +
+ " -s " + networkerServer +
Review Comment:
Same as before, have a look at `Script` usage https://github.com/apache/cloudstack/blob/main/utils/src/main/java/com/cloud/utils/script/Script.java#L48
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r951222998
##########
scripts/vm/hypervisor/kvm/nsrkvmbackup.sh:
##########
@@ -0,0 +1,270 @@
+#!/bin/bash
+
+## Licensed to the Apache Software Foundation (ASF) under one
+## or more contributor license agreements. See the NOTICE file
+## distributed with this work for additional information
+## regarding copyright ownership. The ASF licenses this file
+## to you under the Apache License, Version 2.0 (the
+## "License"); you may not use this file except in compliance
+## with the License. You may obtain a copy of the License at
+##
+## http://www.apache.org/licenses/LICENSE-2.0
+##
+## Unless required by applicable law or agreed to in writing,
+## software distributed under the License is distributed on an
+## "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+## KIND, either express or implied. See the License for the
+## specific language governing permissions and limitations
+## under the License.
+
+
+version=0.9.1
+
+
+OPTIND=1
+
+verb=0
+logDir="/nsr/logs/cloudstack/"
+snapPrefix="CSBKP_$RANDOM_"
+clusterClient=""
+networkerServer=""
+hvVersion=""
+libvVersion=""
+apiVersion=""
+kvmDName=""
+kvmDUuid=""
+logFile=""
+mediaPool=""
+retentionTime=""
+
+
+log () {
+
+ [[ "$verb" -eq 1 ]] && builtin echo "$@"
+ if [[ "$1" == "-ne" || "$1" == "-e" || "$1" == "-n" ]]; then
+ builtin echo -e "$(date '+%Y-%m-%d %H-%M-%S>')" "${@: 2}" >> "$logFile"
+ else
+ builtin echo "$(date '+%Y-%m-%d %H-%M-%S>')" "$@" >> "$logFile"
+ fi
+
+}
+
+vercomp(){
+ local a b IFS=. -; set -f
+ printf -v a %08d $1; printf -v b %08d $3
+ test $a "$2" $b
+}
+usage() {
+ echo "
+
+Usage:[-v] [-h] [-l log_dir] [-dr] [-s networker_server] [-c networker_cluster_client] [-t target_vm] [-u target_uuid] [-p snapprefix] [-P media_pool ] [-R retention_time ]
+
+Options:
+ -h Help and usage
+ -v Enable verbose mode
+ -l log_dir. Specify log directory. Default is /nsr/logs/cloudstack
+ -s networker_server Specifiy the EMC Networker server we are going to use
+ -c networker_cluster_client Specify the EMC Networker client CLUSTER to use
+ -t target_vm KVM domain to backup
+ -u target_uuid KVM domain to backup
+ -p Snapshot Prefix for backups
+ -P mediaPool EMC Networker Media Pool
+ -R retention_time Backup retention time
+
+Supplements Apache Cloudstack B&R Framework EMC Networker plugin and performs the backup of the Virtual Machines
+"
+
+}
+
+sanity_checks() {
+
+ log "Performing environment sanity checks..."
+ log -ne "\t[1] Checking if Networker is installed\t"
+ if [[ $(systemctl list-unit-files | grep networker) = *networker* ]]; then
+ log "Success"
+ else
+ log "Failure"
+ log -e "\n\tNetworker Service NOT FOUND. Make sure that Networker client is properly installed"
+ exit 1
+ fi
+
+ log -ne "\t[2] Checking if Networker is running\t"
+ if [[ $(systemctl is-active networker) = *active* ]]; then
+ log "Success"
+ else
+ log "Failure"
+ log -e "\n\tNetworker Service is not running. Investigate Networker logs, startup server and try again"
+ exit 2
+ fi
+ log -ne "\t[3] Checking Networker DNS Resolution\t"
+ if [[ $(getent hosts "$networkerServer") = *$networkerServer* ]]; then
+ log "Success"
+ else
+ log "Failure"
+ log -e "\n\tNetworker Server cannot be resolved. Backups will most probably fail. Consider adding the ip/hostname to /etc/host or fix DNS resolution"
+ exit 3
+ fi
+
+ log -ne "\t[4] Checking QEMU / Libvirt Versions \t"
+ hvVersion=$(virsh version | grep hypervisor | awk '{print $(NF)}')
Review Comment:
I would avoid mixing information one can execute via java/libvirt-library; is this run on KVM agents on or mgmt server?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] github-actions[bot] commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1185563915
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1188435911
@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1185728179
UI build: :heavy_check_mark:
Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6550 (SL-JID-1966)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1185223448
@rohityadavcloud , what is the status of the jacoco validator? I've seen unexplained failures in other PRs too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1207847745
@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1285468857
@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1363984091
> @fermosan sorry to be bugging you. I don´t think the package-lock is intended to be deleted, just to remain unchanged.
@DaanHoogland No prob. I pulled from origin/main and added
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1363437445
<b>Trillian test result (tid-5613)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 43891 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6550-t5613-kvm-centos7.zip
Smoke tests completed. 105 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_08_upgrade_kubernetes_ha_cluster | `Failure` | 590.78 | test_kubernetes_clusters.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1358630991
<b>Trillian test result (tid-5566)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 43733 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6550-t5566-kvm-centos7.zip
Smoke tests completed. 104 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_04_autoscale_kubernetes_cluster | `Failure` | 32.90 | test_kubernetes_clusters.py
test_08_upgrade_kubernetes_ha_cluster | `Failure` | 569.24 | test_kubernetes_clusters.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1362688071
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 5055
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan closed pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan closed pull request #6550: Emc networker b&r
URL: https://github.com/apache/cloudstack/pull/6550
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r980158793
##########
scripts/vm/hypervisor/kvm/nsrkvmrestore.sh:
##########
@@ -0,0 +1,234 @@
+#!/bin/bash
+
+## Licensed to the Apache Software Foundation (ASF) under one
+## or more contributor license agreements. See the NOTICE file
+## distributed with this work for additional information
+## regarding copyright ownership. The ASF licenses this file
+## to you under the Apache License, Version 2.0 (the
+## "License"); you may not use this file except in compliance
+## with the License. You may obtain a copy of the License at
+##
+## http://www.apache.org/licenses/LICENSE-2.0
+##
+## Unless required by applicable law or agreed to in writing,
+## software distributed under the License is distributed on an
+## "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+## KIND, either express or implied. See the License for the
+## specific language governing permissions and limitations
+## under the License.
+
+
+version=0.9.1
+
+
+OPTIND=1
+
+verb=0
+logDir="/nsr/logs/cloudstack/"
+clusterClient=""
+networkerServer=""
+hvVersion=""
+libvVersion=""
+apiVersion=""
+logFile=""
+destination=""
+volume=""
+ssid=""
+newvolumeUuid=""
+POOLPATH=""
+
+log () {
+ [[ "$verb" -eq 1 ]] && builtin echo "$@"
+ if [[ "$1" == "-ne" || "$1" == "-e" || "$1" == "-n" ]]; then
+ builtin echo -e "$(date '+%Y-%m-%d %H-%M-%S>')" "${@: 2}" >> "$logFile"
+ else
+ builtin echo "$(date '+%Y-%m-%d %H-%M-%S>')" "$@" >> "$logFile"
+ fi
+}
+
+vercomp(){
+ local a b IFS=. -; set -f
+ printf -v a %08d $1; printf -v b %08d $3
+ test $a "$2" $b
+}
+
+
+usage() {
+ echo "
+
+Usage:[-v] [-h] [-l log_dir] [-s networker_server] [-c networker_cluster_client] [-S ssid] [-d networker_destination_client ] [ -a volume ] [ -p pool_local_path ]
+
+Options:
+ -h Help and usage
+ -v Enable log mode
+ -l log_dir. Specify log directory. Default is /nsr/logs/cloudstack
+ -s networker_server Specify the EMC Networker server we are going to use
+ -c networker_cluster_client Specify the EMC Networker client CLUSTER to use
+ -d networker_destination_client Specify the EMC Networker client that is going to be used for the restore
+ -a volume Specify volume to restore
+ -p pool_local_path Local Path to Pool
+ -S ssid Specify SSID
+Supplements Apache Cloudstack B&R Framework EMC Networker plugin and performs the backup of the Virtual Machines
+"
+
+}
+
+sanity_checks() {
+ log "Performing environment sanity checks..."
+
+ log -ne "\t[1] Checking if Networker is installed\t"
+ if [[ $(systemctl list-unit-files | grep networker) = *networker* ]]; then
+ log "Success"
+ else
+ log "Failure"
+ log -e "\n\tNetworker Service NOT FOUND. Make sure that Networker client is properly installed"
+ exit 1
+ fi
+
+ log -ne "\t[2] Checking if Networker is running\t"
+ if [[ $(systemctl is-active networker) = *active* ]]; then
+ log "Success"
+ else
+ log "Failure"
+ log -e "\n\tNetworker Service is not running. Investigate Networker logs, startup server and try again"
+ exit 2
+ fi
+ log -ne "\t[3] Checking Networker DNS Resolution\t"
+ if [[ $(getent hosts "$networkerServer") = *$networkerServer* ]]; then
+ log "Success"
+ else
+ log "Failure"
+ log -e "\n\tNetworker Server cannot be resolved. Backups will most probably fail. Consider adding the ip/hostname to /etc/host or fix DNS resolution"
+ exit 3
+ fi
+
+ log -ne "\t[4] Checking QEMU / Libvirt Versions \t"
+ hvVersion=$(virsh version | grep hypervisor | awk '{print $(NF)}')
+ libvVersion=$(virsh version | grep libvirt | awk '{print $(NF)}' | tail -n 1)
+ apiVersion=$(virsh version | grep API | awk '{print $(NF)}')
Review Comment:
This is running on kvm host
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r979472121
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/NetworkerClient.java:
##########
@@ -0,0 +1,360 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r991589195
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/NetworkerBackupProvider.java:
##########
@@ -0,0 +1,637 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.backup;
+
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.host.HostVO;
+import com.cloud.host.Status;
+import com.cloud.host.dao.HostDao;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.StoragePoolHostVO;
+import com.cloud.storage.Volume;
+import com.cloud.storage.VolumeVO;
+import com.cloud.storage.dao.StoragePoolHostDao;
+import com.cloud.storage.dao.VolumeDao;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.component.AdapterBase;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallbackNoReturn;
+import com.cloud.utils.db.TransactionStatus;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.ssh.SSHCmdHelper;
+import com.cloud.utils.ssh.SshException;
+import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.dao.VMInstanceDao;
+import org.apache.cloudstack.api.InternalIdentity;
+import org.apache.cloudstack.backup.dao.BackupDao;
+import org.apache.cloudstack.backup.dao.BackupOfferingDaoImpl;
+import org.apache.cloudstack.backup.networker.NetworkerClient;
+import org.apache.cloudstack.framework.config.ConfigKey;
+import org.apache.cloudstack.framework.config.Configurable;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.log4j.Logger;
+import org.apache.xml.utils.URI;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+import javax.inject.Inject;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.HashMap;
+import java.util.Date;
+import java.util.Objects;
+import java.util.UUID;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+public class NetworkerBackupProvider extends AdapterBase implements BackupProvider, Configurable {
+
+ public static final String BACKUP_IDENTIFIER = "-CSBKP-";
+ private static final Logger LOG = Logger.getLogger(NetworkerBackupProvider.class);
+ public ConfigKey<String> NetworkerUrl = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.url", "https://localhost:9090/nwrestapi/v3",
+ "The EMC Networker API URL.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerUsername = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.username", "administrator",
+ "The EMC Networker API username.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerPassword = new ConfigKey<>("Secure", String.class,
+ "backup.plugin.networker.password", "password",
+ "The EMC Networker API password.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerMediaPool = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.pool", "Default",
+ "The EMC Networker Media Pool", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerValidateSSLSecurity = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.validate.ssl", "false",
+ "Validate the SSL certificate when connecting to EMC Networker API service.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Integer> NetworkerApiRequestTimeout = new ConfigKey<>("Advanced", Integer.class,
+ "backup.plugin.networker.request.timeout", "300",
+ "The EMC Networker API request timeout in seconds.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerClientVerboseLogs = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.client.verbosity", "false",
+ "Produce Verbose logs in Hypervisor", true, ConfigKey.Scope.Zone);
+
+ @Inject
+ private BackupDao backupDao;
+
+ @Inject
+ private HostDao hostDao;
+
+ @Inject
+ private ConfigurationDao configDao;
+
+ @Inject
+ private ClusterDao clusterDao;
+
+ @Inject
+ private VolumeDao volumeDao;
+
+ @Inject
+ private StoragePoolHostDao storagePoolHostDao;
+
+ @Inject
+ private VMInstanceDao vmInstanceDao;
+
+ private static String getUrlDomain(String url) throws URISyntaxException {
+ URI uri;
+ try {
+ uri = new URI(url);
+ } catch (URI.MalformedURIException e) {
+ throw new CloudRuntimeException("Failed to cast URI");
+ }
+
+ return uri.getHost();
+ }
+
+ @Override
+ public ConfigKey<?>[] getConfigKeys() {
+ return new ConfigKey[]{
+ NetworkerUrl,
+ NetworkerUsername,
+ NetworkerPassword,
+ NetworkerValidateSSLSecurity,
+ NetworkerApiRequestTimeout,
+ NetworkerMediaPool,
+ NetworkerClientVerboseLogs
+ };
+ }
+
+ @Override
+ public String getName() {
+ return "networker";
+ }
+
+ @Override
+ public String getDescription() {
+ return "EMC Networker Backup Plugin";
+ }
+
+ @Override
+ public String getConfigComponentName() {
+ return BackupService.class.getSimpleName();
+ }
+
+ protected HostVO getLastVMHypervisorHost(VirtualMachine vm) {
+ HostVO host;
+ Long hostId = vm.getLastHostId();
+
+ if (hostId == null) {
+ LOG.debug("Cannot find last host for vm. This should never happen, please check your database.");
+ return null;
+ }
+ host = hostDao.findById(hostId);
+
+ if ( host.getStatus() == Status.Up ) {
+ return host;
+ } else {
+ // Try to find a host in the same cluster
+ List<HostVO> altClusterHosts = hostDao.findHypervisorHostInCluster(host.getClusterId());
+ for (final HostVO candidateClusterHost : altClusterHosts) {
+ if ( candidateClusterHost.getStatus() == Status.Up ) {
+ LOG.debug("Found Host " + candidateClusterHost.getName());
+ return candidateClusterHost;
+ }
+ }
+ }
+ // Try to find a Host in the zone
+ List<HostVO> altZoneHosts = hostDao.findByDataCenterId(host.getDataCenterId());
+ for (final HostVO candidateZoneHost : altZoneHosts) {
+ if ( candidateZoneHost.getStatus() == Status.Up && candidateZoneHost.getHypervisorType() == Hypervisor.HypervisorType.KVM ) {
+ LOG.debug("Found Host " + candidateZoneHost.getName());
+ return candidateZoneHost;
+ }
+ }
+ return null;
+ }
+
+ protected HostVO getRunningVMHypervisorHost(VirtualMachine vm) {
+
+ HostVO host;
+ Long hostId = vm.getHostId();
+
+ if (hostId == null) {
+ throw new CloudRuntimeException("Unable to find the HYPERVISOR for " + vm.getName() + ". Make sure the virtual machine is running");
+ }
+
+ host = hostDao.findById(hostId);
+
+ return host;
+ }
+
+ protected String getVMHypervisorCluster(HostVO host) {
+
+ return clusterDao.findById(host.getClusterId()).getName();
+ }
+
+ protected Ternary<String, String, String> getKVMHyperisorCredentials(HostVO host) {
+
+ String username = null;
+ String password = null;
+ String privateKey = null;
+
+ if (host != null && host.getHypervisorType() == Hypervisor.HypervisorType.KVM) {
+ hostDao.loadDetails(host);
+ password = host.getDetail("password");
+ username = host.getDetail("username");
+ privateKey = configDao.getValue("ssh.privatekey");
+ }
+ if ((password == null && privateKey == null) || username == null) {
+ throw new CloudRuntimeException("Cannot find login credentials for HYPERVISOR " + Objects.requireNonNull(host).getUuid());
+ }
+
+ return new Ternary<>(username, password, privateKey);
+ }
+
+ private String executeBackupCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+ String nstRegex = "\\bcompleted savetime=([0-9]{10})";
+ Pattern saveTimePattern = Pattern.compile(nstRegex);
+
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ Matcher saveTimeMatcher = saveTimePattern.matcher(result.getStdOut());
+ if (saveTimeMatcher.find()) {
+ return saveTimeMatcher.group(1);
+ }
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+
+ return null;
+ }
+ private boolean executeRestoreCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ return true;
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+ }
+
+ private NetworkerClient getClient(final Long zoneId) {
+ try {
+ return new NetworkerClient(NetworkerUrl.valueIn(zoneId), NetworkerUsername.valueIn(zoneId), NetworkerPassword.valueIn(zoneId),
+ NetworkerValidateSSLSecurity.valueIn(zoneId), NetworkerApiRequestTimeout.valueIn(zoneId));
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException("Failed to parse EMC Networker API URL: " + e.getMessage());
+ } catch (NoSuchAlgorithmException | KeyManagementException e) {
+ LOG.error("Failed to build EMC Networker API client due to: ", e);
+ }
+ throw new CloudRuntimeException("Failed to build EMC Networker API client");
+ }
+
+ @Override
+ public List<BackupOffering> listBackupOfferings(Long zoneId) {
+ List<BackupOffering> policies = new ArrayList<>();
+ for (final BackupOffering policy : getClient(zoneId).listPolicies()) {
+ if (!policy.getName().contains(BACKUP_IDENTIFIER)) {
+ policies.add(policy);
+ }
+ }
+
+ return policies;
+ }
+
+ @Override
+ public boolean isValidProviderOffering(Long zoneId, String uuid) {
+ List<BackupOffering> policies = listBackupOfferings(zoneId);
+ if (CollectionUtils.isEmpty(policies)) {
+ return false;
+ }
+ for (final BackupOffering policy : policies) {
+ if (Objects.equals(policy.getExternalId(), uuid)) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ @Override
+ public boolean assignVMToBackupOffering(VirtualMachine vm, BackupOffering backupOffering) { return true; }
+
+ @Override
+ public boolean removeVMFromBackupOffering(VirtualMachine vm) {
+ LOG.debug("Removing VirtualMachine from Backup offering and Deleting any existing backups");
+
+ List<String> backupsTaken = getClient(vm.getDataCenterId()).getBackupsForVm(vm);
+
+ for (String backupId : backupsTaken) {
+ LOG.debug("Trying to remove backup with id" + backupId);
+ getClient(vm.getDataCenterId()).deleteBackupForVM(backupId);
+ }
+
+ return true;
+ }
+
+ @Override
+ public boolean restoreVMFromBackup(VirtualMachine vm, Backup backup) {
+ LOG.debug("Restoring vm " + vm.getUuid() + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+ String networkerServer;
+ HostVO hostVO;
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return false;
+ }
+
+ // Find where the VM was last running
+ hostVO = getLastVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -S " + SSID;
+
+ if ( Boolean.TRUE.equals(NetworkerClientVerboseLogs.value()) )
+ command = command + " -v ";
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for VM ID " + vm.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return true;
+ } else {
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return false;
+ }
+ }
+
+ @Override
+ public Pair<Boolean, String> restoreBackedUpVolume(Backup backup, String volumeUuid, String hostIp, String dataStoreUuid) {
+ LOG.debug("Restoring volume " + volumeUuid + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+
+ String networkerServer;
+ VolumeVO volume = volumeDao.findByUuid(volumeUuid);
+ VMInstanceVO backupSourceVm = vmInstanceDao.findById(backup.getVmId());
+ StoragePoolHostVO dataStore = storagePoolHostDao.findByUuid(dataStoreUuid);
+ HostVO hostVO = hostDao.findByIp(hostIp);
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+ final String clusterName = networkerBackup.getClientHostname();
+ final String destinationNetworkerClient = hostVO.getName().split("\\.")[0];
+ Long restoredVolumeDiskSize = 0L;
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return null;
+ }
+
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ // Find volume size from backup vols
+ for ( Backup.VolumeInfo VMVolToRestore : backupSourceVm.getBackupVolumeList()) {
+ if (VMVolToRestore.getUuid().equals(volumeUuid))
+ restoredVolumeDiskSize = (VMVolToRestore.getSize());
+ }
+
+ VolumeVO restoredVolume = new VolumeVO(Volume.Type.DATADISK, null, backup.getZoneId(),
+ backup.getDomainId(), backup.getAccountId(), 0, null,
+ backup.getSize(), null, null, null);
+
+ restoredVolume.setName("RV-"+volume.getName());
+ restoredVolume.setProvisioningType(volume.getProvisioningType());
+ restoredVolume.setUpdated(new Date());
+ restoredVolume.setUuid(UUID.randomUUID().toString());
+ restoredVolume.setRemoved(null);
+ restoredVolume.setDisplayVolume(true);
+ restoredVolume.setPoolId(volume.getPoolId());
+ restoredVolume.setPath(restoredVolume.getUuid());
+ restoredVolume.setState(Volume.State.Copying);
+ restoredVolume.setSize(restoredVolumeDiskSize);
+ restoredVolume.setDiskOfferingId(volume.getDiskOfferingId());
+
+ try {
+ volumeDao.persist(restoredVolume);
+ } catch (Exception e) {
+ throw new CloudRuntimeException("Unable to craft restored volume due to: "+e);
+ }
+
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -c " + clusterName +
+ " -d " + destinationNetworkerClient +
+ " -n " + restoredVolume.getUuid() +
+ " -p " + dataStore.getLocalPath() +
+ " -a " + volume.getUuid();
+
+ if ( Boolean.TRUE.equals(NetworkerClientVerboseLogs.value()) )
+ command = command + " -v ";
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for Volume UUID " + volume.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return new Pair<>(true,restoredVolume.getUuid());
+ } else {
+ volumeDao.expunge(restoredVolume.getId());
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return null;
+ }
+ }
+
+ @Override
+ public boolean takeBackup(VirtualMachine vm) {
+
+ String networkerServer;
+ String clusterName;
+
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ // Find where the VM is currently running
+ HostVO hostVO = getRunningVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ // Get retention Period for our Backup
+ BackupOfferingVO vmBackupOffering = new BackupOfferingDaoImpl().findById(vm.getBackupOfferingId());
+ final String backupProviderPolicyId = vmBackupOffering.getExternalId();
+ String backupRentionPeriod = getClient(vm.getDataCenterId()).getBackupPolicyRetentionInterval(backupProviderPolicyId);
+
+ if ( backupRentionPeriod == null ) {
+ LOG.warn("There is no retention setting for Emc Networker Policy, setting default for 1 day");
+ backupRentionPeriod="1 Day";
+ }
+
+ // Get Cluster
+ clusterName = getVMHypervisorCluster(hostVO);
+
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmbackup.sh" +
+ " -s " + networkerServer +
+ " -R '" + backupRentionPeriod + "'" +
+ " -P " + NetworkerMediaPool.valueIn(vm.getDataCenterId()) +
+ " -c " + clusterName +
+ " -u " + vm.getUuid() +
+ " -t " + vm.getName();
+ if ( Boolean.TRUE.equals(NetworkerClientVerboseLogs.value()) )
+ command = command + " -v ";
+
+ LOG.debug("Starting backup for VM ID " + vm.getUuid() + " on Networker provider");
+ Date backupJobStart = new Date();
+
+ String saveTime = executeBackupCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command);
+ BackupVO backup = getClient(vm.getDataCenterId()).registerBackupForVm(vm, backupJobStart, saveTime);
+ if (backup != null) {
+ backupDao.persist(backup);
+ return true;
+ } else {
+ // We need to handle this rare situation where backup is successful but can't be registered properly.
+ return false;
+ }
+ }
+
+ @Override
+ public boolean deleteBackup(Backup backup) {
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+
+ if (getClient(zoneId).deleteBackupForVM(externalBackupId)) {
+ LOG.debug("EMC Networker successfully deleted backup with id " + externalBackupId);
+ return true;
+ } else {
+ LOG.debug("There was an error removing the backup with id " + externalBackupId + " from EMC NEtworker");
+ }
+ return false;
+ }
+
+ @Override
+ public Map<VirtualMachine, Backup.Metric> getBackupMetrics(Long zoneId, List<VirtualMachine> vms) {
+ final Map<VirtualMachine, Backup.Metric> metrics = new HashMap<>();
+ Long vmBackupSize=0L;
+ Long vmBackupProtectedSize=0L;
+
+ if (CollectionUtils.isEmpty(vms)) {
+ LOG.warn("Unable to get VM Backup Metrics because the list of VMs is empty.");
+ return metrics;
+ }
+
+ for (final VirtualMachine vm : vms) {
+ for ( Backup.VolumeInfo thisVMVol : vm.getBackupVolumeList()) {
+ vmBackupSize += (thisVMVol.getSize() / 1024L / 1024L);
+ }
+ final ArrayList<String> vmBackups = getClient(zoneId).getBackupsForVm(vm);
+ for ( String vmBackup : vmBackups ) {
+ NetworkerBackup vmNwBackup = getClient(zoneId).getNetworkerBackupInfo(vmBackup);
+ vmBackupProtectedSize+= vmNwBackup.getSize().getValue() / 1024L;
+ }
+ Backup.Metric vmBackupMetric = new Backup.Metric(vmBackupSize,vmBackupProtectedSize);
+ LOG.debug(String.format("Metrics for VM [uuid: %s, name: %s] is [backup size: %s, data size: %s].", vm.getUuid(),
+ vm.getInstanceName(), vmBackupMetric.getBackupSize(), vmBackupMetric.getDataSize()));
+ metrics.put(vm, vmBackupMetric);
+ }
+ return metrics;
+ }
+
+ @Override
+ public void syncBackups(VirtualMachine vm, Backup.Metric metric) {
Review Comment:
Yes. An advanced administrator can create backups on demand from third party scripts or manually. I explain more at line 593.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r991675972
##########
server/src/main/java/com/cloud/hypervisor/KVMGuru.java:
##########
@@ -204,7 +250,7 @@ protected Pair<Long, Integer> getHostMaxMemoryAndCpuCores(HostVO host, VirtualMa
if (lastHost != null) {
maxHostMemory = lastHost.getTotalMemory();
maxHostCpuCore = lastHost.getCpus();
- s_logger.debug(String.format("Retrieved memory and cpu max values {\"memory\": %s, \"cpu\": %s} from %s last %s.", maxHostMemory, maxHostCpuCore, vmDescription, lastHost.toString()));
+ s_logger.debug(String.format("Retrieved memory and cpu max values {\"memory\": %s, \"cpu\": %s} from %s last %s.", maxHostMemory, maxHostCpuCore, vmDescription, lastHost));
Review Comment:
lasthost() is already outputing string.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1252050514
> @DaanHoogland @rohityadavcloud Hello, how we can kick in sonar ?
@fermosan , maybe do as @rohityadavcloud suggested and rebase your code on latest, then force push it to your branch.
Did you manage to run sonar locally?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r951221692
##########
scripts/vm/hypervisor/kvm/nsrkvmbackup.sh:
##########
@@ -0,0 +1,270 @@
+#!/bin/bash
+
+## Licensed to the Apache Software Foundation (ASF) under one
+## or more contributor license agreements. See the NOTICE file
+## distributed with this work for additional information
+## regarding copyright ownership. The ASF licenses this file
+## to you under the Apache License, Version 2.0 (the
+## "License"); you may not use this file except in compliance
+## with the License. You may obtain a copy of the License at
+##
+## http://www.apache.org/licenses/LICENSE-2.0
+##
+## Unless required by applicable law or agreed to in writing,
+## software distributed under the License is distributed on an
+## "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+## KIND, either express or implied. See the License for the
+## specific language governing permissions and limitations
+## under the License.
+
Review Comment:
minor nit - extra newline
##########
scripts/vm/hypervisor/kvm/nsrkvmbackup.sh:
##########
@@ -0,0 +1,270 @@
+#!/bin/bash
+
+## Licensed to the Apache Software Foundation (ASF) under one
+## or more contributor license agreements. See the NOTICE file
+## distributed with this work for additional information
+## regarding copyright ownership. The ASF licenses this file
+## to you under the Apache License, Version 2.0 (the
+## "License"); you may not use this file except in compliance
+## with the License. You may obtain a copy of the License at
+##
+## http://www.apache.org/licenses/LICENSE-2.0
+##
+## Unless required by applicable law or agreed to in writing,
+## software distributed under the License is distributed on an
+## "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+## KIND, either express or implied. See the License for the
+## specific language governing permissions and limitations
+## under the License.
+
+
+version=0.9.1
+
+
Review Comment:
minor nit - extra newline
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r967635911
##########
plugins/backup/networker/src/test/java/org/apache/backup/networker/NetworkerClientTest.java:
##########
@@ -0,0 +1,741 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.backup.networker;
+
+import static com.github.tomakehurst.wiremock.client.WireMock.aResponse;
+import static com.github.tomakehurst.wiremock.client.WireMock.delete;
+import static com.github.tomakehurst.wiremock.client.WireMock.deleteRequestedFor;
+import static com.github.tomakehurst.wiremock.client.WireMock.get;
+import static com.github.tomakehurst.wiremock.client.WireMock.getRequestedFor;
+import static com.github.tomakehurst.wiremock.client.WireMock.urlEqualTo;
+import static com.github.tomakehurst.wiremock.client.WireMock.urlMatching;
+import static com.github.tomakehurst.wiremock.client.WireMock.verify;
+import static org.apache.cloudstack.backup.NetworkerBackupProvider.BACKUP_IDENTIFIER;
+
+import java.text.SimpleDateFormat;
+import java.time.Instant;
+import java.util.Date;
+import java.util.List;
+
+
+import com.cloud.vm.VMInstanceVO;
+import com.github.tomakehurst.wiremock.client.VerificationException;
+import org.apache.cloudstack.backup.BackupOffering;
+import org.apache.cloudstack.backup.BackupVO;
+import org.apache.cloudstack.backup.networker.NetworkerClient;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Rule;
+import org.junit.Test;
+
+
+import com.github.tomakehurst.wiremock.client.BasicCredentials;
+import com.github.tomakehurst.wiremock.junit.WireMockRule;
+
+
+public class NetworkerClientTest {
+ private final String adminUsername = "administrator";
+ private final String adminPassword = "password";
+ private final int port = 9399;
+ private final String url = "http://localhost:" + port + "/nwrestapi/v3";
+
+ private NetworkerClient client;
+
+
Review Comment:
fixed in latest commit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1218239720
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1201198838
@DaanHoogland a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1184273971
> > @DaanHoogland @rohityadavcloud @harikrishna-patnala Is JaCoCo failing for some particular reason ?
>
> are you fully ready yet, @fermosan ? let's not worry about jococo untill we are ready to merge.
Our tests look good. We are running some scenarios now. Surely, there will be another commit for the documentation pull in cloudstack-documentation.
I am currently looking at the following scenario. Maybe you can suggest something.
The backups are meant to expire at predefined retention days configured by the administrator on the Networker side.
This will happen automatically and Cloudstack B&R Framework will be unaware of this.
It looks like BackupSyncTask in B&R implementation can be utilized to do that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r920544705
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/Traditional.java:
##########
@@ -0,0 +1,299 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "forceBackupLevel",
+ "browsePeriod",
+ "estimate",
+ "enableDDRetentionLock",
+ "ddRetentionLockTime",
+ "destinationPool",
+ "timestampFormat",
+ "verifySyntheticFull",
+ "revertToFullWhenSyntheticFullFails",
+ "fileInactivityThresholdInDays",
+ "fileInactivityAlertThresholdPercentage"
+})
+@Generated("jsonschema2pojo")
+public class Traditional implements Serializable {
+
+ private final static long serialVersionUID = -6349295255710627836L;
+ @JsonProperty("forceBackupLevel")
+ private String forceBackupLevel;
+ @JsonProperty("browsePeriod")
+ private String browsePeriod;
+ @JsonProperty("estimate")
+ private Boolean estimate;
+ @JsonProperty("enableDDRetentionLock")
+ private Boolean enableDDRetentionLock;
+ @JsonProperty("ddRetentionLockTime")
+ private String ddRetentionLockTime;
+ @JsonProperty("destinationPool")
+ private String destinationPool;
+ @JsonProperty("timestampFormat")
+ private String timestampFormat;
+ @JsonProperty("verifySyntheticFull")
+ private Boolean verifySyntheticFull;
+ @JsonProperty("revertToFullWhenSyntheticFullFails")
+ private Boolean revertToFullWhenSyntheticFullFails;
+ @JsonProperty("fileInactivityThresholdInDays")
+ private Integer fileInactivityThresholdInDays;
+ @JsonProperty("fileInactivityAlertThresholdPercentage")
+ private Integer fileInactivityAlertThresholdPercentage;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public Traditional() {
+ }
+
+ /**
+ * @param enableDDRetentionLock
+ * @param revertToFullWhenSyntheticFullFails
+ * @param fileInactivityThresholdInDays
+ * @param destinationPool
+ * @param fileInactivityAlertThresholdPercentage
+ * @param estimate
+ * @param ddRetentionLockTime
+ * @param timestampFormat
+ * @param forceBackupLevel
+ * @param verifySyntheticFull
+ * @param browsePeriod
+ */
+ public Traditional(String forceBackupLevel, String browsePeriod, Boolean estimate, Boolean enableDDRetentionLock, String ddRetentionLockTime, String destinationPool, String timestampFormat, Boolean verifySyntheticFull, Boolean revertToFullWhenSyntheticFullFails, Integer fileInactivityThresholdInDays, Integer fileInactivityAlertThresholdPercentage) {
+ super();
+ this.forceBackupLevel = forceBackupLevel;
+ this.browsePeriod = browsePeriod;
+ this.estimate = estimate;
+ this.enableDDRetentionLock = enableDDRetentionLock;
+ this.ddRetentionLockTime = ddRetentionLockTime;
+ this.destinationPool = destinationPool;
+ this.timestampFormat = timestampFormat;
+ this.verifySyntheticFull = verifySyntheticFull;
+ this.revertToFullWhenSyntheticFullFails = revertToFullWhenSyntheticFullFails;
+ this.fileInactivityThresholdInDays = fileInactivityThresholdInDays;
+ this.fileInactivityAlertThresholdPercentage = fileInactivityAlertThresholdPercentage;
+ }
+
+ @JsonProperty("forceBackupLevel")
+ public String getForceBackupLevel() {
+ return forceBackupLevel;
+ }
+
+ @JsonProperty("forceBackupLevel")
+ public void setForceBackupLevel(String forceBackupLevel) {
+ this.forceBackupLevel = forceBackupLevel;
+ }
+
+ @JsonProperty("browsePeriod")
+ public String getBrowsePeriod() {
+ return browsePeriod;
+ }
+
+ @JsonProperty("browsePeriod")
+ public void setBrowsePeriod(String browsePeriod) {
+ this.browsePeriod = browsePeriod;
+ }
+
+ @JsonProperty("estimate")
+ public Boolean getEstimate() {
+ return estimate;
+ }
+
+ @JsonProperty("estimate")
+ public void setEstimate(Boolean estimate) {
+ this.estimate = estimate;
+ }
+
+ @JsonProperty("enableDDRetentionLock")
+ public Boolean getEnableDDRetentionLock() {
+ return enableDDRetentionLock;
+ }
+
+ @JsonProperty("enableDDRetentionLock")
+ public void setEnableDDRetentionLock(Boolean enableDDRetentionLock) {
+ this.enableDDRetentionLock = enableDDRetentionLock;
+ }
+
+ @JsonProperty("ddRetentionLockTime")
+ public String getDdRetentionLockTime() {
+ return ddRetentionLockTime;
+ }
+
+ @JsonProperty("ddRetentionLockTime")
+ public void setDdRetentionLockTime(String ddRetentionLockTime) {
+ this.ddRetentionLockTime = ddRetentionLockTime;
+ }
+
+ @JsonProperty("destinationPool")
+ public String getDestinationPool() {
+ return destinationPool;
+ }
+
+ @JsonProperty("destinationPool")
+ public void setDestinationPool(String destinationPool) {
+ this.destinationPool = destinationPool;
+ }
+
+ @JsonProperty("timestampFormat")
+ public String getTimestampFormat() {
+ return timestampFormat;
+ }
+
+ @JsonProperty("timestampFormat")
+ public void setTimestampFormat(String timestampFormat) {
+ this.timestampFormat = timestampFormat;
+ }
+
+ @JsonProperty("verifySyntheticFull")
+ public Boolean getVerifySyntheticFull() {
+ return verifySyntheticFull;
+ }
+
+ @JsonProperty("verifySyntheticFull")
+ public void setVerifySyntheticFull(Boolean verifySyntheticFull) {
+ this.verifySyntheticFull = verifySyntheticFull;
+ }
+
+ @JsonProperty("revertToFullWhenSyntheticFullFails")
+ public Boolean getRevertToFullWhenSyntheticFullFails() {
+ return revertToFullWhenSyntheticFullFails;
+ }
+
+ @JsonProperty("revertToFullWhenSyntheticFullFails")
+ public void setRevertToFullWhenSyntheticFullFails(Boolean revertToFullWhenSyntheticFullFails) {
+ this.revertToFullWhenSyntheticFullFails = revertToFullWhenSyntheticFullFails;
+ }
+
+ @JsonProperty("fileInactivityThresholdInDays")
+ public Integer getFileInactivityThresholdInDays() {
+ return fileInactivityThresholdInDays;
+ }
+
+ @JsonProperty("fileInactivityThresholdInDays")
+ public void setFileInactivityThresholdInDays(Integer fileInactivityThresholdInDays) {
+ this.fileInactivityThresholdInDays = fileInactivityThresholdInDays;
+ }
+
+ @JsonProperty("fileInactivityAlertThresholdPercentage")
+ public Integer getFileInactivityAlertThresholdPercentage() {
+ return fileInactivityAlertThresholdPercentage;
+ }
+
+ @JsonProperty("fileInactivityAlertThresholdPercentage")
+ public void setFileInactivityAlertThresholdPercentage(Integer fileInactivityAlertThresholdPercentage) {
+ this.fileInactivityAlertThresholdPercentage = fileInactivityAlertThresholdPercentage;
+ }
+
+ @Override
+ public String toString() {
+ StringBuilder sb = new StringBuilder();
+ sb.append(Traditional.class.getName()).append('@').append(Integer.toHexString(System.identityHashCode(this))).append('[');
+ sb.append("forceBackupLevel");
+ sb.append('=');
+ sb.append(((this.forceBackupLevel == null) ? "<null>" : this.forceBackupLevel));
+ sb.append(',');
+ sb.append("browsePeriod");
+ sb.append('=');
+ sb.append(((this.browsePeriod == null) ? "<null>" : this.browsePeriod));
+ sb.append(',');
+ sb.append("estimate");
+ sb.append('=');
+ sb.append(((this.estimate == null) ? "<null>" : this.estimate));
+ sb.append(',');
+ sb.append("enableDDRetentionLock");
+ sb.append('=');
+ sb.append(((this.enableDDRetentionLock == null) ? "<null>" : this.enableDDRetentionLock));
+ sb.append(',');
+ sb.append("ddRetentionLockTime");
+ sb.append('=');
+ sb.append(((this.ddRetentionLockTime == null) ? "<null>" : this.ddRetentionLockTime));
+ sb.append(',');
+ sb.append("destinationPool");
+ sb.append('=');
+ sb.append(((this.destinationPool == null) ? "<null>" : this.destinationPool));
+ sb.append(',');
+ sb.append("timestampFormat");
+ sb.append('=');
+ sb.append(((this.timestampFormat == null) ? "<null>" : this.timestampFormat));
+ sb.append(',');
+ sb.append("verifySyntheticFull");
+ sb.append('=');
+ sb.append(((this.verifySyntheticFull == null) ? "<null>" : this.verifySyntheticFull));
+ sb.append(',');
+ sb.append("revertToFullWhenSyntheticFullFails");
+ sb.append('=');
+ sb.append(((this.revertToFullWhenSyntheticFullFails == null) ? "<null>" : this.revertToFullWhenSyntheticFullFails));
+ sb.append(',');
+ sb.append("fileInactivityThresholdInDays");
+ sb.append('=');
+ sb.append(((this.fileInactivityThresholdInDays == null) ? "<null>" : this.fileInactivityThresholdInDays));
+ sb.append(',');
+ sb.append("fileInactivityAlertThresholdPercentage");
+ sb.append('=');
+ sb.append(((this.fileInactivityAlertThresholdPercentage == null) ? "<null>" : this.fileInactivityAlertThresholdPercentage));
+ sb.append(',');
+ if (sb.charAt((sb.length() - 1)) == ',') {
+ sb.setCharAt((sb.length() - 1), ']');
+ } else {
+ sb.append(']');
+ }
+ return sb.toString();
+ }
+
+ @Override
+ public int hashCode() {
+ int result = 1;
+ result = ((result * 31) + ((this.enableDDRetentionLock == null) ? 0 : this.enableDDRetentionLock.hashCode()));
+ result = ((result * 31) + ((this.revertToFullWhenSyntheticFullFails == null) ? 0 : this.revertToFullWhenSyntheticFullFails.hashCode()));
+ result = ((result * 31) + ((this.fileInactivityThresholdInDays == null) ? 0 : this.fileInactivityThresholdInDays.hashCode()));
+ result = ((result * 31) + ((this.destinationPool == null) ? 0 : this.destinationPool.hashCode()));
+ result = ((result * 31) + ((this.fileInactivityAlertThresholdPercentage == null) ? 0 : this.fileInactivityAlertThresholdPercentage.hashCode()));
+ result = ((result * 31) + ((this.estimate == null) ? 0 : this.estimate.hashCode()));
+ result = ((result * 31) + ((this.ddRetentionLockTime == null) ? 0 : this.ddRetentionLockTime.hashCode()));
+ result = ((result * 31) + ((this.timestampFormat == null) ? 0 : this.timestampFormat.hashCode()));
+ result = ((result * 31) + ((this.forceBackupLevel == null) ? 0 : this.forceBackupLevel.hashCode()));
+ result = ((result * 31) + ((this.verifySyntheticFull == null) ? 0 : this.verifySyntheticFull.hashCode()));
+ result = ((result * 31) + ((this.browsePeriod == null) ? 0 : this.browsePeriod.hashCode()));
+ return result;
+ }
+
+ @Override
+ public boolean equals(Object other) {
+ if (other == this) {
+ return true;
+ }
+ if ((other instanceof Traditional) == false) {
+ return false;
+ }
+ Traditional rhs = ((Traditional) other);
+ return ((((((((((((this.enableDDRetentionLock == rhs.enableDDRetentionLock) || ((this.enableDDRetentionLock != null) && this.enableDDRetentionLock.equals(rhs.enableDDRetentionLock))) && ((this.revertToFullWhenSyntheticFullFails == rhs.revertToFullWhenSyntheticFullFails) || ((this.revertToFullWhenSyntheticFullFails != null) && this.revertToFullWhenSyntheticFullFails.equals(rhs.revertToFullWhenSyntheticFullFails)))) && ((this.fileInactivityThresholdInDays == rhs.fileInactivityThresholdInDays) || ((this.fileInactivityThresholdInDays != null) && this.fileInactivityThresholdInDays.equals(rhs.fileInactivityThresholdInDays)))) && ((this.destinationPool == rhs.destinationPool) || ((this.destinationPool != null) && this.destinationPool.equals(rhs.destinationPool)))) && ((this.fileInactivityAlertThresholdPercentage == rhs.fileInactivityAlertThresholdPercentage) || ((this.fileInactivityAlertThresholdPercentage != null) &&
+ this.fileInactivityAlertThresholdPercentage.equals(rhs.fileInactivityAlertThresholdPercentage)))) && ((this.estimate == rhs.estimate) || ((this.estimate != null) && this.estimate.equals(rhs.estimate)))) && ((this.ddRetentionLockTime == rhs.ddRetentionLockTime) || ((this.ddRetentionLockTime != null) && this.ddRetentionLockTime.equals(rhs.ddRetentionLockTime)))) && ((this.timestampFormat == rhs.timestampFormat) || ((this.timestampFormat != null) && this.timestampFormat.equals(rhs.timestampFormat)))) && ((this.forceBackupLevel == rhs.forceBackupLevel) || ((this.forceBackupLevel != null) && this.forceBackupLevel.equals(rhs.forceBackupLevel)))) && ((this.verifySyntheticFull == rhs.verifySyntheticFull) || ((this.verifySyntheticFull != null) && this.verifySyntheticFull.equals(rhs.verifySyntheticFull)))) && ((this.browsePeriod == rhs.browsePeriod) || ((this.browsePeriod != null) && this.browsePeriod.equals(rhs.browsePeriod))));
Review Comment:
Will be removed since it is not used anywhere.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r920787306
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/ResourceId.java:
##########
@@ -0,0 +1,117 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "id",
+ "sequence"
+})
+@Generated("jsonschema2pojo")
+public class ResourceId implements Serializable {
+
+ private final static long serialVersionUID = -6987764740605099486L;
+ @JsonProperty("id")
+ private String id;
+ @JsonProperty("sequence")
+ private Integer sequence;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public ResourceId() {
+ }
+
+ /**
+ * @param sequence
+ * @param id
+ */
+ public ResourceId(String id, Integer sequence) {
+ super();
+ this.id = id;
+ this.sequence = sequence;
+ }
+
+ @JsonProperty("id")
+ public String getId() {
+ return id;
+ }
+
+ @JsonProperty("id")
+ public void setId(String id) {
+ this.id = id;
+ }
+
+ @JsonProperty("sequence")
+ public Integer getSequence() {
+ return sequence;
+ }
+
+ @JsonProperty("sequence")
+ public void setSequence(Integer sequence) {
+ this.sequence = sequence;
+ }
+
+ @Override
+ public String toString() {
Review Comment:
Transitioned.
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/Traditional.java:
##########
@@ -0,0 +1,299 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "forceBackupLevel",
+ "browsePeriod",
+ "estimate",
+ "enableDDRetentionLock",
+ "ddRetentionLockTime",
+ "destinationPool",
+ "timestampFormat",
+ "verifySyntheticFull",
+ "revertToFullWhenSyntheticFullFails",
+ "fileInactivityThresholdInDays",
+ "fileInactivityAlertThresholdPercentage"
+})
+@Generated("jsonschema2pojo")
+public class Traditional implements Serializable {
+
+ private final static long serialVersionUID = -6349295255710627836L;
+ @JsonProperty("forceBackupLevel")
+ private String forceBackupLevel;
+ @JsonProperty("browsePeriod")
+ private String browsePeriod;
+ @JsonProperty("estimate")
+ private Boolean estimate;
+ @JsonProperty("enableDDRetentionLock")
+ private Boolean enableDDRetentionLock;
+ @JsonProperty("ddRetentionLockTime")
+ private String ddRetentionLockTime;
+ @JsonProperty("destinationPool")
+ private String destinationPool;
+ @JsonProperty("timestampFormat")
+ private String timestampFormat;
+ @JsonProperty("verifySyntheticFull")
+ private Boolean verifySyntheticFull;
+ @JsonProperty("revertToFullWhenSyntheticFullFails")
+ private Boolean revertToFullWhenSyntheticFullFails;
+ @JsonProperty("fileInactivityThresholdInDays")
+ private Integer fileInactivityThresholdInDays;
+ @JsonProperty("fileInactivityAlertThresholdPercentage")
+ private Integer fileInactivityAlertThresholdPercentage;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public Traditional() {
+ }
+
+ /**
+ * @param enableDDRetentionLock
+ * @param revertToFullWhenSyntheticFullFails
+ * @param fileInactivityThresholdInDays
+ * @param destinationPool
+ * @param fileInactivityAlertThresholdPercentage
+ * @param estimate
+ * @param ddRetentionLockTime
+ * @param timestampFormat
+ * @param forceBackupLevel
+ * @param verifySyntheticFull
+ * @param browsePeriod
+ */
+ public Traditional(String forceBackupLevel, String browsePeriod, Boolean estimate, Boolean enableDDRetentionLock, String ddRetentionLockTime, String destinationPool, String timestampFormat, Boolean verifySyntheticFull, Boolean revertToFullWhenSyntheticFullFails, Integer fileInactivityThresholdInDays, Integer fileInactivityAlertThresholdPercentage) {
+ super();
+ this.forceBackupLevel = forceBackupLevel;
+ this.browsePeriod = browsePeriod;
+ this.estimate = estimate;
+ this.enableDDRetentionLock = enableDDRetentionLock;
+ this.ddRetentionLockTime = ddRetentionLockTime;
+ this.destinationPool = destinationPool;
+ this.timestampFormat = timestampFormat;
+ this.verifySyntheticFull = verifySyntheticFull;
+ this.revertToFullWhenSyntheticFullFails = revertToFullWhenSyntheticFullFails;
+ this.fileInactivityThresholdInDays = fileInactivityThresholdInDays;
+ this.fileInactivityAlertThresholdPercentage = fileInactivityAlertThresholdPercentage;
+ }
+
+ @JsonProperty("forceBackupLevel")
+ public String getForceBackupLevel() {
+ return forceBackupLevel;
+ }
+
+ @JsonProperty("forceBackupLevel")
+ public void setForceBackupLevel(String forceBackupLevel) {
+ this.forceBackupLevel = forceBackupLevel;
+ }
+
+ @JsonProperty("browsePeriod")
+ public String getBrowsePeriod() {
+ return browsePeriod;
+ }
+
+ @JsonProperty("browsePeriod")
+ public void setBrowsePeriod(String browsePeriod) {
+ this.browsePeriod = browsePeriod;
+ }
+
+ @JsonProperty("estimate")
+ public Boolean getEstimate() {
+ return estimate;
+ }
+
+ @JsonProperty("estimate")
+ public void setEstimate(Boolean estimate) {
+ this.estimate = estimate;
+ }
+
+ @JsonProperty("enableDDRetentionLock")
+ public Boolean getEnableDDRetentionLock() {
+ return enableDDRetentionLock;
+ }
+
+ @JsonProperty("enableDDRetentionLock")
+ public void setEnableDDRetentionLock(Boolean enableDDRetentionLock) {
+ this.enableDDRetentionLock = enableDDRetentionLock;
+ }
+
+ @JsonProperty("ddRetentionLockTime")
+ public String getDdRetentionLockTime() {
+ return ddRetentionLockTime;
+ }
+
+ @JsonProperty("ddRetentionLockTime")
+ public void setDdRetentionLockTime(String ddRetentionLockTime) {
+ this.ddRetentionLockTime = ddRetentionLockTime;
+ }
+
+ @JsonProperty("destinationPool")
+ public String getDestinationPool() {
+ return destinationPool;
+ }
+
+ @JsonProperty("destinationPool")
+ public void setDestinationPool(String destinationPool) {
+ this.destinationPool = destinationPool;
+ }
+
+ @JsonProperty("timestampFormat")
+ public String getTimestampFormat() {
+ return timestampFormat;
+ }
+
+ @JsonProperty("timestampFormat")
+ public void setTimestampFormat(String timestampFormat) {
+ this.timestampFormat = timestampFormat;
+ }
+
+ @JsonProperty("verifySyntheticFull")
+ public Boolean getVerifySyntheticFull() {
+ return verifySyntheticFull;
+ }
+
+ @JsonProperty("verifySyntheticFull")
+ public void setVerifySyntheticFull(Boolean verifySyntheticFull) {
+ this.verifySyntheticFull = verifySyntheticFull;
+ }
+
+ @JsonProperty("revertToFullWhenSyntheticFullFails")
+ public Boolean getRevertToFullWhenSyntheticFullFails() {
+ return revertToFullWhenSyntheticFullFails;
+ }
+
+ @JsonProperty("revertToFullWhenSyntheticFullFails")
+ public void setRevertToFullWhenSyntheticFullFails(Boolean revertToFullWhenSyntheticFullFails) {
+ this.revertToFullWhenSyntheticFullFails = revertToFullWhenSyntheticFullFails;
+ }
+
+ @JsonProperty("fileInactivityThresholdInDays")
+ public Integer getFileInactivityThresholdInDays() {
+ return fileInactivityThresholdInDays;
+ }
+
+ @JsonProperty("fileInactivityThresholdInDays")
+ public void setFileInactivityThresholdInDays(Integer fileInactivityThresholdInDays) {
+ this.fileInactivityThresholdInDays = fileInactivityThresholdInDays;
+ }
+
+ @JsonProperty("fileInactivityAlertThresholdPercentage")
+ public Integer getFileInactivityAlertThresholdPercentage() {
+ return fileInactivityAlertThresholdPercentage;
+ }
+
+ @JsonProperty("fileInactivityAlertThresholdPercentage")
+ public void setFileInactivityAlertThresholdPercentage(Integer fileInactivityAlertThresholdPercentage) {
+ this.fileInactivityAlertThresholdPercentage = fileInactivityAlertThresholdPercentage;
+ }
+
+ @Override
+ public String toString() {
Review Comment:
Transitioned.
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/SummaryNotification.java:
##########
@@ -0,0 +1,117 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "command",
+ "executeOn"
+})
+@Generated("jsonschema2pojo")
+public class SummaryNotification implements Serializable {
+
+ private final static long serialVersionUID = -1142867788732047105L;
+ @JsonProperty("command")
+ private String command;
+ @JsonProperty("executeOn")
+ private String executeOn;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public SummaryNotification() {
+ }
+
+ /**
+ * @param executeOn
+ * @param command
+ */
+ public SummaryNotification(String command, String executeOn) {
+ super();
+ this.command = command;
+ this.executeOn = executeOn;
+ }
+
+ @JsonProperty("command")
+ public String getCommand() {
+ return command;
+ }
+
+ @JsonProperty("command")
+ public void setCommand(String command) {
+ this.command = command;
+ }
+
+ @JsonProperty("executeOn")
+ public String getExecuteOn() {
+ return executeOn;
+ }
+
+ @JsonProperty("executeOn")
+ public void setExecuteOn(String executeOn) {
+ this.executeOn = executeOn;
+ }
+
+ @Override
+ public String toString() {
Review Comment:
Transitioned.
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/Size.java:
##########
@@ -0,0 +1,117 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "unit",
+ "value"
+})
+@Generated("jsonschema2pojo")
+public class Size implements Serializable {
+
+ private final static long serialVersionUID = -8270448162523429622L;
+ @JsonProperty("unit")
+ private String unit;
+ @JsonProperty("value")
+ private Integer value;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public Size() {
+ }
+
+ /**
+ * @param unit
+ * @param value
+ */
+ public Size(String unit, Integer value) {
+ super();
+ this.unit = unit;
+ this.value = value;
+ }
+
+ @JsonProperty("unit")
+ public String getUnit() {
+ return unit;
+ }
+
+ @JsonProperty("unit")
+ public void setUnit(String unit) {
+ this.unit = unit;
+ }
+
+ @JsonProperty("value")
+ public Integer getValue() {
+ return value;
+ }
+
+ @JsonProperty("value")
+ public void setValue(Integer value) {
+ this.value = value;
+ }
+
+ @Override
+ public String toString() {
Review Comment:
Transitioned.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] acs-robot commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
acs-robot commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1184041405
Found UI changes, kicking a new UI QA build
@blueorangutan ui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r920101661
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/ProtectionPolicy.java:
##########
@@ -0,0 +1,222 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+import java.util.List;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "comment",
+ "links",
+ "name",
+ "policyProtectionEnable",
+ "policyProtectionPeriod",
+ "resourceId",
+ "summaryNotification",
+ "workflows"
+})
+@Generated("jsonschema2pojo")
+public class ProtectionPolicy implements Serializable {
+
+ private final static long serialVersionUID = 5407494949453441445L;
+ @JsonProperty("comment")
+ private String comment;
+ @JsonProperty("links")
+ private List<Link> links = null;
+ @JsonProperty("name")
+ private String name;
+ @JsonProperty("policyProtectionEnable")
+ private Boolean policyProtectionEnable;
+ @JsonProperty("policyProtectionPeriod")
+ private String policyProtectionPeriod;
+ @JsonProperty("resourceId")
+ private ResourceId resourceId;
+ @JsonProperty("summaryNotification")
+ private SummaryNotification summaryNotification;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public ProtectionPolicy() {
+ }
+
+ /**
+ * @param policyProtectionEnable
+ * @param policyProtectionPeriod
+ * @param summaryNotification
+ * @param resourceId
+ * @param name
+ * @param comment
+ * @param links
+ */
+ public ProtectionPolicy(String comment, List<Link> links, String name, Boolean policyProtectionEnable, String policyProtectionPeriod, ResourceId resourceId, SummaryNotification summaryNotification) {
+ super();
+ this.comment = comment;
+ this.links = links;
+ this.name = name;
+ this.policyProtectionEnable = policyProtectionEnable;
+ this.policyProtectionPeriod = policyProtectionPeriod;
+ this.resourceId = resourceId;
+ this.summaryNotification = summaryNotification;
+ }
+
+ @JsonProperty("comment")
+ public String getComment() {
+ return comment;
+ }
+
+ @JsonProperty("comment")
+ public void setComment(String comment) {
+ this.comment = comment;
+ }
+
+ @JsonProperty("links")
+ public List<Link> getLinks() {
+ return links;
+ }
+
+ @JsonProperty("links")
+ public void setLinks(List<Link> links) {
+ this.links = links;
+ }
+
+ @JsonProperty("name")
+ public String getName() {
+ return name;
+ }
+
+ @JsonProperty("name")
+ public void setName(String name) {
+ this.name = name;
+ }
+
+ @JsonProperty("policyProtectionEnable")
+ public Boolean getPolicyProtectionEnable() {
+ return policyProtectionEnable;
+ }
+
+ @JsonProperty("policyProtectionEnable")
+ public void setPolicyProtectionEnable(Boolean policyProtectionEnable) {
+ this.policyProtectionEnable = policyProtectionEnable;
+ }
+
+ @JsonProperty("policyProtectionPeriod")
+ public String getPolicyProtectionPeriod() {
+ return policyProtectionPeriod;
+ }
+
+ @JsonProperty("policyProtectionPeriod")
+ public void setPolicyProtectionPeriod(String policyProtectionPeriod) {
+ this.policyProtectionPeriod = policyProtectionPeriod;
+ }
+
+ @JsonProperty("resourceId")
+ public ResourceId getResourceId() {
+ return resourceId;
+ }
+
+ @JsonProperty("resourceId")
+ public void setResourceId(ResourceId resourceId) {
+ this.resourceId = resourceId;
+ }
+
+ @JsonProperty("summaryNotification")
+ public SummaryNotification getSummaryNotification() {
+ return summaryNotification;
+ }
+
+ @JsonProperty("summaryNotification")
+ public void setSummaryNotification(SummaryNotification summaryNotification) {
+ this.summaryNotification = summaryNotification;
+ }
+
+
+ @Override
+ public String toString() {
Review Comment:
please use `ReflectionToStringBuilderUtils`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r920102918
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/ResourceId.java:
##########
@@ -0,0 +1,117 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "id",
+ "sequence"
+})
+@Generated("jsonschema2pojo")
+public class ResourceId implements Serializable {
+
+ private final static long serialVersionUID = -6987764740605099486L;
+ @JsonProperty("id")
+ private String id;
+ @JsonProperty("sequence")
+ private Integer sequence;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public ResourceId() {
+ }
+
+ /**
+ * @param sequence
+ * @param id
+ */
+ public ResourceId(String id, Integer sequence) {
+ super();
+ this.id = id;
+ this.sequence = sequence;
+ }
+
+ @JsonProperty("id")
+ public String getId() {
+ return id;
+ }
+
+ @JsonProperty("id")
+ public void setId(String id) {
+ this.id = id;
+ }
+
+ @JsonProperty("sequence")
+ public Integer getSequence() {
+ return sequence;
+ }
+
+ @JsonProperty("sequence")
+ public void setSequence(Integer sequence) {
+ this.sequence = sequence;
+ }
+
+ @Override
+ public String toString() {
Review Comment:
`ReflectionToStringBuilderUtils`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r920106110
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/Traditional.java:
##########
@@ -0,0 +1,299 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "forceBackupLevel",
+ "browsePeriod",
+ "estimate",
+ "enableDDRetentionLock",
+ "ddRetentionLockTime",
+ "destinationPool",
+ "timestampFormat",
+ "verifySyntheticFull",
+ "revertToFullWhenSyntheticFullFails",
+ "fileInactivityThresholdInDays",
+ "fileInactivityAlertThresholdPercentage"
+})
+@Generated("jsonschema2pojo")
+public class Traditional implements Serializable {
+
+ private final static long serialVersionUID = -6349295255710627836L;
+ @JsonProperty("forceBackupLevel")
+ private String forceBackupLevel;
+ @JsonProperty("browsePeriod")
+ private String browsePeriod;
+ @JsonProperty("estimate")
+ private Boolean estimate;
+ @JsonProperty("enableDDRetentionLock")
+ private Boolean enableDDRetentionLock;
+ @JsonProperty("ddRetentionLockTime")
+ private String ddRetentionLockTime;
+ @JsonProperty("destinationPool")
+ private String destinationPool;
+ @JsonProperty("timestampFormat")
+ private String timestampFormat;
+ @JsonProperty("verifySyntheticFull")
+ private Boolean verifySyntheticFull;
+ @JsonProperty("revertToFullWhenSyntheticFullFails")
+ private Boolean revertToFullWhenSyntheticFullFails;
+ @JsonProperty("fileInactivityThresholdInDays")
+ private Integer fileInactivityThresholdInDays;
+ @JsonProperty("fileInactivityAlertThresholdPercentage")
+ private Integer fileInactivityAlertThresholdPercentage;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public Traditional() {
+ }
+
+ /**
+ * @param enableDDRetentionLock
+ * @param revertToFullWhenSyntheticFullFails
+ * @param fileInactivityThresholdInDays
+ * @param destinationPool
+ * @param fileInactivityAlertThresholdPercentage
+ * @param estimate
+ * @param ddRetentionLockTime
+ * @param timestampFormat
+ * @param forceBackupLevel
+ * @param verifySyntheticFull
+ * @param browsePeriod
+ */
+ public Traditional(String forceBackupLevel, String browsePeriod, Boolean estimate, Boolean enableDDRetentionLock, String ddRetentionLockTime, String destinationPool, String timestampFormat, Boolean verifySyntheticFull, Boolean revertToFullWhenSyntheticFullFails, Integer fileInactivityThresholdInDays, Integer fileInactivityAlertThresholdPercentage) {
+ super();
+ this.forceBackupLevel = forceBackupLevel;
+ this.browsePeriod = browsePeriod;
+ this.estimate = estimate;
+ this.enableDDRetentionLock = enableDDRetentionLock;
+ this.ddRetentionLockTime = ddRetentionLockTime;
+ this.destinationPool = destinationPool;
+ this.timestampFormat = timestampFormat;
+ this.verifySyntheticFull = verifySyntheticFull;
+ this.revertToFullWhenSyntheticFullFails = revertToFullWhenSyntheticFullFails;
+ this.fileInactivityThresholdInDays = fileInactivityThresholdInDays;
+ this.fileInactivityAlertThresholdPercentage = fileInactivityAlertThresholdPercentage;
+ }
+
+ @JsonProperty("forceBackupLevel")
+ public String getForceBackupLevel() {
+ return forceBackupLevel;
+ }
+
+ @JsonProperty("forceBackupLevel")
+ public void setForceBackupLevel(String forceBackupLevel) {
+ this.forceBackupLevel = forceBackupLevel;
+ }
+
+ @JsonProperty("browsePeriod")
+ public String getBrowsePeriod() {
+ return browsePeriod;
+ }
+
+ @JsonProperty("browsePeriod")
+ public void setBrowsePeriod(String browsePeriod) {
+ this.browsePeriod = browsePeriod;
+ }
+
+ @JsonProperty("estimate")
+ public Boolean getEstimate() {
+ return estimate;
+ }
+
+ @JsonProperty("estimate")
+ public void setEstimate(Boolean estimate) {
+ this.estimate = estimate;
+ }
+
+ @JsonProperty("enableDDRetentionLock")
+ public Boolean getEnableDDRetentionLock() {
+ return enableDDRetentionLock;
+ }
+
+ @JsonProperty("enableDDRetentionLock")
+ public void setEnableDDRetentionLock(Boolean enableDDRetentionLock) {
+ this.enableDDRetentionLock = enableDDRetentionLock;
+ }
+
+ @JsonProperty("ddRetentionLockTime")
+ public String getDdRetentionLockTime() {
+ return ddRetentionLockTime;
+ }
+
+ @JsonProperty("ddRetentionLockTime")
+ public void setDdRetentionLockTime(String ddRetentionLockTime) {
+ this.ddRetentionLockTime = ddRetentionLockTime;
+ }
+
+ @JsonProperty("destinationPool")
+ public String getDestinationPool() {
+ return destinationPool;
+ }
+
+ @JsonProperty("destinationPool")
+ public void setDestinationPool(String destinationPool) {
+ this.destinationPool = destinationPool;
+ }
+
+ @JsonProperty("timestampFormat")
+ public String getTimestampFormat() {
+ return timestampFormat;
+ }
+
+ @JsonProperty("timestampFormat")
+ public void setTimestampFormat(String timestampFormat) {
+ this.timestampFormat = timestampFormat;
+ }
+
+ @JsonProperty("verifySyntheticFull")
+ public Boolean getVerifySyntheticFull() {
+ return verifySyntheticFull;
+ }
+
+ @JsonProperty("verifySyntheticFull")
+ public void setVerifySyntheticFull(Boolean verifySyntheticFull) {
+ this.verifySyntheticFull = verifySyntheticFull;
+ }
+
+ @JsonProperty("revertToFullWhenSyntheticFullFails")
+ public Boolean getRevertToFullWhenSyntheticFullFails() {
+ return revertToFullWhenSyntheticFullFails;
+ }
+
+ @JsonProperty("revertToFullWhenSyntheticFullFails")
+ public void setRevertToFullWhenSyntheticFullFails(Boolean revertToFullWhenSyntheticFullFails) {
+ this.revertToFullWhenSyntheticFullFails = revertToFullWhenSyntheticFullFails;
+ }
+
+ @JsonProperty("fileInactivityThresholdInDays")
+ public Integer getFileInactivityThresholdInDays() {
+ return fileInactivityThresholdInDays;
+ }
+
+ @JsonProperty("fileInactivityThresholdInDays")
+ public void setFileInactivityThresholdInDays(Integer fileInactivityThresholdInDays) {
+ this.fileInactivityThresholdInDays = fileInactivityThresholdInDays;
+ }
+
+ @JsonProperty("fileInactivityAlertThresholdPercentage")
+ public Integer getFileInactivityAlertThresholdPercentage() {
+ return fileInactivityAlertThresholdPercentage;
+ }
+
+ @JsonProperty("fileInactivityAlertThresholdPercentage")
+ public void setFileInactivityAlertThresholdPercentage(Integer fileInactivityAlertThresholdPercentage) {
+ this.fileInactivityAlertThresholdPercentage = fileInactivityAlertThresholdPercentage;
+ }
+
+ @Override
+ public String toString() {
Review Comment:
use `ReflectionToStringBuilderUtils`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1204872558
@DaanHoogland a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r1010084794
##########
ui/public/config.json:
##########
@@ -63,5 +63,121 @@
"basicZoneEnabled": true,
"multipleServer": false,
"allowSettingTheme": true,
- "docHelpMappings": {}
+ "docHelpMappings": {
Review Comment:
Can you remove the added changes to be same as https://github.com/apache/cloudstack/blob/main/ui/public/config.json#L66 ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r1013882994
##########
ui/public/config.json:
##########
@@ -63,5 +63,121 @@
"basicZoneEnabled": true,
"multipleServer": false,
"allowSettingTheme": true,
- "docHelpMappings": {}
+ "docHelpMappings": {
Review Comment:
Resolved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1188368142
@rohityadavcloud Can you provide some insight regarding JaCoCo errors ? Is it something with the commit ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland closed pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland closed pull request #6550: Emc networker b&r
URL: https://github.com/apache/cloudstack/pull/6550
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r951222208
##########
scripts/vm/hypervisor/kvm/nsrkvmbackup.sh:
##########
@@ -0,0 +1,270 @@
+#!/bin/bash
+
+## Licensed to the Apache Software Foundation (ASF) under one
+## or more contributor license agreements. See the NOTICE file
+## distributed with this work for additional information
+## regarding copyright ownership. The ASF licenses this file
+## to you under the Apache License, Version 2.0 (the
+## "License"); you may not use this file except in compliance
+## with the License. You may obtain a copy of the License at
+##
+## http://www.apache.org/licenses/LICENSE-2.0
+##
+## Unless required by applicable law or agreed to in writing,
+## software distributed under the License is distributed on an
+## "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+## KIND, either express or implied. See the License for the
+## specific language governing permissions and limitations
+## under the License.
+
+
+version=0.9.1
+
+
+OPTIND=1
+
+verb=0
+logDir="/nsr/logs/cloudstack/"
Review Comment:
should this be /var/log/cloudstack/management/... ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r951223769
##########
scripts/vm/hypervisor/kvm/nsrkvmrestore.sh:
##########
@@ -0,0 +1,234 @@
+#!/bin/bash
+
+## Licensed to the Apache Software Foundation (ASF) under one
+## or more contributor license agreements. See the NOTICE file
+## distributed with this work for additional information
+## regarding copyright ownership. The ASF licenses this file
+## to you under the Apache License, Version 2.0 (the
+## "License"); you may not use this file except in compliance
+## with the License. You may obtain a copy of the License at
+##
+## http://www.apache.org/licenses/LICENSE-2.0
+##
+## Unless required by applicable law or agreed to in writing,
+## software distributed under the License is distributed on an
+## "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+## KIND, either express or implied. See the License for the
+## specific language governing permissions and limitations
+## under the License.
+
+
Review Comment:
minor nit - extra newline
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1199288287
@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1228290462
@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1297514164
<b>Trillian test result (tid-5233)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 40027 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6550-t5233-kvm-centos7.zip
Smoke tests completed. 104 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r1009273511
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/NetworkerBackupProvider.java:
##########
@@ -0,0 +1,646 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.backup;
+
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.host.HostVO;
+import com.cloud.host.Status;
+import com.cloud.host.dao.HostDao;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.StoragePoolHostVO;
+import com.cloud.storage.Volume;
+import com.cloud.storage.VolumeVO;
+import com.cloud.storage.dao.StoragePoolHostDao;
+import com.cloud.storage.dao.VolumeDao;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.component.AdapterBase;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallbackNoReturn;
+import com.cloud.utils.db.TransactionStatus;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.ssh.SshHelper;
+import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.dao.VMInstanceDao;
+import org.apache.cloudstack.api.InternalIdentity;
+import org.apache.cloudstack.backup.dao.BackupDao;
+import org.apache.cloudstack.backup.dao.BackupOfferingDaoImpl;
+import org.apache.cloudstack.backup.networker.NetworkerClient;
+import org.apache.cloudstack.framework.config.ConfigKey;
+import org.apache.cloudstack.framework.config.Configurable;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.log4j.Logger;
+import org.apache.xml.utils.URI;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+import javax.inject.Inject;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.HashMap;
+import java.util.Date;
+import java.util.Objects;
+import java.util.UUID;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+import com.cloud.utils.script.Script;
+
+public class NetworkerBackupProvider extends AdapterBase implements BackupProvider, Configurable {
+
+ public static final String BACKUP_IDENTIFIER = "-CSBKP-";
+ private static final Logger LOG = Logger.getLogger(NetworkerBackupProvider.class);
+
+ public ConfigKey<String> NetworkerUrl = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.url", "https://localhost:9090/nwrestapi/v3",
+ "The EMC Networker API URL.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerUsername = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.username", "administrator",
+ "The EMC Networker API username.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerPassword = new ConfigKey<>("Secure", String.class,
+ "backup.plugin.networker.password", "password",
+ "The EMC Networker API password.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerMediaPool = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.pool", "Default",
+ "The EMC Networker Media Pool", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerValidateSSLSecurity = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.validate.ssl", "false",
+ "Validate the SSL certificate when connecting to EMC Networker API service.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Integer> NetworkerApiRequestTimeout = new ConfigKey<>("Advanced", Integer.class,
+ "backup.plugin.networker.request.timeout", "300",
+ "The EMC Networker API request timeout in seconds.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerClientVerboseLogs = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.client.verbosity", "false",
+ "Produce Verbose logs in Hypervisor", true, ConfigKey.Scope.Zone);
+
+ @Inject
+ private BackupDao backupDao;
+
+ @Inject
+ private HostDao hostDao;
+
+ @Inject
+ private ConfigurationDao configDao;
+
+ @Inject
+ private ClusterDao clusterDao;
+
+ @Inject
+ private VolumeDao volumeDao;
+
+ @Inject
+ private StoragePoolHostDao storagePoolHostDao;
+
+ @Inject
+ private VMInstanceDao vmInstanceDao;
+
+ private static String getUrlDomain(String url) throws URISyntaxException {
+ URI uri;
+ try {
+ uri = new URI(url);
+ } catch (URI.MalformedURIException e) {
+ throw new CloudRuntimeException("Failed to cast URI");
+ }
+
+ return uri.getHost();
+ }
+
+ @Override
+ public ConfigKey<?>[] getConfigKeys() {
+ return new ConfigKey[]{
+ NetworkerUrl,
+ NetworkerUsername,
+ NetworkerPassword,
+ NetworkerValidateSSLSecurity,
+ NetworkerApiRequestTimeout,
+ NetworkerMediaPool,
+ NetworkerClientVerboseLogs
+ };
+ }
+
+ @Override
+ public String getName() {
+ return "networker";
+ }
+
+ @Override
+ public String getDescription() {
+ return "EMC Networker Backup Plugin";
+ }
+
+ @Override
+ public String getConfigComponentName() {
+ return BackupService.class.getSimpleName();
+ }
+
+ protected HostVO getLastVMHypervisorHost(VirtualMachine vm) {
+ HostVO host;
+ Long hostId = vm.getLastHostId();
+
+ if (hostId == null) {
+ LOG.debug("Cannot find last host for vm. This should never happen, please check your database.");
+ return null;
+ }
+ host = hostDao.findById(hostId);
+
+ if ( host.getStatus() == Status.Up ) {
+ return host;
+ } else {
+ // Try to find a host in the same cluster
+ List<HostVO> altClusterHosts = hostDao.findHypervisorHostInCluster(host.getClusterId());
+ for (final HostVO candidateClusterHost : altClusterHosts) {
+ if ( candidateClusterHost.getStatus() == Status.Up ) {
+ LOG.debug("Found Host " + candidateClusterHost.getName());
+ return candidateClusterHost;
+ }
+ }
+ }
+ // Try to find a Host in the zone
+ List<HostVO> altZoneHosts = hostDao.findByDataCenterId(host.getDataCenterId());
+ for (final HostVO candidateZoneHost : altZoneHosts) {
+ if ( candidateZoneHost.getStatus() == Status.Up && candidateZoneHost.getHypervisorType() == Hypervisor.HypervisorType.KVM ) {
+ LOG.debug("Found Host " + candidateZoneHost.getName());
+ return candidateZoneHost;
+ }
+ }
+ return null;
+ }
+
+ protected HostVO getRunningVMHypervisorHost(VirtualMachine vm) {
+
+ HostVO host;
+ Long hostId = vm.getHostId();
+
+ if (hostId == null) {
+ throw new CloudRuntimeException("Unable to find the HYPERVISOR for " + vm.getName() + ". Make sure the virtual machine is running");
+ }
+
+ host = hostDao.findById(hostId);
+
+ return host;
+ }
+
+ protected String getVMHypervisorCluster(HostVO host) {
+
+ return clusterDao.findById(host.getClusterId()).getName();
+ }
+
+ protected Ternary<String, String, String> getKVMHyperisorCredentials(HostVO host) {
+
+ String username = null;
+ String password = null;
+
+ if (host != null && host.getHypervisorType() == Hypervisor.HypervisorType.KVM) {
+ hostDao.loadDetails(host);
+ password = host.getDetail("password");
+ username = host.getDetail("username");
+ }
+ if ( password == null || username == null) {
+ throw new CloudRuntimeException("Cannot find login credentials for HYPERVISOR " + Objects.requireNonNull(host).getUuid());
+ }
+
+ return new Ternary<>(username, password, null);
+ }
+
+ private String executeBackupCommand(HostVO host, String username, String password, String command) {
+ String nstRegex = "\\bcompleted savetime=([0-9]{10})";
+ Pattern saveTimePattern = Pattern.compile(nstRegex);
+
+ try {
+ Pair<Boolean, String> response = SshHelper.sshExecute(host.getPrivateIpAddress(), 22,
+ username, null, password, command, 120000, 120000, 3600000);
+ if (!response.first()) {
+ LOG.error(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, response.second()));
+ } else {
+ LOG.debug(String.format("Networker Backup Results: %s", response.second()));
+ }
+ Matcher saveTimeMatcher = saveTimePattern.matcher(response.second());
+ if (saveTimeMatcher.find()) {
+ LOG.debug(String.format("Got saveTimeMatcher: %s", saveTimeMatcher.group(1)));
+ return saveTimeMatcher.group(1);
+ }
+ } catch (final Exception e) {
+ throw new CloudRuntimeException(String.format("Failed to take backup on host %s due to: %s", host.getName(), e.getMessage()));
+ }
+
+ return null;
+ }
+ private boolean executeRestoreCommand(HostVO host, String username, String password, String command) {
+
+ try {
+ Pair<Boolean, String> response = SshHelper.sshExecute(host.getPrivateIpAddress(), 22,
+ username, null, password, command, 120000, 120000, 3600000);
+
+ if (!response.first()) {
+ LOG.error(String.format("Restore Script failed on HYPERVISOR %s due to: %s", host, response.second()));
+ } else {
+ LOG.debug(String.format("Networker Restore Results: %s",response.second()));
+ return true;
+ }
+ } catch (final Exception e) {
+ throw new CloudRuntimeException(String.format("Failed to restore backup on host %s due to: %s", host.getName(), e.getMessage()));
+ }
+ return false;
+ }
+
+ private NetworkerClient getClient(final Long zoneId) {
+ try {
+ return new NetworkerClient(NetworkerUrl.valueIn(zoneId), NetworkerUsername.valueIn(zoneId), NetworkerPassword.valueIn(zoneId),
+ NetworkerValidateSSLSecurity.valueIn(zoneId), NetworkerApiRequestTimeout.valueIn(zoneId));
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException("Failed to parse EMC Networker API URL: " + e.getMessage());
+ } catch (NoSuchAlgorithmException | KeyManagementException e) {
+ LOG.error("Failed to build EMC Networker API client due to: ", e);
+ }
+ throw new CloudRuntimeException("Failed to build EMC Networker API client");
+ }
+
+ @Override
+ public List<BackupOffering> listBackupOfferings(Long zoneId) {
+ List<BackupOffering> policies = new ArrayList<>();
+ for (final BackupOffering policy : getClient(zoneId).listPolicies()) {
+ if (!policy.getName().contains(BACKUP_IDENTIFIER)) {
+ policies.add(policy);
+ }
+ }
+
+ return policies;
+ }
+
+ @Override
+ public boolean isValidProviderOffering(Long zoneId, String uuid) {
+ List<BackupOffering> policies = listBackupOfferings(zoneId);
+ if (CollectionUtils.isEmpty(policies)) {
+ return false;
+ }
+ for (final BackupOffering policy : policies) {
+ if (Objects.equals(policy.getExternalId(), uuid)) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ @Override
+ public boolean assignVMToBackupOffering(VirtualMachine vm, BackupOffering backupOffering) { return true; }
+
+ @Override
+ public boolean removeVMFromBackupOffering(VirtualMachine vm) {
+ LOG.debug("Removing VirtualMachine from Backup offering and Deleting any existing backups");
+
+ List<String> backupsTaken = getClient(vm.getDataCenterId()).getBackupsForVm(vm);
+
+ for (String backupId : backupsTaken) {
+ LOG.debug("Trying to remove backup with id" + backupId);
+ getClient(vm.getDataCenterId()).deleteBackupForVM(backupId);
+ }
+
+ return true;
+ }
+
+ @Override
+ public boolean restoreVMFromBackup(VirtualMachine vm, Backup backup) {
+ String backupRestorePath = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh";
+ String networkerServer;
+ HostVO hostVO;
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+
+ LOG.debug("Restoring vm " + vm.getUuid() + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return false;
+ }
+
+ // Find where the VM was last running
+ hostVO = getLastVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ final Script script = new Script(backupRestorePath);
+ script.add("-s");
+ script.add(networkerServer);
+ script.add("-S");
+ script.add(SSID);
+
+ if ( Boolean.TRUE.equals(NetworkerClientVerboseLogs.value()) )
+ script.add("-v");
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for VM ID " + vm.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), script.toString()) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return true;
+ } else {
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return false;
+ }
+ }
+
+ @Override
+ public Pair<Boolean, String> restoreBackedUpVolume(Backup backup, String volumeUuid, String hostIp, String dataStoreUuid) {
+ String backupRestorePath = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh";
Review Comment:
With a little bit more work on the EMC side we might be able to run them without sudo. I will need to test and get back to you. Alternatively, how can we utilize the java libvirt functionality on the client side ? is there an example ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1296565297
@rohityadavcloud a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1296564789
@DaanHoogland sorry I was away on holidays, I'll address it this week
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r1009092083
##########
server/src/main/java/com/cloud/hypervisor/KVMGuru.java:
##########
@@ -139,7 +185,7 @@ protected void configureVmOsDescription(VirtualMachine virtualMachine, VirtualMa
guestOsMapping = _guestOsHypervisorDao.findByOsIdAndHypervisor(guestOS.getId(), getHypervisorType().toString(), hostVo.getHypervisorVersion());
}
- if (guestOsMapping == null || hostVo == null) {
+ if (guestOsMapping == null) {
Review Comment:
@fermosan
I read the code from main branch and it seems this can be null if hostId is null in caller at line 126 https://github.com/apache/cloudstack/blob/main/server/src/main/java/com/cloud/hypervisor/KVMGuru.java#L123
##########
server/src/main/java/com/cloud/hypervisor/KVMGuru.java:
##########
@@ -167,9 +213,9 @@ protected void configureVmMemoryAndCpuCores(VirtualMachineTO virtualMachineTo, H
Long maxHostMemory = max.first();
Integer maxHostCpuCore = max.second();
- Long minMemory = virtualMachineTo.getMinRam();
+ long minMemory = virtualMachineTo.getMinRam();
Long maxMemory = minMemory;
- Integer minCpuCores = virtualMachineTo.getCpus();
+ int minCpuCores = virtualMachineTo.getCpus();
Review Comment:
@fermosan ?
##########
ui/public/config.json:
##########
@@ -63,5 +63,121 @@
"basicZoneEnabled": true,
"multipleServer": false,
"allowSettingTheme": true,
- "docHelpMappings": {}
+ "docHelpMappings": {
Review Comment:
Another note about package-lock.json - what version of npm do you have? (I don't remember if we should be committing package-lock.json changes?)
##########
server/src/main/java/com/cloud/hypervisor/KVMGuru.java:
##########
@@ -167,9 +213,9 @@ protected void configureVmMemoryAndCpuCores(VirtualMachineTO virtualMachineTo, H
Long maxHostMemory = max.first();
Integer maxHostCpuCore = max.second();
- Long minMemory = virtualMachineTo.getMinRam();
+ long minMemory = virtualMachineTo.getMinRam();
Review Comment:
@fermosan ?
##########
ui/public/config.json:
##########
@@ -63,5 +63,121 @@
"basicZoneEnabled": true,
"multipleServer": false,
"allowSettingTheme": true,
- "docHelpMappings": {}
+ "docHelpMappings": {
Review Comment:
@fermosan I could be wrong but I think these are generated/auto-generated during build and need not be added to version history. cc @DaanHoogland @shwstppr
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1296678105
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r991524671
##########
plugins/pom.xml:
##########
@@ -59,6 +59,7 @@
<module>api/solidfire-intg-test</module>
<module>backup/dummy</module>
+ <module>backup/networker</module>
Review Comment:
For starters only KVM. It could be ported for VMWARE (with some modifications for supporting vsphere). If DELL plans to support clients for other we can port it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1257262733
UI build: :heavy_check_mark:
Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6550 (SL-JID-2416)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r979468324
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/NetworkerBackupProvider.java:
##########
@@ -0,0 +1,640 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.backup;
+
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.host.HostVO;
+import com.cloud.host.Status;
+import com.cloud.host.dao.HostDao;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.StoragePoolHostVO;
+import com.cloud.storage.Volume;
+import com.cloud.storage.VolumeVO;
+import com.cloud.storage.dao.StoragePoolHostDao;
+import com.cloud.storage.dao.VolumeDao;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.component.AdapterBase;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallbackNoReturn;
+import com.cloud.utils.db.TransactionStatus;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.ssh.SSHCmdHelper;
+import com.cloud.utils.ssh.SshException;
+import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.dao.VMInstanceDao;
+import org.apache.cloudstack.api.InternalIdentity;
+import org.apache.cloudstack.backup.dao.BackupDao;
+import org.apache.cloudstack.backup.dao.BackupOfferingDaoImpl;
+import org.apache.cloudstack.backup.networker.NetworkerClient;
+import org.apache.cloudstack.framework.config.ConfigKey;
+import org.apache.cloudstack.framework.config.Configurable;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.log4j.Logger;
+import org.apache.xml.utils.URI;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+
+
+import javax.inject.Inject;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.HashMap;
+import java.util.Date;
+import java.util.Objects;
+import java.util.UUID;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1283774959
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1286294905
<b>Trillian test result (tid-5178)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 40179 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6550-t5178-kvm-centos7.zip
Smoke tests completed. 103 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_08_upgrade_kubernetes_ha_cluster | `Failure` | 567.35 | test_kubernetes_clusters.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1204936206
Packaging result: :heavy_multiplication_x: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 3898
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1200971529
@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1262597148
I'll try to complete a round of review
@blueorangutan test centos7 vmware-67u3
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1272659412
<b>Trillian test result (tid-5088)</b>
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 37778 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6550-t5088-xenserver-71.zip
Smoke tests completed. 104 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1272713136
<b>Trillian test result (tid-5089)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 45537 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6550-t5089-kvm-centos7.zip
Smoke tests completed. 103 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers | `Failure` | 464.49 | test_vpc_redundant.py
test_05_rvpc_multi_tiers | `Failure` | 426.33 | test_vpc_redundant.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r920544529
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/ProtectionPolicy.java:
##########
@@ -0,0 +1,222 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+import java.util.List;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "comment",
+ "links",
+ "name",
+ "policyProtectionEnable",
+ "policyProtectionPeriod",
+ "resourceId",
+ "summaryNotification",
+ "workflows"
+})
+@Generated("jsonschema2pojo")
+public class ProtectionPolicy implements Serializable {
+
+ private final static long serialVersionUID = 5407494949453441445L;
+ @JsonProperty("comment")
+ private String comment;
+ @JsonProperty("links")
+ private List<Link> links = null;
+ @JsonProperty("name")
+ private String name;
+ @JsonProperty("policyProtectionEnable")
+ private Boolean policyProtectionEnable;
+ @JsonProperty("policyProtectionPeriod")
+ private String policyProtectionPeriod;
+ @JsonProperty("resourceId")
+ private ResourceId resourceId;
+ @JsonProperty("summaryNotification")
+ private SummaryNotification summaryNotification;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public ProtectionPolicy() {
+ }
+
+ /**
+ * @param policyProtectionEnable
+ * @param policyProtectionPeriod
+ * @param summaryNotification
+ * @param resourceId
+ * @param name
+ * @param comment
+ * @param links
+ */
+ public ProtectionPolicy(String comment, List<Link> links, String name, Boolean policyProtectionEnable, String policyProtectionPeriod, ResourceId resourceId, SummaryNotification summaryNotification) {
+ super();
+ this.comment = comment;
+ this.links = links;
+ this.name = name;
+ this.policyProtectionEnable = policyProtectionEnable;
+ this.policyProtectionPeriod = policyProtectionPeriod;
+ this.resourceId = resourceId;
+ this.summaryNotification = summaryNotification;
+ }
+
+ @JsonProperty("comment")
+ public String getComment() {
+ return comment;
+ }
+
+ @JsonProperty("comment")
+ public void setComment(String comment) {
+ this.comment = comment;
+ }
+
+ @JsonProperty("links")
+ public List<Link> getLinks() {
+ return links;
+ }
+
+ @JsonProperty("links")
+ public void setLinks(List<Link> links) {
+ this.links = links;
+ }
+
+ @JsonProperty("name")
+ public String getName() {
+ return name;
+ }
+
+ @JsonProperty("name")
+ public void setName(String name) {
+ this.name = name;
+ }
+
+ @JsonProperty("policyProtectionEnable")
+ public Boolean getPolicyProtectionEnable() {
+ return policyProtectionEnable;
+ }
+
+ @JsonProperty("policyProtectionEnable")
+ public void setPolicyProtectionEnable(Boolean policyProtectionEnable) {
+ this.policyProtectionEnable = policyProtectionEnable;
+ }
+
+ @JsonProperty("policyProtectionPeriod")
+ public String getPolicyProtectionPeriod() {
+ return policyProtectionPeriod;
+ }
+
+ @JsonProperty("policyProtectionPeriod")
+ public void setPolicyProtectionPeriod(String policyProtectionPeriod) {
+ this.policyProtectionPeriod = policyProtectionPeriod;
+ }
+
+ @JsonProperty("resourceId")
+ public ResourceId getResourceId() {
+ return resourceId;
+ }
+
+ @JsonProperty("resourceId")
+ public void setResourceId(ResourceId resourceId) {
+ this.resourceId = resourceId;
+ }
+
+ @JsonProperty("summaryNotification")
+ public SummaryNotification getSummaryNotification() {
+ return summaryNotification;
+ }
+
+ @JsonProperty("summaryNotification")
+ public void setSummaryNotification(SummaryNotification summaryNotification) {
+ this.summaryNotification = summaryNotification;
+ }
+
+
+ @Override
+ public String toString() {
+ StringBuilder sb = new StringBuilder();
+ sb.append(ProtectionPolicy.class.getName()).append('@').append(Integer.toHexString(System.identityHashCode(this))).append('[');
+ sb.append("comment");
+ sb.append('=');
+ sb.append(((this.comment == null) ? "<null>" : this.comment));
+ sb.append(',');
+ sb.append("links");
+ sb.append('=');
+ sb.append(((this.links == null) ? "<null>" : this.links));
+ sb.append(',');
+ sb.append("name");
+ sb.append('=');
+ sb.append(((this.name == null) ? "<null>" : this.name));
+ sb.append(',');
+ sb.append("policyProtectionEnable");
+ sb.append('=');
+ sb.append(((this.policyProtectionEnable == null) ? "<null>" : this.policyProtectionEnable));
+ sb.append(',');
+ sb.append("policyProtectionPeriod");
+ sb.append('=');
+ sb.append(((this.policyProtectionPeriod == null) ? "<null>" : this.policyProtectionPeriod));
+ sb.append(',');
+ sb.append("resourceId");
+ sb.append('=');
+ sb.append(((this.resourceId == null) ? "<null>" : this.resourceId));
+ sb.append(',');
+ sb.append("summaryNotification");
+ sb.append('=');
+ sb.append(((this.summaryNotification == null) ? "<null>" : this.summaryNotification));
+ sb.append(',');
+ if (sb.charAt((sb.length() - 1)) == ',') {
+ sb.setCharAt((sb.length() - 1), ']');
+ } else {
+ sb.append(']');
+ }
+ return sb.toString();
+ }
+
+ @Override
+ public int hashCode() {
+ int result = 1;
+ result = ((result * 31) + ((this.policyProtectionEnable == null) ? 0 : this.policyProtectionEnable.hashCode()));
+ result = ((result * 31) + ((this.policyProtectionPeriod == null) ? 0 : this.policyProtectionPeriod.hashCode()));
+ result = ((result * 31) + ((this.summaryNotification == null) ? 0 : this.summaryNotification.hashCode()));
+ result = ((result * 31) + ((this.resourceId == null) ? 0 : this.resourceId.hashCode()));
+ result = ((result * 31) + ((this.name == null) ? 0 : this.name.hashCode()));
+ result = ((result * 31) + ((this.comment == null) ? 0 : this.comment.hashCode()));
+ result = ((result * 31) + ((this.links == null) ? 0 : this.links.hashCode()));
+ return result;
+ }
+
+ @Override
+ public boolean equals(Object other) {
+ if (other == this) {
+ return true;
+ }
+ if ((other instanceof ProtectionPolicy) == false) {
+ return false;
+ }
+ ProtectionPolicy rhs = ((ProtectionPolicy) other);
+ return (((((((((this.policyProtectionEnable == rhs.policyProtectionEnable) || ((this.policyProtectionEnable != null) && this.policyProtectionEnable.equals(rhs.policyProtectionEnable))) && ((this.policyProtectionPeriod == rhs.policyProtectionPeriod) || ((this.policyProtectionPeriod != null) && this.policyProtectionPeriod.equals(rhs.policyProtectionPeriod)))) && ((this.summaryNotification == rhs.summaryNotification) || ((this.summaryNotification != null) && this.summaryNotification.equals(rhs.summaryNotification)))) &&
Review Comment:
Will be removed since it is not used anywhere.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r920100509
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/NetworkerBackups.java:
##########
@@ -0,0 +1,118 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+import java.util.List;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "backups",
+ "count"
+})
+@Generated("jsonschema2pojo")
+public class NetworkerBackups implements Serializable {
+
+ private final static long serialVersionUID = -3805021350250865454L;
+ @JsonProperty("backups")
+ private List<Backup> backups = null;
+ @JsonProperty("count")
+ private Integer count;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public NetworkerBackups() {
+ }
+
+ /**
+ * @param count
+ * @param backups
+ */
+ public NetworkerBackups(List<Backup> backups, Integer count) {
+ super();
+ this.backups = backups;
+ this.count = count;
+ }
+
+ @JsonProperty("backups")
+ public List<Backup> getBackups() {
+ return backups;
+ }
+
+ @JsonProperty("backups")
+ public void setBackups(List<Backup> backups) {
+ this.backups = backups;
+ }
+
+ @JsonProperty("count")
+ public Integer getCount() {
+ return count;
+ }
+
+ @JsonProperty("count")
+ public void setCount(Integer count) {
+ this.count = count;
+ }
+
+ @Override
+ public String toString() {
Review Comment:
`ReflectionToStringBuilderUtils`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1181986334
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1182029698
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 3754
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1362698148
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1354727695
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_multiplication_x: debian :heavy_check_mark: suse15. SL-JID 4992
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1363994280
Packaging result: :heavy_check_mark: el7 :heavy_multiplication_x: el8 :heavy_multiplication_x: el9 :heavy_multiplication_x: debian :heavy_multiplication_x: suse15. SL-JID 5079
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1367666166
<b>Trillian test result (tid-5688)</b>
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server r8
Total time taken: 51317 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6550-t5688-vmware-67u3.zip
Smoke tests completed. 105 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_07_arping_in_vr | `Failure` | 5.22 | test_diagnostics.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1348017425
@DaanHoogland a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1349388860
Packaging result: :heavy_check_mark: el7 :heavy_multiplication_x: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 4906
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1375730617
As I saw the error on the jacoco build in another PR on the apache jenkins as well, I am going to merge.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland merged pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland merged PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1257261389
@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1184644090
UI build: :heavy_check_mark:
Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6550 (SL-JID-1961)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1183695295
UI build: :heavy_check_mark:
Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6550 (SL-JID-1952)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1184039710
@DaanHoogland @rohityadavcloud @harikrishna-patnala Is JaCoCo failing for some particular reason ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r920099082
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/Instance.java:
##########
@@ -0,0 +1,158 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+import java.util.List;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "clone",
+ "id",
+ "status",
+ "volumeIds"
+})
+@Generated("jsonschema2pojo")
+public class Instance implements Serializable {
+
+ private final static long serialVersionUID = -5708111855130556342L;
+ @JsonProperty("clone")
+ private Boolean clone;
+ @JsonProperty("id")
+ private String id;
+ @JsonProperty("status")
+ private String status;
+ @JsonProperty("volumeIds")
+ private List<String> volumeIds = null;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public Instance() {
+ }
+
+ /**
+ * @param clone
+ * @param volumeIds
+ * @param id
+ * @param status
+ */
+ public Instance(Boolean clone, String id, String status, List<String> volumeIds) {
+ super();
+ this.clone = clone;
+ this.id = id;
+ this.status = status;
+ this.volumeIds = volumeIds;
+ }
+
+ @JsonProperty("clone")
+ public Boolean getClone() {
+ return clone;
+ }
+
+ @JsonProperty("clone")
+ public void setClone(Boolean clone) {
+ this.clone = clone;
+ }
+
+ @JsonProperty("id")
+ public String getId() {
+ return id;
+ }
+
+ @JsonProperty("id")
+ public void setId(String id) {
+ this.id = id;
+ }
+
+ @JsonProperty("status")
+ public String getStatus() {
+ return status;
+ }
+
+ @JsonProperty("status")
+ public void setStatus(String status) {
+ this.status = status;
+ }
+
+ @JsonProperty("volumeIds")
+ public List<String> getVolumeIds() {
+ return volumeIds;
+ }
+
+ @JsonProperty("volumeIds")
+ public void setVolumeIds(List<String> volumeIds) {
+ this.volumeIds = volumeIds;
+ }
+
+ @Override
+ public String toString() {
+ StringBuilder sb = new StringBuilder();
Review Comment:
please use `ReflectionToStringBuilderUtils`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1183236294
nice work @fermosan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] acs-robot commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
acs-robot commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1188435066
Found UI changes, kicking a new UI QA build
@blueorangutan ui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r951224581
##########
scripts/vm/hypervisor/kvm/nsrkvmrestore.sh:
##########
@@ -0,0 +1,234 @@
+#!/bin/bash
+
+## Licensed to the Apache Software Foundation (ASF) under one
+## or more contributor license agreements. See the NOTICE file
+## distributed with this work for additional information
+## regarding copyright ownership. The ASF licenses this file
+## to you under the Apache License, Version 2.0 (the
+## "License"); you may not use this file except in compliance
+## with the License. You may obtain a copy of the License at
+##
+## http://www.apache.org/licenses/LICENSE-2.0
+##
+## Unless required by applicable law or agreed to in writing,
+## software distributed under the License is distributed on an
+## "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+## KIND, either express or implied. See the License for the
+## specific language governing permissions and limitations
+## under the License.
+
+
+version=0.9.1
+
+
+OPTIND=1
+
+verb=0
+logDir="/nsr/logs/cloudstack/"
+clusterClient=""
+networkerServer=""
+hvVersion=""
+libvVersion=""
+apiVersion=""
+logFile=""
+destination=""
+volume=""
+ssid=""
+newvolumeUuid=""
+POOLPATH=""
+
+log () {
+ [[ "$verb" -eq 1 ]] && builtin echo "$@"
+ if [[ "$1" == "-ne" || "$1" == "-e" || "$1" == "-n" ]]; then
+ builtin echo -e "$(date '+%Y-%m-%d %H-%M-%S>')" "${@: 2}" >> "$logFile"
+ else
+ builtin echo "$(date '+%Y-%m-%d %H-%M-%S>')" "$@" >> "$logFile"
+ fi
+}
+
+vercomp(){
+ local a b IFS=. -; set -f
+ printf -v a %08d $1; printf -v b %08d $3
+ test $a "$2" $b
+}
+
+
+usage() {
+ echo "
+
+Usage:[-v] [-h] [-l log_dir] [-s networker_server] [-c networker_cluster_client] [-S ssid] [-d networker_destination_client ] [ -a volume ] [ -p pool_local_path ]
+
+Options:
+ -h Help and usage
+ -v Enable log mode
+ -l log_dir. Specify log directory. Default is /nsr/logs/cloudstack
+ -s networker_server Specify the EMC Networker server we are going to use
+ -c networker_cluster_client Specify the EMC Networker client CLUSTER to use
+ -d networker_destination_client Specify the EMC Networker client that is going to be used for the restore
+ -a volume Specify volume to restore
+ -p pool_local_path Local Path to Pool
+ -S ssid Specify SSID
+Supplements Apache Cloudstack B&R Framework EMC Networker plugin and performs the backup of the Virtual Machines
+"
+
+}
+
+sanity_checks() {
+ log "Performing environment sanity checks..."
+
+ log -ne "\t[1] Checking if Networker is installed\t"
+ if [[ $(systemctl list-unit-files | grep networker) = *networker* ]]; then
+ log "Success"
+ else
+ log "Failure"
+ log -e "\n\tNetworker Service NOT FOUND. Make sure that Networker client is properly installed"
+ exit 1
+ fi
+
+ log -ne "\t[2] Checking if Networker is running\t"
+ if [[ $(systemctl is-active networker) = *active* ]]; then
+ log "Success"
+ else
+ log "Failure"
+ log -e "\n\tNetworker Service is not running. Investigate Networker logs, startup server and try again"
+ exit 2
+ fi
+ log -ne "\t[3] Checking Networker DNS Resolution\t"
+ if [[ $(getent hosts "$networkerServer") = *$networkerServer* ]]; then
+ log "Success"
+ else
+ log "Failure"
+ log -e "\n\tNetworker Server cannot be resolved. Backups will most probably fail. Consider adding the ip/hostname to /etc/host or fix DNS resolution"
+ exit 3
+ fi
+
+ log -ne "\t[4] Checking QEMU / Libvirt Versions \t"
+ hvVersion=$(virsh version | grep hypervisor | awk '{print $(NF)}')
+ libvVersion=$(virsh version | grep libvirt | awk '{print $(NF)}' | tail -n 1)
+ apiVersion=$(virsh version | grep API | awk '{print $(NF)}')
Review Comment:
same as above - general remark if this can be done via the java based code libvirt-java? Does this run on mgmt server or on kvm host?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r951218626
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/NetworkerClient.java:
##########
@@ -0,0 +1,360 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.cloudstack.backup.networker;
+
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nio.TrustAllManager;
+import com.cloud.vm.VirtualMachine;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.cloudstack.api.ApiErrorCode;
+import org.apache.cloudstack.api.ServerApiException;
+import org.apache.cloudstack.backup.BackupOffering;
+import org.apache.cloudstack.backup.BackupVO;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackups;
+import org.apache.cloudstack.backup.networker.api.ProtectionPolicies;
+import org.apache.cloudstack.backup.networker.api.ProtectionPolicy;
+import org.apache.cloudstack.utils.security.SSLUtils;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpResponse;
+import org.apache.http.HttpStatus;
+import org.apache.http.client.HttpClient;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.HttpDelete;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.conn.ConnectTimeoutException;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.impl.client.HttpClientBuilder;
+import org.apache.log4j.Logger;
+
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.X509TrustManager;
+import java.io.IOException;
+import java.net.SocketTimeoutException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.security.SecureRandom;
+import java.text.ParseException;
+import java.text.SimpleDateFormat;
+import java.time.Instant;
+import java.util.ArrayList;
+import java.util.Base64;
+import java.util.Date;
+import java.util.List;
+
+import static org.apache.cloudstack.backup.NetworkerBackupProvider.BACKUP_IDENTIFIER;
+
+
+public class NetworkerClient {
+
+
+ private static final Logger LOG = Logger.getLogger(NetworkerClient.class);
+
+ private final URI apiURI;
+ private final String apiName;
+ private final String apiPassword;
+
+ private final HttpClient httpClient;
+
+
Review Comment:
minor nit - extra \n
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r951226132
##########
server/src/main/java/com/cloud/hypervisor/KVMGuru.java:
##########
@@ -297,4 +343,64 @@ public Map<String, String> getClusterSettings(long vmId) {
return null;
}
-}
+ @Override
+ public VirtualMachine importVirtualMachineFromBackup(long zoneId, long domainId, long accountId, long userId, String vmInternalName, Backup backup) {
+ s_logger.debug(String.format("Trying to import VM [vmInternalName: %s] from Backup [%s].", vmInternalName,
+ ReflectionToStringBuilderUtils.reflectOnlySelectedFields(backup, "id", "uuid", "vmId", "externalId", "backupType")));
+
+ VMInstanceVO vm = _instanceDao.findVMByInstanceNameIncludingRemoved(vmInternalName);
+
+ if (vm == null) {
+ throw new CloudRuntimeException("Cannot find VM: " + vmInternalName);
+ }
+ try {
+ if (vm.getRemoved() == null) {
+ vm.setState(VirtualMachine.State.Stopped);
+ vm.setPowerState(VirtualMachine.PowerState.PowerOff);
+ _instanceDao.update(vm.getId(), vm);
+ }
+ for ( Backup.VolumeInfo VMVolToRestore : vm.getBackupVolumeList()) {
+ VolumeVO volume = _volumeDao.findByUuidIncludingRemoved(VMVolToRestore.getUuid());
+ volume.setState(Volume.State.Ready);
+ _volumeDao.update(volume.getId(), volume);
+ if (VMVolToRestore.getType() == Volume.Type.ROOT) {
+ _volumeDao.update(volume.getId(), volume);
+ _volumeDao.attachVolume(volume.getId(), vm.getId(), 0L);
+ }
+ else if ( VMVolToRestore.getType() == Volume.Type.DATADISK) {
+ List<VolumeVO> vmVolumes = _volumeDao.findByInstance(vm.getId());
+ _volumeDao.update(volume.getId(), volume);
+ _volumeDao.attachVolume(volume.getId(), vm.getId(), getNextAvailableDeviceId(vmVolumes));
+ }
+ }
+ } catch (Exception e) {
+ throw new RuntimeException("Could not restore VM " + vm.getName() + " due to : " + e.getMessage());
+ }
+ return vm;
+ }
+
+
Review Comment:
minor nit - extra newline
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r951218278
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/NetworkerClient.java:
##########
@@ -0,0 +1,360 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.cloudstack.backup.networker;
+
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nio.TrustAllManager;
+import com.cloud.vm.VirtualMachine;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.cloudstack.api.ApiErrorCode;
+import org.apache.cloudstack.api.ServerApiException;
+import org.apache.cloudstack.backup.BackupOffering;
+import org.apache.cloudstack.backup.BackupVO;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackups;
+import org.apache.cloudstack.backup.networker.api.ProtectionPolicies;
+import org.apache.cloudstack.backup.networker.api.ProtectionPolicy;
+import org.apache.cloudstack.utils.security.SSLUtils;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpResponse;
+import org.apache.http.HttpStatus;
+import org.apache.http.client.HttpClient;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.HttpDelete;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.conn.ConnectTimeoutException;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.impl.client.HttpClientBuilder;
+import org.apache.log4j.Logger;
+
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.X509TrustManager;
+import java.io.IOException;
+import java.net.SocketTimeoutException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.security.SecureRandom;
+import java.text.ParseException;
+import java.text.SimpleDateFormat;
+import java.time.Instant;
+import java.util.ArrayList;
+import java.util.Base64;
+import java.util.Date;
+import java.util.List;
+
+import static org.apache.cloudstack.backup.NetworkerBackupProvider.BACKUP_IDENTIFIER;
Review Comment:
nit - Generally, the static field is exported by an `Interface`
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/NetworkerClient.java:
##########
@@ -0,0 +1,360 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.cloudstack.backup.networker;
+
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nio.TrustAllManager;
+import com.cloud.vm.VirtualMachine;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.cloudstack.api.ApiErrorCode;
+import org.apache.cloudstack.api.ServerApiException;
+import org.apache.cloudstack.backup.BackupOffering;
+import org.apache.cloudstack.backup.BackupVO;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackups;
+import org.apache.cloudstack.backup.networker.api.ProtectionPolicies;
+import org.apache.cloudstack.backup.networker.api.ProtectionPolicy;
+import org.apache.cloudstack.utils.security.SSLUtils;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpResponse;
+import org.apache.http.HttpStatus;
+import org.apache.http.client.HttpClient;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.HttpDelete;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.conn.ConnectTimeoutException;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.impl.client.HttpClientBuilder;
+import org.apache.log4j.Logger;
+
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.X509TrustManager;
+import java.io.IOException;
+import java.net.SocketTimeoutException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.security.SecureRandom;
+import java.text.ParseException;
+import java.text.SimpleDateFormat;
+import java.time.Instant;
+import java.util.ArrayList;
+import java.util.Base64;
+import java.util.Date;
+import java.util.List;
+
+import static org.apache.cloudstack.backup.NetworkerBackupProvider.BACKUP_IDENTIFIER;
+
+
+public class NetworkerClient {
+
Review Comment:
minor nit - extra new line
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1202073025
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1201259078
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 3876
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1257616508
@rohityadavcloud can you have a second look in terms of the B&R framework architecture, please?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r991601148
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/NetworkerBackupProvider.java:
##########
@@ -0,0 +1,637 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.backup;
+
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.host.HostVO;
+import com.cloud.host.Status;
+import com.cloud.host.dao.HostDao;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.StoragePoolHostVO;
+import com.cloud.storage.Volume;
+import com.cloud.storage.VolumeVO;
+import com.cloud.storage.dao.StoragePoolHostDao;
+import com.cloud.storage.dao.VolumeDao;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.component.AdapterBase;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallbackNoReturn;
+import com.cloud.utils.db.TransactionStatus;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.ssh.SSHCmdHelper;
+import com.cloud.utils.ssh.SshException;
+import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.dao.VMInstanceDao;
+import org.apache.cloudstack.api.InternalIdentity;
+import org.apache.cloudstack.backup.dao.BackupDao;
+import org.apache.cloudstack.backup.dao.BackupOfferingDaoImpl;
+import org.apache.cloudstack.backup.networker.NetworkerClient;
+import org.apache.cloudstack.framework.config.ConfigKey;
+import org.apache.cloudstack.framework.config.Configurable;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.log4j.Logger;
+import org.apache.xml.utils.URI;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+import javax.inject.Inject;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.HashMap;
+import java.util.Date;
+import java.util.Objects;
+import java.util.UUID;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+public class NetworkerBackupProvider extends AdapterBase implements BackupProvider, Configurable {
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] acs-robot commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
acs-robot commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1281690542
Found UI changes, kicking a new UI QA build
@blueorangutan ui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1306961509
Script.findScript can work in the Cmd handler on the KVM resource (libvirtcomputingresource) and it depends exactly where/how you want to run it.
> minCpuCores / minMemory
> If anyone has any input about the usability of the supposed usability of those in KVMGuru let me know.
On the first issue ^^, @weizhouapache or @shwstppr can you check and advise?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1272525143
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 4400
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] acs-robot commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
acs-robot commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1199287238
Found UI changes, kicking a new UI QA build
@blueorangutan ui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r967635973
##########
scripts/vm/hypervisor/kvm/nsrkvmbackup.sh:
##########
@@ -0,0 +1,270 @@
+#!/bin/bash
+
+## Licensed to the Apache Software Foundation (ASF) under one
+## or more contributor license agreements. See the NOTICE file
+## distributed with this work for additional information
+## regarding copyright ownership. The ASF licenses this file
+## to you under the Apache License, Version 2.0 (the
+## "License"); you may not use this file except in compliance
+## with the License. You may obtain a copy of the License at
+##
+## http://www.apache.org/licenses/LICENSE-2.0
+##
+## Unless required by applicable law or agreed to in writing,
+## software distributed under the License is distributed on an
+## "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+## KIND, either express or implied. See the License for the
+## specific language governing permissions and limitations
+## under the License.
+
Review Comment:
fixed in latest commit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r967639242
##########
server/src/main/java/com/cloud/hypervisor/KVMGuru.java:
##########
@@ -68,6 +83,37 @@ protected KVMGuru() {
super();
}
+ /**
+ * Get next free DeviceId for a KVM Guest
+ */
+
+ protected Long getNextAvailableDeviceId(List<VolumeVO> vmVolumes) {
+
+ int maxDataVolumesSupported;
+ int maxDeviceId;
+ List<String> devIds = new ArrayList<>();
+
+ try {
+ maxDataVolumesSupported = _hypervisorCapabilitiesDao.getMaxDataVolumesLimit(HypervisorType.KVM,"default");
+ int maxDevices = maxDataVolumesSupported + 2; // add 2 to consider devices root volume and cdrom
Review Comment:
This is actually how /main/server/src/main/java/com/cloud/storage/VolumeApiServiceImpl.java@4070 handles the deviceid discovery. The +2 is KVM specific for anticipating root volume and removable cdrom devices as far as i understand.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1218244762
<b>Trillian Build Failed (tid-4723)<b/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1272244585
@rohityadavcloud a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1363810951
> > > @rohityadavcloud @fermosan do we reset the package-lock.json on this? I think we can merge as soon as we have.
> >
> >
> > re-ping, before this gets rained down by BO messages, @rohityadavcloud @fermosan
>
> @DaanHoogland @rohityadavcloud I thnik we can reset.
ok, @fermosan , can you please do that? I'll rerun a quick package build / test as soon as you have. Hope to merge over x-mas ;)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1363957410
@DaanHoogland a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1363997623
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland closed pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland closed pull request #6550: Emc networker b&r
URL: https://github.com/apache/cloudstack/pull/6550
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1376128636
@DaanHoogland Hello and happy new year. We leave this pull as it is for further communication or i delete branch ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1370940414
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: el9 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 5174
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r920097769
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/Backup.java:
##########
@@ -0,0 +1,423 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+import java.util.List;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "attributes",
+ "browseTime",
+ "clientHostname",
+ "clientId",
+ "completionTime",
+ "creationTime",
+ "fileCount",
+ "id",
+ "instances",
+ "level",
+ "links",
+ "name",
+ "retentionTime",
+ "saveTime",
+ "shortId",
+ "size",
+ "type"
+})
+@Generated("jsonschema2pojo")
+public class Backup implements Serializable {
+
+ private final static long serialVersionUID = -4474500098917286405L;
+ @JsonProperty("attributes")
+ private List<Attribute> attributes = null;
+ @JsonProperty("browseTime")
+ private String browseTime;
+ @JsonProperty("clientHostname")
+ private String clientHostname;
+ @JsonProperty("clientId")
+ private String clientId;
+ @JsonProperty("completionTime")
+ private String completionTime;
+ @JsonProperty("creationTime")
+ private String creationTime;
+ @JsonProperty("fileCount")
+ private Integer fileCount;
+ @JsonProperty("id")
+ private String id;
+ @JsonProperty("instances")
+ private List<Instance> instances = null;
+ @JsonProperty("level")
+ private String level;
+ @JsonProperty("links")
+ private List<Link> links = null;
+ @JsonProperty("name")
+ private String name;
+ @JsonProperty("retentionTime")
+ private String retentionTime;
+ @JsonProperty("saveTime")
+ private String saveTime;
+ @JsonProperty("shortId")
+ private String shortId;
+ @JsonProperty("size")
+ private Size size;
+ @JsonProperty("type")
+ private String type;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public Backup() {
+ }
+
+ /**
+ * @param shortId
+ * @param clientId
+ * @param browseTime
+ * @param creationTime
+ * @param instances
+ * @param level
+ * @param retentionTime
+ * @param type
+ * @param fileCount
+ * @param clientHostname
+ * @param completionTime
+ * @param size
+ * @param name
+ * @param attributes
+ * @param links
+ * @param id
+ * @param saveTime
+ */
+ public Backup(List<Attribute> attributes, String browseTime, String clientHostname, String clientId, String completionTime, String creationTime, Integer fileCount, String id, List<Instance> instances, String level, List<Link> links, String name, String retentionTime, String saveTime, String shortId, Size size, String type) {
+ super();
+ this.attributes = attributes;
+ this.browseTime = browseTime;
+ this.clientHostname = clientHostname;
+ this.clientId = clientId;
+ this.completionTime = completionTime;
+ this.creationTime = creationTime;
+ this.fileCount = fileCount;
+ this.id = id;
+ this.instances = instances;
+ this.level = level;
+ this.links = links;
+ this.name = name;
+ this.retentionTime = retentionTime;
+ this.saveTime = saveTime;
+ this.shortId = shortId;
+ this.size = size;
+ this.type = type;
+ }
+
+ @JsonProperty("attributes")
+ public List<Attribute> getAttributes() {
+ return attributes;
+ }
+
+ @JsonProperty("attributes")
+ public void setAttributes(List<Attribute> attributes) {
+ this.attributes = attributes;
+ }
+
+ @JsonProperty("browseTime")
+ public String getBrowseTime() {
+ return browseTime;
+ }
+
+ @JsonProperty("browseTime")
+ public void setBrowseTime(String browseTime) {
+ this.browseTime = browseTime;
+ }
+
+ @JsonProperty("clientHostname")
+ public String getClientHostname() {
+ return clientHostname;
+ }
+
+ @JsonProperty("clientHostname")
+ public void setClientHostname(String clientHostname) {
+ this.clientHostname = clientHostname;
+ }
+
+ @JsonProperty("clientId")
+ public String getClientId() {
+ return clientId;
+ }
+
+ @JsonProperty("clientId")
+ public void setClientId(String clientId) {
+ this.clientId = clientId;
+ }
+
+ @JsonProperty("completionTime")
+ public String getCompletionTime() {
+ return completionTime;
+ }
+
+ @JsonProperty("completionTime")
+ public void setCompletionTime(String completionTime) {
+ this.completionTime = completionTime;
+ }
+
+ @JsonProperty("creationTime")
+ public String getCreationTime() {
+ return creationTime;
+ }
+
+ @JsonProperty("creationTime")
+ public void setCreationTime(String creationTime) {
+ this.creationTime = creationTime;
+ }
+
+ @JsonProperty("fileCount")
+ public Integer getFileCount() {
+ return fileCount;
+ }
+
+ @JsonProperty("fileCount")
+ public void setFileCount(Integer fileCount) {
+ this.fileCount = fileCount;
+ }
+
+ @JsonProperty("id")
+ public String getId() {
+ return id;
+ }
+
+ @JsonProperty("id")
+ public void setId(String id) {
+ this.id = id;
+ }
+
+ @JsonProperty("instances")
+ public List<Instance> getInstances() {
+ return instances;
+ }
+
+ @JsonProperty("instances")
+ public void setInstances(List<Instance> instances) {
+ this.instances = instances;
+ }
+
+ @JsonProperty("level")
+ public String getLevel() {
+ return level;
+ }
+
+ @JsonProperty("level")
+ public void setLevel(String level) {
+ this.level = level;
+ }
+
+ @JsonProperty("links")
+ public List<Link> getLinks() {
+ return links;
+ }
+
+ @JsonProperty("links")
+ public void setLinks(List<Link> links) {
+ this.links = links;
+ }
+
+ @JsonProperty("name")
+ public String getName() {
+ return name;
+ }
+
+ @JsonProperty("name")
+ public void setName(String name) {
+ this.name = name;
+ }
+
+ @JsonProperty("retentionTime")
+ public String getRetentionTime() {
+ return retentionTime;
+ }
+
+ @JsonProperty("retentionTime")
+ public void setRetentionTime(String retentionTime) {
+ this.retentionTime = retentionTime;
+ }
+
+ @JsonProperty("saveTime")
+ public String getSaveTime() {
+ return saveTime;
+ }
+
+ @JsonProperty("saveTime")
+ public void setSaveTime(String saveTime) {
+ this.saveTime = saveTime;
+ }
+
+ @JsonProperty("shortId")
+ public String getShortId() {
+ return shortId;
+ }
+
+ @JsonProperty("shortId")
+ public void setShortId(String shortId) {
+ this.shortId = shortId;
+ }
+
+ @JsonProperty("size")
+ public Size getSize() {
+ return size;
+ }
+
+ @JsonProperty("size")
+ public void setSize(Size size) {
+ this.size = size;
+ }
+
+ @JsonProperty("type")
+ public String getType() {
+ return type;
+ }
+
+ @JsonProperty("type")
+ public void setType(String type) {
+ this.type = type;
+ }
+
+ @Override
+ public String toString() {
+ StringBuilder sb = new StringBuilder();
+ sb.append(Backup.class.getName()).append('@').append(Integer.toHexString(System.identityHashCode(this))).append('[');
+ sb.append("attributes");
+ sb.append('=');
+ sb.append(((this.attributes == null) ? "<null>" : this.attributes));
+ sb.append(',');
+ sb.append("browseTime");
+ sb.append('=');
+ sb.append(((this.browseTime == null) ? "<null>" : this.browseTime));
+ sb.append(',');
+ sb.append("clientHostname");
+ sb.append('=');
+ sb.append(((this.clientHostname == null) ? "<null>" : this.clientHostname));
+ sb.append(',');
+ sb.append("clientId");
+ sb.append('=');
+ sb.append(((this.clientId == null) ? "<null>" : this.clientId));
+ sb.append(',');
+ sb.append("completionTime");
+ sb.append('=');
+ sb.append(((this.completionTime == null) ? "<null>" : this.completionTime));
+ sb.append(',');
+ sb.append("creationTime");
+ sb.append('=');
+ sb.append(((this.creationTime == null) ? "<null>" : this.creationTime));
+ sb.append(',');
+ sb.append("fileCount");
+ sb.append('=');
+ sb.append(((this.fileCount == null) ? "<null>" : this.fileCount));
+ sb.append(',');
+ sb.append("id");
+ sb.append('=');
+ sb.append(((this.id == null) ? "<null>" : this.id));
+ sb.append(',');
+ sb.append("instances");
+ sb.append('=');
+ sb.append(((this.instances == null) ? "<null>" : this.instances));
+ sb.append(',');
+ sb.append("level");
+ sb.append('=');
+ sb.append(((this.level == null) ? "<null>" : this.level));
+ sb.append(',');
+ sb.append("links");
+ sb.append('=');
+ sb.append(((this.links == null) ? "<null>" : this.links));
+ sb.append(',');
+ sb.append("name");
+ sb.append('=');
+ sb.append(((this.name == null) ? "<null>" : this.name));
+ sb.append(',');
+ sb.append("retentionTime");
+ sb.append('=');
+ sb.append(((this.retentionTime == null) ? "<null>" : this.retentionTime));
+ sb.append(',');
+ sb.append("saveTime");
+ sb.append('=');
+ sb.append(((this.saveTime == null) ? "<null>" : this.saveTime));
+ sb.append(',');
+ sb.append("shortId");
+ sb.append('=');
+ sb.append(((this.shortId == null) ? "<null>" : this.shortId));
+ sb.append(',');
+ sb.append("size");
+ sb.append('=');
+ sb.append(((this.size == null) ? "<null>" : this.size));
+ sb.append(',');
+ sb.append("type");
+ sb.append('=');
+ sb.append(((this.type == null) ? "<null>" : this.type));
+ sb.append(',');
+ if (sb.charAt((sb.length() - 1)) == ',') {
+ sb.setCharAt((sb.length() - 1), ']');
+ } else {
+ sb.append(']');
+ }
+ return sb.toString();
+ }
+
+ @Override
+ public int hashCode() {
+ int result = 1;
+ result = ((result * 31) + ((this.shortId == null) ? 0 : this.shortId.hashCode()));
+ result = ((result * 31) + ((this.clientId == null) ? 0 : this.clientId.hashCode()));
+ result = ((result * 31) + ((this.browseTime == null) ? 0 : this.browseTime.hashCode()));
+ result = ((result * 31) + ((this.creationTime == null) ? 0 : this.creationTime.hashCode()));
+ result = ((result * 31) + ((this.instances == null) ? 0 : this.instances.hashCode()));
+ result = ((result * 31) + ((this.level == null) ? 0 : this.level.hashCode()));
+ result = ((result * 31) + ((this.retentionTime == null) ? 0 : this.retentionTime.hashCode()));
+ result = ((result * 31) + ((this.type == null) ? 0 : this.type.hashCode()));
+ result = ((result * 31) + ((this.fileCount == null) ? 0 : this.fileCount.hashCode()));
+ result = ((result * 31) + ((this.clientHostname == null) ? 0 : this.clientHostname.hashCode()));
+ result = ((result * 31) + ((this.completionTime == null) ? 0 : this.completionTime.hashCode()));
+ result = ((result * 31) + ((this.size == null) ? 0 : this.size.hashCode()));
+ result = ((result * 31) + ((this.name == null) ? 0 : this.name.hashCode()));
+ result = ((result * 31) + ((this.attributes == null) ? 0 : this.attributes.hashCode()));
+ result = ((result * 31) + ((this.links == null) ? 0 : this.links.hashCode()));
+ result = ((result * 31) + ((this.id == null) ? 0 : this.id.hashCode()));
+ result = ((result * 31) + ((this.saveTime == null) ? 0 : this.saveTime.hashCode()));
+ return result;
+ }
+
+ @Override
+ public boolean equals(Object other) {
+ if (other == this) {
+ return true;
+ }
+ if ((other instanceof Backup) == false) {
+ return false;
+ }
+ Backup rhs = ((Backup) other);
+ return ((((((((((((((((((this.shortId == rhs.shortId) || ((this.shortId != null) && this.shortId.equals(rhs.shortId))) && ((this.clientId == rhs.clientId) || ((this.clientId != null) && this.clientId.equals(rhs.clientId)))) && ((this.browseTime == rhs.browseTime) || ((this.browseTime != null) &&
+ this.browseTime.equals(rhs.browseTime)))) && ((this.creationTime == rhs.creationTime) || ((this.creationTime != null) && this.creationTime.equals(rhs.creationTime)))) && ((this.instances == rhs.instances) || ((this.instances != null) && this.instances.equals(rhs.instances)))) && ((this.level == rhs.level) || ((this.level != null) &&
+ this.level.equals(rhs.level)))) && ((this.retentionTime == rhs.retentionTime) || ((this.retentionTime != null) && this.retentionTime.equals(rhs.retentionTime)))) && ((this.type == rhs.type) || ((this.type != null) && this.type.equals(rhs.type)))) && ((this.fileCount == rhs.fileCount) || ((this.fileCount != null) && this.fileCount.equals(rhs.fileCount)))) &&
+ ((this.clientHostname == rhs.clientHostname) || ((this.clientHostname != null) && this.clientHostname.equals(rhs.clientHostname)))) && ((this.completionTime == rhs.completionTime) || ((this.completionTime != null) && this.completionTime.equals(rhs.completionTime)))) && ((this.size == rhs.size) || ((this.size != null) && this.size.equals(rhs.size)))) &&
+ ((this.name == rhs.name) || ((this.name != null) && this.name.equals(rhs.name)))) && ((this.attributes == rhs.attributes) || ((this.attributes != null) && this.attributes.equals(rhs.attributes)))) && ((this.links == rhs.links) || ((this.links != null) && this.links.equals(rhs.links)))) && ((this.id == rhs.id) || ((this.id != null) && this.id.equals(rhs.id)))) &&
+ ((this.saveTime == rhs.saveTime) || ((this.saveTime != null) && this.saveTime.equals(rhs.saveTime))));
Review Comment:
can you extract this and make it more readible, please?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r920787068
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/Backup.java:
##########
@@ -0,0 +1,423 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+import java.util.List;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "attributes",
+ "browseTime",
+ "clientHostname",
+ "clientId",
+ "completionTime",
+ "creationTime",
+ "fileCount",
+ "id",
+ "instances",
+ "level",
+ "links",
+ "name",
+ "retentionTime",
+ "saveTime",
+ "shortId",
+ "size",
+ "type"
+})
+@Generated("jsonschema2pojo")
+public class Backup implements Serializable {
+
+ private final static long serialVersionUID = -4474500098917286405L;
+ @JsonProperty("attributes")
+ private List<Attribute> attributes = null;
+ @JsonProperty("browseTime")
+ private String browseTime;
+ @JsonProperty("clientHostname")
+ private String clientHostname;
+ @JsonProperty("clientId")
+ private String clientId;
+ @JsonProperty("completionTime")
+ private String completionTime;
+ @JsonProperty("creationTime")
+ private String creationTime;
+ @JsonProperty("fileCount")
+ private Integer fileCount;
+ @JsonProperty("id")
+ private String id;
+ @JsonProperty("instances")
+ private List<Instance> instances = null;
+ @JsonProperty("level")
+ private String level;
+ @JsonProperty("links")
+ private List<Link> links = null;
+ @JsonProperty("name")
+ private String name;
+ @JsonProperty("retentionTime")
+ private String retentionTime;
+ @JsonProperty("saveTime")
+ private String saveTime;
+ @JsonProperty("shortId")
+ private String shortId;
+ @JsonProperty("size")
+ private Size size;
+ @JsonProperty("type")
+ private String type;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public Backup() {
+ }
+
+ /**
+ * @param shortId
+ * @param clientId
+ * @param browseTime
+ * @param creationTime
+ * @param instances
+ * @param level
+ * @param retentionTime
+ * @param type
+ * @param fileCount
+ * @param clientHostname
+ * @param completionTime
+ * @param size
+ * @param name
+ * @param attributes
+ * @param links
+ * @param id
+ * @param saveTime
+ */
+ public Backup(List<Attribute> attributes, String browseTime, String clientHostname, String clientId, String completionTime, String creationTime, Integer fileCount, String id, List<Instance> instances, String level, List<Link> links, String name, String retentionTime, String saveTime, String shortId, Size size, String type) {
+ super();
+ this.attributes = attributes;
+ this.browseTime = browseTime;
+ this.clientHostname = clientHostname;
+ this.clientId = clientId;
+ this.completionTime = completionTime;
+ this.creationTime = creationTime;
+ this.fileCount = fileCount;
+ this.id = id;
+ this.instances = instances;
+ this.level = level;
+ this.links = links;
+ this.name = name;
+ this.retentionTime = retentionTime;
+ this.saveTime = saveTime;
+ this.shortId = shortId;
+ this.size = size;
+ this.type = type;
+ }
+
+ @JsonProperty("attributes")
+ public List<Attribute> getAttributes() {
+ return attributes;
+ }
+
+ @JsonProperty("attributes")
+ public void setAttributes(List<Attribute> attributes) {
+ this.attributes = attributes;
+ }
+
+ @JsonProperty("browseTime")
+ public String getBrowseTime() {
+ return browseTime;
+ }
+
+ @JsonProperty("browseTime")
+ public void setBrowseTime(String browseTime) {
+ this.browseTime = browseTime;
+ }
+
+ @JsonProperty("clientHostname")
+ public String getClientHostname() {
+ return clientHostname;
+ }
+
+ @JsonProperty("clientHostname")
+ public void setClientHostname(String clientHostname) {
+ this.clientHostname = clientHostname;
+ }
+
+ @JsonProperty("clientId")
+ public String getClientId() {
+ return clientId;
+ }
+
+ @JsonProperty("clientId")
+ public void setClientId(String clientId) {
+ this.clientId = clientId;
+ }
+
+ @JsonProperty("completionTime")
+ public String getCompletionTime() {
+ return completionTime;
+ }
+
+ @JsonProperty("completionTime")
+ public void setCompletionTime(String completionTime) {
+ this.completionTime = completionTime;
+ }
+
+ @JsonProperty("creationTime")
+ public String getCreationTime() {
+ return creationTime;
+ }
+
+ @JsonProperty("creationTime")
+ public void setCreationTime(String creationTime) {
+ this.creationTime = creationTime;
+ }
+
+ @JsonProperty("fileCount")
+ public Integer getFileCount() {
+ return fileCount;
+ }
+
+ @JsonProperty("fileCount")
+ public void setFileCount(Integer fileCount) {
+ this.fileCount = fileCount;
+ }
+
+ @JsonProperty("id")
+ public String getId() {
+ return id;
+ }
+
+ @JsonProperty("id")
+ public void setId(String id) {
+ this.id = id;
+ }
+
+ @JsonProperty("instances")
+ public List<Instance> getInstances() {
+ return instances;
+ }
+
+ @JsonProperty("instances")
+ public void setInstances(List<Instance> instances) {
+ this.instances = instances;
+ }
+
+ @JsonProperty("level")
+ public String getLevel() {
+ return level;
+ }
+
+ @JsonProperty("level")
+ public void setLevel(String level) {
+ this.level = level;
+ }
+
+ @JsonProperty("links")
+ public List<Link> getLinks() {
+ return links;
+ }
+
+ @JsonProperty("links")
+ public void setLinks(List<Link> links) {
+ this.links = links;
+ }
+
+ @JsonProperty("name")
+ public String getName() {
+ return name;
+ }
+
+ @JsonProperty("name")
+ public void setName(String name) {
+ this.name = name;
+ }
+
+ @JsonProperty("retentionTime")
+ public String getRetentionTime() {
+ return retentionTime;
+ }
+
+ @JsonProperty("retentionTime")
+ public void setRetentionTime(String retentionTime) {
+ this.retentionTime = retentionTime;
+ }
+
+ @JsonProperty("saveTime")
+ public String getSaveTime() {
+ return saveTime;
+ }
+
+ @JsonProperty("saveTime")
+ public void setSaveTime(String saveTime) {
+ this.saveTime = saveTime;
+ }
+
+ @JsonProperty("shortId")
+ public String getShortId() {
+ return shortId;
+ }
+
+ @JsonProperty("shortId")
+ public void setShortId(String shortId) {
+ this.shortId = shortId;
+ }
+
+ @JsonProperty("size")
+ public Size getSize() {
+ return size;
+ }
+
+ @JsonProperty("size")
+ public void setSize(Size size) {
+ this.size = size;
+ }
+
+ @JsonProperty("type")
+ public String getType() {
+ return type;
+ }
+
+ @JsonProperty("type")
+ public void setType(String type) {
+ this.type = type;
+ }
+
+ @Override
+ public String toString() {
Review Comment:
Transitioned.
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/Instance.java:
##########
@@ -0,0 +1,158 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+import java.util.List;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "clone",
+ "id",
+ "status",
+ "volumeIds"
+})
+@Generated("jsonschema2pojo")
+public class Instance implements Serializable {
+
+ private final static long serialVersionUID = -5708111855130556342L;
+ @JsonProperty("clone")
+ private Boolean clone;
+ @JsonProperty("id")
+ private String id;
+ @JsonProperty("status")
+ private String status;
+ @JsonProperty("volumeIds")
+ private List<String> volumeIds = null;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public Instance() {
+ }
+
+ /**
+ * @param clone
+ * @param volumeIds
+ * @param id
+ * @param status
+ */
+ public Instance(Boolean clone, String id, String status, List<String> volumeIds) {
+ super();
+ this.clone = clone;
+ this.id = id;
+ this.status = status;
+ this.volumeIds = volumeIds;
+ }
+
+ @JsonProperty("clone")
+ public Boolean getClone() {
+ return clone;
+ }
+
+ @JsonProperty("clone")
+ public void setClone(Boolean clone) {
+ this.clone = clone;
+ }
+
+ @JsonProperty("id")
+ public String getId() {
+ return id;
+ }
+
+ @JsonProperty("id")
+ public void setId(String id) {
+ this.id = id;
+ }
+
+ @JsonProperty("status")
+ public String getStatus() {
+ return status;
+ }
+
+ @JsonProperty("status")
+ public void setStatus(String status) {
+ this.status = status;
+ }
+
+ @JsonProperty("volumeIds")
+ public List<String> getVolumeIds() {
+ return volumeIds;
+ }
+
+ @JsonProperty("volumeIds")
+ public void setVolumeIds(List<String> volumeIds) {
+ this.volumeIds = volumeIds;
+ }
+
+ @Override
+ public String toString() {
+ StringBuilder sb = new StringBuilder();
Review Comment:
Transitioned.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1184038345
@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r920107827
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/ServerBackup.java:
##########
@@ -0,0 +1,178 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "performBootstrap",
+ "performClientFileIndexing",
+ "destinationStorageNode",
+ "retentionPeriod",
+ "destinationPool"
+})
+@Generated("jsonschema2pojo")
+public class ServerBackup implements Serializable {
+
+ private final static long serialVersionUID = -542556595701706880L;
+ @JsonProperty("performBootstrap")
+ private Boolean performBootstrap;
+ @JsonProperty("performClientFileIndexing")
+ private Boolean performClientFileIndexing;
+ @JsonProperty("destinationStorageNode")
+ private String destinationStorageNode;
+ @JsonProperty("retentionPeriod")
+ private String retentionPeriod;
+ @JsonProperty("destinationPool")
+ private String destinationPool;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public ServerBackup() {
+ }
+
+ /**
+ * @param destinationPool
+ * @param performBootstrap
+ * @param performClientFileIndexing
+ * @param destinationStorageNode
+ * @param retentionPeriod
+ */
+ public ServerBackup(Boolean performBootstrap, Boolean performClientFileIndexing, String destinationStorageNode, String retentionPeriod, String destinationPool) {
+ super();
+ this.performBootstrap = performBootstrap;
+ this.performClientFileIndexing = performClientFileIndexing;
+ this.destinationStorageNode = destinationStorageNode;
+ this.retentionPeriod = retentionPeriod;
+ this.destinationPool = destinationPool;
+ }
+
+ @JsonProperty("performBootstrap")
+ public Boolean getPerformBootstrap() {
+ return performBootstrap;
+ }
+
+ @JsonProperty("performBootstrap")
+ public void setPerformBootstrap(Boolean performBootstrap) {
+ this.performBootstrap = performBootstrap;
+ }
+
+ @JsonProperty("performClientFileIndexing")
+ public Boolean getPerformClientFileIndexing() {
+ return performClientFileIndexing;
+ }
+
+ @JsonProperty("performClientFileIndexing")
+ public void setPerformClientFileIndexing(Boolean performClientFileIndexing) {
+ this.performClientFileIndexing = performClientFileIndexing;
+ }
+
+ @JsonProperty("destinationStorageNode")
+ public String getDestinationStorageNode() {
+ return destinationStorageNode;
+ }
+
+ @JsonProperty("destinationStorageNode")
+ public void setDestinationStorageNode(String destinationStorageNode) {
+ this.destinationStorageNode = destinationStorageNode;
+ }
+
+ @JsonProperty("retentionPeriod")
+ public String getRetentionPeriod() {
+ return retentionPeriod;
+ }
+
+ @JsonProperty("retentionPeriod")
+ public void setRetentionPeriod(String retentionPeriod) {
+ this.retentionPeriod = retentionPeriod;
+ }
+
+ @JsonProperty("destinationPool")
+ public String getDestinationPool() {
+ return destinationPool;
+ }
+
+ @JsonProperty("destinationPool")
+ public void setDestinationPool(String destinationPool) {
+ this.destinationPool = destinationPool;
+ }
+
+ @Override
+ public String toString() {
+ StringBuilder sb = new StringBuilder();
+ sb.append(ServerBackup.class.getName()).append('@').append(Integer.toHexString(System.identityHashCode(this))).append('[');
+ sb.append("performBootstrap");
+ sb.append('=');
+ sb.append(((this.performBootstrap == null) ? "<null>" : this.performBootstrap));
+ sb.append(',');
+ sb.append("performClientFileIndexing");
+ sb.append('=');
+ sb.append(((this.performClientFileIndexing == null) ? "<null>" : this.performClientFileIndexing));
+ sb.append(',');
+ sb.append("destinationStorageNode");
+ sb.append('=');
+ sb.append(((this.destinationStorageNode == null) ? "<null>" : this.destinationStorageNode));
+ sb.append(',');
+ sb.append("retentionPeriod");
+ sb.append('=');
+ sb.append(((this.retentionPeriod == null) ? "<null>" : this.retentionPeriod));
+ sb.append(',');
+ sb.append("destinationPool");
+ sb.append('=');
+ sb.append(((this.destinationPool == null) ? "<null>" : this.destinationPool));
+ sb.append(',');
+ if (sb.charAt((sb.length() - 1)) == ',') {
+ sb.setCharAt((sb.length() - 1), ']');
+ } else {
+ sb.append(']');
+ }
+ return sb.toString();
+ }
+
+ @Override
+ public int hashCode() {
+ int result = 1;
+ result = ((result * 31) + ((this.performBootstrap == null) ? 0 : this.performBootstrap.hashCode()));
+ result = ((result * 31) + ((this.performClientFileIndexing == null) ? 0 : this.performClientFileIndexing.hashCode()));
+ result = ((result * 31) + ((this.destinationPool == null) ? 0 : this.destinationPool.hashCode()));
+ result = ((result * 31) + ((this.destinationStorageNode == null) ? 0 : this.destinationStorageNode.hashCode()));
+ result = ((result * 31) + ((this.retentionPeriod == null) ? 0 : this.retentionPeriod.hashCode()));
+ return result;
+ }
+
+ @Override
+ public boolean equals(Object other) {
+ if (other == this) {
+ return true;
+ }
+ if ((other instanceof ServerBackup) == false) {
+ return false;
+ }
+ ServerBackup rhs = ((ServerBackup) other);
+ return ((((((this.performBootstrap == rhs.performBootstrap) || ((this.performBootstrap != null) && this.performBootstrap.equals(rhs.performBootstrap))) && ((this.performClientFileIndexing == rhs.performClientFileIndexing) || ((this.performClientFileIndexing != null) && this.performClientFileIndexing.equals(rhs.performClientFileIndexing)))) && ((this.destinationPool == rhs.destinationPool) || ((this.destinationPool != null) && this.destinationPool.equals(rhs.destinationPool)))) && ((this.destinationStorageNode == rhs.destinationStorageNode) || ((this.destinationStorageNode != null) && this.destinationStorageNode.equals(rhs.destinationStorageNode)))) && ((this.retentionPeriod == rhs.retentionPeriod) || ((this.retentionPeriod != null) && this.retentionPeriod.equals(rhs.retentionPeriod))));
Review Comment:
please simplify
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1184635185
@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1188440225
UI build: :heavy_check_mark:
Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6550 (SL-JID-1976)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] slavkap commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
slavkap commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r926301169
##########
server/src/main/java/com/cloud/hypervisor/KVMGuru.java:
##########
@@ -297,4 +309,49 @@ public Map<String, String> getClusterSettings(long vmId) {
return null;
}
-}
+ @Override
+ public VirtualMachine importVirtualMachineFromBackup(long zoneId, long domainId, long accountId, long userId, String vmInternalName, Backup backup) throws Exception {
+ s_logger.debug(String.format("Trying to import VM [vmInternalName: %s] from Backup [%s].", vmInternalName,
+ ReflectionToStringBuilderUtils.reflectOnlySelectedFields(backup, "id", "uuid", "vmId", "externalId", "backupType")));
+
+ VMInstanceVO vm = _instanceDao.findVMByInstanceNameIncludingRemoved(vmInternalName);
+ if (vm.getRemoved() != null) {
+ vm.setState(VirtualMachine.State.Stopped);
+ vm.setPowerState(VirtualMachine.PowerState.PowerOff);
+ _instanceDao.update(vm.getId(), vm);
+ _instanceDao.unremove(vm.getId());
+ }
+ for (final VolumeVO volume : _volumeDao.findIncludingRemovedByInstanceAndType(vm.getId(), null)) {
+ volume.setState(Volume.State.Ready);
+ volume.setAttached(new Date());
+ _volumeDao.update(volume.getId(), volume);
+ _volumeDao.unremove(volume.getId());
+ }
+
+ return vm;
+ }
+
+
+
+ @Override public boolean attachRestoredVolumeToVirtualMachine(long zoneId, String location, Backup.VolumeInfo volumeInfo, VirtualMachine vm, long poolId, Backup backup)
+ throws Exception {
+
+ VMInstanceVO targetVM = _instanceDao.findVMByInstanceNameIncludingRemoved(vm.getName());
+ List<VolumeVO> devices = _volumeDao.findIncludingRemovedByInstanceAndType(targetVM.getId(), null);
+ VolumeVO restoredVolume = _volumeDao.findByUuid(location);
+ Integer deviceId = devices.size();
+
+
+ if (restoredVolume != null) {
+ restoredVolume.setState(Volume.State.Ready);
+ _volumeDao.update(restoredVolume.getId(), restoredVolume);
+ try {
+ _volumeDao.attachVolume(restoredVolume.getId(), vm.getId(), deviceId);
Review Comment:
Thanks, @fermosan, this should work! You can extract the repeated code from `importVirtualMachineFromBackup` and `attachRestoredVolumeToVirtualMachine` into a method `getDeviceId` or something
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1196777976
@rohityadavcloud @DaanHoogland A quick update. Our tests doing great here. Everything works as expected. We will keep testing for a few more weeks. When is 4.17.1.0 scheduled to be released and approximately when 4.18.0 ? Maybe we can catch the deadline for 4.17.1.0 unless you don't want to introduce new features in a minor version for some reason.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1203971173
@rohityadavcloud @harikrishna-patnala @slavkap CLGTM i don´t think we have anybody other than @fermosan and colleagues to test this. Can you agree we can merge?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r951214352
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/NetworkerBackupProvider.java:
##########
@@ -0,0 +1,640 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.backup;
+
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.host.HostVO;
+import com.cloud.host.Status;
+import com.cloud.host.dao.HostDao;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.StoragePoolHostVO;
+import com.cloud.storage.Volume;
+import com.cloud.storage.VolumeVO;
+import com.cloud.storage.dao.StoragePoolHostDao;
+import com.cloud.storage.dao.VolumeDao;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.component.AdapterBase;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallbackNoReturn;
+import com.cloud.utils.db.TransactionStatus;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.ssh.SSHCmdHelper;
+import com.cloud.utils.ssh.SshException;
+import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.dao.VMInstanceDao;
+import org.apache.cloudstack.api.InternalIdentity;
+import org.apache.cloudstack.backup.dao.BackupDao;
+import org.apache.cloudstack.backup.dao.BackupOfferingDaoImpl;
+import org.apache.cloudstack.backup.networker.NetworkerClient;
+import org.apache.cloudstack.framework.config.ConfigKey;
+import org.apache.cloudstack.framework.config.Configurable;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.log4j.Logger;
+import org.apache.xml.utils.URI;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+
+
+import javax.inject.Inject;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.HashMap;
+import java.util.Date;
+import java.util.Objects;
+import java.util.UUID;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+
+public class NetworkerBackupProvider extends AdapterBase implements BackupProvider, Configurable {
+
+ public static final String BACKUP_IDENTIFIER = "-CSBKP-";
+ private static final Logger LOG = Logger.getLogger(NetworkerBackupProvider.class);
+ public ConfigKey<String> NetworkerUrl = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.url", "https://localhost:9090/nwrestapi/v3",
+ "The EMC Networker API URL.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerUsername = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.username", "administrator",
+ "The EMC Networker API username.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerPassword = new ConfigKey<>("Secure", String.class,
+ "backup.plugin.networker.password", "password",
+ "The EMC Networker API password.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerMediaPool = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.pool", "Default",
+ "The EMC Networker Media Pool", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerValidateSSLSecurity = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.validate.ssl", "false",
+ "Validate the SSL certificate when connecting to EMC Networker API service.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Integer> NetworkerApiRequestTimeout = new ConfigKey<>("Advanced", Integer.class,
+ "backup.plugin.networker.request.timeout", "300",
+ "The EMC Networker API request timeout in seconds.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerClientVerboseLogs = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.client.verbosity", "false",
+ "Produce Verbose logs in Hypervisor", true, ConfigKey.Scope.Zone);
+
+ @Inject
+ private BackupDao backupDao;
+
+ @Inject
+ private HostDao hostDao;
+
+ @Inject
+ private ConfigurationDao configDao;
+
+ @Inject
+ private ClusterDao clusterDao;
+
+ @Inject
+ private VolumeDao volumeDao;
+
+ @Inject
+ private StoragePoolHostDao storagePoolHostDao;
+
+ @Inject
+ private VMInstanceDao vmInstanceDao;
+
+ private static String getUrlDomain(String url) throws URISyntaxException {
+ URI uri;
+ try {
+ uri = new URI(url);
+ } catch (URI.MalformedURIException e) {
+ throw new CloudRuntimeException("Failed to cast URI");
+ }
+
+ return uri.getHost();
+ }
+
+ @Override
+ public ConfigKey<?>[] getConfigKeys() {
+ return new ConfigKey[]{
+ NetworkerUrl,
+ NetworkerUsername,
+ NetworkerPassword,
+ NetworkerValidateSSLSecurity,
+ NetworkerApiRequestTimeout,
+ NetworkerMediaPool,
+ NetworkerClientVerboseLogs
+ };
+ }
+
+ @Override
+ public String getName() {
+ return "networker";
+ }
+
+ @Override
+ public String getDescription() {
+ return "EMC Networker Backup Plugin";
+ }
+
+ @Override
+ public String getConfigComponentName() {
+ return BackupService.class.getSimpleName();
+ }
+
+ protected HostVO getLastVMHypervisorHost(VirtualMachine vm) {
+ HostVO host;
+ Long hostId = vm.getLastHostId();
+
+ if (hostId == null) {
+ LOG.debug("Cannot find last host for vm. This should never happen, please check your database.");
+ return null;
+ }
+ host = hostDao.findById(hostId);
+
+ if ( host.getStatus() == Status.Up ) {
+ return host;
+ } else {
+ // Try to find a host in the same cluster
+ List<HostVO> altClusterHosts = hostDao.findHypervisorHostInCluster(host.getClusterId());
+ for (final HostVO candidateClusterHost : altClusterHosts) {
+ if ( candidateClusterHost.getStatus() == Status.Up ) {
+ LOG.debug("Found Host " + candidateClusterHost.getName());
+ return candidateClusterHost;
+ }
+ }
+ }
+ // Try to find a Host in the zone
+ List<HostVO> altZoneHosts = hostDao.findByDataCenterId(host.getDataCenterId());
+ for (final HostVO candidateZoneHost : altZoneHosts) {
+ if ( candidateZoneHost.getStatus() == Status.Up && candidateZoneHost.getHypervisorType() == Hypervisor.HypervisorType.KVM ) {
+ LOG.debug("Found Host " + candidateZoneHost.getName());
+ return candidateZoneHost;
+ }
+ }
+ return null;
+ }
+
+ protected HostVO getRunningVMHypervisorHost(VirtualMachine vm) {
+
+ HostVO host;
+ Long hostId = vm.getHostId();
+
+ if (hostId == null) {
+ throw new CloudRuntimeException("Unable to find the HYPERVISOR for " + vm.getName() + ". Make sure the virtual machine is running");
+ }
+
+ host = hostDao.findById(hostId);
+
+ return host;
+ }
+
+ protected String getVMHypervisorCluster(HostVO host) {
+
+ return clusterDao.findById(host.getClusterId()).getName();
+ }
+
+ protected Ternary<String, String, String> getKVMHyperisorCredentials(HostVO host) {
+
+ String username = null;
+ String password = null;
+ String privateKey = null;
+
+ if (host != null && host.getHypervisorType() == Hypervisor.HypervisorType.KVM) {
+ hostDao.loadDetails(host);
+ password = host.getDetail("password");
+ username = host.getDetail("username");
+ privateKey = configDao.getValue("ssh.privatekey");
+ }
+ if ((password == null && privateKey == null) || username == null) {
+ throw new CloudRuntimeException("Cannot find login credentials for HYPERVISOR " + Objects.requireNonNull(host).getUuid());
+ }
+
+ return new Ternary<>(username, password, privateKey);
+ }
+
+ private String executeBackupCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+ String nstRegex = "\\bcompleted savetime=([0-9]{10})";
+ Pattern saveTimePattern = Pattern.compile(nstRegex);
+
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ Matcher saveTimeMatcher = saveTimePattern.matcher(result.getStdOut());
+ if (saveTimeMatcher.find()) {
+ return saveTimeMatcher.group(1);
+ }
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+
+ return null;
+ }
+ private boolean executeRestoreCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ return true;
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+ }
+
+ private NetworkerClient getClient(final Long zoneId) {
+ try {
+ return new NetworkerClient(NetworkerUrl.valueIn(zoneId), NetworkerUsername.valueIn(zoneId), NetworkerPassword.valueIn(zoneId),
+ NetworkerValidateSSLSecurity.valueIn(zoneId), NetworkerApiRequestTimeout.valueIn(zoneId));
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException("Failed to parse EMC Networker API URL: " + e.getMessage());
+ } catch (NoSuchAlgorithmException | KeyManagementException e) {
+ LOG.error("Failed to build EMC Networker API client due to: ", e);
+ }
+ throw new CloudRuntimeException("Failed to build EMC Networker API client");
+ }
+
+ @Override
+ public List<BackupOffering> listBackupOfferings(Long zoneId) {
+ List<BackupOffering> policies = new ArrayList<>();
+ for (final BackupOffering policy : getClient(zoneId).listPolicies()) {
+ if (!policy.getName().contains(BACKUP_IDENTIFIER)) {
+ policies.add(policy);
+ }
+ }
+
+ return policies;
+ }
+
+ @Override
+ public boolean isValidProviderOffering(Long zoneId, String uuid) {
+ List<BackupOffering> policies = listBackupOfferings(zoneId);
+ if (CollectionUtils.isEmpty(policies)) {
+ return false;
+ }
+ for (final BackupOffering policy : policies) {
+ if (Objects.equals(policy.getExternalId(), uuid)) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ @Override
+ public boolean assignVMToBackupOffering(VirtualMachine vm, BackupOffering backupOffering) { return true; }
+
+ @Override
+ public boolean removeVMFromBackupOffering(VirtualMachine vm) {
+ LOG.debug("Removing VirtualMachine from Backup offering and Deleting any existing backups");
+
+ List<String> backupsTaken = getClient(vm.getDataCenterId()).getBackupsForVm(vm);
+
+ for (String backupId : backupsTaken) {
+ LOG.debug("Trying to remove backup with id" + backupId);
+ getClient(vm.getDataCenterId()).deleteBackupForVM(backupId);
+ }
+
+ return true;
+ }
+
+ @Override
+ public boolean restoreVMFromBackup(VirtualMachine vm, Backup backup) {
+ LOG.debug("Restoring vm " + vm.getUuid() + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+ String networkerServer;
+ HostVO hostVO;
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return false;
+ }
+
+ // Find where the VM was last running
+ hostVO = getLastVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -S " + SSID;
+
+ if (NetworkerClientVerboseLogs.value())
+ command = command + " -v ";
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for VM ID " + vm.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return true;
+ } else {
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return false;
+ }
+ }
+
+ @Override
+ public Pair<Boolean, String> restoreBackedUpVolume(Backup backup, String volumeUuid, String hostIp, String dataStoreUuid) {
+ LOG.debug("Restoring volume " + volumeUuid + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+
+ String networkerServer;
+ VolumeVO volume = volumeDao.findByUuid(volumeUuid);
+ VMInstanceVO backupSourceVm = vmInstanceDao.findById(backup.getVmId());
+ StoragePoolHostVO dataStore = storagePoolHostDao.findByUuid(dataStoreUuid);
+ HostVO hostVO = hostDao.findByIp(hostIp);
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+ final String clusterName = networkerBackup.getClientHostname();
+ final String destinationNetworkerClient = hostVO.getName().split("\\.")[0];
+ Long restoredVolumeDiskSize = 0L;
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return null;
+ }
+
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ // Find volume size from backup vols
+ for ( Backup.VolumeInfo VMVolToRestore : backupSourceVm.getBackupVolumeList()) {
+ if (VMVolToRestore.getUuid().equals(volumeUuid))
+ restoredVolumeDiskSize = (VMVolToRestore.getSize());
+ }
+
+ VolumeVO restoredVolume = new VolumeVO(Volume.Type.DATADISK, null, backup.getZoneId(),
+ backup.getDomainId(), backup.getAccountId(), 0, null,
+ backup.getSize(), null, null, null);
+
+ restoredVolume.setName("RV-"+volume.getName());
+ restoredVolume.setProvisioningType(volume.getProvisioningType());
+ restoredVolume.setUpdated(new Date());
+ restoredVolume.setUuid(UUID.randomUUID().toString());
+ restoredVolume.setRemoved(null);
+ restoredVolume.setDisplayVolume(true);
+ restoredVolume.setPoolId(volume.getPoolId());
+ restoredVolume.setPath(restoredVolume.getUuid());
+ restoredVolume.setState(Volume.State.Copying);
+ restoredVolume.setSize(restoredVolumeDiskSize);
+ restoredVolume.setDiskOfferingId(volume.getDiskOfferingId());
+
+ try {
+ volumeDao.persist(restoredVolume);
+ } catch (Exception e) {
+ throw new CloudRuntimeException("Unable to craft restored volume due to: "+e);
+ }
+
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -c " + clusterName +
+ " -d " + destinationNetworkerClient +
+ " -n " + restoredVolume.getUuid() +
Review Comment:
needs thinking if Script can be used, otherwise seems reasonable but without any parameter validation/parsing, simply contatening them to a 'sudo cmd' could become a potential attack surface
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r951210720
##########
plugins/backup/networker/pom.xml:
##########
@@ -0,0 +1,56 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <artifactId>cloud-plugin-backup-networker</artifactId>
+ <name>Apache CloudStack Plugin - EMC Networker Backup and Recovery Plugin</name>
+ <parent>
+ <artifactId>cloudstack-plugins</artifactId>
+ <groupId>org.apache.cloudstack</groupId>
+ <version>4.18.0.0-SNAPSHOT</version>
+ <relativePath>../../pom.xml</relativePath>
+ </parent>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.cloudstack</groupId>
+ <artifactId>cloud-plugin-hypervisor-kvm</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.commons</groupId>
+ <artifactId>commons-lang3</artifactId>
+ <version>${cs.commons-lang3.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>com.fasterxml.jackson.core</groupId>
+ <artifactId>jackson-databind</artifactId>
+ <version>2.13.0</version>
Review Comment:
nit - move this to root pom.xml as dependency version property
##########
plugins/backup/networker/pom.xml:
##########
@@ -0,0 +1,56 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <artifactId>cloud-plugin-backup-networker</artifactId>
+ <name>Apache CloudStack Plugin - EMC Networker Backup and Recovery Plugin</name>
+ <parent>
+ <artifactId>cloudstack-plugins</artifactId>
+ <groupId>org.apache.cloudstack</groupId>
+ <version>4.18.0.0-SNAPSHOT</version>
+ <relativePath>../../pom.xml</relativePath>
+ </parent>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.cloudstack</groupId>
+ <artifactId>cloud-plugin-hypervisor-kvm</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.commons</groupId>
+ <artifactId>commons-lang3</artifactId>
+ <version>${cs.commons-lang3.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>com.fasterxml.jackson.core</groupId>
+ <artifactId>jackson-databind</artifactId>
+ <version>2.13.0</version>
+ </dependency>
+ <dependency>
+ <groupId>com.github.tomakehurst</groupId>
+ <artifactId>wiremock-standalone</artifactId>
+ <version>2.27.2</version>
Review Comment:
nit - move this to root pom.xml as dependency version property (see if it already exists?)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1228295528
UI build: :heavy_check_mark:
Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6550 (SL-JID-2232)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r955955139
##########
scripts/vm/hypervisor/kvm/nsrkvmbackup.sh:
##########
@@ -0,0 +1,270 @@
+#!/bin/bash
+
+## Licensed to the Apache Software Foundation (ASF) under one
+## or more contributor license agreements. See the NOTICE file
+## distributed with this work for additional information
+## regarding copyright ownership. The ASF licenses this file
+## to you under the Apache License, Version 2.0 (the
+## "License"); you may not use this file except in compliance
+## with the License. You may obtain a copy of the License at
+##
+## http://www.apache.org/licenses/LICENSE-2.0
+##
+## Unless required by applicable law or agreed to in writing,
+## software distributed under the License is distributed on an
+## "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+## KIND, either express or implied. See the License for the
+## specific language governing permissions and limitations
+## under the License.
+
+
+version=0.9.1
+
+
+OPTIND=1
+
+verb=0
+logDir="/nsr/logs/cloudstack/"
+snapPrefix="CSBKP_$RANDOM_"
+clusterClient=""
+networkerServer=""
+hvVersion=""
+libvVersion=""
+apiVersion=""
+kvmDName=""
+kvmDUuid=""
+logFile=""
+mediaPool=""
+retentionTime=""
+
+
+log () {
+
+ [[ "$verb" -eq 1 ]] && builtin echo "$@"
+ if [[ "$1" == "-ne" || "$1" == "-e" || "$1" == "-n" ]]; then
+ builtin echo -e "$(date '+%Y-%m-%d %H-%M-%S>')" "${@: 2}" >> "$logFile"
+ else
+ builtin echo "$(date '+%Y-%m-%d %H-%M-%S>')" "$@" >> "$logFile"
+ fi
+
+}
+
+vercomp(){
+ local a b IFS=. -; set -f
+ printf -v a %08d $1; printf -v b %08d $3
+ test $a "$2" $b
+}
+usage() {
+ echo "
+
+Usage:[-v] [-h] [-l log_dir] [-dr] [-s networker_server] [-c networker_cluster_client] [-t target_vm] [-u target_uuid] [-p snapprefix] [-P media_pool ] [-R retention_time ]
+
+Options:
+ -h Help and usage
+ -v Enable verbose mode
+ -l log_dir. Specify log directory. Default is /nsr/logs/cloudstack
+ -s networker_server Specifiy the EMC Networker server we are going to use
+ -c networker_cluster_client Specify the EMC Networker client CLUSTER to use
+ -t target_vm KVM domain to backup
+ -u target_uuid KVM domain to backup
+ -p Snapshot Prefix for backups
+ -P mediaPool EMC Networker Media Pool
+ -R retention_time Backup retention time
+
+Supplements Apache Cloudstack B&R Framework EMC Networker plugin and performs the backup of the Virtual Machines
+"
+
+}
+
+sanity_checks() {
+
+ log "Performing environment sanity checks..."
+ log -ne "\t[1] Checking if Networker is installed\t"
+ if [[ $(systemctl list-unit-files | grep networker) = *networker* ]]; then
+ log "Success"
+ else
+ log "Failure"
+ log -e "\n\tNetworker Service NOT FOUND. Make sure that Networker client is properly installed"
+ exit 1
+ fi
+
+ log -ne "\t[2] Checking if Networker is running\t"
+ if [[ $(systemctl is-active networker) = *active* ]]; then
+ log "Success"
+ else
+ log "Failure"
+ log -e "\n\tNetworker Service is not running. Investigate Networker logs, startup server and try again"
+ exit 2
+ fi
+ log -ne "\t[3] Checking Networker DNS Resolution\t"
+ if [[ $(getent hosts "$networkerServer") = *$networkerServer* ]]; then
+ log "Success"
+ else
+ log "Failure"
+ log -e "\n\tNetworker Server cannot be resolved. Backups will most probably fail. Consider adding the ip/hostname to /etc/host or fix DNS resolution"
+ exit 3
+ fi
+
+ log -ne "\t[4] Checking QEMU / Libvirt Versions \t"
+ hvVersion=$(virsh version | grep hypervisor | awk '{print $(NF)}')
+ libvVersion=$(virsh version | grep libvirt | awk '{print $(NF)}' | tail -n 1)
+ apiVersion=$(virsh version | grep API | awk '{print $(NF)}')
+ if vercomp "$hvVersion" \> 2.1.2; then
+ log -n "Success"
+ log -ne "\t\t [ Libvirt: $libvVersion apiVersion: $apiVersion ]"
+ echo
+
+ else
+ log "Failure"
+ log -e "\n\tYour QEMU version $hvVersion is unsupported. Consider upgrading at least to latest QEMU at branch 2"
+ exit 4
+ fi
+
+
+
+ log "Environment Sanity Checks successfully passed"
+}
+
+
+
+echo "
+Cloudstack B&R Framework - EMC Networker backup script
+Version $version
+"
+
+
+backup_domain() {
+
+ name=$1
+ snapName=$2
+ log "Preparing snapshots and gathering information for backing up domain $name under snapshot name $snapName"
+ log "Retention time is $retentionTime"
+
+ declare -A TRGSRC
+ while IFS=',' read -r TARGET SOURCE
+ do
+ if [[ $SOURCE != "-" ]]; then
+ TRGSRC+=(["$TARGET"]="$SOURCE")
+ fi
+ done < <(virsh domblklist "$name" --details | grep file | grep -v 'cdrom' | grep -v 'floppy' | sed 's/ */,/g' | cut -d',' -f 4-)
+ diskspec=""
+ for target in "${!TRGSRC[@]}"; do
+ log -e "\tDisk for $target is at ${TRGSRC[${target}]}"
+ diskspec="$diskspec --diskspec $target,snapshot=external"
+ disks="$disks ${TRGSRC[${target}]} "
+ done
+
+ cmd="$(virsh snapshot-create-as --domain "$name" --name "$snapName" --no-metadata --atomic --quiesce --disk-only "$diskspec")"
Review Comment:
Since it is running on the kvm agent side i am not sure we can do.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1207846820
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1207849866
> @DaanHoogland @slavkap Thank you for your review and approvals. Of course we are going to maintain it since it will be actively used. It will be moved to 4.17.1.0 milestone or stay for 4.18 ?
new features never go in dot releases, so 4.18 I'd say. Also it is targetted to main so the next release made from that branch ;)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1216474341
@rohityadavcloud a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1357489804
@DaanHoogland a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1363956836
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1363970935
@blueorangutan test matrix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1281692213
@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1272540157
@rohityadavcloud a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1272539945
@blueorangutan test matrix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r979840440
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/NetworkerBackupProvider.java:
##########
@@ -0,0 +1,640 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.backup;
+
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.host.HostVO;
+import com.cloud.host.Status;
+import com.cloud.host.dao.HostDao;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.StoragePoolHostVO;
+import com.cloud.storage.Volume;
+import com.cloud.storage.VolumeVO;
+import com.cloud.storage.dao.StoragePoolHostDao;
+import com.cloud.storage.dao.VolumeDao;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.component.AdapterBase;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallbackNoReturn;
+import com.cloud.utils.db.TransactionStatus;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.ssh.SSHCmdHelper;
+import com.cloud.utils.ssh.SshException;
+import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.dao.VMInstanceDao;
+import org.apache.cloudstack.api.InternalIdentity;
+import org.apache.cloudstack.backup.dao.BackupDao;
+import org.apache.cloudstack.backup.dao.BackupOfferingDaoImpl;
+import org.apache.cloudstack.backup.networker.NetworkerClient;
+import org.apache.cloudstack.framework.config.ConfigKey;
+import org.apache.cloudstack.framework.config.Configurable;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.log4j.Logger;
+import org.apache.xml.utils.URI;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+
+
+import javax.inject.Inject;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.HashMap;
+import java.util.Date;
+import java.util.Objects;
+import java.util.UUID;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+
+public class NetworkerBackupProvider extends AdapterBase implements BackupProvider, Configurable {
+
+ public static final String BACKUP_IDENTIFIER = "-CSBKP-";
+ private static final Logger LOG = Logger.getLogger(NetworkerBackupProvider.class);
+ public ConfigKey<String> NetworkerUrl = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.url", "https://localhost:9090/nwrestapi/v3",
+ "The EMC Networker API URL.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerUsername = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.username", "administrator",
+ "The EMC Networker API username.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerPassword = new ConfigKey<>("Secure", String.class,
+ "backup.plugin.networker.password", "password",
+ "The EMC Networker API password.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerMediaPool = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.pool", "Default",
+ "The EMC Networker Media Pool", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerValidateSSLSecurity = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.validate.ssl", "false",
+ "Validate the SSL certificate when connecting to EMC Networker API service.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Integer> NetworkerApiRequestTimeout = new ConfigKey<>("Advanced", Integer.class,
+ "backup.plugin.networker.request.timeout", "300",
+ "The EMC Networker API request timeout in seconds.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerClientVerboseLogs = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.client.verbosity", "false",
+ "Produce Verbose logs in Hypervisor", true, ConfigKey.Scope.Zone);
+
+ @Inject
+ private BackupDao backupDao;
+
+ @Inject
+ private HostDao hostDao;
+
+ @Inject
+ private ConfigurationDao configDao;
+
+ @Inject
+ private ClusterDao clusterDao;
+
+ @Inject
+ private VolumeDao volumeDao;
+
+ @Inject
+ private StoragePoolHostDao storagePoolHostDao;
+
+ @Inject
+ private VMInstanceDao vmInstanceDao;
+
+ private static String getUrlDomain(String url) throws URISyntaxException {
+ URI uri;
+ try {
+ uri = new URI(url);
+ } catch (URI.MalformedURIException e) {
+ throw new CloudRuntimeException("Failed to cast URI");
+ }
+
+ return uri.getHost();
+ }
+
+ @Override
+ public ConfigKey<?>[] getConfigKeys() {
+ return new ConfigKey[]{
+ NetworkerUrl,
+ NetworkerUsername,
+ NetworkerPassword,
+ NetworkerValidateSSLSecurity,
+ NetworkerApiRequestTimeout,
+ NetworkerMediaPool,
+ NetworkerClientVerboseLogs
+ };
+ }
+
+ @Override
+ public String getName() {
+ return "networker";
+ }
+
+ @Override
+ public String getDescription() {
+ return "EMC Networker Backup Plugin";
+ }
+
+ @Override
+ public String getConfigComponentName() {
+ return BackupService.class.getSimpleName();
+ }
+
+ protected HostVO getLastVMHypervisorHost(VirtualMachine vm) {
+ HostVO host;
+ Long hostId = vm.getLastHostId();
+
+ if (hostId == null) {
+ LOG.debug("Cannot find last host for vm. This should never happen, please check your database.");
+ return null;
+ }
+ host = hostDao.findById(hostId);
+
+ if ( host.getStatus() == Status.Up ) {
+ return host;
+ } else {
+ // Try to find a host in the same cluster
+ List<HostVO> altClusterHosts = hostDao.findHypervisorHostInCluster(host.getClusterId());
+ for (final HostVO candidateClusterHost : altClusterHosts) {
+ if ( candidateClusterHost.getStatus() == Status.Up ) {
+ LOG.debug("Found Host " + candidateClusterHost.getName());
+ return candidateClusterHost;
+ }
+ }
+ }
+ // Try to find a Host in the zone
+ List<HostVO> altZoneHosts = hostDao.findByDataCenterId(host.getDataCenterId());
+ for (final HostVO candidateZoneHost : altZoneHosts) {
+ if ( candidateZoneHost.getStatus() == Status.Up && candidateZoneHost.getHypervisorType() == Hypervisor.HypervisorType.KVM ) {
+ LOG.debug("Found Host " + candidateZoneHost.getName());
+ return candidateZoneHost;
+ }
+ }
+ return null;
+ }
+
+ protected HostVO getRunningVMHypervisorHost(VirtualMachine vm) {
+
+ HostVO host;
+ Long hostId = vm.getHostId();
+
+ if (hostId == null) {
+ throw new CloudRuntimeException("Unable to find the HYPERVISOR for " + vm.getName() + ". Make sure the virtual machine is running");
+ }
+
+ host = hostDao.findById(hostId);
+
+ return host;
+ }
+
+ protected String getVMHypervisorCluster(HostVO host) {
+
+ return clusterDao.findById(host.getClusterId()).getName();
+ }
+
+ protected Ternary<String, String, String> getKVMHyperisorCredentials(HostVO host) {
+
+ String username = null;
+ String password = null;
+ String privateKey = null;
+
+ if (host != null && host.getHypervisorType() == Hypervisor.HypervisorType.KVM) {
+ hostDao.loadDetails(host);
+ password = host.getDetail("password");
+ username = host.getDetail("username");
+ privateKey = configDao.getValue("ssh.privatekey");
+ }
+ if ((password == null && privateKey == null) || username == null) {
+ throw new CloudRuntimeException("Cannot find login credentials for HYPERVISOR " + Objects.requireNonNull(host).getUuid());
+ }
+
+ return new Ternary<>(username, password, privateKey);
+ }
+
+ private String executeBackupCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+ String nstRegex = "\\bcompleted savetime=([0-9]{10})";
+ Pattern saveTimePattern = Pattern.compile(nstRegex);
+
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ Matcher saveTimeMatcher = saveTimePattern.matcher(result.getStdOut());
+ if (saveTimeMatcher.find()) {
+ return saveTimeMatcher.group(1);
+ }
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+
+ return null;
+ }
+ private boolean executeRestoreCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ return true;
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+ }
+
+ private NetworkerClient getClient(final Long zoneId) {
+ try {
+ return new NetworkerClient(NetworkerUrl.valueIn(zoneId), NetworkerUsername.valueIn(zoneId), NetworkerPassword.valueIn(zoneId),
+ NetworkerValidateSSLSecurity.valueIn(zoneId), NetworkerApiRequestTimeout.valueIn(zoneId));
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException("Failed to parse EMC Networker API URL: " + e.getMessage());
+ } catch (NoSuchAlgorithmException | KeyManagementException e) {
+ LOG.error("Failed to build EMC Networker API client due to: ", e);
+ }
+ throw new CloudRuntimeException("Failed to build EMC Networker API client");
+ }
+
+ @Override
+ public List<BackupOffering> listBackupOfferings(Long zoneId) {
+ List<BackupOffering> policies = new ArrayList<>();
+ for (final BackupOffering policy : getClient(zoneId).listPolicies()) {
+ if (!policy.getName().contains(BACKUP_IDENTIFIER)) {
+ policies.add(policy);
+ }
+ }
+
+ return policies;
+ }
+
+ @Override
+ public boolean isValidProviderOffering(Long zoneId, String uuid) {
+ List<BackupOffering> policies = listBackupOfferings(zoneId);
+ if (CollectionUtils.isEmpty(policies)) {
+ return false;
+ }
+ for (final BackupOffering policy : policies) {
+ if (Objects.equals(policy.getExternalId(), uuid)) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ @Override
+ public boolean assignVMToBackupOffering(VirtualMachine vm, BackupOffering backupOffering) { return true; }
+
+ @Override
+ public boolean removeVMFromBackupOffering(VirtualMachine vm) {
+ LOG.debug("Removing VirtualMachine from Backup offering and Deleting any existing backups");
+
+ List<String> backupsTaken = getClient(vm.getDataCenterId()).getBackupsForVm(vm);
+
+ for (String backupId : backupsTaken) {
+ LOG.debug("Trying to remove backup with id" + backupId);
+ getClient(vm.getDataCenterId()).deleteBackupForVM(backupId);
+ }
+
+ return true;
+ }
+
+ @Override
+ public boolean restoreVMFromBackup(VirtualMachine vm, Backup backup) {
+ LOG.debug("Restoring vm " + vm.getUuid() + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+ String networkerServer;
+ HostVO hostVO;
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return false;
+ }
+
+ // Find where the VM was last running
+ hostVO = getLastVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -S " + SSID;
+
+ if (NetworkerClientVerboseLogs.value())
+ command = command + " -v ";
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for VM ID " + vm.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return true;
+ } else {
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return false;
+ }
+ }
+
+ @Override
+ public Pair<Boolean, String> restoreBackedUpVolume(Backup backup, String volumeUuid, String hostIp, String dataStoreUuid) {
+ LOG.debug("Restoring volume " + volumeUuid + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+
+ String networkerServer;
+ VolumeVO volume = volumeDao.findByUuid(volumeUuid);
+ VMInstanceVO backupSourceVm = vmInstanceDao.findById(backup.getVmId());
+ StoragePoolHostVO dataStore = storagePoolHostDao.findByUuid(dataStoreUuid);
+ HostVO hostVO = hostDao.findByIp(hostIp);
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+ final String clusterName = networkerBackup.getClientHostname();
+ final String destinationNetworkerClient = hostVO.getName().split("\\.")[0];
+ Long restoredVolumeDiskSize = 0L;
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return null;
+ }
+
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ // Find volume size from backup vols
+ for ( Backup.VolumeInfo VMVolToRestore : backupSourceVm.getBackupVolumeList()) {
+ if (VMVolToRestore.getUuid().equals(volumeUuid))
+ restoredVolumeDiskSize = (VMVolToRestore.getSize());
+ }
+
+ VolumeVO restoredVolume = new VolumeVO(Volume.Type.DATADISK, null, backup.getZoneId(),
+ backup.getDomainId(), backup.getAccountId(), 0, null,
+ backup.getSize(), null, null, null);
+
+ restoredVolume.setName("RV-"+volume.getName());
+ restoredVolume.setProvisioningType(volume.getProvisioningType());
+ restoredVolume.setUpdated(new Date());
+ restoredVolume.setUuid(UUID.randomUUID().toString());
+ restoredVolume.setRemoved(null);
+ restoredVolume.setDisplayVolume(true);
+ restoredVolume.setPoolId(volume.getPoolId());
+ restoredVolume.setPath(restoredVolume.getUuid());
+ restoredVolume.setState(Volume.State.Copying);
+ restoredVolume.setSize(restoredVolumeDiskSize);
+ restoredVolume.setDiskOfferingId(volume.getDiskOfferingId());
+
+ try {
+ volumeDao.persist(restoredVolume);
+ } catch (Exception e) {
+ throw new CloudRuntimeException("Unable to craft restored volume due to: "+e);
+ }
+
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -c " + clusterName +
+ " -d " + destinationNetworkerClient +
+ " -n " + restoredVolume.getUuid() +
+ " -p " + dataStore.getLocalPath() +
+ " -a " + volume.getUuid();
+
+ if (NetworkerClientVerboseLogs.value())
+ command = command + " -v ";
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for Volume UUID " + volume.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return new Pair<>(true,restoredVolume.getUuid());
+ } else {
+ volumeDao.expunge(restoredVolume.getId());
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return null;
+ }
+ }
+
+ @Override
+ public boolean takeBackup(VirtualMachine vm) {
+
+ String networkerServer;
+ String clusterName;
+
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ // Find where the VM is currently running
+ HostVO hostVO = getRunningVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ // Get retention Period for our Backup
+ BackupOfferingVO vmBackupOffering = new BackupOfferingDaoImpl().findById(vm.getBackupOfferingId());
+ final String backupProviderPolicyId = vmBackupOffering.getExternalId();
+ String backupRentionPeriod = getClient(vm.getDataCenterId()).getBackupPolicyRetentionInterval(backupProviderPolicyId);
+
+ if ( backupRentionPeriod == null ) {
+ LOG.warn("There is no retention setting for Emc Networker Policy, setting default for 1 day");
+ backupRentionPeriod="1 Day";
+ }
+
+ // Get Cluster
+ clusterName = getVMHypervisorCluster(hostVO);
+
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmbackup.sh" +
+ " -s " + networkerServer +
+ " -R '" + backupRentionPeriod + "'" +
+ " -P " + NetworkerMediaPool.valueIn(vm.getDataCenterId()) +
+ " -c " + clusterName +
+ " -u " + vm.getUuid() +
+ " -t " + vm.getName();
+ if (NetworkerClientVerboseLogs.value())
+ command = command + " -v ";
+
+ LOG.debug("Starting backup for VM ID " + vm.getUuid() + " on Networker provider");
+ Date backupJobStart = new Date();
+
+ String saveTime = executeBackupCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command);
+ BackupVO backup = getClient(vm.getDataCenterId()).registerBackupForVm(vm, backupJobStart, saveTime);
+ if (backup != null) {
+ backupDao.persist(backup);
+ return true;
+ } else {
+ // We need to handle this rare situation where backup is successful but can't be registered properly.
+ return false;
+ }
+ }
+
+ @Override
+ public boolean deleteBackup(Backup backup) {
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+
+ if (getClient(zoneId).deleteBackupForVM(externalBackupId)) {
+ LOG.debug("EMC Networker successfully deleted backup with id " + externalBackupId);
+ return true;
+ } else {
+ LOG.debug("There was an error removing the backup with id " + externalBackupId + " from EMC NEtworker");
+ }
+ return false;
+ }
+
+ @Override
+ public Map<VirtualMachine, Backup.Metric> getBackupMetrics(Long zoneId, List<VirtualMachine> vms) {
+ final Map<VirtualMachine, Backup.Metric> metrics = new HashMap<>();
+ Long vmBackupSize=0L;
+ Long vmBackupProtectedSize=0L;
+
+ if (CollectionUtils.isEmpty(vms)) {
+ LOG.warn("Unable to get VM Backup Metrics because the list of VMs is empty.");
+ return metrics;
+ }
+
+ for (final VirtualMachine vm : vms) {
+ for ( Backup.VolumeInfo thisVMVol : vm.getBackupVolumeList()) {
+ vmBackupSize += (thisVMVol.getSize() / 1024L / 1024L);
Review Comment:
1024L is supposed to do that. vmBackupSize is already Long.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1257684949
Packaging result: :heavy_multiplication_x: el7 :heavy_check_mark: el8 :heavy_multiplication_x: debian :heavy_multiplication_x: suse15. SL-JID 4274
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r979468728
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/NetworkerBackupOffering.java:
##########
@@ -0,0 +1,82 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.cloudstack.backup.networker;
+
+
+import org.apache.cloudstack.backup.BackupOffering;
+
+import java.util.Date;
+
+
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1228305292
We'll have to do without sonar output on this one I guess. It wasn't triggered, even after re-opening.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r955953437
##########
scripts/vm/hypervisor/kvm/nsrkvmbackup.sh:
##########
@@ -0,0 +1,270 @@
+#!/bin/bash
+
+## Licensed to the Apache Software Foundation (ASF) under one
+## or more contributor license agreements. See the NOTICE file
+## distributed with this work for additional information
+## regarding copyright ownership. The ASF licenses this file
+## to you under the Apache License, Version 2.0 (the
+## "License"); you may not use this file except in compliance
+## with the License. You may obtain a copy of the License at
+##
+## http://www.apache.org/licenses/LICENSE-2.0
+##
+## Unless required by applicable law or agreed to in writing,
+## software distributed under the License is distributed on an
+## "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+## KIND, either express or implied. See the License for the
+## specific language governing permissions and limitations
+## under the License.
+
+
+version=0.9.1
+
+
+OPTIND=1
+
+verb=0
+logDir="/nsr/logs/cloudstack/"
+snapPrefix="CSBKP_$RANDOM_"
+clusterClient=""
+networkerServer=""
+hvVersion=""
+libvVersion=""
+apiVersion=""
+kvmDName=""
+kvmDUuid=""
+logFile=""
+mediaPool=""
+retentionTime=""
+
+
+log () {
+
+ [[ "$verb" -eq 1 ]] && builtin echo "$@"
+ if [[ "$1" == "-ne" || "$1" == "-e" || "$1" == "-n" ]]; then
+ builtin echo -e "$(date '+%Y-%m-%d %H-%M-%S>')" "${@: 2}" >> "$logFile"
+ else
+ builtin echo "$(date '+%Y-%m-%d %H-%M-%S>')" "$@" >> "$logFile"
+ fi
+
+}
+
+vercomp(){
+ local a b IFS=. -; set -f
+ printf -v a %08d $1; printf -v b %08d $3
+ test $a "$2" $b
+}
+usage() {
+ echo "
+
+Usage:[-v] [-h] [-l log_dir] [-dr] [-s networker_server] [-c networker_cluster_client] [-t target_vm] [-u target_uuid] [-p snapprefix] [-P media_pool ] [-R retention_time ]
+
+Options:
+ -h Help and usage
+ -v Enable verbose mode
+ -l log_dir. Specify log directory. Default is /nsr/logs/cloudstack
+ -s networker_server Specifiy the EMC Networker server we are going to use
+ -c networker_cluster_client Specify the EMC Networker client CLUSTER to use
+ -t target_vm KVM domain to backup
+ -u target_uuid KVM domain to backup
+ -p Snapshot Prefix for backups
+ -P mediaPool EMC Networker Media Pool
+ -R retention_time Backup retention time
+
+Supplements Apache Cloudstack B&R Framework EMC Networker plugin and performs the backup of the Virtual Machines
+"
+
+}
+
+sanity_checks() {
+
+ log "Performing environment sanity checks..."
+ log -ne "\t[1] Checking if Networker is installed\t"
+ if [[ $(systemctl list-unit-files | grep networker) = *networker* ]]; then
+ log "Success"
+ else
+ log "Failure"
+ log -e "\n\tNetworker Service NOT FOUND. Make sure that Networker client is properly installed"
+ exit 1
+ fi
+
+ log -ne "\t[2] Checking if Networker is running\t"
+ if [[ $(systemctl is-active networker) = *active* ]]; then
+ log "Success"
+ else
+ log "Failure"
+ log -e "\n\tNetworker Service is not running. Investigate Networker logs, startup server and try again"
+ exit 2
+ fi
+ log -ne "\t[3] Checking Networker DNS Resolution\t"
+ if [[ $(getent hosts "$networkerServer") = *$networkerServer* ]]; then
+ log "Success"
+ else
+ log "Failure"
+ log -e "\n\tNetworker Server cannot be resolved. Backups will most probably fail. Consider adding the ip/hostname to /etc/host or fix DNS resolution"
+ exit 3
+ fi
+
+ log -ne "\t[4] Checking QEMU / Libvirt Versions \t"
+ hvVersion=$(virsh version | grep hypervisor | awk '{print $(NF)}')
Review Comment:
It is running on KVM Agents.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r955954029
##########
scripts/vm/hypervisor/kvm/nsrkvmbackup.sh:
##########
@@ -0,0 +1,270 @@
+#!/bin/bash
+
+## Licensed to the Apache Software Foundation (ASF) under one
+## or more contributor license agreements. See the NOTICE file
+## distributed with this work for additional information
+## regarding copyright ownership. The ASF licenses this file
+## to you under the Apache License, Version 2.0 (the
+## "License"); you may not use this file except in compliance
+## with the License. You may obtain a copy of the License at
+##
+## http://www.apache.org/licenses/LICENSE-2.0
+##
+## Unless required by applicable law or agreed to in writing,
+## software distributed under the License is distributed on an
+## "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+## KIND, either express or implied. See the License for the
+## specific language governing permissions and limitations
+## under the License.
+
+
+version=0.9.1
+
+
+OPTIND=1
+
+verb=0
+logDir="/nsr/logs/cloudstack/"
Review Comment:
Since this is basically talking with EMC Networker client administrators expecting to find logs at the default /nsr/logs directory.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1261630405
<b>Trillian test result (tid-5038)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 40606 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6550-t5038-kvm-centos7.zip
Smoke tests completed. 102 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_08_upgrade_kubernetes_ha_cluster | `Failure` | 567.99 | test_kubernetes_clusters.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1299986299
ping @fermosan you see any chance to address @rohityadavcloud 's remaining comments?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1216473478
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1219981643
<b>Trillian test result (tid-4724)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 43604 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6550-t4724-kvm-centos7.zip
Smoke tests completed. 100 look OK, 1 have errors
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_08_upgrade_kubernetes_ha_cluster | `Failure` | 577.23 | test_kubernetes_clusters.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] acs-robot commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
acs-robot commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1200970503
Found UI changes, kicking a new UI QA build
@blueorangutan ui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r927021781
##########
server/src/main/java/com/cloud/hypervisor/KVMGuru.java:
##########
@@ -297,4 +309,49 @@ public Map<String, String> getClusterSettings(long vmId) {
return null;
}
-}
+ @Override
+ public VirtualMachine importVirtualMachineFromBackup(long zoneId, long domainId, long accountId, long userId, String vmInternalName, Backup backup) throws Exception {
+ s_logger.debug(String.format("Trying to import VM [vmInternalName: %s] from Backup [%s].", vmInternalName,
+ ReflectionToStringBuilderUtils.reflectOnlySelectedFields(backup, "id", "uuid", "vmId", "externalId", "backupType")));
+
+ VMInstanceVO vm = _instanceDao.findVMByInstanceNameIncludingRemoved(vmInternalName);
+ if (vm.getRemoved() != null) {
+ vm.setState(VirtualMachine.State.Stopped);
+ vm.setPowerState(VirtualMachine.PowerState.PowerOff);
+ _instanceDao.update(vm.getId(), vm);
+ _instanceDao.unremove(vm.getId());
+ }
+ for (final VolumeVO volume : _volumeDao.findIncludingRemovedByInstanceAndType(vm.getId(), null)) {
+ volume.setState(Volume.State.Ready);
+ volume.setAttached(new Date());
+ _volumeDao.update(volume.getId(), volume);
+ _volumeDao.unremove(volume.getId());
+ }
+
+ return vm;
+ }
+
+
+
+ @Override public boolean attachRestoredVolumeToVirtualMachine(long zoneId, String location, Backup.VolumeInfo volumeInfo, VirtualMachine vm, long poolId, Backup backup)
+ throws Exception {
+
+ VMInstanceVO targetVM = _instanceDao.findVMByInstanceNameIncludingRemoved(vm.getName());
+ List<VolumeVO> devices = _volumeDao.findIncludingRemovedByInstanceAndType(targetVM.getId(), null);
+ VolumeVO restoredVolume = _volumeDao.findByUuid(location);
+ Integer deviceId = devices.size();
+
+
+ if (restoredVolume != null) {
+ restoredVolume.setState(Volume.State.Ready);
+ _volumeDao.update(restoredVolume.getId(), restoredVolume);
+ try {
+ _volumeDao.attachVolume(restoredVolume.getId(), vm.getId(), deviceId);
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r951216563
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/NetworkerBackupOffering.java:
##########
@@ -0,0 +1,82 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.cloudstack.backup.networker;
+
+
Review Comment:
nit - extra new line
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/NetworkerBackupOffering.java:
##########
@@ -0,0 +1,82 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.cloudstack.backup.networker;
+
+
+import org.apache.cloudstack.backup.BackupOffering;
+
+import java.util.Date;
+
+
Review Comment:
nit - extra new line
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1206258639
@slavkap are your concerns met?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r920543580
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/Action.java:
##########
@@ -0,0 +1,444 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+import java.util.List;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "actionReferSchedule",
+ "actionSpecificData",
+ "comment",
+ "completionNotification",
+ "concurrent",
+ "drivenBy",
+ "enabled",
+ "failureImpact",
+ "hardLimit",
+ "inactivityTimeoutInMin",
+ "name",
+ "parallelism",
+ "retries",
+ "retryDelayInSec",
+ "softLimit",
+ "scheduleActivities",
+ "scheduleOverrides",
+ "schedulePeriod"
+})
+@Generated("jsonschema2pojo")
+public class Action implements Serializable {
+
+ private final static long serialVersionUID = 1750989315434884936L;
+ @JsonProperty("actionReferSchedule")
+ private String actionReferSchedule;
+ @JsonProperty("actionSpecificData")
+ private ActionSpecificData actionSpecificData;
+ @JsonProperty("comment")
+ private String comment;
+ @JsonProperty("completionNotification")
+ private CompletionNotification completionNotification;
+ @JsonProperty("concurrent")
+ private Boolean concurrent;
+ @JsonProperty("drivenBy")
+ private String drivenBy;
+ @JsonProperty("enabled")
+ private Boolean enabled;
+ @JsonProperty("failureImpact")
+ private String failureImpact;
+ @JsonProperty("hardLimit")
+ private String hardLimit;
+ @JsonProperty("inactivityTimeoutInMin")
+ private Integer inactivityTimeoutInMin;
+ @JsonProperty("name")
+ private String name;
+ @JsonProperty("parallelism")
+ private Integer parallelism;
+ @JsonProperty("retries")
+ private Integer retries;
+ @JsonProperty("retryDelayInSec")
+ private Integer retryDelayInSec;
+ @JsonProperty("softLimit")
+ private String softLimit;
+ @JsonProperty("scheduleActivities")
+ private List<String> scheduleActivities = null;
+ @JsonProperty("scheduleOverrides")
+ private List<Object> scheduleOverrides = null;
+ @JsonProperty("schedulePeriod")
+ private String schedulePeriod;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public Action() {
+ }
+
+ /**
+ * @param failureImpact
+ * @param actionSpecificData
+ * @param completionNotification
+ * @param parallelism
+ * @param concurrent
+ * @param retryDelayInSec
+ * @param drivenBy
+ * @param enabled
+ * @param scheduleActivities
+ * @param retries
+ * @param actionReferSchedule
+ * @param name
+ * @param inactivityTimeoutInMin
+ * @param comment
+ * @param hardLimit
+ * @param scheduleOverrides
+ * @param schedulePeriod
+ * @param softLimit
+ */
+ public Action(String actionReferSchedule, ActionSpecificData actionSpecificData, String comment, CompletionNotification completionNotification, Boolean concurrent, String drivenBy, Boolean enabled, String failureImpact, String hardLimit, Integer inactivityTimeoutInMin, String name, Integer parallelism, Integer retries, Integer retryDelayInSec, String softLimit, List<String> scheduleActivities, List<Object> scheduleOverrides, String schedulePeriod) {
+ super();
+ this.actionReferSchedule = actionReferSchedule;
+ this.actionSpecificData = actionSpecificData;
+ this.comment = comment;
+ this.completionNotification = completionNotification;
+ this.concurrent = concurrent;
+ this.drivenBy = drivenBy;
+ this.enabled = enabled;
+ this.failureImpact = failureImpact;
+ this.hardLimit = hardLimit;
+ this.inactivityTimeoutInMin = inactivityTimeoutInMin;
+ this.name = name;
+ this.parallelism = parallelism;
+ this.retries = retries;
+ this.retryDelayInSec = retryDelayInSec;
+ this.softLimit = softLimit;
+ this.scheduleActivities = scheduleActivities;
+ this.scheduleOverrides = scheduleOverrides;
+ this.schedulePeriod = schedulePeriod;
+ }
+
+ @JsonProperty("actionReferSchedule")
+ public String getActionReferSchedule() {
+ return actionReferSchedule;
+ }
+
+ @JsonProperty("actionReferSchedule")
+ public void setActionReferSchedule(String actionReferSchedule) {
+ this.actionReferSchedule = actionReferSchedule;
+ }
+
+ @JsonProperty("actionSpecificData")
+ public ActionSpecificData getActionSpecificData() {
+ return actionSpecificData;
+ }
+
+ @JsonProperty("actionSpecificData")
+ public void setActionSpecificData(ActionSpecificData actionSpecificData) {
+ this.actionSpecificData = actionSpecificData;
+ }
+
+ @JsonProperty("comment")
+ public String getComment() {
+ return comment;
+ }
+
+ @JsonProperty("comment")
+ public void setComment(String comment) {
+ this.comment = comment;
+ }
+
+ @JsonProperty("completionNotification")
+ public CompletionNotification getCompletionNotification() {
+ return completionNotification;
+ }
+
+ @JsonProperty("completionNotification")
+ public void setCompletionNotification(CompletionNotification completionNotification) {
+ this.completionNotification = completionNotification;
+ }
+
+ @JsonProperty("concurrent")
+ public Boolean getConcurrent() {
+ return concurrent;
+ }
+
+ @JsonProperty("concurrent")
+ public void setConcurrent(Boolean concurrent) {
+ this.concurrent = concurrent;
+ }
+
+ @JsonProperty("drivenBy")
+ public String getDrivenBy() {
+ return drivenBy;
+ }
+
+ @JsonProperty("drivenBy")
+ public void setDrivenBy(String drivenBy) {
+ this.drivenBy = drivenBy;
+ }
+
+ @JsonProperty("enabled")
+ public Boolean getEnabled() {
+ return enabled;
+ }
+
+ @JsonProperty("enabled")
+ public void setEnabled(Boolean enabled) {
+ this.enabled = enabled;
+ }
+
+ @JsonProperty("failureImpact")
+ public String getFailureImpact() {
+ return failureImpact;
+ }
+
+ @JsonProperty("failureImpact")
+ public void setFailureImpact(String failureImpact) {
+ this.failureImpact = failureImpact;
+ }
+
+ @JsonProperty("hardLimit")
+ public String getHardLimit() {
+ return hardLimit;
+ }
+
+ @JsonProperty("hardLimit")
+ public void setHardLimit(String hardLimit) {
+ this.hardLimit = hardLimit;
+ }
+
+ @JsonProperty("inactivityTimeoutInMin")
+ public Integer getInactivityTimeoutInMin() {
+ return inactivityTimeoutInMin;
+ }
+
+ @JsonProperty("inactivityTimeoutInMin")
+ public void setInactivityTimeoutInMin(Integer inactivityTimeoutInMin) {
+ this.inactivityTimeoutInMin = inactivityTimeoutInMin;
+ }
+
+ @JsonProperty("name")
+ public String getName() {
+ return name;
+ }
+
+ @JsonProperty("name")
+ public void setName(String name) {
+ this.name = name;
+ }
+
+ @JsonProperty("parallelism")
+ public Integer getParallelism() {
+ return parallelism;
+ }
+
+ @JsonProperty("parallelism")
+ public void setParallelism(Integer parallelism) {
+ this.parallelism = parallelism;
+ }
+
+ @JsonProperty("retries")
+ public Integer getRetries() {
+ return retries;
+ }
+
+ @JsonProperty("retries")
+ public void setRetries(Integer retries) {
+ this.retries = retries;
+ }
+
+ @JsonProperty("retryDelayInSec")
+ public Integer getRetryDelayInSec() {
+ return retryDelayInSec;
+ }
+
+ @JsonProperty("retryDelayInSec")
+ public void setRetryDelayInSec(Integer retryDelayInSec) {
+ this.retryDelayInSec = retryDelayInSec;
+ }
+
+ @JsonProperty("softLimit")
+ public String getSoftLimit() {
+ return softLimit;
+ }
+
+ @JsonProperty("softLimit")
+ public void setSoftLimit(String softLimit) {
+ this.softLimit = softLimit;
+ }
+
+ @JsonProperty("scheduleActivities")
+ public List<String> getScheduleActivities() {
+ return scheduleActivities;
+ }
+
+ @JsonProperty("scheduleActivities")
+ public void setScheduleActivities(List<String> scheduleActivities) {
+ this.scheduleActivities = scheduleActivities;
+ }
+
+ @JsonProperty("scheduleOverrides")
+ public List<Object> getScheduleOverrides() {
+ return scheduleOverrides;
+ }
+
+ @JsonProperty("scheduleOverrides")
+ public void setScheduleOverrides(List<Object> scheduleOverrides) {
+ this.scheduleOverrides = scheduleOverrides;
+ }
+
+ @JsonProperty("schedulePeriod")
+ public String getSchedulePeriod() {
+ return schedulePeriod;
+ }
+
+ @JsonProperty("schedulePeriod")
+ public void setSchedulePeriod(String schedulePeriod) {
+ this.schedulePeriod = schedulePeriod;
+ }
+
+ @Override
+ public String toString() {
+ StringBuilder sb = new StringBuilder();
+ sb.append(Action.class.getName()).append('@').append(Integer.toHexString(System.identityHashCode(this))).append('[');
+ sb.append("actionReferSchedule");
+ sb.append('=');
+ sb.append(((this.actionReferSchedule == null) ? "<null>" : this.actionReferSchedule));
+ sb.append(',');
+ sb.append("actionSpecificData");
+ sb.append('=');
+ sb.append(((this.actionSpecificData == null) ? "<null>" : this.actionSpecificData));
+ sb.append(',');
+ sb.append("comment");
+ sb.append('=');
+ sb.append(((this.comment == null) ? "<null>" : this.comment));
+ sb.append(',');
+ sb.append("completionNotification");
+ sb.append('=');
+ sb.append(((this.completionNotification == null) ? "<null>" : this.completionNotification));
+ sb.append(',');
+ sb.append("concurrent");
+ sb.append('=');
+ sb.append(((this.concurrent == null) ? "<null>" : this.concurrent));
+ sb.append(',');
+ sb.append("drivenBy");
+ sb.append('=');
+ sb.append(((this.drivenBy == null) ? "<null>" : this.drivenBy));
+ sb.append(',');
+ sb.append("enabled");
+ sb.append('=');
+ sb.append(((this.enabled == null) ? "<null>" : this.enabled));
+ sb.append(',');
+ sb.append("failureImpact");
+ sb.append('=');
+ sb.append(((this.failureImpact == null) ? "<null>" : this.failureImpact));
+ sb.append(',');
+ sb.append("hardLimit");
+ sb.append('=');
+ sb.append(((this.hardLimit == null) ? "<null>" : this.hardLimit));
+ sb.append(',');
+ sb.append("inactivityTimeoutInMin");
+ sb.append('=');
+ sb.append(((this.inactivityTimeoutInMin == null) ? "<null>" : this.inactivityTimeoutInMin));
+ sb.append(',');
+ sb.append("name");
+ sb.append('=');
+ sb.append(((this.name == null) ? "<null>" : this.name));
+ sb.append(',');
+ sb.append("parallelism");
+ sb.append('=');
+ sb.append(((this.parallelism == null) ? "<null>" : this.parallelism));
+ sb.append(',');
+ sb.append("retries");
+ sb.append('=');
+ sb.append(((this.retries == null) ? "<null>" : this.retries));
+ sb.append(',');
+ sb.append("retryDelayInSec");
+ sb.append('=');
+ sb.append(((this.retryDelayInSec == null) ? "<null>" : this.retryDelayInSec));
+ sb.append(',');
+ sb.append("softLimit");
+ sb.append('=');
+ sb.append(((this.softLimit == null) ? "<null>" : this.softLimit));
+ sb.append(',');
+ sb.append("scheduleActivities");
+ sb.append('=');
+ sb.append(((this.scheduleActivities == null) ? "<null>" : this.scheduleActivities));
+ sb.append(',');
+ sb.append("scheduleOverrides");
+ sb.append('=');
+ sb.append(((this.scheduleOverrides == null) ? "<null>" : this.scheduleOverrides));
+ sb.append(',');
+ sb.append("schedulePeriod");
+ sb.append('=');
+ sb.append(((this.schedulePeriod == null) ? "<null>" : this.schedulePeriod));
+ sb.append(',');
+ if (sb.charAt((sb.length() - 1)) == ',') {
+ sb.setCharAt((sb.length() - 1), ']');
+ } else {
+ sb.append(']');
+ }
+ return sb.toString();
+ }
+
+ @Override
+ public int hashCode() {
+ int result = 1;
+ result = ((result * 31) + ((this.failureImpact == null) ? 0 : this.failureImpact.hashCode()));
+ result = ((result * 31) + ((this.actionSpecificData == null) ? 0 : this.actionSpecificData.hashCode()));
+ result = ((result * 31) + ((this.completionNotification == null) ? 0 : this.completionNotification.hashCode()));
+ result = ((result * 31) + ((this.parallelism == null) ? 0 : this.parallelism.hashCode()));
+ result = ((result * 31) + ((this.concurrent == null) ? 0 : this.concurrent.hashCode()));
+ result = ((result * 31) + ((this.retryDelayInSec == null) ? 0 : this.retryDelayInSec.hashCode()));
+ result = ((result * 31) + ((this.drivenBy == null) ? 0 : this.drivenBy.hashCode()));
+ result = ((result * 31) + ((this.enabled == null) ? 0 : this.enabled.hashCode()));
+ result = ((result * 31) + ((this.scheduleActivities == null) ? 0 : this.scheduleActivities.hashCode()));
+ result = ((result * 31) + ((this.retries == null) ? 0 : this.retries.hashCode()));
+ result = ((result * 31) + ((this.actionReferSchedule == null) ? 0 : this.actionReferSchedule.hashCode()));
+ result = ((result * 31) + ((this.name == null) ? 0 : this.name.hashCode()));
+ result = ((result * 31) + ((this.inactivityTimeoutInMin == null) ? 0 : this.inactivityTimeoutInMin.hashCode()));
+ result = ((result * 31) + ((this.comment == null) ? 0 : this.comment.hashCode()));
+ result = ((result * 31) + ((this.hardLimit == null) ? 0 : this.hardLimit.hashCode()));
+ result = ((result * 31) + ((this.scheduleOverrides == null) ? 0 : this.scheduleOverrides.hashCode()));
+ result = ((result * 31) + ((this.schedulePeriod == null) ? 0 : this.schedulePeriod.hashCode()));
+ result = ((result * 31) + ((this.softLimit == null) ? 0 : this.softLimit.hashCode()));
+ return result;
+ }
+
+ @Override
+ public boolean equals(Object other) {
+ if (other == this) {
+ return true;
+ }
+ if ((other instanceof Action) == false) {
+ return false;
+ }
+ Action rhs = ((Action) other);
+ return (((((((((((((((((((this.failureImpact == rhs.failureImpact) || ((this.failureImpact != null) && this.failureImpact.equals(rhs.failureImpact))) && ((this.actionSpecificData == rhs.actionSpecificData) || ((this.actionSpecificData != null) && this.actionSpecificData.equals(rhs.actionSpecificData)))) && ((this.completionNotification == rhs.completionNotification) || ((this.completionNotification != null) &&
+ this.completionNotification.equals(rhs.completionNotification)))) && ((this.parallelism == rhs.parallelism) || ((this.parallelism != null) && this.parallelism.equals(rhs.parallelism)))) && ((this.concurrent == rhs.concurrent) || ((this.concurrent != null) && this.concurrent.equals(rhs.concurrent)))) && ((this.retryDelayInSec == rhs.retryDelayInSec) || ((this.retryDelayInSec != null) && this.retryDelayInSec.equals(rhs.retryDelayInSec)))) && ((this.drivenBy == rhs.drivenBy) || ((this.drivenBy != null) &&
+ this.drivenBy.equals(rhs.drivenBy)))) && ((this.enabled == rhs.enabled) || ((this.enabled != null) && this.enabled.equals(rhs.enabled)))) && ((this.scheduleActivities == rhs.scheduleActivities) || ((this.scheduleActivities != null) && this.scheduleActivities.equals(rhs.scheduleActivities)))) && ((this.retries == rhs.retries) || ((this.retries != null) && this.retries.equals(rhs.retries)))) &&
+ ((this.actionReferSchedule == rhs.actionReferSchedule) || ((this.actionReferSchedule != null) && this.actionReferSchedule.equals(rhs.actionReferSchedule)))) && ((this.name == rhs.name) || ((this.name != null) && this.name.equals(rhs.name)))) && ((this.inactivityTimeoutInMin == rhs.inactivityTimeoutInMin) || ((this.inactivityTimeoutInMin != null) && this.inactivityTimeoutInMin.equals(rhs.inactivityTimeoutInMin)))) && ((this.comment == rhs.comment) || ((this.comment != null) &&
+ this.comment.equals(rhs.comment)))) && ((this.hardLimit == rhs.hardLimit) || ((this.hardLimit != null) && this.hardLimit.equals(rhs.hardLimit)))) && ((this.scheduleOverrides == rhs.scheduleOverrides) || ((this.scheduleOverrides != null) && this.scheduleOverrides.equals(rhs.scheduleOverrides)))) && ((this.schedulePeriod == rhs.schedulePeriod) || ((this.schedulePeriod != null) && this.schedulePeriod.equals(rhs.schedulePeriod)))) &&
+ ((this.softLimit == rhs.softLimit) || ((this.softLimit != null) && this.softLimit.equals(rhs.softLimit))));
+ }
Review Comment:
Will be removed since it is not used anywhere.
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/Backup.java:
##########
@@ -0,0 +1,423 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+import java.util.List;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "attributes",
+ "browseTime",
+ "clientHostname",
+ "clientId",
+ "completionTime",
+ "creationTime",
+ "fileCount",
+ "id",
+ "instances",
+ "level",
+ "links",
+ "name",
+ "retentionTime",
+ "saveTime",
+ "shortId",
+ "size",
+ "type"
+})
+@Generated("jsonschema2pojo")
+public class Backup implements Serializable {
+
+ private final static long serialVersionUID = -4474500098917286405L;
+ @JsonProperty("attributes")
+ private List<Attribute> attributes = null;
+ @JsonProperty("browseTime")
+ private String browseTime;
+ @JsonProperty("clientHostname")
+ private String clientHostname;
+ @JsonProperty("clientId")
+ private String clientId;
+ @JsonProperty("completionTime")
+ private String completionTime;
+ @JsonProperty("creationTime")
+ private String creationTime;
+ @JsonProperty("fileCount")
+ private Integer fileCount;
+ @JsonProperty("id")
+ private String id;
+ @JsonProperty("instances")
+ private List<Instance> instances = null;
+ @JsonProperty("level")
+ private String level;
+ @JsonProperty("links")
+ private List<Link> links = null;
+ @JsonProperty("name")
+ private String name;
+ @JsonProperty("retentionTime")
+ private String retentionTime;
+ @JsonProperty("saveTime")
+ private String saveTime;
+ @JsonProperty("shortId")
+ private String shortId;
+ @JsonProperty("size")
+ private Size size;
+ @JsonProperty("type")
+ private String type;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public Backup() {
+ }
+
+ /**
+ * @param shortId
+ * @param clientId
+ * @param browseTime
+ * @param creationTime
+ * @param instances
+ * @param level
+ * @param retentionTime
+ * @param type
+ * @param fileCount
+ * @param clientHostname
+ * @param completionTime
+ * @param size
+ * @param name
+ * @param attributes
+ * @param links
+ * @param id
+ * @param saveTime
+ */
+ public Backup(List<Attribute> attributes, String browseTime, String clientHostname, String clientId, String completionTime, String creationTime, Integer fileCount, String id, List<Instance> instances, String level, List<Link> links, String name, String retentionTime, String saveTime, String shortId, Size size, String type) {
+ super();
+ this.attributes = attributes;
+ this.browseTime = browseTime;
+ this.clientHostname = clientHostname;
+ this.clientId = clientId;
+ this.completionTime = completionTime;
+ this.creationTime = creationTime;
+ this.fileCount = fileCount;
+ this.id = id;
+ this.instances = instances;
+ this.level = level;
+ this.links = links;
+ this.name = name;
+ this.retentionTime = retentionTime;
+ this.saveTime = saveTime;
+ this.shortId = shortId;
+ this.size = size;
+ this.type = type;
+ }
+
+ @JsonProperty("attributes")
+ public List<Attribute> getAttributes() {
+ return attributes;
+ }
+
+ @JsonProperty("attributes")
+ public void setAttributes(List<Attribute> attributes) {
+ this.attributes = attributes;
+ }
+
+ @JsonProperty("browseTime")
+ public String getBrowseTime() {
+ return browseTime;
+ }
+
+ @JsonProperty("browseTime")
+ public void setBrowseTime(String browseTime) {
+ this.browseTime = browseTime;
+ }
+
+ @JsonProperty("clientHostname")
+ public String getClientHostname() {
+ return clientHostname;
+ }
+
+ @JsonProperty("clientHostname")
+ public void setClientHostname(String clientHostname) {
+ this.clientHostname = clientHostname;
+ }
+
+ @JsonProperty("clientId")
+ public String getClientId() {
+ return clientId;
+ }
+
+ @JsonProperty("clientId")
+ public void setClientId(String clientId) {
+ this.clientId = clientId;
+ }
+
+ @JsonProperty("completionTime")
+ public String getCompletionTime() {
+ return completionTime;
+ }
+
+ @JsonProperty("completionTime")
+ public void setCompletionTime(String completionTime) {
+ this.completionTime = completionTime;
+ }
+
+ @JsonProperty("creationTime")
+ public String getCreationTime() {
+ return creationTime;
+ }
+
+ @JsonProperty("creationTime")
+ public void setCreationTime(String creationTime) {
+ this.creationTime = creationTime;
+ }
+
+ @JsonProperty("fileCount")
+ public Integer getFileCount() {
+ return fileCount;
+ }
+
+ @JsonProperty("fileCount")
+ public void setFileCount(Integer fileCount) {
+ this.fileCount = fileCount;
+ }
+
+ @JsonProperty("id")
+ public String getId() {
+ return id;
+ }
+
+ @JsonProperty("id")
+ public void setId(String id) {
+ this.id = id;
+ }
+
+ @JsonProperty("instances")
+ public List<Instance> getInstances() {
+ return instances;
+ }
+
+ @JsonProperty("instances")
+ public void setInstances(List<Instance> instances) {
+ this.instances = instances;
+ }
+
+ @JsonProperty("level")
+ public String getLevel() {
+ return level;
+ }
+
+ @JsonProperty("level")
+ public void setLevel(String level) {
+ this.level = level;
+ }
+
+ @JsonProperty("links")
+ public List<Link> getLinks() {
+ return links;
+ }
+
+ @JsonProperty("links")
+ public void setLinks(List<Link> links) {
+ this.links = links;
+ }
+
+ @JsonProperty("name")
+ public String getName() {
+ return name;
+ }
+
+ @JsonProperty("name")
+ public void setName(String name) {
+ this.name = name;
+ }
+
+ @JsonProperty("retentionTime")
+ public String getRetentionTime() {
+ return retentionTime;
+ }
+
+ @JsonProperty("retentionTime")
+ public void setRetentionTime(String retentionTime) {
+ this.retentionTime = retentionTime;
+ }
+
+ @JsonProperty("saveTime")
+ public String getSaveTime() {
+ return saveTime;
+ }
+
+ @JsonProperty("saveTime")
+ public void setSaveTime(String saveTime) {
+ this.saveTime = saveTime;
+ }
+
+ @JsonProperty("shortId")
+ public String getShortId() {
+ return shortId;
+ }
+
+ @JsonProperty("shortId")
+ public void setShortId(String shortId) {
+ this.shortId = shortId;
+ }
+
+ @JsonProperty("size")
+ public Size getSize() {
+ return size;
+ }
+
+ @JsonProperty("size")
+ public void setSize(Size size) {
+ this.size = size;
+ }
+
+ @JsonProperty("type")
+ public String getType() {
+ return type;
+ }
+
+ @JsonProperty("type")
+ public void setType(String type) {
+ this.type = type;
+ }
+
+ @Override
+ public String toString() {
+ StringBuilder sb = new StringBuilder();
+ sb.append(Backup.class.getName()).append('@').append(Integer.toHexString(System.identityHashCode(this))).append('[');
+ sb.append("attributes");
+ sb.append('=');
+ sb.append(((this.attributes == null) ? "<null>" : this.attributes));
+ sb.append(',');
+ sb.append("browseTime");
+ sb.append('=');
+ sb.append(((this.browseTime == null) ? "<null>" : this.browseTime));
+ sb.append(',');
+ sb.append("clientHostname");
+ sb.append('=');
+ sb.append(((this.clientHostname == null) ? "<null>" : this.clientHostname));
+ sb.append(',');
+ sb.append("clientId");
+ sb.append('=');
+ sb.append(((this.clientId == null) ? "<null>" : this.clientId));
+ sb.append(',');
+ sb.append("completionTime");
+ sb.append('=');
+ sb.append(((this.completionTime == null) ? "<null>" : this.completionTime));
+ sb.append(',');
+ sb.append("creationTime");
+ sb.append('=');
+ sb.append(((this.creationTime == null) ? "<null>" : this.creationTime));
+ sb.append(',');
+ sb.append("fileCount");
+ sb.append('=');
+ sb.append(((this.fileCount == null) ? "<null>" : this.fileCount));
+ sb.append(',');
+ sb.append("id");
+ sb.append('=');
+ sb.append(((this.id == null) ? "<null>" : this.id));
+ sb.append(',');
+ sb.append("instances");
+ sb.append('=');
+ sb.append(((this.instances == null) ? "<null>" : this.instances));
+ sb.append(',');
+ sb.append("level");
+ sb.append('=');
+ sb.append(((this.level == null) ? "<null>" : this.level));
+ sb.append(',');
+ sb.append("links");
+ sb.append('=');
+ sb.append(((this.links == null) ? "<null>" : this.links));
+ sb.append(',');
+ sb.append("name");
+ sb.append('=');
+ sb.append(((this.name == null) ? "<null>" : this.name));
+ sb.append(',');
+ sb.append("retentionTime");
+ sb.append('=');
+ sb.append(((this.retentionTime == null) ? "<null>" : this.retentionTime));
+ sb.append(',');
+ sb.append("saveTime");
+ sb.append('=');
+ sb.append(((this.saveTime == null) ? "<null>" : this.saveTime));
+ sb.append(',');
+ sb.append("shortId");
+ sb.append('=');
+ sb.append(((this.shortId == null) ? "<null>" : this.shortId));
+ sb.append(',');
+ sb.append("size");
+ sb.append('=');
+ sb.append(((this.size == null) ? "<null>" : this.size));
+ sb.append(',');
+ sb.append("type");
+ sb.append('=');
+ sb.append(((this.type == null) ? "<null>" : this.type));
+ sb.append(',');
+ if (sb.charAt((sb.length() - 1)) == ',') {
+ sb.setCharAt((sb.length() - 1), ']');
+ } else {
+ sb.append(']');
+ }
+ return sb.toString();
+ }
+
+ @Override
+ public int hashCode() {
+ int result = 1;
+ result = ((result * 31) + ((this.shortId == null) ? 0 : this.shortId.hashCode()));
+ result = ((result * 31) + ((this.clientId == null) ? 0 : this.clientId.hashCode()));
+ result = ((result * 31) + ((this.browseTime == null) ? 0 : this.browseTime.hashCode()));
+ result = ((result * 31) + ((this.creationTime == null) ? 0 : this.creationTime.hashCode()));
+ result = ((result * 31) + ((this.instances == null) ? 0 : this.instances.hashCode()));
+ result = ((result * 31) + ((this.level == null) ? 0 : this.level.hashCode()));
+ result = ((result * 31) + ((this.retentionTime == null) ? 0 : this.retentionTime.hashCode()));
+ result = ((result * 31) + ((this.type == null) ? 0 : this.type.hashCode()));
+ result = ((result * 31) + ((this.fileCount == null) ? 0 : this.fileCount.hashCode()));
+ result = ((result * 31) + ((this.clientHostname == null) ? 0 : this.clientHostname.hashCode()));
+ result = ((result * 31) + ((this.completionTime == null) ? 0 : this.completionTime.hashCode()));
+ result = ((result * 31) + ((this.size == null) ? 0 : this.size.hashCode()));
+ result = ((result * 31) + ((this.name == null) ? 0 : this.name.hashCode()));
+ result = ((result * 31) + ((this.attributes == null) ? 0 : this.attributes.hashCode()));
+ result = ((result * 31) + ((this.links == null) ? 0 : this.links.hashCode()));
+ result = ((result * 31) + ((this.id == null) ? 0 : this.id.hashCode()));
+ result = ((result * 31) + ((this.saveTime == null) ? 0 : this.saveTime.hashCode()));
+ return result;
+ }
+
+ @Override
+ public boolean equals(Object other) {
+ if (other == this) {
+ return true;
+ }
+ if ((other instanceof Backup) == false) {
+ return false;
+ }
+ Backup rhs = ((Backup) other);
+ return ((((((((((((((((((this.shortId == rhs.shortId) || ((this.shortId != null) && this.shortId.equals(rhs.shortId))) && ((this.clientId == rhs.clientId) || ((this.clientId != null) && this.clientId.equals(rhs.clientId)))) && ((this.browseTime == rhs.browseTime) || ((this.browseTime != null) &&
+ this.browseTime.equals(rhs.browseTime)))) && ((this.creationTime == rhs.creationTime) || ((this.creationTime != null) && this.creationTime.equals(rhs.creationTime)))) && ((this.instances == rhs.instances) || ((this.instances != null) && this.instances.equals(rhs.instances)))) && ((this.level == rhs.level) || ((this.level != null) &&
+ this.level.equals(rhs.level)))) && ((this.retentionTime == rhs.retentionTime) || ((this.retentionTime != null) && this.retentionTime.equals(rhs.retentionTime)))) && ((this.type == rhs.type) || ((this.type != null) && this.type.equals(rhs.type)))) && ((this.fileCount == rhs.fileCount) || ((this.fileCount != null) && this.fileCount.equals(rhs.fileCount)))) &&
+ ((this.clientHostname == rhs.clientHostname) || ((this.clientHostname != null) && this.clientHostname.equals(rhs.clientHostname)))) && ((this.completionTime == rhs.completionTime) || ((this.completionTime != null) && this.completionTime.equals(rhs.completionTime)))) && ((this.size == rhs.size) || ((this.size != null) && this.size.equals(rhs.size)))) &&
+ ((this.name == rhs.name) || ((this.name != null) && this.name.equals(rhs.name)))) && ((this.attributes == rhs.attributes) || ((this.attributes != null) && this.attributes.equals(rhs.attributes)))) && ((this.links == rhs.links) || ((this.links != null) && this.links.equals(rhs.links)))) && ((this.id == rhs.id) || ((this.id != null) && this.id.equals(rhs.id)))) &&
+ ((this.saveTime == rhs.saveTime) || ((this.saveTime != null) && this.saveTime.equals(rhs.saveTime))));
Review Comment:
Will be removed since it is not used anywhere.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1183690862
@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1185564765
@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] acs-robot commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
acs-robot commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1185722139
Found UI changes, kicking a new UI QA build
@blueorangutan ui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1196676072
UI build: :heavy_check_mark:
Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6550 (SL-JID-2013)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1204871477
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] acs-robot commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
acs-robot commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1303279989
Found UI changes, kicking a new UI QA build
@blueorangutan ui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1301592982
>
> ping @fermosan you see any chance to address @rohityadavcloud 's remaining comments?
Yes. @DaanHoogland
Updates:
if (guestOsMapping == null || hostVo == null) will be reverted in a coming patch
I am trying to see if we can use the Script.findScript(). I cannot find a way to make this work. The function is called on the management server side and the script cannot be found since it exists on the client.
Furthermore in the development environment the lookup path is under the local development directories.
There is now straight way to get "kvm.script.dir" or "scripts.path" from environ since configure for super is not there.
If there is no way for this to work the full path will be used there. Any ideas ?
I am also working on eliminating the need for sudo execution on the KVM host side (needs some work/testing and more conf in
the EMC networker side). If successful, i will commit here and update the documentation pull as well to reflect that.
For the [ui/public/config.json] I will revert the changes and match main in a coming patch.
Finally for the comments about
int minCpuCores = virtualMachineTo.getCpus();
long minMemory = virtualMachineTo.getMinRam();
The code was there beforehand and as far as I can see it is not used by any other components. It is however used in the tests. I am not sure what is supposed to do and which components are supposed to call it if any. Any inputs ? How we going to proceed on that ? My guess is that it has to do something with the computer offerings. Can those be null under any circumstance ?
Maybe @nvazquez has some input for the above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r967635841
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/NetworkerClient.java:
##########
@@ -0,0 +1,360 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.cloudstack.backup.networker;
+
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nio.TrustAllManager;
+import com.cloud.vm.VirtualMachine;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.cloudstack.api.ApiErrorCode;
+import org.apache.cloudstack.api.ServerApiException;
+import org.apache.cloudstack.backup.BackupOffering;
+import org.apache.cloudstack.backup.BackupVO;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackups;
+import org.apache.cloudstack.backup.networker.api.ProtectionPolicies;
+import org.apache.cloudstack.backup.networker.api.ProtectionPolicy;
+import org.apache.cloudstack.utils.security.SSLUtils;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpResponse;
+import org.apache.http.HttpStatus;
+import org.apache.http.client.HttpClient;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.HttpDelete;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.conn.ConnectTimeoutException;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.impl.client.HttpClientBuilder;
+import org.apache.log4j.Logger;
+
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.X509TrustManager;
+import java.io.IOException;
+import java.net.SocketTimeoutException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.security.SecureRandom;
+import java.text.ParseException;
+import java.text.SimpleDateFormat;
+import java.time.Instant;
+import java.util.ArrayList;
+import java.util.Base64;
+import java.util.Date;
+import java.util.List;
+
+import static org.apache.cloudstack.backup.NetworkerBackupProvider.BACKUP_IDENTIFIER;
+
+
+public class NetworkerClient {
+
Review Comment:
fixed in latest commit
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/NetworkerClient.java:
##########
@@ -0,0 +1,360 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.cloudstack.backup.networker;
+
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nio.TrustAllManager;
+import com.cloud.vm.VirtualMachine;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.cloudstack.api.ApiErrorCode;
+import org.apache.cloudstack.api.ServerApiException;
+import org.apache.cloudstack.backup.BackupOffering;
+import org.apache.cloudstack.backup.BackupVO;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackups;
+import org.apache.cloudstack.backup.networker.api.ProtectionPolicies;
+import org.apache.cloudstack.backup.networker.api.ProtectionPolicy;
+import org.apache.cloudstack.utils.security.SSLUtils;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpResponse;
+import org.apache.http.HttpStatus;
+import org.apache.http.client.HttpClient;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.HttpDelete;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.conn.ConnectTimeoutException;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.impl.client.HttpClientBuilder;
+import org.apache.log4j.Logger;
+
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.X509TrustManager;
+import java.io.IOException;
+import java.net.SocketTimeoutException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.security.SecureRandom;
+import java.text.ParseException;
+import java.text.SimpleDateFormat;
+import java.time.Instant;
+import java.util.ArrayList;
+import java.util.Base64;
+import java.util.Date;
+import java.util.List;
+
+import static org.apache.cloudstack.backup.NetworkerBackupProvider.BACKUP_IDENTIFIER;
+
+
+public class NetworkerClient {
+
+
+ private static final Logger LOG = Logger.getLogger(NetworkerClient.class);
+
+ private final URI apiURI;
+ private final String apiName;
+ private final String apiPassword;
+
+ private final HttpClient httpClient;
+
+
Review Comment:
fixed in latest commit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1363997932
@DaanHoogland a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1365707905
@DaanHoogland a Trillian-Jenkins matrix job (centos7 mgmt + xenserver71, rocky8 mgmt + vmware67u3, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1365707474
@blueorangutan test matrix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1260935302
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1367174458
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: el9 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 5125
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1367189736
@DaanHoogland a Trillian-Jenkins matrix job (centos7 mgmt + xenserver71, rocky8 mgmt + vmware67u3, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1370963760
@blueorangutan test matrix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1371672870
<b>Trillian test result (tid-5749)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 42647 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6550-t5749-kvm-centos7.zip
Smoke tests completed. 105 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_08_upgrade_kubernetes_ha_cluster | `Failure` | 672.15 | test_kubernetes_clusters.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1370766873
@DaanHoogland a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1283840423
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 4504
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r979472256
##########
scripts/vm/hypervisor/kvm/nsrkvmbackup.sh:
##########
@@ -0,0 +1,270 @@
+#!/bin/bash
+
+## Licensed to the Apache Software Foundation (ASF) under one
+## or more contributor license agreements. See the NOTICE file
+## distributed with this work for additional information
+## regarding copyright ownership. The ASF licenses this file
+## to you under the Apache License, Version 2.0 (the
+## "License"); you may not use this file except in compliance
+## with the License. You may obtain a copy of the License at
+##
+## http://www.apache.org/licenses/LICENSE-2.0
+##
+## Unless required by applicable law or agreed to in writing,
+## software distributed under the License is distributed on an
+## "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+## KIND, either express or implied. See the License for the
+## specific language governing permissions and limitations
+## under the License.
+
+
+version=0.9.1
+
+
+OPTIND=1
+
+verb=0
+logDir="/nsr/logs/cloudstack/"
+snapPrefix="CSBKP_$RANDOM_"
+clusterClient=""
+networkerServer=""
+hvVersion=""
+libvVersion=""
+apiVersion=""
+kvmDName=""
+kvmDUuid=""
+logFile=""
+mediaPool=""
+retentionTime=""
+
+
+log () {
+
+ [[ "$verb" -eq 1 ]] && builtin echo "$@"
+ if [[ "$1" == "-ne" || "$1" == "-e" || "$1" == "-n" ]]; then
+ builtin echo -e "$(date '+%Y-%m-%d %H-%M-%S>')" "${@: 2}" >> "$logFile"
+ else
+ builtin echo "$(date '+%Y-%m-%d %H-%M-%S>')" "$@" >> "$logFile"
+ fi
+
+}
+
+vercomp(){
+ local a b IFS=. -; set -f
+ printf -v a %08d $1; printf -v b %08d $3
+ test $a "$2" $b
+}
+usage() {
+ echo "
+
+Usage:[-v] [-h] [-l log_dir] [-dr] [-s networker_server] [-c networker_cluster_client] [-t target_vm] [-u target_uuid] [-p snapprefix] [-P media_pool ] [-R retention_time ]
+
+Options:
+ -h Help and usage
+ -v Enable verbose mode
+ -l log_dir. Specify log directory. Default is /nsr/logs/cloudstack
+ -s networker_server Specifiy the EMC Networker server we are going to use
+ -c networker_cluster_client Specify the EMC Networker client CLUSTER to use
+ -t target_vm KVM domain to backup
+ -u target_uuid KVM domain to backup
+ -p Snapshot Prefix for backups
+ -P mediaPool EMC Networker Media Pool
+ -R retention_time Backup retention time
+
+Supplements Apache Cloudstack B&R Framework EMC Networker plugin and performs the backup of the Virtual Machines
+"
+
+}
+
+sanity_checks() {
+
+ log "Performing environment sanity checks..."
+ log -ne "\t[1] Checking if Networker is installed\t"
+ if [[ $(systemctl list-unit-files | grep networker) = *networker* ]]; then
+ log "Success"
+ else
+ log "Failure"
+ log -e "\n\tNetworker Service NOT FOUND. Make sure that Networker client is properly installed"
+ exit 1
+ fi
+
+ log -ne "\t[2] Checking if Networker is running\t"
+ if [[ $(systemctl is-active networker) = *active* ]]; then
+ log "Success"
+ else
+ log "Failure"
+ log -e "\n\tNetworker Service is not running. Investigate Networker logs, startup server and try again"
+ exit 2
+ fi
+ log -ne "\t[3] Checking Networker DNS Resolution\t"
+ if [[ $(getent hosts "$networkerServer") = *$networkerServer* ]]; then
+ log "Success"
+ else
+ log "Failure"
+ log -e "\n\tNetworker Server cannot be resolved. Backups will most probably fail. Consider adding the ip/hostname to /etc/host or fix DNS resolution"
+ exit 3
+ fi
+
+ log -ne "\t[4] Checking QEMU / Libvirt Versions \t"
+ hvVersion=$(virsh version | grep hypervisor | awk '{print $(NF)}')
+ libvVersion=$(virsh version | grep libvirt | awk '{print $(NF)}' | tail -n 1)
+ apiVersion=$(virsh version | grep API | awk '{print $(NF)}')
+ if vercomp "$hvVersion" \> 2.1.2; then
+ log -n "Success"
+ log -ne "\t\t [ Libvirt: $libvVersion apiVersion: $apiVersion ]"
+ echo
+
+ else
+ log "Failure"
+ log -e "\n\tYour QEMU version $hvVersion is unsupported. Consider upgrading at least to latest QEMU at branch 2"
+ exit 4
+ fi
+
+
+
+ log "Environment Sanity Checks successfully passed"
+}
+
+
+
+echo "
+Cloudstack B&R Framework - EMC Networker backup script
+Version $version
+"
+
+
+backup_domain() {
+
+ name=$1
+ snapName=$2
+ log "Preparing snapshots and gathering information for backing up domain $name under snapshot name $snapName"
+ log "Retention time is $retentionTime"
+
+ declare -A TRGSRC
+ while IFS=',' read -r TARGET SOURCE
+ do
+ if [[ $SOURCE != "-" ]]; then
+ TRGSRC+=(["$TARGET"]="$SOURCE")
+ fi
+ done < <(virsh domblklist "$name" --details | grep file | grep -v 'cdrom' | grep -v 'floppy' | sed 's/ */,/g' | cut -d',' -f 4-)
+ diskspec=""
+ for target in "${!TRGSRC[@]}"; do
+ log -e "\tDisk for $target is at ${TRGSRC[${target}]}"
+ diskspec="$diskspec --diskspec $target,snapshot=external"
+ disks="$disks ${TRGSRC[${target}]} "
+ done
+
+ cmd="$(virsh snapshot-create-as --domain "$name" --name "$snapName" --no-metadata --atomic --quiesce --disk-only "$diskspec")"
+ retVal=$?
+ log "$cmd"
+ if [ "$retVal" -ne 0 ]; then
+ log "Agent not responding, trying to snapshot directly"
+ cmd="$(virsh snapshot-create-as --domain "$name" --name "$snapName" --no-metadata --atomic --disk-only "$diskspec")"
+ retVal=$?
+ if [ "$retVal" -ne 0 ]; then
+ log "Failed to create snapshot for $name"
+ exit 7
+ fi
+ log "Created snapshot(s) for $name"
+ fi
+ cmd="$(save -LL -q -e "${retentionTime}" -s "$networkerServer" -c "$clusterClient" -N "$name" -b "$mediaPool" $disks)"
+ retVal=$?
+ log "$cmd"
+ echo "$cmd" | grep -oE 'savetime=[0-9]{10}'
+ if [ $retVal -ne 0 ]; then
+ log "Unable to backup $disks for $name"
+ else
+ log "Backup $disks for $name completed!!"
+ fi
+
+
+ #Merge changes and conclude
+ SNAPSHOTS="$(virsh domblklist "$name" --details | grep file | grep -v 'cdrom' | grep -v 'floppy' | awk '{print $4}')"
+ for target in "${!TRGSRC[@]}"; do
+ log "Merging Snasphots for $target"
+ cmd="$(virsh blockcommit "$name" "$target" --active --pivot)"
+ retVal=$?
+ log "$cmd"
+ if [ $retVal -ne 0 ]; then
+ log "Unable to merge disk %target changes for domain $name"
+ exit 8
+ fi
+ done
+ #Clean snapshots
+ for snapshot in $SNAPSHOTS; do
+ log "Deleting Snapshot $snapshot"
+ cmd=$(rm "$snapshot")
+ retVal=$?
+ log "$cmd"
+ if [ $retVal -ne 0 ]; then
+ log "Unable to delete snapshot $snapshot"
+ exit 8
+ fi
+ log "Deleted Snapshot: $snapshot"
+ done
+}
+
+while getopts "h?vs:l:c:t:u:p:P:R:" opt; do
+ case "$opt" in
+ h|\?)
+ usage
+ exit 254
+ ;;
+ c) clusterClient="$OPTARG"
+ ;;
+ s) networkerServer="$OPTARG"
+ ;;
+ l) logDir="$OPTARG"
+ ;;
+ t) kvmDName="$OPTARG"
+ ;;
+ u) kvmDUuid="$OPTARG"
+ ;;
+ p) snapPrefix="$OPTARG"
+ ;;
+ P) mediaPool="$OPTARG"
+ ;;
+ R) retentionTime="$OPTARG"
+ ;;
+ v) verb=1
+ ;;
+ esac
+ done
+
+ shift $((OPTIND-1))
+
+ [ "${1:-}" = "--" ] && shift
+
+ if [[ -z "$networkerServer" || -z "$kvmDName" || -z "$clusterClient" || -z "$kvmDUuid" ]]; then
+ usage
+ exit 255
+ fi
+
+ if [ ! -d "$logDir" ]; then
+ mkdir -p "$logDir"
+ fi
+
+ logFile="$logDir/BACKUP-$kvmDName-$(date +'%Y_%m_%d_%I_%M_%p').log"
+
+
+ # Perform Initial sanity checks
+ sanity_checks
+
+
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1257603698
@DaanHoogland a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1279788845
@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] acs-robot commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
acs-robot commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1279788731
Found UI changes, kicking a new UI QA build
@blueorangutan ui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1260454449
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1260494640
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 4299
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r951213446
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/NetworkerBackupProvider.java:
##########
@@ -0,0 +1,640 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.backup;
+
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.host.HostVO;
+import com.cloud.host.Status;
+import com.cloud.host.dao.HostDao;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.StoragePoolHostVO;
+import com.cloud.storage.Volume;
+import com.cloud.storage.VolumeVO;
+import com.cloud.storage.dao.StoragePoolHostDao;
+import com.cloud.storage.dao.VolumeDao;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.component.AdapterBase;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallbackNoReturn;
+import com.cloud.utils.db.TransactionStatus;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.ssh.SSHCmdHelper;
+import com.cloud.utils.ssh.SshException;
+import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.dao.VMInstanceDao;
+import org.apache.cloudstack.api.InternalIdentity;
+import org.apache.cloudstack.backup.dao.BackupDao;
+import org.apache.cloudstack.backup.dao.BackupOfferingDaoImpl;
+import org.apache.cloudstack.backup.networker.NetworkerClient;
+import org.apache.cloudstack.framework.config.ConfigKey;
+import org.apache.cloudstack.framework.config.Configurable;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.log4j.Logger;
+import org.apache.xml.utils.URI;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+
+
+import javax.inject.Inject;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.HashMap;
+import java.util.Date;
+import java.util.Objects;
+import java.util.UUID;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+
+public class NetworkerBackupProvider extends AdapterBase implements BackupProvider, Configurable {
+
+ public static final String BACKUP_IDENTIFIER = "-CSBKP-";
+ private static final Logger LOG = Logger.getLogger(NetworkerBackupProvider.class);
+ public ConfigKey<String> NetworkerUrl = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.url", "https://localhost:9090/nwrestapi/v3",
+ "The EMC Networker API URL.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerUsername = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.username", "administrator",
+ "The EMC Networker API username.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerPassword = new ConfigKey<>("Secure", String.class,
+ "backup.plugin.networker.password", "password",
+ "The EMC Networker API password.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerMediaPool = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.pool", "Default",
+ "The EMC Networker Media Pool", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerValidateSSLSecurity = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.validate.ssl", "false",
+ "Validate the SSL certificate when connecting to EMC Networker API service.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Integer> NetworkerApiRequestTimeout = new ConfigKey<>("Advanced", Integer.class,
+ "backup.plugin.networker.request.timeout", "300",
+ "The EMC Networker API request timeout in seconds.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerClientVerboseLogs = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.client.verbosity", "false",
+ "Produce Verbose logs in Hypervisor", true, ConfigKey.Scope.Zone);
+
+ @Inject
+ private BackupDao backupDao;
+
+ @Inject
+ private HostDao hostDao;
+
+ @Inject
+ private ConfigurationDao configDao;
+
+ @Inject
+ private ClusterDao clusterDao;
+
+ @Inject
+ private VolumeDao volumeDao;
+
+ @Inject
+ private StoragePoolHostDao storagePoolHostDao;
+
+ @Inject
+ private VMInstanceDao vmInstanceDao;
+
+ private static String getUrlDomain(String url) throws URISyntaxException {
+ URI uri;
+ try {
+ uri = new URI(url);
+ } catch (URI.MalformedURIException e) {
+ throw new CloudRuntimeException("Failed to cast URI");
+ }
+
+ return uri.getHost();
+ }
+
+ @Override
+ public ConfigKey<?>[] getConfigKeys() {
+ return new ConfigKey[]{
+ NetworkerUrl,
+ NetworkerUsername,
+ NetworkerPassword,
+ NetworkerValidateSSLSecurity,
+ NetworkerApiRequestTimeout,
+ NetworkerMediaPool,
+ NetworkerClientVerboseLogs
+ };
+ }
+
+ @Override
+ public String getName() {
+ return "networker";
+ }
+
+ @Override
+ public String getDescription() {
+ return "EMC Networker Backup Plugin";
+ }
+
+ @Override
+ public String getConfigComponentName() {
+ return BackupService.class.getSimpleName();
+ }
+
+ protected HostVO getLastVMHypervisorHost(VirtualMachine vm) {
+ HostVO host;
+ Long hostId = vm.getLastHostId();
+
+ if (hostId == null) {
+ LOG.debug("Cannot find last host for vm. This should never happen, please check your database.");
+ return null;
+ }
+ host = hostDao.findById(hostId);
+
+ if ( host.getStatus() == Status.Up ) {
+ return host;
+ } else {
+ // Try to find a host in the same cluster
+ List<HostVO> altClusterHosts = hostDao.findHypervisorHostInCluster(host.getClusterId());
+ for (final HostVO candidateClusterHost : altClusterHosts) {
+ if ( candidateClusterHost.getStatus() == Status.Up ) {
+ LOG.debug("Found Host " + candidateClusterHost.getName());
+ return candidateClusterHost;
+ }
+ }
+ }
+ // Try to find a Host in the zone
+ List<HostVO> altZoneHosts = hostDao.findByDataCenterId(host.getDataCenterId());
+ for (final HostVO candidateZoneHost : altZoneHosts) {
+ if ( candidateZoneHost.getStatus() == Status.Up && candidateZoneHost.getHypervisorType() == Hypervisor.HypervisorType.KVM ) {
+ LOG.debug("Found Host " + candidateZoneHost.getName());
+ return candidateZoneHost;
+ }
+ }
+ return null;
+ }
+
+ protected HostVO getRunningVMHypervisorHost(VirtualMachine vm) {
+
+ HostVO host;
+ Long hostId = vm.getHostId();
+
+ if (hostId == null) {
+ throw new CloudRuntimeException("Unable to find the HYPERVISOR for " + vm.getName() + ". Make sure the virtual machine is running");
+ }
+
+ host = hostDao.findById(hostId);
+
+ return host;
+ }
+
+ protected String getVMHypervisorCluster(HostVO host) {
+
+ return clusterDao.findById(host.getClusterId()).getName();
+ }
+
+ protected Ternary<String, String, String> getKVMHyperisorCredentials(HostVO host) {
+
+ String username = null;
+ String password = null;
+ String privateKey = null;
+
+ if (host != null && host.getHypervisorType() == Hypervisor.HypervisorType.KVM) {
+ hostDao.loadDetails(host);
+ password = host.getDetail("password");
+ username = host.getDetail("username");
+ privateKey = configDao.getValue("ssh.privatekey");
+ }
+ if ((password == null && privateKey == null) || username == null) {
+ throw new CloudRuntimeException("Cannot find login credentials for HYPERVISOR " + Objects.requireNonNull(host).getUuid());
+ }
+
+ return new Ternary<>(username, password, privateKey);
+ }
+
+ private String executeBackupCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+ String nstRegex = "\\bcompleted savetime=([0-9]{10})";
+ Pattern saveTimePattern = Pattern.compile(nstRegex);
+
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ Matcher saveTimeMatcher = saveTimePattern.matcher(result.getStdOut());
+ if (saveTimeMatcher.find()) {
+ return saveTimeMatcher.group(1);
+ }
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+
+ return null;
+ }
+ private boolean executeRestoreCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ return true;
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+ }
+
+ private NetworkerClient getClient(final Long zoneId) {
+ try {
+ return new NetworkerClient(NetworkerUrl.valueIn(zoneId), NetworkerUsername.valueIn(zoneId), NetworkerPassword.valueIn(zoneId),
+ NetworkerValidateSSLSecurity.valueIn(zoneId), NetworkerApiRequestTimeout.valueIn(zoneId));
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException("Failed to parse EMC Networker API URL: " + e.getMessage());
+ } catch (NoSuchAlgorithmException | KeyManagementException e) {
+ LOG.error("Failed to build EMC Networker API client due to: ", e);
+ }
+ throw new CloudRuntimeException("Failed to build EMC Networker API client");
+ }
+
+ @Override
+ public List<BackupOffering> listBackupOfferings(Long zoneId) {
+ List<BackupOffering> policies = new ArrayList<>();
+ for (final BackupOffering policy : getClient(zoneId).listPolicies()) {
+ if (!policy.getName().contains(BACKUP_IDENTIFIER)) {
+ policies.add(policy);
+ }
+ }
+
+ return policies;
+ }
+
+ @Override
+ public boolean isValidProviderOffering(Long zoneId, String uuid) {
+ List<BackupOffering> policies = listBackupOfferings(zoneId);
+ if (CollectionUtils.isEmpty(policies)) {
+ return false;
+ }
+ for (final BackupOffering policy : policies) {
+ if (Objects.equals(policy.getExternalId(), uuid)) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ @Override
+ public boolean assignVMToBackupOffering(VirtualMachine vm, BackupOffering backupOffering) { return true; }
+
+ @Override
+ public boolean removeVMFromBackupOffering(VirtualMachine vm) {
+ LOG.debug("Removing VirtualMachine from Backup offering and Deleting any existing backups");
+
+ List<String> backupsTaken = getClient(vm.getDataCenterId()).getBackupsForVm(vm);
+
+ for (String backupId : backupsTaken) {
+ LOG.debug("Trying to remove backup with id" + backupId);
+ getClient(vm.getDataCenterId()).deleteBackupForVM(backupId);
+ }
+
+ return true;
+ }
+
+ @Override
+ public boolean restoreVMFromBackup(VirtualMachine vm, Backup backup) {
+ LOG.debug("Restoring vm " + vm.getUuid() + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+ String networkerServer;
+ HostVO hostVO;
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return false;
+ }
+
+ // Find where the VM was last running
+ hostVO = getLastVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -S " + SSID;
+
+ if (NetworkerClientVerboseLogs.value())
+ command = command + " -v ";
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for VM ID " + vm.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return true;
+ } else {
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return false;
+ }
+ }
+
+ @Override
+ public Pair<Boolean, String> restoreBackedUpVolume(Backup backup, String volumeUuid, String hostIp, String dataStoreUuid) {
+ LOG.debug("Restoring volume " + volumeUuid + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+
+ String networkerServer;
+ VolumeVO volume = volumeDao.findByUuid(volumeUuid);
+ VMInstanceVO backupSourceVm = vmInstanceDao.findById(backup.getVmId());
+ StoragePoolHostVO dataStore = storagePoolHostDao.findByUuid(dataStoreUuid);
+ HostVO hostVO = hostDao.findByIp(hostIp);
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+ final String clusterName = networkerBackup.getClientHostname();
+ final String destinationNetworkerClient = hostVO.getName().split("\\.")[0];
+ Long restoredVolumeDiskSize = 0L;
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return null;
+ }
+
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ // Find volume size from backup vols
+ for ( Backup.VolumeInfo VMVolToRestore : backupSourceVm.getBackupVolumeList()) {
+ if (VMVolToRestore.getUuid().equals(volumeUuid))
+ restoredVolumeDiskSize = (VMVolToRestore.getSize());
+ }
+
+ VolumeVO restoredVolume = new VolumeVO(Volume.Type.DATADISK, null, backup.getZoneId(),
+ backup.getDomainId(), backup.getAccountId(), 0, null,
+ backup.getSize(), null, null, null);
+
+ restoredVolume.setName("RV-"+volume.getName());
+ restoredVolume.setProvisioningType(volume.getProvisioningType());
+ restoredVolume.setUpdated(new Date());
+ restoredVolume.setUuid(UUID.randomUUID().toString());
+ restoredVolume.setRemoved(null);
+ restoredVolume.setDisplayVolume(true);
+ restoredVolume.setPoolId(volume.getPoolId());
+ restoredVolume.setPath(restoredVolume.getUuid());
+ restoredVolume.setState(Volume.State.Copying);
+ restoredVolume.setSize(restoredVolumeDiskSize);
+ restoredVolume.setDiskOfferingId(volume.getDiskOfferingId());
+
+ try {
+ volumeDao.persist(restoredVolume);
+ } catch (Exception e) {
+ throw new CloudRuntimeException("Unable to craft restored volume due to: "+e);
+ }
+
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
Review Comment:
normally would it make sense to use `Script` class?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r951217347
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/NetworkerBackupOffering.java:
##########
@@ -0,0 +1,82 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.cloudstack.backup.networker;
+
+
+import org.apache.cloudstack.backup.BackupOffering;
+
+import java.util.Date;
+
+
+public class NetworkerBackupOffering implements BackupOffering {
+
+ private String name;
+ private String uid;
+
+ public NetworkerBackupOffering(String name, String uid) {
+ this.name = name;
+ this.uid = uid;
+ }
+
+ @Override
+ public String getExternalId() {
+ return uid;
+ }
+
+ @Override
+ public String getName() {
+ return name;
+ }
+
+ @Override
+ public String getDescription() {
+ return "EMC Networker Backup Offering (Job)";
+ }
+
+ @Override
+ public long getZoneId() {
+ return -1;
+ }
+
+ @Override
+ public boolean isUserDrivenBackupAllowed() {
+ return false;
+ }
+
+ @Override
+ public String getProvider() {
+ return "veeam";
Review Comment:
@fermosan this should be called as name of the provider, so I'm guessing not `veeam`?
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/NetworkerClient.java:
##########
@@ -0,0 +1,360 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
Review Comment:
minor nit - extra newline
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r951219211
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/NetworkerObject.java:
##########
@@ -0,0 +1,34 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.cloudstack.backup.networker;
+
+
Review Comment:
minor nit - extra newline
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r951221912
##########
scripts/vm/hypervisor/kvm/nsrkvmbackup.sh:
##########
@@ -0,0 +1,270 @@
+#!/bin/bash
+
+## Licensed to the Apache Software Foundation (ASF) under one
+## or more contributor license agreements. See the NOTICE file
+## distributed with this work for additional information
+## regarding copyright ownership. The ASF licenses this file
+## to you under the Apache License, Version 2.0 (the
+## "License"); you may not use this file except in compliance
+## with the License. You may obtain a copy of the License at
+##
+## http://www.apache.org/licenses/LICENSE-2.0
+##
+## Unless required by applicable law or agreed to in writing,
+## software distributed under the License is distributed on an
+## "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+## KIND, either express or implied. See the License for the
+## specific language governing permissions and limitations
+## under the License.
+
+
+version=0.9.1
Review Comment:
is this version of EMC networker?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r920099875
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/Link.java:
##########
@@ -0,0 +1,117 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "href",
+ "rel"
+})
+@Generated("jsonschema2pojo")
+public class Link implements Serializable {
+
+ private final static long serialVersionUID = 1110347626425938231L;
+ @JsonProperty("href")
+ private String href;
+ @JsonProperty("rel")
+ private String rel;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public Link() {
+ }
+
+ /**
+ * @param rel
+ * @param href
+ */
+ public Link(String href, String rel) {
+ super();
+ this.href = href;
+ this.rel = rel;
+ }
+
+ @JsonProperty("href")
+ public String getHref() {
+ return href;
+ }
+
+ @JsonProperty("href")
+ public void setHref(String href) {
+ this.href = href;
+ }
+
+ @JsonProperty("rel")
+ public String getRel() {
+ return rel;
+ }
+
+ @JsonProperty("rel")
+ public void setRel(String rel) {
+ this.rel = rel;
+ }
+
+ @Override
+ public String toString() {
Review Comment:
`ReflectionToStringBuilderUtils`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] acs-robot commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
acs-robot commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1183690170
Found UI changes, kicking a new UI QA build
@blueorangutan ui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1184380538
> > > @DaanHoogland @rohityadavcloud @harikrishna-patnala Is JaCoCo failing for some particular reason ?
> >
> >
> > are you fully ready yet, @fermosan ? let's not worry about jococo untill we are ready to merge.
>
> Our tests look good. We are running some scenarios now. Surely, there will be another commit for the documentation pull in cloudstack-documentation.
>
> I am currently looking at the following scenario. Maybe you can suggest something. The backups are meant to expire at predefined retention days configured by the administrator on the Networker side. This will happen automatically and Cloudstack B&R Framework will be unaware of this. It looks like BackupSyncTask in B&R implementation can be utilized to do that.
Yes. That worked like a charm. Will soon commit and we will do some internal testing as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1272244537
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r991605147
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/NetworkerBackupProvider.java:
##########
@@ -0,0 +1,637 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.backup;
+
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.host.HostVO;
+import com.cloud.host.Status;
+import com.cloud.host.dao.HostDao;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.StoragePoolHostVO;
+import com.cloud.storage.Volume;
+import com.cloud.storage.VolumeVO;
+import com.cloud.storage.dao.StoragePoolHostDao;
+import com.cloud.storage.dao.VolumeDao;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.component.AdapterBase;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallbackNoReturn;
+import com.cloud.utils.db.TransactionStatus;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.ssh.SSHCmdHelper;
+import com.cloud.utils.ssh.SshException;
+import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.dao.VMInstanceDao;
+import org.apache.cloudstack.api.InternalIdentity;
+import org.apache.cloudstack.backup.dao.BackupDao;
+import org.apache.cloudstack.backup.dao.BackupOfferingDaoImpl;
+import org.apache.cloudstack.backup.networker.NetworkerClient;
+import org.apache.cloudstack.framework.config.ConfigKey;
+import org.apache.cloudstack.framework.config.Configurable;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.log4j.Logger;
+import org.apache.xml.utils.URI;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+import javax.inject.Inject;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.HashMap;
+import java.util.Date;
+import java.util.Objects;
+import java.util.UUID;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+public class NetworkerBackupProvider extends AdapterBase implements BackupProvider, Configurable {
+
+ public static final String BACKUP_IDENTIFIER = "-CSBKP-";
+ private static final Logger LOG = Logger.getLogger(NetworkerBackupProvider.class);
+ public ConfigKey<String> NetworkerUrl = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.url", "https://localhost:9090/nwrestapi/v3",
+ "The EMC Networker API URL.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerUsername = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.username", "administrator",
+ "The EMC Networker API username.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerPassword = new ConfigKey<>("Secure", String.class,
+ "backup.plugin.networker.password", "password",
+ "The EMC Networker API password.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerMediaPool = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.pool", "Default",
+ "The EMC Networker Media Pool", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerValidateSSLSecurity = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.validate.ssl", "false",
+ "Validate the SSL certificate when connecting to EMC Networker API service.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Integer> NetworkerApiRequestTimeout = new ConfigKey<>("Advanced", Integer.class,
+ "backup.plugin.networker.request.timeout", "300",
+ "The EMC Networker API request timeout in seconds.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerClientVerboseLogs = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.client.verbosity", "false",
+ "Produce Verbose logs in Hypervisor", true, ConfigKey.Scope.Zone);
+
+ @Inject
+ private BackupDao backupDao;
+
+ @Inject
+ private HostDao hostDao;
+
+ @Inject
+ private ConfigurationDao configDao;
+
+ @Inject
+ private ClusterDao clusterDao;
+
+ @Inject
+ private VolumeDao volumeDao;
+
+ @Inject
+ private StoragePoolHostDao storagePoolHostDao;
+
+ @Inject
+ private VMInstanceDao vmInstanceDao;
+
+ private static String getUrlDomain(String url) throws URISyntaxException {
+ URI uri;
+ try {
+ uri = new URI(url);
+ } catch (URI.MalformedURIException e) {
+ throw new CloudRuntimeException("Failed to cast URI");
+ }
+
+ return uri.getHost();
+ }
+
+ @Override
+ public ConfigKey<?>[] getConfigKeys() {
+ return new ConfigKey[]{
+ NetworkerUrl,
+ NetworkerUsername,
+ NetworkerPassword,
+ NetworkerValidateSSLSecurity,
+ NetworkerApiRequestTimeout,
+ NetworkerMediaPool,
+ NetworkerClientVerboseLogs
+ };
+ }
+
+ @Override
+ public String getName() {
+ return "networker";
+ }
+
+ @Override
+ public String getDescription() {
+ return "EMC Networker Backup Plugin";
+ }
+
+ @Override
+ public String getConfigComponentName() {
+ return BackupService.class.getSimpleName();
+ }
+
+ protected HostVO getLastVMHypervisorHost(VirtualMachine vm) {
+ HostVO host;
+ Long hostId = vm.getLastHostId();
+
+ if (hostId == null) {
+ LOG.debug("Cannot find last host for vm. This should never happen, please check your database.");
+ return null;
+ }
+ host = hostDao.findById(hostId);
+
+ if ( host.getStatus() == Status.Up ) {
+ return host;
+ } else {
+ // Try to find a host in the same cluster
+ List<HostVO> altClusterHosts = hostDao.findHypervisorHostInCluster(host.getClusterId());
+ for (final HostVO candidateClusterHost : altClusterHosts) {
+ if ( candidateClusterHost.getStatus() == Status.Up ) {
+ LOG.debug("Found Host " + candidateClusterHost.getName());
+ return candidateClusterHost;
+ }
+ }
+ }
+ // Try to find a Host in the zone
+ List<HostVO> altZoneHosts = hostDao.findByDataCenterId(host.getDataCenterId());
+ for (final HostVO candidateZoneHost : altZoneHosts) {
+ if ( candidateZoneHost.getStatus() == Status.Up && candidateZoneHost.getHypervisorType() == Hypervisor.HypervisorType.KVM ) {
+ LOG.debug("Found Host " + candidateZoneHost.getName());
+ return candidateZoneHost;
+ }
+ }
+ return null;
+ }
+
+ protected HostVO getRunningVMHypervisorHost(VirtualMachine vm) {
+
+ HostVO host;
+ Long hostId = vm.getHostId();
+
+ if (hostId == null) {
+ throw new CloudRuntimeException("Unable to find the HYPERVISOR for " + vm.getName() + ". Make sure the virtual machine is running");
+ }
+
+ host = hostDao.findById(hostId);
+
+ return host;
+ }
+
+ protected String getVMHypervisorCluster(HostVO host) {
+
+ return clusterDao.findById(host.getClusterId()).getName();
+ }
+
+ protected Ternary<String, String, String> getKVMHyperisorCredentials(HostVO host) {
+
+ String username = null;
+ String password = null;
+ String privateKey = null;
+
+ if (host != null && host.getHypervisorType() == Hypervisor.HypervisorType.KVM) {
+ hostDao.loadDetails(host);
+ password = host.getDetail("password");
+ username = host.getDetail("username");
+ privateKey = configDao.getValue("ssh.privatekey");
+ }
+ if ((password == null && privateKey == null) || username == null) {
+ throw new CloudRuntimeException("Cannot find login credentials for HYPERVISOR " + Objects.requireNonNull(host).getUuid());
+ }
+
+ return new Ternary<>(username, password, privateKey);
+ }
+
+ private String executeBackupCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+ String nstRegex = "\\bcompleted savetime=([0-9]{10})";
+ Pattern saveTimePattern = Pattern.compile(nstRegex);
+
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ Matcher saveTimeMatcher = saveTimePattern.matcher(result.getStdOut());
+ if (saveTimeMatcher.find()) {
+ return saveTimeMatcher.group(1);
+ }
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+
+ return null;
+ }
+ private boolean executeRestoreCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ return true;
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+ }
+
+ private NetworkerClient getClient(final Long zoneId) {
+ try {
+ return new NetworkerClient(NetworkerUrl.valueIn(zoneId), NetworkerUsername.valueIn(zoneId), NetworkerPassword.valueIn(zoneId),
+ NetworkerValidateSSLSecurity.valueIn(zoneId), NetworkerApiRequestTimeout.valueIn(zoneId));
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException("Failed to parse EMC Networker API URL: " + e.getMessage());
+ } catch (NoSuchAlgorithmException | KeyManagementException e) {
+ LOG.error("Failed to build EMC Networker API client due to: ", e);
+ }
+ throw new CloudRuntimeException("Failed to build EMC Networker API client");
+ }
+
+ @Override
+ public List<BackupOffering> listBackupOfferings(Long zoneId) {
+ List<BackupOffering> policies = new ArrayList<>();
+ for (final BackupOffering policy : getClient(zoneId).listPolicies()) {
+ if (!policy.getName().contains(BACKUP_IDENTIFIER)) {
+ policies.add(policy);
+ }
+ }
+
+ return policies;
+ }
+
+ @Override
+ public boolean isValidProviderOffering(Long zoneId, String uuid) {
+ List<BackupOffering> policies = listBackupOfferings(zoneId);
+ if (CollectionUtils.isEmpty(policies)) {
+ return false;
+ }
+ for (final BackupOffering policy : policies) {
+ if (Objects.equals(policy.getExternalId(), uuid)) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ @Override
+ public boolean assignVMToBackupOffering(VirtualMachine vm, BackupOffering backupOffering) { return true; }
+
+ @Override
+ public boolean removeVMFromBackupOffering(VirtualMachine vm) {
+ LOG.debug("Removing VirtualMachine from Backup offering and Deleting any existing backups");
+
+ List<String> backupsTaken = getClient(vm.getDataCenterId()).getBackupsForVm(vm);
+
+ for (String backupId : backupsTaken) {
+ LOG.debug("Trying to remove backup with id" + backupId);
+ getClient(vm.getDataCenterId()).deleteBackupForVM(backupId);
+ }
+
+ return true;
+ }
+
+ @Override
+ public boolean restoreVMFromBackup(VirtualMachine vm, Backup backup) {
+ LOG.debug("Restoring vm " + vm.getUuid() + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+ String networkerServer;
+ HostVO hostVO;
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return false;
+ }
+
+ // Find where the VM was last running
+ hostVO = getLastVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -S " + SSID;
+
+ if ( Boolean.TRUE.equals(NetworkerClientVerboseLogs.value()) )
+ command = command + " -v ";
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for VM ID " + vm.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return true;
+ } else {
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return false;
+ }
+ }
+
+ @Override
+ public Pair<Boolean, String> restoreBackedUpVolume(Backup backup, String volumeUuid, String hostIp, String dataStoreUuid) {
+ LOG.debug("Restoring volume " + volumeUuid + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+
+ String networkerServer;
+ VolumeVO volume = volumeDao.findByUuid(volumeUuid);
+ VMInstanceVO backupSourceVm = vmInstanceDao.findById(backup.getVmId());
+ StoragePoolHostVO dataStore = storagePoolHostDao.findByUuid(dataStoreUuid);
+ HostVO hostVO = hostDao.findByIp(hostIp);
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+ final String clusterName = networkerBackup.getClientHostname();
+ final String destinationNetworkerClient = hostVO.getName().split("\\.")[0];
+ Long restoredVolumeDiskSize = 0L;
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return null;
+ }
+
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ // Find volume size from backup vols
+ for ( Backup.VolumeInfo VMVolToRestore : backupSourceVm.getBackupVolumeList()) {
+ if (VMVolToRestore.getUuid().equals(volumeUuid))
+ restoredVolumeDiskSize = (VMVolToRestore.getSize());
+ }
+
+ VolumeVO restoredVolume = new VolumeVO(Volume.Type.DATADISK, null, backup.getZoneId(),
+ backup.getDomainId(), backup.getAccountId(), 0, null,
+ backup.getSize(), null, null, null);
+
+ restoredVolume.setName("RV-"+volume.getName());
+ restoredVolume.setProvisioningType(volume.getProvisioningType());
+ restoredVolume.setUpdated(new Date());
+ restoredVolume.setUuid(UUID.randomUUID().toString());
+ restoredVolume.setRemoved(null);
+ restoredVolume.setDisplayVolume(true);
+ restoredVolume.setPoolId(volume.getPoolId());
+ restoredVolume.setPath(restoredVolume.getUuid());
+ restoredVolume.setState(Volume.State.Copying);
+ restoredVolume.setSize(restoredVolumeDiskSize);
+ restoredVolume.setDiskOfferingId(volume.getDiskOfferingId());
+
+ try {
+ volumeDao.persist(restoredVolume);
+ } catch (Exception e) {
+ throw new CloudRuntimeException("Unable to craft restored volume due to: "+e);
+ }
+
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -c " + clusterName +
+ " -d " + destinationNetworkerClient +
+ " -n " + restoredVolume.getUuid() +
+ " -p " + dataStore.getLocalPath() +
+ " -a " + volume.getUuid();
+
+ if ( Boolean.TRUE.equals(NetworkerClientVerboseLogs.value()) )
+ command = command + " -v ";
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for Volume UUID " + volume.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return new Pair<>(true,restoredVolume.getUuid());
+ } else {
+ volumeDao.expunge(restoredVolume.getId());
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return null;
+ }
+ }
+
+ @Override
+ public boolean takeBackup(VirtualMachine vm) {
+
+ String networkerServer;
+ String clusterName;
+
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ // Find where the VM is currently running
+ HostVO hostVO = getRunningVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ // Get retention Period for our Backup
+ BackupOfferingVO vmBackupOffering = new BackupOfferingDaoImpl().findById(vm.getBackupOfferingId());
+ final String backupProviderPolicyId = vmBackupOffering.getExternalId();
+ String backupRentionPeriod = getClient(vm.getDataCenterId()).getBackupPolicyRetentionInterval(backupProviderPolicyId);
+
+ if ( backupRentionPeriod == null ) {
+ LOG.warn("There is no retention setting for Emc Networker Policy, setting default for 1 day");
+ backupRentionPeriod="1 Day";
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r991587067
##########
server/src/main/java/com/cloud/hypervisor/KVMGuru.java:
##########
@@ -139,7 +185,7 @@ protected void configureVmOsDescription(VirtualMachine virtualMachine, VirtualMa
guestOsMapping = _guestOsHypervisorDao.findByOsIdAndHypervisor(guestOS.getId(), getHypervisorType().toString(), hostVo.getHypervisorVersion());
}
- if (guestOsMapping == null || hostVo == null) {
+ if (guestOsMapping == null) {
Review Comment:
HostVo can never be null at this point asfaik.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r991675215
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/NetworkerBackupOffering.java:
##########
@@ -0,0 +1,75 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.backup.networker;
+
+import org.apache.cloudstack.backup.BackupOffering;
+import java.util.Date;
+public class NetworkerBackupOffering implements BackupOffering {
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1272735431
<b>Trillian test result (tid-5090)</b>
Environment: vmware-65u2 (x2), Advanced Networking with Mgmt server 7
Total time taken: 49189 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6550-t5090-vmware-65u2.zip
Smoke tests completed. 102 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_02_upgrade_kubernetes_cluster | `Failure` | 544.75 | test_kubernetes_clusters.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers | `Failure` | 591.73 | test_vpc_redundant.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] slavkap commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
slavkap commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r924175517
##########
server/src/main/java/com/cloud/hypervisor/KVMGuru.java:
##########
@@ -297,4 +309,49 @@ public Map<String, String> getClusterSettings(long vmId) {
return null;
}
-}
+ @Override
+ public VirtualMachine importVirtualMachineFromBackup(long zoneId, long domainId, long accountId, long userId, String vmInternalName, Backup backup) throws Exception {
+ s_logger.debug(String.format("Trying to import VM [vmInternalName: %s] from Backup [%s].", vmInternalName,
+ ReflectionToStringBuilderUtils.reflectOnlySelectedFields(backup, "id", "uuid", "vmId", "externalId", "backupType")));
+
+ VMInstanceVO vm = _instanceDao.findVMByInstanceNameIncludingRemoved(vmInternalName);
+ if (vm.getRemoved() != null) {
+ vm.setState(VirtualMachine.State.Stopped);
+ vm.setPowerState(VirtualMachine.PowerState.PowerOff);
+ _instanceDao.update(vm.getId(), vm);
+ _instanceDao.unremove(vm.getId());
+ }
+ for (final VolumeVO volume : _volumeDao.findIncludingRemovedByInstanceAndType(vm.getId(), null)) {
+ volume.setState(Volume.State.Ready);
+ volume.setAttached(new Date());
+ _volumeDao.update(volume.getId(), volume);
+ _volumeDao.unremove(volume.getId());
+ }
+
+ return vm;
+ }
+
+
+
+ @Override public boolean attachRestoredVolumeToVirtualMachine(long zoneId, String location, Backup.VolumeInfo volumeInfo, VirtualMachine vm, long poolId, Backup backup)
+ throws Exception {
+
+ VMInstanceVO targetVM = _instanceDao.findVMByInstanceNameIncludingRemoved(vm.getName());
+ List<VolumeVO> devices = _volumeDao.findIncludingRemovedByInstanceAndType(targetVM.getId(), null);
+ VolumeVO restoredVolume = _volumeDao.findByUuid(location);
+ Integer deviceId = devices.size();
+
+
+ if (restoredVolume != null) {
+ restoredVolume.setState(Volume.State.Ready);
+ _volumeDao.update(restoredVolume.getId(), restoredVolume);
+ try {
+ _volumeDao.attachVolume(restoredVolume.getId(), vm.getId(), deviceId);
Review Comment:
Hi, @fermosan, I'm not sure, but I think you will have a problem if you have a VM with more volumes and restore them at some point. They will have the same `deviceId`. The issue will appear at the start of the VM when the `LibvirtComputingResource::createVbd` is invoked.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r924415617
##########
server/src/main/java/com/cloud/hypervisor/KVMGuru.java:
##########
@@ -297,4 +309,49 @@ public Map<String, String> getClusterSettings(long vmId) {
return null;
}
-}
+ @Override
+ public VirtualMachine importVirtualMachineFromBackup(long zoneId, long domainId, long accountId, long userId, String vmInternalName, Backup backup) throws Exception {
+ s_logger.debug(String.format("Trying to import VM [vmInternalName: %s] from Backup [%s].", vmInternalName,
+ ReflectionToStringBuilderUtils.reflectOnlySelectedFields(backup, "id", "uuid", "vmId", "externalId", "backupType")));
+
+ VMInstanceVO vm = _instanceDao.findVMByInstanceNameIncludingRemoved(vmInternalName);
+ if (vm.getRemoved() != null) {
+ vm.setState(VirtualMachine.State.Stopped);
+ vm.setPowerState(VirtualMachine.PowerState.PowerOff);
+ _instanceDao.update(vm.getId(), vm);
+ _instanceDao.unremove(vm.getId());
+ }
+ for (final VolumeVO volume : _volumeDao.findIncludingRemovedByInstanceAndType(vm.getId(), null)) {
+ volume.setState(Volume.State.Ready);
+ volume.setAttached(new Date());
+ _volumeDao.update(volume.getId(), volume);
+ _volumeDao.unremove(volume.getId());
+ }
+
+ return vm;
+ }
+
+
+
+ @Override public boolean attachRestoredVolumeToVirtualMachine(long zoneId, String location, Backup.VolumeInfo volumeInfo, VirtualMachine vm, long poolId, Backup backup)
+ throws Exception {
+
+ VMInstanceVO targetVM = _instanceDao.findVMByInstanceNameIncludingRemoved(vm.getName());
+ List<VolumeVO> devices = _volumeDao.findIncludingRemovedByInstanceAndType(targetVM.getId(), null);
+ VolumeVO restoredVolume = _volumeDao.findByUuid(location);
+ Integer deviceId = devices.size();
+
+
+ if (restoredVolume != null) {
+ restoredVolume.setState(Volume.State.Ready);
+ _volumeDao.update(restoredVolume.getId(), restoredVolume);
+ try {
+ _volumeDao.attachVolume(restoredVolume.getId(), vm.getId(), deviceId);
Review Comment:
Hello @slavkap. We can test this if you think of a specific test case. attachRestoredVolumeToVirtualMachine() is supposed to be called for only one volume at a time from B&R framework (attachVolumeToVM in class BackupManagerImpl).
Each time it is called and depending on the currently attached devices for that vm it will return the next counter.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1298051321
Thanks @fermosan your comments about the framework level provider/settings makes sense (I forgot that was already in place). Kindly review other outstanding comments, I think this is ready for merging once you address them. cc @DaanHoogland @nvazquez
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1303280372
@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] acs-robot commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
acs-robot commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1303273801
Found UI changes, kicking a new UI QA build
@blueorangutan ui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1281695344
UI build: :heavy_check_mark:
Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6550 (SL-JID-2523)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1216519287
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 4000
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1196670817
@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1257264526
> > @DaanHoogland @rohityadavcloud Hello, how we can kick in sonar ?
>
> @fermosan , maybe do as @rohityadavcloud suggested and rebase your code on latest, then force push it to your branch. Did you manage to run sonar locally?
@DaanHoogland We should now be rebased. Can you confirm all is ok so we can move on with the test of the review notes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r979468810
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/NetworkerBackupProvider.java:
##########
@@ -0,0 +1,640 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.backup;
+
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.host.HostVO;
+import com.cloud.host.Status;
+import com.cloud.host.dao.HostDao;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.StoragePoolHostVO;
+import com.cloud.storage.Volume;
+import com.cloud.storage.VolumeVO;
+import com.cloud.storage.dao.StoragePoolHostDao;
+import com.cloud.storage.dao.VolumeDao;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.component.AdapterBase;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallbackNoReturn;
+import com.cloud.utils.db.TransactionStatus;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.ssh.SSHCmdHelper;
+import com.cloud.utils.ssh.SshException;
+import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.dao.VMInstanceDao;
+import org.apache.cloudstack.api.InternalIdentity;
+import org.apache.cloudstack.backup.dao.BackupDao;
+import org.apache.cloudstack.backup.dao.BackupOfferingDaoImpl;
+import org.apache.cloudstack.backup.networker.NetworkerClient;
+import org.apache.cloudstack.framework.config.ConfigKey;
+import org.apache.cloudstack.framework.config.Configurable;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.log4j.Logger;
+import org.apache.xml.utils.URI;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+
+
+import javax.inject.Inject;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.HashMap;
+import java.util.Date;
+import java.util.Objects;
+import java.util.UUID;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+
+public class NetworkerBackupProvider extends AdapterBase implements BackupProvider, Configurable {
+
+ public static final String BACKUP_IDENTIFIER = "-CSBKP-";
+ private static final Logger LOG = Logger.getLogger(NetworkerBackupProvider.class);
+ public ConfigKey<String> NetworkerUrl = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.url", "https://localhost:9090/nwrestapi/v3",
+ "The EMC Networker API URL.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerUsername = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.username", "administrator",
+ "The EMC Networker API username.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerPassword = new ConfigKey<>("Secure", String.class,
+ "backup.plugin.networker.password", "password",
+ "The EMC Networker API password.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerMediaPool = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.pool", "Default",
+ "The EMC Networker Media Pool", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerValidateSSLSecurity = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.validate.ssl", "false",
+ "Validate the SSL certificate when connecting to EMC Networker API service.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Integer> NetworkerApiRequestTimeout = new ConfigKey<>("Advanced", Integer.class,
+ "backup.plugin.networker.request.timeout", "300",
+ "The EMC Networker API request timeout in seconds.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerClientVerboseLogs = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.client.verbosity", "false",
+ "Produce Verbose logs in Hypervisor", true, ConfigKey.Scope.Zone);
+
+ @Inject
+ private BackupDao backupDao;
+
+ @Inject
+ private HostDao hostDao;
+
+ @Inject
+ private ConfigurationDao configDao;
+
+ @Inject
+ private ClusterDao clusterDao;
+
+ @Inject
+ private VolumeDao volumeDao;
+
+ @Inject
+ private StoragePoolHostDao storagePoolHostDao;
+
+ @Inject
+ private VMInstanceDao vmInstanceDao;
+
+ private static String getUrlDomain(String url) throws URISyntaxException {
+ URI uri;
+ try {
+ uri = new URI(url);
+ } catch (URI.MalformedURIException e) {
+ throw new CloudRuntimeException("Failed to cast URI");
+ }
+
+ return uri.getHost();
+ }
+
+ @Override
+ public ConfigKey<?>[] getConfigKeys() {
+ return new ConfigKey[]{
+ NetworkerUrl,
+ NetworkerUsername,
+ NetworkerPassword,
+ NetworkerValidateSSLSecurity,
+ NetworkerApiRequestTimeout,
+ NetworkerMediaPool,
+ NetworkerClientVerboseLogs
+ };
+ }
+
+ @Override
+ public String getName() {
+ return "networker";
+ }
+
+ @Override
+ public String getDescription() {
+ return "EMC Networker Backup Plugin";
+ }
+
+ @Override
+ public String getConfigComponentName() {
+ return BackupService.class.getSimpleName();
+ }
+
+ protected HostVO getLastVMHypervisorHost(VirtualMachine vm) {
+ HostVO host;
+ Long hostId = vm.getLastHostId();
+
+ if (hostId == null) {
+ LOG.debug("Cannot find last host for vm. This should never happen, please check your database.");
+ return null;
+ }
+ host = hostDao.findById(hostId);
+
+ if ( host.getStatus() == Status.Up ) {
+ return host;
+ } else {
+ // Try to find a host in the same cluster
+ List<HostVO> altClusterHosts = hostDao.findHypervisorHostInCluster(host.getClusterId());
+ for (final HostVO candidateClusterHost : altClusterHosts) {
+ if ( candidateClusterHost.getStatus() == Status.Up ) {
+ LOG.debug("Found Host " + candidateClusterHost.getName());
+ return candidateClusterHost;
+ }
+ }
+ }
+ // Try to find a Host in the zone
+ List<HostVO> altZoneHosts = hostDao.findByDataCenterId(host.getDataCenterId());
+ for (final HostVO candidateZoneHost : altZoneHosts) {
+ if ( candidateZoneHost.getStatus() == Status.Up && candidateZoneHost.getHypervisorType() == Hypervisor.HypervisorType.KVM ) {
+ LOG.debug("Found Host " + candidateZoneHost.getName());
+ return candidateZoneHost;
+ }
+ }
+ return null;
+ }
+
+ protected HostVO getRunningVMHypervisorHost(VirtualMachine vm) {
+
+ HostVO host;
+ Long hostId = vm.getHostId();
+
+ if (hostId == null) {
+ throw new CloudRuntimeException("Unable to find the HYPERVISOR for " + vm.getName() + ". Make sure the virtual machine is running");
+ }
+
+ host = hostDao.findById(hostId);
+
+ return host;
+ }
+
+ protected String getVMHypervisorCluster(HostVO host) {
+
+ return clusterDao.findById(host.getClusterId()).getName();
+ }
+
+ protected Ternary<String, String, String> getKVMHyperisorCredentials(HostVO host) {
+
+ String username = null;
+ String password = null;
+ String privateKey = null;
+
+ if (host != null && host.getHypervisorType() == Hypervisor.HypervisorType.KVM) {
+ hostDao.loadDetails(host);
+ password = host.getDetail("password");
+ username = host.getDetail("username");
+ privateKey = configDao.getValue("ssh.privatekey");
+ }
+ if ((password == null && privateKey == null) || username == null) {
+ throw new CloudRuntimeException("Cannot find login credentials for HYPERVISOR " + Objects.requireNonNull(host).getUuid());
+ }
+
+ return new Ternary<>(username, password, privateKey);
+ }
+
+ private String executeBackupCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+ String nstRegex = "\\bcompleted savetime=([0-9]{10})";
+ Pattern saveTimePattern = Pattern.compile(nstRegex);
+
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ Matcher saveTimeMatcher = saveTimePattern.matcher(result.getStdOut());
+ if (saveTimeMatcher.find()) {
+ return saveTimeMatcher.group(1);
+ }
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+
+ return null;
+ }
+ private boolean executeRestoreCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ return true;
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+ }
+
+ private NetworkerClient getClient(final Long zoneId) {
+ try {
+ return new NetworkerClient(NetworkerUrl.valueIn(zoneId), NetworkerUsername.valueIn(zoneId), NetworkerPassword.valueIn(zoneId),
+ NetworkerValidateSSLSecurity.valueIn(zoneId), NetworkerApiRequestTimeout.valueIn(zoneId));
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException("Failed to parse EMC Networker API URL: " + e.getMessage());
+ } catch (NoSuchAlgorithmException | KeyManagementException e) {
+ LOG.error("Failed to build EMC Networker API client due to: ", e);
+ }
+ throw new CloudRuntimeException("Failed to build EMC Networker API client");
+ }
+
+ @Override
+ public List<BackupOffering> listBackupOfferings(Long zoneId) {
+ List<BackupOffering> policies = new ArrayList<>();
+ for (final BackupOffering policy : getClient(zoneId).listPolicies()) {
+ if (!policy.getName().contains(BACKUP_IDENTIFIER)) {
+ policies.add(policy);
+ }
+ }
+
+ return policies;
+ }
+
+ @Override
+ public boolean isValidProviderOffering(Long zoneId, String uuid) {
+ List<BackupOffering> policies = listBackupOfferings(zoneId);
+ if (CollectionUtils.isEmpty(policies)) {
+ return false;
+ }
+ for (final BackupOffering policy : policies) {
+ if (Objects.equals(policy.getExternalId(), uuid)) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ @Override
+ public boolean assignVMToBackupOffering(VirtualMachine vm, BackupOffering backupOffering) { return true; }
+
+ @Override
+ public boolean removeVMFromBackupOffering(VirtualMachine vm) {
+ LOG.debug("Removing VirtualMachine from Backup offering and Deleting any existing backups");
+
+ List<String> backupsTaken = getClient(vm.getDataCenterId()).getBackupsForVm(vm);
+
+ for (String backupId : backupsTaken) {
+ LOG.debug("Trying to remove backup with id" + backupId);
+ getClient(vm.getDataCenterId()).deleteBackupForVM(backupId);
+ }
+
+ return true;
+ }
+
+ @Override
+ public boolean restoreVMFromBackup(VirtualMachine vm, Backup backup) {
+ LOG.debug("Restoring vm " + vm.getUuid() + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+ String networkerServer;
+ HostVO hostVO;
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return false;
+ }
+
+ // Find where the VM was last running
+ hostVO = getLastVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -S " + SSID;
+
+ if (NetworkerClientVerboseLogs.value())
+ command = command + " -v ";
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for VM ID " + vm.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return true;
+ } else {
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return false;
+ }
+ }
+
+ @Override
+ public Pair<Boolean, String> restoreBackedUpVolume(Backup backup, String volumeUuid, String hostIp, String dataStoreUuid) {
+ LOG.debug("Restoring volume " + volumeUuid + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+
+ String networkerServer;
+ VolumeVO volume = volumeDao.findByUuid(volumeUuid);
+ VMInstanceVO backupSourceVm = vmInstanceDao.findById(backup.getVmId());
+ StoragePoolHostVO dataStore = storagePoolHostDao.findByUuid(dataStoreUuid);
+ HostVO hostVO = hostDao.findByIp(hostIp);
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+ final String clusterName = networkerBackup.getClientHostname();
+ final String destinationNetworkerClient = hostVO.getName().split("\\.")[0];
+ Long restoredVolumeDiskSize = 0L;
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return null;
+ }
+
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ // Find volume size from backup vols
+ for ( Backup.VolumeInfo VMVolToRestore : backupSourceVm.getBackupVolumeList()) {
+ if (VMVolToRestore.getUuid().equals(volumeUuid))
+ restoredVolumeDiskSize = (VMVolToRestore.getSize());
+ }
+
+ VolumeVO restoredVolume = new VolumeVO(Volume.Type.DATADISK, null, backup.getZoneId(),
+ backup.getDomainId(), backup.getAccountId(), 0, null,
+ backup.getSize(), null, null, null);
+
+ restoredVolume.setName("RV-"+volume.getName());
+ restoredVolume.setProvisioningType(volume.getProvisioningType());
+ restoredVolume.setUpdated(new Date());
+ restoredVolume.setUuid(UUID.randomUUID().toString());
+ restoredVolume.setRemoved(null);
+ restoredVolume.setDisplayVolume(true);
+ restoredVolume.setPoolId(volume.getPoolId());
+ restoredVolume.setPath(restoredVolume.getUuid());
+ restoredVolume.setState(Volume.State.Copying);
+ restoredVolume.setSize(restoredVolumeDiskSize);
+ restoredVolume.setDiskOfferingId(volume.getDiskOfferingId());
+
+ try {
+ volumeDao.persist(restoredVolume);
+ } catch (Exception e) {
+ throw new CloudRuntimeException("Unable to craft restored volume due to: "+e);
+ }
+
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
Review Comment:
Will have a look
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1257602135
Yes @fermosan, all looks good, I'll re-run the regression tests.
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1283902326
@blueorangutan test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1367144795
@DaanHoogland a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1370964661
@DaanHoogland a Trillian-Jenkins matrix job (centos7 mgmt + xenserver71, rocky8 mgmt + vmware67u3, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1348015638
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1348028306
@rohityadavcloud @weizhouapache @shwstppr @fermosan is there anything left to discuss here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] harikrishna-patnala commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
harikrishna-patnala commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1182730281
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r920544812
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/ServerBackup.java:
##########
@@ -0,0 +1,178 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "performBootstrap",
+ "performClientFileIndexing",
+ "destinationStorageNode",
+ "retentionPeriod",
+ "destinationPool"
+})
+@Generated("jsonschema2pojo")
+public class ServerBackup implements Serializable {
+
+ private final static long serialVersionUID = -542556595701706880L;
+ @JsonProperty("performBootstrap")
+ private Boolean performBootstrap;
+ @JsonProperty("performClientFileIndexing")
+ private Boolean performClientFileIndexing;
+ @JsonProperty("destinationStorageNode")
+ private String destinationStorageNode;
+ @JsonProperty("retentionPeriod")
+ private String retentionPeriod;
+ @JsonProperty("destinationPool")
+ private String destinationPool;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public ServerBackup() {
+ }
+
+ /**
+ * @param destinationPool
+ * @param performBootstrap
+ * @param performClientFileIndexing
+ * @param destinationStorageNode
+ * @param retentionPeriod
+ */
+ public ServerBackup(Boolean performBootstrap, Boolean performClientFileIndexing, String destinationStorageNode, String retentionPeriod, String destinationPool) {
+ super();
+ this.performBootstrap = performBootstrap;
+ this.performClientFileIndexing = performClientFileIndexing;
+ this.destinationStorageNode = destinationStorageNode;
+ this.retentionPeriod = retentionPeriod;
+ this.destinationPool = destinationPool;
+ }
+
+ @JsonProperty("performBootstrap")
+ public Boolean getPerformBootstrap() {
+ return performBootstrap;
+ }
+
+ @JsonProperty("performBootstrap")
+ public void setPerformBootstrap(Boolean performBootstrap) {
+ this.performBootstrap = performBootstrap;
+ }
+
+ @JsonProperty("performClientFileIndexing")
+ public Boolean getPerformClientFileIndexing() {
+ return performClientFileIndexing;
+ }
+
+ @JsonProperty("performClientFileIndexing")
+ public void setPerformClientFileIndexing(Boolean performClientFileIndexing) {
+ this.performClientFileIndexing = performClientFileIndexing;
+ }
+
+ @JsonProperty("destinationStorageNode")
+ public String getDestinationStorageNode() {
+ return destinationStorageNode;
+ }
+
+ @JsonProperty("destinationStorageNode")
+ public void setDestinationStorageNode(String destinationStorageNode) {
+ this.destinationStorageNode = destinationStorageNode;
+ }
+
+ @JsonProperty("retentionPeriod")
+ public String getRetentionPeriod() {
+ return retentionPeriod;
+ }
+
+ @JsonProperty("retentionPeriod")
+ public void setRetentionPeriod(String retentionPeriod) {
+ this.retentionPeriod = retentionPeriod;
+ }
+
+ @JsonProperty("destinationPool")
+ public String getDestinationPool() {
+ return destinationPool;
+ }
+
+ @JsonProperty("destinationPool")
+ public void setDestinationPool(String destinationPool) {
+ this.destinationPool = destinationPool;
+ }
+
+ @Override
+ public String toString() {
+ StringBuilder sb = new StringBuilder();
+ sb.append(ServerBackup.class.getName()).append('@').append(Integer.toHexString(System.identityHashCode(this))).append('[');
+ sb.append("performBootstrap");
+ sb.append('=');
+ sb.append(((this.performBootstrap == null) ? "<null>" : this.performBootstrap));
+ sb.append(',');
+ sb.append("performClientFileIndexing");
+ sb.append('=');
+ sb.append(((this.performClientFileIndexing == null) ? "<null>" : this.performClientFileIndexing));
+ sb.append(',');
+ sb.append("destinationStorageNode");
+ sb.append('=');
+ sb.append(((this.destinationStorageNode == null) ? "<null>" : this.destinationStorageNode));
+ sb.append(',');
+ sb.append("retentionPeriod");
+ sb.append('=');
+ sb.append(((this.retentionPeriod == null) ? "<null>" : this.retentionPeriod));
+ sb.append(',');
+ sb.append("destinationPool");
+ sb.append('=');
+ sb.append(((this.destinationPool == null) ? "<null>" : this.destinationPool));
+ sb.append(',');
+ if (sb.charAt((sb.length() - 1)) == ',') {
+ sb.setCharAt((sb.length() - 1), ']');
+ } else {
+ sb.append(']');
+ }
+ return sb.toString();
+ }
+
+ @Override
+ public int hashCode() {
+ int result = 1;
+ result = ((result * 31) + ((this.performBootstrap == null) ? 0 : this.performBootstrap.hashCode()));
+ result = ((result * 31) + ((this.performClientFileIndexing == null) ? 0 : this.performClientFileIndexing.hashCode()));
+ result = ((result * 31) + ((this.destinationPool == null) ? 0 : this.destinationPool.hashCode()));
+ result = ((result * 31) + ((this.destinationStorageNode == null) ? 0 : this.destinationStorageNode.hashCode()));
+ result = ((result * 31) + ((this.retentionPeriod == null) ? 0 : this.retentionPeriod.hashCode()));
+ return result;
+ }
+
+ @Override
+ public boolean equals(Object other) {
+ if (other == this) {
+ return true;
+ }
+ if ((other instanceof ServerBackup) == false) {
+ return false;
+ }
+ ServerBackup rhs = ((ServerBackup) other);
+ return ((((((this.performBootstrap == rhs.performBootstrap) || ((this.performBootstrap != null) && this.performBootstrap.equals(rhs.performBootstrap))) && ((this.performClientFileIndexing == rhs.performClientFileIndexing) || ((this.performClientFileIndexing != null) && this.performClientFileIndexing.equals(rhs.performClientFileIndexing)))) && ((this.destinationPool == rhs.destinationPool) || ((this.destinationPool != null) && this.destinationPool.equals(rhs.destinationPool)))) && ((this.destinationStorageNode == rhs.destinationStorageNode) || ((this.destinationStorageNode != null) && this.destinationStorageNode.equals(rhs.destinationStorageNode)))) && ((this.retentionPeriod == rhs.retentionPeriod) || ((this.retentionPeriod != null) && this.retentionPeriod.equals(rhs.retentionPeriod))));
Review Comment:
Will be removed since it is not used anywhere.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1200976431
UI build: :heavy_check_mark:
Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6550 (SL-JID-2042)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1185668045
@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1283902601
@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1284743163
<b>Trillian test result (tid-5170)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 44096 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6550-t5170-kvm-centos7.zip
Smoke tests completed. 102 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_08_upgrade_kubernetes_ha_cluster | `Failure` | 621.75 | test_kubernetes_clusters.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers | `Failure` | 453.88 | test_vpc_redundant.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1283695723
> @fermosan did you answer all @rohityadavcloud 's comments? (i.e. do you guys agree this is ready or not?)
@DaanHoogland @rohityadavcloud. There are two pending comments regarding NPEs for minCores and minMemory. I did not implemented them, they were there. I haven't looked all the code to determine if those values can ever be undefined.
We redeployed all the code (with current branch and fixes) to a new environment and run all tests again. Everything was found in good order.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1303287377
> >
>
> > ping @fermosan you see any chance to address @rohityadavcloud 's remaining comments?
>
> Yes. @DaanHoogland Updates: if (guestOsMapping == null || hostVo == null) will be reverted in a coming patch
>
> I am trying to see if we can use the Script.findScript(). I cannot find a way to make this work. The function is called on the management server side and the script cannot be found since it exists on the client. Furthermore in the development environment the lookup path is under the local development directories. There is now straight way to get "kvm.script.dir" or "scripts.path" from environ since configure for super is not there. If there is no way for this to work the full path will be used there. Any ideas ?
>
> I am also working on eliminating the need for sudo execution on the KVM host side (needs some work/testing and more conf in the EMC networker side). If successful, i will commit here and update the documentation pull as well to reflect that.
>
> For the [ui/public/config.json] I will revert the changes and match main in a coming patch.
>
> Finally for the comments about
>
> int minCpuCores = virtualMachineTo.getCpus(); long minMemory = virtualMachineTo.getMinRam();
>
> The code was there beforehand and as far as I can see it is not used by any other components. It is however used in the tests. I am not sure what is supposed to do and which components are supposed to call it if any. Any inputs ? How we going to proceed on that ? My guess is that it has to do something with the computer offerings. Can those be null under any circumstance ?
>
> Maybe @nvazquez has some input for the above.
@DaanHoogland @rohityadavcloud
- Reverted config.json and HostVO null checks
- Eliminated the need to run sudo from cs side. Documentation pull is also updated.
Only two remaining points of discussion are:
minCpuCores / minMemory
If anyone has any input about the usability of the supposed usability of those in KVMGuru let me know.
script.findScr
Couldn't find a way to utilize it from the management side. Will give it another try. If not it will remain as it is for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud closed pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud closed pull request #6550: Emc networker b&r
URL: https://github.com/apache/cloudstack/pull/6550
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1364562721
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1362635420
@blueorangutan package
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1364569229
Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: el9 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 5085
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1367591539
<b>Trillian test result (tid-5687)</b>
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 38887 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6550-t5687-xenserver-71.zip
Smoke tests completed. 106 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1359236796
> Trillian test result (tid-5567) Environment: vmware-65u2 (x2), Advanced Networking with Mgmt server 7 Total time taken: 53426 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6550-t5567-vmware-65u2.zip Smoke tests completed. 103 look OK, 2 have errors, 0 did not run Only failed and skipped tests results shown below:
> Test Result Time (s) Test File
> test_02_create_template_with_checksum_sha1 `Error` 5.16 test_templates.py
> test_03_create_template_with_checksum_sha256 `Error` 5.17 test_templates.py
> test_04_create_template_with_checksum_md5 `Error` 5.15 test_templates.py
> test_02_upgrade_kubernetes_cluster `Failure` 537.80 test_kubernetes_clusters.py
> test_08_upgrade_kubernetes_ha_cluster `Error` 4093.32 test_kubernetes_clusters.py
> test_09_delete_kubernetes_ha_cluster `Failure` 0.05 test_kubernetes_clusters.py
> ContextSuite context=TestKubernetesCluster>:teardown `Error` 45.96 test_kubernetes_clusters.py
the test_templates errors are handled in #7001
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1364070134
@blueorangutan test matrix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland closed pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland closed pull request #6550: Emc networker b&r
URL: https://github.com/apache/cloudstack/pull/6550
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r951218979
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/NetworkerClient.java:
##########
@@ -0,0 +1,360 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.cloudstack.backup.networker;
+
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nio.TrustAllManager;
+import com.cloud.vm.VirtualMachine;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.cloudstack.api.ApiErrorCode;
+import org.apache.cloudstack.api.ServerApiException;
+import org.apache.cloudstack.backup.BackupOffering;
+import org.apache.cloudstack.backup.BackupVO;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackups;
+import org.apache.cloudstack.backup.networker.api.ProtectionPolicies;
+import org.apache.cloudstack.backup.networker.api.ProtectionPolicy;
+import org.apache.cloudstack.utils.security.SSLUtils;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpResponse;
+import org.apache.http.HttpStatus;
+import org.apache.http.client.HttpClient;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.HttpDelete;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.conn.ConnectTimeoutException;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.impl.client.HttpClientBuilder;
+import org.apache.log4j.Logger;
+
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.X509TrustManager;
+import java.io.IOException;
+import java.net.SocketTimeoutException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.security.SecureRandom;
+import java.text.ParseException;
+import java.text.SimpleDateFormat;
+import java.time.Instant;
+import java.util.ArrayList;
+import java.util.Base64;
+import java.util.Date;
+import java.util.List;
+
+import static org.apache.cloudstack.backup.NetworkerBackupProvider.BACKUP_IDENTIFIER;
+
+
+public class NetworkerClient {
+
+
+ private static final Logger LOG = Logger.getLogger(NetworkerClient.class);
+
+ private final URI apiURI;
+ private final String apiName;
+ private final String apiPassword;
+
+ private final HttpClient httpClient;
+
+
+ public NetworkerClient(final String url, final String username, final String password, final boolean validateCertificate, final int timeout) throws URISyntaxException, NoSuchAlgorithmException, KeyManagementException {
+
+ apiName = username;
+ apiPassword = password;
+
+ this.apiURI = new URI(url);
+ final RequestConfig config = RequestConfig.custom()
+ .setConnectTimeout(timeout * 1000)
+ .setConnectionRequestTimeout(timeout * 1000)
+ .setSocketTimeout(timeout * 1000)
+ .build();
+
+ if (!validateCertificate) {
+ final SSLContext sslcontext = SSLUtils.getSSLContext();
+ sslcontext.init(null, new X509TrustManager[]{new TrustAllManager()}, new SecureRandom());
+ final SSLConnectionSocketFactory factory = new SSLConnectionSocketFactory(sslcontext, NoopHostnameVerifier.INSTANCE);
+ this.httpClient = HttpClientBuilder.create()
+ .setDefaultRequestConfig(config)
+ .setSSLSocketFactory(factory)
+ .build();
+ } else {
+ this.httpClient = HttpClientBuilder.create()
+ .setDefaultRequestConfig(config)
+ .build();
+ }
+
+ authenticate(username, password);
+ }
+
+ private void authenticate(final String username, final String password) {
+
+ final HttpGet request = new HttpGet(apiURI.toString());
+ request.setHeader(HttpHeaders.AUTHORIZATION, "Basic " + Base64.getEncoder().encodeToString((username + ":" + password).getBytes()));
+ request.setHeader(HttpHeaders.ACCEPT, "application/json");
+ request.setHeader(HttpHeaders.USER_AGENT, "CloudStack B&R");
+ request.setHeader(HttpHeaders.CONNECTION, "keep-alive");
+ try {
+ final HttpResponse response = httpClient.execute(request);
+ checkAuthFailure(response);
+ if (response.getStatusLine().getStatusCode() != HttpStatus.SC_OK) {
+ throw new CloudRuntimeException("Failed to create and authenticate EMC Networker API client, please check the settings.");
+ }
+ } catch (final IOException e) {
+ throw new CloudRuntimeException("Failed to authenticate Networker API service due to:" + e.getMessage());
+ }
+ }
+
+ private void checkAuthFailure(final HttpResponse response) {
+ if (response != null && response.getStatusLine().getStatusCode() == HttpStatus.SC_UNAUTHORIZED) {
+ throw new ServerApiException(ApiErrorCode.UNAUTHORIZED, "EMC Networker B&R API call unauthorized. Check username/password or contact your backup administrator.");
+ }
+ }
+
+ private void checkResponseOK(final HttpResponse response) {
+ if (response.getStatusLine().getStatusCode() == HttpStatus.SC_NO_CONTENT) {
+ LOG.debug("Requested EMC Networker resource does not exist");
+ return;
+ }
+ if (!(response.getStatusLine().getStatusCode() == HttpStatus.SC_OK ||
+ response.getStatusLine().getStatusCode() == HttpStatus.SC_ACCEPTED) &&
+ response.getStatusLine().getStatusCode() != HttpStatus.SC_NO_CONTENT) {
+ LOG.debug(String.format("HTTP request failed, status code is [%s], response is: [%s].", response.getStatusLine().getStatusCode(), response));
+ throw new ServerApiException(ApiErrorCode.INTERNAL_ERROR, "Got invalid API status code returned by the EMC Networker server");
+ }
+ }
+
+ private void checkResponseTimeOut(final Exception e) {
+ if (e instanceof ConnectTimeoutException || e instanceof SocketTimeoutException) {
+ throw new ServerApiException(ApiErrorCode.RESOURCE_UNAVAILABLE_ERROR, "EMC Networker API operation timed out, please try again.");
+ }
+ }
+
+ private HttpResponse get(final String path) throws IOException {
+ String url = apiURI.toString() + path;
+ final HttpGet request = new HttpGet(url);
+ request.setHeader(HttpHeaders.AUTHORIZATION, "Basic " + Base64.getEncoder().encodeToString((apiName + ":" + apiPassword).getBytes()));
+ request.setHeader(HttpHeaders.ACCEPT, "application/json");
+ request.setHeader(HttpHeaders.USER_AGENT, "CloudStack B&R");
+ final HttpResponse response = httpClient.execute(request);
+ checkAuthFailure(response);
+
+ LOG.debug(String.format("Response received in GET request is: [%s] for URL: [%s].", response.toString(), url));
+ return response;
+ }
+
+ public String getBackupPolicyRetentionInterval(String externalId) {
+ try {
+ final HttpResponse response = get("/global/protectionpolicies/?q=comment:" + BACKUP_IDENTIFIER);
+ checkResponseOK(response);
+ final ObjectMapper jsonMapper = new ObjectMapper();
+ jsonMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
+
+ final ProtectionPolicies protectionPolicies = jsonMapper.readValue(response.getEntity().getContent(), ProtectionPolicies.class);
+
+ if (protectionPolicies == null || protectionPolicies.getProtectionPolicies() == null) {
+ return null;
+ }
+ for (final ProtectionPolicy protectionPolicy : protectionPolicies.getProtectionPolicies()) {
+ if ( protectionPolicy.getResourceId().getId().equals(externalId)) {
+ return protectionPolicy.getPolicyProtectionPeriod();
+ }
+ }
+ } catch (final IOException e) {
+ LOG.error("Failed to get Protection Policy Period from EMC Networker due to:", e);
+ checkResponseTimeOut(e);
+ }
+ return null;
+ }
+ private HttpResponse delete(final String path) throws IOException {
+ String url = apiURI.toString() + path;
+ final HttpDelete request = new HttpDelete(url);
+ request.setHeader(HttpHeaders.AUTHORIZATION, "Basic " + Base64.getEncoder().encodeToString((apiName + ":" + apiPassword).getBytes()));
+ request.setHeader(HttpHeaders.USER_AGENT, "CloudStack B&R");
+ final HttpResponse response = httpClient.execute(request);
+ checkAuthFailure(response);
+
+ LOG.debug(String.format("Response received in DELETE request is: [%s] for URL [%s].", response.toString(), url));
+ return response;
+ }
+
+ public boolean deleteBackupForVM(String externalId) {
+ try {
+ final HttpResponse response = delete("/global/backups/" + externalId);
+ checkResponseOK(response);
+ return true;
+ } catch (final IOException e) {
+ LOG.error("Failed to delete backup from EMC Networker due to:", e);
+ checkResponseTimeOut(e);
+ }
+ return false;
+ }
+
+
+ public BackupVO registerBackupForVm(VirtualMachine vm, Date backupJobStart, String saveTime) {
+ LOG.debug("Querying EMC Networker about latest backup");
+
+ NetworkerBackups networkerBackups;
+ BackupVO backup = new BackupVO();
+
+ SimpleDateFormat formatterDate = new SimpleDateFormat("yyyy-MM-dd");
+ SimpleDateFormat formatterTime = new SimpleDateFormat("HH:mm:ss");
+ SimpleDateFormat formatterDateTime = new SimpleDateFormat("yyy-MM-dd'T'HH:mm:ss");
+
+ String startDate = formatterDate.format(backupJobStart);
+ String startTime = formatterTime.format(backupJobStart);
+ String endDate = formatterDate.format(new Date());
+ String endTime = formatterTime.format(new Date());
+
+ final String searchRange = "['" + startDate + "T" + startTime + "'+TO+'" + endDate + "T" + endTime + "']";
+ String backupJobCriteria;
+
+ try {
+ if ( saveTime != null ) {
+ Instant instant = Instant.ofEpochSecond(Long.parseLong(saveTime));
+ String completionTime = formatterDateTime.format(Date.from(instant));
+ backupJobCriteria = "+and+saveTime:" + "'" + completionTime + "'";
+ }
+ else {
+ backupJobCriteria = "+and+saveTime:" + searchRange;
+ }
+ final HttpResponse response = get("/global/backups/?q=name:" + vm.getName() + backupJobCriteria);
+ checkResponseOK(response);
+ final ObjectMapper jsonMapper = new ObjectMapper();
+ jsonMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
+
+ networkerBackups = jsonMapper.readValue(response.getEntity().getContent(), NetworkerBackups.class);
+ NetworkerBackup networkerLatestBackup = new NetworkerBackup();
+
+ if (networkerBackups == null || networkerBackups.getBackups() == null || networkerBackups.getCount() == 0) {
+ return null;
+ }
+ if (networkerBackups.getCount() == 1) {
+ networkerLatestBackup = networkerBackups.getBackups().get(0);
+ } else {
+ for (final NetworkerBackup networkerBackup : networkerBackups.getBackups()) {
+ LOG.debug("Found Backup :" + networkerBackup.getName());
+ }
+ }
+
+ backup.setVmId(vm.getId());
+ backup.setExternalId(networkerLatestBackup.getId());
+ backup.setType(networkerLatestBackup.getType());
+ backup.setDate(networkerLatestBackup.getCreationTime());
+ backup.setSize(networkerLatestBackup.getSize().getValue());
+ backup.setProtectedSize(networkerLatestBackup.getSize().getValue());
+ backup.setStatus(org.apache.cloudstack.backup.Backup.Status.BackedUp);
+ backup.setBackupOfferingId(vm.getBackupOfferingId());
+ backup.setAccountId(vm.getAccountId());
+ backup.setDomainId(vm.getDomainId());
+ backup.setZoneId(vm.getDataCenterId());
+ return backup;
+ } catch (final IOException e) {
+ LOG.error("Failed to register backup from EMC Networker due to:", e);
+ checkResponseTimeOut(e);
+ }
+ return null;
+ }
+
+
+ public NetworkerBackup getNetworkerBackupInfo(String backupId) {
+ LOG.debug("Trying to get EMC Networker details for backup " + backupId);
+ try {
+ final HttpResponse response = get("/global/backups/?q=id:" + backupId);
+ checkResponseOK(response);
+ final ObjectMapper jsonMapper = new ObjectMapper();
+ jsonMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
+ NetworkerBackups networkerBackups = jsonMapper.readValue(response.getEntity().getContent(), NetworkerBackups.class);
+ NetworkerBackup networkerBackup = networkerBackups.getBackups().get(0);
+ if ( networkerBackup.getShortId() == null ) {
+ return null;
+ }
+ return networkerBackup;
+ } catch (final IOException e) {
+ LOG.error("Failed to list EMC Networker backups due to:", e);
+ checkResponseTimeOut(e);
+ }
+ return null;
+ }
+ public ArrayList<String> getBackupsForVm(VirtualMachine vm) {
+ SimpleDateFormat formatterDateTime = new SimpleDateFormat("yyy-MM-dd'T'HH:mm:ss");
+
+ LOG.debug("Trying to list EMC Networker backups for VM " + vm.getName());
+ try {
+ final HttpResponse response = get("/global/backups/?q=name:" + vm.getName());
+ checkResponseOK(response);
+ final ObjectMapper jsonMapper = new ObjectMapper();
+ jsonMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
+ NetworkerBackups networkerBackups = jsonMapper.readValue(response.getEntity().getContent(), NetworkerBackups.class);
+ final ArrayList<String> backupsTaken = new ArrayList<>();
+ if (networkerBackups == null || networkerBackups.getBackups() == null) {
+ return backupsTaken;
+ }
+ for (final NetworkerBackup backup : networkerBackups.getBackups()) {
+ LOG.debug("Found Backup " + backup.getId());
+ // Backups that have expired on the EMC Networker but not removed yet will not be added
+ try {
+ Date backupRetentionTime = formatterDateTime.parse(backup.getRetentionTime());
+ Date currentTime = new Date();
+ if (currentTime.compareTo(backupRetentionTime) < 0) {
+ backupsTaken.add(backup.getId());
+ }
+ } catch (ParseException e) {
+ throw new RuntimeException(e);
+ }
+ }
+ return backupsTaken;
+ } catch (final IOException e) {
+ LOG.error("Failed to list EMC Networker backups due to:", e);
+ checkResponseTimeOut(e);
+ }
+ return new ArrayList<>();
+ }
+ public List<BackupOffering> listPolicies() {
+ LOG.debug("Trying to list backup EMC Networker Policies we can use");
+ try {
+ final HttpResponse response = get("/global/protectionpolicies/?q=comment:" + BACKUP_IDENTIFIER);
+ checkResponseOK(response);
+ final ObjectMapper jsonMapper = new ObjectMapper();
+ jsonMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
+
+ final ProtectionPolicies protectionPolicies = jsonMapper.readValue(response.getEntity().getContent(), ProtectionPolicies.class);
+
+ final List<BackupOffering> policies = new ArrayList<>();
+
+ if (protectionPolicies == null || protectionPolicies.getProtectionPolicies() == null) {
+ return policies;
+ }
+ for (final ProtectionPolicy protectionPolicy : protectionPolicies.getProtectionPolicies()) {
+ LOG.debug("Found Protection Policy:" + protectionPolicy.getName());
+ policies.add(new NetworkerBackupOffering(protectionPolicy.getName(), protectionPolicy.getResourceId().getId()));
+ }
+ return policies;
+ } catch (final IOException e) {
+ LOG.error("Failed to list EMC Networker Protection Policies jobs due to:", e);
+ checkResponseTimeOut(e);
+ }
+ return new ArrayList<>();
+ }
+}
+
Review Comment:
minor nit - 2x extra newline
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/NetworkerObject.java:
##########
@@ -0,0 +1,34 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
Review Comment:
minor nit - extra newline
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r951217852
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/NetworkerClient.java:
##########
@@ -0,0 +1,360 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.cloudstack.backup.networker;
+
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nio.TrustAllManager;
+import com.cloud.vm.VirtualMachine;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.cloudstack.api.ApiErrorCode;
+import org.apache.cloudstack.api.ServerApiException;
+import org.apache.cloudstack.backup.BackupOffering;
+import org.apache.cloudstack.backup.BackupVO;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackups;
+import org.apache.cloudstack.backup.networker.api.ProtectionPolicies;
+import org.apache.cloudstack.backup.networker.api.ProtectionPolicy;
+import org.apache.cloudstack.utils.security.SSLUtils;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpResponse;
+import org.apache.http.HttpStatus;
+import org.apache.http.client.HttpClient;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.HttpDelete;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.conn.ConnectTimeoutException;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.impl.client.HttpClientBuilder;
+import org.apache.log4j.Logger;
+
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.X509TrustManager;
+import java.io.IOException;
+import java.net.SocketTimeoutException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.security.SecureRandom;
+import java.text.ParseException;
+import java.text.SimpleDateFormat;
+import java.time.Instant;
+import java.util.ArrayList;
+import java.util.Base64;
+import java.util.Date;
+import java.util.List;
+
+import static org.apache.cloudstack.backup.NetworkerBackupProvider.BACKUP_IDENTIFIER;
+
Review Comment:
minor nit - extra new line
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rohityadavcloud commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
rohityadavcloud commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r951223497
##########
scripts/vm/hypervisor/kvm/nsrkvmbackup.sh:
##########
@@ -0,0 +1,270 @@
+#!/bin/bash
+
+## Licensed to the Apache Software Foundation (ASF) under one
+## or more contributor license agreements. See the NOTICE file
+## distributed with this work for additional information
+## regarding copyright ownership. The ASF licenses this file
+## to you under the Apache License, Version 2.0 (the
+## "License"); you may not use this file except in compliance
+## with the License. You may obtain a copy of the License at
+##
+## http://www.apache.org/licenses/LICENSE-2.0
+##
+## Unless required by applicable law or agreed to in writing,
+## software distributed under the License is distributed on an
+## "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+## KIND, either express or implied. See the License for the
+## specific language governing permissions and limitations
+## under the License.
+
+
+version=0.9.1
+
+
+OPTIND=1
+
+verb=0
+logDir="/nsr/logs/cloudstack/"
+snapPrefix="CSBKP_$RANDOM_"
+clusterClient=""
+networkerServer=""
+hvVersion=""
+libvVersion=""
+apiVersion=""
+kvmDName=""
+kvmDUuid=""
+logFile=""
+mediaPool=""
+retentionTime=""
+
+
+log () {
+
+ [[ "$verb" -eq 1 ]] && builtin echo "$@"
+ if [[ "$1" == "-ne" || "$1" == "-e" || "$1" == "-n" ]]; then
+ builtin echo -e "$(date '+%Y-%m-%d %H-%M-%S>')" "${@: 2}" >> "$logFile"
+ else
+ builtin echo "$(date '+%Y-%m-%d %H-%M-%S>')" "$@" >> "$logFile"
+ fi
+
+}
+
+vercomp(){
+ local a b IFS=. -; set -f
+ printf -v a %08d $1; printf -v b %08d $3
+ test $a "$2" $b
+}
+usage() {
+ echo "
+
+Usage:[-v] [-h] [-l log_dir] [-dr] [-s networker_server] [-c networker_cluster_client] [-t target_vm] [-u target_uuid] [-p snapprefix] [-P media_pool ] [-R retention_time ]
+
+Options:
+ -h Help and usage
+ -v Enable verbose mode
+ -l log_dir. Specify log directory. Default is /nsr/logs/cloudstack
+ -s networker_server Specifiy the EMC Networker server we are going to use
+ -c networker_cluster_client Specify the EMC Networker client CLUSTER to use
+ -t target_vm KVM domain to backup
+ -u target_uuid KVM domain to backup
+ -p Snapshot Prefix for backups
+ -P mediaPool EMC Networker Media Pool
+ -R retention_time Backup retention time
+
+Supplements Apache Cloudstack B&R Framework EMC Networker plugin and performs the backup of the Virtual Machines
+"
+
+}
+
+sanity_checks() {
+
+ log "Performing environment sanity checks..."
+ log -ne "\t[1] Checking if Networker is installed\t"
+ if [[ $(systemctl list-unit-files | grep networker) = *networker* ]]; then
+ log "Success"
+ else
+ log "Failure"
+ log -e "\n\tNetworker Service NOT FOUND. Make sure that Networker client is properly installed"
+ exit 1
+ fi
+
+ log -ne "\t[2] Checking if Networker is running\t"
+ if [[ $(systemctl is-active networker) = *active* ]]; then
+ log "Success"
+ else
+ log "Failure"
+ log -e "\n\tNetworker Service is not running. Investigate Networker logs, startup server and try again"
+ exit 2
+ fi
+ log -ne "\t[3] Checking Networker DNS Resolution\t"
+ if [[ $(getent hosts "$networkerServer") = *$networkerServer* ]]; then
+ log "Success"
+ else
+ log "Failure"
+ log -e "\n\tNetworker Server cannot be resolved. Backups will most probably fail. Consider adding the ip/hostname to /etc/host or fix DNS resolution"
+ exit 3
+ fi
+
+ log -ne "\t[4] Checking QEMU / Libvirt Versions \t"
+ hvVersion=$(virsh version | grep hypervisor | awk '{print $(NF)}')
+ libvVersion=$(virsh version | grep libvirt | awk '{print $(NF)}' | tail -n 1)
+ apiVersion=$(virsh version | grep API | awk '{print $(NF)}')
+ if vercomp "$hvVersion" \> 2.1.2; then
+ log -n "Success"
+ log -ne "\t\t [ Libvirt: $libvVersion apiVersion: $apiVersion ]"
+ echo
+
+ else
+ log "Failure"
+ log -e "\n\tYour QEMU version $hvVersion is unsupported. Consider upgrading at least to latest QEMU at branch 2"
+ exit 4
+ fi
+
+
+
+ log "Environment Sanity Checks successfully passed"
+}
+
+
+
+echo "
+Cloudstack B&R Framework - EMC Networker backup script
+Version $version
+"
+
+
+backup_domain() {
+
+ name=$1
+ snapName=$2
+ log "Preparing snapshots and gathering information for backing up domain $name under snapshot name $snapName"
+ log "Retention time is $retentionTime"
+
+ declare -A TRGSRC
+ while IFS=',' read -r TARGET SOURCE
+ do
+ if [[ $SOURCE != "-" ]]; then
+ TRGSRC+=(["$TARGET"]="$SOURCE")
+ fi
+ done < <(virsh domblklist "$name" --details | grep file | grep -v 'cdrom' | grep -v 'floppy' | sed 's/ */,/g' | cut -d',' -f 4-)
+ diskspec=""
+ for target in "${!TRGSRC[@]}"; do
+ log -e "\tDisk for $target is at ${TRGSRC[${target}]}"
+ diskspec="$diskspec --diskspec $target,snapshot=external"
+ disks="$disks ${TRGSRC[${target}]} "
+ done
+
+ cmd="$(virsh snapshot-create-as --domain "$name" --name "$snapName" --no-metadata --atomic --quiesce --disk-only "$diskspec")"
Review Comment:
general remark - could we do this via java using libvirt-java than using virsh?
##########
scripts/vm/hypervisor/kvm/nsrkvmbackup.sh:
##########
@@ -0,0 +1,270 @@
+#!/bin/bash
+
+## Licensed to the Apache Software Foundation (ASF) under one
+## or more contributor license agreements. See the NOTICE file
+## distributed with this work for additional information
+## regarding copyright ownership. The ASF licenses this file
+## to you under the Apache License, Version 2.0 (the
+## "License"); you may not use this file except in compliance
+## with the License. You may obtain a copy of the License at
+##
+## http://www.apache.org/licenses/LICENSE-2.0
+##
+## Unless required by applicable law or agreed to in writing,
+## software distributed under the License is distributed on an
+## "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+## KIND, either express or implied. See the License for the
+## specific language governing permissions and limitations
+## under the License.
+
+
+version=0.9.1
+
+
+OPTIND=1
+
+verb=0
+logDir="/nsr/logs/cloudstack/"
+snapPrefix="CSBKP_$RANDOM_"
+clusterClient=""
+networkerServer=""
+hvVersion=""
+libvVersion=""
+apiVersion=""
+kvmDName=""
+kvmDUuid=""
+logFile=""
+mediaPool=""
+retentionTime=""
+
+
+log () {
+
+ [[ "$verb" -eq 1 ]] && builtin echo "$@"
+ if [[ "$1" == "-ne" || "$1" == "-e" || "$1" == "-n" ]]; then
+ builtin echo -e "$(date '+%Y-%m-%d %H-%M-%S>')" "${@: 2}" >> "$logFile"
+ else
+ builtin echo "$(date '+%Y-%m-%d %H-%M-%S>')" "$@" >> "$logFile"
+ fi
+
+}
+
+vercomp(){
+ local a b IFS=. -; set -f
+ printf -v a %08d $1; printf -v b %08d $3
+ test $a "$2" $b
+}
+usage() {
+ echo "
+
+Usage:[-v] [-h] [-l log_dir] [-dr] [-s networker_server] [-c networker_cluster_client] [-t target_vm] [-u target_uuid] [-p snapprefix] [-P media_pool ] [-R retention_time ]
+
+Options:
+ -h Help and usage
+ -v Enable verbose mode
+ -l log_dir. Specify log directory. Default is /nsr/logs/cloudstack
+ -s networker_server Specifiy the EMC Networker server we are going to use
+ -c networker_cluster_client Specify the EMC Networker client CLUSTER to use
+ -t target_vm KVM domain to backup
+ -u target_uuid KVM domain to backup
+ -p Snapshot Prefix for backups
+ -P mediaPool EMC Networker Media Pool
+ -R retention_time Backup retention time
+
+Supplements Apache Cloudstack B&R Framework EMC Networker plugin and performs the backup of the Virtual Machines
+"
+
+}
+
+sanity_checks() {
+
+ log "Performing environment sanity checks..."
+ log -ne "\t[1] Checking if Networker is installed\t"
+ if [[ $(systemctl list-unit-files | grep networker) = *networker* ]]; then
+ log "Success"
+ else
+ log "Failure"
+ log -e "\n\tNetworker Service NOT FOUND. Make sure that Networker client is properly installed"
+ exit 1
+ fi
+
+ log -ne "\t[2] Checking if Networker is running\t"
+ if [[ $(systemctl is-active networker) = *active* ]]; then
+ log "Success"
+ else
+ log "Failure"
+ log -e "\n\tNetworker Service is not running. Investigate Networker logs, startup server and try again"
+ exit 2
+ fi
+ log -ne "\t[3] Checking Networker DNS Resolution\t"
+ if [[ $(getent hosts "$networkerServer") = *$networkerServer* ]]; then
+ log "Success"
+ else
+ log "Failure"
+ log -e "\n\tNetworker Server cannot be resolved. Backups will most probably fail. Consider adding the ip/hostname to /etc/host or fix DNS resolution"
+ exit 3
+ fi
+
+ log -ne "\t[4] Checking QEMU / Libvirt Versions \t"
+ hvVersion=$(virsh version | grep hypervisor | awk '{print $(NF)}')
+ libvVersion=$(virsh version | grep libvirt | awk '{print $(NF)}' | tail -n 1)
+ apiVersion=$(virsh version | grep API | awk '{print $(NF)}')
+ if vercomp "$hvVersion" \> 2.1.2; then
+ log -n "Success"
+ log -ne "\t\t [ Libvirt: $libvVersion apiVersion: $apiVersion ]"
+ echo
+
+ else
+ log "Failure"
+ log -e "\n\tYour QEMU version $hvVersion is unsupported. Consider upgrading at least to latest QEMU at branch 2"
+ exit 4
+ fi
+
+
+
+ log "Environment Sanity Checks successfully passed"
+}
+
+
+
+echo "
+Cloudstack B&R Framework - EMC Networker backup script
+Version $version
+"
+
+
+backup_domain() {
+
+ name=$1
+ snapName=$2
+ log "Preparing snapshots and gathering information for backing up domain $name under snapshot name $snapName"
+ log "Retention time is $retentionTime"
+
+ declare -A TRGSRC
+ while IFS=',' read -r TARGET SOURCE
+ do
+ if [[ $SOURCE != "-" ]]; then
+ TRGSRC+=(["$TARGET"]="$SOURCE")
+ fi
+ done < <(virsh domblklist "$name" --details | grep file | grep -v 'cdrom' | grep -v 'floppy' | sed 's/ */,/g' | cut -d',' -f 4-)
+ diskspec=""
+ for target in "${!TRGSRC[@]}"; do
+ log -e "\tDisk for $target is at ${TRGSRC[${target}]}"
+ diskspec="$diskspec --diskspec $target,snapshot=external"
+ disks="$disks ${TRGSRC[${target}]} "
+ done
+
+ cmd="$(virsh snapshot-create-as --domain "$name" --name "$snapName" --no-metadata --atomic --quiesce --disk-only "$diskspec")"
+ retVal=$?
+ log "$cmd"
+ if [ "$retVal" -ne 0 ]; then
+ log "Agent not responding, trying to snapshot directly"
+ cmd="$(virsh snapshot-create-as --domain "$name" --name "$snapName" --no-metadata --atomic --disk-only "$diskspec")"
+ retVal=$?
+ if [ "$retVal" -ne 0 ]; then
+ log "Failed to create snapshot for $name"
+ exit 7
+ fi
+ log "Created snapshot(s) for $name"
+ fi
+ cmd="$(save -LL -q -e "${retentionTime}" -s "$networkerServer" -c "$clusterClient" -N "$name" -b "$mediaPool" $disks)"
+ retVal=$?
+ log "$cmd"
+ echo "$cmd" | grep -oE 'savetime=[0-9]{10}'
+ if [ $retVal -ne 0 ]; then
+ log "Unable to backup $disks for $name"
+ else
+ log "Backup $disks for $name completed!!"
+ fi
+
+
+ #Merge changes and conclude
+ SNAPSHOTS="$(virsh domblklist "$name" --details | grep file | grep -v 'cdrom' | grep -v 'floppy' | awk '{print $4}')"
+ for target in "${!TRGSRC[@]}"; do
+ log "Merging Snasphots for $target"
+ cmd="$(virsh blockcommit "$name" "$target" --active --pivot)"
+ retVal=$?
+ log "$cmd"
+ if [ $retVal -ne 0 ]; then
+ log "Unable to merge disk %target changes for domain $name"
+ exit 8
+ fi
+ done
+ #Clean snapshots
+ for snapshot in $SNAPSHOTS; do
+ log "Deleting Snapshot $snapshot"
+ cmd=$(rm "$snapshot")
+ retVal=$?
+ log "$cmd"
+ if [ $retVal -ne 0 ]; then
+ log "Unable to delete snapshot $snapshot"
+ exit 8
+ fi
+ log "Deleted Snapshot: $snapshot"
+ done
+}
+
+while getopts "h?vs:l:c:t:u:p:P:R:" opt; do
+ case "$opt" in
+ h|\?)
+ usage
+ exit 254
+ ;;
+ c) clusterClient="$OPTARG"
+ ;;
+ s) networkerServer="$OPTARG"
+ ;;
+ l) logDir="$OPTARG"
+ ;;
+ t) kvmDName="$OPTARG"
+ ;;
+ u) kvmDUuid="$OPTARG"
+ ;;
+ p) snapPrefix="$OPTARG"
+ ;;
+ P) mediaPool="$OPTARG"
+ ;;
+ R) retentionTime="$OPTARG"
+ ;;
+ v) verb=1
+ ;;
+ esac
+ done
+
+ shift $((OPTIND-1))
+
+ [ "${1:-}" = "--" ] && shift
+
+ if [[ -z "$networkerServer" || -z "$kvmDName" || -z "$clusterClient" || -z "$kvmDUuid" ]]; then
+ usage
+ exit 255
+ fi
+
+ if [ ! -d "$logDir" ]; then
+ mkdir -p "$logDir"
+ fi
+
+ logFile="$logDir/BACKUP-$kvmDName-$(date +'%Y_%m_%d_%I_%M_%p').log"
+
+
+ # Perform Initial sanity checks
+ sanity_checks
+
+
Review Comment:
minor nit - extra newline
##########
scripts/vm/hypervisor/kvm/nsrkvmrestore.sh:
##########
@@ -0,0 +1,234 @@
+#!/bin/bash
+
+## Licensed to the Apache Software Foundation (ASF) under one
+## or more contributor license agreements. See the NOTICE file
+## distributed with this work for additional information
+## regarding copyright ownership. The ASF licenses this file
+## to you under the Apache License, Version 2.0 (the
+## "License"); you may not use this file except in compliance
+## with the License. You may obtain a copy of the License at
+##
+## http://www.apache.org/licenses/LICENSE-2.0
+##
+## Unless required by applicable law or agreed to in writing,
+## software distributed under the License is distributed on an
+## "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+## KIND, either express or implied. See the License for the
+## specific language governing permissions and limitations
+## under the License.
+
+
+version=0.9.1
+
+
Review Comment:
minor nit - extra newline
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r920093278
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/Action.java:
##########
@@ -0,0 +1,444 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+import java.util.List;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "actionReferSchedule",
+ "actionSpecificData",
+ "comment",
+ "completionNotification",
+ "concurrent",
+ "drivenBy",
+ "enabled",
+ "failureImpact",
+ "hardLimit",
+ "inactivityTimeoutInMin",
+ "name",
+ "parallelism",
+ "retries",
+ "retryDelayInSec",
+ "softLimit",
+ "scheduleActivities",
+ "scheduleOverrides",
+ "schedulePeriod"
+})
+@Generated("jsonschema2pojo")
+public class Action implements Serializable {
+
+ private final static long serialVersionUID = 1750989315434884936L;
+ @JsonProperty("actionReferSchedule")
+ private String actionReferSchedule;
+ @JsonProperty("actionSpecificData")
+ private ActionSpecificData actionSpecificData;
+ @JsonProperty("comment")
+ private String comment;
+ @JsonProperty("completionNotification")
+ private CompletionNotification completionNotification;
+ @JsonProperty("concurrent")
+ private Boolean concurrent;
+ @JsonProperty("drivenBy")
+ private String drivenBy;
+ @JsonProperty("enabled")
+ private Boolean enabled;
+ @JsonProperty("failureImpact")
+ private String failureImpact;
+ @JsonProperty("hardLimit")
+ private String hardLimit;
+ @JsonProperty("inactivityTimeoutInMin")
+ private Integer inactivityTimeoutInMin;
+ @JsonProperty("name")
+ private String name;
+ @JsonProperty("parallelism")
+ private Integer parallelism;
+ @JsonProperty("retries")
+ private Integer retries;
+ @JsonProperty("retryDelayInSec")
+ private Integer retryDelayInSec;
+ @JsonProperty("softLimit")
+ private String softLimit;
+ @JsonProperty("scheduleActivities")
+ private List<String> scheduleActivities = null;
+ @JsonProperty("scheduleOverrides")
+ private List<Object> scheduleOverrides = null;
+ @JsonProperty("schedulePeriod")
+ private String schedulePeriod;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public Action() {
+ }
+
+ /**
+ * @param failureImpact
+ * @param actionSpecificData
+ * @param completionNotification
+ * @param parallelism
+ * @param concurrent
+ * @param retryDelayInSec
+ * @param drivenBy
+ * @param enabled
+ * @param scheduleActivities
+ * @param retries
+ * @param actionReferSchedule
+ * @param name
+ * @param inactivityTimeoutInMin
+ * @param comment
+ * @param hardLimit
+ * @param scheduleOverrides
+ * @param schedulePeriod
+ * @param softLimit
+ */
+ public Action(String actionReferSchedule, ActionSpecificData actionSpecificData, String comment, CompletionNotification completionNotification, Boolean concurrent, String drivenBy, Boolean enabled, String failureImpact, String hardLimit, Integer inactivityTimeoutInMin, String name, Integer parallelism, Integer retries, Integer retryDelayInSec, String softLimit, List<String> scheduleActivities, List<Object> scheduleOverrides, String schedulePeriod) {
+ super();
+ this.actionReferSchedule = actionReferSchedule;
+ this.actionSpecificData = actionSpecificData;
+ this.comment = comment;
+ this.completionNotification = completionNotification;
+ this.concurrent = concurrent;
+ this.drivenBy = drivenBy;
+ this.enabled = enabled;
+ this.failureImpact = failureImpact;
+ this.hardLimit = hardLimit;
+ this.inactivityTimeoutInMin = inactivityTimeoutInMin;
+ this.name = name;
+ this.parallelism = parallelism;
+ this.retries = retries;
+ this.retryDelayInSec = retryDelayInSec;
+ this.softLimit = softLimit;
+ this.scheduleActivities = scheduleActivities;
+ this.scheduleOverrides = scheduleOverrides;
+ this.schedulePeriod = schedulePeriod;
+ }
+
+ @JsonProperty("actionReferSchedule")
+ public String getActionReferSchedule() {
+ return actionReferSchedule;
+ }
+
+ @JsonProperty("actionReferSchedule")
+ public void setActionReferSchedule(String actionReferSchedule) {
+ this.actionReferSchedule = actionReferSchedule;
+ }
+
+ @JsonProperty("actionSpecificData")
+ public ActionSpecificData getActionSpecificData() {
+ return actionSpecificData;
+ }
+
+ @JsonProperty("actionSpecificData")
+ public void setActionSpecificData(ActionSpecificData actionSpecificData) {
+ this.actionSpecificData = actionSpecificData;
+ }
+
+ @JsonProperty("comment")
+ public String getComment() {
+ return comment;
+ }
+
+ @JsonProperty("comment")
+ public void setComment(String comment) {
+ this.comment = comment;
+ }
+
+ @JsonProperty("completionNotification")
+ public CompletionNotification getCompletionNotification() {
+ return completionNotification;
+ }
+
+ @JsonProperty("completionNotification")
+ public void setCompletionNotification(CompletionNotification completionNotification) {
+ this.completionNotification = completionNotification;
+ }
+
+ @JsonProperty("concurrent")
+ public Boolean getConcurrent() {
+ return concurrent;
+ }
+
+ @JsonProperty("concurrent")
+ public void setConcurrent(Boolean concurrent) {
+ this.concurrent = concurrent;
+ }
+
+ @JsonProperty("drivenBy")
+ public String getDrivenBy() {
+ return drivenBy;
+ }
+
+ @JsonProperty("drivenBy")
+ public void setDrivenBy(String drivenBy) {
+ this.drivenBy = drivenBy;
+ }
+
+ @JsonProperty("enabled")
+ public Boolean getEnabled() {
+ return enabled;
+ }
+
+ @JsonProperty("enabled")
+ public void setEnabled(Boolean enabled) {
+ this.enabled = enabled;
+ }
+
+ @JsonProperty("failureImpact")
+ public String getFailureImpact() {
+ return failureImpact;
+ }
+
+ @JsonProperty("failureImpact")
+ public void setFailureImpact(String failureImpact) {
+ this.failureImpact = failureImpact;
+ }
+
+ @JsonProperty("hardLimit")
+ public String getHardLimit() {
+ return hardLimit;
+ }
+
+ @JsonProperty("hardLimit")
+ public void setHardLimit(String hardLimit) {
+ this.hardLimit = hardLimit;
+ }
+
+ @JsonProperty("inactivityTimeoutInMin")
+ public Integer getInactivityTimeoutInMin() {
+ return inactivityTimeoutInMin;
+ }
+
+ @JsonProperty("inactivityTimeoutInMin")
+ public void setInactivityTimeoutInMin(Integer inactivityTimeoutInMin) {
+ this.inactivityTimeoutInMin = inactivityTimeoutInMin;
+ }
+
+ @JsonProperty("name")
+ public String getName() {
+ return name;
+ }
+
+ @JsonProperty("name")
+ public void setName(String name) {
+ this.name = name;
+ }
+
+ @JsonProperty("parallelism")
+ public Integer getParallelism() {
+ return parallelism;
+ }
+
+ @JsonProperty("parallelism")
+ public void setParallelism(Integer parallelism) {
+ this.parallelism = parallelism;
+ }
+
+ @JsonProperty("retries")
+ public Integer getRetries() {
+ return retries;
+ }
+
+ @JsonProperty("retries")
+ public void setRetries(Integer retries) {
+ this.retries = retries;
+ }
+
+ @JsonProperty("retryDelayInSec")
+ public Integer getRetryDelayInSec() {
+ return retryDelayInSec;
+ }
+
+ @JsonProperty("retryDelayInSec")
+ public void setRetryDelayInSec(Integer retryDelayInSec) {
+ this.retryDelayInSec = retryDelayInSec;
+ }
+
+ @JsonProperty("softLimit")
+ public String getSoftLimit() {
+ return softLimit;
+ }
+
+ @JsonProperty("softLimit")
+ public void setSoftLimit(String softLimit) {
+ this.softLimit = softLimit;
+ }
+
+ @JsonProperty("scheduleActivities")
+ public List<String> getScheduleActivities() {
+ return scheduleActivities;
+ }
+
+ @JsonProperty("scheduleActivities")
+ public void setScheduleActivities(List<String> scheduleActivities) {
+ this.scheduleActivities = scheduleActivities;
+ }
+
+ @JsonProperty("scheduleOverrides")
+ public List<Object> getScheduleOverrides() {
+ return scheduleOverrides;
+ }
+
+ @JsonProperty("scheduleOverrides")
+ public void setScheduleOverrides(List<Object> scheduleOverrides) {
+ this.scheduleOverrides = scheduleOverrides;
+ }
+
+ @JsonProperty("schedulePeriod")
+ public String getSchedulePeriod() {
+ return schedulePeriod;
+ }
+
+ @JsonProperty("schedulePeriod")
+ public void setSchedulePeriod(String schedulePeriod) {
+ this.schedulePeriod = schedulePeriod;
+ }
+
+ @Override
+ public String toString() {
+ StringBuilder sb = new StringBuilder();
+ sb.append(Action.class.getName()).append('@').append(Integer.toHexString(System.identityHashCode(this))).append('[');
+ sb.append("actionReferSchedule");
+ sb.append('=');
+ sb.append(((this.actionReferSchedule == null) ? "<null>" : this.actionReferSchedule));
+ sb.append(',');
+ sb.append("actionSpecificData");
+ sb.append('=');
+ sb.append(((this.actionSpecificData == null) ? "<null>" : this.actionSpecificData));
+ sb.append(',');
+ sb.append("comment");
+ sb.append('=');
+ sb.append(((this.comment == null) ? "<null>" : this.comment));
+ sb.append(',');
+ sb.append("completionNotification");
+ sb.append('=');
+ sb.append(((this.completionNotification == null) ? "<null>" : this.completionNotification));
+ sb.append(',');
+ sb.append("concurrent");
+ sb.append('=');
+ sb.append(((this.concurrent == null) ? "<null>" : this.concurrent));
+ sb.append(',');
+ sb.append("drivenBy");
+ sb.append('=');
+ sb.append(((this.drivenBy == null) ? "<null>" : this.drivenBy));
+ sb.append(',');
+ sb.append("enabled");
+ sb.append('=');
+ sb.append(((this.enabled == null) ? "<null>" : this.enabled));
+ sb.append(',');
+ sb.append("failureImpact");
+ sb.append('=');
+ sb.append(((this.failureImpact == null) ? "<null>" : this.failureImpact));
+ sb.append(',');
+ sb.append("hardLimit");
+ sb.append('=');
+ sb.append(((this.hardLimit == null) ? "<null>" : this.hardLimit));
+ sb.append(',');
+ sb.append("inactivityTimeoutInMin");
+ sb.append('=');
+ sb.append(((this.inactivityTimeoutInMin == null) ? "<null>" : this.inactivityTimeoutInMin));
+ sb.append(',');
+ sb.append("name");
+ sb.append('=');
+ sb.append(((this.name == null) ? "<null>" : this.name));
+ sb.append(',');
+ sb.append("parallelism");
+ sb.append('=');
+ sb.append(((this.parallelism == null) ? "<null>" : this.parallelism));
+ sb.append(',');
+ sb.append("retries");
+ sb.append('=');
+ sb.append(((this.retries == null) ? "<null>" : this.retries));
+ sb.append(',');
+ sb.append("retryDelayInSec");
+ sb.append('=');
+ sb.append(((this.retryDelayInSec == null) ? "<null>" : this.retryDelayInSec));
+ sb.append(',');
+ sb.append("softLimit");
+ sb.append('=');
+ sb.append(((this.softLimit == null) ? "<null>" : this.softLimit));
+ sb.append(',');
+ sb.append("scheduleActivities");
+ sb.append('=');
+ sb.append(((this.scheduleActivities == null) ? "<null>" : this.scheduleActivities));
+ sb.append(',');
+ sb.append("scheduleOverrides");
+ sb.append('=');
+ sb.append(((this.scheduleOverrides == null) ? "<null>" : this.scheduleOverrides));
+ sb.append(',');
+ sb.append("schedulePeriod");
+ sb.append('=');
+ sb.append(((this.schedulePeriod == null) ? "<null>" : this.schedulePeriod));
+ sb.append(',');
+ if (sb.charAt((sb.length() - 1)) == ',') {
+ sb.setCharAt((sb.length() - 1), ']');
+ } else {
+ sb.append(']');
+ }
+ return sb.toString();
+ }
+
+ @Override
+ public int hashCode() {
+ int result = 1;
+ result = ((result * 31) + ((this.failureImpact == null) ? 0 : this.failureImpact.hashCode()));
+ result = ((result * 31) + ((this.actionSpecificData == null) ? 0 : this.actionSpecificData.hashCode()));
+ result = ((result * 31) + ((this.completionNotification == null) ? 0 : this.completionNotification.hashCode()));
+ result = ((result * 31) + ((this.parallelism == null) ? 0 : this.parallelism.hashCode()));
+ result = ((result * 31) + ((this.concurrent == null) ? 0 : this.concurrent.hashCode()));
+ result = ((result * 31) + ((this.retryDelayInSec == null) ? 0 : this.retryDelayInSec.hashCode()));
+ result = ((result * 31) + ((this.drivenBy == null) ? 0 : this.drivenBy.hashCode()));
+ result = ((result * 31) + ((this.enabled == null) ? 0 : this.enabled.hashCode()));
+ result = ((result * 31) + ((this.scheduleActivities == null) ? 0 : this.scheduleActivities.hashCode()));
+ result = ((result * 31) + ((this.retries == null) ? 0 : this.retries.hashCode()));
+ result = ((result * 31) + ((this.actionReferSchedule == null) ? 0 : this.actionReferSchedule.hashCode()));
+ result = ((result * 31) + ((this.name == null) ? 0 : this.name.hashCode()));
+ result = ((result * 31) + ((this.inactivityTimeoutInMin == null) ? 0 : this.inactivityTimeoutInMin.hashCode()));
+ result = ((result * 31) + ((this.comment == null) ? 0 : this.comment.hashCode()));
+ result = ((result * 31) + ((this.hardLimit == null) ? 0 : this.hardLimit.hashCode()));
+ result = ((result * 31) + ((this.scheduleOverrides == null) ? 0 : this.scheduleOverrides.hashCode()));
+ result = ((result * 31) + ((this.schedulePeriod == null) ? 0 : this.schedulePeriod.hashCode()));
+ result = ((result * 31) + ((this.softLimit == null) ? 0 : this.softLimit.hashCode()));
+ return result;
+ }
+
+ @Override
+ public boolean equals(Object other) {
+ if (other == this) {
+ return true;
+ }
+ if ((other instanceof Action) == false) {
+ return false;
+ }
+ Action rhs = ((Action) other);
+ return (((((((((((((((((((this.failureImpact == rhs.failureImpact) || ((this.failureImpact != null) && this.failureImpact.equals(rhs.failureImpact))) && ((this.actionSpecificData == rhs.actionSpecificData) || ((this.actionSpecificData != null) && this.actionSpecificData.equals(rhs.actionSpecificData)))) && ((this.completionNotification == rhs.completionNotification) || ((this.completionNotification != null) &&
+ this.completionNotification.equals(rhs.completionNotification)))) && ((this.parallelism == rhs.parallelism) || ((this.parallelism != null) && this.parallelism.equals(rhs.parallelism)))) && ((this.concurrent == rhs.concurrent) || ((this.concurrent != null) && this.concurrent.equals(rhs.concurrent)))) && ((this.retryDelayInSec == rhs.retryDelayInSec) || ((this.retryDelayInSec != null) && this.retryDelayInSec.equals(rhs.retryDelayInSec)))) && ((this.drivenBy == rhs.drivenBy) || ((this.drivenBy != null) &&
+ this.drivenBy.equals(rhs.drivenBy)))) && ((this.enabled == rhs.enabled) || ((this.enabled != null) && this.enabled.equals(rhs.enabled)))) && ((this.scheduleActivities == rhs.scheduleActivities) || ((this.scheduleActivities != null) && this.scheduleActivities.equals(rhs.scheduleActivities)))) && ((this.retries == rhs.retries) || ((this.retries != null) && this.retries.equals(rhs.retries)))) &&
+ ((this.actionReferSchedule == rhs.actionReferSchedule) || ((this.actionReferSchedule != null) && this.actionReferSchedule.equals(rhs.actionReferSchedule)))) && ((this.name == rhs.name) || ((this.name != null) && this.name.equals(rhs.name)))) && ((this.inactivityTimeoutInMin == rhs.inactivityTimeoutInMin) || ((this.inactivityTimeoutInMin != null) && this.inactivityTimeoutInMin.equals(rhs.inactivityTimeoutInMin)))) && ((this.comment == rhs.comment) || ((this.comment != null) &&
+ this.comment.equals(rhs.comment)))) && ((this.hardLimit == rhs.hardLimit) || ((this.hardLimit != null) && this.hardLimit.equals(rhs.hardLimit)))) && ((this.scheduleOverrides == rhs.scheduleOverrides) || ((this.scheduleOverrides != null) && this.scheduleOverrides.equals(rhs.scheduleOverrides)))) && ((this.schedulePeriod == rhs.schedulePeriod) || ((this.schedulePeriod != null) && this.schedulePeriod.equals(rhs.schedulePeriod)))) &&
+ ((this.softLimit == rhs.softLimit) || ((this.softLimit != null) && this.softLimit.equals(rhs.softLimit))));
+ }
Review Comment:
can you extract and simplify this in a separate method, please?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r920106582
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/SummaryNotification.java:
##########
@@ -0,0 +1,117 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "command",
+ "executeOn"
+})
+@Generated("jsonschema2pojo")
+public class SummaryNotification implements Serializable {
+
+ private final static long serialVersionUID = -1142867788732047105L;
+ @JsonProperty("command")
+ private String command;
+ @JsonProperty("executeOn")
+ private String executeOn;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public SummaryNotification() {
+ }
+
+ /**
+ * @param executeOn
+ * @param command
+ */
+ public SummaryNotification(String command, String executeOn) {
+ super();
+ this.command = command;
+ this.executeOn = executeOn;
+ }
+
+ @JsonProperty("command")
+ public String getCommand() {
+ return command;
+ }
+
+ @JsonProperty("command")
+ public void setCommand(String command) {
+ this.command = command;
+ }
+
+ @JsonProperty("executeOn")
+ public String getExecuteOn() {
+ return executeOn;
+ }
+
+ @JsonProperty("executeOn")
+ public void setExecuteOn(String executeOn) {
+ this.executeOn = executeOn;
+ }
+
+ @Override
+ public String toString() {
Review Comment:
use `ReflectionToStringBuilderUtils`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r920107518
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/Size.java:
##########
@@ -0,0 +1,117 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "unit",
+ "value"
+})
+@Generated("jsonschema2pojo")
+public class Size implements Serializable {
+
+ private final static long serialVersionUID = -8270448162523429622L;
+ @JsonProperty("unit")
+ private String unit;
+ @JsonProperty("value")
+ private Integer value;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public Size() {
+ }
+
+ /**
+ * @param unit
+ * @param value
+ */
+ public Size(String unit, Integer value) {
+ super();
+ this.unit = unit;
+ this.value = value;
+ }
+
+ @JsonProperty("unit")
+ public String getUnit() {
+ return unit;
+ }
+
+ @JsonProperty("unit")
+ public void setUnit(String unit) {
+ this.unit = unit;
+ }
+
+ @JsonProperty("value")
+ public Integer getValue() {
+ return value;
+ }
+
+ @JsonProperty("value")
+ public void setValue(Integer value) {
+ this.value = value;
+ }
+
+ @Override
+ public String toString() {
Review Comment:
use `ReflectionToStringBuilderUtils`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1184041707
UI build: :heavy_check_mark:
Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6550 (SL-JID-1957)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] acs-robot commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
acs-robot commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1185564251
Found UI changes, kicking a new UI QA build
@blueorangutan ui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1370778961
Packaging result: :heavy_multiplication_x: el7 :heavy_multiplication_x: el8 :heavy_multiplication_x: el9 :heavy_multiplication_x: debian :heavy_multiplication_x: suse15. SL-JID 5170
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r967635854
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/NetworkerClient.java:
##########
@@ -0,0 +1,360 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.cloudstack.backup.networker;
+
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nio.TrustAllManager;
+import com.cloud.vm.VirtualMachine;
+import com.fasterxml.jackson.databind.DeserializationFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.cloudstack.api.ApiErrorCode;
+import org.apache.cloudstack.api.ServerApiException;
+import org.apache.cloudstack.backup.BackupOffering;
+import org.apache.cloudstack.backup.BackupVO;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackups;
+import org.apache.cloudstack.backup.networker.api.ProtectionPolicies;
+import org.apache.cloudstack.backup.networker.api.ProtectionPolicy;
+import org.apache.cloudstack.utils.security.SSLUtils;
+import org.apache.http.HttpHeaders;
+import org.apache.http.HttpResponse;
+import org.apache.http.HttpStatus;
+import org.apache.http.client.HttpClient;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.HttpDelete;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.conn.ConnectTimeoutException;
+import org.apache.http.conn.ssl.NoopHostnameVerifier;
+import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
+import org.apache.http.impl.client.HttpClientBuilder;
+import org.apache.log4j.Logger;
+
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.X509TrustManager;
+import java.io.IOException;
+import java.net.SocketTimeoutException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.security.SecureRandom;
+import java.text.ParseException;
+import java.text.SimpleDateFormat;
+import java.time.Instant;
+import java.util.ArrayList;
+import java.util.Base64;
+import java.util.Date;
+import java.util.List;
+
+import static org.apache.cloudstack.backup.NetworkerBackupProvider.BACKUP_IDENTIFIER;
+
+
+public class NetworkerClient {
+
+
+ private static final Logger LOG = Logger.getLogger(NetworkerClient.class);
+
+ private final URI apiURI;
+ private final String apiName;
+ private final String apiPassword;
+
+ private final HttpClient httpClient;
+
+
+ public NetworkerClient(final String url, final String username, final String password, final boolean validateCertificate, final int timeout) throws URISyntaxException, NoSuchAlgorithmException, KeyManagementException {
+
+ apiName = username;
+ apiPassword = password;
+
+ this.apiURI = new URI(url);
+ final RequestConfig config = RequestConfig.custom()
+ .setConnectTimeout(timeout * 1000)
+ .setConnectionRequestTimeout(timeout * 1000)
+ .setSocketTimeout(timeout * 1000)
+ .build();
+
+ if (!validateCertificate) {
+ final SSLContext sslcontext = SSLUtils.getSSLContext();
+ sslcontext.init(null, new X509TrustManager[]{new TrustAllManager()}, new SecureRandom());
+ final SSLConnectionSocketFactory factory = new SSLConnectionSocketFactory(sslcontext, NoopHostnameVerifier.INSTANCE);
+ this.httpClient = HttpClientBuilder.create()
+ .setDefaultRequestConfig(config)
+ .setSSLSocketFactory(factory)
+ .build();
+ } else {
+ this.httpClient = HttpClientBuilder.create()
+ .setDefaultRequestConfig(config)
+ .build();
+ }
+
+ authenticate(username, password);
+ }
+
+ private void authenticate(final String username, final String password) {
+
+ final HttpGet request = new HttpGet(apiURI.toString());
+ request.setHeader(HttpHeaders.AUTHORIZATION, "Basic " + Base64.getEncoder().encodeToString((username + ":" + password).getBytes()));
+ request.setHeader(HttpHeaders.ACCEPT, "application/json");
+ request.setHeader(HttpHeaders.USER_AGENT, "CloudStack B&R");
+ request.setHeader(HttpHeaders.CONNECTION, "keep-alive");
+ try {
+ final HttpResponse response = httpClient.execute(request);
+ checkAuthFailure(response);
+ if (response.getStatusLine().getStatusCode() != HttpStatus.SC_OK) {
+ throw new CloudRuntimeException("Failed to create and authenticate EMC Networker API client, please check the settings.");
+ }
+ } catch (final IOException e) {
+ throw new CloudRuntimeException("Failed to authenticate Networker API service due to:" + e.getMessage());
+ }
+ }
+
+ private void checkAuthFailure(final HttpResponse response) {
+ if (response != null && response.getStatusLine().getStatusCode() == HttpStatus.SC_UNAUTHORIZED) {
+ throw new ServerApiException(ApiErrorCode.UNAUTHORIZED, "EMC Networker B&R API call unauthorized. Check username/password or contact your backup administrator.");
+ }
+ }
+
+ private void checkResponseOK(final HttpResponse response) {
+ if (response.getStatusLine().getStatusCode() == HttpStatus.SC_NO_CONTENT) {
+ LOG.debug("Requested EMC Networker resource does not exist");
+ return;
+ }
+ if (!(response.getStatusLine().getStatusCode() == HttpStatus.SC_OK ||
+ response.getStatusLine().getStatusCode() == HttpStatus.SC_ACCEPTED) &&
+ response.getStatusLine().getStatusCode() != HttpStatus.SC_NO_CONTENT) {
+ LOG.debug(String.format("HTTP request failed, status code is [%s], response is: [%s].", response.getStatusLine().getStatusCode(), response));
+ throw new ServerApiException(ApiErrorCode.INTERNAL_ERROR, "Got invalid API status code returned by the EMC Networker server");
+ }
+ }
+
+ private void checkResponseTimeOut(final Exception e) {
+ if (e instanceof ConnectTimeoutException || e instanceof SocketTimeoutException) {
+ throw new ServerApiException(ApiErrorCode.RESOURCE_UNAVAILABLE_ERROR, "EMC Networker API operation timed out, please try again.");
+ }
+ }
+
+ private HttpResponse get(final String path) throws IOException {
+ String url = apiURI.toString() + path;
+ final HttpGet request = new HttpGet(url);
+ request.setHeader(HttpHeaders.AUTHORIZATION, "Basic " + Base64.getEncoder().encodeToString((apiName + ":" + apiPassword).getBytes()));
+ request.setHeader(HttpHeaders.ACCEPT, "application/json");
+ request.setHeader(HttpHeaders.USER_AGENT, "CloudStack B&R");
+ final HttpResponse response = httpClient.execute(request);
+ checkAuthFailure(response);
+
+ LOG.debug(String.format("Response received in GET request is: [%s] for URL: [%s].", response.toString(), url));
+ return response;
+ }
+
+ public String getBackupPolicyRetentionInterval(String externalId) {
+ try {
+ final HttpResponse response = get("/global/protectionpolicies/?q=comment:" + BACKUP_IDENTIFIER);
+ checkResponseOK(response);
+ final ObjectMapper jsonMapper = new ObjectMapper();
+ jsonMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
+
+ final ProtectionPolicies protectionPolicies = jsonMapper.readValue(response.getEntity().getContent(), ProtectionPolicies.class);
+
+ if (protectionPolicies == null || protectionPolicies.getProtectionPolicies() == null) {
+ return null;
+ }
+ for (final ProtectionPolicy protectionPolicy : protectionPolicies.getProtectionPolicies()) {
+ if ( protectionPolicy.getResourceId().getId().equals(externalId)) {
+ return protectionPolicy.getPolicyProtectionPeriod();
+ }
+ }
+ } catch (final IOException e) {
+ LOG.error("Failed to get Protection Policy Period from EMC Networker due to:", e);
+ checkResponseTimeOut(e);
+ }
+ return null;
+ }
+ private HttpResponse delete(final String path) throws IOException {
+ String url = apiURI.toString() + path;
+ final HttpDelete request = new HttpDelete(url);
+ request.setHeader(HttpHeaders.AUTHORIZATION, "Basic " + Base64.getEncoder().encodeToString((apiName + ":" + apiPassword).getBytes()));
+ request.setHeader(HttpHeaders.USER_AGENT, "CloudStack B&R");
+ final HttpResponse response = httpClient.execute(request);
+ checkAuthFailure(response);
+
+ LOG.debug(String.format("Response received in DELETE request is: [%s] for URL [%s].", response.toString(), url));
+ return response;
+ }
+
+ public boolean deleteBackupForVM(String externalId) {
+ try {
+ final HttpResponse response = delete("/global/backups/" + externalId);
+ checkResponseOK(response);
+ return true;
+ } catch (final IOException e) {
+ LOG.error("Failed to delete backup from EMC Networker due to:", e);
+ checkResponseTimeOut(e);
+ }
+ return false;
+ }
+
+
+ public BackupVO registerBackupForVm(VirtualMachine vm, Date backupJobStart, String saveTime) {
+ LOG.debug("Querying EMC Networker about latest backup");
+
+ NetworkerBackups networkerBackups;
+ BackupVO backup = new BackupVO();
+
+ SimpleDateFormat formatterDate = new SimpleDateFormat("yyyy-MM-dd");
+ SimpleDateFormat formatterTime = new SimpleDateFormat("HH:mm:ss");
+ SimpleDateFormat formatterDateTime = new SimpleDateFormat("yyy-MM-dd'T'HH:mm:ss");
+
+ String startDate = formatterDate.format(backupJobStart);
+ String startTime = formatterTime.format(backupJobStart);
+ String endDate = formatterDate.format(new Date());
+ String endTime = formatterTime.format(new Date());
+
+ final String searchRange = "['" + startDate + "T" + startTime + "'+TO+'" + endDate + "T" + endTime + "']";
+ String backupJobCriteria;
+
+ try {
+ if ( saveTime != null ) {
+ Instant instant = Instant.ofEpochSecond(Long.parseLong(saveTime));
+ String completionTime = formatterDateTime.format(Date.from(instant));
+ backupJobCriteria = "+and+saveTime:" + "'" + completionTime + "'";
+ }
+ else {
+ backupJobCriteria = "+and+saveTime:" + searchRange;
+ }
+ final HttpResponse response = get("/global/backups/?q=name:" + vm.getName() + backupJobCriteria);
+ checkResponseOK(response);
+ final ObjectMapper jsonMapper = new ObjectMapper();
+ jsonMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
+
+ networkerBackups = jsonMapper.readValue(response.getEntity().getContent(), NetworkerBackups.class);
+ NetworkerBackup networkerLatestBackup = new NetworkerBackup();
+
+ if (networkerBackups == null || networkerBackups.getBackups() == null || networkerBackups.getCount() == 0) {
+ return null;
+ }
+ if (networkerBackups.getCount() == 1) {
+ networkerLatestBackup = networkerBackups.getBackups().get(0);
+ } else {
+ for (final NetworkerBackup networkerBackup : networkerBackups.getBackups()) {
+ LOG.debug("Found Backup :" + networkerBackup.getName());
+ }
+ }
+
+ backup.setVmId(vm.getId());
+ backup.setExternalId(networkerLatestBackup.getId());
+ backup.setType(networkerLatestBackup.getType());
+ backup.setDate(networkerLatestBackup.getCreationTime());
+ backup.setSize(networkerLatestBackup.getSize().getValue());
+ backup.setProtectedSize(networkerLatestBackup.getSize().getValue());
+ backup.setStatus(org.apache.cloudstack.backup.Backup.Status.BackedUp);
+ backup.setBackupOfferingId(vm.getBackupOfferingId());
+ backup.setAccountId(vm.getAccountId());
+ backup.setDomainId(vm.getDomainId());
+ backup.setZoneId(vm.getDataCenterId());
+ return backup;
+ } catch (final IOException e) {
+ LOG.error("Failed to register backup from EMC Networker due to:", e);
+ checkResponseTimeOut(e);
+ }
+ return null;
+ }
+
+
+ public NetworkerBackup getNetworkerBackupInfo(String backupId) {
+ LOG.debug("Trying to get EMC Networker details for backup " + backupId);
+ try {
+ final HttpResponse response = get("/global/backups/?q=id:" + backupId);
+ checkResponseOK(response);
+ final ObjectMapper jsonMapper = new ObjectMapper();
+ jsonMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
+ NetworkerBackups networkerBackups = jsonMapper.readValue(response.getEntity().getContent(), NetworkerBackups.class);
+ NetworkerBackup networkerBackup = networkerBackups.getBackups().get(0);
+ if ( networkerBackup.getShortId() == null ) {
+ return null;
+ }
+ return networkerBackup;
+ } catch (final IOException e) {
+ LOG.error("Failed to list EMC Networker backups due to:", e);
+ checkResponseTimeOut(e);
+ }
+ return null;
+ }
+ public ArrayList<String> getBackupsForVm(VirtualMachine vm) {
+ SimpleDateFormat formatterDateTime = new SimpleDateFormat("yyy-MM-dd'T'HH:mm:ss");
+
+ LOG.debug("Trying to list EMC Networker backups for VM " + vm.getName());
+ try {
+ final HttpResponse response = get("/global/backups/?q=name:" + vm.getName());
+ checkResponseOK(response);
+ final ObjectMapper jsonMapper = new ObjectMapper();
+ jsonMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
+ NetworkerBackups networkerBackups = jsonMapper.readValue(response.getEntity().getContent(), NetworkerBackups.class);
+ final ArrayList<String> backupsTaken = new ArrayList<>();
+ if (networkerBackups == null || networkerBackups.getBackups() == null) {
+ return backupsTaken;
+ }
+ for (final NetworkerBackup backup : networkerBackups.getBackups()) {
+ LOG.debug("Found Backup " + backup.getId());
+ // Backups that have expired on the EMC Networker but not removed yet will not be added
+ try {
+ Date backupRetentionTime = formatterDateTime.parse(backup.getRetentionTime());
+ Date currentTime = new Date();
+ if (currentTime.compareTo(backupRetentionTime) < 0) {
+ backupsTaken.add(backup.getId());
+ }
+ } catch (ParseException e) {
+ throw new RuntimeException(e);
+ }
+ }
+ return backupsTaken;
+ } catch (final IOException e) {
+ LOG.error("Failed to list EMC Networker backups due to:", e);
+ checkResponseTimeOut(e);
+ }
+ return new ArrayList<>();
+ }
+ public List<BackupOffering> listPolicies() {
+ LOG.debug("Trying to list backup EMC Networker Policies we can use");
+ try {
+ final HttpResponse response = get("/global/protectionpolicies/?q=comment:" + BACKUP_IDENTIFIER);
+ checkResponseOK(response);
+ final ObjectMapper jsonMapper = new ObjectMapper();
+ jsonMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
+
+ final ProtectionPolicies protectionPolicies = jsonMapper.readValue(response.getEntity().getContent(), ProtectionPolicies.class);
+
+ final List<BackupOffering> policies = new ArrayList<>();
+
+ if (protectionPolicies == null || protectionPolicies.getProtectionPolicies() == null) {
+ return policies;
+ }
+ for (final ProtectionPolicy protectionPolicy : protectionPolicies.getProtectionPolicies()) {
+ LOG.debug("Found Protection Policy:" + protectionPolicy.getName());
+ policies.add(new NetworkerBackupOffering(protectionPolicy.getName(), protectionPolicy.getResourceId().getId()));
+ }
+ return policies;
+ } catch (final IOException e) {
+ LOG.error("Failed to list EMC Networker Protection Policies jobs due to:", e);
+ checkResponseTimeOut(e);
+ }
+ return new ArrayList<>();
+ }
+}
+
Review Comment:
fixed in latest commit
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/NetworkerObject.java:
##########
@@ -0,0 +1,34 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
Review Comment:
fixed in latest commit
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/NetworkerObject.java:
##########
@@ -0,0 +1,34 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.cloudstack.backup.networker;
+
+
Review Comment:
fixed in latest commit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1206256927
@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r979472042
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/NetworkerBackupOffering.java:
##########
@@ -0,0 +1,82 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.cloudstack.backup.networker;
+
+
+import org.apache.cloudstack.backup.BackupOffering;
+
+import java.util.Date;
+
+
+public class NetworkerBackupOffering implements BackupOffering {
+
+ private String name;
+ private String uid;
+
+ public NetworkerBackupOffering(String name, String uid) {
+ this.name = name;
+ this.uid = uid;
+ }
+
+ @Override
+ public String getExternalId() {
+ return uid;
+ }
+
+ @Override
+ public String getName() {
+ return name;
+ }
+
+ @Override
+ public String getDescription() {
+ return "EMC Networker Backup Offering (Job)";
+ }
+
+ @Override
+ public long getZoneId() {
+ return -1;
+ }
+
+ @Override
+ public boolean isUserDrivenBackupAllowed() {
+ return false;
+ }
+
+ @Override
+ public String getProvider() {
+ return "veeam";
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1272517156
@rohityadavcloud a Jenkins job has been kicked to build packages. It will be bundled with
SystemVM template(s). I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r979468824
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/NetworkerBackupProvider.java:
##########
@@ -0,0 +1,640 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.backup;
+
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.host.HostVO;
+import com.cloud.host.Status;
+import com.cloud.host.dao.HostDao;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.StoragePoolHostVO;
+import com.cloud.storage.Volume;
+import com.cloud.storage.VolumeVO;
+import com.cloud.storage.dao.StoragePoolHostDao;
+import com.cloud.storage.dao.VolumeDao;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.component.AdapterBase;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallbackNoReturn;
+import com.cloud.utils.db.TransactionStatus;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.ssh.SSHCmdHelper;
+import com.cloud.utils.ssh.SshException;
+import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.dao.VMInstanceDao;
+import org.apache.cloudstack.api.InternalIdentity;
+import org.apache.cloudstack.backup.dao.BackupDao;
+import org.apache.cloudstack.backup.dao.BackupOfferingDaoImpl;
+import org.apache.cloudstack.backup.networker.NetworkerClient;
+import org.apache.cloudstack.framework.config.ConfigKey;
+import org.apache.cloudstack.framework.config.Configurable;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.log4j.Logger;
+import org.apache.xml.utils.URI;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+
+
+import javax.inject.Inject;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.HashMap;
+import java.util.Date;
+import java.util.Objects;
+import java.util.UUID;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+
+public class NetworkerBackupProvider extends AdapterBase implements BackupProvider, Configurable {
+
+ public static final String BACKUP_IDENTIFIER = "-CSBKP-";
+ private static final Logger LOG = Logger.getLogger(NetworkerBackupProvider.class);
+ public ConfigKey<String> NetworkerUrl = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.url", "https://localhost:9090/nwrestapi/v3",
+ "The EMC Networker API URL.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerUsername = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.username", "administrator",
+ "The EMC Networker API username.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerPassword = new ConfigKey<>("Secure", String.class,
+ "backup.plugin.networker.password", "password",
+ "The EMC Networker API password.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerMediaPool = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.pool", "Default",
+ "The EMC Networker Media Pool", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerValidateSSLSecurity = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.validate.ssl", "false",
+ "Validate the SSL certificate when connecting to EMC Networker API service.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Integer> NetworkerApiRequestTimeout = new ConfigKey<>("Advanced", Integer.class,
+ "backup.plugin.networker.request.timeout", "300",
+ "The EMC Networker API request timeout in seconds.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerClientVerboseLogs = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.client.verbosity", "false",
+ "Produce Verbose logs in Hypervisor", true, ConfigKey.Scope.Zone);
+
+ @Inject
+ private BackupDao backupDao;
+
+ @Inject
+ private HostDao hostDao;
+
+ @Inject
+ private ConfigurationDao configDao;
+
+ @Inject
+ private ClusterDao clusterDao;
+
+ @Inject
+ private VolumeDao volumeDao;
+
+ @Inject
+ private StoragePoolHostDao storagePoolHostDao;
+
+ @Inject
+ private VMInstanceDao vmInstanceDao;
+
+ private static String getUrlDomain(String url) throws URISyntaxException {
+ URI uri;
+ try {
+ uri = new URI(url);
+ } catch (URI.MalformedURIException e) {
+ throw new CloudRuntimeException("Failed to cast URI");
+ }
+
+ return uri.getHost();
+ }
+
+ @Override
+ public ConfigKey<?>[] getConfigKeys() {
+ return new ConfigKey[]{
+ NetworkerUrl,
+ NetworkerUsername,
+ NetworkerPassword,
+ NetworkerValidateSSLSecurity,
+ NetworkerApiRequestTimeout,
+ NetworkerMediaPool,
+ NetworkerClientVerboseLogs
+ };
+ }
+
+ @Override
+ public String getName() {
+ return "networker";
+ }
+
+ @Override
+ public String getDescription() {
+ return "EMC Networker Backup Plugin";
+ }
+
+ @Override
+ public String getConfigComponentName() {
+ return BackupService.class.getSimpleName();
+ }
+
+ protected HostVO getLastVMHypervisorHost(VirtualMachine vm) {
+ HostVO host;
+ Long hostId = vm.getLastHostId();
+
+ if (hostId == null) {
+ LOG.debug("Cannot find last host for vm. This should never happen, please check your database.");
+ return null;
+ }
+ host = hostDao.findById(hostId);
+
+ if ( host.getStatus() == Status.Up ) {
+ return host;
+ } else {
+ // Try to find a host in the same cluster
+ List<HostVO> altClusterHosts = hostDao.findHypervisorHostInCluster(host.getClusterId());
+ for (final HostVO candidateClusterHost : altClusterHosts) {
+ if ( candidateClusterHost.getStatus() == Status.Up ) {
+ LOG.debug("Found Host " + candidateClusterHost.getName());
+ return candidateClusterHost;
+ }
+ }
+ }
+ // Try to find a Host in the zone
+ List<HostVO> altZoneHosts = hostDao.findByDataCenterId(host.getDataCenterId());
+ for (final HostVO candidateZoneHost : altZoneHosts) {
+ if ( candidateZoneHost.getStatus() == Status.Up && candidateZoneHost.getHypervisorType() == Hypervisor.HypervisorType.KVM ) {
+ LOG.debug("Found Host " + candidateZoneHost.getName());
+ return candidateZoneHost;
+ }
+ }
+ return null;
+ }
+
+ protected HostVO getRunningVMHypervisorHost(VirtualMachine vm) {
+
+ HostVO host;
+ Long hostId = vm.getHostId();
+
+ if (hostId == null) {
+ throw new CloudRuntimeException("Unable to find the HYPERVISOR for " + vm.getName() + ". Make sure the virtual machine is running");
+ }
+
+ host = hostDao.findById(hostId);
+
+ return host;
+ }
+
+ protected String getVMHypervisorCluster(HostVO host) {
+
+ return clusterDao.findById(host.getClusterId()).getName();
+ }
+
+ protected Ternary<String, String, String> getKVMHyperisorCredentials(HostVO host) {
+
+ String username = null;
+ String password = null;
+ String privateKey = null;
+
+ if (host != null && host.getHypervisorType() == Hypervisor.HypervisorType.KVM) {
+ hostDao.loadDetails(host);
+ password = host.getDetail("password");
+ username = host.getDetail("username");
+ privateKey = configDao.getValue("ssh.privatekey");
+ }
+ if ((password == null && privateKey == null) || username == null) {
+ throw new CloudRuntimeException("Cannot find login credentials for HYPERVISOR " + Objects.requireNonNull(host).getUuid());
+ }
+
+ return new Ternary<>(username, password, privateKey);
+ }
+
+ private String executeBackupCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+ String nstRegex = "\\bcompleted savetime=([0-9]{10})";
+ Pattern saveTimePattern = Pattern.compile(nstRegex);
+
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ Matcher saveTimeMatcher = saveTimePattern.matcher(result.getStdOut());
+ if (saveTimeMatcher.find()) {
+ return saveTimeMatcher.group(1);
+ }
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+
+ return null;
+ }
+ private boolean executeRestoreCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ return true;
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+ }
+
+ private NetworkerClient getClient(final Long zoneId) {
+ try {
+ return new NetworkerClient(NetworkerUrl.valueIn(zoneId), NetworkerUsername.valueIn(zoneId), NetworkerPassword.valueIn(zoneId),
+ NetworkerValidateSSLSecurity.valueIn(zoneId), NetworkerApiRequestTimeout.valueIn(zoneId));
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException("Failed to parse EMC Networker API URL: " + e.getMessage());
+ } catch (NoSuchAlgorithmException | KeyManagementException e) {
+ LOG.error("Failed to build EMC Networker API client due to: ", e);
+ }
+ throw new CloudRuntimeException("Failed to build EMC Networker API client");
+ }
+
+ @Override
+ public List<BackupOffering> listBackupOfferings(Long zoneId) {
+ List<BackupOffering> policies = new ArrayList<>();
+ for (final BackupOffering policy : getClient(zoneId).listPolicies()) {
+ if (!policy.getName().contains(BACKUP_IDENTIFIER)) {
+ policies.add(policy);
+ }
+ }
+
+ return policies;
+ }
+
+ @Override
+ public boolean isValidProviderOffering(Long zoneId, String uuid) {
+ List<BackupOffering> policies = listBackupOfferings(zoneId);
+ if (CollectionUtils.isEmpty(policies)) {
+ return false;
+ }
+ for (final BackupOffering policy : policies) {
+ if (Objects.equals(policy.getExternalId(), uuid)) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ @Override
+ public boolean assignVMToBackupOffering(VirtualMachine vm, BackupOffering backupOffering) { return true; }
+
+ @Override
+ public boolean removeVMFromBackupOffering(VirtualMachine vm) {
+ LOG.debug("Removing VirtualMachine from Backup offering and Deleting any existing backups");
+
+ List<String> backupsTaken = getClient(vm.getDataCenterId()).getBackupsForVm(vm);
+
+ for (String backupId : backupsTaken) {
+ LOG.debug("Trying to remove backup with id" + backupId);
+ getClient(vm.getDataCenterId()).deleteBackupForVM(backupId);
+ }
+
+ return true;
+ }
+
+ @Override
+ public boolean restoreVMFromBackup(VirtualMachine vm, Backup backup) {
+ LOG.debug("Restoring vm " + vm.getUuid() + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+ String networkerServer;
+ HostVO hostVO;
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return false;
+ }
+
+ // Find where the VM was last running
+ hostVO = getLastVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -S " + SSID;
+
+ if (NetworkerClientVerboseLogs.value())
+ command = command + " -v ";
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for VM ID " + vm.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return true;
+ } else {
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return false;
+ }
+ }
+
+ @Override
+ public Pair<Boolean, String> restoreBackedUpVolume(Backup backup, String volumeUuid, String hostIp, String dataStoreUuid) {
+ LOG.debug("Restoring volume " + volumeUuid + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+
+ String networkerServer;
+ VolumeVO volume = volumeDao.findByUuid(volumeUuid);
+ VMInstanceVO backupSourceVm = vmInstanceDao.findById(backup.getVmId());
+ StoragePoolHostVO dataStore = storagePoolHostDao.findByUuid(dataStoreUuid);
+ HostVO hostVO = hostDao.findByIp(hostIp);
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+ final String clusterName = networkerBackup.getClientHostname();
+ final String destinationNetworkerClient = hostVO.getName().split("\\.")[0];
+ Long restoredVolumeDiskSize = 0L;
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return null;
+ }
+
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ // Find volume size from backup vols
+ for ( Backup.VolumeInfo VMVolToRestore : backupSourceVm.getBackupVolumeList()) {
+ if (VMVolToRestore.getUuid().equals(volumeUuid))
+ restoredVolumeDiskSize = (VMVolToRestore.getSize());
+ }
+
+ VolumeVO restoredVolume = new VolumeVO(Volume.Type.DATADISK, null, backup.getZoneId(),
+ backup.getDomainId(), backup.getAccountId(), 0, null,
+ backup.getSize(), null, null, null);
+
+ restoredVolume.setName("RV-"+volume.getName());
+ restoredVolume.setProvisioningType(volume.getProvisioningType());
+ restoredVolume.setUpdated(new Date());
+ restoredVolume.setUuid(UUID.randomUUID().toString());
+ restoredVolume.setRemoved(null);
+ restoredVolume.setDisplayVolume(true);
+ restoredVolume.setPoolId(volume.getPoolId());
+ restoredVolume.setPath(restoredVolume.getUuid());
+ restoredVolume.setState(Volume.State.Copying);
+ restoredVolume.setSize(restoredVolumeDiskSize);
+ restoredVolume.setDiskOfferingId(volume.getDiskOfferingId());
+
+ try {
+ volumeDao.persist(restoredVolume);
+ } catch (Exception e) {
+ throw new CloudRuntimeException("Unable to craft restored volume due to: "+e);
+ }
+
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -c " + clusterName +
+ " -d " + destinationNetworkerClient +
+ " -n " + restoredVolume.getUuid() +
Review Comment:
Will have a look
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/NetworkerBackupProvider.java:
##########
@@ -0,0 +1,640 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.backup;
+
+import com.cloud.dc.dao.ClusterDao;
+import com.cloud.host.HostVO;
+import com.cloud.host.Status;
+import com.cloud.host.dao.HostDao;
+import com.cloud.hypervisor.Hypervisor;
+import com.cloud.storage.StoragePoolHostVO;
+import com.cloud.storage.Volume;
+import com.cloud.storage.VolumeVO;
+import com.cloud.storage.dao.StoragePoolHostDao;
+import com.cloud.storage.dao.VolumeDao;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.component.AdapterBase;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallbackNoReturn;
+import com.cloud.utils.db.TransactionStatus;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.ssh.SSHCmdHelper;
+import com.cloud.utils.ssh.SshException;
+import com.cloud.vm.VMInstanceVO;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.dao.VMInstanceDao;
+import org.apache.cloudstack.api.InternalIdentity;
+import org.apache.cloudstack.backup.dao.BackupDao;
+import org.apache.cloudstack.backup.dao.BackupOfferingDaoImpl;
+import org.apache.cloudstack.backup.networker.NetworkerClient;
+import org.apache.cloudstack.framework.config.ConfigKey;
+import org.apache.cloudstack.framework.config.Configurable;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.log4j.Logger;
+import org.apache.xml.utils.URI;
+import org.apache.cloudstack.backup.networker.api.NetworkerBackup;
+
+
+import javax.inject.Inject;
+import java.net.URISyntaxException;
+import java.security.KeyManagementException;
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.HashMap;
+import java.util.Date;
+import java.util.Objects;
+import java.util.UUID;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+
+public class NetworkerBackupProvider extends AdapterBase implements BackupProvider, Configurable {
+
+ public static final String BACKUP_IDENTIFIER = "-CSBKP-";
+ private static final Logger LOG = Logger.getLogger(NetworkerBackupProvider.class);
+ public ConfigKey<String> NetworkerUrl = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.url", "https://localhost:9090/nwrestapi/v3",
+ "The EMC Networker API URL.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerUsername = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.username", "administrator",
+ "The EMC Networker API username.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerPassword = new ConfigKey<>("Secure", String.class,
+ "backup.plugin.networker.password", "password",
+ "The EMC Networker API password.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<String> NetworkerMediaPool = new ConfigKey<>("Advanced", String.class,
+ "backup.plugin.networker.pool", "Default",
+ "The EMC Networker Media Pool", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerValidateSSLSecurity = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.validate.ssl", "false",
+ "Validate the SSL certificate when connecting to EMC Networker API service.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Integer> NetworkerApiRequestTimeout = new ConfigKey<>("Advanced", Integer.class,
+ "backup.plugin.networker.request.timeout", "300",
+ "The EMC Networker API request timeout in seconds.", true, ConfigKey.Scope.Zone);
+
+ private final ConfigKey<Boolean> NetworkerClientVerboseLogs = new ConfigKey<>("Advanced", Boolean.class,
+ "backup.plugin.networker.client.verbosity", "false",
+ "Produce Verbose logs in Hypervisor", true, ConfigKey.Scope.Zone);
+
+ @Inject
+ private BackupDao backupDao;
+
+ @Inject
+ private HostDao hostDao;
+
+ @Inject
+ private ConfigurationDao configDao;
+
+ @Inject
+ private ClusterDao clusterDao;
+
+ @Inject
+ private VolumeDao volumeDao;
+
+ @Inject
+ private StoragePoolHostDao storagePoolHostDao;
+
+ @Inject
+ private VMInstanceDao vmInstanceDao;
+
+ private static String getUrlDomain(String url) throws URISyntaxException {
+ URI uri;
+ try {
+ uri = new URI(url);
+ } catch (URI.MalformedURIException e) {
+ throw new CloudRuntimeException("Failed to cast URI");
+ }
+
+ return uri.getHost();
+ }
+
+ @Override
+ public ConfigKey<?>[] getConfigKeys() {
+ return new ConfigKey[]{
+ NetworkerUrl,
+ NetworkerUsername,
+ NetworkerPassword,
+ NetworkerValidateSSLSecurity,
+ NetworkerApiRequestTimeout,
+ NetworkerMediaPool,
+ NetworkerClientVerboseLogs
+ };
+ }
+
+ @Override
+ public String getName() {
+ return "networker";
+ }
+
+ @Override
+ public String getDescription() {
+ return "EMC Networker Backup Plugin";
+ }
+
+ @Override
+ public String getConfigComponentName() {
+ return BackupService.class.getSimpleName();
+ }
+
+ protected HostVO getLastVMHypervisorHost(VirtualMachine vm) {
+ HostVO host;
+ Long hostId = vm.getLastHostId();
+
+ if (hostId == null) {
+ LOG.debug("Cannot find last host for vm. This should never happen, please check your database.");
+ return null;
+ }
+ host = hostDao.findById(hostId);
+
+ if ( host.getStatus() == Status.Up ) {
+ return host;
+ } else {
+ // Try to find a host in the same cluster
+ List<HostVO> altClusterHosts = hostDao.findHypervisorHostInCluster(host.getClusterId());
+ for (final HostVO candidateClusterHost : altClusterHosts) {
+ if ( candidateClusterHost.getStatus() == Status.Up ) {
+ LOG.debug("Found Host " + candidateClusterHost.getName());
+ return candidateClusterHost;
+ }
+ }
+ }
+ // Try to find a Host in the zone
+ List<HostVO> altZoneHosts = hostDao.findByDataCenterId(host.getDataCenterId());
+ for (final HostVO candidateZoneHost : altZoneHosts) {
+ if ( candidateZoneHost.getStatus() == Status.Up && candidateZoneHost.getHypervisorType() == Hypervisor.HypervisorType.KVM ) {
+ LOG.debug("Found Host " + candidateZoneHost.getName());
+ return candidateZoneHost;
+ }
+ }
+ return null;
+ }
+
+ protected HostVO getRunningVMHypervisorHost(VirtualMachine vm) {
+
+ HostVO host;
+ Long hostId = vm.getHostId();
+
+ if (hostId == null) {
+ throw new CloudRuntimeException("Unable to find the HYPERVISOR for " + vm.getName() + ". Make sure the virtual machine is running");
+ }
+
+ host = hostDao.findById(hostId);
+
+ return host;
+ }
+
+ protected String getVMHypervisorCluster(HostVO host) {
+
+ return clusterDao.findById(host.getClusterId()).getName();
+ }
+
+ protected Ternary<String, String, String> getKVMHyperisorCredentials(HostVO host) {
+
+ String username = null;
+ String password = null;
+ String privateKey = null;
+
+ if (host != null && host.getHypervisorType() == Hypervisor.HypervisorType.KVM) {
+ hostDao.loadDetails(host);
+ password = host.getDetail("password");
+ username = host.getDetail("username");
+ privateKey = configDao.getValue("ssh.privatekey");
+ }
+ if ((password == null && privateKey == null) || username == null) {
+ throw new CloudRuntimeException("Cannot find login credentials for HYPERVISOR " + Objects.requireNonNull(host).getUuid());
+ }
+
+ return new Ternary<>(username, password, privateKey);
+ }
+
+ private String executeBackupCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+ String nstRegex = "\\bcompleted savetime=([0-9]{10})";
+ Pattern saveTimePattern = Pattern.compile(nstRegex);
+
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ Matcher saveTimeMatcher = saveTimePattern.matcher(result.getStdOut());
+ if (saveTimeMatcher.find()) {
+ return saveTimeMatcher.group(1);
+ }
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+
+ return null;
+ }
+ private boolean executeRestoreCommand(HostVO host, String username, String password, String privateKey, String command) {
+
+ SSHCmdHelper.SSHCmdResult result;
+
+
+ final com.trilead.ssh2.Connection connection = SSHCmdHelper.acquireAuthorizedConnection(
+ host.getPrivateIpAddress(), 22, username, password, privateKey);
+ if (connection == null) {
+ throw new CloudRuntimeException(String.format("Failed to connect to SSH at HYPERVISOR %s via IP address [%s].", host, host.getPrivateIpAddress()));
+ }
+ try {
+ result = SSHCmdHelper.sshExecuteCmdOneShot(connection, command);
+ if (result.getReturnCode() != 0) {
+ throw new CloudRuntimeException(String.format("Backup Script failed on HYPERVISOR %s due to: %s", host, result.getStdErr()));
+ }
+ LOG.debug("BACKUP RESULT: " + result);
+ return true;
+ } catch (final SshException e) {
+ throw new CloudRuntimeException(String.format("Command execution on host %s took longer than expected: %s", host, e.getMessage()));
+ }
+ }
+
+ private NetworkerClient getClient(final Long zoneId) {
+ try {
+ return new NetworkerClient(NetworkerUrl.valueIn(zoneId), NetworkerUsername.valueIn(zoneId), NetworkerPassword.valueIn(zoneId),
+ NetworkerValidateSSLSecurity.valueIn(zoneId), NetworkerApiRequestTimeout.valueIn(zoneId));
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException("Failed to parse EMC Networker API URL: " + e.getMessage());
+ } catch (NoSuchAlgorithmException | KeyManagementException e) {
+ LOG.error("Failed to build EMC Networker API client due to: ", e);
+ }
+ throw new CloudRuntimeException("Failed to build EMC Networker API client");
+ }
+
+ @Override
+ public List<BackupOffering> listBackupOfferings(Long zoneId) {
+ List<BackupOffering> policies = new ArrayList<>();
+ for (final BackupOffering policy : getClient(zoneId).listPolicies()) {
+ if (!policy.getName().contains(BACKUP_IDENTIFIER)) {
+ policies.add(policy);
+ }
+ }
+
+ return policies;
+ }
+
+ @Override
+ public boolean isValidProviderOffering(Long zoneId, String uuid) {
+ List<BackupOffering> policies = listBackupOfferings(zoneId);
+ if (CollectionUtils.isEmpty(policies)) {
+ return false;
+ }
+ for (final BackupOffering policy : policies) {
+ if (Objects.equals(policy.getExternalId(), uuid)) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ @Override
+ public boolean assignVMToBackupOffering(VirtualMachine vm, BackupOffering backupOffering) { return true; }
+
+ @Override
+ public boolean removeVMFromBackupOffering(VirtualMachine vm) {
+ LOG.debug("Removing VirtualMachine from Backup offering and Deleting any existing backups");
+
+ List<String> backupsTaken = getClient(vm.getDataCenterId()).getBackupsForVm(vm);
+
+ for (String backupId : backupsTaken) {
+ LOG.debug("Trying to remove backup with id" + backupId);
+ getClient(vm.getDataCenterId()).deleteBackupForVM(backupId);
+ }
+
+ return true;
+ }
+
+ @Override
+ public boolean restoreVMFromBackup(VirtualMachine vm, Backup backup) {
+ LOG.debug("Restoring vm " + vm.getUuid() + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+ String networkerServer;
+ HostVO hostVO;
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return false;
+ }
+
+ // Find where the VM was last running
+ hostVO = getLastVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -S " + SSID;
+
+ if (NetworkerClientVerboseLogs.value())
+ command = command + " -v ";
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for VM ID " + vm.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return true;
+ } else {
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return false;
+ }
+ }
+
+ @Override
+ public Pair<Boolean, String> restoreBackedUpVolume(Backup backup, String volumeUuid, String hostIp, String dataStoreUuid) {
+ LOG.debug("Restoring volume " + volumeUuid + "from backup " + backup.getUuid() + " on the Networker Backup Provider");
+
+ String networkerServer;
+ VolumeVO volume = volumeDao.findByUuid(volumeUuid);
+ VMInstanceVO backupSourceVm = vmInstanceDao.findById(backup.getVmId());
+ StoragePoolHostVO dataStore = storagePoolHostDao.findByUuid(dataStoreUuid);
+ HostVO hostVO = hostDao.findByIp(hostIp);
+
+ final Long zoneId = backup.getZoneId();
+ final String externalBackupId = backup.getExternalId();
+ final NetworkerBackup networkerBackup=getClient(zoneId).getNetworkerBackupInfo(externalBackupId);
+ final String SSID = networkerBackup.getShortId();
+ final String clusterName = networkerBackup.getClientHostname();
+ final String destinationNetworkerClient = hostVO.getName().split("\\.")[0];
+ Long restoredVolumeDiskSize = 0L;
+
+ if ( SSID.isEmpty() ) {
+ LOG.debug("There was an error retrieving the SSID for backup with id " + externalBackupId + " from EMC NEtworker");
+ return null;
+ }
+
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ LOG.debug("The SSID was reported successfully " + externalBackupId);
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ // Find volume size from backup vols
+ for ( Backup.VolumeInfo VMVolToRestore : backupSourceVm.getBackupVolumeList()) {
+ if (VMVolToRestore.getUuid().equals(volumeUuid))
+ restoredVolumeDiskSize = (VMVolToRestore.getSize());
+ }
+
+ VolumeVO restoredVolume = new VolumeVO(Volume.Type.DATADISK, null, backup.getZoneId(),
+ backup.getDomainId(), backup.getAccountId(), 0, null,
+ backup.getSize(), null, null, null);
+
+ restoredVolume.setName("RV-"+volume.getName());
+ restoredVolume.setProvisioningType(volume.getProvisioningType());
+ restoredVolume.setUpdated(new Date());
+ restoredVolume.setUuid(UUID.randomUUID().toString());
+ restoredVolume.setRemoved(null);
+ restoredVolume.setDisplayVolume(true);
+ restoredVolume.setPoolId(volume.getPoolId());
+ restoredVolume.setPath(restoredVolume.getUuid());
+ restoredVolume.setState(Volume.State.Copying);
+ restoredVolume.setSize(restoredVolumeDiskSize);
+ restoredVolume.setDiskOfferingId(volume.getDiskOfferingId());
+
+ try {
+ volumeDao.persist(restoredVolume);
+ } catch (Exception e) {
+ throw new CloudRuntimeException("Unable to craft restored volume due to: "+e);
+ }
+
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmrestore.sh" +
+ " -s " + networkerServer +
+ " -c " + clusterName +
+ " -d " + destinationNetworkerClient +
+ " -n " + restoredVolume.getUuid() +
+ " -p " + dataStore.getLocalPath() +
+ " -a " + volume.getUuid();
+
+ if (NetworkerClientVerboseLogs.value())
+ command = command + " -v ";
+
+ Date restoreJobStart = new Date();
+ LOG.debug("Starting Restore for Volume UUID " + volume.getUuid() + " and SSID" + SSID + " at " + restoreJobStart);
+
+ if ( executeRestoreCommand(hostVO, credentials.first(), credentials.second(), credentials.third(), command) ) {
+ Date restoreJobEnd = new Date();
+ LOG.debug("Restore Job for SSID " + SSID + " completed successfully at " + restoreJobEnd);
+ return new Pair<>(true,restoredVolume.getUuid());
+ } else {
+ volumeDao.expunge(restoredVolume.getId());
+ LOG.debug("Restore Job for SSID " + SSID + " failed!");
+ return null;
+ }
+ }
+
+ @Override
+ public boolean takeBackup(VirtualMachine vm) {
+
+ String networkerServer;
+ String clusterName;
+
+ try {
+ networkerServer = getUrlDomain(NetworkerUrl.value());
+ } catch (URISyntaxException e) {
+ throw new CloudRuntimeException(String.format("Failed to convert API to HOST : %s", e));
+ }
+
+ // Find where the VM is currently running
+ HostVO hostVO = getRunningVMHypervisorHost(vm);
+ // Get credentials for that host
+ Ternary<String, String, String> credentials = getKVMHyperisorCredentials(hostVO);
+ // Get retention Period for our Backup
+ BackupOfferingVO vmBackupOffering = new BackupOfferingDaoImpl().findById(vm.getBackupOfferingId());
+ final String backupProviderPolicyId = vmBackupOffering.getExternalId();
+ String backupRentionPeriod = getClient(vm.getDataCenterId()).getBackupPolicyRetentionInterval(backupProviderPolicyId);
+
+ if ( backupRentionPeriod == null ) {
+ LOG.warn("There is no retention setting for Emc Networker Policy, setting default for 1 day");
+ backupRentionPeriod="1 Day";
+ }
+
+ // Get Cluster
+ clusterName = getVMHypervisorCluster(hostVO);
+
+ String command = "sudo /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/nsrkvmbackup.sh" +
+ " -s " + networkerServer +
Review Comment:
Will have a look
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r979468141
##########
plugins/backup/networker/pom.xml:
##########
@@ -0,0 +1,56 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <artifactId>cloud-plugin-backup-networker</artifactId>
+ <name>Apache CloudStack Plugin - EMC Networker Backup and Recovery Plugin</name>
+ <parent>
+ <artifactId>cloudstack-plugins</artifactId>
+ <groupId>org.apache.cloudstack</groupId>
+ <version>4.18.0.0-SNAPSHOT</version>
+ <relativePath>../../pom.xml</relativePath>
+ </parent>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.cloudstack</groupId>
+ <artifactId>cloud-plugin-hypervisor-kvm</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.commons</groupId>
+ <artifactId>commons-lang3</artifactId>
+ <version>${cs.commons-lang3.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>com.fasterxml.jackson.core</groupId>
+ <artifactId>jackson-databind</artifactId>
+ <version>2.13.0</version>
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1260454958
@DaanHoogland a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] fermosan commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
fermosan commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r920787179
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/Link.java:
##########
@@ -0,0 +1,117 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "href",
+ "rel"
+})
+@Generated("jsonschema2pojo")
+public class Link implements Serializable {
+
+ private final static long serialVersionUID = 1110347626425938231L;
+ @JsonProperty("href")
+ private String href;
+ @JsonProperty("rel")
+ private String rel;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public Link() {
+ }
+
+ /**
+ * @param rel
+ * @param href
+ */
+ public Link(String href, String rel) {
+ super();
+ this.href = href;
+ this.rel = rel;
+ }
+
+ @JsonProperty("href")
+ public String getHref() {
+ return href;
+ }
+
+ @JsonProperty("href")
+ public void setHref(String href) {
+ this.href = href;
+ }
+
+ @JsonProperty("rel")
+ public String getRel() {
+ return rel;
+ }
+
+ @JsonProperty("rel")
+ public void setRel(String rel) {
+ this.rel = rel;
+ }
+
+ @Override
+ public String toString() {
Review Comment:
Transitioned.
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/NetworkerBackups.java:
##########
@@ -0,0 +1,118 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+import java.util.List;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "backups",
+ "count"
+})
+@Generated("jsonschema2pojo")
+public class NetworkerBackups implements Serializable {
+
+ private final static long serialVersionUID = -3805021350250865454L;
+ @JsonProperty("backups")
+ private List<Backup> backups = null;
+ @JsonProperty("count")
+ private Integer count;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public NetworkerBackups() {
+ }
+
+ /**
+ * @param count
+ * @param backups
+ */
+ public NetworkerBackups(List<Backup> backups, Integer count) {
+ super();
+ this.backups = backups;
+ this.count = count;
+ }
+
+ @JsonProperty("backups")
+ public List<Backup> getBackups() {
+ return backups;
+ }
+
+ @JsonProperty("backups")
+ public void setBackups(List<Backup> backups) {
+ this.backups = backups;
+ }
+
+ @JsonProperty("count")
+ public Integer getCount() {
+ return count;
+ }
+
+ @JsonProperty("count")
+ public void setCount(Integer count) {
+ this.count = count;
+ }
+
+ @Override
+ public String toString() {
Review Comment:
Transitioned.
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/ProtectionPolicies.java:
##########
@@ -0,0 +1,118 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+import java.util.List;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "count",
+ "protectionPolicies"
+})
+@Generated("jsonschema2pojo")
+public class ProtectionPolicies implements Serializable {
+
+ private final static long serialVersionUID = 1357846434531599309L;
+ @JsonProperty("count")
+ private Integer count;
+ @JsonProperty("protectionPolicies")
+ private List<ProtectionPolicy> protectionPolicies = null;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public ProtectionPolicies() {
+ }
+
+ /**
+ * @param protectionPolicies
+ * @param count
+ */
+ public ProtectionPolicies(Integer count, List<ProtectionPolicy> protectionPolicies) {
+ super();
+ this.count = count;
+ this.protectionPolicies = protectionPolicies;
+ }
+
+ @JsonProperty("count")
+ public Integer getCount() {
+ return count;
+ }
+
+ @JsonProperty("count")
+ public void setCount(Integer count) {
+ this.count = count;
+ }
+
+ @JsonProperty("protectionPolicies")
+ public List<ProtectionPolicy> getProtectionPolicies() {
+ return protectionPolicies;
+ }
+
+ @JsonProperty("protectionPolicies")
+ public void setProtectionPolicies(List<ProtectionPolicy> protectionPolicies) {
+ this.protectionPolicies = protectionPolicies;
+ }
+
+ @Override
+ public String toString() {
Review Comment:
Transitioned.
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/ProtectionPolicy.java:
##########
@@ -0,0 +1,222 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+import java.util.List;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "comment",
+ "links",
+ "name",
+ "policyProtectionEnable",
+ "policyProtectionPeriod",
+ "resourceId",
+ "summaryNotification",
+ "workflows"
+})
+@Generated("jsonschema2pojo")
+public class ProtectionPolicy implements Serializable {
+
+ private final static long serialVersionUID = 5407494949453441445L;
+ @JsonProperty("comment")
+ private String comment;
+ @JsonProperty("links")
+ private List<Link> links = null;
+ @JsonProperty("name")
+ private String name;
+ @JsonProperty("policyProtectionEnable")
+ private Boolean policyProtectionEnable;
+ @JsonProperty("policyProtectionPeriod")
+ private String policyProtectionPeriod;
+ @JsonProperty("resourceId")
+ private ResourceId resourceId;
+ @JsonProperty("summaryNotification")
+ private SummaryNotification summaryNotification;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public ProtectionPolicy() {
+ }
+
+ /**
+ * @param policyProtectionEnable
+ * @param policyProtectionPeriod
+ * @param summaryNotification
+ * @param resourceId
+ * @param name
+ * @param comment
+ * @param links
+ */
+ public ProtectionPolicy(String comment, List<Link> links, String name, Boolean policyProtectionEnable, String policyProtectionPeriod, ResourceId resourceId, SummaryNotification summaryNotification) {
+ super();
+ this.comment = comment;
+ this.links = links;
+ this.name = name;
+ this.policyProtectionEnable = policyProtectionEnable;
+ this.policyProtectionPeriod = policyProtectionPeriod;
+ this.resourceId = resourceId;
+ this.summaryNotification = summaryNotification;
+ }
+
+ @JsonProperty("comment")
+ public String getComment() {
+ return comment;
+ }
+
+ @JsonProperty("comment")
+ public void setComment(String comment) {
+ this.comment = comment;
+ }
+
+ @JsonProperty("links")
+ public List<Link> getLinks() {
+ return links;
+ }
+
+ @JsonProperty("links")
+ public void setLinks(List<Link> links) {
+ this.links = links;
+ }
+
+ @JsonProperty("name")
+ public String getName() {
+ return name;
+ }
+
+ @JsonProperty("name")
+ public void setName(String name) {
+ this.name = name;
+ }
+
+ @JsonProperty("policyProtectionEnable")
+ public Boolean getPolicyProtectionEnable() {
+ return policyProtectionEnable;
+ }
+
+ @JsonProperty("policyProtectionEnable")
+ public void setPolicyProtectionEnable(Boolean policyProtectionEnable) {
+ this.policyProtectionEnable = policyProtectionEnable;
+ }
+
+ @JsonProperty("policyProtectionPeriod")
+ public String getPolicyProtectionPeriod() {
+ return policyProtectionPeriod;
+ }
+
+ @JsonProperty("policyProtectionPeriod")
+ public void setPolicyProtectionPeriod(String policyProtectionPeriod) {
+ this.policyProtectionPeriod = policyProtectionPeriod;
+ }
+
+ @JsonProperty("resourceId")
+ public ResourceId getResourceId() {
+ return resourceId;
+ }
+
+ @JsonProperty("resourceId")
+ public void setResourceId(ResourceId resourceId) {
+ this.resourceId = resourceId;
+ }
+
+ @JsonProperty("summaryNotification")
+ public SummaryNotification getSummaryNotification() {
+ return summaryNotification;
+ }
+
+ @JsonProperty("summaryNotification")
+ public void setSummaryNotification(SummaryNotification summaryNotification) {
+ this.summaryNotification = summaryNotification;
+ }
+
+
+ @Override
+ public String toString() {
Review Comment:
Transitioned.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] acs-robot commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
acs-robot commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1184037815
Found UI changes, kicking a new UI QA build
@blueorangutan ui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1185565614
UI build: :heavy_multiplication_x:
(SL-JID-1964)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a diff in pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on code in PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#discussion_r920097232
##########
plugins/backup/networker/src/main/java/org/apache/cloudstack/backup/networker/api/Backup.java:
##########
@@ -0,0 +1,423 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.cloudstack.backup.networker.api;
+
+import com.fasterxml.jackson.annotation.JsonInclude;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonPropertyOrder;
+
+import javax.annotation.Generated;
+import java.io.Serializable;
+import java.util.List;
+
+@JsonInclude(JsonInclude.Include.NON_NULL)
+@JsonPropertyOrder({
+ "attributes",
+ "browseTime",
+ "clientHostname",
+ "clientId",
+ "completionTime",
+ "creationTime",
+ "fileCount",
+ "id",
+ "instances",
+ "level",
+ "links",
+ "name",
+ "retentionTime",
+ "saveTime",
+ "shortId",
+ "size",
+ "type"
+})
+@Generated("jsonschema2pojo")
+public class Backup implements Serializable {
+
+ private final static long serialVersionUID = -4474500098917286405L;
+ @JsonProperty("attributes")
+ private List<Attribute> attributes = null;
+ @JsonProperty("browseTime")
+ private String browseTime;
+ @JsonProperty("clientHostname")
+ private String clientHostname;
+ @JsonProperty("clientId")
+ private String clientId;
+ @JsonProperty("completionTime")
+ private String completionTime;
+ @JsonProperty("creationTime")
+ private String creationTime;
+ @JsonProperty("fileCount")
+ private Integer fileCount;
+ @JsonProperty("id")
+ private String id;
+ @JsonProperty("instances")
+ private List<Instance> instances = null;
+ @JsonProperty("level")
+ private String level;
+ @JsonProperty("links")
+ private List<Link> links = null;
+ @JsonProperty("name")
+ private String name;
+ @JsonProperty("retentionTime")
+ private String retentionTime;
+ @JsonProperty("saveTime")
+ private String saveTime;
+ @JsonProperty("shortId")
+ private String shortId;
+ @JsonProperty("size")
+ private Size size;
+ @JsonProperty("type")
+ private String type;
+
+ /**
+ * No args constructor for use in serialization
+ */
+ public Backup() {
+ }
+
+ /**
+ * @param shortId
+ * @param clientId
+ * @param browseTime
+ * @param creationTime
+ * @param instances
+ * @param level
+ * @param retentionTime
+ * @param type
+ * @param fileCount
+ * @param clientHostname
+ * @param completionTime
+ * @param size
+ * @param name
+ * @param attributes
+ * @param links
+ * @param id
+ * @param saveTime
+ */
+ public Backup(List<Attribute> attributes, String browseTime, String clientHostname, String clientId, String completionTime, String creationTime, Integer fileCount, String id, List<Instance> instances, String level, List<Link> links, String name, String retentionTime, String saveTime, String shortId, Size size, String type) {
+ super();
+ this.attributes = attributes;
+ this.browseTime = browseTime;
+ this.clientHostname = clientHostname;
+ this.clientId = clientId;
+ this.completionTime = completionTime;
+ this.creationTime = creationTime;
+ this.fileCount = fileCount;
+ this.id = id;
+ this.instances = instances;
+ this.level = level;
+ this.links = links;
+ this.name = name;
+ this.retentionTime = retentionTime;
+ this.saveTime = saveTime;
+ this.shortId = shortId;
+ this.size = size;
+ this.type = type;
+ }
+
+ @JsonProperty("attributes")
+ public List<Attribute> getAttributes() {
+ return attributes;
+ }
+
+ @JsonProperty("attributes")
+ public void setAttributes(List<Attribute> attributes) {
+ this.attributes = attributes;
+ }
+
+ @JsonProperty("browseTime")
+ public String getBrowseTime() {
+ return browseTime;
+ }
+
+ @JsonProperty("browseTime")
+ public void setBrowseTime(String browseTime) {
+ this.browseTime = browseTime;
+ }
+
+ @JsonProperty("clientHostname")
+ public String getClientHostname() {
+ return clientHostname;
+ }
+
+ @JsonProperty("clientHostname")
+ public void setClientHostname(String clientHostname) {
+ this.clientHostname = clientHostname;
+ }
+
+ @JsonProperty("clientId")
+ public String getClientId() {
+ return clientId;
+ }
+
+ @JsonProperty("clientId")
+ public void setClientId(String clientId) {
+ this.clientId = clientId;
+ }
+
+ @JsonProperty("completionTime")
+ public String getCompletionTime() {
+ return completionTime;
+ }
+
+ @JsonProperty("completionTime")
+ public void setCompletionTime(String completionTime) {
+ this.completionTime = completionTime;
+ }
+
+ @JsonProperty("creationTime")
+ public String getCreationTime() {
+ return creationTime;
+ }
+
+ @JsonProperty("creationTime")
+ public void setCreationTime(String creationTime) {
+ this.creationTime = creationTime;
+ }
+
+ @JsonProperty("fileCount")
+ public Integer getFileCount() {
+ return fileCount;
+ }
+
+ @JsonProperty("fileCount")
+ public void setFileCount(Integer fileCount) {
+ this.fileCount = fileCount;
+ }
+
+ @JsonProperty("id")
+ public String getId() {
+ return id;
+ }
+
+ @JsonProperty("id")
+ public void setId(String id) {
+ this.id = id;
+ }
+
+ @JsonProperty("instances")
+ public List<Instance> getInstances() {
+ return instances;
+ }
+
+ @JsonProperty("instances")
+ public void setInstances(List<Instance> instances) {
+ this.instances = instances;
+ }
+
+ @JsonProperty("level")
+ public String getLevel() {
+ return level;
+ }
+
+ @JsonProperty("level")
+ public void setLevel(String level) {
+ this.level = level;
+ }
+
+ @JsonProperty("links")
+ public List<Link> getLinks() {
+ return links;
+ }
+
+ @JsonProperty("links")
+ public void setLinks(List<Link> links) {
+ this.links = links;
+ }
+
+ @JsonProperty("name")
+ public String getName() {
+ return name;
+ }
+
+ @JsonProperty("name")
+ public void setName(String name) {
+ this.name = name;
+ }
+
+ @JsonProperty("retentionTime")
+ public String getRetentionTime() {
+ return retentionTime;
+ }
+
+ @JsonProperty("retentionTime")
+ public void setRetentionTime(String retentionTime) {
+ this.retentionTime = retentionTime;
+ }
+
+ @JsonProperty("saveTime")
+ public String getSaveTime() {
+ return saveTime;
+ }
+
+ @JsonProperty("saveTime")
+ public void setSaveTime(String saveTime) {
+ this.saveTime = saveTime;
+ }
+
+ @JsonProperty("shortId")
+ public String getShortId() {
+ return shortId;
+ }
+
+ @JsonProperty("shortId")
+ public void setShortId(String shortId) {
+ this.shortId = shortId;
+ }
+
+ @JsonProperty("size")
+ public Size getSize() {
+ return size;
+ }
+
+ @JsonProperty("size")
+ public void setSize(Size size) {
+ this.size = size;
+ }
+
+ @JsonProperty("type")
+ public String getType() {
+ return type;
+ }
+
+ @JsonProperty("type")
+ public void setType(String type) {
+ this.type = type;
+ }
+
+ @Override
+ public String toString() {
Review Comment:
there is a `ReflectionToStringBuilderUtils` to dela with this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] acs-robot commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
acs-robot commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1185667340
Found UI changes, kicking a new UI QA build
@blueorangutan ui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #6550: Emc networker b&r
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on PR #6550:
URL: https://github.com/apache/cloudstack/pull/6550#issuecomment-1185668771
UI build: :heavy_multiplication_x:
(SL-JID-1965)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org