You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by GitBox <gi...@apache.org> on 2020/07/31 04:54:16 UTC
[GitHub] [cloudstack] nvazquez opened a new pull request #4235: [VMware] Full OVF properties support
nvazquez opened a new pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235
## Description
WIP description...
## Types of changes
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
## Screenshots (if appropriate):
## How Has This Been Tested?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] shwstppr commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
shwstppr commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-669098811
@blueorangutan package
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#discussion_r463449233
##########
File path: core/src/main/java/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java
##########
@@ -89,7 +89,7 @@ protected Answer execute(CopyCommand cmd) {
return processor.copyTemplateToPrimaryStorage(cmd);
} else if (srcData.getObjectType() == DataObjectType.TEMPLATE && srcDataStore.getRole() == DataStoreRole.Primary &&
destDataStore.getRole() == DataStoreRole.Primary) {
- //clone template to a volume
+ // FR37 pretend to clone template to a volume but actually create a cloned vm
Review comment:
comment applied?
##########
File path: engine/orchestration/src/main/java/com/cloud/vm/VirtualMachineManagerImpl.java
##########
@@ -1096,7 +1098,7 @@ public void orchestrateStart(final String vmUuid, final Map<VirtualMachineProfil
//do not enter volume reuse for next retry, since we want to look for resources outside the volume's cluster
reuseVolume = false;
continue;
- }
+ } // this is the one that gets thrown for all kinds of reasons and might not have to do with capacity :
Review comment:
this comment is a nice place holder. i think we need to reflect in the message created for this exception a hint as to where to look for the real problem.
##########
File path: engine/orchestration/src/main/java/com/cloud/vm/VirtualMachineManagerImpl.java
##########
@@ -1154,7 +1157,7 @@ public void orchestrateStart(final String vmUuid, final Map<VirtualMachineProfil
handlePath(vmTO.getDisks(), vm.getHypervisorType());
cmds = new Commands(Command.OnError.Stop);
-
+ // FR37 TODO vmTO must contain definitions of disks to be expected for deployAsIs
Review comment:
think this is done as well
##########
File path: api/src/main/java/org/apache/cloudstack/api/net/NetworkPrerequisite.java
##########
@@ -0,0 +1,37 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.api.net;
+
+public interface NetworkPrerequisite {
Review comment:
not sure if we need an interface
##########
File path: api/src/main/java/org/apache/cloudstack/api/response/TemplateOVFPropertyResponse.java
##########
@@ -16,13 +16,18 @@
// under the License.
package org.apache.cloudstack.api.response;
-import com.cloud.agent.api.storage.OVFProperty;
-import com.cloud.serializer.Param;
-import com.google.gson.annotations.SerializedName;
import org.apache.cloudstack.api.ApiConstants;
import org.apache.cloudstack.api.BaseResponse;
import org.apache.cloudstack.api.EntityReference;
+import com.cloud.agent.api.storage.OVFProperty;
+import com.cloud.serializer.Param;
+import com.google.gson.annotations.SerializedName;
+
+/**
+ * the placeholder of parameters to fill for deployment
+ // FR37 TODO remname for generic use
Review comment:
todo: decision to be made and comment to remove
##########
File path: engine/schema/src/main/java/com/cloud/storage/dao/VMTemplatePoolDaoImpl.java
##########
@@ -264,7 +264,7 @@ public boolean templateAvailable(long templateId, long hostId) {
@Override
public VMTemplateStoragePoolVO findByHostTemplate(Long hostId, Long templateId) {
List<VMTemplateStoragePoolVO> result = listByHostTemplate(hostId, templateId);
- return (result.size() == 0) ? null : result.get(1);
+ return (result.size() == 0) ? null : result.get(0); // Not sure how this index outofbound has survived the years, or what the rationale was getting the second/result.get(1)
Review comment:
@rhtyd @nvazquez I don't know how to handle this Do you know of any rationale on the old implementation?
if we find an answer we can remove the comment
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/guru/VmwareVmImplementer.java
##########
@@ -317,19 +369,39 @@ private void handleOvfProperties(VirtualMachineProfile vm, VirtualMachineTO to,
}
}
+ private DiskTO getRootDiskTOFromVM(VirtualMachineProfile vm) {
+ DiskTO rootDiskTO;
+ List<DiskTO> rootDiskList;
+ rootDiskList = vm.getDisks().stream().filter(x -> x.getType() == Volume.Type.ROOT).collect(Collectors.toList());
+ if (rootDiskList.size() != 1) {
+ if (vm.getTemplate().isDeployAsIs()) { // FR37 dirty hack to avoid ISOs, the start command should have added a root disk to
+ rootDiskList = vm.getDisks().stream().filter(x -> x.getType() == null).collect(Collectors.toList());
+ if (rootDiskList.size() < 1) {
+ throw new CloudRuntimeException("Did not find a template to serve as root disk for VM " + vm.getHostName());
+ }
+ } else {
+ throw new CloudRuntimeException("Did not find only one root disk for VM " + vm.getHostName());
+ }
+ }
+ rootDiskTO = rootDiskList.get(0);
+ return rootDiskTO;
+ }
+
+ // TODO FR37 phase out ovf properties in favor of template details; propertyTO remains
private List<OVFPropertyTO> getOvfPropertyList(VirtualMachineProfile vm, Map<String, String> details) {
List<OVFPropertyTO> ovfProperties = new ArrayList<OVFPropertyTO>();
for (String detailKey : details.keySet()) {
- if (detailKey.startsWith(ApiConstants.OVF_PROPERTIES)) {
- String ovfPropKey = detailKey.replace(ApiConstants.OVF_PROPERTIES + "-", "");
- TemplateOVFPropertyVO templateOVFPropertyVO = templateOVFPropertiesDao.findByTemplateAndKey(vm.getTemplateId(), ovfPropKey);
- if (templateOVFPropertyVO == null) {
- LOG.warn(String.format("OVF property %s not found on template, discarding", ovfPropKey));
+ if (detailKey.startsWith(ApiConstants.ACS_PROPERTY)) {
+ OVFPropertyTO propertyTO = templateDetailsDao.findPropertyByTemplateAndKey(vm.getTemplateId(), detailKey);
+ String vmPropertyKey = detailKey.replace(ApiConstants.ACS_PROPERTY + "-", "");
+ if (propertyTO == null) {
+ LOGGER.warn(String.format("OVF property %s not found on template, discarding", vmPropertyKey));
continue;
}
- String ovfValue = details.get(detailKey);
- boolean isPassword = templateOVFPropertyVO.isPassword();
- OVFPropertyTO propertyTO = new OVFPropertyTO(ovfPropKey, ovfValue, isPassword);
+ // FR37 the key is without acs prefix (in the TO)
Review comment:
```suggestion
```
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/StartCommandExecutor.java
##########
@@ -0,0 +1,2247 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.hypervisor.vmware.resource;
+
+import java.io.File;
+import java.rmi.RemoteException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.cloud.agent.api.to.DataTO;
+import com.vmware.vim25.VirtualDiskFlatVer2BackingInfo;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.storage.configdrive.ConfigDrive;
+import org.apache.cloudstack.storage.to.TemplateObjectTO;
+import org.apache.cloudstack.storage.to.VolumeObjectTO;
+import org.apache.cloudstack.utils.volume.VirtualMachineDiskInfo;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.commons.lang.ArrayUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.log4j.Logger;
+
+import com.cloud.agent.api.Command;
+import com.cloud.agent.api.StartAnswer;
+import com.cloud.agent.api.StartCommand;
+import com.cloud.agent.api.storage.OVFPropertyTO;
+import com.cloud.agent.api.to.DataStoreTO;
+import com.cloud.agent.api.to.DiskTO;
+import com.cloud.agent.api.to.NfsTO;
+import com.cloud.agent.api.to.NicTO;
+import com.cloud.agent.api.to.VirtualMachineTO;
+import com.cloud.configuration.Resource;
+import com.cloud.hypervisor.vmware.VmwareResourceException;
+import com.cloud.hypervisor.vmware.manager.VmwareManager;
+import com.cloud.hypervisor.vmware.mo.CustomFieldConstants;
+import com.cloud.hypervisor.vmware.mo.DatacenterMO;
+import com.cloud.hypervisor.vmware.mo.DatastoreFile;
+import com.cloud.hypervisor.vmware.mo.DatastoreMO;
+import com.cloud.hypervisor.vmware.mo.DiskControllerType;
+import com.cloud.hypervisor.vmware.mo.HostMO;
+import com.cloud.hypervisor.vmware.mo.HypervisorHostHelper;
+import com.cloud.hypervisor.vmware.mo.TaskMO;
+import com.cloud.hypervisor.vmware.mo.VirtualEthernetCardType;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineDiskInfoBuilder;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineMO;
+import com.cloud.hypervisor.vmware.mo.VmwareHypervisorHost;
+import com.cloud.hypervisor.vmware.util.VmwareContext;
+import com.cloud.hypervisor.vmware.util.VmwareHelper;
+import com.cloud.network.Networks;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Volume;
+import com.cloud.storage.resource.VmwareStorageLayoutHelper;
+import com.cloud.storage.resource.VmwareStorageProcessor;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nicira.nvp.plugin.NiciraNvpApiVersion;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.VmDetailConstants;
+import com.vmware.vim25.BoolPolicy;
+import com.vmware.vim25.DVPortConfigInfo;
+import com.vmware.vim25.DVPortConfigSpec;
+import com.vmware.vim25.DasVmPriority;
+import com.vmware.vim25.DistributedVirtualPort;
+import com.vmware.vim25.DistributedVirtualSwitchPortConnection;
+import com.vmware.vim25.DistributedVirtualSwitchPortCriteria;
+import com.vmware.vim25.ManagedObjectReference;
+import com.vmware.vim25.OptionValue;
+import com.vmware.vim25.VMwareDVSPortSetting;
+import com.vmware.vim25.VirtualDevice;
+import com.vmware.vim25.VirtualDeviceBackingInfo;
+import com.vmware.vim25.VirtualDeviceConfigSpec;
+import com.vmware.vim25.VirtualDeviceConfigSpecOperation;
+import com.vmware.vim25.VirtualDisk;
+import com.vmware.vim25.VirtualEthernetCard;
+import com.vmware.vim25.VirtualEthernetCardDistributedVirtualPortBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardNetworkBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardOpaqueNetworkBackingInfo;
+import com.vmware.vim25.VirtualMachineBootOptions;
+import com.vmware.vim25.VirtualMachineConfigSpec;
+import com.vmware.vim25.VirtualMachineFileInfo;
+import com.vmware.vim25.VirtualMachineFileLayoutEx;
+import com.vmware.vim25.VirtualMachineGuestOsIdentifier;
+import com.vmware.vim25.VirtualUSBController;
+import com.vmware.vim25.VmConfigInfo;
+import com.vmware.vim25.VmwareDistributedVirtualSwitchVlanIdSpec;
+
+class StartCommandExecutor {
+ private static final Logger LOGGER = Logger.getLogger(StartCommandExecutor.class);
+
+ private final VmwareResource vmwareResource;
+
+ public StartCommandExecutor(VmwareResource vmwareResource) {
+ this.vmwareResource = vmwareResource;
+ }
+
+ // FR37 the monster blob god method: reduce to 320 so far and counting
+ protected StartAnswer execute(StartCommand cmd) {
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Executing resource StartCommand: " + vmwareResource.getGson().toJson(cmd));
+ }
+
+ VirtualMachineTO vmSpec = cmd.getVirtualMachine();
+
+ VirtualMachineData existingVm = null;
+
+ Pair<String, String> names = composeVmNames(vmSpec);
+ String vmInternalCSName = names.first();
+ String vmNameOnVcenter = names.second();
+
+ DiskTO rootDiskTO = null;
+
+ Pair<String, String> controllerInfo = getDiskControllerInfo(vmSpec);
+
+ Boolean systemVm = vmSpec.getType().isUsedBySystem();
+
+ VmwareContext context = vmwareResource.getServiceContext();
+ DatacenterMO dcMo = null;
+ try {
+ VmwareManager mgr = context.getStockObject(VmwareManager.CONTEXT_STOCK_NAME);
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(context);
+ dcMo = new DatacenterMO(hyperHost.getContext(), hyperHost.getHyperHostDatacenter());
+
+ // checkIfVmExistsInVcenter(vmInternalCSName, vmNameOnVcenter, dcMo);
+ // FR37 - We expect VM to be already cloned and available at this point
+ VirtualMachineMO vmMo = dcMo.findVm(vmInternalCSName);
+ if (vmMo == null) {
+ vmMo = dcMo.findVm(vmNameOnVcenter);
+ }
+
+ DiskTO[] specDisks = vmSpec.getDisks();
+ boolean installAsIs = StringUtils.isNotEmpty(vmSpec.getTemplateLocation());
+ // FR37 if startcommand contains enough info: a template url/-location and flag; deploy OVA as is
+ if (vmMo == null && installAsIs) {
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("deploying OVA from %s as is", vmSpec.getTemplateLocation()));
+ }
+ getStorageProcessor().cloneVMFromTemplate(vmSpec.getTemplateName(), vmInternalCSName, vmSpec.getTemplatePrimaryStoreUuid());
+ vmMo = dcMo.findVm(vmInternalCSName);
+ mapSpecDisksToClonedDisks(vmMo, vmInternalCSName, specDisks);
+ }
+
+ // VM may not have been on the same host, relocate to expected host
+ if (vmMo != null) {
+ vmMo.relocate(hyperHost.getMor());
+ // Get updated MO
+ vmMo = hyperHost.findVmOnHyperHost(vmMo.getVmName());
+ }
+
+ String guestOsId = translateGuestOsIdentifier(vmSpec.getArch(), vmSpec.getOs(), vmSpec.getPlatformEmulator()).value();
+ DiskTO[] disks = validateDisks(specDisks);
+ NicTO[] nics = vmSpec.getNics();
+
+ // FIXME: disks logic here, why is disks/volumes during copy not set with pool ID?
+ // FR37 TODO if deployasis a new VM no datastores are known (yet) and we need to get the data store from the tvmspec content library / template location
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails = inferDatastoreDetailsFromDiskInfo(hyperHost, context, disks, cmd);
+ if ((dataStoresDetails == null) || (dataStoresDetails.isEmpty())) {
+ String msg = "Unable to locate datastore details of the volumes to be attached";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ // FR37 - this may not be necessary the cloned VM will have disks and knowledge of datastore paths/location?
+ // throw new Exception(msg);
+ }
+
+ // FR37 - this may need checking, if the first datastore is the right one - ideally it should be datastore where first disk is hosted
+ DatastoreMO dsRootVolumeIsOn = null; //
Review comment:
```suggestion
DatastoreMO dsRootVolumeIsOn = null;
```
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/StartCommandExecutor.java
##########
@@ -0,0 +1,2247 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.hypervisor.vmware.resource;
+
+import java.io.File;
+import java.rmi.RemoteException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.cloud.agent.api.to.DataTO;
+import com.vmware.vim25.VirtualDiskFlatVer2BackingInfo;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.storage.configdrive.ConfigDrive;
+import org.apache.cloudstack.storage.to.TemplateObjectTO;
+import org.apache.cloudstack.storage.to.VolumeObjectTO;
+import org.apache.cloudstack.utils.volume.VirtualMachineDiskInfo;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.commons.lang.ArrayUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.log4j.Logger;
+
+import com.cloud.agent.api.Command;
+import com.cloud.agent.api.StartAnswer;
+import com.cloud.agent.api.StartCommand;
+import com.cloud.agent.api.storage.OVFPropertyTO;
+import com.cloud.agent.api.to.DataStoreTO;
+import com.cloud.agent.api.to.DiskTO;
+import com.cloud.agent.api.to.NfsTO;
+import com.cloud.agent.api.to.NicTO;
+import com.cloud.agent.api.to.VirtualMachineTO;
+import com.cloud.configuration.Resource;
+import com.cloud.hypervisor.vmware.VmwareResourceException;
+import com.cloud.hypervisor.vmware.manager.VmwareManager;
+import com.cloud.hypervisor.vmware.mo.CustomFieldConstants;
+import com.cloud.hypervisor.vmware.mo.DatacenterMO;
+import com.cloud.hypervisor.vmware.mo.DatastoreFile;
+import com.cloud.hypervisor.vmware.mo.DatastoreMO;
+import com.cloud.hypervisor.vmware.mo.DiskControllerType;
+import com.cloud.hypervisor.vmware.mo.HostMO;
+import com.cloud.hypervisor.vmware.mo.HypervisorHostHelper;
+import com.cloud.hypervisor.vmware.mo.TaskMO;
+import com.cloud.hypervisor.vmware.mo.VirtualEthernetCardType;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineDiskInfoBuilder;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineMO;
+import com.cloud.hypervisor.vmware.mo.VmwareHypervisorHost;
+import com.cloud.hypervisor.vmware.util.VmwareContext;
+import com.cloud.hypervisor.vmware.util.VmwareHelper;
+import com.cloud.network.Networks;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Volume;
+import com.cloud.storage.resource.VmwareStorageLayoutHelper;
+import com.cloud.storage.resource.VmwareStorageProcessor;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nicira.nvp.plugin.NiciraNvpApiVersion;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.VmDetailConstants;
+import com.vmware.vim25.BoolPolicy;
+import com.vmware.vim25.DVPortConfigInfo;
+import com.vmware.vim25.DVPortConfigSpec;
+import com.vmware.vim25.DasVmPriority;
+import com.vmware.vim25.DistributedVirtualPort;
+import com.vmware.vim25.DistributedVirtualSwitchPortConnection;
+import com.vmware.vim25.DistributedVirtualSwitchPortCriteria;
+import com.vmware.vim25.ManagedObjectReference;
+import com.vmware.vim25.OptionValue;
+import com.vmware.vim25.VMwareDVSPortSetting;
+import com.vmware.vim25.VirtualDevice;
+import com.vmware.vim25.VirtualDeviceBackingInfo;
+import com.vmware.vim25.VirtualDeviceConfigSpec;
+import com.vmware.vim25.VirtualDeviceConfigSpecOperation;
+import com.vmware.vim25.VirtualDisk;
+import com.vmware.vim25.VirtualEthernetCard;
+import com.vmware.vim25.VirtualEthernetCardDistributedVirtualPortBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardNetworkBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardOpaqueNetworkBackingInfo;
+import com.vmware.vim25.VirtualMachineBootOptions;
+import com.vmware.vim25.VirtualMachineConfigSpec;
+import com.vmware.vim25.VirtualMachineFileInfo;
+import com.vmware.vim25.VirtualMachineFileLayoutEx;
+import com.vmware.vim25.VirtualMachineGuestOsIdentifier;
+import com.vmware.vim25.VirtualUSBController;
+import com.vmware.vim25.VmConfigInfo;
+import com.vmware.vim25.VmwareDistributedVirtualSwitchVlanIdSpec;
+
+class StartCommandExecutor {
+ private static final Logger LOGGER = Logger.getLogger(StartCommandExecutor.class);
+
+ private final VmwareResource vmwareResource;
+
+ public StartCommandExecutor(VmwareResource vmwareResource) {
+ this.vmwareResource = vmwareResource;
+ }
+
+ // FR37 the monster blob god method: reduce to 320 so far and counting
+ protected StartAnswer execute(StartCommand cmd) {
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Executing resource StartCommand: " + vmwareResource.getGson().toJson(cmd));
+ }
+
+ VirtualMachineTO vmSpec = cmd.getVirtualMachine();
+
+ VirtualMachineData existingVm = null;
+
+ Pair<String, String> names = composeVmNames(vmSpec);
+ String vmInternalCSName = names.first();
+ String vmNameOnVcenter = names.second();
+
+ DiskTO rootDiskTO = null;
+
+ Pair<String, String> controllerInfo = getDiskControllerInfo(vmSpec);
+
+ Boolean systemVm = vmSpec.getType().isUsedBySystem();
+
+ VmwareContext context = vmwareResource.getServiceContext();
+ DatacenterMO dcMo = null;
+ try {
+ VmwareManager mgr = context.getStockObject(VmwareManager.CONTEXT_STOCK_NAME);
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(context);
+ dcMo = new DatacenterMO(hyperHost.getContext(), hyperHost.getHyperHostDatacenter());
+
+ // checkIfVmExistsInVcenter(vmInternalCSName, vmNameOnVcenter, dcMo);
+ // FR37 - We expect VM to be already cloned and available at this point
+ VirtualMachineMO vmMo = dcMo.findVm(vmInternalCSName);
+ if (vmMo == null) {
+ vmMo = dcMo.findVm(vmNameOnVcenter);
+ }
+
+ DiskTO[] specDisks = vmSpec.getDisks();
+ boolean installAsIs = StringUtils.isNotEmpty(vmSpec.getTemplateLocation());
+ // FR37 if startcommand contains enough info: a template url/-location and flag; deploy OVA as is
+ if (vmMo == null && installAsIs) {
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("deploying OVA from %s as is", vmSpec.getTemplateLocation()));
+ }
+ getStorageProcessor().cloneVMFromTemplate(vmSpec.getTemplateName(), vmInternalCSName, vmSpec.getTemplatePrimaryStoreUuid());
+ vmMo = dcMo.findVm(vmInternalCSName);
+ mapSpecDisksToClonedDisks(vmMo, vmInternalCSName, specDisks);
+ }
+
+ // VM may not have been on the same host, relocate to expected host
+ if (vmMo != null) {
+ vmMo.relocate(hyperHost.getMor());
+ // Get updated MO
+ vmMo = hyperHost.findVmOnHyperHost(vmMo.getVmName());
+ }
+
+ String guestOsId = translateGuestOsIdentifier(vmSpec.getArch(), vmSpec.getOs(), vmSpec.getPlatformEmulator()).value();
+ DiskTO[] disks = validateDisks(specDisks);
+ NicTO[] nics = vmSpec.getNics();
+
+ // FIXME: disks logic here, why is disks/volumes during copy not set with pool ID?
+ // FR37 TODO if deployasis a new VM no datastores are known (yet) and we need to get the data store from the tvmspec content library / template location
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails = inferDatastoreDetailsFromDiskInfo(hyperHost, context, disks, cmd);
+ if ((dataStoresDetails == null) || (dataStoresDetails.isEmpty())) {
+ String msg = "Unable to locate datastore details of the volumes to be attached";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ // FR37 - this may not be necessary the cloned VM will have disks and knowledge of datastore paths/location?
+ // throw new Exception(msg);
+ }
+
+ // FR37 - this may need checking, if the first datastore is the right one - ideally it should be datastore where first disk is hosted
+ DatastoreMO dsRootVolumeIsOn = null; //
+ if (! installAsIs) {
+ dsRootVolumeIsOn = getDatastoreThatRootDiskIsOn(dataStoresDetails, disks);
+
+ if (dsRootVolumeIsOn == null) {
+ String msg = "Unable to locate datastore details of root volume";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ throw new VmwareResourceException(msg);
+ }
+ }
+
+ VirtualMachineDiskInfoBuilder diskInfoBuilder = null;
+ VirtualDevice[] nicDevices = null;
+ DiskControllerType systemVmScsiControllerType = DiskControllerType.lsilogic;
+ int firstScsiControllerBusNum = 0;
+ int numScsiControllerForSystemVm = 1;
+ boolean hasSnapshot = false;
+ if (vmMo != null) {
+ PrepareRunningVMForConfiguration prepareRunningVMForConfiguration = new PrepareRunningVMForConfiguration(vmInternalCSName, controllerInfo, systemVm, vmMo,
+ systemVmScsiControllerType, firstScsiControllerBusNum, numScsiControllerForSystemVm);
+ prepareRunningVMForConfiguration.invoke();
+ diskInfoBuilder = prepareRunningVMForConfiguration.getDiskInfoBuilder();
+ nicDevices = prepareRunningVMForConfiguration.getNicDevices();
+ hasSnapshot = prepareRunningVMForConfiguration.isHasSnapshot();
+ } else {
+ ManagedObjectReference morDc = hyperHost.getHyperHostDatacenter();
+ assert (morDc != null);
+
+ vmMo = hyperHost.findVmOnPeerHyperHost(vmInternalCSName);
+ if (vmMo != null) {
+ VirtualMachineRecycler virtualMachineRecycler = new VirtualMachineRecycler(vmInternalCSName, controllerInfo, systemVm, hyperHost, vmMo,
+ systemVmScsiControllerType, firstScsiControllerBusNum, numScsiControllerForSystemVm);
+ virtualMachineRecycler.invoke();
+ diskInfoBuilder = virtualMachineRecycler.getDiskInfoBuilder();
+ hasSnapshot = virtualMachineRecycler.isHasSnapshot();
+ nicDevices = virtualMachineRecycler.getNicDevices();
+ } else {
+ // FR37 we just didn't find a VM by name 'vmInternalCSName', so why is this here?
+ existingVm = unregisterOnOtherClusterButHoldOnToOldVmData(vmInternalCSName, dcMo);
+
+ createNewVm(context, vmSpec, installAsIs, vmInternalCSName, vmNameOnVcenter, controllerInfo, systemVm, mgr, hyperHost, guestOsId, disks, dataStoresDetails,
+ dsRootVolumeIsOn);
+ }
+
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ if (vmMo == null) {
+ throw new Exception("Failed to find the newly create or relocated VM. vmName: " + vmInternalCSName);
+ }
+ }
+ // vmMo should now be a stopped VM on the intended host
+ // The number of disks changed must be 0 for install as is, as the VM is a clone
+ int disksChanges = !installAsIs ? disks.length : 0;
+ int totalChangeDevices = disksChanges + nics.length;
+ int hackDeviceCount = 0;
+ if (diskInfoBuilder != null) {
+ hackDeviceCount += diskInfoBuilder.getDiskCount();
+ }
+ hackDeviceCount += nicDevices == null ? 0 : nicDevices.length;
+ // vApp cdrom device
+ // HACK ALERT: ovf properties might not be the only or defining feature of vApps; needs checking
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("current count(s) desired: %d/ found:%d. now adding device to device count for vApp config ISO", totalChangeDevices, hackDeviceCount));
+ }
+ if (vmSpec.getOvfProperties() != null) {
+ totalChangeDevices++;
+ }
+
+ DiskTO volIso = null;
+ if (vmSpec.getType() != VirtualMachine.Type.User) {
+ // system VM needs a patch ISO
+ totalChangeDevices++;
+ } else {
+ volIso = getIsoDiskTO(disks);
+ if (volIso == null)
+ totalChangeDevices++;
+ }
+
+ VirtualMachineConfigSpec vmConfigSpec = new VirtualMachineConfigSpec();
+
+ VmwareHelper.setBasicVmConfig(vmConfigSpec, vmSpec.getCpus(), vmSpec.getMaxSpeed(), vmwareResource.getReservedCpuMHZ(vmSpec), (int)(vmSpec.getMaxRam() / (1024 * 1024)),
+ vmwareResource.getReservedMemoryMb(vmSpec), guestOsId, vmSpec.getLimitCpuUse());
+
+ int numCoresPerSocket = adjustNumberOfCoresPerSocket(vmSpec, vmMo, vmConfigSpec);
+
+ // Check for hotadd settings
+ vmConfigSpec.setMemoryHotAddEnabled(vmMo.isMemoryHotAddSupported(guestOsId));
+
+ String hostApiVersion = ((HostMO)hyperHost).getHostAboutInfo().getApiVersion();
+ if (numCoresPerSocket > 1 && hostApiVersion.compareTo("5.0") < 0) {
+ LOGGER.warn("Dynamic scaling of CPU is not supported for Virtual Machines with multi-core vCPUs in case of ESXi hosts 4.1 and prior. Hence CpuHotAdd will not be"
+ + " enabled for Virtual Machine: " + vmInternalCSName);
+ vmConfigSpec.setCpuHotAddEnabled(false);
+ } else {
+ vmConfigSpec.setCpuHotAddEnabled(vmMo.isCpuHotAddSupported(guestOsId));
+ }
+
+ vmwareResource.configNestedHVSupport(vmMo, vmSpec, vmConfigSpec);
+
+ VirtualDeviceConfigSpec[] deviceConfigSpecArray = new VirtualDeviceConfigSpec[totalChangeDevices];
+ int deviceCount = 0;
+ int ideUnitNumber = 0;
+ int scsiUnitNumber = 0;
+ int ideControllerKey = vmMo.getIDEDeviceControllerKey();
+ int scsiControllerKey = vmMo.getScsiDeviceControllerKeyNoException();
+
+ IsoSetup isoSetup = new IsoSetup(vmSpec, mgr, hyperHost, vmMo, disks, volIso, deviceConfigSpecArray, deviceCount, ideUnitNumber);
+ isoSetup.invoke();
+ deviceCount = isoSetup.getDeviceCount();
+ ideUnitNumber = isoSetup.getIdeUnitNumber();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("setting up %d disks with root from %s", diskInfoBuilder.getDiskCount(), vmwareResource.getGson().toJson(rootDiskTO)));
+ }
+
+ DiskSetup diskSetup = new DiskSetup(vmSpec, rootDiskTO, controllerInfo, context, dcMo, hyperHost, vmMo, disks, dataStoresDetails, diskInfoBuilder, hasSnapshot,
+ deviceConfigSpecArray, deviceCount, ideUnitNumber, scsiUnitNumber, ideControllerKey, scsiControllerKey, installAsIs);
+ diskSetup.invoke();
+
+ rootDiskTO = diskSetup.getRootDiskTO();
+ deviceCount = diskSetup.getDeviceCount();
+ DiskTO[] sortedDisks = diskSetup.getSortedDisks();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("seting up %d disks with root from %s", sortedDisks.length, vmwareResource.getGson().toJson(rootDiskTO)));
+ }
+
+ deviceCount += setupUsbDevicesAndGetCount(vmInternalCSName, vmMo, guestOsId, deviceConfigSpecArray, deviceCount);
+
+ NicSetup nicSetup = new NicSetup(cmd, vmSpec, vmInternalCSName, context, mgr, hyperHost, vmMo, nics, deviceConfigSpecArray, deviceCount, nicDevices);
+ nicSetup.invoke();
+
+ deviceCount = nicSetup.getDeviceCount();
+ int nicMask = nicSetup.getNicMask();
+ int nicCount = nicSetup.getNicCount();
+ Map<String, String> nicUuidToDvSwitchUuid = nicSetup.getNicUuidToDvSwitchUuid();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("device count: %d; nic count %d", deviceCount, nicCount));
+ }
+ for (int j = 0; j < deviceCount; j++)
+ vmConfigSpec.getDeviceChange().add(deviceConfigSpecArray[j]);
+
+ //
+ // Setup VM options
+ //
+
+ // pass boot arguments through machine.id & perform customized options to VMX
+ ArrayList<OptionValue> extraOptions = new ArrayList<>();
+ configBasicExtraOption(extraOptions, vmSpec);
+ configNvpExtraOption(extraOptions, vmSpec, nicUuidToDvSwitchUuid);
+ configCustomExtraOption(extraOptions, vmSpec);
+
+ // config for NCC
+ VirtualMachine.Type vmType = cmd.getVirtualMachine().getType();
+ if (vmType.equals(VirtualMachine.Type.NetScalerVm)) {
+ NicTO mgmtNic = vmSpec.getNics()[0];
+ OptionValue option = new OptionValue();
+ option.setKey("machine.id");
+ option.setValue("ip=" + mgmtNic.getIp() + "&netmask=" + mgmtNic.getNetmask() + "&gateway=" + mgmtNic.getGateway());
+ extraOptions.add(option);
+ }
+
+ // config VNC
+ String keyboardLayout = null;
+ if (vmSpec.getDetails() != null)
+ keyboardLayout = vmSpec.getDetails().get(VmDetailConstants.KEYBOARD);
+ vmConfigSpec.getExtraConfig().addAll(
+ Arrays.asList(vmwareResource.configureVnc(extraOptions.toArray(new OptionValue[0]), hyperHost, vmInternalCSName, vmSpec.getVncPassword(), keyboardLayout)));
+
+ // config video card
+ vmwareResource.configureVideoCard(vmMo, vmSpec, vmConfigSpec);
+
+ // Set OVF properties (if available)
+ Pair<String, List<OVFPropertyTO>> ovfPropsMap = vmSpec.getOvfProperties();
+ VmConfigInfo templateVappConfig;
+ List<OVFPropertyTO> ovfProperties;
+ if (ovfPropsMap != null) {
+ String vmTemplate = ovfPropsMap.first();
+ LOGGER.info("Find VM template " + vmTemplate);
+ VirtualMachineMO vmTemplateMO = dcMo.findVm(vmTemplate);
+ templateVappConfig = vmTemplateMO.getConfigInfo().getVAppConfig();
+ ovfProperties = ovfPropsMap.second();
+ // Set OVF properties (if available)
+ if (CollectionUtils.isNotEmpty(ovfProperties)) {
+ LOGGER.info("Copying OVF properties from template and setting them to the values the user provided");
+ vmwareResource.copyVAppConfigsFromTemplate(templateVappConfig, ovfProperties, vmConfigSpec);
+ }
+ }
+
+ checkBootOptions(vmSpec, vmConfigSpec);
+
+ //
+ // Configure VM
+ //
+ if (!vmMo.configureVm(vmConfigSpec)) {
+ throw new Exception("Failed to configure VM before start. vmName: " + vmInternalCSName);
+ }
+ // FR37 TODO reconcile disks now!!! they are configured, check and add them to the returning vmTO
Review comment:
```suggestion
```
i think, valid?
##########
File path: server/src/main/java/com/cloud/vm/UserVmManagerImpl.java
##########
@@ -4009,6 +4029,56 @@ public UserVmVO doInTransaction(TransactionStatus status) throws InsufficientCap
});
}
+ private void copyNetworkRequirementsToVm(UserVmVO vm, VirtualMachineTemplate template) {
+ if (template.isDeployAsIs()) { // FR37 should be always when we are done
Review comment:
```suggestion
if (template.isDeployAsIs()) { // This should be always when we are done, i.e. after a cooling down period.
```
##########
File path: api/src/main/java/org/apache/cloudstack/api/command/admin/template/RegisterTemplateCmdByAdmin.java
##########
@@ -17,10 +17,32 @@
package org.apache.cloudstack.api.command.admin.template;
import org.apache.cloudstack.api.APICommand;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.api.Parameter;
import org.apache.cloudstack.api.ResponseObject.ResponseView;
import org.apache.cloudstack.api.command.user.template.RegisterTemplateCmd;
import org.apache.cloudstack.api.response.TemplateResponse;
@APICommand(name = "registerTemplate", description = "Registers an existing template into the CloudStack cloud.", responseObject = TemplateResponse.class, responseView = ResponseView.Full,
requestHasSensitiveInfo = false, responseHasSensitiveInfo = false)
-public class RegisterTemplateCmdByAdmin extends RegisterTemplateCmd {}
+public class RegisterTemplateCmdByAdmin extends RegisterTemplateCmd {
+
+ /////////////////////////////////////////////////////
+ //////////////// +API parameter /////////////////////
+ /////////////////////////////////////////////////////
+
+ @Parameter(name= ApiConstants.DEPLOY_AS_IS,
+ type = CommandType.BOOLEAN,
+ description = "true if template should not strip and define disks and networks but leave those to the template definition",
Review comment:
maybe we should add some description about true being the default and false only for backwards incompatible avoidance.
##########
File path: api/src/main/java/com/cloud/agent/api/storage/OVFProperty.java
##########
@@ -19,6 +19,7 @@
package com.cloud.agent.api.storage;
+// FR37 rename
Review comment:
Do we really need this interface? isn't the TO fit for purpose?
##########
File path: engine/schema/src/main/java/com/cloud/storage/TemplateOVFPropertyVO.java
##########
@@ -28,6 +28,7 @@
@Entity
@Table(name = "template_ovf_properties")
+@Deprecated(since = "now" , forRemoval = true)
Review comment:
can we already remove? do we need any conversion? the feature was highly experimental so i think we can just delete.
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/StartCommandExecutor.java
##########
@@ -0,0 +1,2247 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.hypervisor.vmware.resource;
+
+import java.io.File;
+import java.rmi.RemoteException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.cloud.agent.api.to.DataTO;
+import com.vmware.vim25.VirtualDiskFlatVer2BackingInfo;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.storage.configdrive.ConfigDrive;
+import org.apache.cloudstack.storage.to.TemplateObjectTO;
+import org.apache.cloudstack.storage.to.VolumeObjectTO;
+import org.apache.cloudstack.utils.volume.VirtualMachineDiskInfo;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.commons.lang.ArrayUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.log4j.Logger;
+
+import com.cloud.agent.api.Command;
+import com.cloud.agent.api.StartAnswer;
+import com.cloud.agent.api.StartCommand;
+import com.cloud.agent.api.storage.OVFPropertyTO;
+import com.cloud.agent.api.to.DataStoreTO;
+import com.cloud.agent.api.to.DiskTO;
+import com.cloud.agent.api.to.NfsTO;
+import com.cloud.agent.api.to.NicTO;
+import com.cloud.agent.api.to.VirtualMachineTO;
+import com.cloud.configuration.Resource;
+import com.cloud.hypervisor.vmware.VmwareResourceException;
+import com.cloud.hypervisor.vmware.manager.VmwareManager;
+import com.cloud.hypervisor.vmware.mo.CustomFieldConstants;
+import com.cloud.hypervisor.vmware.mo.DatacenterMO;
+import com.cloud.hypervisor.vmware.mo.DatastoreFile;
+import com.cloud.hypervisor.vmware.mo.DatastoreMO;
+import com.cloud.hypervisor.vmware.mo.DiskControllerType;
+import com.cloud.hypervisor.vmware.mo.HostMO;
+import com.cloud.hypervisor.vmware.mo.HypervisorHostHelper;
+import com.cloud.hypervisor.vmware.mo.TaskMO;
+import com.cloud.hypervisor.vmware.mo.VirtualEthernetCardType;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineDiskInfoBuilder;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineMO;
+import com.cloud.hypervisor.vmware.mo.VmwareHypervisorHost;
+import com.cloud.hypervisor.vmware.util.VmwareContext;
+import com.cloud.hypervisor.vmware.util.VmwareHelper;
+import com.cloud.network.Networks;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Volume;
+import com.cloud.storage.resource.VmwareStorageLayoutHelper;
+import com.cloud.storage.resource.VmwareStorageProcessor;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nicira.nvp.plugin.NiciraNvpApiVersion;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.VmDetailConstants;
+import com.vmware.vim25.BoolPolicy;
+import com.vmware.vim25.DVPortConfigInfo;
+import com.vmware.vim25.DVPortConfigSpec;
+import com.vmware.vim25.DasVmPriority;
+import com.vmware.vim25.DistributedVirtualPort;
+import com.vmware.vim25.DistributedVirtualSwitchPortConnection;
+import com.vmware.vim25.DistributedVirtualSwitchPortCriteria;
+import com.vmware.vim25.ManagedObjectReference;
+import com.vmware.vim25.OptionValue;
+import com.vmware.vim25.VMwareDVSPortSetting;
+import com.vmware.vim25.VirtualDevice;
+import com.vmware.vim25.VirtualDeviceBackingInfo;
+import com.vmware.vim25.VirtualDeviceConfigSpec;
+import com.vmware.vim25.VirtualDeviceConfigSpecOperation;
+import com.vmware.vim25.VirtualDisk;
+import com.vmware.vim25.VirtualEthernetCard;
+import com.vmware.vim25.VirtualEthernetCardDistributedVirtualPortBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardNetworkBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardOpaqueNetworkBackingInfo;
+import com.vmware.vim25.VirtualMachineBootOptions;
+import com.vmware.vim25.VirtualMachineConfigSpec;
+import com.vmware.vim25.VirtualMachineFileInfo;
+import com.vmware.vim25.VirtualMachineFileLayoutEx;
+import com.vmware.vim25.VirtualMachineGuestOsIdentifier;
+import com.vmware.vim25.VirtualUSBController;
+import com.vmware.vim25.VmConfigInfo;
+import com.vmware.vim25.VmwareDistributedVirtualSwitchVlanIdSpec;
+
+class StartCommandExecutor {
+ private static final Logger LOGGER = Logger.getLogger(StartCommandExecutor.class);
+
+ private final VmwareResource vmwareResource;
+
+ public StartCommandExecutor(VmwareResource vmwareResource) {
+ this.vmwareResource = vmwareResource;
+ }
+
+ // FR37 the monster blob god method: reduce to 320 so far and counting
+ protected StartAnswer execute(StartCommand cmd) {
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Executing resource StartCommand: " + vmwareResource.getGson().toJson(cmd));
+ }
+
+ VirtualMachineTO vmSpec = cmd.getVirtualMachine();
+
+ VirtualMachineData existingVm = null;
+
+ Pair<String, String> names = composeVmNames(vmSpec);
+ String vmInternalCSName = names.first();
+ String vmNameOnVcenter = names.second();
+
+ DiskTO rootDiskTO = null;
+
+ Pair<String, String> controllerInfo = getDiskControllerInfo(vmSpec);
+
+ Boolean systemVm = vmSpec.getType().isUsedBySystem();
+
+ VmwareContext context = vmwareResource.getServiceContext();
+ DatacenterMO dcMo = null;
+ try {
+ VmwareManager mgr = context.getStockObject(VmwareManager.CONTEXT_STOCK_NAME);
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(context);
+ dcMo = new DatacenterMO(hyperHost.getContext(), hyperHost.getHyperHostDatacenter());
+
+ // checkIfVmExistsInVcenter(vmInternalCSName, vmNameOnVcenter, dcMo);
+ // FR37 - We expect VM to be already cloned and available at this point
+ VirtualMachineMO vmMo = dcMo.findVm(vmInternalCSName);
+ if (vmMo == null) {
+ vmMo = dcMo.findVm(vmNameOnVcenter);
+ }
+
+ DiskTO[] specDisks = vmSpec.getDisks();
+ boolean installAsIs = StringUtils.isNotEmpty(vmSpec.getTemplateLocation());
+ // FR37 if startcommand contains enough info: a template url/-location and flag; deploy OVA as is
+ if (vmMo == null && installAsIs) {
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("deploying OVA from %s as is", vmSpec.getTemplateLocation()));
+ }
+ getStorageProcessor().cloneVMFromTemplate(vmSpec.getTemplateName(), vmInternalCSName, vmSpec.getTemplatePrimaryStoreUuid());
+ vmMo = dcMo.findVm(vmInternalCSName);
+ mapSpecDisksToClonedDisks(vmMo, vmInternalCSName, specDisks);
+ }
+
+ // VM may not have been on the same host, relocate to expected host
+ if (vmMo != null) {
+ vmMo.relocate(hyperHost.getMor());
+ // Get updated MO
+ vmMo = hyperHost.findVmOnHyperHost(vmMo.getVmName());
+ }
+
+ String guestOsId = translateGuestOsIdentifier(vmSpec.getArch(), vmSpec.getOs(), vmSpec.getPlatformEmulator()).value();
+ DiskTO[] disks = validateDisks(specDisks);
+ NicTO[] nics = vmSpec.getNics();
+
+ // FIXME: disks logic here, why is disks/volumes during copy not set with pool ID?
+ // FR37 TODO if deployasis a new VM no datastores are known (yet) and we need to get the data store from the tvmspec content library / template location
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails = inferDatastoreDetailsFromDiskInfo(hyperHost, context, disks, cmd);
+ if ((dataStoresDetails == null) || (dataStoresDetails.isEmpty())) {
+ String msg = "Unable to locate datastore details of the volumes to be attached";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ // FR37 - this may not be necessary the cloned VM will have disks and knowledge of datastore paths/location?
+ // throw new Exception(msg);
+ }
+
+ // FR37 - this may need checking, if the first datastore is the right one - ideally it should be datastore where first disk is hosted
+ DatastoreMO dsRootVolumeIsOn = null; //
+ if (! installAsIs) {
+ dsRootVolumeIsOn = getDatastoreThatRootDiskIsOn(dataStoresDetails, disks);
+
+ if (dsRootVolumeIsOn == null) {
+ String msg = "Unable to locate datastore details of root volume";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ throw new VmwareResourceException(msg);
+ }
+ }
+
+ VirtualMachineDiskInfoBuilder diskInfoBuilder = null;
+ VirtualDevice[] nicDevices = null;
+ DiskControllerType systemVmScsiControllerType = DiskControllerType.lsilogic;
+ int firstScsiControllerBusNum = 0;
+ int numScsiControllerForSystemVm = 1;
+ boolean hasSnapshot = false;
+ if (vmMo != null) {
+ PrepareRunningVMForConfiguration prepareRunningVMForConfiguration = new PrepareRunningVMForConfiguration(vmInternalCSName, controllerInfo, systemVm, vmMo,
+ systemVmScsiControllerType, firstScsiControllerBusNum, numScsiControllerForSystemVm);
+ prepareRunningVMForConfiguration.invoke();
+ diskInfoBuilder = prepareRunningVMForConfiguration.getDiskInfoBuilder();
+ nicDevices = prepareRunningVMForConfiguration.getNicDevices();
+ hasSnapshot = prepareRunningVMForConfiguration.isHasSnapshot();
+ } else {
+ ManagedObjectReference morDc = hyperHost.getHyperHostDatacenter();
+ assert (morDc != null);
+
+ vmMo = hyperHost.findVmOnPeerHyperHost(vmInternalCSName);
+ if (vmMo != null) {
+ VirtualMachineRecycler virtualMachineRecycler = new VirtualMachineRecycler(vmInternalCSName, controllerInfo, systemVm, hyperHost, vmMo,
+ systemVmScsiControllerType, firstScsiControllerBusNum, numScsiControllerForSystemVm);
+ virtualMachineRecycler.invoke();
+ diskInfoBuilder = virtualMachineRecycler.getDiskInfoBuilder();
+ hasSnapshot = virtualMachineRecycler.isHasSnapshot();
+ nicDevices = virtualMachineRecycler.getNicDevices();
+ } else {
+ // FR37 we just didn't find a VM by name 'vmInternalCSName', so why is this here?
+ existingVm = unregisterOnOtherClusterButHoldOnToOldVmData(vmInternalCSName, dcMo);
+
+ createNewVm(context, vmSpec, installAsIs, vmInternalCSName, vmNameOnVcenter, controllerInfo, systemVm, mgr, hyperHost, guestOsId, disks, dataStoresDetails,
+ dsRootVolumeIsOn);
+ }
+
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ if (vmMo == null) {
+ throw new Exception("Failed to find the newly create or relocated VM. vmName: " + vmInternalCSName);
+ }
+ }
+ // vmMo should now be a stopped VM on the intended host
+ // The number of disks changed must be 0 for install as is, as the VM is a clone
+ int disksChanges = !installAsIs ? disks.length : 0;
+ int totalChangeDevices = disksChanges + nics.length;
+ int hackDeviceCount = 0;
+ if (diskInfoBuilder != null) {
+ hackDeviceCount += diskInfoBuilder.getDiskCount();
+ }
+ hackDeviceCount += nicDevices == null ? 0 : nicDevices.length;
+ // vApp cdrom device
+ // HACK ALERT: ovf properties might not be the only or defining feature of vApps; needs checking
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("current count(s) desired: %d/ found:%d. now adding device to device count for vApp config ISO", totalChangeDevices, hackDeviceCount));
+ }
+ if (vmSpec.getOvfProperties() != null) {
+ totalChangeDevices++;
+ }
+
+ DiskTO volIso = null;
+ if (vmSpec.getType() != VirtualMachine.Type.User) {
+ // system VM needs a patch ISO
+ totalChangeDevices++;
+ } else {
+ volIso = getIsoDiskTO(disks);
+ if (volIso == null)
+ totalChangeDevices++;
+ }
+
+ VirtualMachineConfigSpec vmConfigSpec = new VirtualMachineConfigSpec();
+
+ VmwareHelper.setBasicVmConfig(vmConfigSpec, vmSpec.getCpus(), vmSpec.getMaxSpeed(), vmwareResource.getReservedCpuMHZ(vmSpec), (int)(vmSpec.getMaxRam() / (1024 * 1024)),
+ vmwareResource.getReservedMemoryMb(vmSpec), guestOsId, vmSpec.getLimitCpuUse());
+
+ int numCoresPerSocket = adjustNumberOfCoresPerSocket(vmSpec, vmMo, vmConfigSpec);
+
+ // Check for hotadd settings
+ vmConfigSpec.setMemoryHotAddEnabled(vmMo.isMemoryHotAddSupported(guestOsId));
+
+ String hostApiVersion = ((HostMO)hyperHost).getHostAboutInfo().getApiVersion();
+ if (numCoresPerSocket > 1 && hostApiVersion.compareTo("5.0") < 0) {
+ LOGGER.warn("Dynamic scaling of CPU is not supported for Virtual Machines with multi-core vCPUs in case of ESXi hosts 4.1 and prior. Hence CpuHotAdd will not be"
+ + " enabled for Virtual Machine: " + vmInternalCSName);
+ vmConfigSpec.setCpuHotAddEnabled(false);
+ } else {
+ vmConfigSpec.setCpuHotAddEnabled(vmMo.isCpuHotAddSupported(guestOsId));
+ }
+
+ vmwareResource.configNestedHVSupport(vmMo, vmSpec, vmConfigSpec);
+
+ VirtualDeviceConfigSpec[] deviceConfigSpecArray = new VirtualDeviceConfigSpec[totalChangeDevices];
+ int deviceCount = 0;
+ int ideUnitNumber = 0;
+ int scsiUnitNumber = 0;
+ int ideControllerKey = vmMo.getIDEDeviceControllerKey();
+ int scsiControllerKey = vmMo.getScsiDeviceControllerKeyNoException();
+
+ IsoSetup isoSetup = new IsoSetup(vmSpec, mgr, hyperHost, vmMo, disks, volIso, deviceConfigSpecArray, deviceCount, ideUnitNumber);
+ isoSetup.invoke();
+ deviceCount = isoSetup.getDeviceCount();
+ ideUnitNumber = isoSetup.getIdeUnitNumber();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("setting up %d disks with root from %s", diskInfoBuilder.getDiskCount(), vmwareResource.getGson().toJson(rootDiskTO)));
+ }
+
+ DiskSetup diskSetup = new DiskSetup(vmSpec, rootDiskTO, controllerInfo, context, dcMo, hyperHost, vmMo, disks, dataStoresDetails, diskInfoBuilder, hasSnapshot,
+ deviceConfigSpecArray, deviceCount, ideUnitNumber, scsiUnitNumber, ideControllerKey, scsiControllerKey, installAsIs);
+ diskSetup.invoke();
+
+ rootDiskTO = diskSetup.getRootDiskTO();
+ deviceCount = diskSetup.getDeviceCount();
+ DiskTO[] sortedDisks = diskSetup.getSortedDisks();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("seting up %d disks with root from %s", sortedDisks.length, vmwareResource.getGson().toJson(rootDiskTO)));
+ }
+
+ deviceCount += setupUsbDevicesAndGetCount(vmInternalCSName, vmMo, guestOsId, deviceConfigSpecArray, deviceCount);
+
+ NicSetup nicSetup = new NicSetup(cmd, vmSpec, vmInternalCSName, context, mgr, hyperHost, vmMo, nics, deviceConfigSpecArray, deviceCount, nicDevices);
+ nicSetup.invoke();
+
+ deviceCount = nicSetup.getDeviceCount();
+ int nicMask = nicSetup.getNicMask();
+ int nicCount = nicSetup.getNicCount();
+ Map<String, String> nicUuidToDvSwitchUuid = nicSetup.getNicUuidToDvSwitchUuid();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("device count: %d; nic count %d", deviceCount, nicCount));
+ }
+ for (int j = 0; j < deviceCount; j++)
+ vmConfigSpec.getDeviceChange().add(deviceConfigSpecArray[j]);
+
+ //
+ // Setup VM options
+ //
+
+ // pass boot arguments through machine.id & perform customized options to VMX
+ ArrayList<OptionValue> extraOptions = new ArrayList<>();
+ configBasicExtraOption(extraOptions, vmSpec);
+ configNvpExtraOption(extraOptions, vmSpec, nicUuidToDvSwitchUuid);
+ configCustomExtraOption(extraOptions, vmSpec);
+
+ // config for NCC
+ VirtualMachine.Type vmType = cmd.getVirtualMachine().getType();
+ if (vmType.equals(VirtualMachine.Type.NetScalerVm)) {
+ NicTO mgmtNic = vmSpec.getNics()[0];
+ OptionValue option = new OptionValue();
+ option.setKey("machine.id");
+ option.setValue("ip=" + mgmtNic.getIp() + "&netmask=" + mgmtNic.getNetmask() + "&gateway=" + mgmtNic.getGateway());
+ extraOptions.add(option);
+ }
+
+ // config VNC
+ String keyboardLayout = null;
+ if (vmSpec.getDetails() != null)
+ keyboardLayout = vmSpec.getDetails().get(VmDetailConstants.KEYBOARD);
+ vmConfigSpec.getExtraConfig().addAll(
+ Arrays.asList(vmwareResource.configureVnc(extraOptions.toArray(new OptionValue[0]), hyperHost, vmInternalCSName, vmSpec.getVncPassword(), keyboardLayout)));
+
+ // config video card
+ vmwareResource.configureVideoCard(vmMo, vmSpec, vmConfigSpec);
+
+ // Set OVF properties (if available)
+ Pair<String, List<OVFPropertyTO>> ovfPropsMap = vmSpec.getOvfProperties();
+ VmConfigInfo templateVappConfig;
+ List<OVFPropertyTO> ovfProperties;
+ if (ovfPropsMap != null) {
+ String vmTemplate = ovfPropsMap.first();
+ LOGGER.info("Find VM template " + vmTemplate);
+ VirtualMachineMO vmTemplateMO = dcMo.findVm(vmTemplate);
+ templateVappConfig = vmTemplateMO.getConfigInfo().getVAppConfig();
+ ovfProperties = ovfPropsMap.second();
+ // Set OVF properties (if available)
+ if (CollectionUtils.isNotEmpty(ovfProperties)) {
+ LOGGER.info("Copying OVF properties from template and setting them to the values the user provided");
+ vmwareResource.copyVAppConfigsFromTemplate(templateVappConfig, ovfProperties, vmConfigSpec);
+ }
+ }
+
+ checkBootOptions(vmSpec, vmConfigSpec);
+
+ //
+ // Configure VM
+ //
+ if (!vmMo.configureVm(vmConfigSpec)) {
+ throw new Exception("Failed to configure VM before start. vmName: " + vmInternalCSName);
+ }
+ // FR37 TODO reconcile disks now!!! they are configured, check and add them to the returning vmTO
+ if (vmSpec.getType() == VirtualMachine.Type.DomainRouter) {
+ hyperHost.setRestartPriorityForVM(vmMo, DasVmPriority.HIGH.value());
+ }
+
+ // Resizing root disk only when explicit requested by user
+ final Map<String, String> vmDetails = cmd.getVirtualMachine().getDetails();
+ if (rootDiskTO != null && !hasSnapshot && (vmDetails != null && vmDetails.containsKey(ApiConstants.ROOT_DISK_SIZE))) {
+ resizeRootDiskOnVMStart(vmMo, rootDiskTO, hyperHost, context);
+ }
+
+ //
+ // Post Configuration
+ //
+
+ vmMo.setCustomFieldValue(CustomFieldConstants.CLOUD_NIC_MASK, String.valueOf(nicMask));
+ postNvpConfigBeforeStart(vmMo, vmSpec);
+
+ Map<String, Map<String, String>> iqnToData = new HashMap<>();
+
+ postDiskConfigBeforeStart(vmMo, vmSpec, sortedDisks, iqnToData, hyperHost, context, installAsIs);
+
+ //
+ // Power-on VM
+ //
+ if (!vmMo.powerOn()) {
+ throw new Exception("Failed to start VM. vmName: " + vmInternalCSName + " with hostname " + vmNameOnVcenter);
+ }
+
+ if (installAsIs) {
+ // Set disks as the disks path and chain is retrieved from the cloned VM disks
+ cmd.getVirtualMachine().setDisks(disks);
+ }
+
+ StartAnswer startAnswer = new StartAnswer(cmd);
+
+ startAnswer.setIqnToData(iqnToData);
+
+ deleteOldVersionOfTheStartedVM(existingVm, dcMo, vmMo);
+
+ return startAnswer;
+ } catch (Throwable e) {
+ return handleStartFailure(cmd, vmSpec, existingVm, dcMo, e);
+ } finally {
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("finally done with %s", vmwareResource.getGson().toJson(cmd)));
+ }
+ }
+ }
+
+ /**
+ * Modify the specDisks information to match the cloned VM's disks (from vmMo VM)
+ */
+ private void mapSpecDisksToClonedDisks(VirtualMachineMO vmMo, String vmInternalCSName, DiskTO[] specDisks) {
+ try {
+ if (vmMo != null && ArrayUtils.isNotEmpty(specDisks)) {
+ List<VirtualDisk> vmDisks = vmMo.getVirtualDisks();
+ List<DiskTO> sortedDisks = Arrays.asList(sortVolumesByDeviceId(specDisks))
+ .stream()
+ .filter(x -> x.getType() == Volume.Type.ROOT)
+ .collect(Collectors.toList());
+ if (sortedDisks.size() != vmDisks.size()) {
+ LOGGER.error("Different number of root disks spec vs cloned deploy-as-is VM disks: " + sortedDisks.size() + " - " + vmDisks.size());
+ return;
+ }
+ for (int i = 0; i < sortedDisks.size(); i++) {
+ DiskTO specDisk = sortedDisks.get(i);
+ VirtualDisk vmDisk = vmDisks.get(i);
+ DataTO dataVolume = specDisk.getData();
+ if (dataVolume instanceof VolumeObjectTO) {
+ VolumeObjectTO volumeObjectTO = (VolumeObjectTO) dataVolume;
+ if (!volumeObjectTO.getSize().equals(vmDisk.getCapacityInBytes())) {
+ LOGGER.info("Mapped disk size is not the same as the cloned VM disk size: " +
+ volumeObjectTO.getSize() + " - " + vmDisk.getCapacityInBytes());
+ }
+ VirtualDeviceBackingInfo backingInfo = vmDisk.getBacking();
+ if (backingInfo instanceof VirtualDiskFlatVer2BackingInfo) {
+ VirtualDiskFlatVer2BackingInfo backing = (VirtualDiskFlatVer2BackingInfo) backingInfo;
+ String fileName = backing.getFileName();
+ if (StringUtils.isNotBlank(fileName)) {
+ String[] fileNameParts = fileName.split(" ");
+ String datastoreUuid = fileNameParts[0].replace("[", "").replace("]", "");
+ String relativePath = fileNameParts[1].split("/")[1].replace(".vmdk", "");
+ String vmSpecDatastoreUuid = volumeObjectTO.getDataStore().getUuid().replaceAll("-", "");
+ if (!datastoreUuid.equals(vmSpecDatastoreUuid)) {
+ LOGGER.info("Mapped disk datastore UUID is not the same as the cloned VM datastore UUID: " +
+ datastoreUuid + " - " + vmSpecDatastoreUuid);
+ }
+ volumeObjectTO.setPath(relativePath);
+ specDisk.setPath(relativePath);
+ }
+ }
+ }
+ }
+ }
+ } catch (Exception e) {
+ String msg = "Error mapping deploy-as-is VM disks from cloned VM " + vmInternalCSName;
+ LOGGER.error(msg, e);
+ throw new CloudRuntimeException(e);
+ }
+ }
+
+ private StartAnswer handleStartFailure(StartCommand cmd, VirtualMachineTO vmSpec, VirtualMachineData existingVm, DatacenterMO dcMo, Throwable e) {
+ if (e instanceof RemoteException) {
+ LOGGER.warn("Encounter remote exception to vCenter, invalidate VMware session context");
+ vmwareResource.invalidateServiceContext();
+ }
+
+ String msg = "StartCommand failed due to " + VmwareHelper.getExceptionMessage(e);
+ LOGGER.warn(msg, e);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("didn't start VM %s", vmSpec.getName()), e);
+ }
+ StartAnswer startAnswer = new StartAnswer(cmd, msg);
+ if ( e instanceof VmAlreadyExistsInVcenter) {
+ startAnswer.setContextParam("stopRetry", "true");
+ }
+ reRegisterExistingVm(existingVm, dcMo);
+
+ return startAnswer;
+ }
+
+ private void deleteOldVersionOfTheStartedVM(VirtualMachineData existingVm, DatacenterMO dcMo, VirtualMachineMO vmMo) throws Exception {
+ // Since VM was successfully powered-on, if there was an existing VM in a different cluster that was unregistered, delete all the files associated with it.
+ if (existingVm != null && existingVm.vmName != null && existingVm.vmFileLayout != null) {
+ List<String> vmDatastoreNames = new ArrayList<>();
+ for (DatastoreMO vmDatastore : vmMo.getAllDatastores()) {
+ vmDatastoreNames.add(vmDatastore.getName());
+ }
+ // Don't delete files that are in a datastore that is being used by the new VM as well (zone-wide datastore).
+ List<String> skipDatastores = new ArrayList<>();
+ for (DatastoreMO existingDatastore : existingVm.datastores) {
+ if (vmDatastoreNames.contains(existingDatastore.getName())) {
+ skipDatastores.add(existingDatastore.getName());
+ }
+ }
+ vmwareResource.deleteUnregisteredVmFiles(existingVm.vmFileLayout, dcMo, true, skipDatastores);
+ }
+ }
+
+ private int adjustNumberOfCoresPerSocket(VirtualMachineTO vmSpec, VirtualMachineMO vmMo, VirtualMachineConfigSpec vmConfigSpec) throws Exception {
+ // Check for multi-cores per socket settings
+ int numCoresPerSocket = 1;
+ String coresPerSocket = vmSpec.getDetails().get(VmDetailConstants.CPU_CORE_PER_SOCKET);
+ if (coresPerSocket != null) {
+ String apiVersion = HypervisorHostHelper.getVcenterApiVersion(vmMo.getContext());
+ // Property 'numCoresPerSocket' is supported since vSphere API 5.0
+ if (apiVersion.compareTo("5.0") >= 0) {
+ numCoresPerSocket = NumbersUtil.parseInt(coresPerSocket, 1);
+ vmConfigSpec.setNumCoresPerSocket(numCoresPerSocket);
+ }
+ }
+ return numCoresPerSocket;
+ }
+
+ /**
+ // Setup USB devices
+ */
+ private int setupUsbDevicesAndGetCount(String vmInternalCSName, VirtualMachineMO vmMo, String guestOsId, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount)
+ throws Exception {
+ int usbDeviceCount = 0;
+ if (guestOsId.startsWith("darwin")) { //Mac OS
+ VirtualDevice[] devices = vmMo.getMatchedDevices(new Class<?>[] {VirtualUSBController.class});
+ if (devices.length == 0) {
+ LOGGER.debug("No USB Controller device on VM Start. Add USB Controller device for Mac OS VM " + vmInternalCSName);
+
+ //For Mac OS X systems, the EHCI+UHCI controller is enabled by default and is required for USB mouse and keyboard access.
+ VirtualDevice usbControllerDevice = VmwareHelper.prepareUSBControllerDevice();
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ deviceConfigSpecArray[deviceCount].setDevice(usbControllerDevice);
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare USB controller at new device " + vmwareResource.getGson().toJson(deviceConfigSpecArray[deviceCount]));
+
+ deviceCount++;
+ usbDeviceCount++;
+ } else {
+ LOGGER.debug("USB Controller device exists on VM Start for Mac OS VM " + vmInternalCSName);
+ }
+ }
+ return usbDeviceCount;
+ }
+
+ private void createNewVm(VmwareContext context, VirtualMachineTO vmSpec, boolean installAsIs, String vmInternalCSName, String vmNameOnVcenter, Pair<String, String> controllerInfo, Boolean systemVm,
+ VmwareManager mgr, VmwareHypervisorHost hyperHost, String guestOsId, DiskTO[] disks, HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails,
+ DatastoreMO dsRootVolumeIsOn) throws Exception {
+ VirtualMachineMO vmMo;
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = getRootDiskDataStoreDetails(disks, dataStoresDetails);
+
+ assert (vmSpec.getMinSpeed() != null) && (rootDiskDataStoreDetails != null);
+
+ boolean vmFolderExists = rootDiskDataStoreDetails.second().folderExists(String.format("[%s]", rootDiskDataStoreDetails.second().getName()), vmNameOnVcenter);
+ String vmxFileFullPath = dsRootVolumeIsOn.searchFileInSubFolders(vmNameOnVcenter + ".vmx", false, VmwareManager.s_vmwareSearchExcludeFolder.value());
+ if (vmFolderExists && vmxFileFullPath != null) { // VM can be registered only if .vmx is present.
+ registerVm(vmNameOnVcenter, dsRootVolumeIsOn, vmwareResource);
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ if (vmMo != null) {
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Found registered vm " + vmInternalCSName + " at host " + hyperHost.getHyperHostName());
+ }
+ }
+ tearDownVm(vmMo);
+ } else if (installAsIs) {
+// first get all the MORs ManagedObjectReference morPool = hyperHost.getHyperHostOwnerResourcePool();
+// get the base VM vmMo = hyperHost.findVmOnHyperHost(vm.template.getPath());
+// do templateVm.createLinkedClone(vmInternalCSName, morBaseSnapshot, dcMo.getVmFolder(), morPool, morDatastore)
+// or hyperHost....createLinkedOrFullClone(templateVm, volume, dcMo, vmMo, morDatastore, dsMo, vmInternalCSName, morPool);
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ // At this point vmMo points to the cloned VM
+ } else {
+ if (!hyperHost
+ .createBlankVm(vmNameOnVcenter, vmInternalCSName, vmSpec.getCpus(), vmSpec.getMaxSpeed(), vmwareResource.getReservedCpuMHZ(vmSpec), vmSpec.getLimitCpuUse(), (int)(vmSpec.getMaxRam() / Resource.ResourceType.bytesToMiB), vmwareResource.getReservedMemoryMb(vmSpec), guestOsId,
+ rootDiskDataStoreDetails.first(), false, controllerInfo, systemVm)) {
+ throw new Exception("Failed to create VM. vmName: " + vmInternalCSName);
+ }
+ }
+ }
+
+ /**
+ * If a VM with the same name is found in a different cluster in the DC, unregister the old VM and configure a new VM (cold-migration).
+ */
+ private VirtualMachineData unregisterOnOtherClusterButHoldOnToOldVmData(String vmInternalCSName, DatacenterMO dcMo) throws Exception {
+ VirtualMachineMO existingVmInDc = dcMo.findVm(vmInternalCSName);
+ VirtualMachineData existingVm = null;
+ if (existingVmInDc != null) {
+ existingVm = new VirtualMachineData();
+ existingVm.vmName = existingVmInDc.getName();
+ existingVm.vmFileInfo = existingVmInDc.getFileInfo();
+ existingVm.vmFileLayout = existingVmInDc.getFileLayout();
+ existingVm.datastores = existingVmInDc.getAllDatastores();
+ LOGGER.info("Found VM: " + vmInternalCSName + " on a host in a different cluster. Unregistering the exisitng VM.");
+ existingVmInDc.unregisterVm();
+ }
+ return existingVm;
+ }
+
+ private Pair<ManagedObjectReference, DatastoreMO> getRootDiskDataStoreDetails(DiskTO[] disks, HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails) {
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = null;
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ROOT) {
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (managed) {
+ String datastoreName = VmwareResource.getDatastoreName(details.get(DiskTO.IQN));
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(datastoreName);
+ } else {
+ DataStoreTO primaryStore = vol.getData().getDataStore();
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(primaryStore.getUuid());
+ }
+ }
+ }
+ return rootDiskDataStoreDetails;
+ }
+
+ /**
+ * Since VM start failed, if there was an existing VM in a different cluster that was unregistered, register it back.
+ *
+ * @param dcMo is guaranteed to be not null since we have noticed there is an existing VM in the dc (using that mo)
+ */
+ private void reRegisterExistingVm(VirtualMachineData existingVm, DatacenterMO dcMo) {
+ if (existingVm != null && existingVm.vmName != null && existingVm.vmFileInfo != null) {
+ LOGGER.debug("Since VM start failed, registering back an existing VM: " + existingVm.vmName + " that was unregistered");
+ try {
+ DatastoreFile fileInDatastore = new DatastoreFile(existingVm.vmFileInfo.getVmPathName());
+ DatastoreMO existingVmDsMo = new DatastoreMO(dcMo.getContext(), dcMo.findDatastore(fileInDatastore.getDatastoreName()));
+ registerVm(existingVm.vmName, existingVmDsMo, vmwareResource);
+ } catch (Exception ex) {
+ String message = "Failed to register an existing VM: " + existingVm.vmName + " due to " + VmwareHelper.getExceptionMessage(ex);
+ LOGGER.warn(message, ex);
+ }
+ }
+ }
+
+ private void checkIfVmExistsInVcenter(String vmInternalCSName, String vmNameOnVcenter, DatacenterMO dcMo) throws VmAlreadyExistsInVcenter, Exception {
+ // Validate VM name is unique in Datacenter
+ VirtualMachineMO vmInVcenter = dcMo.checkIfVmAlreadyExistsInVcenter(vmNameOnVcenter, vmInternalCSName);
+ if (vmInVcenter != null) {
+ String msg = "VM with name: " + vmNameOnVcenter + " already exists in vCenter.";
+ LOGGER.error(msg);
+ throw new VmAlreadyExistsInVcenter(msg);
+ }
+ }
+
+ private Pair<String, String> getDiskControllerInfo(VirtualMachineTO vmSpec) {
+ String dataDiskController = vmSpec.getDetails().get(VmDetailConstants.DATA_DISK_CONTROLLER);
+ String rootDiskController = vmSpec.getDetails().get(VmDetailConstants.ROOT_DISK_CONTROLLER);
+ // If root disk controller is scsi, then data disk controller would also be scsi instead of using 'osdefault'
+ // This helps avoid mix of different scsi subtype controllers in instance.
+ if (DiskControllerType.osdefault == DiskControllerType.getType(dataDiskController) && DiskControllerType.lsilogic == DiskControllerType.getType(rootDiskController)) {
+ dataDiskController = DiskControllerType.scsi.toString();
+ }
+
+ // Validate the controller types
+ dataDiskController = DiskControllerType.getType(dataDiskController).toString();
+ rootDiskController = DiskControllerType.getType(rootDiskController).toString();
+
+ if (DiskControllerType.getType(rootDiskController) == DiskControllerType.none) {
+ throw new CloudRuntimeException("Invalid root disk controller detected : " + rootDiskController);
+ }
+ if (DiskControllerType.getType(dataDiskController) == DiskControllerType.none) {
+ throw new CloudRuntimeException("Invalid data disk controller detected : " + dataDiskController);
+ }
+
+ return new Pair<>(rootDiskController, dataDiskController);
+ }
+
+ /**
+ * Registers the vm to the inventory given the vmx file.
+ * @param vmName
+ * @param dsMo
+ * @param vmwareResource
+ */
+ private void registerVm(String vmName, DatastoreMO dsMo, VmwareResource vmwareResource) throws Exception {
+
+ //1st param
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(vmwareResource.getServiceContext());
+ ManagedObjectReference dcMor = hyperHost.getHyperHostDatacenter();
+ DatacenterMO dataCenterMo = new DatacenterMO(vmwareResource.getServiceContext(), dcMor);
+ ManagedObjectReference vmFolderMor = dataCenterMo.getVmFolder();
+
+ //2nd param
+ String vmxFilePath = dsMo.searchFileInSubFolders(vmName + ".vmx", false, VmwareManager.s_vmwareSearchExcludeFolder.value());
+
+ // 5th param
+ ManagedObjectReference morPool = hyperHost.getHyperHostOwnerResourcePool();
+
+ ManagedObjectReference morTask = vmwareResource.getServiceContext().getService().registerVMTask(vmFolderMor, vmxFilePath, vmName, false, morPool, hyperHost.getMor());
+ boolean result = vmwareResource.getServiceContext().getVimClient().waitForTask(morTask);
+ if (!result) {
+ throw new Exception("Unable to register vm due to " + TaskMO.getTaskFailureInfo(vmwareResource.getServiceContext(), morTask));
+ } else {
+ vmwareResource.getServiceContext().waitForTaskProgressDone(morTask);
+ }
+
+ }
+
+ // Pair<internal CS name, vCenter display name>
+ private Pair<String, String> composeVmNames(VirtualMachineTO vmSpec) {
+ String vmInternalCSName = vmSpec.getName();
+ String vmNameOnVcenter = vmSpec.getName();
+ if (VmwareResource.instanceNameFlag && vmSpec.getHostName() != null) {
+ vmNameOnVcenter = vmSpec.getHostName();
+ }
+ return new Pair<String, String>(vmInternalCSName, vmNameOnVcenter);
+ }
+
+ private VirtualMachineGuestOsIdentifier translateGuestOsIdentifier(String cpuArchitecture, String guestOs, String cloudGuestOs) {
+ if (cpuArchitecture == null) {
+ LOGGER.warn("CPU arch is not set, default to i386. guest os: " + guestOs);
+ cpuArchitecture = "i386";
+ }
+
+ if (cloudGuestOs == null) {
+ LOGGER.warn("Guest OS mapping name is not set for guest os: " + guestOs);
+ }
+
+ VirtualMachineGuestOsIdentifier identifier = null;
+ try {
+ if (cloudGuestOs != null) {
+ identifier = VirtualMachineGuestOsIdentifier.fromValue(cloudGuestOs);
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Using mapping name : " + identifier.toString());
+ }
+ }
+ } catch (IllegalArgumentException e) {
+ LOGGER.warn("Unable to find Guest OS Identifier in VMware for mapping name: " + cloudGuestOs + ". Continuing with defaults.");
+ }
+ if (identifier != null) {
+ return identifier;
+ }
+
+ if (cpuArchitecture.equalsIgnoreCase("x86_64")) {
+ return VirtualMachineGuestOsIdentifier.OTHER_GUEST_64;
+ }
+ return VirtualMachineGuestOsIdentifier.OTHER_GUEST;
+ }
+
+ private DiskTO[] validateDisks(DiskTO[] disks) {
+ List<DiskTO> validatedDisks = new ArrayList<DiskTO>();
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() != Volume.Type.ISO) {
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ if (primaryStore.getUuid() != null && !primaryStore.getUuid().isEmpty()) {
+ validatedDisks.add(vol);
+ }
+ } else if (vol.getType() == Volume.Type.ISO) {
+ TemplateObjectTO templateTO = (TemplateObjectTO) vol.getData();
+ if (templateTO.getPath() != null && !templateTO.getPath().isEmpty()) {
+ validatedDisks.add(vol);
+ }
+ } else {
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Drop invalid disk option, volumeTO: " + vmwareResource.getGson().toJson(vol));
+ }
+ }
+ }
+ Collections.sort(validatedDisks, (d1, d2) -> d1.getDiskSeq().compareTo(d2.getDiskSeq()));
+ return validatedDisks.toArray(new DiskTO[0]);
+ }
+
+ private HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> inferDatastoreDetailsFromDiskInfo(VmwareHypervisorHost hyperHost, VmwareContext context,
+ DiskTO[] disks, Command cmd) throws Exception {
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> mapIdToMors = new HashMap<>();
+
+ assert (hyperHost != null) && (context != null);
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() != Volume.Type.ISO) {
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ String poolUuid = primaryStore.getUuid();
+
+ if (mapIdToMors.get(poolUuid) == null) {
+ boolean isManaged = false;
+ Map<String, String> details = vol.getDetails();
+
+ if (details != null) {
+ isManaged = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (isManaged) {
+ String iScsiName = details.get(DiskTO.IQN); // details should not be null for managed storage (it may or may not be null for non-managed storage)
+ String datastoreName = VmwareResource.getDatastoreName(iScsiName);
+ ManagedObjectReference morDatastore = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, datastoreName);
+
+ // if the datastore is not present, we need to discover the iSCSI device that will support it,
+ // create the datastore, and create a VMDK file in the datastore
+ if (morDatastore == null) {
+ final String vmdkPath = vmwareResource.getVmdkPath(volumeTO.getPath());
+
+ morDatastore = getStorageProcessor().prepareManagedStorage(context, hyperHost, null, iScsiName,
+ details.get(DiskTO.STORAGE_HOST), Integer.parseInt(details.get(DiskTO.STORAGE_PORT)),
+ vmdkPath,
+ details.get(DiskTO.CHAP_INITIATOR_USERNAME), details.get(DiskTO.CHAP_INITIATOR_SECRET),
+ details.get(DiskTO.CHAP_TARGET_USERNAME), details.get(DiskTO.CHAP_TARGET_SECRET),
+ Long.parseLong(details.get(DiskTO.VOLUME_SIZE)), cmd);
+
+ DatastoreMO dsMo = new DatastoreMO(vmwareResource.getServiceContext(), morDatastore);
+
+ final String datastoreVolumePath;
+
+ if (vmdkPath != null) {
+ datastoreVolumePath = dsMo.getDatastorePath(vmdkPath + VmwareResource.VMDK_EXTENSION);
+ } else {
+ datastoreVolumePath = dsMo.getDatastorePath(dsMo.getName() + VmwareResource.VMDK_EXTENSION);
+ }
+
+ volumeTO.setPath(datastoreVolumePath);
+ vol.setPath(datastoreVolumePath);
+ }
+
+ mapIdToMors.put(datastoreName, new Pair<>(morDatastore, new DatastoreMO(context, morDatastore)));
+ } else {
+ ManagedObjectReference morDatastore = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, poolUuid);
+
+ if (morDatastore == null) {
+ String msg = "Failed to get the mounted datastore for the volume's pool " + poolUuid;
+
+ LOGGER.error(msg);
+
+ throw new Exception(msg);
+ }
+
+ mapIdToMors.put(poolUuid, new Pair<>(morDatastore, new DatastoreMO(context, morDatastore)));
+ }
+ }
+ }
+ }
+
+ return mapIdToMors;
+ }
+
+ private VmwareStorageProcessor getStorageProcessor() {
+ return vmwareResource.getStorageProcessor();
+ }
+
+ private DatastoreMO getDatastoreThatRootDiskIsOn(HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails, DiskTO disks[]) {
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = null;
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ROOT) {
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (managed) {
+ String datastoreName = VmwareResource.getDatastoreName(details.get(DiskTO.IQN));
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(datastoreName);
+
+ break;
+ } else {
+ DataStoreTO primaryStore = vol.getData().getDataStore();
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(primaryStore.getUuid());
+
+ break;
+ }
+ }
+ }
+
+ if (rootDiskDataStoreDetails != null) {
+ return rootDiskDataStoreDetails.second();
+ }
+
+ return null;
+ }
+
+ private void ensureDiskControllers(VirtualMachineMO vmMo, Pair<String, String> controllerInfo) throws Exception {
+ if (vmMo == null) {
+ return;
+ }
+
+ String msg;
+ String rootDiskController = controllerInfo.first();
+ String dataDiskController = controllerInfo.second();
+ String scsiDiskController;
+ String recommendedDiskController = null;
+
+ if (VmwareHelper.isControllerOsRecommended(dataDiskController) || VmwareHelper.isControllerOsRecommended(rootDiskController)) {
+ recommendedDiskController = vmMo.getRecommendedDiskController(null);
+ }
+ scsiDiskController = HypervisorHostHelper.getScsiController(new Pair<String, String>(rootDiskController, dataDiskController), recommendedDiskController);
+ if (scsiDiskController == null) {
+ return;
+ }
+
+ vmMo.getScsiDeviceControllerKeyNoException();
+ // This VM needs SCSI controllers.
+ // Get count of existing scsi controllers. Helps not to attempt to create more than the maximum allowed 4
+ // Get maximum among the bus numbers in use by scsi controllers. Safe to pick maximum, because we always go sequential allocating bus numbers.
+ Ternary<Integer, Integer, DiskControllerType> scsiControllerInfo = vmMo.getScsiControllerInfo();
+ int requiredNumScsiControllers = VmwareHelper.MAX_SCSI_CONTROLLER_COUNT - scsiControllerInfo.first();
+ int availableBusNum = scsiControllerInfo.second() + 1; // method returned current max. bus number
+
+ if (requiredNumScsiControllers == 0) {
+ return;
+ }
+ if (scsiControllerInfo.first() > 0) {
+ // For VMs which already have a SCSI controller, do NOT attempt to add any more SCSI controllers & return the sub type.
+ // For Legacy VMs would have only 1 LsiLogic Parallel SCSI controller, and doesn't require more.
+ // For VMs created post device ordering support, 4 SCSI subtype controllers are ensured during deployment itself. No need to add more.
+ // For fresh VM deployment only, all required controllers should be ensured.
+ return;
+ }
+ ensureScsiDiskControllers(vmMo, scsiDiskController, requiredNumScsiControllers, availableBusNum);
+ }
+
+ private void ensureScsiDiskControllers(VirtualMachineMO vmMo, String scsiDiskController, int requiredNumScsiControllers, int availableBusNum) throws Exception {
+ // Pick the sub type of scsi
+ if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.pvscsi) {
+ if (!vmMo.isPvScsiSupported()) {
+ String msg = "This VM doesn't support Vmware Paravirtual SCSI controller for virtual disks, because the virtual hardware version is less than 7.";
+ throw new Exception(msg);
+ }
+ vmMo.ensurePvScsiDeviceController(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.lsisas1068) {
+ vmMo.ensureLsiLogicSasDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.buslogic) {
+ vmMo.ensureBusLogicDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.lsilogic) {
+ vmMo.ensureLsiLogicDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ }
+ }
+
+ private void tearDownVm(VirtualMachineMO vmMo) throws Exception {
+
+ if (vmMo == null)
+ return;
+
+ boolean hasSnapshot = false;
+ hasSnapshot = vmMo.hasSnapshot();
+ if (!hasSnapshot)
+ vmMo.tearDownDevices(new Class<?>[]{VirtualDisk.class, VirtualEthernetCard.class});
+ else
+ vmMo.tearDownDevices(new Class<?>[]{VirtualEthernetCard.class});
+ vmMo.ensureScsiDeviceController();
+ }
+
+ private static DiskTO getIsoDiskTO(DiskTO[] disks) {
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ISO) {
+ return vol;
+ }
+ }
+ return null;
+ }
+
+ private static DiskTO[] sortVolumesByDeviceId(DiskTO[] volumes) {
+
+ List<DiskTO> listForSort = new ArrayList<DiskTO>();
+ for (DiskTO vol : volumes) {
+ listForSort.add(vol);
+ }
+ Collections.sort(listForSort, new Comparator<DiskTO>() {
+
+ @Override
+ public int compare(DiskTO arg0, DiskTO arg1) {
+ if (arg0.getDiskSeq() < arg1.getDiskSeq()) {
+ return -1;
+ } else if (arg0.getDiskSeq().equals(arg1.getDiskSeq())) {
+ return 0;
+ }
+
+ return 1;
+ }
+ });
+
+ return listForSort.toArray(new DiskTO[0]);
+ }
+
+ private void postDiskConfigBeforeStart(VirtualMachineMO vmMo, VirtualMachineTO vmSpec, DiskTO[] sortedDisks,
+ Map<String, Map<String, String>> iqnToData, VmwareHypervisorHost hyperHost, VmwareContext context, boolean installAsIs)
+ throws Exception {
+ VirtualMachineDiskInfoBuilder diskInfoBuilder = vmMo.getDiskInfoBuilder();
+
+ for (DiskTO vol : sortedDisks) {
+ //TODO: Map existing disks to the ones returned in the answer
+ if (vol.getType() == Volume.Type.ISO)
+ continue;
+
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+
+ VirtualMachineDiskInfo diskInfo = getMatchingExistingDisk(diskInfoBuilder, vol, hyperHost, context);
+ assert (diskInfo != null);
+
+ String[] diskChain = diskInfo.getDiskChain();
+ assert (diskChain.length > 0);
+
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ DatastoreFile file = new DatastoreFile(diskChain[0]);
+
+ if (managed) {
+ DatastoreFile originalFile = new DatastoreFile(volumeTO.getPath());
+
+ if (!file.getFileBaseName().equalsIgnoreCase(originalFile.getFileBaseName())) {
+ if (LOGGER.isInfoEnabled())
+ LOGGER.info("Detected disk-chain top file change on volume: " + volumeTO.getId() + " " + volumeTO.getPath() + " -> " + diskChain[0]);
+ }
+ } else {
+ if (!file.getFileBaseName().equalsIgnoreCase(volumeTO.getPath())) {
+ if (LOGGER.isInfoEnabled())
+ LOGGER.info("Detected disk-chain top file change on volume: " + volumeTO.getId() + " " + volumeTO.getPath() + " -> " + file.getFileBaseName());
+ }
+ }
+
+ VolumeObjectTO volInSpec = getVolumeInSpec(vmSpec, volumeTO);
+
+ if (volInSpec != null) {
+ if (managed) {
+ Map<String, String> data = new HashMap<>();
+
+ String datastoreVolumePath = diskChain[0];
+
+ data.put(StartAnswer.PATH, datastoreVolumePath);
+ data.put(StartAnswer.IMAGE_FORMAT, Storage.ImageFormat.OVA.toString());
+
+ iqnToData.put(details.get(DiskTO.IQN), data);
+
+ vol.setPath(datastoreVolumePath);
+ volumeTO.setPath(datastoreVolumePath);
+ volInSpec.setPath(datastoreVolumePath);
+ } else {
+ volInSpec.setPath(file.getFileBaseName());
+ }
+ volInSpec.setChainInfo(vmwareResource.getGson().toJson(diskInfo));
+ }
+ }
+ }
+
+ private void checkBootOptions(VirtualMachineTO vmSpec, VirtualMachineConfigSpec vmConfigSpec) {
+ String bootMode = null;
+ if (vmSpec.getDetails().containsKey(VmDetailConstants.BOOT_MODE)) {
+ bootMode = vmSpec.getDetails().get(VmDetailConstants.BOOT_MODE);
+ }
+ if (null == bootMode) {
+ bootMode = ApiConstants.BootType.BIOS.toString();
+ }
+
+ setBootOptions(vmSpec, bootMode, vmConfigSpec);
+ }
+
+ private void setBootOptions(VirtualMachineTO vmSpec, String bootMode, VirtualMachineConfigSpec vmConfigSpec) {
+ VirtualMachineBootOptions bootOptions = null;
+ if (StringUtils.isNotBlank(bootMode) && !bootMode.equalsIgnoreCase("bios")) {
+ vmConfigSpec.setFirmware("efi");
+ if (vmSpec.getDetails().containsKey(ApiConstants.BootType.UEFI.toString()) && "secure".equalsIgnoreCase(vmSpec.getDetails().get(ApiConstants.BootType.UEFI.toString()))) {
+ if (bootOptions == null) {
+ bootOptions = new VirtualMachineBootOptions();
+ }
+ bootOptions.setEfiSecureBootEnabled(true);
+ }
+ }
+ if (vmSpec.isEnterHardwareSetup()) {
+ if (bootOptions == null) {
+ bootOptions = new VirtualMachineBootOptions();
+ }
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("configuring VM '%s' to enter hardware setup",vmSpec.getName()));
+ }
+ bootOptions.setEnterBIOSSetup(vmSpec.isEnterHardwareSetup());
+ }
+ if (bootOptions != null) {
+ vmConfigSpec.setBootOptions(bootOptions);
+ }
+ }
+
+ private void resizeRootDiskOnVMStart(VirtualMachineMO vmMo, DiskTO rootDiskTO, VmwareHypervisorHost hyperHost, VmwareContext context) throws Exception {
+ final Pair<VirtualDisk, String> vdisk = vmwareResource.getVirtualDiskInfo(vmMo, VmwareResource.appendFileType(rootDiskTO.getPath(), VmwareResource.VMDK_EXTENSION));
+ assert (vdisk != null);
+
+ Long reqSize = 0L;
+ final VolumeObjectTO volumeTO = ((VolumeObjectTO) rootDiskTO.getData());
+ if (volumeTO != null) {
+ reqSize = volumeTO.getSize() / 1024;
+ }
+ final VirtualDisk disk = vdisk.first();
+ if (reqSize > disk.getCapacityInKB()) {
+ final VirtualMachineDiskInfo diskInfo = getMatchingExistingDisk(vmMo.getDiskInfoBuilder(), rootDiskTO, hyperHost, context);
+ assert (diskInfo != null);
+ final String[] diskChain = diskInfo.getDiskChain();
+
+ if (diskChain != null && diskChain.length > 1) {
+ LOGGER.warn("Disk chain length for the VM is greater than one, this is not supported");
+ throw new CloudRuntimeException("Unsupported VM disk chain length: " + diskChain.length);
+ }
+
+ boolean resizingSupported = false;
+ String deviceBusName = diskInfo.getDiskDeviceBusName();
+ if (deviceBusName != null && (deviceBusName.toLowerCase().contains("scsi") || deviceBusName.toLowerCase().contains("lsi"))) {
+ resizingSupported = true;
+ }
+ if (!resizingSupported) {
+ LOGGER.warn("Resizing of root disk is only support for scsi device/bus, the provide VM's disk device bus name is " + diskInfo.getDiskDeviceBusName());
+ throw new CloudRuntimeException("Unsupported VM root disk device bus: " + diskInfo.getDiskDeviceBusName());
+ }
+
+ disk.setCapacityInKB(reqSize);
+ VirtualMachineConfigSpec vmConfigSpec = new VirtualMachineConfigSpec();
+ VirtualDeviceConfigSpec deviceConfigSpec = new VirtualDeviceConfigSpec();
+ deviceConfigSpec.setDevice(disk);
+ deviceConfigSpec.setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ vmConfigSpec.getDeviceChange().add(deviceConfigSpec);
+ if (!vmMo.configureVm(vmConfigSpec)) {
+ throw new Exception("Failed to configure VM for given root disk size. vmName: " + vmMo.getName());
+ }
+ }
+ }
+
+ private static void postNvpConfigBeforeStart(VirtualMachineMO vmMo, VirtualMachineTO vmSpec) throws Exception {
+ /**
+ * We need to configure the port on the DV switch after the host is
+ * connected. So make this happen between the configure and start of
+ * the VM
+ */
+ int nicIndex = 0;
+ for (NicTO nicTo : sortNicsByDeviceId(vmSpec.getNics())) {
+ if (nicTo.getBroadcastType() == Networks.BroadcastDomainType.Lswitch) {
+ // We need to create a port with a unique vlan and pass the key to the nic device
+ LOGGER.trace("Nic " + nicTo.toString() + " is connected to an NVP logicalswitch");
+ VirtualDevice nicVirtualDevice = vmMo.getNicDeviceByIndex(nicIndex);
+ if (nicVirtualDevice == null) {
+ throw new Exception("Failed to find a VirtualDevice for nic " + nicIndex); //FIXME Generic exceptions are bad
+ }
+ VirtualDeviceBackingInfo backing = nicVirtualDevice.getBacking();
+ if (backing instanceof VirtualEthernetCardDistributedVirtualPortBackingInfo) {
+ // This NIC is connected to a Distributed Virtual Switch
+ VirtualEthernetCardDistributedVirtualPortBackingInfo portInfo = (VirtualEthernetCardDistributedVirtualPortBackingInfo) backing;
+ DistributedVirtualSwitchPortConnection port = portInfo.getPort();
+ String portKey = port.getPortKey();
+ String portGroupKey = port.getPortgroupKey();
+ String dvSwitchUuid = port.getSwitchUuid();
+
+ LOGGER.debug("NIC " + nicTo.toString() + " is connected to dvSwitch " + dvSwitchUuid + " pg " + portGroupKey + " port " + portKey);
+
+ ManagedObjectReference dvSwitchManager = vmMo.getContext().getVimClient().getServiceContent().getDvSwitchManager();
+ ManagedObjectReference dvSwitch = vmMo.getContext().getVimClient().getService().queryDvsByUuid(dvSwitchManager, dvSwitchUuid);
+
+ // Get all ports
+ DistributedVirtualSwitchPortCriteria criteria = new DistributedVirtualSwitchPortCriteria();
+ criteria.setInside(true);
+ criteria.getPortgroupKey().add(portGroupKey);
+ List<DistributedVirtualPort> dvPorts = vmMo.getContext().getVimClient().getService().fetchDVPorts(dvSwitch, criteria);
+
+ DistributedVirtualPort vmDvPort = null;
+ List<Integer> usedVlans = new ArrayList<Integer>();
+ for (DistributedVirtualPort dvPort : dvPorts) {
+ // Find the port for this NIC by portkey
+ if (portKey.equals(dvPort.getKey())) {
+ vmDvPort = dvPort;
+ }
+ VMwareDVSPortSetting settings = (VMwareDVSPortSetting) dvPort.getConfig().getSetting();
+ VmwareDistributedVirtualSwitchVlanIdSpec vlanId = (VmwareDistributedVirtualSwitchVlanIdSpec) settings.getVlan();
+ LOGGER.trace("Found port " + dvPort.getKey() + " with vlan " + vlanId.getVlanId());
+ if (vlanId.getVlanId() > 0 && vlanId.getVlanId() < 4095) {
+ usedVlans.add(vlanId.getVlanId());
+ }
+ }
+
+ if (vmDvPort == null) {
+ throw new Exception("Empty port list from dvSwitch for nic " + nicTo.toString());
+ }
+
+ DVPortConfigInfo dvPortConfigInfo = vmDvPort.getConfig();
+ VMwareDVSPortSetting settings = (VMwareDVSPortSetting) dvPortConfigInfo.getSetting();
+
+ VmwareDistributedVirtualSwitchVlanIdSpec vlanId = (VmwareDistributedVirtualSwitchVlanIdSpec) settings.getVlan();
+ BoolPolicy blocked = settings.getBlocked();
+ if (blocked.isValue() == Boolean.TRUE) {
+ LOGGER.trace("Port is blocked, set a vlanid and unblock");
+ DVPortConfigSpec dvPortConfigSpec = new DVPortConfigSpec();
+ VMwareDVSPortSetting edittedSettings = new VMwareDVSPortSetting();
+ // Unblock
+ blocked.setValue(Boolean.FALSE);
+ blocked.setInherited(Boolean.FALSE);
+ edittedSettings.setBlocked(blocked);
+ // Set vlan
+ int i;
+ for (i = 1; i < 4095; i++) {
+ if (!usedVlans.contains(i))
+ break;
+ }
+ vlanId.setVlanId(i); // FIXME should be a determined
+ // based on usage
+ vlanId.setInherited(false);
+ edittedSettings.setVlan(vlanId);
+
+ dvPortConfigSpec.setSetting(edittedSettings);
+ dvPortConfigSpec.setOperation("edit");
+ dvPortConfigSpec.setKey(portKey);
+ List<DVPortConfigSpec> dvPortConfigSpecs = new ArrayList<DVPortConfigSpec>();
+ dvPortConfigSpecs.add(dvPortConfigSpec);
+ ManagedObjectReference task = vmMo.getContext().getVimClient().getService().reconfigureDVPortTask(dvSwitch, dvPortConfigSpecs);
+ if (!vmMo.getContext().getVimClient().waitForTask(task)) {
+ throw new Exception("Failed to configure the dvSwitch port for nic " + nicTo.toString());
+ }
+ LOGGER.debug("NIC " + nicTo.toString() + " connected to vlan " + i);
+ } else {
+ LOGGER.trace("Port already configured and set to vlan " + vlanId.getVlanId());
+ }
+ } else if (backing instanceof VirtualEthernetCardNetworkBackingInfo) {
+ // This NIC is connected to a Virtual Switch
+ // Nothing to do
+ } else if (backing instanceof VirtualEthernetCardOpaqueNetworkBackingInfo) {
+ //if NSX API VERSION >= 4.2, connect to br-int (nsx.network), do not create portgroup else previous behaviour
+ //OK, connected to OpaqueNetwork
+ } else {
+ LOGGER.error("nic device backing is of type " + backing.getClass().getName());
+ throw new Exception("Incompatible backing for a VirtualDevice for nic " + nicIndex); //FIXME Generic exceptions are bad
+ }
+ }
+ nicIndex++;
+ }
+ }
+
+ private VirtualMachineDiskInfo getMatchingExistingDisk(VirtualMachineDiskInfoBuilder diskInfoBuilder, DiskTO vol, VmwareHypervisorHost hyperHost, VmwareContext context)
+ throws Exception {
+ if (diskInfoBuilder != null) {
+ VolumeObjectTO volume = (VolumeObjectTO) vol.getData();
+
+ String dsName = null;
+ String diskBackingFileBaseName = null;
+
+ Map<String, String> details = vol.getDetails();
+ boolean isManaged = details != null && Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+
+ if (isManaged) {
+ String iScsiName = details.get(DiskTO.IQN);
+
+ // if the storage is managed, iScsiName should not be null
+ dsName = VmwareResource.getDatastoreName(iScsiName);
+
+ diskBackingFileBaseName = new DatastoreFile(volume.getPath()).getFileBaseName();
+ } else {
+ ManagedObjectReference morDs = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, volume.getDataStore().getUuid());
+ DatastoreMO dsMo = new DatastoreMO(context, morDs);
+
+ dsName = dsMo.getName();
+
+ diskBackingFileBaseName = volume.getPath();
+ }
+
+ VirtualMachineDiskInfo diskInfo = diskInfoBuilder.getDiskInfoByBackingFileBaseName(diskBackingFileBaseName, dsName);
+ if (diskInfo != null) {
+ LOGGER.info("Found existing disk info from volume path: " + volume.getPath());
+ return diskInfo;
+ } else {
+ String chainInfo = volume.getChainInfo();
+ if (chainInfo != null) {
+ VirtualMachineDiskInfo infoInChain = vmwareResource.getGson().fromJson(chainInfo, VirtualMachineDiskInfo.class);
+ if (infoInChain != null) {
+ String[] disks = infoInChain.getDiskChain();
+ if (disks.length > 0) {
+ for (String diskPath : disks) {
+ DatastoreFile file = new DatastoreFile(diskPath);
+ diskInfo = diskInfoBuilder.getDiskInfoByBackingFileBaseName(file.getFileBaseName(), dsName);
+ if (diskInfo != null) {
+ LOGGER.info("Found existing disk from chain info: " + diskPath);
+ return diskInfo;
+ }
+ }
+ }
+
+ if (diskInfo == null) {
+ diskInfo = diskInfoBuilder.getDiskInfoByDeviceBusName(infoInChain.getDiskDeviceBusName());
+ if (diskInfo != null) {
+ LOGGER.info("Found existing disk from from chain device bus information: " + infoInChain.getDiskDeviceBusName());
+ return diskInfo;
+ }
+ }
+ }
+ }
+ }
+ }
+
+ return null;
+ }
+
+ private static VolumeObjectTO getVolumeInSpec(VirtualMachineTO vmSpec, VolumeObjectTO srcVol) {
+ for (DiskTO disk : vmSpec.getDisks()) {
+ if (disk.getData() instanceof VolumeObjectTO) {
+ VolumeObjectTO vol = (VolumeObjectTO) disk.getData();
+ if (vol.getId() == srcVol.getId())
+ return vol;
+ }
+ }
+
+ return null;
+ }
+
+ /**
+ * Generate the mac sequence from the nics.
+ */
+ protected String generateMacSequence(NicTO[] nics) {
+ if (nics.length == 0) {
+ return "";
+ }
+
+ StringBuffer sbMacSequence = new StringBuffer();
+ for (NicTO nicTo : sortNicsByDeviceId(nics)) {
+ sbMacSequence.append(nicTo.getMac()).append("|");
+ }
+ if (!sbMacSequence.toString().isEmpty()) {
+ sbMacSequence.deleteCharAt(sbMacSequence.length() - 1); //Remove extra '|' char appended at the end
+ }
+
+ return sbMacSequence.toString();
+ }
+
+ static NicTO[] sortNicsByDeviceId(NicTO[] nics) {
+
+ List<NicTO> listForSort = new ArrayList<NicTO>();
+ for (NicTO nic : nics) {
+ listForSort.add(nic);
+ }
+ Collections.sort(listForSort, new Comparator<NicTO>() {
+
+ @Override
+ public int compare(NicTO arg0, NicTO arg1) {
+ if (arg0.getDeviceId() < arg1.getDeviceId()) {
+ return -1;
+ } else if (arg0.getDeviceId() == arg1.getDeviceId()) {
+ return 0;
+ }
+
+ return 1;
+ }
+ });
+
+ return listForSort.toArray(new NicTO[0]);
+ }
+
+ private static void configBasicExtraOption(List<OptionValue> extraOptions, VirtualMachineTO vmSpec) {
+ OptionValue newVal = new OptionValue();
+ newVal.setKey("machine.id");
+ newVal.setValue(vmSpec.getBootArgs());
+ extraOptions.add(newVal);
+
+ newVal = new OptionValue();
+ newVal.setKey("devices.hotplug");
+ newVal.setValue("true");
+ extraOptions.add(newVal);
+ }
+
+ private static void configNvpExtraOption(List<OptionValue> extraOptions, VirtualMachineTO vmSpec, Map<String, String> nicUuidToDvSwitchUuid) {
+ /**
+ * Extra Config : nvp.vm-uuid = uuid
+ * - Required for Nicira NVP integration
+ */
+ OptionValue newVal = new OptionValue();
+ newVal.setKey("nvp.vm-uuid");
+ newVal.setValue(vmSpec.getUuid());
+ extraOptions.add(newVal);
+
+ /**
+ * Extra Config : nvp.iface-id.<num> = uuid
+ * - Required for Nicira NVP integration
+ */
+ int nicNum = 0;
+ for (NicTO nicTo : sortNicsByDeviceId(vmSpec.getNics())) {
+ if (nicTo.getUuid() != null) {
+ newVal = new OptionValue();
+ newVal.setKey("nvp.iface-id." + nicNum);
+ newVal.setValue(nicTo.getUuid());
+ extraOptions.add(newVal);
+ }
+ nicNum++;
+ }
+ }
+
+ private static void configCustomExtraOption(List<OptionValue> extraOptions, VirtualMachineTO vmSpec) {
+ // we no longer to validation anymore
+ for (Map.Entry<String, String> entry : vmSpec.getDetails().entrySet()) {
+ if (entry.getKey().equalsIgnoreCase(VmDetailConstants.BOOT_MODE)) {
+ continue;
+ }
+ OptionValue newVal = new OptionValue();
+ newVal.setKey(entry.getKey());
+ newVal.setValue(entry.getValue());
+ extraOptions.add(newVal);
+ }
+ }
+
+ private class VmAlreadyExistsInVcenter extends Exception {
+ public VmAlreadyExistsInVcenter(String msg) {
+ }
+ }
+
+ private class VirtualMachineData {
+ String vmName = null;
+ VirtualMachineFileInfo vmFileInfo = null;
+ VirtualMachineFileLayoutEx vmFileLayout = null;
+ List<DatastoreMO> datastores = new ArrayList<>();
+ }
+
+ private class VirtualMachineRecycler {
+ private String vmInternalCSName;
+ private Pair<String, String> controllerInfo;
+ private Boolean systemVm;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private DiskControllerType systemVmScsiControllerType;
+ private int firstScsiControllerBusNum;
+ private int numScsiControllerForSystemVm;
+ private VirtualMachineDiskInfoBuilder diskInfoBuilder;
+ private boolean hasSnapshot;
+ private VirtualDevice[] nicDevices;
+
+ public VirtualMachineRecycler(String vmInternalCSName, Pair<String, String> controllerInfo, Boolean systemVm, VmwareHypervisorHost hyperHost, VirtualMachineMO vmMo,
+ DiskControllerType systemVmScsiControllerType, int firstScsiControllerBusNum, int numScsiControllerForSystemVm) {
+ this.vmInternalCSName = vmInternalCSName;
+ this.controllerInfo = controllerInfo;
+ this.systemVm = systemVm;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.systemVmScsiControllerType = systemVmScsiControllerType;
+ this.firstScsiControllerBusNum = firstScsiControllerBusNum;
+ this.numScsiControllerForSystemVm = numScsiControllerForSystemVm;
+ }
+
+ public VirtualMachineDiskInfoBuilder getDiskInfoBuilder() {
+ return diskInfoBuilder;
+ }
+
+ public boolean isHasSnapshot() {
+ return hasSnapshot;
+ }
+
+ public VirtualMachineRecycler invoke() throws Exception {
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Found vm " + vmInternalCSName + " at other host, relocate to " + hyperHost.getHyperHostName());
+ }
+
+ takeVmFromOtherHyperHost(hyperHost, vmInternalCSName);
+
+ if (VmwareResource.getVmPowerState(vmMo) != VirtualMachine.PowerState.PowerOff)
+ vmMo.safePowerOff(vmwareResource.getShutdownWaitMs());
+
+ diskInfoBuilder = vmMo.getDiskInfoBuilder();
+ hasSnapshot = vmMo.hasSnapshot();
+ nicDevices = vmMo.getNicDevices();
+ if (!hasSnapshot)
+ vmMo.tearDownDevices(new Class<?>[] {VirtualDisk.class, VirtualEthernetCard.class});
+ else
+ vmMo.tearDownDevices(new Class<?>[] {VirtualEthernetCard.class});
+
+ if (systemVm) {
+ // System volumes doesn't require more than 1 SCSI controller as there is no requirement for data volumes.
+ ensureScsiDiskControllers(vmMo, systemVmScsiControllerType.toString(), numScsiControllerForSystemVm, firstScsiControllerBusNum);
+ } else {
+ ensureDiskControllers(vmMo, controllerInfo);
+ }
+ return this;
+ }
+
+ private VirtualMachineMO takeVmFromOtherHyperHost(VmwareHypervisorHost hyperHost, String vmName) throws Exception {
+
+ VirtualMachineMO vmMo = hyperHost.findVmOnPeerHyperHost(vmName);
+ if (vmMo != null) {
+ ManagedObjectReference morTargetPhysicalHost = hyperHost.findMigrationTarget(vmMo);
+ if (morTargetPhysicalHost == null) {
+ String msg = "VM " + vmName + " is on other host and we have no resource available to migrate and start it here";
+ LOGGER.error(msg);
+ throw new Exception(msg);
+ }
+
+ if (!vmMo.relocate(morTargetPhysicalHost)) {
+ String msg = "VM " + vmName + " is on other host and we failed to relocate it here";
+ LOGGER.error(msg);
+ throw new Exception(msg);
+ }
+
+ return vmMo;
+ }
+ return null;
+ }
+
+ public VirtualDevice[] getNicDevices() {
+ return nicDevices;
+ }
+ }
+
+ private class PrepareSytemVMPatchISOMethod {
+ private VmwareManager mgr;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int i;
+ private int ideUnitNumber;
+
+ public PrepareSytemVMPatchISOMethod(VmwareManager mgr, VmwareHypervisorHost hyperHost, VirtualMachineMO vmMo, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int i, int ideUnitNumber) {
+ this.mgr = mgr;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.i = i;
+ this.ideUnitNumber = ideUnitNumber;
+ }
+
+ public int getI() {
+ return i;
+ }
+
+ public int getIdeUnitNumber() {
+ return ideUnitNumber;
+ }
+
+ public PrepareSytemVMPatchISOMethod invoke() throws Exception {
+ // attach ISO (for patching of system VM)
+ DatastoreMO secDsMo = getDatastoreMOForSecStore(mgr, hyperHost);
+
+ deviceConfigSpecArray[i] = new VirtualDeviceConfigSpec();
+ Pair<VirtualDevice, Boolean> isoInfo = VmwareHelper
+ .prepareIsoDevice(vmMo, String.format("[%s] systemvm/%s", secDsMo.getName(), mgr.getSystemVMIsoFileNameOnDatastore()), secDsMo.getMor(), true, true,
+ ideUnitNumber++, i + 1);
+ deviceConfigSpecArray[i].setDevice(isoInfo.first());
+ if (isoInfo.second()) {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at new device " + vmwareResource.getGson().toJson(isoInfo.first()));
+ deviceConfigSpecArray[i].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+ } else {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+ deviceConfigSpecArray[i].setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ }
+ i++;
+ return this;
+ }
+
+ }
+ private DatastoreMO getDatastoreMOForSecStore(VmwareManager mgr, VmwareHypervisorHost hyperHost) throws Exception {
+ Pair<String, Long> secStoreUrlAndId = mgr.getSecondaryStorageStoreUrlAndId(Long.parseLong(vmwareResource.getDcId()));
+ String secStoreUrl = secStoreUrlAndId.first();
+ Long secStoreId = secStoreUrlAndId.second();
+ if (secStoreUrl == null) {
+ String msg = "secondary storage for dc " + vmwareResource.getDcId() + " is not ready yet?";
+ throw new Exception(msg);
+ }
+ mgr.prepareSecondaryStorageStore(secStoreUrl, secStoreId);
+
+ ManagedObjectReference morSecDs = vmwareResource.prepareSecondaryDatastoreOnHost(secStoreUrl);
+ if (morSecDs == null) {
+ String msg = "Failed to prepare secondary storage on host, secondary store url: " + secStoreUrl;
+ throw new Exception(msg);
+ }
+ return new DatastoreMO(hyperHost.getContext(), morSecDs);
+ }
+
+ /**
+ // Setup ROOT/DATA disk devices
+ */
+ private class DiskSetup {
+ private VirtualMachineTO vmSpec;
+ private DiskTO rootDiskTO;
+ private Pair<String, String> controllerInfo;
+ private VmwareContext context;
+ private DatacenterMO dcMo;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private DiskTO[] disks;
+ private HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails;
+ private VirtualMachineDiskInfoBuilder diskInfoBuilder;
+ private boolean hasSnapshot;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int deviceCount;
+ private int ideUnitNumber;
+ private int scsiUnitNumber;
+ private int ideControllerKey;
+ private int scsiControllerKey;
+ private DiskTO[] sortedDisks;
+ private boolean installAsIs;
+
+ public DiskSetup(VirtualMachineTO vmSpec, DiskTO rootDiskTO, Pair<String, String> controllerInfo, VmwareContext context, DatacenterMO dcMo, VmwareHypervisorHost hyperHost,
+ VirtualMachineMO vmMo, DiskTO[] disks, HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails, VirtualMachineDiskInfoBuilder diskInfoBuilder,
+ boolean hasSnapshot, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount, int ideUnitNumber, int scsiUnitNumber, int ideControllerKey, int scsiControllerKey, boolean installAsIs) {
+ this.vmSpec = vmSpec;
+ this.rootDiskTO = rootDiskTO;
+ this.controllerInfo = controllerInfo;
+ this.context = context;
+ this.dcMo = dcMo;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.disks = disks;
+ this.dataStoresDetails = dataStoresDetails;
+ this.diskInfoBuilder = diskInfoBuilder;
+ this.hasSnapshot = hasSnapshot;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.deviceCount = deviceCount;
+ this.ideUnitNumber = ideUnitNumber;
+ this.scsiUnitNumber = scsiUnitNumber;
+ this.ideControllerKey = ideControllerKey;
+ this.scsiControllerKey = scsiControllerKey;
+ this.installAsIs = installAsIs;
+ }
+
+ public DiskTO getRootDiskTO() {
+ return rootDiskTO;
+ }
+
+ public int getDeviceCount() {
+ return deviceCount;
+ }
+
+ public DiskTO[] getSortedDisks() {
+ return sortedDisks;
+ }
+
+ public DiskSetup invoke() throws Exception {
+ int controllerKey;
+ sortedDisks = sortVolumesByDeviceId(disks);
+ for (DiskTO vol : sortedDisks) {
+ if (vol.getType() == Volume.Type.ISO || installAsIs) {
+ continue;
+ }
+
+ VirtualMachineDiskInfo matchingExistingDisk = getMatchingExistingDisk(diskInfoBuilder, vol, hyperHost, context);
+ controllerKey = getDiskController(matchingExistingDisk, vol, vmSpec, ideControllerKey, scsiControllerKey);
+ String diskController = getDiskController(vmMo, matchingExistingDisk, vol, controllerInfo);
+
+ if (DiskControllerType.getType(diskController) == DiskControllerType.osdefault) {
+ diskController = vmMo.getRecommendedDiskController(null);
+ }
+ if (DiskControllerType.getType(diskController) == DiskControllerType.ide) {
+ controllerKey = vmMo.getIDEControllerKey(ideUnitNumber);
+ if (vol.getType() == Volume.Type.DATADISK) {
+ // Could be result of flip due to user configured setting or "osdefault" for data disks
+ // Ensure maximum of 2 data volumes over IDE controller, 3 includeing root volume
+ if (vmMo.getNumberOfVirtualDisks() > 3) {
+ throw new CloudRuntimeException(
+ "Found more than 3 virtual disks attached to this VM [" + vmMo.getVmName() + "]. Unable to implement the disks over " + diskController + " controller, as maximum number of devices supported over IDE controller is 4 includeing CDROM device.");
+ }
+ }
+ } else {
+ if (VmwareHelper.isReservedScsiDeviceNumber(scsiUnitNumber)) {
+ scsiUnitNumber++;
+ }
+
+ controllerKey = vmMo.getScsiDiskControllerKeyNoException(diskController, scsiUnitNumber);
+ if (controllerKey == -1) {
+ // This may happen for ROOT legacy VMs which doesn't have recommended disk controller when global configuration parameter 'vmware.root.disk.controller' is set to "osdefault"
+ // Retrieve existing controller and use.
+ Ternary<Integer, Integer, DiskControllerType> vmScsiControllerInfo = vmMo.getScsiControllerInfo();
+ DiskControllerType existingControllerType = vmScsiControllerInfo.third();
+ controllerKey = vmMo.getScsiDiskControllerKeyNoException(existingControllerType.toString(), scsiUnitNumber);
+ }
+ }
+ if (!hasSnapshot) {
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+
+ VolumeObjectTO volumeTO = (VolumeObjectTO)vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+ String iScsiName = null;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ iScsiName = details.get(DiskTO.IQN);
+ }
+
+ // if the storage is managed, iScsiName should not be null
+ String datastoreName = managed ? VmwareResource.getDatastoreName(iScsiName) : primaryStore.getUuid();
+ Pair<ManagedObjectReference, DatastoreMO> volumeDsDetails = dataStoresDetails.get(datastoreName);
+
+ assert (volumeDsDetails != null);
+
+ String[] diskChain = syncDiskChain(dcMo, vmMo, vmSpec, vol, matchingExistingDisk, dataStoresDetails);
+
+ int deviceNumber = -1;
+ if (controllerKey == vmMo.getIDEControllerKey(ideUnitNumber)) {
+ deviceNumber = ideUnitNumber % VmwareHelper.MAX_ALLOWED_DEVICES_IDE_CONTROLLER;
+ ideUnitNumber++;
+ } else {
+ deviceNumber = scsiUnitNumber % VmwareHelper.MAX_ALLOWED_DEVICES_SCSI_CONTROLLER;
+ scsiUnitNumber++;
+ }
+ VirtualDevice device = VmwareHelper.prepareDiskDevice(vmMo, null, controllerKey, diskChain, volumeDsDetails.first(), deviceNumber, deviceCount + 1);
+ if (vol.getType() == Volume.Type.ROOT)
+ rootDiskTO = vol;
+ deviceConfigSpecArray[deviceCount].setDevice(device);
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare volume at new device " + vmwareResource.getGson().toJson(device));
+
+ deviceCount++;
+ } else {
+ if (controllerKey == vmMo.getIDEControllerKey(ideUnitNumber))
+ ideUnitNumber++;
+ else
+ scsiUnitNumber++;
+ }
+ }
+ return this;
+ }
+
+ private int getDiskController(VirtualMachineDiskInfo matchingExistingDisk, DiskTO vol, VirtualMachineTO vmSpec, int ideControllerKey, int scsiControllerKey) {
+
+ int controllerKey;
+ if (matchingExistingDisk != null) {
+ LOGGER.info("Chose disk controller based on existing information: " + matchingExistingDisk.getDiskDeviceBusName());
+ if (matchingExistingDisk.getDiskDeviceBusName().startsWith("ide"))
+ return ideControllerKey;
+ else
+ return scsiControllerKey;
+ }
+
+ if (vol.getType() == Volume.Type.ROOT) {
+ Map<String, String> vmDetails = vmSpec.getDetails();
+ if (vmDetails != null && vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER) != null) {
+ if (vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER).equalsIgnoreCase("scsi")) {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> scsi, based on root disk controller settings: "
+ + vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER));
+ controllerKey = scsiControllerKey;
+ } else {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> ide, based on root disk controller settings: "
+ + vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER));
+ controllerKey = ideControllerKey;
+ }
+ } else {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> scsi. due to null root disk controller setting");
+ controllerKey = scsiControllerKey;
+ }
+
+ } else {
+ // DATA volume always use SCSI device
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> scsi");
+ controllerKey = scsiControllerKey;
+ }
+
+ return controllerKey;
+ }
+
+ private String getDiskController(VirtualMachineMO vmMo, VirtualMachineDiskInfo matchingExistingDisk, DiskTO vol, Pair<String, String> controllerInfo) throws Exception {
+ int controllerKey;
+ DiskControllerType controllerType = DiskControllerType.none;
+ if (matchingExistingDisk != null) {
+ String currentBusName = matchingExistingDisk.getDiskDeviceBusName();
+ if (currentBusName != null) {
+ LOGGER.info("Chose disk controller based on existing information: " + currentBusName);
+ if (currentBusName.startsWith("ide")) {
+ controllerType = DiskControllerType.ide;
+ } else if (currentBusName.startsWith("scsi")) {
+ controllerType = DiskControllerType.scsi;
+ }
+ }
+ if (controllerType == DiskControllerType.scsi || controllerType == DiskControllerType.none) {
+ Ternary<Integer, Integer, DiskControllerType> vmScsiControllerInfo = vmMo.getScsiControllerInfo();
+ controllerType = vmScsiControllerInfo.third();
+ }
+ return controllerType.toString();
+ }
+
+ if (vol.getType() == Volume.Type.ROOT) {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> " + controllerInfo.first()
+ + ", based on root disk controller settings at global configuration setting.");
+ return controllerInfo.first();
+ } else {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> " + controllerInfo.second()
+ + ", based on default data disk controller setting i.e. Operating system recommended."); // Need to bring in global configuration setting & template level setting.
+ return controllerInfo.second();
+ }
+ }
+
+ // return the finalized disk chain for startup, from top to bottom
+ private String[] syncDiskChain(DatacenterMO dcMo, VirtualMachineMO vmMo, VirtualMachineTO vmSpec, DiskTO vol, VirtualMachineDiskInfo diskInfo,
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails) throws Exception {
+
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ Map<String, String> details = vol.getDetails();
+ boolean isManaged = false;
+ String iScsiName = null;
+
+ if (details != null) {
+ isManaged = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ iScsiName = details.get(DiskTO.IQN);
+ }
+
+ // if the storage is managed, iScsiName should not be null
+ String datastoreName = isManaged ? VmwareResource.getDatastoreName(iScsiName) : primaryStore.getUuid();
+ Pair<ManagedObjectReference, DatastoreMO> volumeDsDetails = dataStoresDetails.get(datastoreName);
+
+ if (volumeDsDetails == null) {
+ throw new Exception("Primary datastore " + primaryStore.getUuid() + " is not mounted on host.");
+ }
+
+ DatastoreMO dsMo = volumeDsDetails.second();
+
+ // we will honor vCenter's meta if it exists
+ if (diskInfo != null) {
+ // to deal with run-time upgrade to maintain the new datastore folder structure
+ String disks[] = diskInfo.getDiskChain();
+ for (int i = 0; i < disks.length; i++) {
+ DatastoreFile file = new DatastoreFile(disks[i]);
+ if (!isManaged && file.getDir() != null && file.getDir().isEmpty()) {
+ LOGGER.info("Perform run-time datastore folder upgrade. sync " + disks[i] + " to VM folder");
+ disks[i] = VmwareStorageLayoutHelper.syncVolumeToVmDefaultFolder(dcMo, vmMo.getName(), dsMo, file.getFileBaseName(), VmwareManager.s_vmwareSearchExcludeFolder.value());
+ }
+ }
+ return disks;
+ }
+
+ final String datastoreDiskPath;
+
+ if (isManaged) {
+ String vmdkPath = new DatastoreFile(volumeTO.getPath()).getFileBaseName();
+
+ if (volumeTO.getVolumeType() == Volume.Type.ROOT) {
+ if (vmdkPath == null) {
+ vmdkPath = volumeTO.getName();
+ }
+
+ datastoreDiskPath = VmwareStorageLayoutHelper.syncVolumeToVmDefaultFolder(dcMo, vmMo.getName(), dsMo, vmdkPath);
+ } else {
+ if (vmdkPath == null) {
+ vmdkPath = dsMo.getName();
+ }
+
+ datastoreDiskPath = dsMo.getDatastorePath(vmdkPath + VmwareResource.VMDK_EXTENSION);
+ }
+ } else {
+ datastoreDiskPath = VmwareStorageLayoutHelper.syncVolumeToVmDefaultFolder(dcMo, vmMo.getName(), dsMo, volumeTO.getPath(), VmwareManager.s_vmwareSearchExcludeFolder.value());
+ }
+
+ if (!dsMo.fileExists(datastoreDiskPath)) {
+ LOGGER.warn("Volume " + volumeTO.getId() + " does not seem to exist on datastore, out of sync? path: " + datastoreDiskPath);
+ }
+
+ return new String[]{datastoreDiskPath};
+ }
+ }
+
+ /**
+ // Setup NIC devices
+ */
+ private class NicSetup {
+ private StartCommand cmd;
+ private VirtualMachineTO vmSpec;
+ private String vmInternalCSName;
+ private VmwareContext context;
+ private VmwareManager mgr;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private NicTO[] nics;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int deviceCount;
+ private int nicMask;
+ private int nicCount;
+ private Map<String, String> nicUuidToDvSwitchUuid;
+ private VirtualDevice[] nicDevices;
+
+ public NicSetup(StartCommand cmd, VirtualMachineTO vmSpec, String vmInternalCSName, VmwareContext context, VmwareManager mgr, VmwareHypervisorHost hyperHost,
+ VirtualMachineMO vmMo, NicTO[] nics, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount, VirtualDevice[] nicDevices) {
+ this.cmd = cmd;
+ this.vmSpec = vmSpec;
+ this.vmInternalCSName = vmInternalCSName;
+ this.context = context;
+ this.mgr = mgr;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.nics = nics;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.deviceCount = deviceCount;
+ this.nicDevices = nicDevices;
+ }
+
+ public int getDeviceCount() {
+ return deviceCount;
+ }
+
+ public int getNicMask() {
+ return nicMask;
+ }
+
+ public int getNicCount() {
+ return nicCount;
+ }
+
+ public Map<String, String> getNicUuidToDvSwitchUuid() {
+ return nicUuidToDvSwitchUuid;
+ }
+
+ public NicSetup invoke() throws Exception {
+ VirtualDevice nic;
+ nicMask = 0;
+ nicCount = 0;
+
+ if (vmSpec.getType() == VirtualMachine.Type.DomainRouter) {
+ doDomainRouterSetup();
+ }
+
+ VirtualEthernetCardType nicDeviceType = VirtualEthernetCardType.valueOf(vmSpec.getDetails().get(VmDetailConstants.NIC_ADAPTER));
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("VM " + vmInternalCSName + " will be started with NIC device type: " + nicDeviceType);
+ }
+
+ NiciraNvpApiVersion.logNiciraApiVersion();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.debug(String.format("deciding for VM '%s' to use orchestrated NICs or as is.", vmInternalCSName));
+ }
+ nicUuidToDvSwitchUuid = new HashMap<>();
+ LOGGER.info(String.format("adding %d nics to VM '%s'", nics.length, vmInternalCSName));
+ for (NicTO nicTo : sortNicsByDeviceId(nics)) {
+ LOGGER.info("Prepare NIC device based on NicTO: " + vmwareResource.getGson().toJson(nicTo));
+
+ boolean configureVServiceInNexus = (nicTo.getType() == Networks.TrafficType.Guest) && (vmSpec.getDetails().containsKey("ConfigureVServiceInNexus"));
+ VirtualMachine.Type vmType = cmd.getVirtualMachine().getType();
+ Pair<ManagedObjectReference, String> networkInfo = vmwareResource.prepareNetworkFromNicInfo(vmMo.getRunningHost(), nicTo, configureVServiceInNexus, vmType);
+ if ((nicTo.getBroadcastType() != Networks.BroadcastDomainType.Lswitch) || (nicTo.getBroadcastType() == Networks.BroadcastDomainType.Lswitch && NiciraNvpApiVersion.isApiVersionLowerThan("4.2"))) {
+ if (VmwareHelper.isDvPortGroup(networkInfo.first())) {
+ String dvSwitchUuid;
+ ManagedObjectReference dcMor = hyperHost.getHyperHostDatacenter();
+ DatacenterMO dataCenterMo = new DatacenterMO(context, dcMor);
+ ManagedObjectReference dvsMor = dataCenterMo.getDvSwitchMor(networkInfo.first());
+ dvSwitchUuid = dataCenterMo.getDvSwitchUuid(dvsMor);
+ LOGGER.info("Preparing NIC device on dvSwitch : " + dvSwitchUuid);
+ nic = VmwareHelper.prepareDvNicDevice(vmMo, networkInfo.first(), nicDeviceType, networkInfo.second(), dvSwitchUuid, nicTo.getMac(), deviceCount + 1, true, true);
+ if (nicTo.getUuid() != null) {
+ nicUuidToDvSwitchUuid.put(nicTo.getUuid(), dvSwitchUuid);
+ }
+ } else {
+ LOGGER.info("Preparing NIC device on network " + networkInfo.second());
+ nic = VmwareHelper.prepareNicDevice(vmMo, networkInfo.first(), nicDeviceType, networkInfo.second(), nicTo.getMac(), deviceCount + 1, true, true);
+ }
+ } else {
+ //if NSX API VERSION >= 4.2, connect to br-int (nsx.network), do not create portgroup else previous behaviour
+ nic = VmwareHelper.prepareNicOpaque(vmMo, nicDeviceType, networkInfo.second(), nicTo.getMac(), deviceCount + 1, true, true);
+ }
+
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ deviceConfigSpecArray[deviceCount].setDevice(nic);
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare NIC at new device " + vmwareResource.getGson().toJson(deviceConfigSpecArray[deviceCount]));
+
+ // this is really a hacking for DomR, upon DomR startup, we will reset all the NIC allocation after eth3
+ if (nicCount < 3)
+ nicMask |= (1 << nicCount);
+
+ deviceCount++;
+ nicCount++;
+ }
+ return this;
+ }
+
+ private void doDomainRouterSetup() throws Exception {
+ int extraPublicNics = mgr.getRouterExtraPublicNics();
+ if (extraPublicNics > 0 && vmSpec.getDetails().containsKey("PeerRouterInstanceName")) {
+ //Set identical MAC address for RvR on extra public interfaces
+ String peerRouterInstanceName = vmSpec.getDetails().get("PeerRouterInstanceName");
+
+ VirtualMachineMO peerVmMo = hyperHost.findVmOnHyperHost(peerRouterInstanceName);
+ if (peerVmMo == null) {
+ peerVmMo = hyperHost.findVmOnPeerHyperHost(peerRouterInstanceName);
+ }
+
+ if (peerVmMo != null) {
+ String oldMacSequence = generateMacSequence(nics);
+
+ for (int nicIndex = nics.length - extraPublicNics; nicIndex < nics.length; nicIndex++) {
+ VirtualDevice nicDevice = peerVmMo.getNicDeviceByIndex(nics[nicIndex].getDeviceId());
+ if (nicDevice != null) {
+ String mac = ((VirtualEthernetCard)nicDevice).getMacAddress();
+ if (mac != null) {
+ LOGGER.info("Use same MAC as previous RvR, the MAC is " + mac + " for extra NIC with device id: " + nics[nicIndex].getDeviceId());
+ nics[nicIndex].setMac(mac);
+ }
+ }
+ }
+
+ if (!StringUtils.isBlank(vmSpec.getBootArgs())) {
+ String newMacSequence = generateMacSequence(nics);
+ vmSpec.setBootArgs(vmwareResource.replaceNicsMacSequenceInBootArgs(oldMacSequence, newMacSequence, vmSpec));
+ }
+ }
+ }
+ }
+ }
+
+ private class IsoSetup {
+ private VirtualMachineTO vmSpec;
+ private VmwareManager mgr;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private DiskTO[] disks;
+ private DiskTO volIso;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int deviceCount;
+ private int ideUnitNumber;
+
+ public IsoSetup(VirtualMachineTO vmSpec, VmwareManager mgr, VmwareHypervisorHost hyperHost, VirtualMachineMO vmMo, DiskTO[] disks, DiskTO volIso,
+ VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount, int ideUnitNumber) {
+ this.vmSpec = vmSpec;
+ this.mgr = mgr;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.disks = disks;
+ this.volIso = volIso;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.deviceCount = deviceCount;
+ this.ideUnitNumber = ideUnitNumber;
+ }
+
+ public int getDeviceCount() {
+ return deviceCount;
+ }
+
+ public int getIdeUnitNumber() {
+ return ideUnitNumber;
+ }
+
+ public IsoSetup invoke() throws Exception {
+ //
+ // Setup ISO device
+ //
+
+ // vAPP ISO
+ // FR37 the native deploy mechs should create this for us
+ if (vmSpec.getOvfProperties() != null) {
+ if (LOGGER.isTraceEnabled()) {
+ // FR37 TODO add more usefull info (if we keep this bit
+ LOGGER.trace("adding iso for properties for 'xxx'");
+ }
Review comment:
```suggestion
if (LOGGER.isTraceEnabled()) {
LOGGER.trace(String.format("adding iso for properties for '%s'", vmMo.getName()));
}
```
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/StartCommandExecutor.java
##########
@@ -0,0 +1,2247 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.hypervisor.vmware.resource;
+
+import java.io.File;
+import java.rmi.RemoteException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.cloud.agent.api.to.DataTO;
+import com.vmware.vim25.VirtualDiskFlatVer2BackingInfo;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.storage.configdrive.ConfigDrive;
+import org.apache.cloudstack.storage.to.TemplateObjectTO;
+import org.apache.cloudstack.storage.to.VolumeObjectTO;
+import org.apache.cloudstack.utils.volume.VirtualMachineDiskInfo;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.commons.lang.ArrayUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.log4j.Logger;
+
+import com.cloud.agent.api.Command;
+import com.cloud.agent.api.StartAnswer;
+import com.cloud.agent.api.StartCommand;
+import com.cloud.agent.api.storage.OVFPropertyTO;
+import com.cloud.agent.api.to.DataStoreTO;
+import com.cloud.agent.api.to.DiskTO;
+import com.cloud.agent.api.to.NfsTO;
+import com.cloud.agent.api.to.NicTO;
+import com.cloud.agent.api.to.VirtualMachineTO;
+import com.cloud.configuration.Resource;
+import com.cloud.hypervisor.vmware.VmwareResourceException;
+import com.cloud.hypervisor.vmware.manager.VmwareManager;
+import com.cloud.hypervisor.vmware.mo.CustomFieldConstants;
+import com.cloud.hypervisor.vmware.mo.DatacenterMO;
+import com.cloud.hypervisor.vmware.mo.DatastoreFile;
+import com.cloud.hypervisor.vmware.mo.DatastoreMO;
+import com.cloud.hypervisor.vmware.mo.DiskControllerType;
+import com.cloud.hypervisor.vmware.mo.HostMO;
+import com.cloud.hypervisor.vmware.mo.HypervisorHostHelper;
+import com.cloud.hypervisor.vmware.mo.TaskMO;
+import com.cloud.hypervisor.vmware.mo.VirtualEthernetCardType;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineDiskInfoBuilder;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineMO;
+import com.cloud.hypervisor.vmware.mo.VmwareHypervisorHost;
+import com.cloud.hypervisor.vmware.util.VmwareContext;
+import com.cloud.hypervisor.vmware.util.VmwareHelper;
+import com.cloud.network.Networks;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Volume;
+import com.cloud.storage.resource.VmwareStorageLayoutHelper;
+import com.cloud.storage.resource.VmwareStorageProcessor;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nicira.nvp.plugin.NiciraNvpApiVersion;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.VmDetailConstants;
+import com.vmware.vim25.BoolPolicy;
+import com.vmware.vim25.DVPortConfigInfo;
+import com.vmware.vim25.DVPortConfigSpec;
+import com.vmware.vim25.DasVmPriority;
+import com.vmware.vim25.DistributedVirtualPort;
+import com.vmware.vim25.DistributedVirtualSwitchPortConnection;
+import com.vmware.vim25.DistributedVirtualSwitchPortCriteria;
+import com.vmware.vim25.ManagedObjectReference;
+import com.vmware.vim25.OptionValue;
+import com.vmware.vim25.VMwareDVSPortSetting;
+import com.vmware.vim25.VirtualDevice;
+import com.vmware.vim25.VirtualDeviceBackingInfo;
+import com.vmware.vim25.VirtualDeviceConfigSpec;
+import com.vmware.vim25.VirtualDeviceConfigSpecOperation;
+import com.vmware.vim25.VirtualDisk;
+import com.vmware.vim25.VirtualEthernetCard;
+import com.vmware.vim25.VirtualEthernetCardDistributedVirtualPortBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardNetworkBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardOpaqueNetworkBackingInfo;
+import com.vmware.vim25.VirtualMachineBootOptions;
+import com.vmware.vim25.VirtualMachineConfigSpec;
+import com.vmware.vim25.VirtualMachineFileInfo;
+import com.vmware.vim25.VirtualMachineFileLayoutEx;
+import com.vmware.vim25.VirtualMachineGuestOsIdentifier;
+import com.vmware.vim25.VirtualUSBController;
+import com.vmware.vim25.VmConfigInfo;
+import com.vmware.vim25.VmwareDistributedVirtualSwitchVlanIdSpec;
+
+class StartCommandExecutor {
+ private static final Logger LOGGER = Logger.getLogger(StartCommandExecutor.class);
+
+ private final VmwareResource vmwareResource;
+
+ public StartCommandExecutor(VmwareResource vmwareResource) {
+ this.vmwareResource = vmwareResource;
+ }
+
+ // FR37 the monster blob god method: reduce to 320 so far and counting
+ protected StartAnswer execute(StartCommand cmd) {
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Executing resource StartCommand: " + vmwareResource.getGson().toJson(cmd));
+ }
+
+ VirtualMachineTO vmSpec = cmd.getVirtualMachine();
+
+ VirtualMachineData existingVm = null;
+
+ Pair<String, String> names = composeVmNames(vmSpec);
+ String vmInternalCSName = names.first();
+ String vmNameOnVcenter = names.second();
+
+ DiskTO rootDiskTO = null;
+
+ Pair<String, String> controllerInfo = getDiskControllerInfo(vmSpec);
+
+ Boolean systemVm = vmSpec.getType().isUsedBySystem();
+
+ VmwareContext context = vmwareResource.getServiceContext();
+ DatacenterMO dcMo = null;
+ try {
+ VmwareManager mgr = context.getStockObject(VmwareManager.CONTEXT_STOCK_NAME);
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(context);
+ dcMo = new DatacenterMO(hyperHost.getContext(), hyperHost.getHyperHostDatacenter());
+
+ // checkIfVmExistsInVcenter(vmInternalCSName, vmNameOnVcenter, dcMo);
+ // FR37 - We expect VM to be already cloned and available at this point
+ VirtualMachineMO vmMo = dcMo.findVm(vmInternalCSName);
+ if (vmMo == null) {
+ vmMo = dcMo.findVm(vmNameOnVcenter);
+ }
+
+ DiskTO[] specDisks = vmSpec.getDisks();
+ boolean installAsIs = StringUtils.isNotEmpty(vmSpec.getTemplateLocation());
+ // FR37 if startcommand contains enough info: a template url/-location and flag; deploy OVA as is
+ if (vmMo == null && installAsIs) {
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("deploying OVA from %s as is", vmSpec.getTemplateLocation()));
+ }
+ getStorageProcessor().cloneVMFromTemplate(vmSpec.getTemplateName(), vmInternalCSName, vmSpec.getTemplatePrimaryStoreUuid());
+ vmMo = dcMo.findVm(vmInternalCSName);
+ mapSpecDisksToClonedDisks(vmMo, vmInternalCSName, specDisks);
+ }
+
+ // VM may not have been on the same host, relocate to expected host
+ if (vmMo != null) {
+ vmMo.relocate(hyperHost.getMor());
+ // Get updated MO
+ vmMo = hyperHost.findVmOnHyperHost(vmMo.getVmName());
+ }
+
+ String guestOsId = translateGuestOsIdentifier(vmSpec.getArch(), vmSpec.getOs(), vmSpec.getPlatformEmulator()).value();
+ DiskTO[] disks = validateDisks(specDisks);
+ NicTO[] nics = vmSpec.getNics();
+
+ // FIXME: disks logic here, why is disks/volumes during copy not set with pool ID?
+ // FR37 TODO if deployasis a new VM no datastores are known (yet) and we need to get the data store from the tvmspec content library / template location
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails = inferDatastoreDetailsFromDiskInfo(hyperHost, context, disks, cmd);
+ if ((dataStoresDetails == null) || (dataStoresDetails.isEmpty())) {
+ String msg = "Unable to locate datastore details of the volumes to be attached";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ // FR37 - this may not be necessary the cloned VM will have disks and knowledge of datastore paths/location?
+ // throw new Exception(msg);
+ }
+
+ // FR37 - this may need checking, if the first datastore is the right one - ideally it should be datastore where first disk is hosted
+ DatastoreMO dsRootVolumeIsOn = null; //
+ if (! installAsIs) {
+ dsRootVolumeIsOn = getDatastoreThatRootDiskIsOn(dataStoresDetails, disks);
+
+ if (dsRootVolumeIsOn == null) {
+ String msg = "Unable to locate datastore details of root volume";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ throw new VmwareResourceException(msg);
+ }
+ }
+
+ VirtualMachineDiskInfoBuilder diskInfoBuilder = null;
+ VirtualDevice[] nicDevices = null;
+ DiskControllerType systemVmScsiControllerType = DiskControllerType.lsilogic;
+ int firstScsiControllerBusNum = 0;
+ int numScsiControllerForSystemVm = 1;
+ boolean hasSnapshot = false;
+ if (vmMo != null) {
+ PrepareRunningVMForConfiguration prepareRunningVMForConfiguration = new PrepareRunningVMForConfiguration(vmInternalCSName, controllerInfo, systemVm, vmMo,
+ systemVmScsiControllerType, firstScsiControllerBusNum, numScsiControllerForSystemVm);
+ prepareRunningVMForConfiguration.invoke();
+ diskInfoBuilder = prepareRunningVMForConfiguration.getDiskInfoBuilder();
+ nicDevices = prepareRunningVMForConfiguration.getNicDevices();
+ hasSnapshot = prepareRunningVMForConfiguration.isHasSnapshot();
+ } else {
+ ManagedObjectReference morDc = hyperHost.getHyperHostDatacenter();
+ assert (morDc != null);
+
+ vmMo = hyperHost.findVmOnPeerHyperHost(vmInternalCSName);
+ if (vmMo != null) {
+ VirtualMachineRecycler virtualMachineRecycler = new VirtualMachineRecycler(vmInternalCSName, controllerInfo, systemVm, hyperHost, vmMo,
+ systemVmScsiControllerType, firstScsiControllerBusNum, numScsiControllerForSystemVm);
+ virtualMachineRecycler.invoke();
+ diskInfoBuilder = virtualMachineRecycler.getDiskInfoBuilder();
+ hasSnapshot = virtualMachineRecycler.isHasSnapshot();
+ nicDevices = virtualMachineRecycler.getNicDevices();
+ } else {
+ // FR37 we just didn't find a VM by name 'vmInternalCSName', so why is this here?
+ existingVm = unregisterOnOtherClusterButHoldOnToOldVmData(vmInternalCSName, dcMo);
+
+ createNewVm(context, vmSpec, installAsIs, vmInternalCSName, vmNameOnVcenter, controllerInfo, systemVm, mgr, hyperHost, guestOsId, disks, dataStoresDetails,
+ dsRootVolumeIsOn);
+ }
+
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ if (vmMo == null) {
+ throw new Exception("Failed to find the newly create or relocated VM. vmName: " + vmInternalCSName);
+ }
+ }
+ // vmMo should now be a stopped VM on the intended host
+ // The number of disks changed must be 0 for install as is, as the VM is a clone
+ int disksChanges = !installAsIs ? disks.length : 0;
+ int totalChangeDevices = disksChanges + nics.length;
+ int hackDeviceCount = 0;
+ if (diskInfoBuilder != null) {
+ hackDeviceCount += diskInfoBuilder.getDiskCount();
+ }
+ hackDeviceCount += nicDevices == null ? 0 : nicDevices.length;
+ // vApp cdrom device
+ // HACK ALERT: ovf properties might not be the only or defining feature of vApps; needs checking
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("current count(s) desired: %d/ found:%d. now adding device to device count for vApp config ISO", totalChangeDevices, hackDeviceCount));
+ }
+ if (vmSpec.getOvfProperties() != null) {
+ totalChangeDevices++;
+ }
+
+ DiskTO volIso = null;
+ if (vmSpec.getType() != VirtualMachine.Type.User) {
+ // system VM needs a patch ISO
+ totalChangeDevices++;
+ } else {
+ volIso = getIsoDiskTO(disks);
+ if (volIso == null)
+ totalChangeDevices++;
+ }
+
+ VirtualMachineConfigSpec vmConfigSpec = new VirtualMachineConfigSpec();
+
+ VmwareHelper.setBasicVmConfig(vmConfigSpec, vmSpec.getCpus(), vmSpec.getMaxSpeed(), vmwareResource.getReservedCpuMHZ(vmSpec), (int)(vmSpec.getMaxRam() / (1024 * 1024)),
+ vmwareResource.getReservedMemoryMb(vmSpec), guestOsId, vmSpec.getLimitCpuUse());
+
+ int numCoresPerSocket = adjustNumberOfCoresPerSocket(vmSpec, vmMo, vmConfigSpec);
+
+ // Check for hotadd settings
+ vmConfigSpec.setMemoryHotAddEnabled(vmMo.isMemoryHotAddSupported(guestOsId));
+
+ String hostApiVersion = ((HostMO)hyperHost).getHostAboutInfo().getApiVersion();
+ if (numCoresPerSocket > 1 && hostApiVersion.compareTo("5.0") < 0) {
+ LOGGER.warn("Dynamic scaling of CPU is not supported for Virtual Machines with multi-core vCPUs in case of ESXi hosts 4.1 and prior. Hence CpuHotAdd will not be"
+ + " enabled for Virtual Machine: " + vmInternalCSName);
+ vmConfigSpec.setCpuHotAddEnabled(false);
+ } else {
+ vmConfigSpec.setCpuHotAddEnabled(vmMo.isCpuHotAddSupported(guestOsId));
+ }
+
+ vmwareResource.configNestedHVSupport(vmMo, vmSpec, vmConfigSpec);
+
+ VirtualDeviceConfigSpec[] deviceConfigSpecArray = new VirtualDeviceConfigSpec[totalChangeDevices];
+ int deviceCount = 0;
+ int ideUnitNumber = 0;
+ int scsiUnitNumber = 0;
+ int ideControllerKey = vmMo.getIDEDeviceControllerKey();
+ int scsiControllerKey = vmMo.getScsiDeviceControllerKeyNoException();
+
+ IsoSetup isoSetup = new IsoSetup(vmSpec, mgr, hyperHost, vmMo, disks, volIso, deviceConfigSpecArray, deviceCount, ideUnitNumber);
+ isoSetup.invoke();
+ deviceCount = isoSetup.getDeviceCount();
+ ideUnitNumber = isoSetup.getIdeUnitNumber();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("setting up %d disks with root from %s", diskInfoBuilder.getDiskCount(), vmwareResource.getGson().toJson(rootDiskTO)));
+ }
+
+ DiskSetup diskSetup = new DiskSetup(vmSpec, rootDiskTO, controllerInfo, context, dcMo, hyperHost, vmMo, disks, dataStoresDetails, diskInfoBuilder, hasSnapshot,
+ deviceConfigSpecArray, deviceCount, ideUnitNumber, scsiUnitNumber, ideControllerKey, scsiControllerKey, installAsIs);
+ diskSetup.invoke();
+
+ rootDiskTO = diskSetup.getRootDiskTO();
+ deviceCount = diskSetup.getDeviceCount();
+ DiskTO[] sortedDisks = diskSetup.getSortedDisks();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("seting up %d disks with root from %s", sortedDisks.length, vmwareResource.getGson().toJson(rootDiskTO)));
+ }
+
+ deviceCount += setupUsbDevicesAndGetCount(vmInternalCSName, vmMo, guestOsId, deviceConfigSpecArray, deviceCount);
+
+ NicSetup nicSetup = new NicSetup(cmd, vmSpec, vmInternalCSName, context, mgr, hyperHost, vmMo, nics, deviceConfigSpecArray, deviceCount, nicDevices);
+ nicSetup.invoke();
+
+ deviceCount = nicSetup.getDeviceCount();
+ int nicMask = nicSetup.getNicMask();
+ int nicCount = nicSetup.getNicCount();
+ Map<String, String> nicUuidToDvSwitchUuid = nicSetup.getNicUuidToDvSwitchUuid();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("device count: %d; nic count %d", deviceCount, nicCount));
+ }
+ for (int j = 0; j < deviceCount; j++)
+ vmConfigSpec.getDeviceChange().add(deviceConfigSpecArray[j]);
+
+ //
+ // Setup VM options
+ //
+
+ // pass boot arguments through machine.id & perform customized options to VMX
+ ArrayList<OptionValue> extraOptions = new ArrayList<>();
+ configBasicExtraOption(extraOptions, vmSpec);
+ configNvpExtraOption(extraOptions, vmSpec, nicUuidToDvSwitchUuid);
+ configCustomExtraOption(extraOptions, vmSpec);
+
+ // config for NCC
+ VirtualMachine.Type vmType = cmd.getVirtualMachine().getType();
+ if (vmType.equals(VirtualMachine.Type.NetScalerVm)) {
+ NicTO mgmtNic = vmSpec.getNics()[0];
+ OptionValue option = new OptionValue();
+ option.setKey("machine.id");
+ option.setValue("ip=" + mgmtNic.getIp() + "&netmask=" + mgmtNic.getNetmask() + "&gateway=" + mgmtNic.getGateway());
+ extraOptions.add(option);
+ }
+
+ // config VNC
+ String keyboardLayout = null;
+ if (vmSpec.getDetails() != null)
+ keyboardLayout = vmSpec.getDetails().get(VmDetailConstants.KEYBOARD);
+ vmConfigSpec.getExtraConfig().addAll(
+ Arrays.asList(vmwareResource.configureVnc(extraOptions.toArray(new OptionValue[0]), hyperHost, vmInternalCSName, vmSpec.getVncPassword(), keyboardLayout)));
+
+ // config video card
+ vmwareResource.configureVideoCard(vmMo, vmSpec, vmConfigSpec);
+
+ // Set OVF properties (if available)
+ Pair<String, List<OVFPropertyTO>> ovfPropsMap = vmSpec.getOvfProperties();
+ VmConfigInfo templateVappConfig;
+ List<OVFPropertyTO> ovfProperties;
+ if (ovfPropsMap != null) {
+ String vmTemplate = ovfPropsMap.first();
+ LOGGER.info("Find VM template " + vmTemplate);
+ VirtualMachineMO vmTemplateMO = dcMo.findVm(vmTemplate);
+ templateVappConfig = vmTemplateMO.getConfigInfo().getVAppConfig();
+ ovfProperties = ovfPropsMap.second();
+ // Set OVF properties (if available)
+ if (CollectionUtils.isNotEmpty(ovfProperties)) {
+ LOGGER.info("Copying OVF properties from template and setting them to the values the user provided");
+ vmwareResource.copyVAppConfigsFromTemplate(templateVappConfig, ovfProperties, vmConfigSpec);
+ }
+ }
+
+ checkBootOptions(vmSpec, vmConfigSpec);
+
+ //
+ // Configure VM
+ //
+ if (!vmMo.configureVm(vmConfigSpec)) {
+ throw new Exception("Failed to configure VM before start. vmName: " + vmInternalCSName);
+ }
+ // FR37 TODO reconcile disks now!!! they are configured, check and add them to the returning vmTO
+ if (vmSpec.getType() == VirtualMachine.Type.DomainRouter) {
+ hyperHost.setRestartPriorityForVM(vmMo, DasVmPriority.HIGH.value());
+ }
+
+ // Resizing root disk only when explicit requested by user
+ final Map<String, String> vmDetails = cmd.getVirtualMachine().getDetails();
+ if (rootDiskTO != null && !hasSnapshot && (vmDetails != null && vmDetails.containsKey(ApiConstants.ROOT_DISK_SIZE))) {
+ resizeRootDiskOnVMStart(vmMo, rootDiskTO, hyperHost, context);
+ }
+
+ //
+ // Post Configuration
+ //
+
+ vmMo.setCustomFieldValue(CustomFieldConstants.CLOUD_NIC_MASK, String.valueOf(nicMask));
+ postNvpConfigBeforeStart(vmMo, vmSpec);
+
+ Map<String, Map<String, String>> iqnToData = new HashMap<>();
+
+ postDiskConfigBeforeStart(vmMo, vmSpec, sortedDisks, iqnToData, hyperHost, context, installAsIs);
+
+ //
+ // Power-on VM
+ //
+ if (!vmMo.powerOn()) {
+ throw new Exception("Failed to start VM. vmName: " + vmInternalCSName + " with hostname " + vmNameOnVcenter);
+ }
+
+ if (installAsIs) {
+ // Set disks as the disks path and chain is retrieved from the cloned VM disks
+ cmd.getVirtualMachine().setDisks(disks);
+ }
+
+ StartAnswer startAnswer = new StartAnswer(cmd);
+
+ startAnswer.setIqnToData(iqnToData);
+
+ deleteOldVersionOfTheStartedVM(existingVm, dcMo, vmMo);
+
+ return startAnswer;
+ } catch (Throwable e) {
+ return handleStartFailure(cmd, vmSpec, existingVm, dcMo, e);
+ } finally {
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("finally done with %s", vmwareResource.getGson().toJson(cmd)));
+ }
+ }
+ }
+
+ /**
+ * Modify the specDisks information to match the cloned VM's disks (from vmMo VM)
+ */
+ private void mapSpecDisksToClonedDisks(VirtualMachineMO vmMo, String vmInternalCSName, DiskTO[] specDisks) {
+ try {
+ if (vmMo != null && ArrayUtils.isNotEmpty(specDisks)) {
+ List<VirtualDisk> vmDisks = vmMo.getVirtualDisks();
+ List<DiskTO> sortedDisks = Arrays.asList(sortVolumesByDeviceId(specDisks))
+ .stream()
+ .filter(x -> x.getType() == Volume.Type.ROOT)
+ .collect(Collectors.toList());
+ if (sortedDisks.size() != vmDisks.size()) {
+ LOGGER.error("Different number of root disks spec vs cloned deploy-as-is VM disks: " + sortedDisks.size() + " - " + vmDisks.size());
+ return;
+ }
+ for (int i = 0; i < sortedDisks.size(); i++) {
+ DiskTO specDisk = sortedDisks.get(i);
+ VirtualDisk vmDisk = vmDisks.get(i);
+ DataTO dataVolume = specDisk.getData();
+ if (dataVolume instanceof VolumeObjectTO) {
+ VolumeObjectTO volumeObjectTO = (VolumeObjectTO) dataVolume;
+ if (!volumeObjectTO.getSize().equals(vmDisk.getCapacityInBytes())) {
+ LOGGER.info("Mapped disk size is not the same as the cloned VM disk size: " +
+ volumeObjectTO.getSize() + " - " + vmDisk.getCapacityInBytes());
+ }
+ VirtualDeviceBackingInfo backingInfo = vmDisk.getBacking();
+ if (backingInfo instanceof VirtualDiskFlatVer2BackingInfo) {
+ VirtualDiskFlatVer2BackingInfo backing = (VirtualDiskFlatVer2BackingInfo) backingInfo;
+ String fileName = backing.getFileName();
+ if (StringUtils.isNotBlank(fileName)) {
+ String[] fileNameParts = fileName.split(" ");
+ String datastoreUuid = fileNameParts[0].replace("[", "").replace("]", "");
+ String relativePath = fileNameParts[1].split("/")[1].replace(".vmdk", "");
+ String vmSpecDatastoreUuid = volumeObjectTO.getDataStore().getUuid().replaceAll("-", "");
+ if (!datastoreUuid.equals(vmSpecDatastoreUuid)) {
+ LOGGER.info("Mapped disk datastore UUID is not the same as the cloned VM datastore UUID: " +
+ datastoreUuid + " - " + vmSpecDatastoreUuid);
+ }
+ volumeObjectTO.setPath(relativePath);
+ specDisk.setPath(relativePath);
+ }
+ }
+ }
+ }
+ }
+ } catch (Exception e) {
+ String msg = "Error mapping deploy-as-is VM disks from cloned VM " + vmInternalCSName;
+ LOGGER.error(msg, e);
+ throw new CloudRuntimeException(e);
+ }
+ }
+
+ private StartAnswer handleStartFailure(StartCommand cmd, VirtualMachineTO vmSpec, VirtualMachineData existingVm, DatacenterMO dcMo, Throwable e) {
+ if (e instanceof RemoteException) {
+ LOGGER.warn("Encounter remote exception to vCenter, invalidate VMware session context");
+ vmwareResource.invalidateServiceContext();
+ }
+
+ String msg = "StartCommand failed due to " + VmwareHelper.getExceptionMessage(e);
+ LOGGER.warn(msg, e);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("didn't start VM %s", vmSpec.getName()), e);
+ }
+ StartAnswer startAnswer = new StartAnswer(cmd, msg);
+ if ( e instanceof VmAlreadyExistsInVcenter) {
+ startAnswer.setContextParam("stopRetry", "true");
+ }
+ reRegisterExistingVm(existingVm, dcMo);
+
+ return startAnswer;
+ }
+
+ private void deleteOldVersionOfTheStartedVM(VirtualMachineData existingVm, DatacenterMO dcMo, VirtualMachineMO vmMo) throws Exception {
+ // Since VM was successfully powered-on, if there was an existing VM in a different cluster that was unregistered, delete all the files associated with it.
+ if (existingVm != null && existingVm.vmName != null && existingVm.vmFileLayout != null) {
+ List<String> vmDatastoreNames = new ArrayList<>();
+ for (DatastoreMO vmDatastore : vmMo.getAllDatastores()) {
+ vmDatastoreNames.add(vmDatastore.getName());
+ }
+ // Don't delete files that are in a datastore that is being used by the new VM as well (zone-wide datastore).
+ List<String> skipDatastores = new ArrayList<>();
+ for (DatastoreMO existingDatastore : existingVm.datastores) {
+ if (vmDatastoreNames.contains(existingDatastore.getName())) {
+ skipDatastores.add(existingDatastore.getName());
+ }
+ }
+ vmwareResource.deleteUnregisteredVmFiles(existingVm.vmFileLayout, dcMo, true, skipDatastores);
+ }
+ }
+
+ private int adjustNumberOfCoresPerSocket(VirtualMachineTO vmSpec, VirtualMachineMO vmMo, VirtualMachineConfigSpec vmConfigSpec) throws Exception {
+ // Check for multi-cores per socket settings
+ int numCoresPerSocket = 1;
+ String coresPerSocket = vmSpec.getDetails().get(VmDetailConstants.CPU_CORE_PER_SOCKET);
+ if (coresPerSocket != null) {
+ String apiVersion = HypervisorHostHelper.getVcenterApiVersion(vmMo.getContext());
+ // Property 'numCoresPerSocket' is supported since vSphere API 5.0
+ if (apiVersion.compareTo("5.0") >= 0) {
+ numCoresPerSocket = NumbersUtil.parseInt(coresPerSocket, 1);
+ vmConfigSpec.setNumCoresPerSocket(numCoresPerSocket);
+ }
+ }
+ return numCoresPerSocket;
+ }
+
+ /**
+ // Setup USB devices
+ */
+ private int setupUsbDevicesAndGetCount(String vmInternalCSName, VirtualMachineMO vmMo, String guestOsId, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount)
+ throws Exception {
+ int usbDeviceCount = 0;
+ if (guestOsId.startsWith("darwin")) { //Mac OS
+ VirtualDevice[] devices = vmMo.getMatchedDevices(new Class<?>[] {VirtualUSBController.class});
+ if (devices.length == 0) {
+ LOGGER.debug("No USB Controller device on VM Start. Add USB Controller device for Mac OS VM " + vmInternalCSName);
+
+ //For Mac OS X systems, the EHCI+UHCI controller is enabled by default and is required for USB mouse and keyboard access.
+ VirtualDevice usbControllerDevice = VmwareHelper.prepareUSBControllerDevice();
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ deviceConfigSpecArray[deviceCount].setDevice(usbControllerDevice);
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare USB controller at new device " + vmwareResource.getGson().toJson(deviceConfigSpecArray[deviceCount]));
+
+ deviceCount++;
+ usbDeviceCount++;
+ } else {
+ LOGGER.debug("USB Controller device exists on VM Start for Mac OS VM " + vmInternalCSName);
+ }
+ }
+ return usbDeviceCount;
+ }
+
+ private void createNewVm(VmwareContext context, VirtualMachineTO vmSpec, boolean installAsIs, String vmInternalCSName, String vmNameOnVcenter, Pair<String, String> controllerInfo, Boolean systemVm,
+ VmwareManager mgr, VmwareHypervisorHost hyperHost, String guestOsId, DiskTO[] disks, HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails,
+ DatastoreMO dsRootVolumeIsOn) throws Exception {
+ VirtualMachineMO vmMo;
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = getRootDiskDataStoreDetails(disks, dataStoresDetails);
+
+ assert (vmSpec.getMinSpeed() != null) && (rootDiskDataStoreDetails != null);
+
+ boolean vmFolderExists = rootDiskDataStoreDetails.second().folderExists(String.format("[%s]", rootDiskDataStoreDetails.second().getName()), vmNameOnVcenter);
+ String vmxFileFullPath = dsRootVolumeIsOn.searchFileInSubFolders(vmNameOnVcenter + ".vmx", false, VmwareManager.s_vmwareSearchExcludeFolder.value());
+ if (vmFolderExists && vmxFileFullPath != null) { // VM can be registered only if .vmx is present.
+ registerVm(vmNameOnVcenter, dsRootVolumeIsOn, vmwareResource);
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ if (vmMo != null) {
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Found registered vm " + vmInternalCSName + " at host " + hyperHost.getHyperHostName());
+ }
+ }
+ tearDownVm(vmMo);
+ } else if (installAsIs) {
+// first get all the MORs ManagedObjectReference morPool = hyperHost.getHyperHostOwnerResourcePool();
+// get the base VM vmMo = hyperHost.findVmOnHyperHost(vm.template.getPath());
+// do templateVm.createLinkedClone(vmInternalCSName, morBaseSnapshot, dcMo.getVmFolder(), morPool, morDatastore)
+// or hyperHost....createLinkedOrFullClone(templateVm, volume, dcMo, vmMo, morDatastore, dsMo, vmInternalCSName, morPool);
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ // At this point vmMo points to the cloned VM
+ } else {
+ if (!hyperHost
+ .createBlankVm(vmNameOnVcenter, vmInternalCSName, vmSpec.getCpus(), vmSpec.getMaxSpeed(), vmwareResource.getReservedCpuMHZ(vmSpec), vmSpec.getLimitCpuUse(), (int)(vmSpec.getMaxRam() / Resource.ResourceType.bytesToMiB), vmwareResource.getReservedMemoryMb(vmSpec), guestOsId,
+ rootDiskDataStoreDetails.first(), false, controllerInfo, systemVm)) {
+ throw new Exception("Failed to create VM. vmName: " + vmInternalCSName);
+ }
+ }
+ }
+
+ /**
+ * If a VM with the same name is found in a different cluster in the DC, unregister the old VM and configure a new VM (cold-migration).
+ */
+ private VirtualMachineData unregisterOnOtherClusterButHoldOnToOldVmData(String vmInternalCSName, DatacenterMO dcMo) throws Exception {
+ VirtualMachineMO existingVmInDc = dcMo.findVm(vmInternalCSName);
+ VirtualMachineData existingVm = null;
+ if (existingVmInDc != null) {
+ existingVm = new VirtualMachineData();
+ existingVm.vmName = existingVmInDc.getName();
+ existingVm.vmFileInfo = existingVmInDc.getFileInfo();
+ existingVm.vmFileLayout = existingVmInDc.getFileLayout();
+ existingVm.datastores = existingVmInDc.getAllDatastores();
+ LOGGER.info("Found VM: " + vmInternalCSName + " on a host in a different cluster. Unregistering the exisitng VM.");
+ existingVmInDc.unregisterVm();
+ }
+ return existingVm;
+ }
+
+ private Pair<ManagedObjectReference, DatastoreMO> getRootDiskDataStoreDetails(DiskTO[] disks, HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails) {
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = null;
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ROOT) {
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (managed) {
+ String datastoreName = VmwareResource.getDatastoreName(details.get(DiskTO.IQN));
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(datastoreName);
+ } else {
+ DataStoreTO primaryStore = vol.getData().getDataStore();
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(primaryStore.getUuid());
+ }
+ }
+ }
+ return rootDiskDataStoreDetails;
+ }
+
+ /**
+ * Since VM start failed, if there was an existing VM in a different cluster that was unregistered, register it back.
+ *
+ * @param dcMo is guaranteed to be not null since we have noticed there is an existing VM in the dc (using that mo)
+ */
+ private void reRegisterExistingVm(VirtualMachineData existingVm, DatacenterMO dcMo) {
+ if (existingVm != null && existingVm.vmName != null && existingVm.vmFileInfo != null) {
+ LOGGER.debug("Since VM start failed, registering back an existing VM: " + existingVm.vmName + " that was unregistered");
+ try {
+ DatastoreFile fileInDatastore = new DatastoreFile(existingVm.vmFileInfo.getVmPathName());
+ DatastoreMO existingVmDsMo = new DatastoreMO(dcMo.getContext(), dcMo.findDatastore(fileInDatastore.getDatastoreName()));
+ registerVm(existingVm.vmName, existingVmDsMo, vmwareResource);
+ } catch (Exception ex) {
+ String message = "Failed to register an existing VM: " + existingVm.vmName + " due to " + VmwareHelper.getExceptionMessage(ex);
+ LOGGER.warn(message, ex);
+ }
+ }
+ }
+
+ private void checkIfVmExistsInVcenter(String vmInternalCSName, String vmNameOnVcenter, DatacenterMO dcMo) throws VmAlreadyExistsInVcenter, Exception {
+ // Validate VM name is unique in Datacenter
+ VirtualMachineMO vmInVcenter = dcMo.checkIfVmAlreadyExistsInVcenter(vmNameOnVcenter, vmInternalCSName);
+ if (vmInVcenter != null) {
+ String msg = "VM with name: " + vmNameOnVcenter + " already exists in vCenter.";
+ LOGGER.error(msg);
+ throw new VmAlreadyExistsInVcenter(msg);
+ }
+ }
+
+ private Pair<String, String> getDiskControllerInfo(VirtualMachineTO vmSpec) {
+ String dataDiskController = vmSpec.getDetails().get(VmDetailConstants.DATA_DISK_CONTROLLER);
+ String rootDiskController = vmSpec.getDetails().get(VmDetailConstants.ROOT_DISK_CONTROLLER);
+ // If root disk controller is scsi, then data disk controller would also be scsi instead of using 'osdefault'
+ // This helps avoid mix of different scsi subtype controllers in instance.
+ if (DiskControllerType.osdefault == DiskControllerType.getType(dataDiskController) && DiskControllerType.lsilogic == DiskControllerType.getType(rootDiskController)) {
+ dataDiskController = DiskControllerType.scsi.toString();
+ }
+
+ // Validate the controller types
+ dataDiskController = DiskControllerType.getType(dataDiskController).toString();
+ rootDiskController = DiskControllerType.getType(rootDiskController).toString();
+
+ if (DiskControllerType.getType(rootDiskController) == DiskControllerType.none) {
+ throw new CloudRuntimeException("Invalid root disk controller detected : " + rootDiskController);
+ }
+ if (DiskControllerType.getType(dataDiskController) == DiskControllerType.none) {
+ throw new CloudRuntimeException("Invalid data disk controller detected : " + dataDiskController);
+ }
+
+ return new Pair<>(rootDiskController, dataDiskController);
+ }
+
+ /**
+ * Registers the vm to the inventory given the vmx file.
+ * @param vmName
+ * @param dsMo
+ * @param vmwareResource
+ */
+ private void registerVm(String vmName, DatastoreMO dsMo, VmwareResource vmwareResource) throws Exception {
+
+ //1st param
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(vmwareResource.getServiceContext());
+ ManagedObjectReference dcMor = hyperHost.getHyperHostDatacenter();
+ DatacenterMO dataCenterMo = new DatacenterMO(vmwareResource.getServiceContext(), dcMor);
+ ManagedObjectReference vmFolderMor = dataCenterMo.getVmFolder();
+
+ //2nd param
+ String vmxFilePath = dsMo.searchFileInSubFolders(vmName + ".vmx", false, VmwareManager.s_vmwareSearchExcludeFolder.value());
+
+ // 5th param
+ ManagedObjectReference morPool = hyperHost.getHyperHostOwnerResourcePool();
+
+ ManagedObjectReference morTask = vmwareResource.getServiceContext().getService().registerVMTask(vmFolderMor, vmxFilePath, vmName, false, morPool, hyperHost.getMor());
+ boolean result = vmwareResource.getServiceContext().getVimClient().waitForTask(morTask);
+ if (!result) {
+ throw new Exception("Unable to register vm due to " + TaskMO.getTaskFailureInfo(vmwareResource.getServiceContext(), morTask));
+ } else {
+ vmwareResource.getServiceContext().waitForTaskProgressDone(morTask);
+ }
+
+ }
+
+ // Pair<internal CS name, vCenter display name>
+ private Pair<String, String> composeVmNames(VirtualMachineTO vmSpec) {
+ String vmInternalCSName = vmSpec.getName();
+ String vmNameOnVcenter = vmSpec.getName();
+ if (VmwareResource.instanceNameFlag && vmSpec.getHostName() != null) {
+ vmNameOnVcenter = vmSpec.getHostName();
+ }
+ return new Pair<String, String>(vmInternalCSName, vmNameOnVcenter);
+ }
+
+ private VirtualMachineGuestOsIdentifier translateGuestOsIdentifier(String cpuArchitecture, String guestOs, String cloudGuestOs) {
+ if (cpuArchitecture == null) {
+ LOGGER.warn("CPU arch is not set, default to i386. guest os: " + guestOs);
+ cpuArchitecture = "i386";
+ }
+
+ if (cloudGuestOs == null) {
+ LOGGER.warn("Guest OS mapping name is not set for guest os: " + guestOs);
+ }
+
+ VirtualMachineGuestOsIdentifier identifier = null;
+ try {
+ if (cloudGuestOs != null) {
+ identifier = VirtualMachineGuestOsIdentifier.fromValue(cloudGuestOs);
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Using mapping name : " + identifier.toString());
+ }
+ }
+ } catch (IllegalArgumentException e) {
+ LOGGER.warn("Unable to find Guest OS Identifier in VMware for mapping name: " + cloudGuestOs + ". Continuing with defaults.");
+ }
+ if (identifier != null) {
+ return identifier;
+ }
+
+ if (cpuArchitecture.equalsIgnoreCase("x86_64")) {
+ return VirtualMachineGuestOsIdentifier.OTHER_GUEST_64;
+ }
+ return VirtualMachineGuestOsIdentifier.OTHER_GUEST;
+ }
+
+ private DiskTO[] validateDisks(DiskTO[] disks) {
+ List<DiskTO> validatedDisks = new ArrayList<DiskTO>();
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() != Volume.Type.ISO) {
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ if (primaryStore.getUuid() != null && !primaryStore.getUuid().isEmpty()) {
+ validatedDisks.add(vol);
+ }
+ } else if (vol.getType() == Volume.Type.ISO) {
+ TemplateObjectTO templateTO = (TemplateObjectTO) vol.getData();
+ if (templateTO.getPath() != null && !templateTO.getPath().isEmpty()) {
+ validatedDisks.add(vol);
+ }
+ } else {
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Drop invalid disk option, volumeTO: " + vmwareResource.getGson().toJson(vol));
+ }
+ }
+ }
+ Collections.sort(validatedDisks, (d1, d2) -> d1.getDiskSeq().compareTo(d2.getDiskSeq()));
+ return validatedDisks.toArray(new DiskTO[0]);
+ }
+
+ private HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> inferDatastoreDetailsFromDiskInfo(VmwareHypervisorHost hyperHost, VmwareContext context,
+ DiskTO[] disks, Command cmd) throws Exception {
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> mapIdToMors = new HashMap<>();
+
+ assert (hyperHost != null) && (context != null);
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() != Volume.Type.ISO) {
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ String poolUuid = primaryStore.getUuid();
+
+ if (mapIdToMors.get(poolUuid) == null) {
+ boolean isManaged = false;
+ Map<String, String> details = vol.getDetails();
+
+ if (details != null) {
+ isManaged = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (isManaged) {
+ String iScsiName = details.get(DiskTO.IQN); // details should not be null for managed storage (it may or may not be null for non-managed storage)
+ String datastoreName = VmwareResource.getDatastoreName(iScsiName);
+ ManagedObjectReference morDatastore = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, datastoreName);
+
+ // if the datastore is not present, we need to discover the iSCSI device that will support it,
+ // create the datastore, and create a VMDK file in the datastore
+ if (morDatastore == null) {
+ final String vmdkPath = vmwareResource.getVmdkPath(volumeTO.getPath());
+
+ morDatastore = getStorageProcessor().prepareManagedStorage(context, hyperHost, null, iScsiName,
+ details.get(DiskTO.STORAGE_HOST), Integer.parseInt(details.get(DiskTO.STORAGE_PORT)),
+ vmdkPath,
+ details.get(DiskTO.CHAP_INITIATOR_USERNAME), details.get(DiskTO.CHAP_INITIATOR_SECRET),
+ details.get(DiskTO.CHAP_TARGET_USERNAME), details.get(DiskTO.CHAP_TARGET_SECRET),
+ Long.parseLong(details.get(DiskTO.VOLUME_SIZE)), cmd);
+
+ DatastoreMO dsMo = new DatastoreMO(vmwareResource.getServiceContext(), morDatastore);
+
+ final String datastoreVolumePath;
+
+ if (vmdkPath != null) {
+ datastoreVolumePath = dsMo.getDatastorePath(vmdkPath + VmwareResource.VMDK_EXTENSION);
+ } else {
+ datastoreVolumePath = dsMo.getDatastorePath(dsMo.getName() + VmwareResource.VMDK_EXTENSION);
+ }
+
+ volumeTO.setPath(datastoreVolumePath);
+ vol.setPath(datastoreVolumePath);
+ }
+
+ mapIdToMors.put(datastoreName, new Pair<>(morDatastore, new DatastoreMO(context, morDatastore)));
+ } else {
+ ManagedObjectReference morDatastore = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, poolUuid);
+
+ if (morDatastore == null) {
+ String msg = "Failed to get the mounted datastore for the volume's pool " + poolUuid;
+
+ LOGGER.error(msg);
+
+ throw new Exception(msg);
+ }
+
+ mapIdToMors.put(poolUuid, new Pair<>(morDatastore, new DatastoreMO(context, morDatastore)));
+ }
+ }
+ }
+ }
+
+ return mapIdToMors;
+ }
+
+ private VmwareStorageProcessor getStorageProcessor() {
+ return vmwareResource.getStorageProcessor();
+ }
+
+ private DatastoreMO getDatastoreThatRootDiskIsOn(HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails, DiskTO disks[]) {
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = null;
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ROOT) {
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (managed) {
+ String datastoreName = VmwareResource.getDatastoreName(details.get(DiskTO.IQN));
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(datastoreName);
+
+ break;
+ } else {
+ DataStoreTO primaryStore = vol.getData().getDataStore();
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(primaryStore.getUuid());
+
+ break;
+ }
+ }
+ }
+
+ if (rootDiskDataStoreDetails != null) {
+ return rootDiskDataStoreDetails.second();
+ }
+
+ return null;
+ }
+
+ private void ensureDiskControllers(VirtualMachineMO vmMo, Pair<String, String> controllerInfo) throws Exception {
+ if (vmMo == null) {
+ return;
+ }
+
+ String msg;
+ String rootDiskController = controllerInfo.first();
+ String dataDiskController = controllerInfo.second();
+ String scsiDiskController;
+ String recommendedDiskController = null;
+
+ if (VmwareHelper.isControllerOsRecommended(dataDiskController) || VmwareHelper.isControllerOsRecommended(rootDiskController)) {
+ recommendedDiskController = vmMo.getRecommendedDiskController(null);
+ }
+ scsiDiskController = HypervisorHostHelper.getScsiController(new Pair<String, String>(rootDiskController, dataDiskController), recommendedDiskController);
+ if (scsiDiskController == null) {
+ return;
+ }
+
+ vmMo.getScsiDeviceControllerKeyNoException();
+ // This VM needs SCSI controllers.
+ // Get count of existing scsi controllers. Helps not to attempt to create more than the maximum allowed 4
+ // Get maximum among the bus numbers in use by scsi controllers. Safe to pick maximum, because we always go sequential allocating bus numbers.
+ Ternary<Integer, Integer, DiskControllerType> scsiControllerInfo = vmMo.getScsiControllerInfo();
+ int requiredNumScsiControllers = VmwareHelper.MAX_SCSI_CONTROLLER_COUNT - scsiControllerInfo.first();
+ int availableBusNum = scsiControllerInfo.second() + 1; // method returned current max. bus number
+
+ if (requiredNumScsiControllers == 0) {
+ return;
+ }
+ if (scsiControllerInfo.first() > 0) {
+ // For VMs which already have a SCSI controller, do NOT attempt to add any more SCSI controllers & return the sub type.
+ // For Legacy VMs would have only 1 LsiLogic Parallel SCSI controller, and doesn't require more.
+ // For VMs created post device ordering support, 4 SCSI subtype controllers are ensured during deployment itself. No need to add more.
+ // For fresh VM deployment only, all required controllers should be ensured.
+ return;
+ }
+ ensureScsiDiskControllers(vmMo, scsiDiskController, requiredNumScsiControllers, availableBusNum);
+ }
+
+ private void ensureScsiDiskControllers(VirtualMachineMO vmMo, String scsiDiskController, int requiredNumScsiControllers, int availableBusNum) throws Exception {
+ // Pick the sub type of scsi
+ if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.pvscsi) {
+ if (!vmMo.isPvScsiSupported()) {
+ String msg = "This VM doesn't support Vmware Paravirtual SCSI controller for virtual disks, because the virtual hardware version is less than 7.";
+ throw new Exception(msg);
+ }
+ vmMo.ensurePvScsiDeviceController(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.lsisas1068) {
+ vmMo.ensureLsiLogicSasDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.buslogic) {
+ vmMo.ensureBusLogicDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.lsilogic) {
+ vmMo.ensureLsiLogicDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ }
+ }
+
+ private void tearDownVm(VirtualMachineMO vmMo) throws Exception {
+
+ if (vmMo == null)
+ return;
+
+ boolean hasSnapshot = false;
+ hasSnapshot = vmMo.hasSnapshot();
+ if (!hasSnapshot)
+ vmMo.tearDownDevices(new Class<?>[]{VirtualDisk.class, VirtualEthernetCard.class});
+ else
+ vmMo.tearDownDevices(new Class<?>[]{VirtualEthernetCard.class});
+ vmMo.ensureScsiDeviceController();
+ }
+
+ private static DiskTO getIsoDiskTO(DiskTO[] disks) {
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ISO) {
+ return vol;
+ }
+ }
+ return null;
+ }
+
+ private static DiskTO[] sortVolumesByDeviceId(DiskTO[] volumes) {
+
+ List<DiskTO> listForSort = new ArrayList<DiskTO>();
+ for (DiskTO vol : volumes) {
+ listForSort.add(vol);
+ }
+ Collections.sort(listForSort, new Comparator<DiskTO>() {
+
+ @Override
+ public int compare(DiskTO arg0, DiskTO arg1) {
+ if (arg0.getDiskSeq() < arg1.getDiskSeq()) {
+ return -1;
+ } else if (arg0.getDiskSeq().equals(arg1.getDiskSeq())) {
+ return 0;
+ }
+
+ return 1;
+ }
+ });
+
+ return listForSort.toArray(new DiskTO[0]);
+ }
+
+ private void postDiskConfigBeforeStart(VirtualMachineMO vmMo, VirtualMachineTO vmSpec, DiskTO[] sortedDisks,
+ Map<String, Map<String, String>> iqnToData, VmwareHypervisorHost hyperHost, VmwareContext context, boolean installAsIs)
+ throws Exception {
+ VirtualMachineDiskInfoBuilder diskInfoBuilder = vmMo.getDiskInfoBuilder();
+
+ for (DiskTO vol : sortedDisks) {
+ //TODO: Map existing disks to the ones returned in the answer
+ if (vol.getType() == Volume.Type.ISO)
+ continue;
+
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+
+ VirtualMachineDiskInfo diskInfo = getMatchingExistingDisk(diskInfoBuilder, vol, hyperHost, context);
+ assert (diskInfo != null);
+
+ String[] diskChain = diskInfo.getDiskChain();
+ assert (diskChain.length > 0);
+
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ DatastoreFile file = new DatastoreFile(diskChain[0]);
+
+ if (managed) {
+ DatastoreFile originalFile = new DatastoreFile(volumeTO.getPath());
+
+ if (!file.getFileBaseName().equalsIgnoreCase(originalFile.getFileBaseName())) {
+ if (LOGGER.isInfoEnabled())
+ LOGGER.info("Detected disk-chain top file change on volume: " + volumeTO.getId() + " " + volumeTO.getPath() + " -> " + diskChain[0]);
+ }
+ } else {
+ if (!file.getFileBaseName().equalsIgnoreCase(volumeTO.getPath())) {
+ if (LOGGER.isInfoEnabled())
+ LOGGER.info("Detected disk-chain top file change on volume: " + volumeTO.getId() + " " + volumeTO.getPath() + " -> " + file.getFileBaseName());
+ }
+ }
+
+ VolumeObjectTO volInSpec = getVolumeInSpec(vmSpec, volumeTO);
+
+ if (volInSpec != null) {
+ if (managed) {
+ Map<String, String> data = new HashMap<>();
+
+ String datastoreVolumePath = diskChain[0];
+
+ data.put(StartAnswer.PATH, datastoreVolumePath);
+ data.put(StartAnswer.IMAGE_FORMAT, Storage.ImageFormat.OVA.toString());
+
+ iqnToData.put(details.get(DiskTO.IQN), data);
+
+ vol.setPath(datastoreVolumePath);
+ volumeTO.setPath(datastoreVolumePath);
+ volInSpec.setPath(datastoreVolumePath);
+ } else {
+ volInSpec.setPath(file.getFileBaseName());
+ }
+ volInSpec.setChainInfo(vmwareResource.getGson().toJson(diskInfo));
+ }
+ }
+ }
+
+ private void checkBootOptions(VirtualMachineTO vmSpec, VirtualMachineConfigSpec vmConfigSpec) {
+ String bootMode = null;
+ if (vmSpec.getDetails().containsKey(VmDetailConstants.BOOT_MODE)) {
+ bootMode = vmSpec.getDetails().get(VmDetailConstants.BOOT_MODE);
+ }
+ if (null == bootMode) {
+ bootMode = ApiConstants.BootType.BIOS.toString();
+ }
+
+ setBootOptions(vmSpec, bootMode, vmConfigSpec);
+ }
+
+ private void setBootOptions(VirtualMachineTO vmSpec, String bootMode, VirtualMachineConfigSpec vmConfigSpec) {
+ VirtualMachineBootOptions bootOptions = null;
+ if (StringUtils.isNotBlank(bootMode) && !bootMode.equalsIgnoreCase("bios")) {
+ vmConfigSpec.setFirmware("efi");
+ if (vmSpec.getDetails().containsKey(ApiConstants.BootType.UEFI.toString()) && "secure".equalsIgnoreCase(vmSpec.getDetails().get(ApiConstants.BootType.UEFI.toString()))) {
+ if (bootOptions == null) {
+ bootOptions = new VirtualMachineBootOptions();
+ }
+ bootOptions.setEfiSecureBootEnabled(true);
+ }
+ }
+ if (vmSpec.isEnterHardwareSetup()) {
+ if (bootOptions == null) {
+ bootOptions = new VirtualMachineBootOptions();
+ }
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("configuring VM '%s' to enter hardware setup",vmSpec.getName()));
+ }
+ bootOptions.setEnterBIOSSetup(vmSpec.isEnterHardwareSetup());
+ }
+ if (bootOptions != null) {
+ vmConfigSpec.setBootOptions(bootOptions);
+ }
+ }
+
+ private void resizeRootDiskOnVMStart(VirtualMachineMO vmMo, DiskTO rootDiskTO, VmwareHypervisorHost hyperHost, VmwareContext context) throws Exception {
+ final Pair<VirtualDisk, String> vdisk = vmwareResource.getVirtualDiskInfo(vmMo, VmwareResource.appendFileType(rootDiskTO.getPath(), VmwareResource.VMDK_EXTENSION));
+ assert (vdisk != null);
+
+ Long reqSize = 0L;
+ final VolumeObjectTO volumeTO = ((VolumeObjectTO) rootDiskTO.getData());
+ if (volumeTO != null) {
+ reqSize = volumeTO.getSize() / 1024;
+ }
+ final VirtualDisk disk = vdisk.first();
+ if (reqSize > disk.getCapacityInKB()) {
+ final VirtualMachineDiskInfo diskInfo = getMatchingExistingDisk(vmMo.getDiskInfoBuilder(), rootDiskTO, hyperHost, context);
+ assert (diskInfo != null);
+ final String[] diskChain = diskInfo.getDiskChain();
+
+ if (diskChain != null && diskChain.length > 1) {
+ LOGGER.warn("Disk chain length for the VM is greater than one, this is not supported");
+ throw new CloudRuntimeException("Unsupported VM disk chain length: " + diskChain.length);
+ }
+
+ boolean resizingSupported = false;
+ String deviceBusName = diskInfo.getDiskDeviceBusName();
+ if (deviceBusName != null && (deviceBusName.toLowerCase().contains("scsi") || deviceBusName.toLowerCase().contains("lsi"))) {
+ resizingSupported = true;
+ }
+ if (!resizingSupported) {
+ LOGGER.warn("Resizing of root disk is only support for scsi device/bus, the provide VM's disk device bus name is " + diskInfo.getDiskDeviceBusName());
+ throw new CloudRuntimeException("Unsupported VM root disk device bus: " + diskInfo.getDiskDeviceBusName());
+ }
+
+ disk.setCapacityInKB(reqSize);
+ VirtualMachineConfigSpec vmConfigSpec = new VirtualMachineConfigSpec();
+ VirtualDeviceConfigSpec deviceConfigSpec = new VirtualDeviceConfigSpec();
+ deviceConfigSpec.setDevice(disk);
+ deviceConfigSpec.setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ vmConfigSpec.getDeviceChange().add(deviceConfigSpec);
+ if (!vmMo.configureVm(vmConfigSpec)) {
+ throw new Exception("Failed to configure VM for given root disk size. vmName: " + vmMo.getName());
+ }
+ }
+ }
+
+ private static void postNvpConfigBeforeStart(VirtualMachineMO vmMo, VirtualMachineTO vmSpec) throws Exception {
+ /**
+ * We need to configure the port on the DV switch after the host is
+ * connected. So make this happen between the configure and start of
+ * the VM
+ */
+ int nicIndex = 0;
+ for (NicTO nicTo : sortNicsByDeviceId(vmSpec.getNics())) {
+ if (nicTo.getBroadcastType() == Networks.BroadcastDomainType.Lswitch) {
+ // We need to create a port with a unique vlan and pass the key to the nic device
+ LOGGER.trace("Nic " + nicTo.toString() + " is connected to an NVP logicalswitch");
+ VirtualDevice nicVirtualDevice = vmMo.getNicDeviceByIndex(nicIndex);
+ if (nicVirtualDevice == null) {
+ throw new Exception("Failed to find a VirtualDevice for nic " + nicIndex); //FIXME Generic exceptions are bad
+ }
+ VirtualDeviceBackingInfo backing = nicVirtualDevice.getBacking();
+ if (backing instanceof VirtualEthernetCardDistributedVirtualPortBackingInfo) {
+ // This NIC is connected to a Distributed Virtual Switch
+ VirtualEthernetCardDistributedVirtualPortBackingInfo portInfo = (VirtualEthernetCardDistributedVirtualPortBackingInfo) backing;
+ DistributedVirtualSwitchPortConnection port = portInfo.getPort();
+ String portKey = port.getPortKey();
+ String portGroupKey = port.getPortgroupKey();
+ String dvSwitchUuid = port.getSwitchUuid();
+
+ LOGGER.debug("NIC " + nicTo.toString() + " is connected to dvSwitch " + dvSwitchUuid + " pg " + portGroupKey + " port " + portKey);
+
+ ManagedObjectReference dvSwitchManager = vmMo.getContext().getVimClient().getServiceContent().getDvSwitchManager();
+ ManagedObjectReference dvSwitch = vmMo.getContext().getVimClient().getService().queryDvsByUuid(dvSwitchManager, dvSwitchUuid);
+
+ // Get all ports
+ DistributedVirtualSwitchPortCriteria criteria = new DistributedVirtualSwitchPortCriteria();
+ criteria.setInside(true);
+ criteria.getPortgroupKey().add(portGroupKey);
+ List<DistributedVirtualPort> dvPorts = vmMo.getContext().getVimClient().getService().fetchDVPorts(dvSwitch, criteria);
+
+ DistributedVirtualPort vmDvPort = null;
+ List<Integer> usedVlans = new ArrayList<Integer>();
+ for (DistributedVirtualPort dvPort : dvPorts) {
+ // Find the port for this NIC by portkey
+ if (portKey.equals(dvPort.getKey())) {
+ vmDvPort = dvPort;
+ }
+ VMwareDVSPortSetting settings = (VMwareDVSPortSetting) dvPort.getConfig().getSetting();
+ VmwareDistributedVirtualSwitchVlanIdSpec vlanId = (VmwareDistributedVirtualSwitchVlanIdSpec) settings.getVlan();
+ LOGGER.trace("Found port " + dvPort.getKey() + " with vlan " + vlanId.getVlanId());
+ if (vlanId.getVlanId() > 0 && vlanId.getVlanId() < 4095) {
+ usedVlans.add(vlanId.getVlanId());
+ }
+ }
+
+ if (vmDvPort == null) {
+ throw new Exception("Empty port list from dvSwitch for nic " + nicTo.toString());
+ }
+
+ DVPortConfigInfo dvPortConfigInfo = vmDvPort.getConfig();
+ VMwareDVSPortSetting settings = (VMwareDVSPortSetting) dvPortConfigInfo.getSetting();
+
+ VmwareDistributedVirtualSwitchVlanIdSpec vlanId = (VmwareDistributedVirtualSwitchVlanIdSpec) settings.getVlan();
+ BoolPolicy blocked = settings.getBlocked();
+ if (blocked.isValue() == Boolean.TRUE) {
+ LOGGER.trace("Port is blocked, set a vlanid and unblock");
+ DVPortConfigSpec dvPortConfigSpec = new DVPortConfigSpec();
+ VMwareDVSPortSetting edittedSettings = new VMwareDVSPortSetting();
+ // Unblock
+ blocked.setValue(Boolean.FALSE);
+ blocked.setInherited(Boolean.FALSE);
+ edittedSettings.setBlocked(blocked);
+ // Set vlan
+ int i;
+ for (i = 1; i < 4095; i++) {
+ if (!usedVlans.contains(i))
+ break;
+ }
+ vlanId.setVlanId(i); // FIXME should be a determined
+ // based on usage
+ vlanId.setInherited(false);
+ edittedSettings.setVlan(vlanId);
+
+ dvPortConfigSpec.setSetting(edittedSettings);
+ dvPortConfigSpec.setOperation("edit");
+ dvPortConfigSpec.setKey(portKey);
+ List<DVPortConfigSpec> dvPortConfigSpecs = new ArrayList<DVPortConfigSpec>();
+ dvPortConfigSpecs.add(dvPortConfigSpec);
+ ManagedObjectReference task = vmMo.getContext().getVimClient().getService().reconfigureDVPortTask(dvSwitch, dvPortConfigSpecs);
+ if (!vmMo.getContext().getVimClient().waitForTask(task)) {
+ throw new Exception("Failed to configure the dvSwitch port for nic " + nicTo.toString());
+ }
+ LOGGER.debug("NIC " + nicTo.toString() + " connected to vlan " + i);
+ } else {
+ LOGGER.trace("Port already configured and set to vlan " + vlanId.getVlanId());
+ }
+ } else if (backing instanceof VirtualEthernetCardNetworkBackingInfo) {
+ // This NIC is connected to a Virtual Switch
+ // Nothing to do
+ } else if (backing instanceof VirtualEthernetCardOpaqueNetworkBackingInfo) {
+ //if NSX API VERSION >= 4.2, connect to br-int (nsx.network), do not create portgroup else previous behaviour
+ //OK, connected to OpaqueNetwork
+ } else {
+ LOGGER.error("nic device backing is of type " + backing.getClass().getName());
+ throw new Exception("Incompatible backing for a VirtualDevice for nic " + nicIndex); //FIXME Generic exceptions are bad
+ }
+ }
+ nicIndex++;
+ }
+ }
+
+ private VirtualMachineDiskInfo getMatchingExistingDisk(VirtualMachineDiskInfoBuilder diskInfoBuilder, DiskTO vol, VmwareHypervisorHost hyperHost, VmwareContext context)
+ throws Exception {
+ if (diskInfoBuilder != null) {
+ VolumeObjectTO volume = (VolumeObjectTO) vol.getData();
+
+ String dsName = null;
+ String diskBackingFileBaseName = null;
+
+ Map<String, String> details = vol.getDetails();
+ boolean isManaged = details != null && Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+
+ if (isManaged) {
+ String iScsiName = details.get(DiskTO.IQN);
+
+ // if the storage is managed, iScsiName should not be null
+ dsName = VmwareResource.getDatastoreName(iScsiName);
+
+ diskBackingFileBaseName = new DatastoreFile(volume.getPath()).getFileBaseName();
+ } else {
+ ManagedObjectReference morDs = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, volume.getDataStore().getUuid());
+ DatastoreMO dsMo = new DatastoreMO(context, morDs);
+
+ dsName = dsMo.getName();
+
+ diskBackingFileBaseName = volume.getPath();
+ }
+
+ VirtualMachineDiskInfo diskInfo = diskInfoBuilder.getDiskInfoByBackingFileBaseName(diskBackingFileBaseName, dsName);
+ if (diskInfo != null) {
+ LOGGER.info("Found existing disk info from volume path: " + volume.getPath());
+ return diskInfo;
+ } else {
+ String chainInfo = volume.getChainInfo();
+ if (chainInfo != null) {
+ VirtualMachineDiskInfo infoInChain = vmwareResource.getGson().fromJson(chainInfo, VirtualMachineDiskInfo.class);
+ if (infoInChain != null) {
+ String[] disks = infoInChain.getDiskChain();
+ if (disks.length > 0) {
+ for (String diskPath : disks) {
+ DatastoreFile file = new DatastoreFile(diskPath);
+ diskInfo = diskInfoBuilder.getDiskInfoByBackingFileBaseName(file.getFileBaseName(), dsName);
+ if (diskInfo != null) {
+ LOGGER.info("Found existing disk from chain info: " + diskPath);
+ return diskInfo;
+ }
+ }
+ }
+
+ if (diskInfo == null) {
+ diskInfo = diskInfoBuilder.getDiskInfoByDeviceBusName(infoInChain.getDiskDeviceBusName());
+ if (diskInfo != null) {
+ LOGGER.info("Found existing disk from from chain device bus information: " + infoInChain.getDiskDeviceBusName());
+ return diskInfo;
+ }
+ }
+ }
+ }
+ }
+ }
+
+ return null;
+ }
+
+ private static VolumeObjectTO getVolumeInSpec(VirtualMachineTO vmSpec, VolumeObjectTO srcVol) {
+ for (DiskTO disk : vmSpec.getDisks()) {
+ if (disk.getData() instanceof VolumeObjectTO) {
+ VolumeObjectTO vol = (VolumeObjectTO) disk.getData();
+ if (vol.getId() == srcVol.getId())
+ return vol;
+ }
+ }
+
+ return null;
+ }
+
+ /**
+ * Generate the mac sequence from the nics.
+ */
+ protected String generateMacSequence(NicTO[] nics) {
+ if (nics.length == 0) {
+ return "";
+ }
+
+ StringBuffer sbMacSequence = new StringBuffer();
+ for (NicTO nicTo : sortNicsByDeviceId(nics)) {
+ sbMacSequence.append(nicTo.getMac()).append("|");
+ }
+ if (!sbMacSequence.toString().isEmpty()) {
+ sbMacSequence.deleteCharAt(sbMacSequence.length() - 1); //Remove extra '|' char appended at the end
+ }
+
+ return sbMacSequence.toString();
+ }
+
+ static NicTO[] sortNicsByDeviceId(NicTO[] nics) {
+
+ List<NicTO> listForSort = new ArrayList<NicTO>();
+ for (NicTO nic : nics) {
+ listForSort.add(nic);
+ }
+ Collections.sort(listForSort, new Comparator<NicTO>() {
+
+ @Override
+ public int compare(NicTO arg0, NicTO arg1) {
+ if (arg0.getDeviceId() < arg1.getDeviceId()) {
+ return -1;
+ } else if (arg0.getDeviceId() == arg1.getDeviceId()) {
+ return 0;
+ }
+
+ return 1;
+ }
+ });
+
+ return listForSort.toArray(new NicTO[0]);
+ }
+
+ private static void configBasicExtraOption(List<OptionValue> extraOptions, VirtualMachineTO vmSpec) {
+ OptionValue newVal = new OptionValue();
+ newVal.setKey("machine.id");
+ newVal.setValue(vmSpec.getBootArgs());
+ extraOptions.add(newVal);
+
+ newVal = new OptionValue();
+ newVal.setKey("devices.hotplug");
+ newVal.setValue("true");
+ extraOptions.add(newVal);
+ }
+
+ private static void configNvpExtraOption(List<OptionValue> extraOptions, VirtualMachineTO vmSpec, Map<String, String> nicUuidToDvSwitchUuid) {
+ /**
+ * Extra Config : nvp.vm-uuid = uuid
+ * - Required for Nicira NVP integration
+ */
+ OptionValue newVal = new OptionValue();
+ newVal.setKey("nvp.vm-uuid");
+ newVal.setValue(vmSpec.getUuid());
+ extraOptions.add(newVal);
+
+ /**
+ * Extra Config : nvp.iface-id.<num> = uuid
+ * - Required for Nicira NVP integration
+ */
+ int nicNum = 0;
+ for (NicTO nicTo : sortNicsByDeviceId(vmSpec.getNics())) {
+ if (nicTo.getUuid() != null) {
+ newVal = new OptionValue();
+ newVal.setKey("nvp.iface-id." + nicNum);
+ newVal.setValue(nicTo.getUuid());
+ extraOptions.add(newVal);
+ }
+ nicNum++;
+ }
+ }
+
+ private static void configCustomExtraOption(List<OptionValue> extraOptions, VirtualMachineTO vmSpec) {
+ // we no longer to validation anymore
+ for (Map.Entry<String, String> entry : vmSpec.getDetails().entrySet()) {
+ if (entry.getKey().equalsIgnoreCase(VmDetailConstants.BOOT_MODE)) {
+ continue;
+ }
+ OptionValue newVal = new OptionValue();
+ newVal.setKey(entry.getKey());
+ newVal.setValue(entry.getValue());
+ extraOptions.add(newVal);
+ }
+ }
+
+ private class VmAlreadyExistsInVcenter extends Exception {
+ public VmAlreadyExistsInVcenter(String msg) {
+ }
+ }
+
+ private class VirtualMachineData {
+ String vmName = null;
+ VirtualMachineFileInfo vmFileInfo = null;
+ VirtualMachineFileLayoutEx vmFileLayout = null;
+ List<DatastoreMO> datastores = new ArrayList<>();
+ }
+
+ private class VirtualMachineRecycler {
+ private String vmInternalCSName;
+ private Pair<String, String> controllerInfo;
+ private Boolean systemVm;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private DiskControllerType systemVmScsiControllerType;
+ private int firstScsiControllerBusNum;
+ private int numScsiControllerForSystemVm;
+ private VirtualMachineDiskInfoBuilder diskInfoBuilder;
+ private boolean hasSnapshot;
+ private VirtualDevice[] nicDevices;
+
+ public VirtualMachineRecycler(String vmInternalCSName, Pair<String, String> controllerInfo, Boolean systemVm, VmwareHypervisorHost hyperHost, VirtualMachineMO vmMo,
+ DiskControllerType systemVmScsiControllerType, int firstScsiControllerBusNum, int numScsiControllerForSystemVm) {
+ this.vmInternalCSName = vmInternalCSName;
+ this.controllerInfo = controllerInfo;
+ this.systemVm = systemVm;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.systemVmScsiControllerType = systemVmScsiControllerType;
+ this.firstScsiControllerBusNum = firstScsiControllerBusNum;
+ this.numScsiControllerForSystemVm = numScsiControllerForSystemVm;
+ }
+
+ public VirtualMachineDiskInfoBuilder getDiskInfoBuilder() {
+ return diskInfoBuilder;
+ }
+
+ public boolean isHasSnapshot() {
+ return hasSnapshot;
+ }
+
+ public VirtualMachineRecycler invoke() throws Exception {
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Found vm " + vmInternalCSName + " at other host, relocate to " + hyperHost.getHyperHostName());
+ }
+
+ takeVmFromOtherHyperHost(hyperHost, vmInternalCSName);
+
+ if (VmwareResource.getVmPowerState(vmMo) != VirtualMachine.PowerState.PowerOff)
+ vmMo.safePowerOff(vmwareResource.getShutdownWaitMs());
+
+ diskInfoBuilder = vmMo.getDiskInfoBuilder();
+ hasSnapshot = vmMo.hasSnapshot();
+ nicDevices = vmMo.getNicDevices();
+ if (!hasSnapshot)
+ vmMo.tearDownDevices(new Class<?>[] {VirtualDisk.class, VirtualEthernetCard.class});
+ else
+ vmMo.tearDownDevices(new Class<?>[] {VirtualEthernetCard.class});
+
+ if (systemVm) {
+ // System volumes doesn't require more than 1 SCSI controller as there is no requirement for data volumes.
Review comment:
```suggestion
// System volumes don't require more than 1 SCSI controller as there is no requirement for data volumes.
```
##########
File path: core/src/main/java/com/cloud/storage/template/OVAProcessor.java
##########
@@ -53,73 +63,140 @@ public FormatInfo process(String templatePath, ImageFormat format, String templa
@Override
public FormatInfo process(String templatePath, ImageFormat format, String templateName, long processTimeout) throws InternalErrorException {
- if (format != null) {
- if (s_logger.isInfoEnabled()) {
- s_logger.info("We currently don't handle conversion from " + format + " to OVA.");
- }
+ if (! conversionChecks(format)){
return null;
}
- s_logger.info("Template processing. templatePath: " + templatePath + ", templateName: " + templateName);
+ LOGGER.info("Template processing. templatePath: " + templatePath + ", templateName: " + templateName);
String templateFilePath = templatePath + File.separator + templateName + "." + ImageFormat.OVA.getFileExtension();
if (!_storage.exists(templateFilePath)) {
- if (s_logger.isInfoEnabled()) {
- s_logger.info("Unable to find the vmware template file: " + templateFilePath);
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Unable to find the vmware template file: " + templateFilePath);
}
return null;
}
- s_logger.info("Template processing - untar OVA package. templatePath: " + templatePath + ", templateName: " + templateName);
- String templateFileFullPath = templatePath + File.separator + templateName + "." + ImageFormat.OVA.getFileExtension();
- File templateFile = new File(templateFileFullPath);
- Script command = new Script("tar", processTimeout, s_logger);
- command.add("--no-same-owner");
- command.add("--no-same-permissions");
- command.add("-xf", templateFileFullPath);
- command.setWorkDir(templateFile.getParent());
- String result = command.execute();
- if (result != null) {
- s_logger.info("failed to untar OVA package due to " + result + ". templatePath: " + templatePath + ", templateName: " + templateName);
- throw new InternalErrorException("failed to untar OVA package");
+ String templateFileFullPath = unpackOva(templatePath, templateName, processTimeout);
+
+ setFileSystemAccessRights(templatePath);
+
+ FormatInfo info = createFormatInfo(templatePath, templateName, templateFilePath, templateFileFullPath);
+
+ // The intention is to use the ova file as is for deployment and use processing result only for
+ // - property assessment and
+ // - reconsiliation of
+ // - - disks,
+ // - - networks and
+ // - - compute dimensions.
+ return info;
+ }
+
+ private FormatInfo createFormatInfo(String templatePath, String templateName, String templateFilePath, String templateFileFullPath) throws InternalErrorException {
+ FormatInfo info = new FormatInfo();
+ info.format = ImageFormat.OVA;
+ info.filename = templateName + "." + ImageFormat.OVA.getFileExtension();
+ info.size = _storage.getSize(templateFilePath);
+ info.virtualSize = getTemplateVirtualSize(templatePath, info.filename);
+ validateOva(templateFileFullPath, info);
+
+ return info;
+ }
+
+ /**
+ * side effect; properties are added to the info
+ *
+ * @throws InternalErrorException on an invalid ova contents
+ */
+ private void validateOva(String templateFileFullPath, FormatInfo info) throws InternalErrorException {
+ String ovfFilePath = getOVFFilePath(templateFileFullPath);
+ OVFHelper ovfHelper = new OVFHelper();
+ Document doc = ovfHelper.getDocumentFromFile(ovfFilePath);
+
+ List<DatadiskTO> disks = ovfHelper.getOVFVolumeInfoFromFile(ovfFilePath, doc);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("Found %d disks in template %s", CollectionUtils.isNotEmpty(disks) ? disks.size() : 0, ovfFilePath));
+ }
+ if (CollectionUtils.isNotEmpty(disks)) {
+ info.disks = disks;
+ }
+
+ List<NetworkPrerequisiteTO> nets = ovfHelper.getNetPrerequisitesFromDocument(doc);
+ if (CollectionUtils.isNotEmpty(nets)) {
+ LOGGER.info("Found " + nets.size() + " prerequisite networks");
+ info.networks = nets;
+ } else if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("no net prerequisites found in template %s", ovfFilePath));
+ }
+
+ List<OVFPropertyTO> ovfProperties = ovfHelper.getConfigurableOVFPropertiesFromDocument(doc);
+ if (CollectionUtils.isNotEmpty(ovfProperties)) {
+ LOGGER.info("Found " + ovfProperties.size() + " configurable OVF properties");
+ info.ovfProperties = ovfProperties;
+ } else if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("no ovf properties found in template %s", ovfFilePath));
}
- command = new Script("chmod", 0, s_logger);
+ OVFVirtualHardwareSectionTO hardwareSection = ovfHelper.getVirtualHardwareSectionFromDocument(doc);
+ List<OVFConfigurationTO> configurations = hardwareSection.getConfigurations();
+ if (CollectionUtils.isNotEmpty(configurations)) {
+ LOGGER.info("Found " + configurations.size() + " deployment option configurations");
+ }
+ List<OVFVirtualHardwareItemTO> hardwareItems = hardwareSection.getCommonHardwareItems();
+ if (CollectionUtils.isNotEmpty(hardwareItems)) {
+ LOGGER.info("Found " + hardwareItems.size() + " virtual hardware items");
+ }
+ info.hardwareSection = hardwareSection;
+
+ // FR37 TODO add any user queries that are required for this OVA
Review comment:
todo? configurations and property values are done (eula considdered implicit?) are there any we have not considered yet?
##########
File path: engine/orchestration/src/main/java/com/cloud/vm/VirtualMachineManagerImpl.java
##########
@@ -1136,6 +1138,7 @@ public void orchestrateStart(final String vmUuid, final Map<VirtualMachineProfil
try {
_networkMgr.prepare(vmProfile, new DeployDestination(dest.getDataCenter(), dest.getPod(), null, null, dest.getStorageForDisks()), ctx);
if (vm.getHypervisorType() != HypervisorType.BareMetal) {
+ // FR37 TODO do not create a copy volume task for deploy as is on vmware ?
Review comment:
this comment can be removed (done)
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/StartCommandExecutor.java
##########
@@ -0,0 +1,2247 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.hypervisor.vmware.resource;
+
+import java.io.File;
+import java.rmi.RemoteException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.cloud.agent.api.to.DataTO;
+import com.vmware.vim25.VirtualDiskFlatVer2BackingInfo;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.storage.configdrive.ConfigDrive;
+import org.apache.cloudstack.storage.to.TemplateObjectTO;
+import org.apache.cloudstack.storage.to.VolumeObjectTO;
+import org.apache.cloudstack.utils.volume.VirtualMachineDiskInfo;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.commons.lang.ArrayUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.log4j.Logger;
+
+import com.cloud.agent.api.Command;
+import com.cloud.agent.api.StartAnswer;
+import com.cloud.agent.api.StartCommand;
+import com.cloud.agent.api.storage.OVFPropertyTO;
+import com.cloud.agent.api.to.DataStoreTO;
+import com.cloud.agent.api.to.DiskTO;
+import com.cloud.agent.api.to.NfsTO;
+import com.cloud.agent.api.to.NicTO;
+import com.cloud.agent.api.to.VirtualMachineTO;
+import com.cloud.configuration.Resource;
+import com.cloud.hypervisor.vmware.VmwareResourceException;
+import com.cloud.hypervisor.vmware.manager.VmwareManager;
+import com.cloud.hypervisor.vmware.mo.CustomFieldConstants;
+import com.cloud.hypervisor.vmware.mo.DatacenterMO;
+import com.cloud.hypervisor.vmware.mo.DatastoreFile;
+import com.cloud.hypervisor.vmware.mo.DatastoreMO;
+import com.cloud.hypervisor.vmware.mo.DiskControllerType;
+import com.cloud.hypervisor.vmware.mo.HostMO;
+import com.cloud.hypervisor.vmware.mo.HypervisorHostHelper;
+import com.cloud.hypervisor.vmware.mo.TaskMO;
+import com.cloud.hypervisor.vmware.mo.VirtualEthernetCardType;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineDiskInfoBuilder;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineMO;
+import com.cloud.hypervisor.vmware.mo.VmwareHypervisorHost;
+import com.cloud.hypervisor.vmware.util.VmwareContext;
+import com.cloud.hypervisor.vmware.util.VmwareHelper;
+import com.cloud.network.Networks;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Volume;
+import com.cloud.storage.resource.VmwareStorageLayoutHelper;
+import com.cloud.storage.resource.VmwareStorageProcessor;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nicira.nvp.plugin.NiciraNvpApiVersion;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.VmDetailConstants;
+import com.vmware.vim25.BoolPolicy;
+import com.vmware.vim25.DVPortConfigInfo;
+import com.vmware.vim25.DVPortConfigSpec;
+import com.vmware.vim25.DasVmPriority;
+import com.vmware.vim25.DistributedVirtualPort;
+import com.vmware.vim25.DistributedVirtualSwitchPortConnection;
+import com.vmware.vim25.DistributedVirtualSwitchPortCriteria;
+import com.vmware.vim25.ManagedObjectReference;
+import com.vmware.vim25.OptionValue;
+import com.vmware.vim25.VMwareDVSPortSetting;
+import com.vmware.vim25.VirtualDevice;
+import com.vmware.vim25.VirtualDeviceBackingInfo;
+import com.vmware.vim25.VirtualDeviceConfigSpec;
+import com.vmware.vim25.VirtualDeviceConfigSpecOperation;
+import com.vmware.vim25.VirtualDisk;
+import com.vmware.vim25.VirtualEthernetCard;
+import com.vmware.vim25.VirtualEthernetCardDistributedVirtualPortBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardNetworkBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardOpaqueNetworkBackingInfo;
+import com.vmware.vim25.VirtualMachineBootOptions;
+import com.vmware.vim25.VirtualMachineConfigSpec;
+import com.vmware.vim25.VirtualMachineFileInfo;
+import com.vmware.vim25.VirtualMachineFileLayoutEx;
+import com.vmware.vim25.VirtualMachineGuestOsIdentifier;
+import com.vmware.vim25.VirtualUSBController;
+import com.vmware.vim25.VmConfigInfo;
+import com.vmware.vim25.VmwareDistributedVirtualSwitchVlanIdSpec;
+
+class StartCommandExecutor {
+ private static final Logger LOGGER = Logger.getLogger(StartCommandExecutor.class);
+
+ private final VmwareResource vmwareResource;
+
+ public StartCommandExecutor(VmwareResource vmwareResource) {
+ this.vmwareResource = vmwareResource;
+ }
+
+ // FR37 the monster blob god method: reduce to 320 so far and counting
Review comment:
```suggestion
// TODO this is a monster blob god method: reduced to 320 lines so far and still needs chopping up
```
##########
File path: engine/orchestration/src/main/java/org/apache/cloudstack/engine/orchestration/VolumeOrchestrator.java
##########
@@ -1174,6 +1221,10 @@ public void prepareForMigration(VirtualMachineProfile vm, DeployDestination dest
private List<VolumeTask> getTasks(List<VolumeVO> vols, Map<Volume, StoragePool> destVols, VirtualMachineProfile vm) throws StorageUnavailableException {
boolean recreate = RecreatableSystemVmEnabled.value();
List<VolumeTask> tasks = new ArrayList<VolumeTask>();
+ // FR37 TODO: is it this easy?
+// if (vm.getTemplate().isDeployAsIs()) {
+// return tasks;
+// }
Review comment:
should be gone i think
```suggestion
```
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/guru/VmwareVmImplementer.java
##########
@@ -184,18 +192,66 @@ VirtualMachineTO implement(VirtualMachineProfile vm, VirtualMachineTO to, long c
}
List<OVFPropertyTO> ovfProperties = getOvfPropertyList(vm, details);
-
handleOvfProperties(vm, to, details, ovfProperties);
+ // FR37 TODO add required nics here or let the start executor copy them from the base template?
Review comment:
isn't this already handled somewhere?
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/StartCommandExecutor.java
##########
@@ -0,0 +1,2247 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.hypervisor.vmware.resource;
+
+import java.io.File;
+import java.rmi.RemoteException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.cloud.agent.api.to.DataTO;
+import com.vmware.vim25.VirtualDiskFlatVer2BackingInfo;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.storage.configdrive.ConfigDrive;
+import org.apache.cloudstack.storage.to.TemplateObjectTO;
+import org.apache.cloudstack.storage.to.VolumeObjectTO;
+import org.apache.cloudstack.utils.volume.VirtualMachineDiskInfo;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.commons.lang.ArrayUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.log4j.Logger;
+
+import com.cloud.agent.api.Command;
+import com.cloud.agent.api.StartAnswer;
+import com.cloud.agent.api.StartCommand;
+import com.cloud.agent.api.storage.OVFPropertyTO;
+import com.cloud.agent.api.to.DataStoreTO;
+import com.cloud.agent.api.to.DiskTO;
+import com.cloud.agent.api.to.NfsTO;
+import com.cloud.agent.api.to.NicTO;
+import com.cloud.agent.api.to.VirtualMachineTO;
+import com.cloud.configuration.Resource;
+import com.cloud.hypervisor.vmware.VmwareResourceException;
+import com.cloud.hypervisor.vmware.manager.VmwareManager;
+import com.cloud.hypervisor.vmware.mo.CustomFieldConstants;
+import com.cloud.hypervisor.vmware.mo.DatacenterMO;
+import com.cloud.hypervisor.vmware.mo.DatastoreFile;
+import com.cloud.hypervisor.vmware.mo.DatastoreMO;
+import com.cloud.hypervisor.vmware.mo.DiskControllerType;
+import com.cloud.hypervisor.vmware.mo.HostMO;
+import com.cloud.hypervisor.vmware.mo.HypervisorHostHelper;
+import com.cloud.hypervisor.vmware.mo.TaskMO;
+import com.cloud.hypervisor.vmware.mo.VirtualEthernetCardType;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineDiskInfoBuilder;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineMO;
+import com.cloud.hypervisor.vmware.mo.VmwareHypervisorHost;
+import com.cloud.hypervisor.vmware.util.VmwareContext;
+import com.cloud.hypervisor.vmware.util.VmwareHelper;
+import com.cloud.network.Networks;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Volume;
+import com.cloud.storage.resource.VmwareStorageLayoutHelper;
+import com.cloud.storage.resource.VmwareStorageProcessor;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nicira.nvp.plugin.NiciraNvpApiVersion;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.VmDetailConstants;
+import com.vmware.vim25.BoolPolicy;
+import com.vmware.vim25.DVPortConfigInfo;
+import com.vmware.vim25.DVPortConfigSpec;
+import com.vmware.vim25.DasVmPriority;
+import com.vmware.vim25.DistributedVirtualPort;
+import com.vmware.vim25.DistributedVirtualSwitchPortConnection;
+import com.vmware.vim25.DistributedVirtualSwitchPortCriteria;
+import com.vmware.vim25.ManagedObjectReference;
+import com.vmware.vim25.OptionValue;
+import com.vmware.vim25.VMwareDVSPortSetting;
+import com.vmware.vim25.VirtualDevice;
+import com.vmware.vim25.VirtualDeviceBackingInfo;
+import com.vmware.vim25.VirtualDeviceConfigSpec;
+import com.vmware.vim25.VirtualDeviceConfigSpecOperation;
+import com.vmware.vim25.VirtualDisk;
+import com.vmware.vim25.VirtualEthernetCard;
+import com.vmware.vim25.VirtualEthernetCardDistributedVirtualPortBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardNetworkBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardOpaqueNetworkBackingInfo;
+import com.vmware.vim25.VirtualMachineBootOptions;
+import com.vmware.vim25.VirtualMachineConfigSpec;
+import com.vmware.vim25.VirtualMachineFileInfo;
+import com.vmware.vim25.VirtualMachineFileLayoutEx;
+import com.vmware.vim25.VirtualMachineGuestOsIdentifier;
+import com.vmware.vim25.VirtualUSBController;
+import com.vmware.vim25.VmConfigInfo;
+import com.vmware.vim25.VmwareDistributedVirtualSwitchVlanIdSpec;
+
+class StartCommandExecutor {
+ private static final Logger LOGGER = Logger.getLogger(StartCommandExecutor.class);
+
+ private final VmwareResource vmwareResource;
+
+ public StartCommandExecutor(VmwareResource vmwareResource) {
+ this.vmwareResource = vmwareResource;
+ }
+
+ // FR37 the monster blob god method: reduce to 320 so far and counting
+ protected StartAnswer execute(StartCommand cmd) {
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Executing resource StartCommand: " + vmwareResource.getGson().toJson(cmd));
+ }
+
+ VirtualMachineTO vmSpec = cmd.getVirtualMachine();
+
+ VirtualMachineData existingVm = null;
+
+ Pair<String, String> names = composeVmNames(vmSpec);
+ String vmInternalCSName = names.first();
+ String vmNameOnVcenter = names.second();
+
+ DiskTO rootDiskTO = null;
+
+ Pair<String, String> controllerInfo = getDiskControllerInfo(vmSpec);
+
+ Boolean systemVm = vmSpec.getType().isUsedBySystem();
+
+ VmwareContext context = vmwareResource.getServiceContext();
+ DatacenterMO dcMo = null;
+ try {
+ VmwareManager mgr = context.getStockObject(VmwareManager.CONTEXT_STOCK_NAME);
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(context);
+ dcMo = new DatacenterMO(hyperHost.getContext(), hyperHost.getHyperHostDatacenter());
+
+ // checkIfVmExistsInVcenter(vmInternalCSName, vmNameOnVcenter, dcMo);
+ // FR37 - We expect VM to be already cloned and available at this point
+ VirtualMachineMO vmMo = dcMo.findVm(vmInternalCSName);
+ if (vmMo == null) {
+ vmMo = dcMo.findVm(vmNameOnVcenter);
+ }
+
+ DiskTO[] specDisks = vmSpec.getDisks();
+ boolean installAsIs = StringUtils.isNotEmpty(vmSpec.getTemplateLocation());
+ // FR37 if startcommand contains enough info: a template url/-location and flag; deploy OVA as is
+ if (vmMo == null && installAsIs) {
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("deploying OVA from %s as is", vmSpec.getTemplateLocation()));
+ }
+ getStorageProcessor().cloneVMFromTemplate(vmSpec.getTemplateName(), vmInternalCSName, vmSpec.getTemplatePrimaryStoreUuid());
+ vmMo = dcMo.findVm(vmInternalCSName);
+ mapSpecDisksToClonedDisks(vmMo, vmInternalCSName, specDisks);
+ }
+
+ // VM may not have been on the same host, relocate to expected host
+ if (vmMo != null) {
+ vmMo.relocate(hyperHost.getMor());
+ // Get updated MO
+ vmMo = hyperHost.findVmOnHyperHost(vmMo.getVmName());
+ }
+
+ String guestOsId = translateGuestOsIdentifier(vmSpec.getArch(), vmSpec.getOs(), vmSpec.getPlatformEmulator()).value();
+ DiskTO[] disks = validateDisks(specDisks);
+ NicTO[] nics = vmSpec.getNics();
+
+ // FIXME: disks logic here, why is disks/volumes during copy not set with pool ID?
+ // FR37 TODO if deployasis a new VM no datastores are known (yet) and we need to get the data store from the tvmspec content library / template location
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails = inferDatastoreDetailsFromDiskInfo(hyperHost, context, disks, cmd);
+ if ((dataStoresDetails == null) || (dataStoresDetails.isEmpty())) {
+ String msg = "Unable to locate datastore details of the volumes to be attached";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ // FR37 - this may not be necessary the cloned VM will have disks and knowledge of datastore paths/location?
+ // throw new Exception(msg);
+ }
+
+ // FR37 - this may need checking, if the first datastore is the right one - ideally it should be datastore where first disk is hosted
+ DatastoreMO dsRootVolumeIsOn = null; //
+ if (! installAsIs) {
+ dsRootVolumeIsOn = getDatastoreThatRootDiskIsOn(dataStoresDetails, disks);
+
+ if (dsRootVolumeIsOn == null) {
+ String msg = "Unable to locate datastore details of root volume";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ throw new VmwareResourceException(msg);
+ }
+ }
+
+ VirtualMachineDiskInfoBuilder diskInfoBuilder = null;
+ VirtualDevice[] nicDevices = null;
+ DiskControllerType systemVmScsiControllerType = DiskControllerType.lsilogic;
+ int firstScsiControllerBusNum = 0;
+ int numScsiControllerForSystemVm = 1;
+ boolean hasSnapshot = false;
+ if (vmMo != null) {
+ PrepareRunningVMForConfiguration prepareRunningVMForConfiguration = new PrepareRunningVMForConfiguration(vmInternalCSName, controllerInfo, systemVm, vmMo,
+ systemVmScsiControllerType, firstScsiControllerBusNum, numScsiControllerForSystemVm);
+ prepareRunningVMForConfiguration.invoke();
+ diskInfoBuilder = prepareRunningVMForConfiguration.getDiskInfoBuilder();
+ nicDevices = prepareRunningVMForConfiguration.getNicDevices();
+ hasSnapshot = prepareRunningVMForConfiguration.isHasSnapshot();
+ } else {
+ ManagedObjectReference morDc = hyperHost.getHyperHostDatacenter();
+ assert (morDc != null);
+
+ vmMo = hyperHost.findVmOnPeerHyperHost(vmInternalCSName);
+ if (vmMo != null) {
+ VirtualMachineRecycler virtualMachineRecycler = new VirtualMachineRecycler(vmInternalCSName, controllerInfo, systemVm, hyperHost, vmMo,
+ systemVmScsiControllerType, firstScsiControllerBusNum, numScsiControllerForSystemVm);
+ virtualMachineRecycler.invoke();
+ diskInfoBuilder = virtualMachineRecycler.getDiskInfoBuilder();
+ hasSnapshot = virtualMachineRecycler.isHasSnapshot();
+ nicDevices = virtualMachineRecycler.getNicDevices();
+ } else {
+ // FR37 we just didn't find a VM by name 'vmInternalCSName', so why is this here?
Review comment:
this needs addressing
##########
File path: core/src/main/java/com/cloud/storage/template/OVAProcessor.java
##########
@@ -53,73 +63,140 @@ public FormatInfo process(String templatePath, ImageFormat format, String templa
@Override
public FormatInfo process(String templatePath, ImageFormat format, String templateName, long processTimeout) throws InternalErrorException {
- if (format != null) {
- if (s_logger.isInfoEnabled()) {
- s_logger.info("We currently don't handle conversion from " + format + " to OVA.");
- }
+ if (! conversionChecks(format)){
return null;
}
- s_logger.info("Template processing. templatePath: " + templatePath + ", templateName: " + templateName);
+ LOGGER.info("Template processing. templatePath: " + templatePath + ", templateName: " + templateName);
String templateFilePath = templatePath + File.separator + templateName + "." + ImageFormat.OVA.getFileExtension();
if (!_storage.exists(templateFilePath)) {
- if (s_logger.isInfoEnabled()) {
- s_logger.info("Unable to find the vmware template file: " + templateFilePath);
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Unable to find the vmware template file: " + templateFilePath);
}
return null;
}
- s_logger.info("Template processing - untar OVA package. templatePath: " + templatePath + ", templateName: " + templateName);
- String templateFileFullPath = templatePath + File.separator + templateName + "." + ImageFormat.OVA.getFileExtension();
- File templateFile = new File(templateFileFullPath);
- Script command = new Script("tar", processTimeout, s_logger);
- command.add("--no-same-owner");
- command.add("--no-same-permissions");
- command.add("-xf", templateFileFullPath);
- command.setWorkDir(templateFile.getParent());
- String result = command.execute();
- if (result != null) {
- s_logger.info("failed to untar OVA package due to " + result + ". templatePath: " + templatePath + ", templateName: " + templateName);
- throw new InternalErrorException("failed to untar OVA package");
+ String templateFileFullPath = unpackOva(templatePath, templateName, processTimeout);
+
+ setFileSystemAccessRights(templatePath);
+
+ FormatInfo info = createFormatInfo(templatePath, templateName, templateFilePath, templateFileFullPath);
+
+ // The intention is to use the ova file as is for deployment and use processing result only for
+ // - property assessment and
+ // - reconsiliation of
+ // - - disks,
+ // - - networks and
+ // - - compute dimensions.
Review comment:
```suggestion
// The ova will be deploying as is and the info created here, is to do
// - property assessment and
// - reconsiliation of
// - - disks,
// - - networks and
// - - compute dimensions.
```
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/guru/VMwareGuru.java
##########
@@ -261,8 +264,13 @@ long getClusterId(long vmId) {
return new Pair<Boolean, Long>(Boolean.FALSE, new Long(hostId));
}
- if (destData.getObjectType() == DataObjectType.VOLUME && destStoreTO.getRole() == DataStoreRole.Primary && srcData.getObjectType() == DataObjectType.TEMPLATE
- && srcStoreTO.getRole() == DataStoreRole.Primary) {
+ if (destData.getObjectType() == DataObjectType.VOLUME && destStoreTO.getRole() == DataStoreRole.Primary
+ && srcData.getObjectType() == DataObjectType.TEMPLATE && srcStoreTO.getRole() == DataStoreRole.Primary) {
+ needDelegation = false;
+ } else
+ // FR37 TODO remove or as possible improvement: check if the template is meant to be deployed as is and delegate if it isn't
Review comment:
So the decision here is: do we delegate vApp OVF processing to the SSVM or do we do it in the MS/resouce plugin
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/StartCommandExecutor.java
##########
@@ -0,0 +1,2247 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.hypervisor.vmware.resource;
+
+import java.io.File;
+import java.rmi.RemoteException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.cloud.agent.api.to.DataTO;
+import com.vmware.vim25.VirtualDiskFlatVer2BackingInfo;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.storage.configdrive.ConfigDrive;
+import org.apache.cloudstack.storage.to.TemplateObjectTO;
+import org.apache.cloudstack.storage.to.VolumeObjectTO;
+import org.apache.cloudstack.utils.volume.VirtualMachineDiskInfo;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.commons.lang.ArrayUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.log4j.Logger;
+
+import com.cloud.agent.api.Command;
+import com.cloud.agent.api.StartAnswer;
+import com.cloud.agent.api.StartCommand;
+import com.cloud.agent.api.storage.OVFPropertyTO;
+import com.cloud.agent.api.to.DataStoreTO;
+import com.cloud.agent.api.to.DiskTO;
+import com.cloud.agent.api.to.NfsTO;
+import com.cloud.agent.api.to.NicTO;
+import com.cloud.agent.api.to.VirtualMachineTO;
+import com.cloud.configuration.Resource;
+import com.cloud.hypervisor.vmware.VmwareResourceException;
+import com.cloud.hypervisor.vmware.manager.VmwareManager;
+import com.cloud.hypervisor.vmware.mo.CustomFieldConstants;
+import com.cloud.hypervisor.vmware.mo.DatacenterMO;
+import com.cloud.hypervisor.vmware.mo.DatastoreFile;
+import com.cloud.hypervisor.vmware.mo.DatastoreMO;
+import com.cloud.hypervisor.vmware.mo.DiskControllerType;
+import com.cloud.hypervisor.vmware.mo.HostMO;
+import com.cloud.hypervisor.vmware.mo.HypervisorHostHelper;
+import com.cloud.hypervisor.vmware.mo.TaskMO;
+import com.cloud.hypervisor.vmware.mo.VirtualEthernetCardType;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineDiskInfoBuilder;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineMO;
+import com.cloud.hypervisor.vmware.mo.VmwareHypervisorHost;
+import com.cloud.hypervisor.vmware.util.VmwareContext;
+import com.cloud.hypervisor.vmware.util.VmwareHelper;
+import com.cloud.network.Networks;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Volume;
+import com.cloud.storage.resource.VmwareStorageLayoutHelper;
+import com.cloud.storage.resource.VmwareStorageProcessor;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nicira.nvp.plugin.NiciraNvpApiVersion;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.VmDetailConstants;
+import com.vmware.vim25.BoolPolicy;
+import com.vmware.vim25.DVPortConfigInfo;
+import com.vmware.vim25.DVPortConfigSpec;
+import com.vmware.vim25.DasVmPriority;
+import com.vmware.vim25.DistributedVirtualPort;
+import com.vmware.vim25.DistributedVirtualSwitchPortConnection;
+import com.vmware.vim25.DistributedVirtualSwitchPortCriteria;
+import com.vmware.vim25.ManagedObjectReference;
+import com.vmware.vim25.OptionValue;
+import com.vmware.vim25.VMwareDVSPortSetting;
+import com.vmware.vim25.VirtualDevice;
+import com.vmware.vim25.VirtualDeviceBackingInfo;
+import com.vmware.vim25.VirtualDeviceConfigSpec;
+import com.vmware.vim25.VirtualDeviceConfigSpecOperation;
+import com.vmware.vim25.VirtualDisk;
+import com.vmware.vim25.VirtualEthernetCard;
+import com.vmware.vim25.VirtualEthernetCardDistributedVirtualPortBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardNetworkBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardOpaqueNetworkBackingInfo;
+import com.vmware.vim25.VirtualMachineBootOptions;
+import com.vmware.vim25.VirtualMachineConfigSpec;
+import com.vmware.vim25.VirtualMachineFileInfo;
+import com.vmware.vim25.VirtualMachineFileLayoutEx;
+import com.vmware.vim25.VirtualMachineGuestOsIdentifier;
+import com.vmware.vim25.VirtualUSBController;
+import com.vmware.vim25.VmConfigInfo;
+import com.vmware.vim25.VmwareDistributedVirtualSwitchVlanIdSpec;
+
+class StartCommandExecutor {
+ private static final Logger LOGGER = Logger.getLogger(StartCommandExecutor.class);
+
+ private final VmwareResource vmwareResource;
+
+ public StartCommandExecutor(VmwareResource vmwareResource) {
+ this.vmwareResource = vmwareResource;
+ }
+
+ // FR37 the monster blob god method: reduce to 320 so far and counting
+ protected StartAnswer execute(StartCommand cmd) {
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Executing resource StartCommand: " + vmwareResource.getGson().toJson(cmd));
+ }
+
+ VirtualMachineTO vmSpec = cmd.getVirtualMachine();
+
+ VirtualMachineData existingVm = null;
+
+ Pair<String, String> names = composeVmNames(vmSpec);
+ String vmInternalCSName = names.first();
+ String vmNameOnVcenter = names.second();
+
+ DiskTO rootDiskTO = null;
+
+ Pair<String, String> controllerInfo = getDiskControllerInfo(vmSpec);
+
+ Boolean systemVm = vmSpec.getType().isUsedBySystem();
+
+ VmwareContext context = vmwareResource.getServiceContext();
+ DatacenterMO dcMo = null;
+ try {
+ VmwareManager mgr = context.getStockObject(VmwareManager.CONTEXT_STOCK_NAME);
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(context);
+ dcMo = new DatacenterMO(hyperHost.getContext(), hyperHost.getHyperHostDatacenter());
+
+ // checkIfVmExistsInVcenter(vmInternalCSName, vmNameOnVcenter, dcMo);
+ // FR37 - We expect VM to be already cloned and available at this point
+ VirtualMachineMO vmMo = dcMo.findVm(vmInternalCSName);
+ if (vmMo == null) {
+ vmMo = dcMo.findVm(vmNameOnVcenter);
+ }
+
+ DiskTO[] specDisks = vmSpec.getDisks();
+ boolean installAsIs = StringUtils.isNotEmpty(vmSpec.getTemplateLocation());
+ // FR37 if startcommand contains enough info: a template url/-location and flag; deploy OVA as is
Review comment:
```suggestion
// If startcommand contains enough info: a template url/-location and flag; deploy OVA as is
// we create the VM as clone from the base image directly instead of copying disks
// from different templates
```
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/StartCommandExecutor.java
##########
@@ -0,0 +1,2247 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.hypervisor.vmware.resource;
+
+import java.io.File;
+import java.rmi.RemoteException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.cloud.agent.api.to.DataTO;
+import com.vmware.vim25.VirtualDiskFlatVer2BackingInfo;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.storage.configdrive.ConfigDrive;
+import org.apache.cloudstack.storage.to.TemplateObjectTO;
+import org.apache.cloudstack.storage.to.VolumeObjectTO;
+import org.apache.cloudstack.utils.volume.VirtualMachineDiskInfo;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.commons.lang.ArrayUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.log4j.Logger;
+
+import com.cloud.agent.api.Command;
+import com.cloud.agent.api.StartAnswer;
+import com.cloud.agent.api.StartCommand;
+import com.cloud.agent.api.storage.OVFPropertyTO;
+import com.cloud.agent.api.to.DataStoreTO;
+import com.cloud.agent.api.to.DiskTO;
+import com.cloud.agent.api.to.NfsTO;
+import com.cloud.agent.api.to.NicTO;
+import com.cloud.agent.api.to.VirtualMachineTO;
+import com.cloud.configuration.Resource;
+import com.cloud.hypervisor.vmware.VmwareResourceException;
+import com.cloud.hypervisor.vmware.manager.VmwareManager;
+import com.cloud.hypervisor.vmware.mo.CustomFieldConstants;
+import com.cloud.hypervisor.vmware.mo.DatacenterMO;
+import com.cloud.hypervisor.vmware.mo.DatastoreFile;
+import com.cloud.hypervisor.vmware.mo.DatastoreMO;
+import com.cloud.hypervisor.vmware.mo.DiskControllerType;
+import com.cloud.hypervisor.vmware.mo.HostMO;
+import com.cloud.hypervisor.vmware.mo.HypervisorHostHelper;
+import com.cloud.hypervisor.vmware.mo.TaskMO;
+import com.cloud.hypervisor.vmware.mo.VirtualEthernetCardType;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineDiskInfoBuilder;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineMO;
+import com.cloud.hypervisor.vmware.mo.VmwareHypervisorHost;
+import com.cloud.hypervisor.vmware.util.VmwareContext;
+import com.cloud.hypervisor.vmware.util.VmwareHelper;
+import com.cloud.network.Networks;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Volume;
+import com.cloud.storage.resource.VmwareStorageLayoutHelper;
+import com.cloud.storage.resource.VmwareStorageProcessor;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nicira.nvp.plugin.NiciraNvpApiVersion;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.VmDetailConstants;
+import com.vmware.vim25.BoolPolicy;
+import com.vmware.vim25.DVPortConfigInfo;
+import com.vmware.vim25.DVPortConfigSpec;
+import com.vmware.vim25.DasVmPriority;
+import com.vmware.vim25.DistributedVirtualPort;
+import com.vmware.vim25.DistributedVirtualSwitchPortConnection;
+import com.vmware.vim25.DistributedVirtualSwitchPortCriteria;
+import com.vmware.vim25.ManagedObjectReference;
+import com.vmware.vim25.OptionValue;
+import com.vmware.vim25.VMwareDVSPortSetting;
+import com.vmware.vim25.VirtualDevice;
+import com.vmware.vim25.VirtualDeviceBackingInfo;
+import com.vmware.vim25.VirtualDeviceConfigSpec;
+import com.vmware.vim25.VirtualDeviceConfigSpecOperation;
+import com.vmware.vim25.VirtualDisk;
+import com.vmware.vim25.VirtualEthernetCard;
+import com.vmware.vim25.VirtualEthernetCardDistributedVirtualPortBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardNetworkBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardOpaqueNetworkBackingInfo;
+import com.vmware.vim25.VirtualMachineBootOptions;
+import com.vmware.vim25.VirtualMachineConfigSpec;
+import com.vmware.vim25.VirtualMachineFileInfo;
+import com.vmware.vim25.VirtualMachineFileLayoutEx;
+import com.vmware.vim25.VirtualMachineGuestOsIdentifier;
+import com.vmware.vim25.VirtualUSBController;
+import com.vmware.vim25.VmConfigInfo;
+import com.vmware.vim25.VmwareDistributedVirtualSwitchVlanIdSpec;
+
+class StartCommandExecutor {
+ private static final Logger LOGGER = Logger.getLogger(StartCommandExecutor.class);
+
+ private final VmwareResource vmwareResource;
+
+ public StartCommandExecutor(VmwareResource vmwareResource) {
+ this.vmwareResource = vmwareResource;
+ }
+
+ // FR37 the monster blob god method: reduce to 320 so far and counting
+ protected StartAnswer execute(StartCommand cmd) {
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Executing resource StartCommand: " + vmwareResource.getGson().toJson(cmd));
+ }
+
+ VirtualMachineTO vmSpec = cmd.getVirtualMachine();
+
+ VirtualMachineData existingVm = null;
+
+ Pair<String, String> names = composeVmNames(vmSpec);
+ String vmInternalCSName = names.first();
+ String vmNameOnVcenter = names.second();
+
+ DiskTO rootDiskTO = null;
+
+ Pair<String, String> controllerInfo = getDiskControllerInfo(vmSpec);
+
+ Boolean systemVm = vmSpec.getType().isUsedBySystem();
+
+ VmwareContext context = vmwareResource.getServiceContext();
+ DatacenterMO dcMo = null;
+ try {
+ VmwareManager mgr = context.getStockObject(VmwareManager.CONTEXT_STOCK_NAME);
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(context);
+ dcMo = new DatacenterMO(hyperHost.getContext(), hyperHost.getHyperHostDatacenter());
+
+ // checkIfVmExistsInVcenter(vmInternalCSName, vmNameOnVcenter, dcMo);
+ // FR37 - We expect VM to be already cloned and available at this point
+ VirtualMachineMO vmMo = dcMo.findVm(vmInternalCSName);
+ if (vmMo == null) {
+ vmMo = dcMo.findVm(vmNameOnVcenter);
+ }
+
+ DiskTO[] specDisks = vmSpec.getDisks();
+ boolean installAsIs = StringUtils.isNotEmpty(vmSpec.getTemplateLocation());
+ // FR37 if startcommand contains enough info: a template url/-location and flag; deploy OVA as is
+ if (vmMo == null && installAsIs) {
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("deploying OVA from %s as is", vmSpec.getTemplateLocation()));
+ }
+ getStorageProcessor().cloneVMFromTemplate(vmSpec.getTemplateName(), vmInternalCSName, vmSpec.getTemplatePrimaryStoreUuid());
+ vmMo = dcMo.findVm(vmInternalCSName);
+ mapSpecDisksToClonedDisks(vmMo, vmInternalCSName, specDisks);
+ }
+
+ // VM may not have been on the same host, relocate to expected host
+ if (vmMo != null) {
+ vmMo.relocate(hyperHost.getMor());
+ // Get updated MO
+ vmMo = hyperHost.findVmOnHyperHost(vmMo.getVmName());
+ }
+
+ String guestOsId = translateGuestOsIdentifier(vmSpec.getArch(), vmSpec.getOs(), vmSpec.getPlatformEmulator()).value();
+ DiskTO[] disks = validateDisks(specDisks);
+ NicTO[] nics = vmSpec.getNics();
+
+ // FIXME: disks logic here, why is disks/volumes during copy not set with pool ID?
+ // FR37 TODO if deployasis a new VM no datastores are known (yet) and we need to get the data store from the tvmspec content library / template location
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails = inferDatastoreDetailsFromDiskInfo(hyperHost, context, disks, cmd);
+ if ((dataStoresDetails == null) || (dataStoresDetails.isEmpty())) {
+ String msg = "Unable to locate datastore details of the volumes to be attached";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ // FR37 - this may not be necessary the cloned VM will have disks and knowledge of datastore paths/location?
+ // throw new Exception(msg);
+ }
+
+ // FR37 - this may need checking, if the first datastore is the right one - ideally it should be datastore where first disk is hosted
+ DatastoreMO dsRootVolumeIsOn = null; //
+ if (! installAsIs) {
+ dsRootVolumeIsOn = getDatastoreThatRootDiskIsOn(dataStoresDetails, disks);
+
+ if (dsRootVolumeIsOn == null) {
+ String msg = "Unable to locate datastore details of root volume";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ throw new VmwareResourceException(msg);
+ }
+ }
+
+ VirtualMachineDiskInfoBuilder diskInfoBuilder = null;
+ VirtualDevice[] nicDevices = null;
+ DiskControllerType systemVmScsiControllerType = DiskControllerType.lsilogic;
+ int firstScsiControllerBusNum = 0;
+ int numScsiControllerForSystemVm = 1;
+ boolean hasSnapshot = false;
+ if (vmMo != null) {
+ PrepareRunningVMForConfiguration prepareRunningVMForConfiguration = new PrepareRunningVMForConfiguration(vmInternalCSName, controllerInfo, systemVm, vmMo,
+ systemVmScsiControllerType, firstScsiControllerBusNum, numScsiControllerForSystemVm);
+ prepareRunningVMForConfiguration.invoke();
+ diskInfoBuilder = prepareRunningVMForConfiguration.getDiskInfoBuilder();
+ nicDevices = prepareRunningVMForConfiguration.getNicDevices();
+ hasSnapshot = prepareRunningVMForConfiguration.isHasSnapshot();
+ } else {
+ ManagedObjectReference morDc = hyperHost.getHyperHostDatacenter();
+ assert (morDc != null);
+
+ vmMo = hyperHost.findVmOnPeerHyperHost(vmInternalCSName);
+ if (vmMo != null) {
+ VirtualMachineRecycler virtualMachineRecycler = new VirtualMachineRecycler(vmInternalCSName, controllerInfo, systemVm, hyperHost, vmMo,
+ systemVmScsiControllerType, firstScsiControllerBusNum, numScsiControllerForSystemVm);
+ virtualMachineRecycler.invoke();
+ diskInfoBuilder = virtualMachineRecycler.getDiskInfoBuilder();
+ hasSnapshot = virtualMachineRecycler.isHasSnapshot();
+ nicDevices = virtualMachineRecycler.getNicDevices();
+ } else {
+ // FR37 we just didn't find a VM by name 'vmInternalCSName', so why is this here?
+ existingVm = unregisterOnOtherClusterButHoldOnToOldVmData(vmInternalCSName, dcMo);
+
+ createNewVm(context, vmSpec, installAsIs, vmInternalCSName, vmNameOnVcenter, controllerInfo, systemVm, mgr, hyperHost, guestOsId, disks, dataStoresDetails,
+ dsRootVolumeIsOn);
+ }
+
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ if (vmMo == null) {
+ throw new Exception("Failed to find the newly create or relocated VM. vmName: " + vmInternalCSName);
+ }
+ }
+ // vmMo should now be a stopped VM on the intended host
+ // The number of disks changed must be 0 for install as is, as the VM is a clone
+ int disksChanges = !installAsIs ? disks.length : 0;
+ int totalChangeDevices = disksChanges + nics.length;
+ int hackDeviceCount = 0;
+ if (diskInfoBuilder != null) {
+ hackDeviceCount += diskInfoBuilder.getDiskCount();
+ }
+ hackDeviceCount += nicDevices == null ? 0 : nicDevices.length;
+ // vApp cdrom device
+ // HACK ALERT: ovf properties might not be the only or defining feature of vApps; needs checking
Review comment:
```suggestion
```
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/guru/VmwareVmImplementer.java
##########
@@ -317,19 +369,39 @@ private void handleOvfProperties(VirtualMachineProfile vm, VirtualMachineTO to,
}
}
+ private DiskTO getRootDiskTOFromVM(VirtualMachineProfile vm) {
+ DiskTO rootDiskTO;
+ List<DiskTO> rootDiskList;
+ rootDiskList = vm.getDisks().stream().filter(x -> x.getType() == Volume.Type.ROOT).collect(Collectors.toList());
+ if (rootDiskList.size() != 1) {
+ if (vm.getTemplate().isDeployAsIs()) { // FR37 dirty hack to avoid ISOs, the start command should have added a root disk to
+ rootDiskList = vm.getDisks().stream().filter(x -> x.getType() == null).collect(Collectors.toList());
+ if (rootDiskList.size() < 1) {
+ throw new CloudRuntimeException("Did not find a template to serve as root disk for VM " + vm.getHostName());
+ }
+ } else {
+ throw new CloudRuntimeException("Did not find only one root disk for VM " + vm.getHostName());
+ }
+ }
+ rootDiskTO = rootDiskList.get(0);
+ return rootDiskTO;
+ }
+
+ // TODO FR37 phase out ovf properties in favor of template details; propertyTO remains
private List<OVFPropertyTO> getOvfPropertyList(VirtualMachineProfile vm, Map<String, String> details) {
List<OVFPropertyTO> ovfProperties = new ArrayList<OVFPropertyTO>();
for (String detailKey : details.keySet()) {
- if (detailKey.startsWith(ApiConstants.OVF_PROPERTIES)) {
- String ovfPropKey = detailKey.replace(ApiConstants.OVF_PROPERTIES + "-", "");
- TemplateOVFPropertyVO templateOVFPropertyVO = templateOVFPropertiesDao.findByTemplateAndKey(vm.getTemplateId(), ovfPropKey);
- if (templateOVFPropertyVO == null) {
- LOG.warn(String.format("OVF property %s not found on template, discarding", ovfPropKey));
+ if (detailKey.startsWith(ApiConstants.ACS_PROPERTY)) {
+ OVFPropertyTO propertyTO = templateDetailsDao.findPropertyByTemplateAndKey(vm.getTemplateId(), detailKey);
+ String vmPropertyKey = detailKey.replace(ApiConstants.ACS_PROPERTY + "-", "");
+ if (propertyTO == null) {
+ LOGGER.warn(String.format("OVF property %s not found on template, discarding", vmPropertyKey));
continue;
}
- String ovfValue = details.get(detailKey);
- boolean isPassword = templateOVFPropertyVO.isPassword();
- OVFPropertyTO propertyTO = new OVFPropertyTO(ovfPropKey, ovfValue, isPassword);
+ // FR37 the key is without acs prefix (in the TO)
+ propertyTO.setKey(vmPropertyKey);
+ // FR37 if the UI send the whole json we should just copy it otherwise take the json from the template and set the value on it
Review comment:
```suggestion
```
not true anyway
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/guru/VmwareVmImplementer.java
##########
@@ -114,12 +115,19 @@ void setGlobalNestedVPerVMEnabled(Boolean globalNestedVPerVMEnabled) {
this.globalNestedVPerVMEnabled = globalNestedVPerVMEnabled;
}
- VirtualMachineTO implement(VirtualMachineProfile vm, VirtualMachineTO to, long clusterId) {
- to.setBootloader(VirtualMachineTemplate.BootloaderType.HVM);
-
+ VirtualMachineTO implement(VirtualMachineProfile vm, VirtualMachineTO to, long clusterId, boolean deployAsIs) {
+ to.setBootloader(VirtualMachineTemplate.BootloaderType.HVM);
+ deployAsIs |= vm.getTemplate().isDeployAsIs();
+ HostVO host = hostDao.findById(vm.getVirtualMachine().getHostId());
+ // FR37 if VmwareImplementAsIsAndReconsiliate add secondary storage or some other encoding of the OVA file to the start command,
+ // FR37 so the url for the original OVA can be used for deployment
+ if (deployAsIs) {
+ // FR37 we need to make sure the primary storage for the template is known and whether this is a new deployment
+ storeTemplateLocationInTO(vm, to, host.getId());
+ }
Review comment:
these comments can go?
```suggestion
// add base image or secondary storage path of the OVA file to the start command,
if (deployAsIs) {
// FR37 TODO we need to make sure the primary storage for the template is known and whether this is a new deployment, I.E. do we need to clone the base image (again)
storeTemplateLocationInTO(vm, to, host.getId());
}
```
that last comment will still need addressing
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/guru/VmwareVmImplementer.java
##########
@@ -317,19 +369,39 @@ private void handleOvfProperties(VirtualMachineProfile vm, VirtualMachineTO to,
}
}
+ private DiskTO getRootDiskTOFromVM(VirtualMachineProfile vm) {
+ DiskTO rootDiskTO;
+ List<DiskTO> rootDiskList;
+ rootDiskList = vm.getDisks().stream().filter(x -> x.getType() == Volume.Type.ROOT).collect(Collectors.toList());
+ if (rootDiskList.size() != 1) {
+ if (vm.getTemplate().isDeployAsIs()) { // FR37 dirty hack to avoid ISOs, the start command should have added a root disk to
+ rootDiskList = vm.getDisks().stream().filter(x -> x.getType() == null).collect(Collectors.toList());
+ if (rootDiskList.size() < 1) {
+ throw new CloudRuntimeException("Did not find a template to serve as root disk for VM " + vm.getHostName());
+ }
+ } else {
+ throw new CloudRuntimeException("Did not find only one root disk for VM " + vm.getHostName());
+ }
+ }
+ rootDiskTO = rootDiskList.get(0);
+ return rootDiskTO;
+ }
+
+ // TODO FR37 phase out ovf properties in favor of template details; propertyTO remains
Review comment:
i think this comment can go
```suggestion
```
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/StartCommandExecutor.java
##########
@@ -0,0 +1,2247 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.hypervisor.vmware.resource;
+
+import java.io.File;
+import java.rmi.RemoteException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.cloud.agent.api.to.DataTO;
+import com.vmware.vim25.VirtualDiskFlatVer2BackingInfo;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.storage.configdrive.ConfigDrive;
+import org.apache.cloudstack.storage.to.TemplateObjectTO;
+import org.apache.cloudstack.storage.to.VolumeObjectTO;
+import org.apache.cloudstack.utils.volume.VirtualMachineDiskInfo;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.commons.lang.ArrayUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.log4j.Logger;
+
+import com.cloud.agent.api.Command;
+import com.cloud.agent.api.StartAnswer;
+import com.cloud.agent.api.StartCommand;
+import com.cloud.agent.api.storage.OVFPropertyTO;
+import com.cloud.agent.api.to.DataStoreTO;
+import com.cloud.agent.api.to.DiskTO;
+import com.cloud.agent.api.to.NfsTO;
+import com.cloud.agent.api.to.NicTO;
+import com.cloud.agent.api.to.VirtualMachineTO;
+import com.cloud.configuration.Resource;
+import com.cloud.hypervisor.vmware.VmwareResourceException;
+import com.cloud.hypervisor.vmware.manager.VmwareManager;
+import com.cloud.hypervisor.vmware.mo.CustomFieldConstants;
+import com.cloud.hypervisor.vmware.mo.DatacenterMO;
+import com.cloud.hypervisor.vmware.mo.DatastoreFile;
+import com.cloud.hypervisor.vmware.mo.DatastoreMO;
+import com.cloud.hypervisor.vmware.mo.DiskControllerType;
+import com.cloud.hypervisor.vmware.mo.HostMO;
+import com.cloud.hypervisor.vmware.mo.HypervisorHostHelper;
+import com.cloud.hypervisor.vmware.mo.TaskMO;
+import com.cloud.hypervisor.vmware.mo.VirtualEthernetCardType;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineDiskInfoBuilder;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineMO;
+import com.cloud.hypervisor.vmware.mo.VmwareHypervisorHost;
+import com.cloud.hypervisor.vmware.util.VmwareContext;
+import com.cloud.hypervisor.vmware.util.VmwareHelper;
+import com.cloud.network.Networks;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Volume;
+import com.cloud.storage.resource.VmwareStorageLayoutHelper;
+import com.cloud.storage.resource.VmwareStorageProcessor;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nicira.nvp.plugin.NiciraNvpApiVersion;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.VmDetailConstants;
+import com.vmware.vim25.BoolPolicy;
+import com.vmware.vim25.DVPortConfigInfo;
+import com.vmware.vim25.DVPortConfigSpec;
+import com.vmware.vim25.DasVmPriority;
+import com.vmware.vim25.DistributedVirtualPort;
+import com.vmware.vim25.DistributedVirtualSwitchPortConnection;
+import com.vmware.vim25.DistributedVirtualSwitchPortCriteria;
+import com.vmware.vim25.ManagedObjectReference;
+import com.vmware.vim25.OptionValue;
+import com.vmware.vim25.VMwareDVSPortSetting;
+import com.vmware.vim25.VirtualDevice;
+import com.vmware.vim25.VirtualDeviceBackingInfo;
+import com.vmware.vim25.VirtualDeviceConfigSpec;
+import com.vmware.vim25.VirtualDeviceConfigSpecOperation;
+import com.vmware.vim25.VirtualDisk;
+import com.vmware.vim25.VirtualEthernetCard;
+import com.vmware.vim25.VirtualEthernetCardDistributedVirtualPortBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardNetworkBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardOpaqueNetworkBackingInfo;
+import com.vmware.vim25.VirtualMachineBootOptions;
+import com.vmware.vim25.VirtualMachineConfigSpec;
+import com.vmware.vim25.VirtualMachineFileInfo;
+import com.vmware.vim25.VirtualMachineFileLayoutEx;
+import com.vmware.vim25.VirtualMachineGuestOsIdentifier;
+import com.vmware.vim25.VirtualUSBController;
+import com.vmware.vim25.VmConfigInfo;
+import com.vmware.vim25.VmwareDistributedVirtualSwitchVlanIdSpec;
+
+class StartCommandExecutor {
+ private static final Logger LOGGER = Logger.getLogger(StartCommandExecutor.class);
+
+ private final VmwareResource vmwareResource;
+
+ public StartCommandExecutor(VmwareResource vmwareResource) {
+ this.vmwareResource = vmwareResource;
+ }
+
+ // FR37 the monster blob god method: reduce to 320 so far and counting
+ protected StartAnswer execute(StartCommand cmd) {
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Executing resource StartCommand: " + vmwareResource.getGson().toJson(cmd));
+ }
+
+ VirtualMachineTO vmSpec = cmd.getVirtualMachine();
+
+ VirtualMachineData existingVm = null;
+
+ Pair<String, String> names = composeVmNames(vmSpec);
+ String vmInternalCSName = names.first();
+ String vmNameOnVcenter = names.second();
+
+ DiskTO rootDiskTO = null;
+
+ Pair<String, String> controllerInfo = getDiskControllerInfo(vmSpec);
+
+ Boolean systemVm = vmSpec.getType().isUsedBySystem();
+
+ VmwareContext context = vmwareResource.getServiceContext();
+ DatacenterMO dcMo = null;
+ try {
+ VmwareManager mgr = context.getStockObject(VmwareManager.CONTEXT_STOCK_NAME);
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(context);
+ dcMo = new DatacenterMO(hyperHost.getContext(), hyperHost.getHyperHostDatacenter());
+
+ // checkIfVmExistsInVcenter(vmInternalCSName, vmNameOnVcenter, dcMo);
+ // FR37 - We expect VM to be already cloned and available at this point
Review comment:
```suggestion
```
##########
File path: server/src/main/java/com/cloud/api/query/dao/TemplateJoinDaoImpl.java
##########
@@ -290,15 +295,22 @@ public TemplateResponse newUpdateResponse(TemplateJoinVO result) {
@Override
public TemplateResponse setTemplateResponse(ResponseView view, TemplateResponse templateResponse, TemplateJoinVO template) {
+ Gson gson = new Gson();
// update details map
- if (template.getDetailName() != null) {
- Map<String, String> details = templateResponse.getDetails();
- if (details == null) {
- details = new HashMap<>();
+ String key = template.getDetailName();
+ if (key != null) {
+ // FR37 TODO check properties and network prerequisites and if details is one of those fill those instead of detail
Review comment:
addressed? do we still treat attributes other than properties in a special way and in fact, do we need to treat properties in a special way?
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/StartCommandExecutor.java
##########
@@ -0,0 +1,2247 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.hypervisor.vmware.resource;
+
+import java.io.File;
+import java.rmi.RemoteException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.cloud.agent.api.to.DataTO;
+import com.vmware.vim25.VirtualDiskFlatVer2BackingInfo;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.storage.configdrive.ConfigDrive;
+import org.apache.cloudstack.storage.to.TemplateObjectTO;
+import org.apache.cloudstack.storage.to.VolumeObjectTO;
+import org.apache.cloudstack.utils.volume.VirtualMachineDiskInfo;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.commons.lang.ArrayUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.log4j.Logger;
+
+import com.cloud.agent.api.Command;
+import com.cloud.agent.api.StartAnswer;
+import com.cloud.agent.api.StartCommand;
+import com.cloud.agent.api.storage.OVFPropertyTO;
+import com.cloud.agent.api.to.DataStoreTO;
+import com.cloud.agent.api.to.DiskTO;
+import com.cloud.agent.api.to.NfsTO;
+import com.cloud.agent.api.to.NicTO;
+import com.cloud.agent.api.to.VirtualMachineTO;
+import com.cloud.configuration.Resource;
+import com.cloud.hypervisor.vmware.VmwareResourceException;
+import com.cloud.hypervisor.vmware.manager.VmwareManager;
+import com.cloud.hypervisor.vmware.mo.CustomFieldConstants;
+import com.cloud.hypervisor.vmware.mo.DatacenterMO;
+import com.cloud.hypervisor.vmware.mo.DatastoreFile;
+import com.cloud.hypervisor.vmware.mo.DatastoreMO;
+import com.cloud.hypervisor.vmware.mo.DiskControllerType;
+import com.cloud.hypervisor.vmware.mo.HostMO;
+import com.cloud.hypervisor.vmware.mo.HypervisorHostHelper;
+import com.cloud.hypervisor.vmware.mo.TaskMO;
+import com.cloud.hypervisor.vmware.mo.VirtualEthernetCardType;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineDiskInfoBuilder;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineMO;
+import com.cloud.hypervisor.vmware.mo.VmwareHypervisorHost;
+import com.cloud.hypervisor.vmware.util.VmwareContext;
+import com.cloud.hypervisor.vmware.util.VmwareHelper;
+import com.cloud.network.Networks;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Volume;
+import com.cloud.storage.resource.VmwareStorageLayoutHelper;
+import com.cloud.storage.resource.VmwareStorageProcessor;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nicira.nvp.plugin.NiciraNvpApiVersion;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.VmDetailConstants;
+import com.vmware.vim25.BoolPolicy;
+import com.vmware.vim25.DVPortConfigInfo;
+import com.vmware.vim25.DVPortConfigSpec;
+import com.vmware.vim25.DasVmPriority;
+import com.vmware.vim25.DistributedVirtualPort;
+import com.vmware.vim25.DistributedVirtualSwitchPortConnection;
+import com.vmware.vim25.DistributedVirtualSwitchPortCriteria;
+import com.vmware.vim25.ManagedObjectReference;
+import com.vmware.vim25.OptionValue;
+import com.vmware.vim25.VMwareDVSPortSetting;
+import com.vmware.vim25.VirtualDevice;
+import com.vmware.vim25.VirtualDeviceBackingInfo;
+import com.vmware.vim25.VirtualDeviceConfigSpec;
+import com.vmware.vim25.VirtualDeviceConfigSpecOperation;
+import com.vmware.vim25.VirtualDisk;
+import com.vmware.vim25.VirtualEthernetCard;
+import com.vmware.vim25.VirtualEthernetCardDistributedVirtualPortBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardNetworkBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardOpaqueNetworkBackingInfo;
+import com.vmware.vim25.VirtualMachineBootOptions;
+import com.vmware.vim25.VirtualMachineConfigSpec;
+import com.vmware.vim25.VirtualMachineFileInfo;
+import com.vmware.vim25.VirtualMachineFileLayoutEx;
+import com.vmware.vim25.VirtualMachineGuestOsIdentifier;
+import com.vmware.vim25.VirtualUSBController;
+import com.vmware.vim25.VmConfigInfo;
+import com.vmware.vim25.VmwareDistributedVirtualSwitchVlanIdSpec;
+
+class StartCommandExecutor {
+ private static final Logger LOGGER = Logger.getLogger(StartCommandExecutor.class);
+
+ private final VmwareResource vmwareResource;
+
+ public StartCommandExecutor(VmwareResource vmwareResource) {
+ this.vmwareResource = vmwareResource;
+ }
+
+ // FR37 the monster blob god method: reduce to 320 so far and counting
+ protected StartAnswer execute(StartCommand cmd) {
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Executing resource StartCommand: " + vmwareResource.getGson().toJson(cmd));
+ }
+
+ VirtualMachineTO vmSpec = cmd.getVirtualMachine();
+
+ VirtualMachineData existingVm = null;
+
+ Pair<String, String> names = composeVmNames(vmSpec);
+ String vmInternalCSName = names.first();
+ String vmNameOnVcenter = names.second();
+
+ DiskTO rootDiskTO = null;
+
+ Pair<String, String> controllerInfo = getDiskControllerInfo(vmSpec);
+
+ Boolean systemVm = vmSpec.getType().isUsedBySystem();
+
+ VmwareContext context = vmwareResource.getServiceContext();
+ DatacenterMO dcMo = null;
+ try {
+ VmwareManager mgr = context.getStockObject(VmwareManager.CONTEXT_STOCK_NAME);
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(context);
+ dcMo = new DatacenterMO(hyperHost.getContext(), hyperHost.getHyperHostDatacenter());
+
+ // checkIfVmExistsInVcenter(vmInternalCSName, vmNameOnVcenter, dcMo);
+ // FR37 - We expect VM to be already cloned and available at this point
+ VirtualMachineMO vmMo = dcMo.findVm(vmInternalCSName);
+ if (vmMo == null) {
+ vmMo = dcMo.findVm(vmNameOnVcenter);
+ }
+
+ DiskTO[] specDisks = vmSpec.getDisks();
+ boolean installAsIs = StringUtils.isNotEmpty(vmSpec.getTemplateLocation());
+ // FR37 if startcommand contains enough info: a template url/-location and flag; deploy OVA as is
+ if (vmMo == null && installAsIs) {
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("deploying OVA from %s as is", vmSpec.getTemplateLocation()));
+ }
+ getStorageProcessor().cloneVMFromTemplate(vmSpec.getTemplateName(), vmInternalCSName, vmSpec.getTemplatePrimaryStoreUuid());
+ vmMo = dcMo.findVm(vmInternalCSName);
+ mapSpecDisksToClonedDisks(vmMo, vmInternalCSName, specDisks);
+ }
+
+ // VM may not have been on the same host, relocate to expected host
+ if (vmMo != null) {
+ vmMo.relocate(hyperHost.getMor());
+ // Get updated MO
+ vmMo = hyperHost.findVmOnHyperHost(vmMo.getVmName());
+ }
+
+ String guestOsId = translateGuestOsIdentifier(vmSpec.getArch(), vmSpec.getOs(), vmSpec.getPlatformEmulator()).value();
+ DiskTO[] disks = validateDisks(specDisks);
+ NicTO[] nics = vmSpec.getNics();
+
+ // FIXME: disks logic here, why is disks/volumes during copy not set with pool ID?
+ // FR37 TODO if deployasis a new VM no datastores are known (yet) and we need to get the data store from the tvmspec content library / template location
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails = inferDatastoreDetailsFromDiskInfo(hyperHost, context, disks, cmd);
+ if ((dataStoresDetails == null) || (dataStoresDetails.isEmpty())) {
+ String msg = "Unable to locate datastore details of the volumes to be attached";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ // FR37 - this may not be necessary the cloned VM will have disks and knowledge of datastore paths/location?
+ // throw new Exception(msg);
Review comment:
```suggestion
```
##########
File path: server/src/main/java/com/cloud/hypervisor/HypervisorGuruBase.java
##########
@@ -215,7 +215,7 @@ protected VirtualMachineTO toVirtualMachineTO(VirtualMachineProfile vmProfile) {
to.setNics(nics);
to.setDisks(vmProfile.getDisks().toArray(new DiskTO[vmProfile.getDisks().size()]));
-
+ // FR37 if this is a new VM to be deployed as is from a template we need to pass the mary storage somehow, now only for actual disks is a primary storage passed
Review comment:
not sure but i think we can
```suggestion
```
if not fully handled yet we might want to change to
```suggestion
// If this is a new VM to be deployed as is from a template we need to pass the primary storage somehow, now only for actual disks is a primary storage passed
```
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/storage/resource/VmwareStorageProcessor.java
##########
@@ -558,9 +576,27 @@ private String getOVFFilePath(String srcOVAFileName) {
return new Pair<>(vmMo, virtualSize);
}
+ private void deployTemplateToContentLibrary(VmwareHypervisorHost hyperHost, DatastoreMO datastoreMo, String secondaryStorageUrl, String templatePathAtSecondaryStorage,
+ String templateUuid, String srcOVAFileName, String srcFileName) throws Exception {
+ String storeName = getSecondaryDatastoreUUID(secondaryStorageUrl);
+ ManagedObjectReference morSecDatastore = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, storeName);
+ if (morSecDatastore == null) {
+ morSecDatastore = prepareSecondaryDatastoreOnHost(secondaryStorageUrl);
+ }
+ DatastoreMO secDsMo = new DatastoreMO(datastoreMo.getContext(), morSecDatastore);
+ DatastoreSummary secDatastoresummary = secDsMo.getSummary();
+
+ String ovfFile = getOVFFile(srcOVAFileName);
+ boolean importResult = true;//contentLibraryService.importOvf(datastoreMo.getContext(), secDatastoresummary.getUrl() + templatePathAtSecondaryStorage, ovfFile, datastoreMo.getName(), templateUuid);
+ if (!importResult) {
+ s_logger.warn("Failed to import ovf into the content library: " + srcFileName);
+ }
+ }
+
Review comment:
not used yet
```suggestion
```
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/storage/resource/VmwareStorageProcessor.java
##########
@@ -558,9 +576,27 @@ private String getOVFFilePath(String srcOVAFileName) {
return new Pair<>(vmMo, virtualSize);
}
+ private void deployTemplateToContentLibrary(VmwareHypervisorHost hyperHost, DatastoreMO datastoreMo, String secondaryStorageUrl, String templatePathAtSecondaryStorage,
+ String templateUuid, String srcOVAFileName, String srcFileName) throws Exception {
+ String storeName = getSecondaryDatastoreUUID(secondaryStorageUrl);
+ ManagedObjectReference morSecDatastore = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, storeName);
+ if (morSecDatastore == null) {
+ morSecDatastore = prepareSecondaryDatastoreOnHost(secondaryStorageUrl);
+ }
+ DatastoreMO secDsMo = new DatastoreMO(datastoreMo.getContext(), morSecDatastore);
+ DatastoreSummary secDatastoresummary = secDsMo.getSummary();
+
+ String ovfFile = getOVFFile(srcOVAFileName);
+ boolean importResult = true;//contentLibraryService.importOvf(datastoreMo.getContext(), secDatastoresummary.getUrl() + templatePathAtSecondaryStorage, ovfFile, datastoreMo.getName(), templateUuid);
+ if (!importResult) {
+ s_logger.warn("Failed to import ovf into the content library: " + srcFileName);
+ }
+ }
+
@Override
public Answer copyTemplateToPrimaryStorage(CopyCommand cmd) {
DataTO srcData = cmd.getSrcTO();
+ // FR37 TODO find where TO is created and make sure deployAsIs is set correctly
Review comment:
done
```suggestion
```
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/StartCommandExecutor.java
##########
@@ -0,0 +1,2247 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.hypervisor.vmware.resource;
+
+import java.io.File;
+import java.rmi.RemoteException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.cloud.agent.api.to.DataTO;
+import com.vmware.vim25.VirtualDiskFlatVer2BackingInfo;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.storage.configdrive.ConfigDrive;
+import org.apache.cloudstack.storage.to.TemplateObjectTO;
+import org.apache.cloudstack.storage.to.VolumeObjectTO;
+import org.apache.cloudstack.utils.volume.VirtualMachineDiskInfo;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.commons.lang.ArrayUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.log4j.Logger;
+
+import com.cloud.agent.api.Command;
+import com.cloud.agent.api.StartAnswer;
+import com.cloud.agent.api.StartCommand;
+import com.cloud.agent.api.storage.OVFPropertyTO;
+import com.cloud.agent.api.to.DataStoreTO;
+import com.cloud.agent.api.to.DiskTO;
+import com.cloud.agent.api.to.NfsTO;
+import com.cloud.agent.api.to.NicTO;
+import com.cloud.agent.api.to.VirtualMachineTO;
+import com.cloud.configuration.Resource;
+import com.cloud.hypervisor.vmware.VmwareResourceException;
+import com.cloud.hypervisor.vmware.manager.VmwareManager;
+import com.cloud.hypervisor.vmware.mo.CustomFieldConstants;
+import com.cloud.hypervisor.vmware.mo.DatacenterMO;
+import com.cloud.hypervisor.vmware.mo.DatastoreFile;
+import com.cloud.hypervisor.vmware.mo.DatastoreMO;
+import com.cloud.hypervisor.vmware.mo.DiskControllerType;
+import com.cloud.hypervisor.vmware.mo.HostMO;
+import com.cloud.hypervisor.vmware.mo.HypervisorHostHelper;
+import com.cloud.hypervisor.vmware.mo.TaskMO;
+import com.cloud.hypervisor.vmware.mo.VirtualEthernetCardType;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineDiskInfoBuilder;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineMO;
+import com.cloud.hypervisor.vmware.mo.VmwareHypervisorHost;
+import com.cloud.hypervisor.vmware.util.VmwareContext;
+import com.cloud.hypervisor.vmware.util.VmwareHelper;
+import com.cloud.network.Networks;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Volume;
+import com.cloud.storage.resource.VmwareStorageLayoutHelper;
+import com.cloud.storage.resource.VmwareStorageProcessor;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nicira.nvp.plugin.NiciraNvpApiVersion;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.VmDetailConstants;
+import com.vmware.vim25.BoolPolicy;
+import com.vmware.vim25.DVPortConfigInfo;
+import com.vmware.vim25.DVPortConfigSpec;
+import com.vmware.vim25.DasVmPriority;
+import com.vmware.vim25.DistributedVirtualPort;
+import com.vmware.vim25.DistributedVirtualSwitchPortConnection;
+import com.vmware.vim25.DistributedVirtualSwitchPortCriteria;
+import com.vmware.vim25.ManagedObjectReference;
+import com.vmware.vim25.OptionValue;
+import com.vmware.vim25.VMwareDVSPortSetting;
+import com.vmware.vim25.VirtualDevice;
+import com.vmware.vim25.VirtualDeviceBackingInfo;
+import com.vmware.vim25.VirtualDeviceConfigSpec;
+import com.vmware.vim25.VirtualDeviceConfigSpecOperation;
+import com.vmware.vim25.VirtualDisk;
+import com.vmware.vim25.VirtualEthernetCard;
+import com.vmware.vim25.VirtualEthernetCardDistributedVirtualPortBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardNetworkBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardOpaqueNetworkBackingInfo;
+import com.vmware.vim25.VirtualMachineBootOptions;
+import com.vmware.vim25.VirtualMachineConfigSpec;
+import com.vmware.vim25.VirtualMachineFileInfo;
+import com.vmware.vim25.VirtualMachineFileLayoutEx;
+import com.vmware.vim25.VirtualMachineGuestOsIdentifier;
+import com.vmware.vim25.VirtualUSBController;
+import com.vmware.vim25.VmConfigInfo;
+import com.vmware.vim25.VmwareDistributedVirtualSwitchVlanIdSpec;
+
+class StartCommandExecutor {
+ private static final Logger LOGGER = Logger.getLogger(StartCommandExecutor.class);
+
+ private final VmwareResource vmwareResource;
+
+ public StartCommandExecutor(VmwareResource vmwareResource) {
+ this.vmwareResource = vmwareResource;
+ }
+
+ // FR37 the monster blob god method: reduce to 320 so far and counting
+ protected StartAnswer execute(StartCommand cmd) {
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Executing resource StartCommand: " + vmwareResource.getGson().toJson(cmd));
+ }
+
+ VirtualMachineTO vmSpec = cmd.getVirtualMachine();
+
+ VirtualMachineData existingVm = null;
+
+ Pair<String, String> names = composeVmNames(vmSpec);
+ String vmInternalCSName = names.first();
+ String vmNameOnVcenter = names.second();
+
+ DiskTO rootDiskTO = null;
+
+ Pair<String, String> controllerInfo = getDiskControllerInfo(vmSpec);
+
+ Boolean systemVm = vmSpec.getType().isUsedBySystem();
+
+ VmwareContext context = vmwareResource.getServiceContext();
+ DatacenterMO dcMo = null;
+ try {
+ VmwareManager mgr = context.getStockObject(VmwareManager.CONTEXT_STOCK_NAME);
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(context);
+ dcMo = new DatacenterMO(hyperHost.getContext(), hyperHost.getHyperHostDatacenter());
+
+ // checkIfVmExistsInVcenter(vmInternalCSName, vmNameOnVcenter, dcMo);
+ // FR37 - We expect VM to be already cloned and available at this point
+ VirtualMachineMO vmMo = dcMo.findVm(vmInternalCSName);
+ if (vmMo == null) {
+ vmMo = dcMo.findVm(vmNameOnVcenter);
+ }
+
+ DiskTO[] specDisks = vmSpec.getDisks();
+ boolean installAsIs = StringUtils.isNotEmpty(vmSpec.getTemplateLocation());
+ // FR37 if startcommand contains enough info: a template url/-location and flag; deploy OVA as is
+ if (vmMo == null && installAsIs) {
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("deploying OVA from %s as is", vmSpec.getTemplateLocation()));
+ }
+ getStorageProcessor().cloneVMFromTemplate(vmSpec.getTemplateName(), vmInternalCSName, vmSpec.getTemplatePrimaryStoreUuid());
+ vmMo = dcMo.findVm(vmInternalCSName);
+ mapSpecDisksToClonedDisks(vmMo, vmInternalCSName, specDisks);
+ }
+
+ // VM may not have been on the same host, relocate to expected host
+ if (vmMo != null) {
+ vmMo.relocate(hyperHost.getMor());
+ // Get updated MO
+ vmMo = hyperHost.findVmOnHyperHost(vmMo.getVmName());
+ }
+
+ String guestOsId = translateGuestOsIdentifier(vmSpec.getArch(), vmSpec.getOs(), vmSpec.getPlatformEmulator()).value();
+ DiskTO[] disks = validateDisks(specDisks);
+ NicTO[] nics = vmSpec.getNics();
+
+ // FIXME: disks logic here, why is disks/volumes during copy not set with pool ID?
+ // FR37 TODO if deployasis a new VM no datastores are known (yet) and we need to get the data store from the tvmspec content library / template location
Review comment:
needs validation and addressing
##########
File path: server/src/main/java/com/cloud/vm/UserVmManagerImpl.java
##########
@@ -4009,6 +4029,56 @@ public UserVmVO doInTransaction(TransactionStatus status) throws InsufficientCap
});
}
+ private void copyNetworkRequirementsToVm(UserVmVO vm, VirtualMachineTemplate template) {
+ if (template.isDeployAsIs()) { // FR37 should be always when we are done
+ List<VMTemplateDetailVO> details = templateDetailsDao.listDetailsByTemplateId(template.getId(), ImageStore.REQUIRED_NETWORK_PREFIX);
+ for (VMTemplateDetailVO detail : details) {
+ vm.setDetail(detail.getName(), detail.getValue());
+ }
+ }
+ }
+
+ private void copyDiskDetailsToVm(UserVmVO vm, VirtualMachineTemplate template) {
+ if (template.isDeployAsIs()) { // FR37 should be always when we are done
Review comment:
```suggestion
if (template.isDeployAsIs()) { // This should be always when we are done, i.e. after a cooling down period.
```
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/VmwareResource.java
##########
@@ -346,21 +332,23 @@
public static final String VMDK_EXTENSION = ".vmdk";
private static final Random RANDOM = new Random(System.nanoTime());
+ private final StartCommandExecutor startCommandExecutor = new StartCommandExecutor(this);
+
+ // FR37 Does this need to be a setting?
Review comment:
remove comment after decision made
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/storage/resource/VmwareStorageProcessor.java
##########
@@ -765,6 +800,7 @@ private boolean createVMFullClone(VirtualMachineMO vmTemplate, DatacenterMO dcMo
return true;
}
+ // FR37 TODO: OR make sure deploy is done in this method from template
Review comment:
```suggestion
```
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/StartCommandExecutor.java
##########
@@ -0,0 +1,2247 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.hypervisor.vmware.resource;
+
+import java.io.File;
+import java.rmi.RemoteException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.cloud.agent.api.to.DataTO;
+import com.vmware.vim25.VirtualDiskFlatVer2BackingInfo;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.storage.configdrive.ConfigDrive;
+import org.apache.cloudstack.storage.to.TemplateObjectTO;
+import org.apache.cloudstack.storage.to.VolumeObjectTO;
+import org.apache.cloudstack.utils.volume.VirtualMachineDiskInfo;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.commons.lang.ArrayUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.log4j.Logger;
+
+import com.cloud.agent.api.Command;
+import com.cloud.agent.api.StartAnswer;
+import com.cloud.agent.api.StartCommand;
+import com.cloud.agent.api.storage.OVFPropertyTO;
+import com.cloud.agent.api.to.DataStoreTO;
+import com.cloud.agent.api.to.DiskTO;
+import com.cloud.agent.api.to.NfsTO;
+import com.cloud.agent.api.to.NicTO;
+import com.cloud.agent.api.to.VirtualMachineTO;
+import com.cloud.configuration.Resource;
+import com.cloud.hypervisor.vmware.VmwareResourceException;
+import com.cloud.hypervisor.vmware.manager.VmwareManager;
+import com.cloud.hypervisor.vmware.mo.CustomFieldConstants;
+import com.cloud.hypervisor.vmware.mo.DatacenterMO;
+import com.cloud.hypervisor.vmware.mo.DatastoreFile;
+import com.cloud.hypervisor.vmware.mo.DatastoreMO;
+import com.cloud.hypervisor.vmware.mo.DiskControllerType;
+import com.cloud.hypervisor.vmware.mo.HostMO;
+import com.cloud.hypervisor.vmware.mo.HypervisorHostHelper;
+import com.cloud.hypervisor.vmware.mo.TaskMO;
+import com.cloud.hypervisor.vmware.mo.VirtualEthernetCardType;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineDiskInfoBuilder;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineMO;
+import com.cloud.hypervisor.vmware.mo.VmwareHypervisorHost;
+import com.cloud.hypervisor.vmware.util.VmwareContext;
+import com.cloud.hypervisor.vmware.util.VmwareHelper;
+import com.cloud.network.Networks;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Volume;
+import com.cloud.storage.resource.VmwareStorageLayoutHelper;
+import com.cloud.storage.resource.VmwareStorageProcessor;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nicira.nvp.plugin.NiciraNvpApiVersion;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.VmDetailConstants;
+import com.vmware.vim25.BoolPolicy;
+import com.vmware.vim25.DVPortConfigInfo;
+import com.vmware.vim25.DVPortConfigSpec;
+import com.vmware.vim25.DasVmPriority;
+import com.vmware.vim25.DistributedVirtualPort;
+import com.vmware.vim25.DistributedVirtualSwitchPortConnection;
+import com.vmware.vim25.DistributedVirtualSwitchPortCriteria;
+import com.vmware.vim25.ManagedObjectReference;
+import com.vmware.vim25.OptionValue;
+import com.vmware.vim25.VMwareDVSPortSetting;
+import com.vmware.vim25.VirtualDevice;
+import com.vmware.vim25.VirtualDeviceBackingInfo;
+import com.vmware.vim25.VirtualDeviceConfigSpec;
+import com.vmware.vim25.VirtualDeviceConfigSpecOperation;
+import com.vmware.vim25.VirtualDisk;
+import com.vmware.vim25.VirtualEthernetCard;
+import com.vmware.vim25.VirtualEthernetCardDistributedVirtualPortBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardNetworkBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardOpaqueNetworkBackingInfo;
+import com.vmware.vim25.VirtualMachineBootOptions;
+import com.vmware.vim25.VirtualMachineConfigSpec;
+import com.vmware.vim25.VirtualMachineFileInfo;
+import com.vmware.vim25.VirtualMachineFileLayoutEx;
+import com.vmware.vim25.VirtualMachineGuestOsIdentifier;
+import com.vmware.vim25.VirtualUSBController;
+import com.vmware.vim25.VmConfigInfo;
+import com.vmware.vim25.VmwareDistributedVirtualSwitchVlanIdSpec;
+
+class StartCommandExecutor {
+ private static final Logger LOGGER = Logger.getLogger(StartCommandExecutor.class);
+
+ private final VmwareResource vmwareResource;
+
+ public StartCommandExecutor(VmwareResource vmwareResource) {
+ this.vmwareResource = vmwareResource;
+ }
+
+ // FR37 the monster blob god method: reduce to 320 so far and counting
+ protected StartAnswer execute(StartCommand cmd) {
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Executing resource StartCommand: " + vmwareResource.getGson().toJson(cmd));
+ }
+
+ VirtualMachineTO vmSpec = cmd.getVirtualMachine();
+
+ VirtualMachineData existingVm = null;
+
+ Pair<String, String> names = composeVmNames(vmSpec);
+ String vmInternalCSName = names.first();
+ String vmNameOnVcenter = names.second();
+
+ DiskTO rootDiskTO = null;
+
+ Pair<String, String> controllerInfo = getDiskControllerInfo(vmSpec);
+
+ Boolean systemVm = vmSpec.getType().isUsedBySystem();
+
+ VmwareContext context = vmwareResource.getServiceContext();
+ DatacenterMO dcMo = null;
+ try {
+ VmwareManager mgr = context.getStockObject(VmwareManager.CONTEXT_STOCK_NAME);
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(context);
+ dcMo = new DatacenterMO(hyperHost.getContext(), hyperHost.getHyperHostDatacenter());
+
+ // checkIfVmExistsInVcenter(vmInternalCSName, vmNameOnVcenter, dcMo);
+ // FR37 - We expect VM to be already cloned and available at this point
+ VirtualMachineMO vmMo = dcMo.findVm(vmInternalCSName);
+ if (vmMo == null) {
+ vmMo = dcMo.findVm(vmNameOnVcenter);
+ }
+
+ DiskTO[] specDisks = vmSpec.getDisks();
+ boolean installAsIs = StringUtils.isNotEmpty(vmSpec.getTemplateLocation());
+ // FR37 if startcommand contains enough info: a template url/-location and flag; deploy OVA as is
+ if (vmMo == null && installAsIs) {
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("deploying OVA from %s as is", vmSpec.getTemplateLocation()));
+ }
+ getStorageProcessor().cloneVMFromTemplate(vmSpec.getTemplateName(), vmInternalCSName, vmSpec.getTemplatePrimaryStoreUuid());
+ vmMo = dcMo.findVm(vmInternalCSName);
+ mapSpecDisksToClonedDisks(vmMo, vmInternalCSName, specDisks);
+ }
+
+ // VM may not have been on the same host, relocate to expected host
+ if (vmMo != null) {
+ vmMo.relocate(hyperHost.getMor());
+ // Get updated MO
+ vmMo = hyperHost.findVmOnHyperHost(vmMo.getVmName());
+ }
+
+ String guestOsId = translateGuestOsIdentifier(vmSpec.getArch(), vmSpec.getOs(), vmSpec.getPlatformEmulator()).value();
+ DiskTO[] disks = validateDisks(specDisks);
+ NicTO[] nics = vmSpec.getNics();
+
+ // FIXME: disks logic here, why is disks/volumes during copy not set with pool ID?
+ // FR37 TODO if deployasis a new VM no datastores are known (yet) and we need to get the data store from the tvmspec content library / template location
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails = inferDatastoreDetailsFromDiskInfo(hyperHost, context, disks, cmd);
+ if ((dataStoresDetails == null) || (dataStoresDetails.isEmpty())) {
+ String msg = "Unable to locate datastore details of the volumes to be attached";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ // FR37 - this may not be necessary the cloned VM will have disks and knowledge of datastore paths/location?
+ // throw new Exception(msg);
+ }
+
+ // FR37 - this may need checking, if the first datastore is the right one - ideally it should be datastore where first disk is hosted
+ DatastoreMO dsRootVolumeIsOn = null; //
+ if (! installAsIs) {
+ dsRootVolumeIsOn = getDatastoreThatRootDiskIsOn(dataStoresDetails, disks);
+
+ if (dsRootVolumeIsOn == null) {
+ String msg = "Unable to locate datastore details of root volume";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ throw new VmwareResourceException(msg);
+ }
+ }
+
+ VirtualMachineDiskInfoBuilder diskInfoBuilder = null;
+ VirtualDevice[] nicDevices = null;
+ DiskControllerType systemVmScsiControllerType = DiskControllerType.lsilogic;
+ int firstScsiControllerBusNum = 0;
+ int numScsiControllerForSystemVm = 1;
+ boolean hasSnapshot = false;
+ if (vmMo != null) {
+ PrepareRunningVMForConfiguration prepareRunningVMForConfiguration = new PrepareRunningVMForConfiguration(vmInternalCSName, controllerInfo, systemVm, vmMo,
+ systemVmScsiControllerType, firstScsiControllerBusNum, numScsiControllerForSystemVm);
+ prepareRunningVMForConfiguration.invoke();
+ diskInfoBuilder = prepareRunningVMForConfiguration.getDiskInfoBuilder();
+ nicDevices = prepareRunningVMForConfiguration.getNicDevices();
+ hasSnapshot = prepareRunningVMForConfiguration.isHasSnapshot();
+ } else {
+ ManagedObjectReference morDc = hyperHost.getHyperHostDatacenter();
+ assert (morDc != null);
+
+ vmMo = hyperHost.findVmOnPeerHyperHost(vmInternalCSName);
+ if (vmMo != null) {
+ VirtualMachineRecycler virtualMachineRecycler = new VirtualMachineRecycler(vmInternalCSName, controllerInfo, systemVm, hyperHost, vmMo,
+ systemVmScsiControllerType, firstScsiControllerBusNum, numScsiControllerForSystemVm);
+ virtualMachineRecycler.invoke();
+ diskInfoBuilder = virtualMachineRecycler.getDiskInfoBuilder();
+ hasSnapshot = virtualMachineRecycler.isHasSnapshot();
+ nicDevices = virtualMachineRecycler.getNicDevices();
+ } else {
+ // FR37 we just didn't find a VM by name 'vmInternalCSName', so why is this here?
+ existingVm = unregisterOnOtherClusterButHoldOnToOldVmData(vmInternalCSName, dcMo);
+
+ createNewVm(context, vmSpec, installAsIs, vmInternalCSName, vmNameOnVcenter, controllerInfo, systemVm, mgr, hyperHost, guestOsId, disks, dataStoresDetails,
+ dsRootVolumeIsOn);
+ }
+
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ if (vmMo == null) {
+ throw new Exception("Failed to find the newly create or relocated VM. vmName: " + vmInternalCSName);
+ }
+ }
+ // vmMo should now be a stopped VM on the intended host
+ // The number of disks changed must be 0 for install as is, as the VM is a clone
+ int disksChanges = !installAsIs ? disks.length : 0;
+ int totalChangeDevices = disksChanges + nics.length;
+ int hackDeviceCount = 0;
+ if (diskInfoBuilder != null) {
+ hackDeviceCount += diskInfoBuilder.getDiskCount();
+ }
+ hackDeviceCount += nicDevices == null ? 0 : nicDevices.length;
+ // vApp cdrom device
+ // HACK ALERT: ovf properties might not be the only or defining feature of vApps; needs checking
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("current count(s) desired: %d/ found:%d. now adding device to device count for vApp config ISO", totalChangeDevices, hackDeviceCount));
+ }
+ if (vmSpec.getOvfProperties() != null) {
+ totalChangeDevices++;
+ }
+
+ DiskTO volIso = null;
+ if (vmSpec.getType() != VirtualMachine.Type.User) {
+ // system VM needs a patch ISO
+ totalChangeDevices++;
+ } else {
+ volIso = getIsoDiskTO(disks);
+ if (volIso == null)
+ totalChangeDevices++;
+ }
+
+ VirtualMachineConfigSpec vmConfigSpec = new VirtualMachineConfigSpec();
+
+ VmwareHelper.setBasicVmConfig(vmConfigSpec, vmSpec.getCpus(), vmSpec.getMaxSpeed(), vmwareResource.getReservedCpuMHZ(vmSpec), (int)(vmSpec.getMaxRam() / (1024 * 1024)),
+ vmwareResource.getReservedMemoryMb(vmSpec), guestOsId, vmSpec.getLimitCpuUse());
+
+ int numCoresPerSocket = adjustNumberOfCoresPerSocket(vmSpec, vmMo, vmConfigSpec);
+
+ // Check for hotadd settings
+ vmConfigSpec.setMemoryHotAddEnabled(vmMo.isMemoryHotAddSupported(guestOsId));
+
+ String hostApiVersion = ((HostMO)hyperHost).getHostAboutInfo().getApiVersion();
+ if (numCoresPerSocket > 1 && hostApiVersion.compareTo("5.0") < 0) {
+ LOGGER.warn("Dynamic scaling of CPU is not supported for Virtual Machines with multi-core vCPUs in case of ESXi hosts 4.1 and prior. Hence CpuHotAdd will not be"
+ + " enabled for Virtual Machine: " + vmInternalCSName);
+ vmConfigSpec.setCpuHotAddEnabled(false);
+ } else {
+ vmConfigSpec.setCpuHotAddEnabled(vmMo.isCpuHotAddSupported(guestOsId));
+ }
+
+ vmwareResource.configNestedHVSupport(vmMo, vmSpec, vmConfigSpec);
+
+ VirtualDeviceConfigSpec[] deviceConfigSpecArray = new VirtualDeviceConfigSpec[totalChangeDevices];
+ int deviceCount = 0;
+ int ideUnitNumber = 0;
+ int scsiUnitNumber = 0;
+ int ideControllerKey = vmMo.getIDEDeviceControllerKey();
+ int scsiControllerKey = vmMo.getScsiDeviceControllerKeyNoException();
+
+ IsoSetup isoSetup = new IsoSetup(vmSpec, mgr, hyperHost, vmMo, disks, volIso, deviceConfigSpecArray, deviceCount, ideUnitNumber);
+ isoSetup.invoke();
+ deviceCount = isoSetup.getDeviceCount();
+ ideUnitNumber = isoSetup.getIdeUnitNumber();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("setting up %d disks with root from %s", diskInfoBuilder.getDiskCount(), vmwareResource.getGson().toJson(rootDiskTO)));
+ }
+
+ DiskSetup diskSetup = new DiskSetup(vmSpec, rootDiskTO, controllerInfo, context, dcMo, hyperHost, vmMo, disks, dataStoresDetails, diskInfoBuilder, hasSnapshot,
+ deviceConfigSpecArray, deviceCount, ideUnitNumber, scsiUnitNumber, ideControllerKey, scsiControllerKey, installAsIs);
+ diskSetup.invoke();
+
+ rootDiskTO = diskSetup.getRootDiskTO();
+ deviceCount = diskSetup.getDeviceCount();
+ DiskTO[] sortedDisks = diskSetup.getSortedDisks();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("seting up %d disks with root from %s", sortedDisks.length, vmwareResource.getGson().toJson(rootDiskTO)));
+ }
+
+ deviceCount += setupUsbDevicesAndGetCount(vmInternalCSName, vmMo, guestOsId, deviceConfigSpecArray, deviceCount);
+
+ NicSetup nicSetup = new NicSetup(cmd, vmSpec, vmInternalCSName, context, mgr, hyperHost, vmMo, nics, deviceConfigSpecArray, deviceCount, nicDevices);
+ nicSetup.invoke();
+
+ deviceCount = nicSetup.getDeviceCount();
+ int nicMask = nicSetup.getNicMask();
+ int nicCount = nicSetup.getNicCount();
+ Map<String, String> nicUuidToDvSwitchUuid = nicSetup.getNicUuidToDvSwitchUuid();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("device count: %d; nic count %d", deviceCount, nicCount));
+ }
+ for (int j = 0; j < deviceCount; j++)
+ vmConfigSpec.getDeviceChange().add(deviceConfigSpecArray[j]);
+
+ //
+ // Setup VM options
+ //
+
+ // pass boot arguments through machine.id & perform customized options to VMX
+ ArrayList<OptionValue> extraOptions = new ArrayList<>();
+ configBasicExtraOption(extraOptions, vmSpec);
+ configNvpExtraOption(extraOptions, vmSpec, nicUuidToDvSwitchUuid);
+ configCustomExtraOption(extraOptions, vmSpec);
+
+ // config for NCC
+ VirtualMachine.Type vmType = cmd.getVirtualMachine().getType();
+ if (vmType.equals(VirtualMachine.Type.NetScalerVm)) {
+ NicTO mgmtNic = vmSpec.getNics()[0];
+ OptionValue option = new OptionValue();
+ option.setKey("machine.id");
+ option.setValue("ip=" + mgmtNic.getIp() + "&netmask=" + mgmtNic.getNetmask() + "&gateway=" + mgmtNic.getGateway());
+ extraOptions.add(option);
+ }
+
+ // config VNC
+ String keyboardLayout = null;
+ if (vmSpec.getDetails() != null)
+ keyboardLayout = vmSpec.getDetails().get(VmDetailConstants.KEYBOARD);
+ vmConfigSpec.getExtraConfig().addAll(
+ Arrays.asList(vmwareResource.configureVnc(extraOptions.toArray(new OptionValue[0]), hyperHost, vmInternalCSName, vmSpec.getVncPassword(), keyboardLayout)));
+
+ // config video card
+ vmwareResource.configureVideoCard(vmMo, vmSpec, vmConfigSpec);
+
+ // Set OVF properties (if available)
+ Pair<String, List<OVFPropertyTO>> ovfPropsMap = vmSpec.getOvfProperties();
+ VmConfigInfo templateVappConfig;
+ List<OVFPropertyTO> ovfProperties;
+ if (ovfPropsMap != null) {
+ String vmTemplate = ovfPropsMap.first();
+ LOGGER.info("Find VM template " + vmTemplate);
+ VirtualMachineMO vmTemplateMO = dcMo.findVm(vmTemplate);
+ templateVappConfig = vmTemplateMO.getConfigInfo().getVAppConfig();
+ ovfProperties = ovfPropsMap.second();
+ // Set OVF properties (if available)
+ if (CollectionUtils.isNotEmpty(ovfProperties)) {
+ LOGGER.info("Copying OVF properties from template and setting them to the values the user provided");
+ vmwareResource.copyVAppConfigsFromTemplate(templateVappConfig, ovfProperties, vmConfigSpec);
+ }
+ }
+
+ checkBootOptions(vmSpec, vmConfigSpec);
+
+ //
+ // Configure VM
+ //
+ if (!vmMo.configureVm(vmConfigSpec)) {
+ throw new Exception("Failed to configure VM before start. vmName: " + vmInternalCSName);
+ }
+ // FR37 TODO reconcile disks now!!! they are configured, check and add them to the returning vmTO
+ if (vmSpec.getType() == VirtualMachine.Type.DomainRouter) {
+ hyperHost.setRestartPriorityForVM(vmMo, DasVmPriority.HIGH.value());
+ }
+
+ // Resizing root disk only when explicit requested by user
+ final Map<String, String> vmDetails = cmd.getVirtualMachine().getDetails();
+ if (rootDiskTO != null && !hasSnapshot && (vmDetails != null && vmDetails.containsKey(ApiConstants.ROOT_DISK_SIZE))) {
+ resizeRootDiskOnVMStart(vmMo, rootDiskTO, hyperHost, context);
+ }
+
+ //
+ // Post Configuration
+ //
+
+ vmMo.setCustomFieldValue(CustomFieldConstants.CLOUD_NIC_MASK, String.valueOf(nicMask));
+ postNvpConfigBeforeStart(vmMo, vmSpec);
+
+ Map<String, Map<String, String>> iqnToData = new HashMap<>();
+
+ postDiskConfigBeforeStart(vmMo, vmSpec, sortedDisks, iqnToData, hyperHost, context, installAsIs);
+
+ //
+ // Power-on VM
+ //
+ if (!vmMo.powerOn()) {
+ throw new Exception("Failed to start VM. vmName: " + vmInternalCSName + " with hostname " + vmNameOnVcenter);
+ }
+
+ if (installAsIs) {
+ // Set disks as the disks path and chain is retrieved from the cloned VM disks
+ cmd.getVirtualMachine().setDisks(disks);
+ }
+
+ StartAnswer startAnswer = new StartAnswer(cmd);
+
+ startAnswer.setIqnToData(iqnToData);
+
+ deleteOldVersionOfTheStartedVM(existingVm, dcMo, vmMo);
+
+ return startAnswer;
+ } catch (Throwable e) {
+ return handleStartFailure(cmd, vmSpec, existingVm, dcMo, e);
+ } finally {
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("finally done with %s", vmwareResource.getGson().toJson(cmd)));
+ }
+ }
+ }
+
+ /**
+ * Modify the specDisks information to match the cloned VM's disks (from vmMo VM)
+ */
+ private void mapSpecDisksToClonedDisks(VirtualMachineMO vmMo, String vmInternalCSName, DiskTO[] specDisks) {
+ try {
+ if (vmMo != null && ArrayUtils.isNotEmpty(specDisks)) {
+ List<VirtualDisk> vmDisks = vmMo.getVirtualDisks();
+ List<DiskTO> sortedDisks = Arrays.asList(sortVolumesByDeviceId(specDisks))
+ .stream()
+ .filter(x -> x.getType() == Volume.Type.ROOT)
+ .collect(Collectors.toList());
+ if (sortedDisks.size() != vmDisks.size()) {
+ LOGGER.error("Different number of root disks spec vs cloned deploy-as-is VM disks: " + sortedDisks.size() + " - " + vmDisks.size());
+ return;
+ }
+ for (int i = 0; i < sortedDisks.size(); i++) {
+ DiskTO specDisk = sortedDisks.get(i);
+ VirtualDisk vmDisk = vmDisks.get(i);
+ DataTO dataVolume = specDisk.getData();
+ if (dataVolume instanceof VolumeObjectTO) {
+ VolumeObjectTO volumeObjectTO = (VolumeObjectTO) dataVolume;
+ if (!volumeObjectTO.getSize().equals(vmDisk.getCapacityInBytes())) {
+ LOGGER.info("Mapped disk size is not the same as the cloned VM disk size: " +
+ volumeObjectTO.getSize() + " - " + vmDisk.getCapacityInBytes());
+ }
+ VirtualDeviceBackingInfo backingInfo = vmDisk.getBacking();
+ if (backingInfo instanceof VirtualDiskFlatVer2BackingInfo) {
+ VirtualDiskFlatVer2BackingInfo backing = (VirtualDiskFlatVer2BackingInfo) backingInfo;
+ String fileName = backing.getFileName();
+ if (StringUtils.isNotBlank(fileName)) {
+ String[] fileNameParts = fileName.split(" ");
+ String datastoreUuid = fileNameParts[0].replace("[", "").replace("]", "");
+ String relativePath = fileNameParts[1].split("/")[1].replace(".vmdk", "");
+ String vmSpecDatastoreUuid = volumeObjectTO.getDataStore().getUuid().replaceAll("-", "");
+ if (!datastoreUuid.equals(vmSpecDatastoreUuid)) {
+ LOGGER.info("Mapped disk datastore UUID is not the same as the cloned VM datastore UUID: " +
+ datastoreUuid + " - " + vmSpecDatastoreUuid);
+ }
+ volumeObjectTO.setPath(relativePath);
+ specDisk.setPath(relativePath);
+ }
+ }
+ }
+ }
+ }
+ } catch (Exception e) {
+ String msg = "Error mapping deploy-as-is VM disks from cloned VM " + vmInternalCSName;
+ LOGGER.error(msg, e);
+ throw new CloudRuntimeException(e);
+ }
+ }
+
+ private StartAnswer handleStartFailure(StartCommand cmd, VirtualMachineTO vmSpec, VirtualMachineData existingVm, DatacenterMO dcMo, Throwable e) {
+ if (e instanceof RemoteException) {
+ LOGGER.warn("Encounter remote exception to vCenter, invalidate VMware session context");
+ vmwareResource.invalidateServiceContext();
+ }
+
+ String msg = "StartCommand failed due to " + VmwareHelper.getExceptionMessage(e);
+ LOGGER.warn(msg, e);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("didn't start VM %s", vmSpec.getName()), e);
+ }
+ StartAnswer startAnswer = new StartAnswer(cmd, msg);
+ if ( e instanceof VmAlreadyExistsInVcenter) {
+ startAnswer.setContextParam("stopRetry", "true");
+ }
+ reRegisterExistingVm(existingVm, dcMo);
+
+ return startAnswer;
+ }
+
+ private void deleteOldVersionOfTheStartedVM(VirtualMachineData existingVm, DatacenterMO dcMo, VirtualMachineMO vmMo) throws Exception {
+ // Since VM was successfully powered-on, if there was an existing VM in a different cluster that was unregistered, delete all the files associated with it.
+ if (existingVm != null && existingVm.vmName != null && existingVm.vmFileLayout != null) {
+ List<String> vmDatastoreNames = new ArrayList<>();
+ for (DatastoreMO vmDatastore : vmMo.getAllDatastores()) {
+ vmDatastoreNames.add(vmDatastore.getName());
+ }
+ // Don't delete files that are in a datastore that is being used by the new VM as well (zone-wide datastore).
+ List<String> skipDatastores = new ArrayList<>();
+ for (DatastoreMO existingDatastore : existingVm.datastores) {
+ if (vmDatastoreNames.contains(existingDatastore.getName())) {
+ skipDatastores.add(existingDatastore.getName());
+ }
+ }
+ vmwareResource.deleteUnregisteredVmFiles(existingVm.vmFileLayout, dcMo, true, skipDatastores);
+ }
+ }
+
+ private int adjustNumberOfCoresPerSocket(VirtualMachineTO vmSpec, VirtualMachineMO vmMo, VirtualMachineConfigSpec vmConfigSpec) throws Exception {
+ // Check for multi-cores per socket settings
+ int numCoresPerSocket = 1;
+ String coresPerSocket = vmSpec.getDetails().get(VmDetailConstants.CPU_CORE_PER_SOCKET);
+ if (coresPerSocket != null) {
+ String apiVersion = HypervisorHostHelper.getVcenterApiVersion(vmMo.getContext());
+ // Property 'numCoresPerSocket' is supported since vSphere API 5.0
+ if (apiVersion.compareTo("5.0") >= 0) {
+ numCoresPerSocket = NumbersUtil.parseInt(coresPerSocket, 1);
+ vmConfigSpec.setNumCoresPerSocket(numCoresPerSocket);
+ }
+ }
+ return numCoresPerSocket;
+ }
+
+ /**
+ // Setup USB devices
+ */
+ private int setupUsbDevicesAndGetCount(String vmInternalCSName, VirtualMachineMO vmMo, String guestOsId, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount)
+ throws Exception {
+ int usbDeviceCount = 0;
+ if (guestOsId.startsWith("darwin")) { //Mac OS
+ VirtualDevice[] devices = vmMo.getMatchedDevices(new Class<?>[] {VirtualUSBController.class});
+ if (devices.length == 0) {
+ LOGGER.debug("No USB Controller device on VM Start. Add USB Controller device for Mac OS VM " + vmInternalCSName);
+
+ //For Mac OS X systems, the EHCI+UHCI controller is enabled by default and is required for USB mouse and keyboard access.
+ VirtualDevice usbControllerDevice = VmwareHelper.prepareUSBControllerDevice();
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ deviceConfigSpecArray[deviceCount].setDevice(usbControllerDevice);
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare USB controller at new device " + vmwareResource.getGson().toJson(deviceConfigSpecArray[deviceCount]));
+
+ deviceCount++;
+ usbDeviceCount++;
+ } else {
+ LOGGER.debug("USB Controller device exists on VM Start for Mac OS VM " + vmInternalCSName);
+ }
+ }
+ return usbDeviceCount;
+ }
+
+ private void createNewVm(VmwareContext context, VirtualMachineTO vmSpec, boolean installAsIs, String vmInternalCSName, String vmNameOnVcenter, Pair<String, String> controllerInfo, Boolean systemVm,
+ VmwareManager mgr, VmwareHypervisorHost hyperHost, String guestOsId, DiskTO[] disks, HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails,
+ DatastoreMO dsRootVolumeIsOn) throws Exception {
+ VirtualMachineMO vmMo;
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = getRootDiskDataStoreDetails(disks, dataStoresDetails);
+
+ assert (vmSpec.getMinSpeed() != null) && (rootDiskDataStoreDetails != null);
+
+ boolean vmFolderExists = rootDiskDataStoreDetails.second().folderExists(String.format("[%s]", rootDiskDataStoreDetails.second().getName()), vmNameOnVcenter);
+ String vmxFileFullPath = dsRootVolumeIsOn.searchFileInSubFolders(vmNameOnVcenter + ".vmx", false, VmwareManager.s_vmwareSearchExcludeFolder.value());
+ if (vmFolderExists && vmxFileFullPath != null) { // VM can be registered only if .vmx is present.
+ registerVm(vmNameOnVcenter, dsRootVolumeIsOn, vmwareResource);
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ if (vmMo != null) {
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Found registered vm " + vmInternalCSName + " at host " + hyperHost.getHyperHostName());
+ }
+ }
+ tearDownVm(vmMo);
+ } else if (installAsIs) {
+// first get all the MORs ManagedObjectReference morPool = hyperHost.getHyperHostOwnerResourcePool();
+// get the base VM vmMo = hyperHost.findVmOnHyperHost(vm.template.getPath());
+// do templateVm.createLinkedClone(vmInternalCSName, morBaseSnapshot, dcMo.getVmFolder(), morPool, morDatastore)
+// or hyperHost....createLinkedOrFullClone(templateVm, volume, dcMo, vmMo, morDatastore, dsMo, vmInternalCSName, morPool);
Review comment:
does this block of comments still hold value? if not
```suggestion
} else if (installAsIs) {
```
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/storage/resource/VmwareStorageProcessor.java
##########
@@ -477,19 +478,33 @@ private String getOVFFilePath(String srcOVAFileName) {
return null;
}
- private Pair<VirtualMachineMO, Long> copyTemplateFromSecondaryToPrimary(VmwareHypervisorHost hyperHost, DatastoreMO datastoreMo, String secondaryStorageUrl,
- String templatePathAtSecondaryStorage, String templateName, String templateUuid,
- boolean createSnapshot, Integer nfsVersion) throws Exception {
- s_logger.info("Executing copyTemplateFromSecondaryToPrimary. secondaryStorage: " + secondaryStorageUrl + ", templatePathAtSecondaryStorage: " +
- templatePathAtSecondaryStorage + ", templateName: " + templateName);
+ private String getOVFFile(String srcOVAFileName) {
+ File file = new File(srcOVAFileName);
+
+ String[] files = _storage.listFiles(file.getParent());
+ if (files != null) {
+ for (String fileName : files) {
+ if (fileName.toLowerCase().endsWith(".ovf")) {
+ File ovfFile = new File(fileName);
+ return ovfFile.getName();
+ }
+ }
+ }
+ return null;
+ }
+
+ private Pair<VirtualMachineMO, Long> copyTemplateFromSecondaryToPrimary(VmwareHypervisorHost hyperHost, DatastoreMO datastoreMo, String secondaryStorageUrl, String templatePathAtSecondaryStorage, String templateName, String templateUuid,
+ boolean createSnapshot, Integer nfsVersion, boolean deployAsIs) throws Exception {
+ s_logger.info(String.format("Executing copyTemplateFromSecondaryToPrimary. secondaryStorage: %s, templatePathAtSecondaryStorage: %s, templateName: %s, deployAsIs: %s",
+ secondaryStorageUrl, templatePathAtSecondaryStorage, templateName, deployAsIs));
String secondaryMountPoint = mountService.getMountPoint(secondaryStorageUrl, nfsVersion);
s_logger.info("Secondary storage mount point: " + secondaryMountPoint);
String srcOVAFileName =
VmwareStorageLayoutHelper.getTemplateOnSecStorageFilePath(secondaryMountPoint, templatePathAtSecondaryStorage, templateName,
ImageFormat.OVA.getFileExtension());
-
+ // FR37 consider extension: ova or ovf?
Review comment:
```suggestion
```
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/storage/resource/VmwareStorageProcessor.java
##########
@@ -818,64 +855,41 @@ public Answer cloneVolumeFromBaseTemplate(CopyCommand cmd) {
}
}
} else {
+ // FR37 check for deployasis and deploy from content library if possible
+ // FR37 check if we should be here for an existing VM
Review comment:
needs addressing
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/StartCommandExecutor.java
##########
@@ -0,0 +1,2247 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.hypervisor.vmware.resource;
+
+import java.io.File;
+import java.rmi.RemoteException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.cloud.agent.api.to.DataTO;
+import com.vmware.vim25.VirtualDiskFlatVer2BackingInfo;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.storage.configdrive.ConfigDrive;
+import org.apache.cloudstack.storage.to.TemplateObjectTO;
+import org.apache.cloudstack.storage.to.VolumeObjectTO;
+import org.apache.cloudstack.utils.volume.VirtualMachineDiskInfo;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.commons.lang.ArrayUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.log4j.Logger;
+
+import com.cloud.agent.api.Command;
+import com.cloud.agent.api.StartAnswer;
+import com.cloud.agent.api.StartCommand;
+import com.cloud.agent.api.storage.OVFPropertyTO;
+import com.cloud.agent.api.to.DataStoreTO;
+import com.cloud.agent.api.to.DiskTO;
+import com.cloud.agent.api.to.NfsTO;
+import com.cloud.agent.api.to.NicTO;
+import com.cloud.agent.api.to.VirtualMachineTO;
+import com.cloud.configuration.Resource;
+import com.cloud.hypervisor.vmware.VmwareResourceException;
+import com.cloud.hypervisor.vmware.manager.VmwareManager;
+import com.cloud.hypervisor.vmware.mo.CustomFieldConstants;
+import com.cloud.hypervisor.vmware.mo.DatacenterMO;
+import com.cloud.hypervisor.vmware.mo.DatastoreFile;
+import com.cloud.hypervisor.vmware.mo.DatastoreMO;
+import com.cloud.hypervisor.vmware.mo.DiskControllerType;
+import com.cloud.hypervisor.vmware.mo.HostMO;
+import com.cloud.hypervisor.vmware.mo.HypervisorHostHelper;
+import com.cloud.hypervisor.vmware.mo.TaskMO;
+import com.cloud.hypervisor.vmware.mo.VirtualEthernetCardType;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineDiskInfoBuilder;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineMO;
+import com.cloud.hypervisor.vmware.mo.VmwareHypervisorHost;
+import com.cloud.hypervisor.vmware.util.VmwareContext;
+import com.cloud.hypervisor.vmware.util.VmwareHelper;
+import com.cloud.network.Networks;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Volume;
+import com.cloud.storage.resource.VmwareStorageLayoutHelper;
+import com.cloud.storage.resource.VmwareStorageProcessor;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nicira.nvp.plugin.NiciraNvpApiVersion;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.VmDetailConstants;
+import com.vmware.vim25.BoolPolicy;
+import com.vmware.vim25.DVPortConfigInfo;
+import com.vmware.vim25.DVPortConfigSpec;
+import com.vmware.vim25.DasVmPriority;
+import com.vmware.vim25.DistributedVirtualPort;
+import com.vmware.vim25.DistributedVirtualSwitchPortConnection;
+import com.vmware.vim25.DistributedVirtualSwitchPortCriteria;
+import com.vmware.vim25.ManagedObjectReference;
+import com.vmware.vim25.OptionValue;
+import com.vmware.vim25.VMwareDVSPortSetting;
+import com.vmware.vim25.VirtualDevice;
+import com.vmware.vim25.VirtualDeviceBackingInfo;
+import com.vmware.vim25.VirtualDeviceConfigSpec;
+import com.vmware.vim25.VirtualDeviceConfigSpecOperation;
+import com.vmware.vim25.VirtualDisk;
+import com.vmware.vim25.VirtualEthernetCard;
+import com.vmware.vim25.VirtualEthernetCardDistributedVirtualPortBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardNetworkBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardOpaqueNetworkBackingInfo;
+import com.vmware.vim25.VirtualMachineBootOptions;
+import com.vmware.vim25.VirtualMachineConfigSpec;
+import com.vmware.vim25.VirtualMachineFileInfo;
+import com.vmware.vim25.VirtualMachineFileLayoutEx;
+import com.vmware.vim25.VirtualMachineGuestOsIdentifier;
+import com.vmware.vim25.VirtualUSBController;
+import com.vmware.vim25.VmConfigInfo;
+import com.vmware.vim25.VmwareDistributedVirtualSwitchVlanIdSpec;
+
+class StartCommandExecutor {
+ private static final Logger LOGGER = Logger.getLogger(StartCommandExecutor.class);
+
+ private final VmwareResource vmwareResource;
+
+ public StartCommandExecutor(VmwareResource vmwareResource) {
+ this.vmwareResource = vmwareResource;
+ }
+
+ // FR37 the monster blob god method: reduce to 320 so far and counting
+ protected StartAnswer execute(StartCommand cmd) {
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Executing resource StartCommand: " + vmwareResource.getGson().toJson(cmd));
+ }
+
+ VirtualMachineTO vmSpec = cmd.getVirtualMachine();
+
+ VirtualMachineData existingVm = null;
+
+ Pair<String, String> names = composeVmNames(vmSpec);
+ String vmInternalCSName = names.first();
+ String vmNameOnVcenter = names.second();
+
+ DiskTO rootDiskTO = null;
+
+ Pair<String, String> controllerInfo = getDiskControllerInfo(vmSpec);
+
+ Boolean systemVm = vmSpec.getType().isUsedBySystem();
+
+ VmwareContext context = vmwareResource.getServiceContext();
+ DatacenterMO dcMo = null;
+ try {
+ VmwareManager mgr = context.getStockObject(VmwareManager.CONTEXT_STOCK_NAME);
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(context);
+ dcMo = new DatacenterMO(hyperHost.getContext(), hyperHost.getHyperHostDatacenter());
+
+ // checkIfVmExistsInVcenter(vmInternalCSName, vmNameOnVcenter, dcMo);
+ // FR37 - We expect VM to be already cloned and available at this point
+ VirtualMachineMO vmMo = dcMo.findVm(vmInternalCSName);
+ if (vmMo == null) {
+ vmMo = dcMo.findVm(vmNameOnVcenter);
+ }
+
+ DiskTO[] specDisks = vmSpec.getDisks();
+ boolean installAsIs = StringUtils.isNotEmpty(vmSpec.getTemplateLocation());
+ // FR37 if startcommand contains enough info: a template url/-location and flag; deploy OVA as is
+ if (vmMo == null && installAsIs) {
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("deploying OVA from %s as is", vmSpec.getTemplateLocation()));
+ }
+ getStorageProcessor().cloneVMFromTemplate(vmSpec.getTemplateName(), vmInternalCSName, vmSpec.getTemplatePrimaryStoreUuid());
+ vmMo = dcMo.findVm(vmInternalCSName);
+ mapSpecDisksToClonedDisks(vmMo, vmInternalCSName, specDisks);
+ }
+
+ // VM may not have been on the same host, relocate to expected host
+ if (vmMo != null) {
+ vmMo.relocate(hyperHost.getMor());
+ // Get updated MO
+ vmMo = hyperHost.findVmOnHyperHost(vmMo.getVmName());
+ }
+
+ String guestOsId = translateGuestOsIdentifier(vmSpec.getArch(), vmSpec.getOs(), vmSpec.getPlatformEmulator()).value();
+ DiskTO[] disks = validateDisks(specDisks);
+ NicTO[] nics = vmSpec.getNics();
+
+ // FIXME: disks logic here, why is disks/volumes during copy not set with pool ID?
+ // FR37 TODO if deployasis a new VM no datastores are known (yet) and we need to get the data store from the tvmspec content library / template location
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails = inferDatastoreDetailsFromDiskInfo(hyperHost, context, disks, cmd);
+ if ((dataStoresDetails == null) || (dataStoresDetails.isEmpty())) {
+ String msg = "Unable to locate datastore details of the volumes to be attached";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ // FR37 - this may not be necessary the cloned VM will have disks and knowledge of datastore paths/location?
+ // throw new Exception(msg);
+ }
+
+ // FR37 - this may need checking, if the first datastore is the right one - ideally it should be datastore where first disk is hosted
+ DatastoreMO dsRootVolumeIsOn = null; //
+ if (! installAsIs) {
+ dsRootVolumeIsOn = getDatastoreThatRootDiskIsOn(dataStoresDetails, disks);
+
+ if (dsRootVolumeIsOn == null) {
+ String msg = "Unable to locate datastore details of root volume";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ throw new VmwareResourceException(msg);
+ }
+ }
+
+ VirtualMachineDiskInfoBuilder diskInfoBuilder = null;
+ VirtualDevice[] nicDevices = null;
+ DiskControllerType systemVmScsiControllerType = DiskControllerType.lsilogic;
+ int firstScsiControllerBusNum = 0;
+ int numScsiControllerForSystemVm = 1;
+ boolean hasSnapshot = false;
+ if (vmMo != null) {
+ PrepareRunningVMForConfiguration prepareRunningVMForConfiguration = new PrepareRunningVMForConfiguration(vmInternalCSName, controllerInfo, systemVm, vmMo,
+ systemVmScsiControllerType, firstScsiControllerBusNum, numScsiControllerForSystemVm);
+ prepareRunningVMForConfiguration.invoke();
+ diskInfoBuilder = prepareRunningVMForConfiguration.getDiskInfoBuilder();
+ nicDevices = prepareRunningVMForConfiguration.getNicDevices();
+ hasSnapshot = prepareRunningVMForConfiguration.isHasSnapshot();
+ } else {
+ ManagedObjectReference morDc = hyperHost.getHyperHostDatacenter();
+ assert (morDc != null);
+
+ vmMo = hyperHost.findVmOnPeerHyperHost(vmInternalCSName);
+ if (vmMo != null) {
+ VirtualMachineRecycler virtualMachineRecycler = new VirtualMachineRecycler(vmInternalCSName, controllerInfo, systemVm, hyperHost, vmMo,
+ systemVmScsiControllerType, firstScsiControllerBusNum, numScsiControllerForSystemVm);
+ virtualMachineRecycler.invoke();
+ diskInfoBuilder = virtualMachineRecycler.getDiskInfoBuilder();
+ hasSnapshot = virtualMachineRecycler.isHasSnapshot();
+ nicDevices = virtualMachineRecycler.getNicDevices();
+ } else {
+ // FR37 we just didn't find a VM by name 'vmInternalCSName', so why is this here?
+ existingVm = unregisterOnOtherClusterButHoldOnToOldVmData(vmInternalCSName, dcMo);
+
+ createNewVm(context, vmSpec, installAsIs, vmInternalCSName, vmNameOnVcenter, controllerInfo, systemVm, mgr, hyperHost, guestOsId, disks, dataStoresDetails,
+ dsRootVolumeIsOn);
+ }
+
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ if (vmMo == null) {
+ throw new Exception("Failed to find the newly create or relocated VM. vmName: " + vmInternalCSName);
+ }
+ }
+ // vmMo should now be a stopped VM on the intended host
+ // The number of disks changed must be 0 for install as is, as the VM is a clone
+ int disksChanges = !installAsIs ? disks.length : 0;
+ int totalChangeDevices = disksChanges + nics.length;
+ int hackDeviceCount = 0;
+ if (diskInfoBuilder != null) {
+ hackDeviceCount += diskInfoBuilder.getDiskCount();
+ }
+ hackDeviceCount += nicDevices == null ? 0 : nicDevices.length;
+ // vApp cdrom device
+ // HACK ALERT: ovf properties might not be the only or defining feature of vApps; needs checking
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("current count(s) desired: %d/ found:%d. now adding device to device count for vApp config ISO", totalChangeDevices, hackDeviceCount));
+ }
+ if (vmSpec.getOvfProperties() != null) {
+ totalChangeDevices++;
+ }
+
+ DiskTO volIso = null;
+ if (vmSpec.getType() != VirtualMachine.Type.User) {
+ // system VM needs a patch ISO
+ totalChangeDevices++;
+ } else {
+ volIso = getIsoDiskTO(disks);
+ if (volIso == null)
+ totalChangeDevices++;
+ }
+
+ VirtualMachineConfigSpec vmConfigSpec = new VirtualMachineConfigSpec();
+
+ VmwareHelper.setBasicVmConfig(vmConfigSpec, vmSpec.getCpus(), vmSpec.getMaxSpeed(), vmwareResource.getReservedCpuMHZ(vmSpec), (int)(vmSpec.getMaxRam() / (1024 * 1024)),
+ vmwareResource.getReservedMemoryMb(vmSpec), guestOsId, vmSpec.getLimitCpuUse());
+
+ int numCoresPerSocket = adjustNumberOfCoresPerSocket(vmSpec, vmMo, vmConfigSpec);
+
+ // Check for hotadd settings
+ vmConfigSpec.setMemoryHotAddEnabled(vmMo.isMemoryHotAddSupported(guestOsId));
+
+ String hostApiVersion = ((HostMO)hyperHost).getHostAboutInfo().getApiVersion();
+ if (numCoresPerSocket > 1 && hostApiVersion.compareTo("5.0") < 0) {
+ LOGGER.warn("Dynamic scaling of CPU is not supported for Virtual Machines with multi-core vCPUs in case of ESXi hosts 4.1 and prior. Hence CpuHotAdd will not be"
+ + " enabled for Virtual Machine: " + vmInternalCSName);
+ vmConfigSpec.setCpuHotAddEnabled(false);
+ } else {
+ vmConfigSpec.setCpuHotAddEnabled(vmMo.isCpuHotAddSupported(guestOsId));
+ }
+
+ vmwareResource.configNestedHVSupport(vmMo, vmSpec, vmConfigSpec);
+
+ VirtualDeviceConfigSpec[] deviceConfigSpecArray = new VirtualDeviceConfigSpec[totalChangeDevices];
+ int deviceCount = 0;
+ int ideUnitNumber = 0;
+ int scsiUnitNumber = 0;
+ int ideControllerKey = vmMo.getIDEDeviceControllerKey();
+ int scsiControllerKey = vmMo.getScsiDeviceControllerKeyNoException();
+
+ IsoSetup isoSetup = new IsoSetup(vmSpec, mgr, hyperHost, vmMo, disks, volIso, deviceConfigSpecArray, deviceCount, ideUnitNumber);
+ isoSetup.invoke();
+ deviceCount = isoSetup.getDeviceCount();
+ ideUnitNumber = isoSetup.getIdeUnitNumber();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("setting up %d disks with root from %s", diskInfoBuilder.getDiskCount(), vmwareResource.getGson().toJson(rootDiskTO)));
+ }
+
+ DiskSetup diskSetup = new DiskSetup(vmSpec, rootDiskTO, controllerInfo, context, dcMo, hyperHost, vmMo, disks, dataStoresDetails, diskInfoBuilder, hasSnapshot,
+ deviceConfigSpecArray, deviceCount, ideUnitNumber, scsiUnitNumber, ideControllerKey, scsiControllerKey, installAsIs);
+ diskSetup.invoke();
+
+ rootDiskTO = diskSetup.getRootDiskTO();
+ deviceCount = diskSetup.getDeviceCount();
+ DiskTO[] sortedDisks = diskSetup.getSortedDisks();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("seting up %d disks with root from %s", sortedDisks.length, vmwareResource.getGson().toJson(rootDiskTO)));
+ }
+
+ deviceCount += setupUsbDevicesAndGetCount(vmInternalCSName, vmMo, guestOsId, deviceConfigSpecArray, deviceCount);
+
+ NicSetup nicSetup = new NicSetup(cmd, vmSpec, vmInternalCSName, context, mgr, hyperHost, vmMo, nics, deviceConfigSpecArray, deviceCount, nicDevices);
+ nicSetup.invoke();
+
+ deviceCount = nicSetup.getDeviceCount();
+ int nicMask = nicSetup.getNicMask();
+ int nicCount = nicSetup.getNicCount();
+ Map<String, String> nicUuidToDvSwitchUuid = nicSetup.getNicUuidToDvSwitchUuid();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("device count: %d; nic count %d", deviceCount, nicCount));
+ }
+ for (int j = 0; j < deviceCount; j++)
+ vmConfigSpec.getDeviceChange().add(deviceConfigSpecArray[j]);
+
+ //
+ // Setup VM options
+ //
+
+ // pass boot arguments through machine.id & perform customized options to VMX
+ ArrayList<OptionValue> extraOptions = new ArrayList<>();
+ configBasicExtraOption(extraOptions, vmSpec);
+ configNvpExtraOption(extraOptions, vmSpec, nicUuidToDvSwitchUuid);
+ configCustomExtraOption(extraOptions, vmSpec);
+
+ // config for NCC
+ VirtualMachine.Type vmType = cmd.getVirtualMachine().getType();
+ if (vmType.equals(VirtualMachine.Type.NetScalerVm)) {
+ NicTO mgmtNic = vmSpec.getNics()[0];
+ OptionValue option = new OptionValue();
+ option.setKey("machine.id");
+ option.setValue("ip=" + mgmtNic.getIp() + "&netmask=" + mgmtNic.getNetmask() + "&gateway=" + mgmtNic.getGateway());
+ extraOptions.add(option);
+ }
+
+ // config VNC
+ String keyboardLayout = null;
+ if (vmSpec.getDetails() != null)
+ keyboardLayout = vmSpec.getDetails().get(VmDetailConstants.KEYBOARD);
+ vmConfigSpec.getExtraConfig().addAll(
+ Arrays.asList(vmwareResource.configureVnc(extraOptions.toArray(new OptionValue[0]), hyperHost, vmInternalCSName, vmSpec.getVncPassword(), keyboardLayout)));
+
+ // config video card
+ vmwareResource.configureVideoCard(vmMo, vmSpec, vmConfigSpec);
+
+ // Set OVF properties (if available)
+ Pair<String, List<OVFPropertyTO>> ovfPropsMap = vmSpec.getOvfProperties();
+ VmConfigInfo templateVappConfig;
+ List<OVFPropertyTO> ovfProperties;
+ if (ovfPropsMap != null) {
+ String vmTemplate = ovfPropsMap.first();
+ LOGGER.info("Find VM template " + vmTemplate);
+ VirtualMachineMO vmTemplateMO = dcMo.findVm(vmTemplate);
+ templateVappConfig = vmTemplateMO.getConfigInfo().getVAppConfig();
+ ovfProperties = ovfPropsMap.second();
+ // Set OVF properties (if available)
+ if (CollectionUtils.isNotEmpty(ovfProperties)) {
+ LOGGER.info("Copying OVF properties from template and setting them to the values the user provided");
+ vmwareResource.copyVAppConfigsFromTemplate(templateVappConfig, ovfProperties, vmConfigSpec);
+ }
+ }
+
+ checkBootOptions(vmSpec, vmConfigSpec);
+
+ //
+ // Configure VM
+ //
+ if (!vmMo.configureVm(vmConfigSpec)) {
+ throw new Exception("Failed to configure VM before start. vmName: " + vmInternalCSName);
+ }
+ // FR37 TODO reconcile disks now!!! they are configured, check and add them to the returning vmTO
+ if (vmSpec.getType() == VirtualMachine.Type.DomainRouter) {
+ hyperHost.setRestartPriorityForVM(vmMo, DasVmPriority.HIGH.value());
+ }
+
+ // Resizing root disk only when explicit requested by user
+ final Map<String, String> vmDetails = cmd.getVirtualMachine().getDetails();
+ if (rootDiskTO != null && !hasSnapshot && (vmDetails != null && vmDetails.containsKey(ApiConstants.ROOT_DISK_SIZE))) {
+ resizeRootDiskOnVMStart(vmMo, rootDiskTO, hyperHost, context);
+ }
+
+ //
+ // Post Configuration
+ //
+
+ vmMo.setCustomFieldValue(CustomFieldConstants.CLOUD_NIC_MASK, String.valueOf(nicMask));
+ postNvpConfigBeforeStart(vmMo, vmSpec);
+
+ Map<String, Map<String, String>> iqnToData = new HashMap<>();
+
+ postDiskConfigBeforeStart(vmMo, vmSpec, sortedDisks, iqnToData, hyperHost, context, installAsIs);
+
+ //
+ // Power-on VM
+ //
+ if (!vmMo.powerOn()) {
+ throw new Exception("Failed to start VM. vmName: " + vmInternalCSName + " with hostname " + vmNameOnVcenter);
+ }
+
+ if (installAsIs) {
+ // Set disks as the disks path and chain is retrieved from the cloned VM disks
+ cmd.getVirtualMachine().setDisks(disks);
+ }
+
+ StartAnswer startAnswer = new StartAnswer(cmd);
+
+ startAnswer.setIqnToData(iqnToData);
+
+ deleteOldVersionOfTheStartedVM(existingVm, dcMo, vmMo);
+
+ return startAnswer;
+ } catch (Throwable e) {
+ return handleStartFailure(cmd, vmSpec, existingVm, dcMo, e);
+ } finally {
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("finally done with %s", vmwareResource.getGson().toJson(cmd)));
+ }
+ }
+ }
+
+ /**
+ * Modify the specDisks information to match the cloned VM's disks (from vmMo VM)
+ */
+ private void mapSpecDisksToClonedDisks(VirtualMachineMO vmMo, String vmInternalCSName, DiskTO[] specDisks) {
+ try {
+ if (vmMo != null && ArrayUtils.isNotEmpty(specDisks)) {
+ List<VirtualDisk> vmDisks = vmMo.getVirtualDisks();
+ List<DiskTO> sortedDisks = Arrays.asList(sortVolumesByDeviceId(specDisks))
+ .stream()
+ .filter(x -> x.getType() == Volume.Type.ROOT)
+ .collect(Collectors.toList());
+ if (sortedDisks.size() != vmDisks.size()) {
+ LOGGER.error("Different number of root disks spec vs cloned deploy-as-is VM disks: " + sortedDisks.size() + " - " + vmDisks.size());
+ return;
+ }
+ for (int i = 0; i < sortedDisks.size(); i++) {
+ DiskTO specDisk = sortedDisks.get(i);
+ VirtualDisk vmDisk = vmDisks.get(i);
+ DataTO dataVolume = specDisk.getData();
+ if (dataVolume instanceof VolumeObjectTO) {
+ VolumeObjectTO volumeObjectTO = (VolumeObjectTO) dataVolume;
+ if (!volumeObjectTO.getSize().equals(vmDisk.getCapacityInBytes())) {
+ LOGGER.info("Mapped disk size is not the same as the cloned VM disk size: " +
+ volumeObjectTO.getSize() + " - " + vmDisk.getCapacityInBytes());
+ }
+ VirtualDeviceBackingInfo backingInfo = vmDisk.getBacking();
+ if (backingInfo instanceof VirtualDiskFlatVer2BackingInfo) {
+ VirtualDiskFlatVer2BackingInfo backing = (VirtualDiskFlatVer2BackingInfo) backingInfo;
+ String fileName = backing.getFileName();
+ if (StringUtils.isNotBlank(fileName)) {
+ String[] fileNameParts = fileName.split(" ");
+ String datastoreUuid = fileNameParts[0].replace("[", "").replace("]", "");
+ String relativePath = fileNameParts[1].split("/")[1].replace(".vmdk", "");
+ String vmSpecDatastoreUuid = volumeObjectTO.getDataStore().getUuid().replaceAll("-", "");
+ if (!datastoreUuid.equals(vmSpecDatastoreUuid)) {
+ LOGGER.info("Mapped disk datastore UUID is not the same as the cloned VM datastore UUID: " +
+ datastoreUuid + " - " + vmSpecDatastoreUuid);
+ }
+ volumeObjectTO.setPath(relativePath);
+ specDisk.setPath(relativePath);
+ }
+ }
+ }
+ }
+ }
+ } catch (Exception e) {
+ String msg = "Error mapping deploy-as-is VM disks from cloned VM " + vmInternalCSName;
+ LOGGER.error(msg, e);
+ throw new CloudRuntimeException(e);
+ }
+ }
+
+ private StartAnswer handleStartFailure(StartCommand cmd, VirtualMachineTO vmSpec, VirtualMachineData existingVm, DatacenterMO dcMo, Throwable e) {
+ if (e instanceof RemoteException) {
+ LOGGER.warn("Encounter remote exception to vCenter, invalidate VMware session context");
+ vmwareResource.invalidateServiceContext();
+ }
+
+ String msg = "StartCommand failed due to " + VmwareHelper.getExceptionMessage(e);
+ LOGGER.warn(msg, e);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("didn't start VM %s", vmSpec.getName()), e);
+ }
+ StartAnswer startAnswer = new StartAnswer(cmd, msg);
+ if ( e instanceof VmAlreadyExistsInVcenter) {
+ startAnswer.setContextParam("stopRetry", "true");
+ }
+ reRegisterExistingVm(existingVm, dcMo);
+
+ return startAnswer;
+ }
+
+ private void deleteOldVersionOfTheStartedVM(VirtualMachineData existingVm, DatacenterMO dcMo, VirtualMachineMO vmMo) throws Exception {
+ // Since VM was successfully powered-on, if there was an existing VM in a different cluster that was unregistered, delete all the files associated with it.
+ if (existingVm != null && existingVm.vmName != null && existingVm.vmFileLayout != null) {
+ List<String> vmDatastoreNames = new ArrayList<>();
+ for (DatastoreMO vmDatastore : vmMo.getAllDatastores()) {
+ vmDatastoreNames.add(vmDatastore.getName());
+ }
+ // Don't delete files that are in a datastore that is being used by the new VM as well (zone-wide datastore).
+ List<String> skipDatastores = new ArrayList<>();
+ for (DatastoreMO existingDatastore : existingVm.datastores) {
+ if (vmDatastoreNames.contains(existingDatastore.getName())) {
+ skipDatastores.add(existingDatastore.getName());
+ }
+ }
+ vmwareResource.deleteUnregisteredVmFiles(existingVm.vmFileLayout, dcMo, true, skipDatastores);
+ }
+ }
+
+ private int adjustNumberOfCoresPerSocket(VirtualMachineTO vmSpec, VirtualMachineMO vmMo, VirtualMachineConfigSpec vmConfigSpec) throws Exception {
+ // Check for multi-cores per socket settings
+ int numCoresPerSocket = 1;
+ String coresPerSocket = vmSpec.getDetails().get(VmDetailConstants.CPU_CORE_PER_SOCKET);
+ if (coresPerSocket != null) {
+ String apiVersion = HypervisorHostHelper.getVcenterApiVersion(vmMo.getContext());
+ // Property 'numCoresPerSocket' is supported since vSphere API 5.0
+ if (apiVersion.compareTo("5.0") >= 0) {
+ numCoresPerSocket = NumbersUtil.parseInt(coresPerSocket, 1);
+ vmConfigSpec.setNumCoresPerSocket(numCoresPerSocket);
+ }
+ }
+ return numCoresPerSocket;
+ }
+
+ /**
+ // Setup USB devices
+ */
+ private int setupUsbDevicesAndGetCount(String vmInternalCSName, VirtualMachineMO vmMo, String guestOsId, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount)
+ throws Exception {
+ int usbDeviceCount = 0;
+ if (guestOsId.startsWith("darwin")) { //Mac OS
+ VirtualDevice[] devices = vmMo.getMatchedDevices(new Class<?>[] {VirtualUSBController.class});
+ if (devices.length == 0) {
+ LOGGER.debug("No USB Controller device on VM Start. Add USB Controller device for Mac OS VM " + vmInternalCSName);
+
+ //For Mac OS X systems, the EHCI+UHCI controller is enabled by default and is required for USB mouse and keyboard access.
+ VirtualDevice usbControllerDevice = VmwareHelper.prepareUSBControllerDevice();
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ deviceConfigSpecArray[deviceCount].setDevice(usbControllerDevice);
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare USB controller at new device " + vmwareResource.getGson().toJson(deviceConfigSpecArray[deviceCount]));
+
+ deviceCount++;
+ usbDeviceCount++;
+ } else {
+ LOGGER.debug("USB Controller device exists on VM Start for Mac OS VM " + vmInternalCSName);
+ }
+ }
+ return usbDeviceCount;
+ }
+
+ private void createNewVm(VmwareContext context, VirtualMachineTO vmSpec, boolean installAsIs, String vmInternalCSName, String vmNameOnVcenter, Pair<String, String> controllerInfo, Boolean systemVm,
+ VmwareManager mgr, VmwareHypervisorHost hyperHost, String guestOsId, DiskTO[] disks, HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails,
+ DatastoreMO dsRootVolumeIsOn) throws Exception {
+ VirtualMachineMO vmMo;
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = getRootDiskDataStoreDetails(disks, dataStoresDetails);
+
+ assert (vmSpec.getMinSpeed() != null) && (rootDiskDataStoreDetails != null);
+
+ boolean vmFolderExists = rootDiskDataStoreDetails.second().folderExists(String.format("[%s]", rootDiskDataStoreDetails.second().getName()), vmNameOnVcenter);
+ String vmxFileFullPath = dsRootVolumeIsOn.searchFileInSubFolders(vmNameOnVcenter + ".vmx", false, VmwareManager.s_vmwareSearchExcludeFolder.value());
+ if (vmFolderExists && vmxFileFullPath != null) { // VM can be registered only if .vmx is present.
+ registerVm(vmNameOnVcenter, dsRootVolumeIsOn, vmwareResource);
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ if (vmMo != null) {
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Found registered vm " + vmInternalCSName + " at host " + hyperHost.getHyperHostName());
+ }
+ }
+ tearDownVm(vmMo);
+ } else if (installAsIs) {
+// first get all the MORs ManagedObjectReference morPool = hyperHost.getHyperHostOwnerResourcePool();
+// get the base VM vmMo = hyperHost.findVmOnHyperHost(vm.template.getPath());
+// do templateVm.createLinkedClone(vmInternalCSName, morBaseSnapshot, dcMo.getVmFolder(), morPool, morDatastore)
+// or hyperHost....createLinkedOrFullClone(templateVm, volume, dcMo, vmMo, morDatastore, dsMo, vmInternalCSName, morPool);
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ // At this point vmMo points to the cloned VM
+ } else {
+ if (!hyperHost
+ .createBlankVm(vmNameOnVcenter, vmInternalCSName, vmSpec.getCpus(), vmSpec.getMaxSpeed(), vmwareResource.getReservedCpuMHZ(vmSpec), vmSpec.getLimitCpuUse(), (int)(vmSpec.getMaxRam() / Resource.ResourceType.bytesToMiB), vmwareResource.getReservedMemoryMb(vmSpec), guestOsId,
+ rootDiskDataStoreDetails.first(), false, controllerInfo, systemVm)) {
+ throw new Exception("Failed to create VM. vmName: " + vmInternalCSName);
+ }
+ }
+ }
+
+ /**
+ * If a VM with the same name is found in a different cluster in the DC, unregister the old VM and configure a new VM (cold-migration).
+ */
+ private VirtualMachineData unregisterOnOtherClusterButHoldOnToOldVmData(String vmInternalCSName, DatacenterMO dcMo) throws Exception {
+ VirtualMachineMO existingVmInDc = dcMo.findVm(vmInternalCSName);
+ VirtualMachineData existingVm = null;
+ if (existingVmInDc != null) {
+ existingVm = new VirtualMachineData();
+ existingVm.vmName = existingVmInDc.getName();
+ existingVm.vmFileInfo = existingVmInDc.getFileInfo();
+ existingVm.vmFileLayout = existingVmInDc.getFileLayout();
+ existingVm.datastores = existingVmInDc.getAllDatastores();
+ LOGGER.info("Found VM: " + vmInternalCSName + " on a host in a different cluster. Unregistering the exisitng VM.");
+ existingVmInDc.unregisterVm();
+ }
+ return existingVm;
+ }
+
+ private Pair<ManagedObjectReference, DatastoreMO> getRootDiskDataStoreDetails(DiskTO[] disks, HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails) {
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = null;
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ROOT) {
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (managed) {
+ String datastoreName = VmwareResource.getDatastoreName(details.get(DiskTO.IQN));
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(datastoreName);
+ } else {
+ DataStoreTO primaryStore = vol.getData().getDataStore();
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(primaryStore.getUuid());
+ }
+ }
+ }
+ return rootDiskDataStoreDetails;
+ }
+
+ /**
+ * Since VM start failed, if there was an existing VM in a different cluster that was unregistered, register it back.
+ *
+ * @param dcMo is guaranteed to be not null since we have noticed there is an existing VM in the dc (using that mo)
+ */
+ private void reRegisterExistingVm(VirtualMachineData existingVm, DatacenterMO dcMo) {
+ if (existingVm != null && existingVm.vmName != null && existingVm.vmFileInfo != null) {
+ LOGGER.debug("Since VM start failed, registering back an existing VM: " + existingVm.vmName + " that was unregistered");
+ try {
+ DatastoreFile fileInDatastore = new DatastoreFile(existingVm.vmFileInfo.getVmPathName());
+ DatastoreMO existingVmDsMo = new DatastoreMO(dcMo.getContext(), dcMo.findDatastore(fileInDatastore.getDatastoreName()));
+ registerVm(existingVm.vmName, existingVmDsMo, vmwareResource);
+ } catch (Exception ex) {
+ String message = "Failed to register an existing VM: " + existingVm.vmName + " due to " + VmwareHelper.getExceptionMessage(ex);
+ LOGGER.warn(message, ex);
+ }
+ }
+ }
+
+ private void checkIfVmExistsInVcenter(String vmInternalCSName, String vmNameOnVcenter, DatacenterMO dcMo) throws VmAlreadyExistsInVcenter, Exception {
+ // Validate VM name is unique in Datacenter
+ VirtualMachineMO vmInVcenter = dcMo.checkIfVmAlreadyExistsInVcenter(vmNameOnVcenter, vmInternalCSName);
+ if (vmInVcenter != null) {
+ String msg = "VM with name: " + vmNameOnVcenter + " already exists in vCenter.";
+ LOGGER.error(msg);
+ throw new VmAlreadyExistsInVcenter(msg);
+ }
+ }
+
+ private Pair<String, String> getDiskControllerInfo(VirtualMachineTO vmSpec) {
+ String dataDiskController = vmSpec.getDetails().get(VmDetailConstants.DATA_DISK_CONTROLLER);
+ String rootDiskController = vmSpec.getDetails().get(VmDetailConstants.ROOT_DISK_CONTROLLER);
+ // If root disk controller is scsi, then data disk controller would also be scsi instead of using 'osdefault'
+ // This helps avoid mix of different scsi subtype controllers in instance.
+ if (DiskControllerType.osdefault == DiskControllerType.getType(dataDiskController) && DiskControllerType.lsilogic == DiskControllerType.getType(rootDiskController)) {
+ dataDiskController = DiskControllerType.scsi.toString();
+ }
+
+ // Validate the controller types
+ dataDiskController = DiskControllerType.getType(dataDiskController).toString();
+ rootDiskController = DiskControllerType.getType(rootDiskController).toString();
+
+ if (DiskControllerType.getType(rootDiskController) == DiskControllerType.none) {
+ throw new CloudRuntimeException("Invalid root disk controller detected : " + rootDiskController);
+ }
+ if (DiskControllerType.getType(dataDiskController) == DiskControllerType.none) {
+ throw new CloudRuntimeException("Invalid data disk controller detected : " + dataDiskController);
+ }
+
+ return new Pair<>(rootDiskController, dataDiskController);
+ }
+
+ /**
+ * Registers the vm to the inventory given the vmx file.
+ * @param vmName
+ * @param dsMo
+ * @param vmwareResource
+ */
+ private void registerVm(String vmName, DatastoreMO dsMo, VmwareResource vmwareResource) throws Exception {
+
+ //1st param
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(vmwareResource.getServiceContext());
+ ManagedObjectReference dcMor = hyperHost.getHyperHostDatacenter();
+ DatacenterMO dataCenterMo = new DatacenterMO(vmwareResource.getServiceContext(), dcMor);
+ ManagedObjectReference vmFolderMor = dataCenterMo.getVmFolder();
+
+ //2nd param
+ String vmxFilePath = dsMo.searchFileInSubFolders(vmName + ".vmx", false, VmwareManager.s_vmwareSearchExcludeFolder.value());
+
+ // 5th param
+ ManagedObjectReference morPool = hyperHost.getHyperHostOwnerResourcePool();
+
+ ManagedObjectReference morTask = vmwareResource.getServiceContext().getService().registerVMTask(vmFolderMor, vmxFilePath, vmName, false, morPool, hyperHost.getMor());
+ boolean result = vmwareResource.getServiceContext().getVimClient().waitForTask(morTask);
+ if (!result) {
+ throw new Exception("Unable to register vm due to " + TaskMO.getTaskFailureInfo(vmwareResource.getServiceContext(), morTask));
+ } else {
+ vmwareResource.getServiceContext().waitForTaskProgressDone(morTask);
+ }
+
+ }
+
+ // Pair<internal CS name, vCenter display name>
+ private Pair<String, String> composeVmNames(VirtualMachineTO vmSpec) {
+ String vmInternalCSName = vmSpec.getName();
+ String vmNameOnVcenter = vmSpec.getName();
+ if (VmwareResource.instanceNameFlag && vmSpec.getHostName() != null) {
+ vmNameOnVcenter = vmSpec.getHostName();
+ }
+ return new Pair<String, String>(vmInternalCSName, vmNameOnVcenter);
+ }
+
+ private VirtualMachineGuestOsIdentifier translateGuestOsIdentifier(String cpuArchitecture, String guestOs, String cloudGuestOs) {
+ if (cpuArchitecture == null) {
+ LOGGER.warn("CPU arch is not set, default to i386. guest os: " + guestOs);
+ cpuArchitecture = "i386";
+ }
+
+ if (cloudGuestOs == null) {
+ LOGGER.warn("Guest OS mapping name is not set for guest os: " + guestOs);
+ }
+
+ VirtualMachineGuestOsIdentifier identifier = null;
+ try {
+ if (cloudGuestOs != null) {
+ identifier = VirtualMachineGuestOsIdentifier.fromValue(cloudGuestOs);
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Using mapping name : " + identifier.toString());
+ }
+ }
+ } catch (IllegalArgumentException e) {
+ LOGGER.warn("Unable to find Guest OS Identifier in VMware for mapping name: " + cloudGuestOs + ". Continuing with defaults.");
+ }
+ if (identifier != null) {
+ return identifier;
+ }
+
+ if (cpuArchitecture.equalsIgnoreCase("x86_64")) {
+ return VirtualMachineGuestOsIdentifier.OTHER_GUEST_64;
+ }
+ return VirtualMachineGuestOsIdentifier.OTHER_GUEST;
+ }
+
+ private DiskTO[] validateDisks(DiskTO[] disks) {
+ List<DiskTO> validatedDisks = new ArrayList<DiskTO>();
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() != Volume.Type.ISO) {
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ if (primaryStore.getUuid() != null && !primaryStore.getUuid().isEmpty()) {
+ validatedDisks.add(vol);
+ }
+ } else if (vol.getType() == Volume.Type.ISO) {
+ TemplateObjectTO templateTO = (TemplateObjectTO) vol.getData();
+ if (templateTO.getPath() != null && !templateTO.getPath().isEmpty()) {
+ validatedDisks.add(vol);
+ }
+ } else {
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Drop invalid disk option, volumeTO: " + vmwareResource.getGson().toJson(vol));
+ }
+ }
+ }
+ Collections.sort(validatedDisks, (d1, d2) -> d1.getDiskSeq().compareTo(d2.getDiskSeq()));
+ return validatedDisks.toArray(new DiskTO[0]);
+ }
+
+ private HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> inferDatastoreDetailsFromDiskInfo(VmwareHypervisorHost hyperHost, VmwareContext context,
+ DiskTO[] disks, Command cmd) throws Exception {
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> mapIdToMors = new HashMap<>();
+
+ assert (hyperHost != null) && (context != null);
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() != Volume.Type.ISO) {
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ String poolUuid = primaryStore.getUuid();
+
+ if (mapIdToMors.get(poolUuid) == null) {
+ boolean isManaged = false;
+ Map<String, String> details = vol.getDetails();
+
+ if (details != null) {
+ isManaged = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (isManaged) {
+ String iScsiName = details.get(DiskTO.IQN); // details should not be null for managed storage (it may or may not be null for non-managed storage)
+ String datastoreName = VmwareResource.getDatastoreName(iScsiName);
+ ManagedObjectReference morDatastore = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, datastoreName);
+
+ // if the datastore is not present, we need to discover the iSCSI device that will support it,
+ // create the datastore, and create a VMDK file in the datastore
+ if (morDatastore == null) {
+ final String vmdkPath = vmwareResource.getVmdkPath(volumeTO.getPath());
+
+ morDatastore = getStorageProcessor().prepareManagedStorage(context, hyperHost, null, iScsiName,
+ details.get(DiskTO.STORAGE_HOST), Integer.parseInt(details.get(DiskTO.STORAGE_PORT)),
+ vmdkPath,
+ details.get(DiskTO.CHAP_INITIATOR_USERNAME), details.get(DiskTO.CHAP_INITIATOR_SECRET),
+ details.get(DiskTO.CHAP_TARGET_USERNAME), details.get(DiskTO.CHAP_TARGET_SECRET),
+ Long.parseLong(details.get(DiskTO.VOLUME_SIZE)), cmd);
+
+ DatastoreMO dsMo = new DatastoreMO(vmwareResource.getServiceContext(), morDatastore);
+
+ final String datastoreVolumePath;
+
+ if (vmdkPath != null) {
+ datastoreVolumePath = dsMo.getDatastorePath(vmdkPath + VmwareResource.VMDK_EXTENSION);
+ } else {
+ datastoreVolumePath = dsMo.getDatastorePath(dsMo.getName() + VmwareResource.VMDK_EXTENSION);
+ }
+
+ volumeTO.setPath(datastoreVolumePath);
+ vol.setPath(datastoreVolumePath);
+ }
+
+ mapIdToMors.put(datastoreName, new Pair<>(morDatastore, new DatastoreMO(context, morDatastore)));
+ } else {
+ ManagedObjectReference morDatastore = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, poolUuid);
+
+ if (morDatastore == null) {
+ String msg = "Failed to get the mounted datastore for the volume's pool " + poolUuid;
+
+ LOGGER.error(msg);
+
+ throw new Exception(msg);
+ }
+
+ mapIdToMors.put(poolUuid, new Pair<>(morDatastore, new DatastoreMO(context, morDatastore)));
+ }
+ }
+ }
+ }
+
+ return mapIdToMors;
+ }
+
+ private VmwareStorageProcessor getStorageProcessor() {
+ return vmwareResource.getStorageProcessor();
+ }
+
+ private DatastoreMO getDatastoreThatRootDiskIsOn(HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails, DiskTO disks[]) {
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = null;
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ROOT) {
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (managed) {
+ String datastoreName = VmwareResource.getDatastoreName(details.get(DiskTO.IQN));
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(datastoreName);
+
+ break;
+ } else {
+ DataStoreTO primaryStore = vol.getData().getDataStore();
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(primaryStore.getUuid());
+
+ break;
+ }
+ }
+ }
+
+ if (rootDiskDataStoreDetails != null) {
+ return rootDiskDataStoreDetails.second();
+ }
+
+ return null;
+ }
+
+ private void ensureDiskControllers(VirtualMachineMO vmMo, Pair<String, String> controllerInfo) throws Exception {
+ if (vmMo == null) {
+ return;
+ }
+
+ String msg;
+ String rootDiskController = controllerInfo.first();
+ String dataDiskController = controllerInfo.second();
+ String scsiDiskController;
+ String recommendedDiskController = null;
+
+ if (VmwareHelper.isControllerOsRecommended(dataDiskController) || VmwareHelper.isControllerOsRecommended(rootDiskController)) {
+ recommendedDiskController = vmMo.getRecommendedDiskController(null);
+ }
+ scsiDiskController = HypervisorHostHelper.getScsiController(new Pair<String, String>(rootDiskController, dataDiskController), recommendedDiskController);
+ if (scsiDiskController == null) {
+ return;
+ }
+
+ vmMo.getScsiDeviceControllerKeyNoException();
+ // This VM needs SCSI controllers.
+ // Get count of existing scsi controllers. Helps not to attempt to create more than the maximum allowed 4
+ // Get maximum among the bus numbers in use by scsi controllers. Safe to pick maximum, because we always go sequential allocating bus numbers.
+ Ternary<Integer, Integer, DiskControllerType> scsiControllerInfo = vmMo.getScsiControllerInfo();
+ int requiredNumScsiControllers = VmwareHelper.MAX_SCSI_CONTROLLER_COUNT - scsiControllerInfo.first();
+ int availableBusNum = scsiControllerInfo.second() + 1; // method returned current max. bus number
+
+ if (requiredNumScsiControllers == 0) {
+ return;
+ }
+ if (scsiControllerInfo.first() > 0) {
+ // For VMs which already have a SCSI controller, do NOT attempt to add any more SCSI controllers & return the sub type.
+ // For Legacy VMs would have only 1 LsiLogic Parallel SCSI controller, and doesn't require more.
+ // For VMs created post device ordering support, 4 SCSI subtype controllers are ensured during deployment itself. No need to add more.
+ // For fresh VM deployment only, all required controllers should be ensured.
+ return;
+ }
+ ensureScsiDiskControllers(vmMo, scsiDiskController, requiredNumScsiControllers, availableBusNum);
+ }
+
+ private void ensureScsiDiskControllers(VirtualMachineMO vmMo, String scsiDiskController, int requiredNumScsiControllers, int availableBusNum) throws Exception {
+ // Pick the sub type of scsi
+ if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.pvscsi) {
+ if (!vmMo.isPvScsiSupported()) {
+ String msg = "This VM doesn't support Vmware Paravirtual SCSI controller for virtual disks, because the virtual hardware version is less than 7.";
+ throw new Exception(msg);
+ }
+ vmMo.ensurePvScsiDeviceController(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.lsisas1068) {
+ vmMo.ensureLsiLogicSasDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.buslogic) {
+ vmMo.ensureBusLogicDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.lsilogic) {
+ vmMo.ensureLsiLogicDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ }
+ }
+
+ private void tearDownVm(VirtualMachineMO vmMo) throws Exception {
+
+ if (vmMo == null)
+ return;
+
+ boolean hasSnapshot = false;
+ hasSnapshot = vmMo.hasSnapshot();
+ if (!hasSnapshot)
+ vmMo.tearDownDevices(new Class<?>[]{VirtualDisk.class, VirtualEthernetCard.class});
+ else
+ vmMo.tearDownDevices(new Class<?>[]{VirtualEthernetCard.class});
+ vmMo.ensureScsiDeviceController();
+ }
+
+ private static DiskTO getIsoDiskTO(DiskTO[] disks) {
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ISO) {
+ return vol;
+ }
+ }
+ return null;
+ }
+
+ private static DiskTO[] sortVolumesByDeviceId(DiskTO[] volumes) {
+
+ List<DiskTO> listForSort = new ArrayList<DiskTO>();
+ for (DiskTO vol : volumes) {
+ listForSort.add(vol);
+ }
+ Collections.sort(listForSort, new Comparator<DiskTO>() {
+
+ @Override
+ public int compare(DiskTO arg0, DiskTO arg1) {
+ if (arg0.getDiskSeq() < arg1.getDiskSeq()) {
+ return -1;
+ } else if (arg0.getDiskSeq().equals(arg1.getDiskSeq())) {
+ return 0;
+ }
+
+ return 1;
+ }
+ });
+
+ return listForSort.toArray(new DiskTO[0]);
+ }
+
+ private void postDiskConfigBeforeStart(VirtualMachineMO vmMo, VirtualMachineTO vmSpec, DiskTO[] sortedDisks,
+ Map<String, Map<String, String>> iqnToData, VmwareHypervisorHost hyperHost, VmwareContext context, boolean installAsIs)
+ throws Exception {
+ VirtualMachineDiskInfoBuilder diskInfoBuilder = vmMo.getDiskInfoBuilder();
+
+ for (DiskTO vol : sortedDisks) {
+ //TODO: Map existing disks to the ones returned in the answer
+ if (vol.getType() == Volume.Type.ISO)
+ continue;
+
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+
+ VirtualMachineDiskInfo diskInfo = getMatchingExistingDisk(diskInfoBuilder, vol, hyperHost, context);
+ assert (diskInfo != null);
+
+ String[] diskChain = diskInfo.getDiskChain();
+ assert (diskChain.length > 0);
+
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ DatastoreFile file = new DatastoreFile(diskChain[0]);
+
+ if (managed) {
+ DatastoreFile originalFile = new DatastoreFile(volumeTO.getPath());
+
+ if (!file.getFileBaseName().equalsIgnoreCase(originalFile.getFileBaseName())) {
+ if (LOGGER.isInfoEnabled())
+ LOGGER.info("Detected disk-chain top file change on volume: " + volumeTO.getId() + " " + volumeTO.getPath() + " -> " + diskChain[0]);
+ }
+ } else {
+ if (!file.getFileBaseName().equalsIgnoreCase(volumeTO.getPath())) {
+ if (LOGGER.isInfoEnabled())
+ LOGGER.info("Detected disk-chain top file change on volume: " + volumeTO.getId() + " " + volumeTO.getPath() + " -> " + file.getFileBaseName());
+ }
+ }
+
+ VolumeObjectTO volInSpec = getVolumeInSpec(vmSpec, volumeTO);
+
+ if (volInSpec != null) {
+ if (managed) {
+ Map<String, String> data = new HashMap<>();
+
+ String datastoreVolumePath = diskChain[0];
+
+ data.put(StartAnswer.PATH, datastoreVolumePath);
+ data.put(StartAnswer.IMAGE_FORMAT, Storage.ImageFormat.OVA.toString());
+
+ iqnToData.put(details.get(DiskTO.IQN), data);
+
+ vol.setPath(datastoreVolumePath);
+ volumeTO.setPath(datastoreVolumePath);
+ volInSpec.setPath(datastoreVolumePath);
+ } else {
+ volInSpec.setPath(file.getFileBaseName());
+ }
+ volInSpec.setChainInfo(vmwareResource.getGson().toJson(diskInfo));
+ }
+ }
+ }
+
+ private void checkBootOptions(VirtualMachineTO vmSpec, VirtualMachineConfigSpec vmConfigSpec) {
+ String bootMode = null;
+ if (vmSpec.getDetails().containsKey(VmDetailConstants.BOOT_MODE)) {
+ bootMode = vmSpec.getDetails().get(VmDetailConstants.BOOT_MODE);
+ }
+ if (null == bootMode) {
+ bootMode = ApiConstants.BootType.BIOS.toString();
+ }
+
+ setBootOptions(vmSpec, bootMode, vmConfigSpec);
+ }
+
+ private void setBootOptions(VirtualMachineTO vmSpec, String bootMode, VirtualMachineConfigSpec vmConfigSpec) {
+ VirtualMachineBootOptions bootOptions = null;
+ if (StringUtils.isNotBlank(bootMode) && !bootMode.equalsIgnoreCase("bios")) {
+ vmConfigSpec.setFirmware("efi");
+ if (vmSpec.getDetails().containsKey(ApiConstants.BootType.UEFI.toString()) && "secure".equalsIgnoreCase(vmSpec.getDetails().get(ApiConstants.BootType.UEFI.toString()))) {
+ if (bootOptions == null) {
+ bootOptions = new VirtualMachineBootOptions();
+ }
+ bootOptions.setEfiSecureBootEnabled(true);
+ }
+ }
+ if (vmSpec.isEnterHardwareSetup()) {
+ if (bootOptions == null) {
+ bootOptions = new VirtualMachineBootOptions();
+ }
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("configuring VM '%s' to enter hardware setup",vmSpec.getName()));
+ }
+ bootOptions.setEnterBIOSSetup(vmSpec.isEnterHardwareSetup());
+ }
+ if (bootOptions != null) {
+ vmConfigSpec.setBootOptions(bootOptions);
+ }
+ }
+
+ private void resizeRootDiskOnVMStart(VirtualMachineMO vmMo, DiskTO rootDiskTO, VmwareHypervisorHost hyperHost, VmwareContext context) throws Exception {
+ final Pair<VirtualDisk, String> vdisk = vmwareResource.getVirtualDiskInfo(vmMo, VmwareResource.appendFileType(rootDiskTO.getPath(), VmwareResource.VMDK_EXTENSION));
+ assert (vdisk != null);
+
+ Long reqSize = 0L;
+ final VolumeObjectTO volumeTO = ((VolumeObjectTO) rootDiskTO.getData());
+ if (volumeTO != null) {
+ reqSize = volumeTO.getSize() / 1024;
+ }
+ final VirtualDisk disk = vdisk.first();
+ if (reqSize > disk.getCapacityInKB()) {
+ final VirtualMachineDiskInfo diskInfo = getMatchingExistingDisk(vmMo.getDiskInfoBuilder(), rootDiskTO, hyperHost, context);
+ assert (diskInfo != null);
+ final String[] diskChain = diskInfo.getDiskChain();
+
+ if (diskChain != null && diskChain.length > 1) {
+ LOGGER.warn("Disk chain length for the VM is greater than one, this is not supported");
+ throw new CloudRuntimeException("Unsupported VM disk chain length: " + diskChain.length);
+ }
+
+ boolean resizingSupported = false;
+ String deviceBusName = diskInfo.getDiskDeviceBusName();
+ if (deviceBusName != null && (deviceBusName.toLowerCase().contains("scsi") || deviceBusName.toLowerCase().contains("lsi"))) {
+ resizingSupported = true;
+ }
+ if (!resizingSupported) {
+ LOGGER.warn("Resizing of root disk is only support for scsi device/bus, the provide VM's disk device bus name is " + diskInfo.getDiskDeviceBusName());
+ throw new CloudRuntimeException("Unsupported VM root disk device bus: " + diskInfo.getDiskDeviceBusName());
+ }
+
+ disk.setCapacityInKB(reqSize);
+ VirtualMachineConfigSpec vmConfigSpec = new VirtualMachineConfigSpec();
+ VirtualDeviceConfigSpec deviceConfigSpec = new VirtualDeviceConfigSpec();
+ deviceConfigSpec.setDevice(disk);
+ deviceConfigSpec.setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ vmConfigSpec.getDeviceChange().add(deviceConfigSpec);
+ if (!vmMo.configureVm(vmConfigSpec)) {
+ throw new Exception("Failed to configure VM for given root disk size. vmName: " + vmMo.getName());
+ }
+ }
+ }
+
+ private static void postNvpConfigBeforeStart(VirtualMachineMO vmMo, VirtualMachineTO vmSpec) throws Exception {
+ /**
+ * We need to configure the port on the DV switch after the host is
+ * connected. So make this happen between the configure and start of
+ * the VM
+ */
Review comment:
```suggestion
/**
* We need to configure the port on the DV switch after the host is
* connected. So make this happen between the configure and start of
* the VM
*/
private static void postNvpConfigBeforeStart(VirtualMachineMO vmMo, VirtualMachineTO vmSpec) throws Exception {
```
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/storage/resource/VmwareStorageProcessor.java
##########
@@ -891,6 +905,65 @@ public Answer cloneVolumeFromBaseTemplate(CopyCommand cmd) {
}
}
+ private void createLinkedOrFullClone(TemplateObjectTO template, VolumeObjectTO volume, DatacenterMO dcMo, VirtualMachineMO vmMo, ManagedObjectReference morDatastore,
+ DatastoreMO dsMo, String cloneName, ManagedObjectReference morPool) throws Exception {
+ if (template.getSize() != null) {
+ _fullCloneFlag = volume.getSize() > template.getSize() || _fullCloneFlag;
+ }
+ if (!_fullCloneFlag) {
+ createVMLinkedClone(vmMo, dcMo, cloneName, morDatastore, morPool);
+ } else {
+ createVMFullClone(vmMo, dcMo, dsMo, cloneName, morDatastore, morPool);
+ }
+ }
+
+ private void restoreVmMo(VolumeObjectTO volume, String searchExcludedFolders, DatacenterMO dcMo, DatastoreMO dsMo, String vmdkFileBaseName) throws Exception {
+ // restoreVM - move the new ROOT disk into corresponding VM folder
+ // FR37 TODO is this needed?
Review comment:
needs addressing
##########
File path: server/src/main/java/com/cloud/vm/UserVmManagerImpl.java
##########
@@ -5142,36 +5212,36 @@ public UserVm createVirtualMachine(DeployVMCmd cmd) throws InsufficientCapacityE
Boolean displayVm = cmd.isDisplayVm();
String keyboard = cmd.getKeyboard();
Map<Long, DiskOffering> dataDiskTemplateToDiskOfferingMap = cmd.getDataDiskTemplateToDiskOfferingMap();
- Map<String, String> userVmOVFProperties = cmd.getVmOVFProperties();
+ Map<String, String> userVmProperties = cmd.getVmProperties();
if (zone.getNetworkType() == NetworkType.Basic) {
if (cmd.getNetworkIds() != null) {
throw new InvalidParameterValueException("Can't specify network Ids in Basic zone");
} else {
vm = createBasicSecurityGroupVirtualMachine(zone, serviceOffering, template, getSecurityGroupIdList(cmd), owner, name, displayName, diskOfferingId,
size , group , cmd.getHypervisor(), cmd.getHttpMethod(), userData , sshKeyPairName , cmd.getIpToNetworkMap(), addrs, displayVm , keyboard , cmd.getAffinityGroupIdList(),
cmd.getDetails(), cmd.getCustomId(), cmd.getDhcpOptionsMap(),
- dataDiskTemplateToDiskOfferingMap, userVmOVFProperties);
+ dataDiskTemplateToDiskOfferingMap, userVmProperties);
}
} else {
if (zone.isSecurityGroupEnabled()) {
vm = createAdvancedSecurityGroupVirtualMachine(zone, serviceOffering, template, cmd.getNetworkIds(), getSecurityGroupIdList(cmd), owner, name,
displayName, diskOfferingId, size, group, cmd.getHypervisor(), cmd.getHttpMethod(), userData, sshKeyPairName, cmd.getIpToNetworkMap(), addrs, displayVm, keyboard,
cmd.getAffinityGroupIdList(), cmd.getDetails(), cmd.getCustomId(), cmd.getDhcpOptionsMap(),
- dataDiskTemplateToDiskOfferingMap, userVmOVFProperties);
+ dataDiskTemplateToDiskOfferingMap, userVmProperties);
} else {
if (cmd.getSecurityGroupIdList() != null && !cmd.getSecurityGroupIdList().isEmpty()) {
throw new InvalidParameterValueException("Can't create vm with security groups; security group feature is not enabled per zone");
}
vm = createAdvancedVirtualMachine(zone, serviceOffering, template, cmd.getNetworkIds(), owner, name, displayName, diskOfferingId, size, group,
cmd.getHypervisor(), cmd.getHttpMethod(), userData, sshKeyPairName, cmd.getIpToNetworkMap(), addrs, displayVm, keyboard, cmd.getAffinityGroupIdList(), cmd.getDetails(),
- cmd.getCustomId(), cmd.getDhcpOptionsMap(), dataDiskTemplateToDiskOfferingMap, userVmOVFProperties);
+ cmd.getCustomId(), cmd.getDhcpOptionsMap(), dataDiskTemplateToDiskOfferingMap, userVmProperties);
}
}
// check if this templateId has a child ISO
List<VMTemplateVO> child_templates = _templateDao.listByParentTemplatetId(templateId);
for (VMTemplateVO tmpl: child_templates){
- if (tmpl.getFormat() == Storage.ImageFormat.ISO){
+ if (tmpl.getFormat() == Storage.ImageFormat.ISO){ // FR37 why only ISO?
s_logger.info("MDOV trying to attach disk to the VM " + tmpl.getId() + " vmid=" + vm.getId());
_tmplService.attachIso(tmpl.getId(), vm.getId());
}
Review comment:
```suggestion
if (tmpl.getFormat() == Storage.ImageFormat.ISO){ // Note that this is only done for ISOs.
s_logger.info("MultiDiskOVA trying to attach disk to the VM " + tmpl.getId() + " vmid=" + vm.getId());
_tmplService.attachIso(tmpl.getId(), vm.getId());
}
```
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/storage/resource/VmwareStorageProcessor.java
##########
@@ -818,64 +855,41 @@ public Answer cloneVolumeFromBaseTemplate(CopyCommand cmd) {
}
}
} else {
+ // FR37 check for deployasis and deploy from content library if possible
Review comment:
```suggestion
```
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/storage/resource/VmwareStorageProcessor.java
##########
@@ -818,64 +855,41 @@ public Answer cloneVolumeFromBaseTemplate(CopyCommand cmd) {
}
}
} else {
+ // FR37 check for deployasis and deploy from content library if possible
+ // FR37 check if we should be here for an existing VM
String templatePath = template.getPath();
- VirtualMachineMO vmTemplate = VmwareHelper.pickOneVmOnRunningHost(dcMo.findVmByNameAndLabel(templatePath), true);
- if (vmTemplate == null) {
- s_logger.warn("Template host in vSphere is not in connected state, request template reload");
- return new CopyCmdAnswer("Template host in vSphere is not in connected state, request template reload");
- }
+ if (template.isDeployAsIs()) {
+ if(s_logger.isDebugEnabled()) {
+ s_logger.trace("No need to create volume as template is already copied in primary storage");
+ }
- ManagedObjectReference morPool = hyperHost.getHyperHostOwnerResourcePool();
- ManagedObjectReference morCluster = hyperHost.getHyperHostCluster();
- if (template.getSize() != null){
- _fullCloneFlag = volume.getSize() > template.getSize() ? true : _fullCloneFlag;
- }
- if (!_fullCloneFlag) {
- createVMLinkedClone(vmTemplate, dcMo, vmdkName, morDatastore, morPool);
+ ManagedObjectReference morPool = hyperHost.getHyperHostOwnerResourcePool();
+ vmMo = hyperHost.findVmOnHyperHost(template.getPath());
+ createLinkedOrFullClone(template, volume, dcMo, vmMo, morDatastore, dsMo, cloneVMName, morPool);
+ // At this point vmMo points to the cloned VM
+ // TODO: should we check if vmMo has no vmdks i.e. a template with iso only?
Review comment:
is this addressed?
##########
File path: server/src/main/java/com/cloud/template/TemplateManagerImpl.java
##########
@@ -334,11 +335,15 @@ public VirtualMachineTemplate registerTemplate(RegisterTemplateCmd cmd) throws U
}
if (cmd.isRoutingType() != null) {
if (!_accountService.isRootAdmin(account.getId())) {
+ // FR37 than why is it not in RegisterTemplateCmdByAdmin at least?
Review comment:
address or delete (minor issue)
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/StartCommandExecutor.java
##########
@@ -0,0 +1,2247 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.hypervisor.vmware.resource;
+
+import java.io.File;
+import java.rmi.RemoteException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.cloud.agent.api.to.DataTO;
+import com.vmware.vim25.VirtualDiskFlatVer2BackingInfo;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.storage.configdrive.ConfigDrive;
+import org.apache.cloudstack.storage.to.TemplateObjectTO;
+import org.apache.cloudstack.storage.to.VolumeObjectTO;
+import org.apache.cloudstack.utils.volume.VirtualMachineDiskInfo;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.commons.lang.ArrayUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.log4j.Logger;
+
+import com.cloud.agent.api.Command;
+import com.cloud.agent.api.StartAnswer;
+import com.cloud.agent.api.StartCommand;
+import com.cloud.agent.api.storage.OVFPropertyTO;
+import com.cloud.agent.api.to.DataStoreTO;
+import com.cloud.agent.api.to.DiskTO;
+import com.cloud.agent.api.to.NfsTO;
+import com.cloud.agent.api.to.NicTO;
+import com.cloud.agent.api.to.VirtualMachineTO;
+import com.cloud.configuration.Resource;
+import com.cloud.hypervisor.vmware.VmwareResourceException;
+import com.cloud.hypervisor.vmware.manager.VmwareManager;
+import com.cloud.hypervisor.vmware.mo.CustomFieldConstants;
+import com.cloud.hypervisor.vmware.mo.DatacenterMO;
+import com.cloud.hypervisor.vmware.mo.DatastoreFile;
+import com.cloud.hypervisor.vmware.mo.DatastoreMO;
+import com.cloud.hypervisor.vmware.mo.DiskControllerType;
+import com.cloud.hypervisor.vmware.mo.HostMO;
+import com.cloud.hypervisor.vmware.mo.HypervisorHostHelper;
+import com.cloud.hypervisor.vmware.mo.TaskMO;
+import com.cloud.hypervisor.vmware.mo.VirtualEthernetCardType;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineDiskInfoBuilder;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineMO;
+import com.cloud.hypervisor.vmware.mo.VmwareHypervisorHost;
+import com.cloud.hypervisor.vmware.util.VmwareContext;
+import com.cloud.hypervisor.vmware.util.VmwareHelper;
+import com.cloud.network.Networks;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Volume;
+import com.cloud.storage.resource.VmwareStorageLayoutHelper;
+import com.cloud.storage.resource.VmwareStorageProcessor;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nicira.nvp.plugin.NiciraNvpApiVersion;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.VmDetailConstants;
+import com.vmware.vim25.BoolPolicy;
+import com.vmware.vim25.DVPortConfigInfo;
+import com.vmware.vim25.DVPortConfigSpec;
+import com.vmware.vim25.DasVmPriority;
+import com.vmware.vim25.DistributedVirtualPort;
+import com.vmware.vim25.DistributedVirtualSwitchPortConnection;
+import com.vmware.vim25.DistributedVirtualSwitchPortCriteria;
+import com.vmware.vim25.ManagedObjectReference;
+import com.vmware.vim25.OptionValue;
+import com.vmware.vim25.VMwareDVSPortSetting;
+import com.vmware.vim25.VirtualDevice;
+import com.vmware.vim25.VirtualDeviceBackingInfo;
+import com.vmware.vim25.VirtualDeviceConfigSpec;
+import com.vmware.vim25.VirtualDeviceConfigSpecOperation;
+import com.vmware.vim25.VirtualDisk;
+import com.vmware.vim25.VirtualEthernetCard;
+import com.vmware.vim25.VirtualEthernetCardDistributedVirtualPortBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardNetworkBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardOpaqueNetworkBackingInfo;
+import com.vmware.vim25.VirtualMachineBootOptions;
+import com.vmware.vim25.VirtualMachineConfigSpec;
+import com.vmware.vim25.VirtualMachineFileInfo;
+import com.vmware.vim25.VirtualMachineFileLayoutEx;
+import com.vmware.vim25.VirtualMachineGuestOsIdentifier;
+import com.vmware.vim25.VirtualUSBController;
+import com.vmware.vim25.VmConfigInfo;
+import com.vmware.vim25.VmwareDistributedVirtualSwitchVlanIdSpec;
+
+class StartCommandExecutor {
+ private static final Logger LOGGER = Logger.getLogger(StartCommandExecutor.class);
+
+ private final VmwareResource vmwareResource;
+
+ public StartCommandExecutor(VmwareResource vmwareResource) {
+ this.vmwareResource = vmwareResource;
+ }
+
+ // FR37 the monster blob god method: reduce to 320 so far and counting
+ protected StartAnswer execute(StartCommand cmd) {
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Executing resource StartCommand: " + vmwareResource.getGson().toJson(cmd));
+ }
+
+ VirtualMachineTO vmSpec = cmd.getVirtualMachine();
+
+ VirtualMachineData existingVm = null;
+
+ Pair<String, String> names = composeVmNames(vmSpec);
+ String vmInternalCSName = names.first();
+ String vmNameOnVcenter = names.second();
+
+ DiskTO rootDiskTO = null;
+
+ Pair<String, String> controllerInfo = getDiskControllerInfo(vmSpec);
+
+ Boolean systemVm = vmSpec.getType().isUsedBySystem();
+
+ VmwareContext context = vmwareResource.getServiceContext();
+ DatacenterMO dcMo = null;
+ try {
+ VmwareManager mgr = context.getStockObject(VmwareManager.CONTEXT_STOCK_NAME);
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(context);
+ dcMo = new DatacenterMO(hyperHost.getContext(), hyperHost.getHyperHostDatacenter());
+
+ // checkIfVmExistsInVcenter(vmInternalCSName, vmNameOnVcenter, dcMo);
+ // FR37 - We expect VM to be already cloned and available at this point
+ VirtualMachineMO vmMo = dcMo.findVm(vmInternalCSName);
+ if (vmMo == null) {
+ vmMo = dcMo.findVm(vmNameOnVcenter);
+ }
+
+ DiskTO[] specDisks = vmSpec.getDisks();
+ boolean installAsIs = StringUtils.isNotEmpty(vmSpec.getTemplateLocation());
+ // FR37 if startcommand contains enough info: a template url/-location and flag; deploy OVA as is
+ if (vmMo == null && installAsIs) {
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("deploying OVA from %s as is", vmSpec.getTemplateLocation()));
+ }
+ getStorageProcessor().cloneVMFromTemplate(vmSpec.getTemplateName(), vmInternalCSName, vmSpec.getTemplatePrimaryStoreUuid());
+ vmMo = dcMo.findVm(vmInternalCSName);
+ mapSpecDisksToClonedDisks(vmMo, vmInternalCSName, specDisks);
+ }
+
+ // VM may not have been on the same host, relocate to expected host
+ if (vmMo != null) {
+ vmMo.relocate(hyperHost.getMor());
+ // Get updated MO
+ vmMo = hyperHost.findVmOnHyperHost(vmMo.getVmName());
+ }
+
+ String guestOsId = translateGuestOsIdentifier(vmSpec.getArch(), vmSpec.getOs(), vmSpec.getPlatformEmulator()).value();
+ DiskTO[] disks = validateDisks(specDisks);
+ NicTO[] nics = vmSpec.getNics();
+
+ // FIXME: disks logic here, why is disks/volumes during copy not set with pool ID?
+ // FR37 TODO if deployasis a new VM no datastores are known (yet) and we need to get the data store from the tvmspec content library / template location
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails = inferDatastoreDetailsFromDiskInfo(hyperHost, context, disks, cmd);
+ if ((dataStoresDetails == null) || (dataStoresDetails.isEmpty())) {
+ String msg = "Unable to locate datastore details of the volumes to be attached";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ // FR37 - this may not be necessary the cloned VM will have disks and knowledge of datastore paths/location?
+ // throw new Exception(msg);
+ }
+
+ // FR37 - this may need checking, if the first datastore is the right one - ideally it should be datastore where first disk is hosted
+ DatastoreMO dsRootVolumeIsOn = null; //
+ if (! installAsIs) {
+ dsRootVolumeIsOn = getDatastoreThatRootDiskIsOn(dataStoresDetails, disks);
+
+ if (dsRootVolumeIsOn == null) {
+ String msg = "Unable to locate datastore details of root volume";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ throw new VmwareResourceException(msg);
+ }
+ }
+
+ VirtualMachineDiskInfoBuilder diskInfoBuilder = null;
+ VirtualDevice[] nicDevices = null;
+ DiskControllerType systemVmScsiControllerType = DiskControllerType.lsilogic;
+ int firstScsiControllerBusNum = 0;
+ int numScsiControllerForSystemVm = 1;
+ boolean hasSnapshot = false;
+ if (vmMo != null) {
+ PrepareRunningVMForConfiguration prepareRunningVMForConfiguration = new PrepareRunningVMForConfiguration(vmInternalCSName, controllerInfo, systemVm, vmMo,
+ systemVmScsiControllerType, firstScsiControllerBusNum, numScsiControllerForSystemVm);
+ prepareRunningVMForConfiguration.invoke();
+ diskInfoBuilder = prepareRunningVMForConfiguration.getDiskInfoBuilder();
+ nicDevices = prepareRunningVMForConfiguration.getNicDevices();
+ hasSnapshot = prepareRunningVMForConfiguration.isHasSnapshot();
+ } else {
+ ManagedObjectReference morDc = hyperHost.getHyperHostDatacenter();
+ assert (morDc != null);
+
+ vmMo = hyperHost.findVmOnPeerHyperHost(vmInternalCSName);
+ if (vmMo != null) {
+ VirtualMachineRecycler virtualMachineRecycler = new VirtualMachineRecycler(vmInternalCSName, controllerInfo, systemVm, hyperHost, vmMo,
+ systemVmScsiControllerType, firstScsiControllerBusNum, numScsiControllerForSystemVm);
+ virtualMachineRecycler.invoke();
+ diskInfoBuilder = virtualMachineRecycler.getDiskInfoBuilder();
+ hasSnapshot = virtualMachineRecycler.isHasSnapshot();
+ nicDevices = virtualMachineRecycler.getNicDevices();
+ } else {
+ // FR37 we just didn't find a VM by name 'vmInternalCSName', so why is this here?
+ existingVm = unregisterOnOtherClusterButHoldOnToOldVmData(vmInternalCSName, dcMo);
+
+ createNewVm(context, vmSpec, installAsIs, vmInternalCSName, vmNameOnVcenter, controllerInfo, systemVm, mgr, hyperHost, guestOsId, disks, dataStoresDetails,
+ dsRootVolumeIsOn);
+ }
+
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ if (vmMo == null) {
+ throw new Exception("Failed to find the newly create or relocated VM. vmName: " + vmInternalCSName);
+ }
+ }
+ // vmMo should now be a stopped VM on the intended host
+ // The number of disks changed must be 0 for install as is, as the VM is a clone
+ int disksChanges = !installAsIs ? disks.length : 0;
+ int totalChangeDevices = disksChanges + nics.length;
+ int hackDeviceCount = 0;
+ if (diskInfoBuilder != null) {
+ hackDeviceCount += diskInfoBuilder.getDiskCount();
+ }
+ hackDeviceCount += nicDevices == null ? 0 : nicDevices.length;
+ // vApp cdrom device
+ // HACK ALERT: ovf properties might not be the only or defining feature of vApps; needs checking
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("current count(s) desired: %d/ found:%d. now adding device to device count for vApp config ISO", totalChangeDevices, hackDeviceCount));
+ }
+ if (vmSpec.getOvfProperties() != null) {
+ totalChangeDevices++;
+ }
+
+ DiskTO volIso = null;
+ if (vmSpec.getType() != VirtualMachine.Type.User) {
+ // system VM needs a patch ISO
+ totalChangeDevices++;
+ } else {
+ volIso = getIsoDiskTO(disks);
+ if (volIso == null)
+ totalChangeDevices++;
+ }
+
+ VirtualMachineConfigSpec vmConfigSpec = new VirtualMachineConfigSpec();
+
+ VmwareHelper.setBasicVmConfig(vmConfigSpec, vmSpec.getCpus(), vmSpec.getMaxSpeed(), vmwareResource.getReservedCpuMHZ(vmSpec), (int)(vmSpec.getMaxRam() / (1024 * 1024)),
+ vmwareResource.getReservedMemoryMb(vmSpec), guestOsId, vmSpec.getLimitCpuUse());
+
+ int numCoresPerSocket = adjustNumberOfCoresPerSocket(vmSpec, vmMo, vmConfigSpec);
+
+ // Check for hotadd settings
+ vmConfigSpec.setMemoryHotAddEnabled(vmMo.isMemoryHotAddSupported(guestOsId));
+
+ String hostApiVersion = ((HostMO)hyperHost).getHostAboutInfo().getApiVersion();
+ if (numCoresPerSocket > 1 && hostApiVersion.compareTo("5.0") < 0) {
+ LOGGER.warn("Dynamic scaling of CPU is not supported for Virtual Machines with multi-core vCPUs in case of ESXi hosts 4.1 and prior. Hence CpuHotAdd will not be"
+ + " enabled for Virtual Machine: " + vmInternalCSName);
+ vmConfigSpec.setCpuHotAddEnabled(false);
+ } else {
+ vmConfigSpec.setCpuHotAddEnabled(vmMo.isCpuHotAddSupported(guestOsId));
+ }
+
+ vmwareResource.configNestedHVSupport(vmMo, vmSpec, vmConfigSpec);
+
+ VirtualDeviceConfigSpec[] deviceConfigSpecArray = new VirtualDeviceConfigSpec[totalChangeDevices];
+ int deviceCount = 0;
+ int ideUnitNumber = 0;
+ int scsiUnitNumber = 0;
+ int ideControllerKey = vmMo.getIDEDeviceControllerKey();
+ int scsiControllerKey = vmMo.getScsiDeviceControllerKeyNoException();
+
+ IsoSetup isoSetup = new IsoSetup(vmSpec, mgr, hyperHost, vmMo, disks, volIso, deviceConfigSpecArray, deviceCount, ideUnitNumber);
+ isoSetup.invoke();
+ deviceCount = isoSetup.getDeviceCount();
+ ideUnitNumber = isoSetup.getIdeUnitNumber();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("setting up %d disks with root from %s", diskInfoBuilder.getDiskCount(), vmwareResource.getGson().toJson(rootDiskTO)));
+ }
+
+ DiskSetup diskSetup = new DiskSetup(vmSpec, rootDiskTO, controllerInfo, context, dcMo, hyperHost, vmMo, disks, dataStoresDetails, diskInfoBuilder, hasSnapshot,
+ deviceConfigSpecArray, deviceCount, ideUnitNumber, scsiUnitNumber, ideControllerKey, scsiControllerKey, installAsIs);
+ diskSetup.invoke();
+
+ rootDiskTO = diskSetup.getRootDiskTO();
+ deviceCount = diskSetup.getDeviceCount();
+ DiskTO[] sortedDisks = diskSetup.getSortedDisks();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("seting up %d disks with root from %s", sortedDisks.length, vmwareResource.getGson().toJson(rootDiskTO)));
+ }
+
+ deviceCount += setupUsbDevicesAndGetCount(vmInternalCSName, vmMo, guestOsId, deviceConfigSpecArray, deviceCount);
+
+ NicSetup nicSetup = new NicSetup(cmd, vmSpec, vmInternalCSName, context, mgr, hyperHost, vmMo, nics, deviceConfigSpecArray, deviceCount, nicDevices);
+ nicSetup.invoke();
+
+ deviceCount = nicSetup.getDeviceCount();
+ int nicMask = nicSetup.getNicMask();
+ int nicCount = nicSetup.getNicCount();
+ Map<String, String> nicUuidToDvSwitchUuid = nicSetup.getNicUuidToDvSwitchUuid();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("device count: %d; nic count %d", deviceCount, nicCount));
+ }
+ for (int j = 0; j < deviceCount; j++)
+ vmConfigSpec.getDeviceChange().add(deviceConfigSpecArray[j]);
+
+ //
+ // Setup VM options
+ //
+
+ // pass boot arguments through machine.id & perform customized options to VMX
+ ArrayList<OptionValue> extraOptions = new ArrayList<>();
+ configBasicExtraOption(extraOptions, vmSpec);
+ configNvpExtraOption(extraOptions, vmSpec, nicUuidToDvSwitchUuid);
+ configCustomExtraOption(extraOptions, vmSpec);
+
+ // config for NCC
+ VirtualMachine.Type vmType = cmd.getVirtualMachine().getType();
+ if (vmType.equals(VirtualMachine.Type.NetScalerVm)) {
+ NicTO mgmtNic = vmSpec.getNics()[0];
+ OptionValue option = new OptionValue();
+ option.setKey("machine.id");
+ option.setValue("ip=" + mgmtNic.getIp() + "&netmask=" + mgmtNic.getNetmask() + "&gateway=" + mgmtNic.getGateway());
+ extraOptions.add(option);
+ }
+
+ // config VNC
+ String keyboardLayout = null;
+ if (vmSpec.getDetails() != null)
+ keyboardLayout = vmSpec.getDetails().get(VmDetailConstants.KEYBOARD);
+ vmConfigSpec.getExtraConfig().addAll(
+ Arrays.asList(vmwareResource.configureVnc(extraOptions.toArray(new OptionValue[0]), hyperHost, vmInternalCSName, vmSpec.getVncPassword(), keyboardLayout)));
+
+ // config video card
+ vmwareResource.configureVideoCard(vmMo, vmSpec, vmConfigSpec);
+
+ // Set OVF properties (if available)
+ Pair<String, List<OVFPropertyTO>> ovfPropsMap = vmSpec.getOvfProperties();
+ VmConfigInfo templateVappConfig;
+ List<OVFPropertyTO> ovfProperties;
+ if (ovfPropsMap != null) {
+ String vmTemplate = ovfPropsMap.first();
+ LOGGER.info("Find VM template " + vmTemplate);
+ VirtualMachineMO vmTemplateMO = dcMo.findVm(vmTemplate);
+ templateVappConfig = vmTemplateMO.getConfigInfo().getVAppConfig();
+ ovfProperties = ovfPropsMap.second();
+ // Set OVF properties (if available)
+ if (CollectionUtils.isNotEmpty(ovfProperties)) {
+ LOGGER.info("Copying OVF properties from template and setting them to the values the user provided");
+ vmwareResource.copyVAppConfigsFromTemplate(templateVappConfig, ovfProperties, vmConfigSpec);
+ }
+ }
+
+ checkBootOptions(vmSpec, vmConfigSpec);
+
+ //
+ // Configure VM
+ //
+ if (!vmMo.configureVm(vmConfigSpec)) {
+ throw new Exception("Failed to configure VM before start. vmName: " + vmInternalCSName);
+ }
+ // FR37 TODO reconcile disks now!!! they are configured, check and add them to the returning vmTO
+ if (vmSpec.getType() == VirtualMachine.Type.DomainRouter) {
+ hyperHost.setRestartPriorityForVM(vmMo, DasVmPriority.HIGH.value());
+ }
+
+ // Resizing root disk only when explicit requested by user
+ final Map<String, String> vmDetails = cmd.getVirtualMachine().getDetails();
+ if (rootDiskTO != null && !hasSnapshot && (vmDetails != null && vmDetails.containsKey(ApiConstants.ROOT_DISK_SIZE))) {
+ resizeRootDiskOnVMStart(vmMo, rootDiskTO, hyperHost, context);
+ }
+
+ //
+ // Post Configuration
+ //
+
+ vmMo.setCustomFieldValue(CustomFieldConstants.CLOUD_NIC_MASK, String.valueOf(nicMask));
+ postNvpConfigBeforeStart(vmMo, vmSpec);
+
+ Map<String, Map<String, String>> iqnToData = new HashMap<>();
+
+ postDiskConfigBeforeStart(vmMo, vmSpec, sortedDisks, iqnToData, hyperHost, context, installAsIs);
+
+ //
+ // Power-on VM
+ //
+ if (!vmMo.powerOn()) {
+ throw new Exception("Failed to start VM. vmName: " + vmInternalCSName + " with hostname " + vmNameOnVcenter);
+ }
+
+ if (installAsIs) {
+ // Set disks as the disks path and chain is retrieved from the cloned VM disks
+ cmd.getVirtualMachine().setDisks(disks);
+ }
+
+ StartAnswer startAnswer = new StartAnswer(cmd);
+
+ startAnswer.setIqnToData(iqnToData);
+
+ deleteOldVersionOfTheStartedVM(existingVm, dcMo, vmMo);
+
+ return startAnswer;
+ } catch (Throwable e) {
+ return handleStartFailure(cmd, vmSpec, existingVm, dcMo, e);
+ } finally {
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("finally done with %s", vmwareResource.getGson().toJson(cmd)));
+ }
+ }
+ }
+
+ /**
+ * Modify the specDisks information to match the cloned VM's disks (from vmMo VM)
+ */
+ private void mapSpecDisksToClonedDisks(VirtualMachineMO vmMo, String vmInternalCSName, DiskTO[] specDisks) {
+ try {
+ if (vmMo != null && ArrayUtils.isNotEmpty(specDisks)) {
+ List<VirtualDisk> vmDisks = vmMo.getVirtualDisks();
+ List<DiskTO> sortedDisks = Arrays.asList(sortVolumesByDeviceId(specDisks))
+ .stream()
+ .filter(x -> x.getType() == Volume.Type.ROOT)
+ .collect(Collectors.toList());
+ if (sortedDisks.size() != vmDisks.size()) {
+ LOGGER.error("Different number of root disks spec vs cloned deploy-as-is VM disks: " + sortedDisks.size() + " - " + vmDisks.size());
+ return;
+ }
+ for (int i = 0; i < sortedDisks.size(); i++) {
+ DiskTO specDisk = sortedDisks.get(i);
+ VirtualDisk vmDisk = vmDisks.get(i);
+ DataTO dataVolume = specDisk.getData();
+ if (dataVolume instanceof VolumeObjectTO) {
+ VolumeObjectTO volumeObjectTO = (VolumeObjectTO) dataVolume;
+ if (!volumeObjectTO.getSize().equals(vmDisk.getCapacityInBytes())) {
+ LOGGER.info("Mapped disk size is not the same as the cloned VM disk size: " +
+ volumeObjectTO.getSize() + " - " + vmDisk.getCapacityInBytes());
+ }
+ VirtualDeviceBackingInfo backingInfo = vmDisk.getBacking();
+ if (backingInfo instanceof VirtualDiskFlatVer2BackingInfo) {
+ VirtualDiskFlatVer2BackingInfo backing = (VirtualDiskFlatVer2BackingInfo) backingInfo;
+ String fileName = backing.getFileName();
+ if (StringUtils.isNotBlank(fileName)) {
+ String[] fileNameParts = fileName.split(" ");
+ String datastoreUuid = fileNameParts[0].replace("[", "").replace("]", "");
+ String relativePath = fileNameParts[1].split("/")[1].replace(".vmdk", "");
+ String vmSpecDatastoreUuid = volumeObjectTO.getDataStore().getUuid().replaceAll("-", "");
+ if (!datastoreUuid.equals(vmSpecDatastoreUuid)) {
+ LOGGER.info("Mapped disk datastore UUID is not the same as the cloned VM datastore UUID: " +
+ datastoreUuid + " - " + vmSpecDatastoreUuid);
+ }
+ volumeObjectTO.setPath(relativePath);
+ specDisk.setPath(relativePath);
+ }
+ }
+ }
+ }
+ }
+ } catch (Exception e) {
+ String msg = "Error mapping deploy-as-is VM disks from cloned VM " + vmInternalCSName;
+ LOGGER.error(msg, e);
+ throw new CloudRuntimeException(e);
+ }
+ }
+
+ private StartAnswer handleStartFailure(StartCommand cmd, VirtualMachineTO vmSpec, VirtualMachineData existingVm, DatacenterMO dcMo, Throwable e) {
+ if (e instanceof RemoteException) {
+ LOGGER.warn("Encounter remote exception to vCenter, invalidate VMware session context");
+ vmwareResource.invalidateServiceContext();
+ }
+
+ String msg = "StartCommand failed due to " + VmwareHelper.getExceptionMessage(e);
+ LOGGER.warn(msg, e);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("didn't start VM %s", vmSpec.getName()), e);
+ }
+ StartAnswer startAnswer = new StartAnswer(cmd, msg);
+ if ( e instanceof VmAlreadyExistsInVcenter) {
+ startAnswer.setContextParam("stopRetry", "true");
+ }
+ reRegisterExistingVm(existingVm, dcMo);
+
+ return startAnswer;
+ }
+
+ private void deleteOldVersionOfTheStartedVM(VirtualMachineData existingVm, DatacenterMO dcMo, VirtualMachineMO vmMo) throws Exception {
+ // Since VM was successfully powered-on, if there was an existing VM in a different cluster that was unregistered, delete all the files associated with it.
+ if (existingVm != null && existingVm.vmName != null && existingVm.vmFileLayout != null) {
+ List<String> vmDatastoreNames = new ArrayList<>();
+ for (DatastoreMO vmDatastore : vmMo.getAllDatastores()) {
+ vmDatastoreNames.add(vmDatastore.getName());
+ }
+ // Don't delete files that are in a datastore that is being used by the new VM as well (zone-wide datastore).
+ List<String> skipDatastores = new ArrayList<>();
+ for (DatastoreMO existingDatastore : existingVm.datastores) {
+ if (vmDatastoreNames.contains(existingDatastore.getName())) {
+ skipDatastores.add(existingDatastore.getName());
+ }
+ }
+ vmwareResource.deleteUnregisteredVmFiles(existingVm.vmFileLayout, dcMo, true, skipDatastores);
+ }
+ }
+
+ private int adjustNumberOfCoresPerSocket(VirtualMachineTO vmSpec, VirtualMachineMO vmMo, VirtualMachineConfigSpec vmConfigSpec) throws Exception {
+ // Check for multi-cores per socket settings
+ int numCoresPerSocket = 1;
+ String coresPerSocket = vmSpec.getDetails().get(VmDetailConstants.CPU_CORE_PER_SOCKET);
+ if (coresPerSocket != null) {
+ String apiVersion = HypervisorHostHelper.getVcenterApiVersion(vmMo.getContext());
+ // Property 'numCoresPerSocket' is supported since vSphere API 5.0
+ if (apiVersion.compareTo("5.0") >= 0) {
+ numCoresPerSocket = NumbersUtil.parseInt(coresPerSocket, 1);
+ vmConfigSpec.setNumCoresPerSocket(numCoresPerSocket);
+ }
+ }
+ return numCoresPerSocket;
+ }
+
+ /**
+ // Setup USB devices
+ */
+ private int setupUsbDevicesAndGetCount(String vmInternalCSName, VirtualMachineMO vmMo, String guestOsId, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount)
+ throws Exception {
+ int usbDeviceCount = 0;
+ if (guestOsId.startsWith("darwin")) { //Mac OS
+ VirtualDevice[] devices = vmMo.getMatchedDevices(new Class<?>[] {VirtualUSBController.class});
+ if (devices.length == 0) {
+ LOGGER.debug("No USB Controller device on VM Start. Add USB Controller device for Mac OS VM " + vmInternalCSName);
+
+ //For Mac OS X systems, the EHCI+UHCI controller is enabled by default and is required for USB mouse and keyboard access.
+ VirtualDevice usbControllerDevice = VmwareHelper.prepareUSBControllerDevice();
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ deviceConfigSpecArray[deviceCount].setDevice(usbControllerDevice);
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare USB controller at new device " + vmwareResource.getGson().toJson(deviceConfigSpecArray[deviceCount]));
+
+ deviceCount++;
+ usbDeviceCount++;
+ } else {
+ LOGGER.debug("USB Controller device exists on VM Start for Mac OS VM " + vmInternalCSName);
+ }
+ }
+ return usbDeviceCount;
+ }
+
+ private void createNewVm(VmwareContext context, VirtualMachineTO vmSpec, boolean installAsIs, String vmInternalCSName, String vmNameOnVcenter, Pair<String, String> controllerInfo, Boolean systemVm,
+ VmwareManager mgr, VmwareHypervisorHost hyperHost, String guestOsId, DiskTO[] disks, HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails,
+ DatastoreMO dsRootVolumeIsOn) throws Exception {
+ VirtualMachineMO vmMo;
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = getRootDiskDataStoreDetails(disks, dataStoresDetails);
+
+ assert (vmSpec.getMinSpeed() != null) && (rootDiskDataStoreDetails != null);
+
+ boolean vmFolderExists = rootDiskDataStoreDetails.second().folderExists(String.format("[%s]", rootDiskDataStoreDetails.second().getName()), vmNameOnVcenter);
+ String vmxFileFullPath = dsRootVolumeIsOn.searchFileInSubFolders(vmNameOnVcenter + ".vmx", false, VmwareManager.s_vmwareSearchExcludeFolder.value());
+ if (vmFolderExists && vmxFileFullPath != null) { // VM can be registered only if .vmx is present.
+ registerVm(vmNameOnVcenter, dsRootVolumeIsOn, vmwareResource);
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ if (vmMo != null) {
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Found registered vm " + vmInternalCSName + " at host " + hyperHost.getHyperHostName());
+ }
+ }
+ tearDownVm(vmMo);
+ } else if (installAsIs) {
+// first get all the MORs ManagedObjectReference morPool = hyperHost.getHyperHostOwnerResourcePool();
+// get the base VM vmMo = hyperHost.findVmOnHyperHost(vm.template.getPath());
+// do templateVm.createLinkedClone(vmInternalCSName, morBaseSnapshot, dcMo.getVmFolder(), morPool, morDatastore)
+// or hyperHost....createLinkedOrFullClone(templateVm, volume, dcMo, vmMo, morDatastore, dsMo, vmInternalCSName, morPool);
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ // At this point vmMo points to the cloned VM
+ } else {
+ if (!hyperHost
+ .createBlankVm(vmNameOnVcenter, vmInternalCSName, vmSpec.getCpus(), vmSpec.getMaxSpeed(), vmwareResource.getReservedCpuMHZ(vmSpec), vmSpec.getLimitCpuUse(), (int)(vmSpec.getMaxRam() / Resource.ResourceType.bytesToMiB), vmwareResource.getReservedMemoryMb(vmSpec), guestOsId,
+ rootDiskDataStoreDetails.first(), false, controllerInfo, systemVm)) {
+ throw new Exception("Failed to create VM. vmName: " + vmInternalCSName);
+ }
+ }
+ }
+
+ /**
+ * If a VM with the same name is found in a different cluster in the DC, unregister the old VM and configure a new VM (cold-migration).
+ */
+ private VirtualMachineData unregisterOnOtherClusterButHoldOnToOldVmData(String vmInternalCSName, DatacenterMO dcMo) throws Exception {
+ VirtualMachineMO existingVmInDc = dcMo.findVm(vmInternalCSName);
+ VirtualMachineData existingVm = null;
+ if (existingVmInDc != null) {
+ existingVm = new VirtualMachineData();
+ existingVm.vmName = existingVmInDc.getName();
+ existingVm.vmFileInfo = existingVmInDc.getFileInfo();
+ existingVm.vmFileLayout = existingVmInDc.getFileLayout();
+ existingVm.datastores = existingVmInDc.getAllDatastores();
+ LOGGER.info("Found VM: " + vmInternalCSName + " on a host in a different cluster. Unregistering the exisitng VM.");
+ existingVmInDc.unregisterVm();
+ }
+ return existingVm;
+ }
+
+ private Pair<ManagedObjectReference, DatastoreMO> getRootDiskDataStoreDetails(DiskTO[] disks, HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails) {
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = null;
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ROOT) {
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (managed) {
+ String datastoreName = VmwareResource.getDatastoreName(details.get(DiskTO.IQN));
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(datastoreName);
+ } else {
+ DataStoreTO primaryStore = vol.getData().getDataStore();
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(primaryStore.getUuid());
+ }
+ }
+ }
+ return rootDiskDataStoreDetails;
+ }
+
+ /**
+ * Since VM start failed, if there was an existing VM in a different cluster that was unregistered, register it back.
+ *
+ * @param dcMo is guaranteed to be not null since we have noticed there is an existing VM in the dc (using that mo)
+ */
+ private void reRegisterExistingVm(VirtualMachineData existingVm, DatacenterMO dcMo) {
+ if (existingVm != null && existingVm.vmName != null && existingVm.vmFileInfo != null) {
+ LOGGER.debug("Since VM start failed, registering back an existing VM: " + existingVm.vmName + " that was unregistered");
+ try {
+ DatastoreFile fileInDatastore = new DatastoreFile(existingVm.vmFileInfo.getVmPathName());
+ DatastoreMO existingVmDsMo = new DatastoreMO(dcMo.getContext(), dcMo.findDatastore(fileInDatastore.getDatastoreName()));
+ registerVm(existingVm.vmName, existingVmDsMo, vmwareResource);
+ } catch (Exception ex) {
+ String message = "Failed to register an existing VM: " + existingVm.vmName + " due to " + VmwareHelper.getExceptionMessage(ex);
+ LOGGER.warn(message, ex);
+ }
+ }
+ }
+
+ private void checkIfVmExistsInVcenter(String vmInternalCSName, String vmNameOnVcenter, DatacenterMO dcMo) throws VmAlreadyExistsInVcenter, Exception {
+ // Validate VM name is unique in Datacenter
+ VirtualMachineMO vmInVcenter = dcMo.checkIfVmAlreadyExistsInVcenter(vmNameOnVcenter, vmInternalCSName);
+ if (vmInVcenter != null) {
+ String msg = "VM with name: " + vmNameOnVcenter + " already exists in vCenter.";
+ LOGGER.error(msg);
+ throw new VmAlreadyExistsInVcenter(msg);
+ }
+ }
+
+ private Pair<String, String> getDiskControllerInfo(VirtualMachineTO vmSpec) {
+ String dataDiskController = vmSpec.getDetails().get(VmDetailConstants.DATA_DISK_CONTROLLER);
+ String rootDiskController = vmSpec.getDetails().get(VmDetailConstants.ROOT_DISK_CONTROLLER);
+ // If root disk controller is scsi, then data disk controller would also be scsi instead of using 'osdefault'
+ // This helps avoid mix of different scsi subtype controllers in instance.
+ if (DiskControllerType.osdefault == DiskControllerType.getType(dataDiskController) && DiskControllerType.lsilogic == DiskControllerType.getType(rootDiskController)) {
+ dataDiskController = DiskControllerType.scsi.toString();
+ }
+
+ // Validate the controller types
+ dataDiskController = DiskControllerType.getType(dataDiskController).toString();
+ rootDiskController = DiskControllerType.getType(rootDiskController).toString();
+
+ if (DiskControllerType.getType(rootDiskController) == DiskControllerType.none) {
+ throw new CloudRuntimeException("Invalid root disk controller detected : " + rootDiskController);
+ }
+ if (DiskControllerType.getType(dataDiskController) == DiskControllerType.none) {
+ throw new CloudRuntimeException("Invalid data disk controller detected : " + dataDiskController);
+ }
+
+ return new Pair<>(rootDiskController, dataDiskController);
+ }
+
+ /**
+ * Registers the vm to the inventory given the vmx file.
+ * @param vmName
+ * @param dsMo
+ * @param vmwareResource
+ */
+ private void registerVm(String vmName, DatastoreMO dsMo, VmwareResource vmwareResource) throws Exception {
+
+ //1st param
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(vmwareResource.getServiceContext());
+ ManagedObjectReference dcMor = hyperHost.getHyperHostDatacenter();
+ DatacenterMO dataCenterMo = new DatacenterMO(vmwareResource.getServiceContext(), dcMor);
+ ManagedObjectReference vmFolderMor = dataCenterMo.getVmFolder();
+
+ //2nd param
+ String vmxFilePath = dsMo.searchFileInSubFolders(vmName + ".vmx", false, VmwareManager.s_vmwareSearchExcludeFolder.value());
+
+ // 5th param
+ ManagedObjectReference morPool = hyperHost.getHyperHostOwnerResourcePool();
+
+ ManagedObjectReference morTask = vmwareResource.getServiceContext().getService().registerVMTask(vmFolderMor, vmxFilePath, vmName, false, morPool, hyperHost.getMor());
+ boolean result = vmwareResource.getServiceContext().getVimClient().waitForTask(morTask);
+ if (!result) {
+ throw new Exception("Unable to register vm due to " + TaskMO.getTaskFailureInfo(vmwareResource.getServiceContext(), morTask));
+ } else {
+ vmwareResource.getServiceContext().waitForTaskProgressDone(morTask);
+ }
+
+ }
+
+ // Pair<internal CS name, vCenter display name>
+ private Pair<String, String> composeVmNames(VirtualMachineTO vmSpec) {
+ String vmInternalCSName = vmSpec.getName();
+ String vmNameOnVcenter = vmSpec.getName();
+ if (VmwareResource.instanceNameFlag && vmSpec.getHostName() != null) {
+ vmNameOnVcenter = vmSpec.getHostName();
+ }
+ return new Pair<String, String>(vmInternalCSName, vmNameOnVcenter);
+ }
+
+ private VirtualMachineGuestOsIdentifier translateGuestOsIdentifier(String cpuArchitecture, String guestOs, String cloudGuestOs) {
+ if (cpuArchitecture == null) {
+ LOGGER.warn("CPU arch is not set, default to i386. guest os: " + guestOs);
+ cpuArchitecture = "i386";
+ }
+
+ if (cloudGuestOs == null) {
+ LOGGER.warn("Guest OS mapping name is not set for guest os: " + guestOs);
+ }
+
+ VirtualMachineGuestOsIdentifier identifier = null;
+ try {
+ if (cloudGuestOs != null) {
+ identifier = VirtualMachineGuestOsIdentifier.fromValue(cloudGuestOs);
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Using mapping name : " + identifier.toString());
+ }
+ }
+ } catch (IllegalArgumentException e) {
+ LOGGER.warn("Unable to find Guest OS Identifier in VMware for mapping name: " + cloudGuestOs + ". Continuing with defaults.");
+ }
+ if (identifier != null) {
+ return identifier;
+ }
+
+ if (cpuArchitecture.equalsIgnoreCase("x86_64")) {
+ return VirtualMachineGuestOsIdentifier.OTHER_GUEST_64;
+ }
+ return VirtualMachineGuestOsIdentifier.OTHER_GUEST;
+ }
+
+ private DiskTO[] validateDisks(DiskTO[] disks) {
+ List<DiskTO> validatedDisks = new ArrayList<DiskTO>();
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() != Volume.Type.ISO) {
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ if (primaryStore.getUuid() != null && !primaryStore.getUuid().isEmpty()) {
+ validatedDisks.add(vol);
+ }
+ } else if (vol.getType() == Volume.Type.ISO) {
+ TemplateObjectTO templateTO = (TemplateObjectTO) vol.getData();
+ if (templateTO.getPath() != null && !templateTO.getPath().isEmpty()) {
+ validatedDisks.add(vol);
+ }
+ } else {
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Drop invalid disk option, volumeTO: " + vmwareResource.getGson().toJson(vol));
+ }
+ }
+ }
+ Collections.sort(validatedDisks, (d1, d2) -> d1.getDiskSeq().compareTo(d2.getDiskSeq()));
+ return validatedDisks.toArray(new DiskTO[0]);
+ }
+
+ private HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> inferDatastoreDetailsFromDiskInfo(VmwareHypervisorHost hyperHost, VmwareContext context,
+ DiskTO[] disks, Command cmd) throws Exception {
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> mapIdToMors = new HashMap<>();
+
+ assert (hyperHost != null) && (context != null);
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() != Volume.Type.ISO) {
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ String poolUuid = primaryStore.getUuid();
+
+ if (mapIdToMors.get(poolUuid) == null) {
+ boolean isManaged = false;
+ Map<String, String> details = vol.getDetails();
+
+ if (details != null) {
+ isManaged = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (isManaged) {
+ String iScsiName = details.get(DiskTO.IQN); // details should not be null for managed storage (it may or may not be null for non-managed storage)
+ String datastoreName = VmwareResource.getDatastoreName(iScsiName);
+ ManagedObjectReference morDatastore = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, datastoreName);
+
+ // if the datastore is not present, we need to discover the iSCSI device that will support it,
+ // create the datastore, and create a VMDK file in the datastore
+ if (morDatastore == null) {
+ final String vmdkPath = vmwareResource.getVmdkPath(volumeTO.getPath());
+
+ morDatastore = getStorageProcessor().prepareManagedStorage(context, hyperHost, null, iScsiName,
+ details.get(DiskTO.STORAGE_HOST), Integer.parseInt(details.get(DiskTO.STORAGE_PORT)),
+ vmdkPath,
+ details.get(DiskTO.CHAP_INITIATOR_USERNAME), details.get(DiskTO.CHAP_INITIATOR_SECRET),
+ details.get(DiskTO.CHAP_TARGET_USERNAME), details.get(DiskTO.CHAP_TARGET_SECRET),
+ Long.parseLong(details.get(DiskTO.VOLUME_SIZE)), cmd);
+
+ DatastoreMO dsMo = new DatastoreMO(vmwareResource.getServiceContext(), morDatastore);
+
+ final String datastoreVolumePath;
+
+ if (vmdkPath != null) {
+ datastoreVolumePath = dsMo.getDatastorePath(vmdkPath + VmwareResource.VMDK_EXTENSION);
+ } else {
+ datastoreVolumePath = dsMo.getDatastorePath(dsMo.getName() + VmwareResource.VMDK_EXTENSION);
+ }
+
+ volumeTO.setPath(datastoreVolumePath);
+ vol.setPath(datastoreVolumePath);
+ }
+
+ mapIdToMors.put(datastoreName, new Pair<>(morDatastore, new DatastoreMO(context, morDatastore)));
+ } else {
+ ManagedObjectReference morDatastore = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, poolUuid);
+
+ if (morDatastore == null) {
+ String msg = "Failed to get the mounted datastore for the volume's pool " + poolUuid;
+
+ LOGGER.error(msg);
+
+ throw new Exception(msg);
+ }
+
+ mapIdToMors.put(poolUuid, new Pair<>(morDatastore, new DatastoreMO(context, morDatastore)));
+ }
+ }
+ }
+ }
+
+ return mapIdToMors;
+ }
+
+ private VmwareStorageProcessor getStorageProcessor() {
+ return vmwareResource.getStorageProcessor();
+ }
+
+ private DatastoreMO getDatastoreThatRootDiskIsOn(HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails, DiskTO disks[]) {
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = null;
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ROOT) {
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (managed) {
+ String datastoreName = VmwareResource.getDatastoreName(details.get(DiskTO.IQN));
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(datastoreName);
+
+ break;
+ } else {
+ DataStoreTO primaryStore = vol.getData().getDataStore();
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(primaryStore.getUuid());
+
+ break;
+ }
+ }
+ }
+
+ if (rootDiskDataStoreDetails != null) {
+ return rootDiskDataStoreDetails.second();
+ }
+
+ return null;
+ }
+
+ private void ensureDiskControllers(VirtualMachineMO vmMo, Pair<String, String> controllerInfo) throws Exception {
+ if (vmMo == null) {
+ return;
+ }
+
+ String msg;
+ String rootDiskController = controllerInfo.first();
+ String dataDiskController = controllerInfo.second();
+ String scsiDiskController;
+ String recommendedDiskController = null;
+
+ if (VmwareHelper.isControllerOsRecommended(dataDiskController) || VmwareHelper.isControllerOsRecommended(rootDiskController)) {
+ recommendedDiskController = vmMo.getRecommendedDiskController(null);
+ }
+ scsiDiskController = HypervisorHostHelper.getScsiController(new Pair<String, String>(rootDiskController, dataDiskController), recommendedDiskController);
+ if (scsiDiskController == null) {
+ return;
+ }
+
+ vmMo.getScsiDeviceControllerKeyNoException();
+ // This VM needs SCSI controllers.
+ // Get count of existing scsi controllers. Helps not to attempt to create more than the maximum allowed 4
+ // Get maximum among the bus numbers in use by scsi controllers. Safe to pick maximum, because we always go sequential allocating bus numbers.
+ Ternary<Integer, Integer, DiskControllerType> scsiControllerInfo = vmMo.getScsiControllerInfo();
+ int requiredNumScsiControllers = VmwareHelper.MAX_SCSI_CONTROLLER_COUNT - scsiControllerInfo.first();
+ int availableBusNum = scsiControllerInfo.second() + 1; // method returned current max. bus number
+
+ if (requiredNumScsiControllers == 0) {
+ return;
+ }
+ if (scsiControllerInfo.first() > 0) {
+ // For VMs which already have a SCSI controller, do NOT attempt to add any more SCSI controllers & return the sub type.
+ // For Legacy VMs would have only 1 LsiLogic Parallel SCSI controller, and doesn't require more.
+ // For VMs created post device ordering support, 4 SCSI subtype controllers are ensured during deployment itself. No need to add more.
+ // For fresh VM deployment only, all required controllers should be ensured.
+ return;
+ }
+ ensureScsiDiskControllers(vmMo, scsiDiskController, requiredNumScsiControllers, availableBusNum);
+ }
+
+ private void ensureScsiDiskControllers(VirtualMachineMO vmMo, String scsiDiskController, int requiredNumScsiControllers, int availableBusNum) throws Exception {
+ // Pick the sub type of scsi
+ if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.pvscsi) {
+ if (!vmMo.isPvScsiSupported()) {
+ String msg = "This VM doesn't support Vmware Paravirtual SCSI controller for virtual disks, because the virtual hardware version is less than 7.";
+ throw new Exception(msg);
+ }
+ vmMo.ensurePvScsiDeviceController(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.lsisas1068) {
+ vmMo.ensureLsiLogicSasDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.buslogic) {
+ vmMo.ensureBusLogicDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.lsilogic) {
+ vmMo.ensureLsiLogicDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ }
+ }
+
+ private void tearDownVm(VirtualMachineMO vmMo) throws Exception {
+
+ if (vmMo == null)
+ return;
+
+ boolean hasSnapshot = false;
+ hasSnapshot = vmMo.hasSnapshot();
+ if (!hasSnapshot)
+ vmMo.tearDownDevices(new Class<?>[]{VirtualDisk.class, VirtualEthernetCard.class});
+ else
+ vmMo.tearDownDevices(new Class<?>[]{VirtualEthernetCard.class});
+ vmMo.ensureScsiDeviceController();
+ }
+
+ private static DiskTO getIsoDiskTO(DiskTO[] disks) {
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ISO) {
+ return vol;
+ }
+ }
+ return null;
+ }
+
+ private static DiskTO[] sortVolumesByDeviceId(DiskTO[] volumes) {
+
+ List<DiskTO> listForSort = new ArrayList<DiskTO>();
+ for (DiskTO vol : volumes) {
+ listForSort.add(vol);
+ }
+ Collections.sort(listForSort, new Comparator<DiskTO>() {
+
+ @Override
+ public int compare(DiskTO arg0, DiskTO arg1) {
+ if (arg0.getDiskSeq() < arg1.getDiskSeq()) {
+ return -1;
+ } else if (arg0.getDiskSeq().equals(arg1.getDiskSeq())) {
+ return 0;
+ }
+
+ return 1;
+ }
+ });
+
+ return listForSort.toArray(new DiskTO[0]);
+ }
+
+ private void postDiskConfigBeforeStart(VirtualMachineMO vmMo, VirtualMachineTO vmSpec, DiskTO[] sortedDisks,
+ Map<String, Map<String, String>> iqnToData, VmwareHypervisorHost hyperHost, VmwareContext context, boolean installAsIs)
+ throws Exception {
+ VirtualMachineDiskInfoBuilder diskInfoBuilder = vmMo.getDiskInfoBuilder();
+
+ for (DiskTO vol : sortedDisks) {
+ //TODO: Map existing disks to the ones returned in the answer
+ if (vol.getType() == Volume.Type.ISO)
+ continue;
+
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+
+ VirtualMachineDiskInfo diskInfo = getMatchingExistingDisk(diskInfoBuilder, vol, hyperHost, context);
+ assert (diskInfo != null);
+
+ String[] diskChain = diskInfo.getDiskChain();
+ assert (diskChain.length > 0);
+
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ DatastoreFile file = new DatastoreFile(diskChain[0]);
+
+ if (managed) {
+ DatastoreFile originalFile = new DatastoreFile(volumeTO.getPath());
+
+ if (!file.getFileBaseName().equalsIgnoreCase(originalFile.getFileBaseName())) {
+ if (LOGGER.isInfoEnabled())
+ LOGGER.info("Detected disk-chain top file change on volume: " + volumeTO.getId() + " " + volumeTO.getPath() + " -> " + diskChain[0]);
+ }
+ } else {
+ if (!file.getFileBaseName().equalsIgnoreCase(volumeTO.getPath())) {
+ if (LOGGER.isInfoEnabled())
+ LOGGER.info("Detected disk-chain top file change on volume: " + volumeTO.getId() + " " + volumeTO.getPath() + " -> " + file.getFileBaseName());
+ }
+ }
+
+ VolumeObjectTO volInSpec = getVolumeInSpec(vmSpec, volumeTO);
+
+ if (volInSpec != null) {
+ if (managed) {
+ Map<String, String> data = new HashMap<>();
+
+ String datastoreVolumePath = diskChain[0];
+
+ data.put(StartAnswer.PATH, datastoreVolumePath);
+ data.put(StartAnswer.IMAGE_FORMAT, Storage.ImageFormat.OVA.toString());
+
+ iqnToData.put(details.get(DiskTO.IQN), data);
+
+ vol.setPath(datastoreVolumePath);
+ volumeTO.setPath(datastoreVolumePath);
+ volInSpec.setPath(datastoreVolumePath);
+ } else {
+ volInSpec.setPath(file.getFileBaseName());
+ }
+ volInSpec.setChainInfo(vmwareResource.getGson().toJson(diskInfo));
+ }
+ }
+ }
+
+ private void checkBootOptions(VirtualMachineTO vmSpec, VirtualMachineConfigSpec vmConfigSpec) {
+ String bootMode = null;
+ if (vmSpec.getDetails().containsKey(VmDetailConstants.BOOT_MODE)) {
+ bootMode = vmSpec.getDetails().get(VmDetailConstants.BOOT_MODE);
+ }
+ if (null == bootMode) {
+ bootMode = ApiConstants.BootType.BIOS.toString();
+ }
+
+ setBootOptions(vmSpec, bootMode, vmConfigSpec);
+ }
+
+ private void setBootOptions(VirtualMachineTO vmSpec, String bootMode, VirtualMachineConfigSpec vmConfigSpec) {
+ VirtualMachineBootOptions bootOptions = null;
+ if (StringUtils.isNotBlank(bootMode) && !bootMode.equalsIgnoreCase("bios")) {
+ vmConfigSpec.setFirmware("efi");
+ if (vmSpec.getDetails().containsKey(ApiConstants.BootType.UEFI.toString()) && "secure".equalsIgnoreCase(vmSpec.getDetails().get(ApiConstants.BootType.UEFI.toString()))) {
+ if (bootOptions == null) {
+ bootOptions = new VirtualMachineBootOptions();
+ }
+ bootOptions.setEfiSecureBootEnabled(true);
+ }
+ }
+ if (vmSpec.isEnterHardwareSetup()) {
+ if (bootOptions == null) {
+ bootOptions = new VirtualMachineBootOptions();
+ }
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("configuring VM '%s' to enter hardware setup",vmSpec.getName()));
+ }
+ bootOptions.setEnterBIOSSetup(vmSpec.isEnterHardwareSetup());
+ }
+ if (bootOptions != null) {
+ vmConfigSpec.setBootOptions(bootOptions);
+ }
+ }
+
+ private void resizeRootDiskOnVMStart(VirtualMachineMO vmMo, DiskTO rootDiskTO, VmwareHypervisorHost hyperHost, VmwareContext context) throws Exception {
+ final Pair<VirtualDisk, String> vdisk = vmwareResource.getVirtualDiskInfo(vmMo, VmwareResource.appendFileType(rootDiskTO.getPath(), VmwareResource.VMDK_EXTENSION));
+ assert (vdisk != null);
+
+ Long reqSize = 0L;
+ final VolumeObjectTO volumeTO = ((VolumeObjectTO) rootDiskTO.getData());
+ if (volumeTO != null) {
+ reqSize = volumeTO.getSize() / 1024;
+ }
+ final VirtualDisk disk = vdisk.first();
+ if (reqSize > disk.getCapacityInKB()) {
+ final VirtualMachineDiskInfo diskInfo = getMatchingExistingDisk(vmMo.getDiskInfoBuilder(), rootDiskTO, hyperHost, context);
+ assert (diskInfo != null);
+ final String[] diskChain = diskInfo.getDiskChain();
+
+ if (diskChain != null && diskChain.length > 1) {
+ LOGGER.warn("Disk chain length for the VM is greater than one, this is not supported");
+ throw new CloudRuntimeException("Unsupported VM disk chain length: " + diskChain.length);
+ }
+
+ boolean resizingSupported = false;
+ String deviceBusName = diskInfo.getDiskDeviceBusName();
+ if (deviceBusName != null && (deviceBusName.toLowerCase().contains("scsi") || deviceBusName.toLowerCase().contains("lsi"))) {
+ resizingSupported = true;
+ }
+ if (!resizingSupported) {
+ LOGGER.warn("Resizing of root disk is only support for scsi device/bus, the provide VM's disk device bus name is " + diskInfo.getDiskDeviceBusName());
+ throw new CloudRuntimeException("Unsupported VM root disk device bus: " + diskInfo.getDiskDeviceBusName());
+ }
+
+ disk.setCapacityInKB(reqSize);
+ VirtualMachineConfigSpec vmConfigSpec = new VirtualMachineConfigSpec();
+ VirtualDeviceConfigSpec deviceConfigSpec = new VirtualDeviceConfigSpec();
+ deviceConfigSpec.setDevice(disk);
+ deviceConfigSpec.setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ vmConfigSpec.getDeviceChange().add(deviceConfigSpec);
+ if (!vmMo.configureVm(vmConfigSpec)) {
+ throw new Exception("Failed to configure VM for given root disk size. vmName: " + vmMo.getName());
+ }
+ }
+ }
+
+ private static void postNvpConfigBeforeStart(VirtualMachineMO vmMo, VirtualMachineTO vmSpec) throws Exception {
+ /**
+ * We need to configure the port on the DV switch after the host is
+ * connected. So make this happen between the configure and start of
+ * the VM
+ */
+ int nicIndex = 0;
+ for (NicTO nicTo : sortNicsByDeviceId(vmSpec.getNics())) {
+ if (nicTo.getBroadcastType() == Networks.BroadcastDomainType.Lswitch) {
+ // We need to create a port with a unique vlan and pass the key to the nic device
+ LOGGER.trace("Nic " + nicTo.toString() + " is connected to an NVP logicalswitch");
+ VirtualDevice nicVirtualDevice = vmMo.getNicDeviceByIndex(nicIndex);
+ if (nicVirtualDevice == null) {
+ throw new Exception("Failed to find a VirtualDevice for nic " + nicIndex); //FIXME Generic exceptions are bad
+ }
+ VirtualDeviceBackingInfo backing = nicVirtualDevice.getBacking();
+ if (backing instanceof VirtualEthernetCardDistributedVirtualPortBackingInfo) {
+ // This NIC is connected to a Distributed Virtual Switch
+ VirtualEthernetCardDistributedVirtualPortBackingInfo portInfo = (VirtualEthernetCardDistributedVirtualPortBackingInfo) backing;
+ DistributedVirtualSwitchPortConnection port = portInfo.getPort();
+ String portKey = port.getPortKey();
+ String portGroupKey = port.getPortgroupKey();
+ String dvSwitchUuid = port.getSwitchUuid();
+
+ LOGGER.debug("NIC " + nicTo.toString() + " is connected to dvSwitch " + dvSwitchUuid + " pg " + portGroupKey + " port " + portKey);
+
+ ManagedObjectReference dvSwitchManager = vmMo.getContext().getVimClient().getServiceContent().getDvSwitchManager();
+ ManagedObjectReference dvSwitch = vmMo.getContext().getVimClient().getService().queryDvsByUuid(dvSwitchManager, dvSwitchUuid);
+
+ // Get all ports
+ DistributedVirtualSwitchPortCriteria criteria = new DistributedVirtualSwitchPortCriteria();
+ criteria.setInside(true);
+ criteria.getPortgroupKey().add(portGroupKey);
+ List<DistributedVirtualPort> dvPorts = vmMo.getContext().getVimClient().getService().fetchDVPorts(dvSwitch, criteria);
+
+ DistributedVirtualPort vmDvPort = null;
+ List<Integer> usedVlans = new ArrayList<Integer>();
+ for (DistributedVirtualPort dvPort : dvPorts) {
+ // Find the port for this NIC by portkey
+ if (portKey.equals(dvPort.getKey())) {
+ vmDvPort = dvPort;
+ }
+ VMwareDVSPortSetting settings = (VMwareDVSPortSetting) dvPort.getConfig().getSetting();
+ VmwareDistributedVirtualSwitchVlanIdSpec vlanId = (VmwareDistributedVirtualSwitchVlanIdSpec) settings.getVlan();
+ LOGGER.trace("Found port " + dvPort.getKey() + " with vlan " + vlanId.getVlanId());
+ if (vlanId.getVlanId() > 0 && vlanId.getVlanId() < 4095) {
+ usedVlans.add(vlanId.getVlanId());
+ }
+ }
+
+ if (vmDvPort == null) {
+ throw new Exception("Empty port list from dvSwitch for nic " + nicTo.toString());
+ }
+
+ DVPortConfigInfo dvPortConfigInfo = vmDvPort.getConfig();
+ VMwareDVSPortSetting settings = (VMwareDVSPortSetting) dvPortConfigInfo.getSetting();
+
+ VmwareDistributedVirtualSwitchVlanIdSpec vlanId = (VmwareDistributedVirtualSwitchVlanIdSpec) settings.getVlan();
+ BoolPolicy blocked = settings.getBlocked();
+ if (blocked.isValue() == Boolean.TRUE) {
+ LOGGER.trace("Port is blocked, set a vlanid and unblock");
+ DVPortConfigSpec dvPortConfigSpec = new DVPortConfigSpec();
+ VMwareDVSPortSetting edittedSettings = new VMwareDVSPortSetting();
+ // Unblock
+ blocked.setValue(Boolean.FALSE);
+ blocked.setInherited(Boolean.FALSE);
+ edittedSettings.setBlocked(blocked);
+ // Set vlan
+ int i;
+ for (i = 1; i < 4095; i++) {
+ if (!usedVlans.contains(i))
+ break;
+ }
+ vlanId.setVlanId(i); // FIXME should be a determined
+ // based on usage
+ vlanId.setInherited(false);
+ edittedSettings.setVlan(vlanId);
+
+ dvPortConfigSpec.setSetting(edittedSettings);
+ dvPortConfigSpec.setOperation("edit");
+ dvPortConfigSpec.setKey(portKey);
+ List<DVPortConfigSpec> dvPortConfigSpecs = new ArrayList<DVPortConfigSpec>();
+ dvPortConfigSpecs.add(dvPortConfigSpec);
+ ManagedObjectReference task = vmMo.getContext().getVimClient().getService().reconfigureDVPortTask(dvSwitch, dvPortConfigSpecs);
+ if (!vmMo.getContext().getVimClient().waitForTask(task)) {
+ throw new Exception("Failed to configure the dvSwitch port for nic " + nicTo.toString());
+ }
+ LOGGER.debug("NIC " + nicTo.toString() + " connected to vlan " + i);
+ } else {
+ LOGGER.trace("Port already configured and set to vlan " + vlanId.getVlanId());
+ }
+ } else if (backing instanceof VirtualEthernetCardNetworkBackingInfo) {
+ // This NIC is connected to a Virtual Switch
+ // Nothing to do
+ } else if (backing instanceof VirtualEthernetCardOpaqueNetworkBackingInfo) {
+ //if NSX API VERSION >= 4.2, connect to br-int (nsx.network), do not create portgroup else previous behaviour
+ //OK, connected to OpaqueNetwork
+ } else {
+ LOGGER.error("nic device backing is of type " + backing.getClass().getName());
+ throw new Exception("Incompatible backing for a VirtualDevice for nic " + nicIndex); //FIXME Generic exceptions are bad
+ }
+ }
+ nicIndex++;
+ }
+ }
+
+ private VirtualMachineDiskInfo getMatchingExistingDisk(VirtualMachineDiskInfoBuilder diskInfoBuilder, DiskTO vol, VmwareHypervisorHost hyperHost, VmwareContext context)
+ throws Exception {
+ if (diskInfoBuilder != null) {
+ VolumeObjectTO volume = (VolumeObjectTO) vol.getData();
+
+ String dsName = null;
+ String diskBackingFileBaseName = null;
+
+ Map<String, String> details = vol.getDetails();
+ boolean isManaged = details != null && Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+
+ if (isManaged) {
+ String iScsiName = details.get(DiskTO.IQN);
+
+ // if the storage is managed, iScsiName should not be null
+ dsName = VmwareResource.getDatastoreName(iScsiName);
+
+ diskBackingFileBaseName = new DatastoreFile(volume.getPath()).getFileBaseName();
+ } else {
+ ManagedObjectReference morDs = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, volume.getDataStore().getUuid());
+ DatastoreMO dsMo = new DatastoreMO(context, morDs);
+
+ dsName = dsMo.getName();
+
+ diskBackingFileBaseName = volume.getPath();
+ }
+
+ VirtualMachineDiskInfo diskInfo = diskInfoBuilder.getDiskInfoByBackingFileBaseName(diskBackingFileBaseName, dsName);
+ if (diskInfo != null) {
+ LOGGER.info("Found existing disk info from volume path: " + volume.getPath());
+ return diskInfo;
+ } else {
+ String chainInfo = volume.getChainInfo();
+ if (chainInfo != null) {
+ VirtualMachineDiskInfo infoInChain = vmwareResource.getGson().fromJson(chainInfo, VirtualMachineDiskInfo.class);
+ if (infoInChain != null) {
+ String[] disks = infoInChain.getDiskChain();
+ if (disks.length > 0) {
+ for (String diskPath : disks) {
+ DatastoreFile file = new DatastoreFile(diskPath);
+ diskInfo = diskInfoBuilder.getDiskInfoByBackingFileBaseName(file.getFileBaseName(), dsName);
+ if (diskInfo != null) {
+ LOGGER.info("Found existing disk from chain info: " + diskPath);
+ return diskInfo;
+ }
+ }
+ }
+
+ if (diskInfo == null) {
+ diskInfo = diskInfoBuilder.getDiskInfoByDeviceBusName(infoInChain.getDiskDeviceBusName());
+ if (diskInfo != null) {
+ LOGGER.info("Found existing disk from from chain device bus information: " + infoInChain.getDiskDeviceBusName());
+ return diskInfo;
+ }
+ }
+ }
+ }
+ }
+ }
+
+ return null;
+ }
+
+ private static VolumeObjectTO getVolumeInSpec(VirtualMachineTO vmSpec, VolumeObjectTO srcVol) {
+ for (DiskTO disk : vmSpec.getDisks()) {
+ if (disk.getData() instanceof VolumeObjectTO) {
+ VolumeObjectTO vol = (VolumeObjectTO) disk.getData();
+ if (vol.getId() == srcVol.getId())
+ return vol;
+ }
+ }
+
+ return null;
+ }
+
+ /**
+ * Generate the mac sequence from the nics.
+ */
+ protected String generateMacSequence(NicTO[] nics) {
+ if (nics.length == 0) {
+ return "";
+ }
+
+ StringBuffer sbMacSequence = new StringBuffer();
+ for (NicTO nicTo : sortNicsByDeviceId(nics)) {
+ sbMacSequence.append(nicTo.getMac()).append("|");
+ }
+ if (!sbMacSequence.toString().isEmpty()) {
+ sbMacSequence.deleteCharAt(sbMacSequence.length() - 1); //Remove extra '|' char appended at the end
+ }
+
+ return sbMacSequence.toString();
+ }
+
+ static NicTO[] sortNicsByDeviceId(NicTO[] nics) {
+
+ List<NicTO> listForSort = new ArrayList<NicTO>();
+ for (NicTO nic : nics) {
+ listForSort.add(nic);
+ }
+ Collections.sort(listForSort, new Comparator<NicTO>() {
+
+ @Override
+ public int compare(NicTO arg0, NicTO arg1) {
+ if (arg0.getDeviceId() < arg1.getDeviceId()) {
+ return -1;
+ } else if (arg0.getDeviceId() == arg1.getDeviceId()) {
+ return 0;
+ }
+
+ return 1;
+ }
+ });
+
+ return listForSort.toArray(new NicTO[0]);
+ }
+
+ private static void configBasicExtraOption(List<OptionValue> extraOptions, VirtualMachineTO vmSpec) {
+ OptionValue newVal = new OptionValue();
+ newVal.setKey("machine.id");
+ newVal.setValue(vmSpec.getBootArgs());
+ extraOptions.add(newVal);
+
+ newVal = new OptionValue();
+ newVal.setKey("devices.hotplug");
+ newVal.setValue("true");
+ extraOptions.add(newVal);
+ }
+
+ private static void configNvpExtraOption(List<OptionValue> extraOptions, VirtualMachineTO vmSpec, Map<String, String> nicUuidToDvSwitchUuid) {
+ /**
+ * Extra Config : nvp.vm-uuid = uuid
+ * - Required for Nicira NVP integration
+ */
+ OptionValue newVal = new OptionValue();
+ newVal.setKey("nvp.vm-uuid");
+ newVal.setValue(vmSpec.getUuid());
+ extraOptions.add(newVal);
+
+ /**
+ * Extra Config : nvp.iface-id.<num> = uuid
+ * - Required for Nicira NVP integration
+ */
+ int nicNum = 0;
+ for (NicTO nicTo : sortNicsByDeviceId(vmSpec.getNics())) {
+ if (nicTo.getUuid() != null) {
+ newVal = new OptionValue();
+ newVal.setKey("nvp.iface-id." + nicNum);
+ newVal.setValue(nicTo.getUuid());
+ extraOptions.add(newVal);
+ }
+ nicNum++;
+ }
+ }
+
+ private static void configCustomExtraOption(List<OptionValue> extraOptions, VirtualMachineTO vmSpec) {
+ // we no longer to validation anymore
+ for (Map.Entry<String, String> entry : vmSpec.getDetails().entrySet()) {
+ if (entry.getKey().equalsIgnoreCase(VmDetailConstants.BOOT_MODE)) {
+ continue;
+ }
+ OptionValue newVal = new OptionValue();
+ newVal.setKey(entry.getKey());
+ newVal.setValue(entry.getValue());
+ extraOptions.add(newVal);
+ }
+ }
+
+ private class VmAlreadyExistsInVcenter extends Exception {
+ public VmAlreadyExistsInVcenter(String msg) {
+ }
+ }
+
+ private class VirtualMachineData {
+ String vmName = null;
+ VirtualMachineFileInfo vmFileInfo = null;
+ VirtualMachineFileLayoutEx vmFileLayout = null;
+ List<DatastoreMO> datastores = new ArrayList<>();
+ }
+
+ private class VirtualMachineRecycler {
+ private String vmInternalCSName;
+ private Pair<String, String> controllerInfo;
+ private Boolean systemVm;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private DiskControllerType systemVmScsiControllerType;
+ private int firstScsiControllerBusNum;
+ private int numScsiControllerForSystemVm;
+ private VirtualMachineDiskInfoBuilder diskInfoBuilder;
+ private boolean hasSnapshot;
+ private VirtualDevice[] nicDevices;
+
+ public VirtualMachineRecycler(String vmInternalCSName, Pair<String, String> controllerInfo, Boolean systemVm, VmwareHypervisorHost hyperHost, VirtualMachineMO vmMo,
+ DiskControllerType systemVmScsiControllerType, int firstScsiControllerBusNum, int numScsiControllerForSystemVm) {
+ this.vmInternalCSName = vmInternalCSName;
+ this.controllerInfo = controllerInfo;
+ this.systemVm = systemVm;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.systemVmScsiControllerType = systemVmScsiControllerType;
+ this.firstScsiControllerBusNum = firstScsiControllerBusNum;
+ this.numScsiControllerForSystemVm = numScsiControllerForSystemVm;
+ }
+
+ public VirtualMachineDiskInfoBuilder getDiskInfoBuilder() {
+ return diskInfoBuilder;
+ }
+
+ public boolean isHasSnapshot() {
+ return hasSnapshot;
+ }
+
+ public VirtualMachineRecycler invoke() throws Exception {
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Found vm " + vmInternalCSName + " at other host, relocate to " + hyperHost.getHyperHostName());
+ }
+
+ takeVmFromOtherHyperHost(hyperHost, vmInternalCSName);
+
+ if (VmwareResource.getVmPowerState(vmMo) != VirtualMachine.PowerState.PowerOff)
+ vmMo.safePowerOff(vmwareResource.getShutdownWaitMs());
+
+ diskInfoBuilder = vmMo.getDiskInfoBuilder();
+ hasSnapshot = vmMo.hasSnapshot();
+ nicDevices = vmMo.getNicDevices();
+ if (!hasSnapshot)
+ vmMo.tearDownDevices(new Class<?>[] {VirtualDisk.class, VirtualEthernetCard.class});
+ else
+ vmMo.tearDownDevices(new Class<?>[] {VirtualEthernetCard.class});
+
+ if (systemVm) {
+ // System volumes doesn't require more than 1 SCSI controller as there is no requirement for data volumes.
+ ensureScsiDiskControllers(vmMo, systemVmScsiControllerType.toString(), numScsiControllerForSystemVm, firstScsiControllerBusNum);
+ } else {
+ ensureDiskControllers(vmMo, controllerInfo);
+ }
+ return this;
+ }
+
+ private VirtualMachineMO takeVmFromOtherHyperHost(VmwareHypervisorHost hyperHost, String vmName) throws Exception {
+
+ VirtualMachineMO vmMo = hyperHost.findVmOnPeerHyperHost(vmName);
+ if (vmMo != null) {
+ ManagedObjectReference morTargetPhysicalHost = hyperHost.findMigrationTarget(vmMo);
+ if (morTargetPhysicalHost == null) {
+ String msg = "VM " + vmName + " is on other host and we have no resource available to migrate and start it here";
+ LOGGER.error(msg);
+ throw new Exception(msg);
+ }
+
+ if (!vmMo.relocate(morTargetPhysicalHost)) {
+ String msg = "VM " + vmName + " is on other host and we failed to relocate it here";
+ LOGGER.error(msg);
+ throw new Exception(msg);
+ }
+
+ return vmMo;
+ }
+ return null;
+ }
+
+ public VirtualDevice[] getNicDevices() {
+ return nicDevices;
+ }
+ }
+
+ private class PrepareSytemVMPatchISOMethod {
+ private VmwareManager mgr;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int i;
+ private int ideUnitNumber;
+
+ public PrepareSytemVMPatchISOMethod(VmwareManager mgr, VmwareHypervisorHost hyperHost, VirtualMachineMO vmMo, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int i, int ideUnitNumber) {
+ this.mgr = mgr;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.i = i;
+ this.ideUnitNumber = ideUnitNumber;
+ }
+
+ public int getI() {
+ return i;
+ }
+
+ public int getIdeUnitNumber() {
+ return ideUnitNumber;
+ }
+
+ public PrepareSytemVMPatchISOMethod invoke() throws Exception {
+ // attach ISO (for patching of system VM)
+ DatastoreMO secDsMo = getDatastoreMOForSecStore(mgr, hyperHost);
+
+ deviceConfigSpecArray[i] = new VirtualDeviceConfigSpec();
+ Pair<VirtualDevice, Boolean> isoInfo = VmwareHelper
+ .prepareIsoDevice(vmMo, String.format("[%s] systemvm/%s", secDsMo.getName(), mgr.getSystemVMIsoFileNameOnDatastore()), secDsMo.getMor(), true, true,
+ ideUnitNumber++, i + 1);
+ deviceConfigSpecArray[i].setDevice(isoInfo.first());
+ if (isoInfo.second()) {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at new device " + vmwareResource.getGson().toJson(isoInfo.first()));
+ deviceConfigSpecArray[i].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+ } else {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+ deviceConfigSpecArray[i].setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ }
+ i++;
+ return this;
+ }
+
+ }
+ private DatastoreMO getDatastoreMOForSecStore(VmwareManager mgr, VmwareHypervisorHost hyperHost) throws Exception {
+ Pair<String, Long> secStoreUrlAndId = mgr.getSecondaryStorageStoreUrlAndId(Long.parseLong(vmwareResource.getDcId()));
+ String secStoreUrl = secStoreUrlAndId.first();
+ Long secStoreId = secStoreUrlAndId.second();
+ if (secStoreUrl == null) {
+ String msg = "secondary storage for dc " + vmwareResource.getDcId() + " is not ready yet?";
+ throw new Exception(msg);
+ }
+ mgr.prepareSecondaryStorageStore(secStoreUrl, secStoreId);
+
+ ManagedObjectReference morSecDs = vmwareResource.prepareSecondaryDatastoreOnHost(secStoreUrl);
+ if (morSecDs == null) {
+ String msg = "Failed to prepare secondary storage on host, secondary store url: " + secStoreUrl;
+ throw new Exception(msg);
+ }
+ return new DatastoreMO(hyperHost.getContext(), morSecDs);
+ }
+
+ /**
+ // Setup ROOT/DATA disk devices
+ */
+ private class DiskSetup {
+ private VirtualMachineTO vmSpec;
+ private DiskTO rootDiskTO;
+ private Pair<String, String> controllerInfo;
+ private VmwareContext context;
+ private DatacenterMO dcMo;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private DiskTO[] disks;
+ private HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails;
+ private VirtualMachineDiskInfoBuilder diskInfoBuilder;
+ private boolean hasSnapshot;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int deviceCount;
+ private int ideUnitNumber;
+ private int scsiUnitNumber;
+ private int ideControllerKey;
+ private int scsiControllerKey;
+ private DiskTO[] sortedDisks;
+ private boolean installAsIs;
+
+ public DiskSetup(VirtualMachineTO vmSpec, DiskTO rootDiskTO, Pair<String, String> controllerInfo, VmwareContext context, DatacenterMO dcMo, VmwareHypervisorHost hyperHost,
+ VirtualMachineMO vmMo, DiskTO[] disks, HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails, VirtualMachineDiskInfoBuilder diskInfoBuilder,
+ boolean hasSnapshot, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount, int ideUnitNumber, int scsiUnitNumber, int ideControllerKey, int scsiControllerKey, boolean installAsIs) {
+ this.vmSpec = vmSpec;
+ this.rootDiskTO = rootDiskTO;
+ this.controllerInfo = controllerInfo;
+ this.context = context;
+ this.dcMo = dcMo;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.disks = disks;
+ this.dataStoresDetails = dataStoresDetails;
+ this.diskInfoBuilder = diskInfoBuilder;
+ this.hasSnapshot = hasSnapshot;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.deviceCount = deviceCount;
+ this.ideUnitNumber = ideUnitNumber;
+ this.scsiUnitNumber = scsiUnitNumber;
+ this.ideControllerKey = ideControllerKey;
+ this.scsiControllerKey = scsiControllerKey;
+ this.installAsIs = installAsIs;
+ }
+
+ public DiskTO getRootDiskTO() {
+ return rootDiskTO;
+ }
+
+ public int getDeviceCount() {
+ return deviceCount;
+ }
+
+ public DiskTO[] getSortedDisks() {
+ return sortedDisks;
+ }
+
+ public DiskSetup invoke() throws Exception {
+ int controllerKey;
+ sortedDisks = sortVolumesByDeviceId(disks);
+ for (DiskTO vol : sortedDisks) {
+ if (vol.getType() == Volume.Type.ISO || installAsIs) {
+ continue;
+ }
+
+ VirtualMachineDiskInfo matchingExistingDisk = getMatchingExistingDisk(diskInfoBuilder, vol, hyperHost, context);
+ controllerKey = getDiskController(matchingExistingDisk, vol, vmSpec, ideControllerKey, scsiControllerKey);
+ String diskController = getDiskController(vmMo, matchingExistingDisk, vol, controllerInfo);
+
+ if (DiskControllerType.getType(diskController) == DiskControllerType.osdefault) {
+ diskController = vmMo.getRecommendedDiskController(null);
+ }
+ if (DiskControllerType.getType(diskController) == DiskControllerType.ide) {
+ controllerKey = vmMo.getIDEControllerKey(ideUnitNumber);
+ if (vol.getType() == Volume.Type.DATADISK) {
+ // Could be result of flip due to user configured setting or "osdefault" for data disks
+ // Ensure maximum of 2 data volumes over IDE controller, 3 includeing root volume
+ if (vmMo.getNumberOfVirtualDisks() > 3) {
+ throw new CloudRuntimeException(
+ "Found more than 3 virtual disks attached to this VM [" + vmMo.getVmName() + "]. Unable to implement the disks over " + diskController + " controller, as maximum number of devices supported over IDE controller is 4 includeing CDROM device.");
+ }
+ }
+ } else {
+ if (VmwareHelper.isReservedScsiDeviceNumber(scsiUnitNumber)) {
+ scsiUnitNumber++;
+ }
+
+ controllerKey = vmMo.getScsiDiskControllerKeyNoException(diskController, scsiUnitNumber);
+ if (controllerKey == -1) {
+ // This may happen for ROOT legacy VMs which doesn't have recommended disk controller when global configuration parameter 'vmware.root.disk.controller' is set to "osdefault"
+ // Retrieve existing controller and use.
+ Ternary<Integer, Integer, DiskControllerType> vmScsiControllerInfo = vmMo.getScsiControllerInfo();
+ DiskControllerType existingControllerType = vmScsiControllerInfo.third();
+ controllerKey = vmMo.getScsiDiskControllerKeyNoException(existingControllerType.toString(), scsiUnitNumber);
+ }
+ }
+ if (!hasSnapshot) {
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+
+ VolumeObjectTO volumeTO = (VolumeObjectTO)vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+ String iScsiName = null;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ iScsiName = details.get(DiskTO.IQN);
+ }
+
+ // if the storage is managed, iScsiName should not be null
+ String datastoreName = managed ? VmwareResource.getDatastoreName(iScsiName) : primaryStore.getUuid();
+ Pair<ManagedObjectReference, DatastoreMO> volumeDsDetails = dataStoresDetails.get(datastoreName);
+
+ assert (volumeDsDetails != null);
+
+ String[] diskChain = syncDiskChain(dcMo, vmMo, vmSpec, vol, matchingExistingDisk, dataStoresDetails);
+
+ int deviceNumber = -1;
+ if (controllerKey == vmMo.getIDEControllerKey(ideUnitNumber)) {
+ deviceNumber = ideUnitNumber % VmwareHelper.MAX_ALLOWED_DEVICES_IDE_CONTROLLER;
+ ideUnitNumber++;
+ } else {
+ deviceNumber = scsiUnitNumber % VmwareHelper.MAX_ALLOWED_DEVICES_SCSI_CONTROLLER;
+ scsiUnitNumber++;
+ }
+ VirtualDevice device = VmwareHelper.prepareDiskDevice(vmMo, null, controllerKey, diskChain, volumeDsDetails.first(), deviceNumber, deviceCount + 1);
+ if (vol.getType() == Volume.Type.ROOT)
+ rootDiskTO = vol;
+ deviceConfigSpecArray[deviceCount].setDevice(device);
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare volume at new device " + vmwareResource.getGson().toJson(device));
+
+ deviceCount++;
+ } else {
+ if (controllerKey == vmMo.getIDEControllerKey(ideUnitNumber))
+ ideUnitNumber++;
+ else
+ scsiUnitNumber++;
+ }
+ }
+ return this;
+ }
+
+ private int getDiskController(VirtualMachineDiskInfo matchingExistingDisk, DiskTO vol, VirtualMachineTO vmSpec, int ideControllerKey, int scsiControllerKey) {
+
+ int controllerKey;
+ if (matchingExistingDisk != null) {
+ LOGGER.info("Chose disk controller based on existing information: " + matchingExistingDisk.getDiskDeviceBusName());
+ if (matchingExistingDisk.getDiskDeviceBusName().startsWith("ide"))
+ return ideControllerKey;
+ else
+ return scsiControllerKey;
+ }
+
+ if (vol.getType() == Volume.Type.ROOT) {
+ Map<String, String> vmDetails = vmSpec.getDetails();
+ if (vmDetails != null && vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER) != null) {
+ if (vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER).equalsIgnoreCase("scsi")) {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> scsi, based on root disk controller settings: "
+ + vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER));
+ controllerKey = scsiControllerKey;
+ } else {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> ide, based on root disk controller settings: "
+ + vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER));
+ controllerKey = ideControllerKey;
+ }
+ } else {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> scsi. due to null root disk controller setting");
+ controllerKey = scsiControllerKey;
+ }
+
+ } else {
+ // DATA volume always use SCSI device
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> scsi");
+ controllerKey = scsiControllerKey;
+ }
+
+ return controllerKey;
+ }
+
+ private String getDiskController(VirtualMachineMO vmMo, VirtualMachineDiskInfo matchingExistingDisk, DiskTO vol, Pair<String, String> controllerInfo) throws Exception {
+ int controllerKey;
+ DiskControllerType controllerType = DiskControllerType.none;
+ if (matchingExistingDisk != null) {
+ String currentBusName = matchingExistingDisk.getDiskDeviceBusName();
+ if (currentBusName != null) {
+ LOGGER.info("Chose disk controller based on existing information: " + currentBusName);
+ if (currentBusName.startsWith("ide")) {
+ controllerType = DiskControllerType.ide;
+ } else if (currentBusName.startsWith("scsi")) {
+ controllerType = DiskControllerType.scsi;
+ }
+ }
+ if (controllerType == DiskControllerType.scsi || controllerType == DiskControllerType.none) {
+ Ternary<Integer, Integer, DiskControllerType> vmScsiControllerInfo = vmMo.getScsiControllerInfo();
+ controllerType = vmScsiControllerInfo.third();
+ }
+ return controllerType.toString();
+ }
+
+ if (vol.getType() == Volume.Type.ROOT) {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> " + controllerInfo.first()
+ + ", based on root disk controller settings at global configuration setting.");
+ return controllerInfo.first();
+ } else {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> " + controllerInfo.second()
+ + ", based on default data disk controller setting i.e. Operating system recommended."); // Need to bring in global configuration setting & template level setting.
+ return controllerInfo.second();
+ }
+ }
+
+ // return the finalized disk chain for startup, from top to bottom
+ private String[] syncDiskChain(DatacenterMO dcMo, VirtualMachineMO vmMo, VirtualMachineTO vmSpec, DiskTO vol, VirtualMachineDiskInfo diskInfo,
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails) throws Exception {
+
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ Map<String, String> details = vol.getDetails();
+ boolean isManaged = false;
+ String iScsiName = null;
+
+ if (details != null) {
+ isManaged = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ iScsiName = details.get(DiskTO.IQN);
+ }
+
+ // if the storage is managed, iScsiName should not be null
+ String datastoreName = isManaged ? VmwareResource.getDatastoreName(iScsiName) : primaryStore.getUuid();
+ Pair<ManagedObjectReference, DatastoreMO> volumeDsDetails = dataStoresDetails.get(datastoreName);
+
+ if (volumeDsDetails == null) {
+ throw new Exception("Primary datastore " + primaryStore.getUuid() + " is not mounted on host.");
+ }
+
+ DatastoreMO dsMo = volumeDsDetails.second();
+
+ // we will honor vCenter's meta if it exists
+ if (diskInfo != null) {
+ // to deal with run-time upgrade to maintain the new datastore folder structure
+ String disks[] = diskInfo.getDiskChain();
+ for (int i = 0; i < disks.length; i++) {
+ DatastoreFile file = new DatastoreFile(disks[i]);
+ if (!isManaged && file.getDir() != null && file.getDir().isEmpty()) {
+ LOGGER.info("Perform run-time datastore folder upgrade. sync " + disks[i] + " to VM folder");
+ disks[i] = VmwareStorageLayoutHelper.syncVolumeToVmDefaultFolder(dcMo, vmMo.getName(), dsMo, file.getFileBaseName(), VmwareManager.s_vmwareSearchExcludeFolder.value());
+ }
+ }
+ return disks;
+ }
+
+ final String datastoreDiskPath;
+
+ if (isManaged) {
+ String vmdkPath = new DatastoreFile(volumeTO.getPath()).getFileBaseName();
+
+ if (volumeTO.getVolumeType() == Volume.Type.ROOT) {
+ if (vmdkPath == null) {
+ vmdkPath = volumeTO.getName();
+ }
+
+ datastoreDiskPath = VmwareStorageLayoutHelper.syncVolumeToVmDefaultFolder(dcMo, vmMo.getName(), dsMo, vmdkPath);
+ } else {
+ if (vmdkPath == null) {
+ vmdkPath = dsMo.getName();
+ }
+
+ datastoreDiskPath = dsMo.getDatastorePath(vmdkPath + VmwareResource.VMDK_EXTENSION);
+ }
+ } else {
+ datastoreDiskPath = VmwareStorageLayoutHelper.syncVolumeToVmDefaultFolder(dcMo, vmMo.getName(), dsMo, volumeTO.getPath(), VmwareManager.s_vmwareSearchExcludeFolder.value());
+ }
+
+ if (!dsMo.fileExists(datastoreDiskPath)) {
+ LOGGER.warn("Volume " + volumeTO.getId() + " does not seem to exist on datastore, out of sync? path: " + datastoreDiskPath);
+ }
+
+ return new String[]{datastoreDiskPath};
+ }
+ }
+
+ /**
+ // Setup NIC devices
+ */
+ private class NicSetup {
+ private StartCommand cmd;
+ private VirtualMachineTO vmSpec;
+ private String vmInternalCSName;
+ private VmwareContext context;
+ private VmwareManager mgr;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private NicTO[] nics;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int deviceCount;
+ private int nicMask;
+ private int nicCount;
+ private Map<String, String> nicUuidToDvSwitchUuid;
+ private VirtualDevice[] nicDevices;
+
+ public NicSetup(StartCommand cmd, VirtualMachineTO vmSpec, String vmInternalCSName, VmwareContext context, VmwareManager mgr, VmwareHypervisorHost hyperHost,
+ VirtualMachineMO vmMo, NicTO[] nics, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount, VirtualDevice[] nicDevices) {
+ this.cmd = cmd;
+ this.vmSpec = vmSpec;
+ this.vmInternalCSName = vmInternalCSName;
+ this.context = context;
+ this.mgr = mgr;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.nics = nics;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.deviceCount = deviceCount;
+ this.nicDevices = nicDevices;
+ }
+
+ public int getDeviceCount() {
+ return deviceCount;
+ }
+
+ public int getNicMask() {
+ return nicMask;
+ }
+
+ public int getNicCount() {
+ return nicCount;
+ }
+
+ public Map<String, String> getNicUuidToDvSwitchUuid() {
+ return nicUuidToDvSwitchUuid;
+ }
+
+ public NicSetup invoke() throws Exception {
+ VirtualDevice nic;
+ nicMask = 0;
+ nicCount = 0;
+
+ if (vmSpec.getType() == VirtualMachine.Type.DomainRouter) {
+ doDomainRouterSetup();
+ }
+
+ VirtualEthernetCardType nicDeviceType = VirtualEthernetCardType.valueOf(vmSpec.getDetails().get(VmDetailConstants.NIC_ADAPTER));
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("VM " + vmInternalCSName + " will be started with NIC device type: " + nicDeviceType);
+ }
+
+ NiciraNvpApiVersion.logNiciraApiVersion();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.debug(String.format("deciding for VM '%s' to use orchestrated NICs or as is.", vmInternalCSName));
+ }
+ nicUuidToDvSwitchUuid = new HashMap<>();
+ LOGGER.info(String.format("adding %d nics to VM '%s'", nics.length, vmInternalCSName));
+ for (NicTO nicTo : sortNicsByDeviceId(nics)) {
+ LOGGER.info("Prepare NIC device based on NicTO: " + vmwareResource.getGson().toJson(nicTo));
+
+ boolean configureVServiceInNexus = (nicTo.getType() == Networks.TrafficType.Guest) && (vmSpec.getDetails().containsKey("ConfigureVServiceInNexus"));
+ VirtualMachine.Type vmType = cmd.getVirtualMachine().getType();
+ Pair<ManagedObjectReference, String> networkInfo = vmwareResource.prepareNetworkFromNicInfo(vmMo.getRunningHost(), nicTo, configureVServiceInNexus, vmType);
+ if ((nicTo.getBroadcastType() != Networks.BroadcastDomainType.Lswitch) || (nicTo.getBroadcastType() == Networks.BroadcastDomainType.Lswitch && NiciraNvpApiVersion.isApiVersionLowerThan("4.2"))) {
+ if (VmwareHelper.isDvPortGroup(networkInfo.first())) {
+ String dvSwitchUuid;
+ ManagedObjectReference dcMor = hyperHost.getHyperHostDatacenter();
+ DatacenterMO dataCenterMo = new DatacenterMO(context, dcMor);
+ ManagedObjectReference dvsMor = dataCenterMo.getDvSwitchMor(networkInfo.first());
+ dvSwitchUuid = dataCenterMo.getDvSwitchUuid(dvsMor);
+ LOGGER.info("Preparing NIC device on dvSwitch : " + dvSwitchUuid);
+ nic = VmwareHelper.prepareDvNicDevice(vmMo, networkInfo.first(), nicDeviceType, networkInfo.second(), dvSwitchUuid, nicTo.getMac(), deviceCount + 1, true, true);
+ if (nicTo.getUuid() != null) {
+ nicUuidToDvSwitchUuid.put(nicTo.getUuid(), dvSwitchUuid);
+ }
+ } else {
+ LOGGER.info("Preparing NIC device on network " + networkInfo.second());
+ nic = VmwareHelper.prepareNicDevice(vmMo, networkInfo.first(), nicDeviceType, networkInfo.second(), nicTo.getMac(), deviceCount + 1, true, true);
+ }
+ } else {
+ //if NSX API VERSION >= 4.2, connect to br-int (nsx.network), do not create portgroup else previous behaviour
+ nic = VmwareHelper.prepareNicOpaque(vmMo, nicDeviceType, networkInfo.second(), nicTo.getMac(), deviceCount + 1, true, true);
+ }
+
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ deviceConfigSpecArray[deviceCount].setDevice(nic);
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare NIC at new device " + vmwareResource.getGson().toJson(deviceConfigSpecArray[deviceCount]));
+
+ // this is really a hacking for DomR, upon DomR startup, we will reset all the NIC allocation after eth3
+ if (nicCount < 3)
+ nicMask |= (1 << nicCount);
+
+ deviceCount++;
+ nicCount++;
+ }
+ return this;
+ }
+
+ private void doDomainRouterSetup() throws Exception {
+ int extraPublicNics = mgr.getRouterExtraPublicNics();
+ if (extraPublicNics > 0 && vmSpec.getDetails().containsKey("PeerRouterInstanceName")) {
+ //Set identical MAC address for RvR on extra public interfaces
+ String peerRouterInstanceName = vmSpec.getDetails().get("PeerRouterInstanceName");
+
+ VirtualMachineMO peerVmMo = hyperHost.findVmOnHyperHost(peerRouterInstanceName);
+ if (peerVmMo == null) {
+ peerVmMo = hyperHost.findVmOnPeerHyperHost(peerRouterInstanceName);
+ }
+
+ if (peerVmMo != null) {
+ String oldMacSequence = generateMacSequence(nics);
+
+ for (int nicIndex = nics.length - extraPublicNics; nicIndex < nics.length; nicIndex++) {
+ VirtualDevice nicDevice = peerVmMo.getNicDeviceByIndex(nics[nicIndex].getDeviceId());
+ if (nicDevice != null) {
+ String mac = ((VirtualEthernetCard)nicDevice).getMacAddress();
+ if (mac != null) {
+ LOGGER.info("Use same MAC as previous RvR, the MAC is " + mac + " for extra NIC with device id: " + nics[nicIndex].getDeviceId());
+ nics[nicIndex].setMac(mac);
+ }
+ }
+ }
+
+ if (!StringUtils.isBlank(vmSpec.getBootArgs())) {
+ String newMacSequence = generateMacSequence(nics);
+ vmSpec.setBootArgs(vmwareResource.replaceNicsMacSequenceInBootArgs(oldMacSequence, newMacSequence, vmSpec));
+ }
+ }
+ }
+ }
+ }
+
+ private class IsoSetup {
+ private VirtualMachineTO vmSpec;
+ private VmwareManager mgr;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private DiskTO[] disks;
+ private DiskTO volIso;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int deviceCount;
+ private int ideUnitNumber;
+
+ public IsoSetup(VirtualMachineTO vmSpec, VmwareManager mgr, VmwareHypervisorHost hyperHost, VirtualMachineMO vmMo, DiskTO[] disks, DiskTO volIso,
+ VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount, int ideUnitNumber) {
+ this.vmSpec = vmSpec;
+ this.mgr = mgr;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.disks = disks;
+ this.volIso = volIso;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.deviceCount = deviceCount;
+ this.ideUnitNumber = ideUnitNumber;
+ }
+
+ public int getDeviceCount() {
+ return deviceCount;
+ }
+
+ public int getIdeUnitNumber() {
+ return ideUnitNumber;
+ }
+
+ public IsoSetup invoke() throws Exception {
+ //
+ // Setup ISO device
+ //
+
+ // vAPP ISO
+ // FR37 the native deploy mechs should create this for us
Review comment:
```suggestion
// the native deploy mechs should create this for us
```
, this really needs checking!
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/StartCommandExecutor.java
##########
@@ -0,0 +1,2247 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.hypervisor.vmware.resource;
+
+import java.io.File;
+import java.rmi.RemoteException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.cloud.agent.api.to.DataTO;
+import com.vmware.vim25.VirtualDiskFlatVer2BackingInfo;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.storage.configdrive.ConfigDrive;
+import org.apache.cloudstack.storage.to.TemplateObjectTO;
+import org.apache.cloudstack.storage.to.VolumeObjectTO;
+import org.apache.cloudstack.utils.volume.VirtualMachineDiskInfo;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.commons.lang.ArrayUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.log4j.Logger;
+
+import com.cloud.agent.api.Command;
+import com.cloud.agent.api.StartAnswer;
+import com.cloud.agent.api.StartCommand;
+import com.cloud.agent.api.storage.OVFPropertyTO;
+import com.cloud.agent.api.to.DataStoreTO;
+import com.cloud.agent.api.to.DiskTO;
+import com.cloud.agent.api.to.NfsTO;
+import com.cloud.agent.api.to.NicTO;
+import com.cloud.agent.api.to.VirtualMachineTO;
+import com.cloud.configuration.Resource;
+import com.cloud.hypervisor.vmware.VmwareResourceException;
+import com.cloud.hypervisor.vmware.manager.VmwareManager;
+import com.cloud.hypervisor.vmware.mo.CustomFieldConstants;
+import com.cloud.hypervisor.vmware.mo.DatacenterMO;
+import com.cloud.hypervisor.vmware.mo.DatastoreFile;
+import com.cloud.hypervisor.vmware.mo.DatastoreMO;
+import com.cloud.hypervisor.vmware.mo.DiskControllerType;
+import com.cloud.hypervisor.vmware.mo.HostMO;
+import com.cloud.hypervisor.vmware.mo.HypervisorHostHelper;
+import com.cloud.hypervisor.vmware.mo.TaskMO;
+import com.cloud.hypervisor.vmware.mo.VirtualEthernetCardType;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineDiskInfoBuilder;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineMO;
+import com.cloud.hypervisor.vmware.mo.VmwareHypervisorHost;
+import com.cloud.hypervisor.vmware.util.VmwareContext;
+import com.cloud.hypervisor.vmware.util.VmwareHelper;
+import com.cloud.network.Networks;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Volume;
+import com.cloud.storage.resource.VmwareStorageLayoutHelper;
+import com.cloud.storage.resource.VmwareStorageProcessor;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nicira.nvp.plugin.NiciraNvpApiVersion;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.VmDetailConstants;
+import com.vmware.vim25.BoolPolicy;
+import com.vmware.vim25.DVPortConfigInfo;
+import com.vmware.vim25.DVPortConfigSpec;
+import com.vmware.vim25.DasVmPriority;
+import com.vmware.vim25.DistributedVirtualPort;
+import com.vmware.vim25.DistributedVirtualSwitchPortConnection;
+import com.vmware.vim25.DistributedVirtualSwitchPortCriteria;
+import com.vmware.vim25.ManagedObjectReference;
+import com.vmware.vim25.OptionValue;
+import com.vmware.vim25.VMwareDVSPortSetting;
+import com.vmware.vim25.VirtualDevice;
+import com.vmware.vim25.VirtualDeviceBackingInfo;
+import com.vmware.vim25.VirtualDeviceConfigSpec;
+import com.vmware.vim25.VirtualDeviceConfigSpecOperation;
+import com.vmware.vim25.VirtualDisk;
+import com.vmware.vim25.VirtualEthernetCard;
+import com.vmware.vim25.VirtualEthernetCardDistributedVirtualPortBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardNetworkBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardOpaqueNetworkBackingInfo;
+import com.vmware.vim25.VirtualMachineBootOptions;
+import com.vmware.vim25.VirtualMachineConfigSpec;
+import com.vmware.vim25.VirtualMachineFileInfo;
+import com.vmware.vim25.VirtualMachineFileLayoutEx;
+import com.vmware.vim25.VirtualMachineGuestOsIdentifier;
+import com.vmware.vim25.VirtualUSBController;
+import com.vmware.vim25.VmConfigInfo;
+import com.vmware.vim25.VmwareDistributedVirtualSwitchVlanIdSpec;
+
+class StartCommandExecutor {
+ private static final Logger LOGGER = Logger.getLogger(StartCommandExecutor.class);
+
+ private final VmwareResource vmwareResource;
+
+ public StartCommandExecutor(VmwareResource vmwareResource) {
+ this.vmwareResource = vmwareResource;
+ }
+
+ // FR37 the monster blob god method: reduce to 320 so far and counting
+ protected StartAnswer execute(StartCommand cmd) {
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Executing resource StartCommand: " + vmwareResource.getGson().toJson(cmd));
+ }
+
+ VirtualMachineTO vmSpec = cmd.getVirtualMachine();
+
+ VirtualMachineData existingVm = null;
+
+ Pair<String, String> names = composeVmNames(vmSpec);
+ String vmInternalCSName = names.first();
+ String vmNameOnVcenter = names.second();
+
+ DiskTO rootDiskTO = null;
+
+ Pair<String, String> controllerInfo = getDiskControllerInfo(vmSpec);
+
+ Boolean systemVm = vmSpec.getType().isUsedBySystem();
+
+ VmwareContext context = vmwareResource.getServiceContext();
+ DatacenterMO dcMo = null;
+ try {
+ VmwareManager mgr = context.getStockObject(VmwareManager.CONTEXT_STOCK_NAME);
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(context);
+ dcMo = new DatacenterMO(hyperHost.getContext(), hyperHost.getHyperHostDatacenter());
+
+ // checkIfVmExistsInVcenter(vmInternalCSName, vmNameOnVcenter, dcMo);
+ // FR37 - We expect VM to be already cloned and available at this point
+ VirtualMachineMO vmMo = dcMo.findVm(vmInternalCSName);
+ if (vmMo == null) {
+ vmMo = dcMo.findVm(vmNameOnVcenter);
+ }
+
+ DiskTO[] specDisks = vmSpec.getDisks();
+ boolean installAsIs = StringUtils.isNotEmpty(vmSpec.getTemplateLocation());
+ // FR37 if startcommand contains enough info: a template url/-location and flag; deploy OVA as is
+ if (vmMo == null && installAsIs) {
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("deploying OVA from %s as is", vmSpec.getTemplateLocation()));
+ }
+ getStorageProcessor().cloneVMFromTemplate(vmSpec.getTemplateName(), vmInternalCSName, vmSpec.getTemplatePrimaryStoreUuid());
+ vmMo = dcMo.findVm(vmInternalCSName);
+ mapSpecDisksToClonedDisks(vmMo, vmInternalCSName, specDisks);
+ }
+
+ // VM may not have been on the same host, relocate to expected host
+ if (vmMo != null) {
+ vmMo.relocate(hyperHost.getMor());
+ // Get updated MO
+ vmMo = hyperHost.findVmOnHyperHost(vmMo.getVmName());
+ }
+
+ String guestOsId = translateGuestOsIdentifier(vmSpec.getArch(), vmSpec.getOs(), vmSpec.getPlatformEmulator()).value();
+ DiskTO[] disks = validateDisks(specDisks);
+ NicTO[] nics = vmSpec.getNics();
+
+ // FIXME: disks logic here, why is disks/volumes during copy not set with pool ID?
+ // FR37 TODO if deployasis a new VM no datastores are known (yet) and we need to get the data store from the tvmspec content library / template location
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails = inferDatastoreDetailsFromDiskInfo(hyperHost, context, disks, cmd);
+ if ((dataStoresDetails == null) || (dataStoresDetails.isEmpty())) {
+ String msg = "Unable to locate datastore details of the volumes to be attached";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ // FR37 - this may not be necessary the cloned VM will have disks and knowledge of datastore paths/location?
+ // throw new Exception(msg);
+ }
+
+ // FR37 - this may need checking, if the first datastore is the right one - ideally it should be datastore where first disk is hosted
+ DatastoreMO dsRootVolumeIsOn = null; //
+ if (! installAsIs) {
+ dsRootVolumeIsOn = getDatastoreThatRootDiskIsOn(dataStoresDetails, disks);
+
+ if (dsRootVolumeIsOn == null) {
+ String msg = "Unable to locate datastore details of root volume";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ throw new VmwareResourceException(msg);
+ }
+ }
+
+ VirtualMachineDiskInfoBuilder diskInfoBuilder = null;
+ VirtualDevice[] nicDevices = null;
+ DiskControllerType systemVmScsiControllerType = DiskControllerType.lsilogic;
+ int firstScsiControllerBusNum = 0;
+ int numScsiControllerForSystemVm = 1;
+ boolean hasSnapshot = false;
+ if (vmMo != null) {
+ PrepareRunningVMForConfiguration prepareRunningVMForConfiguration = new PrepareRunningVMForConfiguration(vmInternalCSName, controllerInfo, systemVm, vmMo,
+ systemVmScsiControllerType, firstScsiControllerBusNum, numScsiControllerForSystemVm);
+ prepareRunningVMForConfiguration.invoke();
+ diskInfoBuilder = prepareRunningVMForConfiguration.getDiskInfoBuilder();
+ nicDevices = prepareRunningVMForConfiguration.getNicDevices();
+ hasSnapshot = prepareRunningVMForConfiguration.isHasSnapshot();
+ } else {
+ ManagedObjectReference morDc = hyperHost.getHyperHostDatacenter();
+ assert (morDc != null);
+
+ vmMo = hyperHost.findVmOnPeerHyperHost(vmInternalCSName);
+ if (vmMo != null) {
+ VirtualMachineRecycler virtualMachineRecycler = new VirtualMachineRecycler(vmInternalCSName, controllerInfo, systemVm, hyperHost, vmMo,
+ systemVmScsiControllerType, firstScsiControllerBusNum, numScsiControllerForSystemVm);
+ virtualMachineRecycler.invoke();
+ diskInfoBuilder = virtualMachineRecycler.getDiskInfoBuilder();
+ hasSnapshot = virtualMachineRecycler.isHasSnapshot();
+ nicDevices = virtualMachineRecycler.getNicDevices();
+ } else {
+ // FR37 we just didn't find a VM by name 'vmInternalCSName', so why is this here?
+ existingVm = unregisterOnOtherClusterButHoldOnToOldVmData(vmInternalCSName, dcMo);
+
+ createNewVm(context, vmSpec, installAsIs, vmInternalCSName, vmNameOnVcenter, controllerInfo, systemVm, mgr, hyperHost, guestOsId, disks, dataStoresDetails,
+ dsRootVolumeIsOn);
+ }
+
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ if (vmMo == null) {
+ throw new Exception("Failed to find the newly create or relocated VM. vmName: " + vmInternalCSName);
+ }
+ }
+ // vmMo should now be a stopped VM on the intended host
+ // The number of disks changed must be 0 for install as is, as the VM is a clone
+ int disksChanges = !installAsIs ? disks.length : 0;
+ int totalChangeDevices = disksChanges + nics.length;
+ int hackDeviceCount = 0;
+ if (diskInfoBuilder != null) {
+ hackDeviceCount += diskInfoBuilder.getDiskCount();
+ }
+ hackDeviceCount += nicDevices == null ? 0 : nicDevices.length;
+ // vApp cdrom device
+ // HACK ALERT: ovf properties might not be the only or defining feature of vApps; needs checking
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("current count(s) desired: %d/ found:%d. now adding device to device count for vApp config ISO", totalChangeDevices, hackDeviceCount));
+ }
+ if (vmSpec.getOvfProperties() != null) {
+ totalChangeDevices++;
+ }
+
+ DiskTO volIso = null;
+ if (vmSpec.getType() != VirtualMachine.Type.User) {
+ // system VM needs a patch ISO
+ totalChangeDevices++;
+ } else {
+ volIso = getIsoDiskTO(disks);
+ if (volIso == null)
+ totalChangeDevices++;
+ }
+
+ VirtualMachineConfigSpec vmConfigSpec = new VirtualMachineConfigSpec();
+
+ VmwareHelper.setBasicVmConfig(vmConfigSpec, vmSpec.getCpus(), vmSpec.getMaxSpeed(), vmwareResource.getReservedCpuMHZ(vmSpec), (int)(vmSpec.getMaxRam() / (1024 * 1024)),
+ vmwareResource.getReservedMemoryMb(vmSpec), guestOsId, vmSpec.getLimitCpuUse());
+
+ int numCoresPerSocket = adjustNumberOfCoresPerSocket(vmSpec, vmMo, vmConfigSpec);
+
+ // Check for hotadd settings
+ vmConfigSpec.setMemoryHotAddEnabled(vmMo.isMemoryHotAddSupported(guestOsId));
+
+ String hostApiVersion = ((HostMO)hyperHost).getHostAboutInfo().getApiVersion();
+ if (numCoresPerSocket > 1 && hostApiVersion.compareTo("5.0") < 0) {
+ LOGGER.warn("Dynamic scaling of CPU is not supported for Virtual Machines with multi-core vCPUs in case of ESXi hosts 4.1 and prior. Hence CpuHotAdd will not be"
+ + " enabled for Virtual Machine: " + vmInternalCSName);
+ vmConfigSpec.setCpuHotAddEnabled(false);
+ } else {
+ vmConfigSpec.setCpuHotAddEnabled(vmMo.isCpuHotAddSupported(guestOsId));
+ }
+
+ vmwareResource.configNestedHVSupport(vmMo, vmSpec, vmConfigSpec);
+
+ VirtualDeviceConfigSpec[] deviceConfigSpecArray = new VirtualDeviceConfigSpec[totalChangeDevices];
+ int deviceCount = 0;
+ int ideUnitNumber = 0;
+ int scsiUnitNumber = 0;
+ int ideControllerKey = vmMo.getIDEDeviceControllerKey();
+ int scsiControllerKey = vmMo.getScsiDeviceControllerKeyNoException();
+
+ IsoSetup isoSetup = new IsoSetup(vmSpec, mgr, hyperHost, vmMo, disks, volIso, deviceConfigSpecArray, deviceCount, ideUnitNumber);
+ isoSetup.invoke();
+ deviceCount = isoSetup.getDeviceCount();
+ ideUnitNumber = isoSetup.getIdeUnitNumber();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("setting up %d disks with root from %s", diskInfoBuilder.getDiskCount(), vmwareResource.getGson().toJson(rootDiskTO)));
+ }
+
+ DiskSetup diskSetup = new DiskSetup(vmSpec, rootDiskTO, controllerInfo, context, dcMo, hyperHost, vmMo, disks, dataStoresDetails, diskInfoBuilder, hasSnapshot,
+ deviceConfigSpecArray, deviceCount, ideUnitNumber, scsiUnitNumber, ideControllerKey, scsiControllerKey, installAsIs);
+ diskSetup.invoke();
+
+ rootDiskTO = diskSetup.getRootDiskTO();
+ deviceCount = diskSetup.getDeviceCount();
+ DiskTO[] sortedDisks = diskSetup.getSortedDisks();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("seting up %d disks with root from %s", sortedDisks.length, vmwareResource.getGson().toJson(rootDiskTO)));
+ }
+
+ deviceCount += setupUsbDevicesAndGetCount(vmInternalCSName, vmMo, guestOsId, deviceConfigSpecArray, deviceCount);
+
+ NicSetup nicSetup = new NicSetup(cmd, vmSpec, vmInternalCSName, context, mgr, hyperHost, vmMo, nics, deviceConfigSpecArray, deviceCount, nicDevices);
+ nicSetup.invoke();
+
+ deviceCount = nicSetup.getDeviceCount();
+ int nicMask = nicSetup.getNicMask();
+ int nicCount = nicSetup.getNicCount();
+ Map<String, String> nicUuidToDvSwitchUuid = nicSetup.getNicUuidToDvSwitchUuid();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("device count: %d; nic count %d", deviceCount, nicCount));
+ }
+ for (int j = 0; j < deviceCount; j++)
+ vmConfigSpec.getDeviceChange().add(deviceConfigSpecArray[j]);
+
+ //
+ // Setup VM options
+ //
+
+ // pass boot arguments through machine.id & perform customized options to VMX
+ ArrayList<OptionValue> extraOptions = new ArrayList<>();
+ configBasicExtraOption(extraOptions, vmSpec);
+ configNvpExtraOption(extraOptions, vmSpec, nicUuidToDvSwitchUuid);
+ configCustomExtraOption(extraOptions, vmSpec);
+
+ // config for NCC
+ VirtualMachine.Type vmType = cmd.getVirtualMachine().getType();
+ if (vmType.equals(VirtualMachine.Type.NetScalerVm)) {
+ NicTO mgmtNic = vmSpec.getNics()[0];
+ OptionValue option = new OptionValue();
+ option.setKey("machine.id");
+ option.setValue("ip=" + mgmtNic.getIp() + "&netmask=" + mgmtNic.getNetmask() + "&gateway=" + mgmtNic.getGateway());
+ extraOptions.add(option);
+ }
+
+ // config VNC
+ String keyboardLayout = null;
+ if (vmSpec.getDetails() != null)
+ keyboardLayout = vmSpec.getDetails().get(VmDetailConstants.KEYBOARD);
+ vmConfigSpec.getExtraConfig().addAll(
+ Arrays.asList(vmwareResource.configureVnc(extraOptions.toArray(new OptionValue[0]), hyperHost, vmInternalCSName, vmSpec.getVncPassword(), keyboardLayout)));
+
+ // config video card
+ vmwareResource.configureVideoCard(vmMo, vmSpec, vmConfigSpec);
+
+ // Set OVF properties (if available)
+ Pair<String, List<OVFPropertyTO>> ovfPropsMap = vmSpec.getOvfProperties();
+ VmConfigInfo templateVappConfig;
+ List<OVFPropertyTO> ovfProperties;
+ if (ovfPropsMap != null) {
+ String vmTemplate = ovfPropsMap.first();
+ LOGGER.info("Find VM template " + vmTemplate);
+ VirtualMachineMO vmTemplateMO = dcMo.findVm(vmTemplate);
+ templateVappConfig = vmTemplateMO.getConfigInfo().getVAppConfig();
+ ovfProperties = ovfPropsMap.second();
+ // Set OVF properties (if available)
+ if (CollectionUtils.isNotEmpty(ovfProperties)) {
+ LOGGER.info("Copying OVF properties from template and setting them to the values the user provided");
+ vmwareResource.copyVAppConfigsFromTemplate(templateVappConfig, ovfProperties, vmConfigSpec);
+ }
+ }
+
+ checkBootOptions(vmSpec, vmConfigSpec);
+
+ //
+ // Configure VM
+ //
+ if (!vmMo.configureVm(vmConfigSpec)) {
+ throw new Exception("Failed to configure VM before start. vmName: " + vmInternalCSName);
+ }
+ // FR37 TODO reconcile disks now!!! they are configured, check and add them to the returning vmTO
+ if (vmSpec.getType() == VirtualMachine.Type.DomainRouter) {
+ hyperHost.setRestartPriorityForVM(vmMo, DasVmPriority.HIGH.value());
+ }
+
+ // Resizing root disk only when explicit requested by user
+ final Map<String, String> vmDetails = cmd.getVirtualMachine().getDetails();
+ if (rootDiskTO != null && !hasSnapshot && (vmDetails != null && vmDetails.containsKey(ApiConstants.ROOT_DISK_SIZE))) {
+ resizeRootDiskOnVMStart(vmMo, rootDiskTO, hyperHost, context);
+ }
+
+ //
+ // Post Configuration
+ //
+
+ vmMo.setCustomFieldValue(CustomFieldConstants.CLOUD_NIC_MASK, String.valueOf(nicMask));
+ postNvpConfigBeforeStart(vmMo, vmSpec);
+
+ Map<String, Map<String, String>> iqnToData = new HashMap<>();
+
+ postDiskConfigBeforeStart(vmMo, vmSpec, sortedDisks, iqnToData, hyperHost, context, installAsIs);
+
+ //
+ // Power-on VM
+ //
+ if (!vmMo.powerOn()) {
+ throw new Exception("Failed to start VM. vmName: " + vmInternalCSName + " with hostname " + vmNameOnVcenter);
+ }
+
+ if (installAsIs) {
+ // Set disks as the disks path and chain is retrieved from the cloned VM disks
+ cmd.getVirtualMachine().setDisks(disks);
+ }
+
+ StartAnswer startAnswer = new StartAnswer(cmd);
+
+ startAnswer.setIqnToData(iqnToData);
+
+ deleteOldVersionOfTheStartedVM(existingVm, dcMo, vmMo);
+
+ return startAnswer;
+ } catch (Throwable e) {
+ return handleStartFailure(cmd, vmSpec, existingVm, dcMo, e);
+ } finally {
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("finally done with %s", vmwareResource.getGson().toJson(cmd)));
+ }
+ }
+ }
+
+ /**
+ * Modify the specDisks information to match the cloned VM's disks (from vmMo VM)
+ */
+ private void mapSpecDisksToClonedDisks(VirtualMachineMO vmMo, String vmInternalCSName, DiskTO[] specDisks) {
+ try {
+ if (vmMo != null && ArrayUtils.isNotEmpty(specDisks)) {
+ List<VirtualDisk> vmDisks = vmMo.getVirtualDisks();
+ List<DiskTO> sortedDisks = Arrays.asList(sortVolumesByDeviceId(specDisks))
+ .stream()
+ .filter(x -> x.getType() == Volume.Type.ROOT)
+ .collect(Collectors.toList());
+ if (sortedDisks.size() != vmDisks.size()) {
+ LOGGER.error("Different number of root disks spec vs cloned deploy-as-is VM disks: " + sortedDisks.size() + " - " + vmDisks.size());
+ return;
+ }
+ for (int i = 0; i < sortedDisks.size(); i++) {
+ DiskTO specDisk = sortedDisks.get(i);
+ VirtualDisk vmDisk = vmDisks.get(i);
+ DataTO dataVolume = specDisk.getData();
+ if (dataVolume instanceof VolumeObjectTO) {
+ VolumeObjectTO volumeObjectTO = (VolumeObjectTO) dataVolume;
+ if (!volumeObjectTO.getSize().equals(vmDisk.getCapacityInBytes())) {
+ LOGGER.info("Mapped disk size is not the same as the cloned VM disk size: " +
+ volumeObjectTO.getSize() + " - " + vmDisk.getCapacityInBytes());
+ }
+ VirtualDeviceBackingInfo backingInfo = vmDisk.getBacking();
+ if (backingInfo instanceof VirtualDiskFlatVer2BackingInfo) {
+ VirtualDiskFlatVer2BackingInfo backing = (VirtualDiskFlatVer2BackingInfo) backingInfo;
+ String fileName = backing.getFileName();
+ if (StringUtils.isNotBlank(fileName)) {
+ String[] fileNameParts = fileName.split(" ");
+ String datastoreUuid = fileNameParts[0].replace("[", "").replace("]", "");
+ String relativePath = fileNameParts[1].split("/")[1].replace(".vmdk", "");
+ String vmSpecDatastoreUuid = volumeObjectTO.getDataStore().getUuid().replaceAll("-", "");
+ if (!datastoreUuid.equals(vmSpecDatastoreUuid)) {
+ LOGGER.info("Mapped disk datastore UUID is not the same as the cloned VM datastore UUID: " +
+ datastoreUuid + " - " + vmSpecDatastoreUuid);
+ }
+ volumeObjectTO.setPath(relativePath);
+ specDisk.setPath(relativePath);
+ }
+ }
+ }
+ }
+ }
+ } catch (Exception e) {
+ String msg = "Error mapping deploy-as-is VM disks from cloned VM " + vmInternalCSName;
+ LOGGER.error(msg, e);
+ throw new CloudRuntimeException(e);
+ }
+ }
+
+ private StartAnswer handleStartFailure(StartCommand cmd, VirtualMachineTO vmSpec, VirtualMachineData existingVm, DatacenterMO dcMo, Throwable e) {
+ if (e instanceof RemoteException) {
+ LOGGER.warn("Encounter remote exception to vCenter, invalidate VMware session context");
+ vmwareResource.invalidateServiceContext();
+ }
+
+ String msg = "StartCommand failed due to " + VmwareHelper.getExceptionMessage(e);
+ LOGGER.warn(msg, e);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("didn't start VM %s", vmSpec.getName()), e);
+ }
+ StartAnswer startAnswer = new StartAnswer(cmd, msg);
+ if ( e instanceof VmAlreadyExistsInVcenter) {
+ startAnswer.setContextParam("stopRetry", "true");
+ }
+ reRegisterExistingVm(existingVm, dcMo);
+
+ return startAnswer;
+ }
+
+ private void deleteOldVersionOfTheStartedVM(VirtualMachineData existingVm, DatacenterMO dcMo, VirtualMachineMO vmMo) throws Exception {
+ // Since VM was successfully powered-on, if there was an existing VM in a different cluster that was unregistered, delete all the files associated with it.
+ if (existingVm != null && existingVm.vmName != null && existingVm.vmFileLayout != null) {
+ List<String> vmDatastoreNames = new ArrayList<>();
+ for (DatastoreMO vmDatastore : vmMo.getAllDatastores()) {
+ vmDatastoreNames.add(vmDatastore.getName());
+ }
+ // Don't delete files that are in a datastore that is being used by the new VM as well (zone-wide datastore).
+ List<String> skipDatastores = new ArrayList<>();
+ for (DatastoreMO existingDatastore : existingVm.datastores) {
+ if (vmDatastoreNames.contains(existingDatastore.getName())) {
+ skipDatastores.add(existingDatastore.getName());
+ }
+ }
+ vmwareResource.deleteUnregisteredVmFiles(existingVm.vmFileLayout, dcMo, true, skipDatastores);
+ }
+ }
+
+ private int adjustNumberOfCoresPerSocket(VirtualMachineTO vmSpec, VirtualMachineMO vmMo, VirtualMachineConfigSpec vmConfigSpec) throws Exception {
+ // Check for multi-cores per socket settings
+ int numCoresPerSocket = 1;
+ String coresPerSocket = vmSpec.getDetails().get(VmDetailConstants.CPU_CORE_PER_SOCKET);
+ if (coresPerSocket != null) {
+ String apiVersion = HypervisorHostHelper.getVcenterApiVersion(vmMo.getContext());
+ // Property 'numCoresPerSocket' is supported since vSphere API 5.0
+ if (apiVersion.compareTo("5.0") >= 0) {
+ numCoresPerSocket = NumbersUtil.parseInt(coresPerSocket, 1);
+ vmConfigSpec.setNumCoresPerSocket(numCoresPerSocket);
+ }
+ }
+ return numCoresPerSocket;
+ }
+
+ /**
+ // Setup USB devices
+ */
+ private int setupUsbDevicesAndGetCount(String vmInternalCSName, VirtualMachineMO vmMo, String guestOsId, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount)
+ throws Exception {
+ int usbDeviceCount = 0;
+ if (guestOsId.startsWith("darwin")) { //Mac OS
+ VirtualDevice[] devices = vmMo.getMatchedDevices(new Class<?>[] {VirtualUSBController.class});
+ if (devices.length == 0) {
+ LOGGER.debug("No USB Controller device on VM Start. Add USB Controller device for Mac OS VM " + vmInternalCSName);
+
+ //For Mac OS X systems, the EHCI+UHCI controller is enabled by default and is required for USB mouse and keyboard access.
+ VirtualDevice usbControllerDevice = VmwareHelper.prepareUSBControllerDevice();
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ deviceConfigSpecArray[deviceCount].setDevice(usbControllerDevice);
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare USB controller at new device " + vmwareResource.getGson().toJson(deviceConfigSpecArray[deviceCount]));
+
+ deviceCount++;
+ usbDeviceCount++;
+ } else {
+ LOGGER.debug("USB Controller device exists on VM Start for Mac OS VM " + vmInternalCSName);
+ }
+ }
+ return usbDeviceCount;
+ }
+
+ private void createNewVm(VmwareContext context, VirtualMachineTO vmSpec, boolean installAsIs, String vmInternalCSName, String vmNameOnVcenter, Pair<String, String> controllerInfo, Boolean systemVm,
+ VmwareManager mgr, VmwareHypervisorHost hyperHost, String guestOsId, DiskTO[] disks, HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails,
+ DatastoreMO dsRootVolumeIsOn) throws Exception {
+ VirtualMachineMO vmMo;
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = getRootDiskDataStoreDetails(disks, dataStoresDetails);
+
+ assert (vmSpec.getMinSpeed() != null) && (rootDiskDataStoreDetails != null);
+
+ boolean vmFolderExists = rootDiskDataStoreDetails.second().folderExists(String.format("[%s]", rootDiskDataStoreDetails.second().getName()), vmNameOnVcenter);
+ String vmxFileFullPath = dsRootVolumeIsOn.searchFileInSubFolders(vmNameOnVcenter + ".vmx", false, VmwareManager.s_vmwareSearchExcludeFolder.value());
+ if (vmFolderExists && vmxFileFullPath != null) { // VM can be registered only if .vmx is present.
+ registerVm(vmNameOnVcenter, dsRootVolumeIsOn, vmwareResource);
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ if (vmMo != null) {
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Found registered vm " + vmInternalCSName + " at host " + hyperHost.getHyperHostName());
+ }
+ }
+ tearDownVm(vmMo);
+ } else if (installAsIs) {
+// first get all the MORs ManagedObjectReference morPool = hyperHost.getHyperHostOwnerResourcePool();
+// get the base VM vmMo = hyperHost.findVmOnHyperHost(vm.template.getPath());
+// do templateVm.createLinkedClone(vmInternalCSName, morBaseSnapshot, dcMo.getVmFolder(), morPool, morDatastore)
+// or hyperHost....createLinkedOrFullClone(templateVm, volume, dcMo, vmMo, morDatastore, dsMo, vmInternalCSName, morPool);
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ // At this point vmMo points to the cloned VM
+ } else {
+ if (!hyperHost
+ .createBlankVm(vmNameOnVcenter, vmInternalCSName, vmSpec.getCpus(), vmSpec.getMaxSpeed(), vmwareResource.getReservedCpuMHZ(vmSpec), vmSpec.getLimitCpuUse(), (int)(vmSpec.getMaxRam() / Resource.ResourceType.bytesToMiB), vmwareResource.getReservedMemoryMb(vmSpec), guestOsId,
+ rootDiskDataStoreDetails.first(), false, controllerInfo, systemVm)) {
+ throw new Exception("Failed to create VM. vmName: " + vmInternalCSName);
+ }
+ }
+ }
+
+ /**
+ * If a VM with the same name is found in a different cluster in the DC, unregister the old VM and configure a new VM (cold-migration).
+ */
+ private VirtualMachineData unregisterOnOtherClusterButHoldOnToOldVmData(String vmInternalCSName, DatacenterMO dcMo) throws Exception {
+ VirtualMachineMO existingVmInDc = dcMo.findVm(vmInternalCSName);
+ VirtualMachineData existingVm = null;
+ if (existingVmInDc != null) {
+ existingVm = new VirtualMachineData();
+ existingVm.vmName = existingVmInDc.getName();
+ existingVm.vmFileInfo = existingVmInDc.getFileInfo();
+ existingVm.vmFileLayout = existingVmInDc.getFileLayout();
+ existingVm.datastores = existingVmInDc.getAllDatastores();
+ LOGGER.info("Found VM: " + vmInternalCSName + " on a host in a different cluster. Unregistering the exisitng VM.");
+ existingVmInDc.unregisterVm();
+ }
+ return existingVm;
+ }
+
+ private Pair<ManagedObjectReference, DatastoreMO> getRootDiskDataStoreDetails(DiskTO[] disks, HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails) {
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = null;
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ROOT) {
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (managed) {
+ String datastoreName = VmwareResource.getDatastoreName(details.get(DiskTO.IQN));
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(datastoreName);
+ } else {
+ DataStoreTO primaryStore = vol.getData().getDataStore();
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(primaryStore.getUuid());
+ }
+ }
+ }
+ return rootDiskDataStoreDetails;
+ }
+
+ /**
+ * Since VM start failed, if there was an existing VM in a different cluster that was unregistered, register it back.
+ *
+ * @param dcMo is guaranteed to be not null since we have noticed there is an existing VM in the dc (using that mo)
+ */
+ private void reRegisterExistingVm(VirtualMachineData existingVm, DatacenterMO dcMo) {
+ if (existingVm != null && existingVm.vmName != null && existingVm.vmFileInfo != null) {
+ LOGGER.debug("Since VM start failed, registering back an existing VM: " + existingVm.vmName + " that was unregistered");
+ try {
+ DatastoreFile fileInDatastore = new DatastoreFile(existingVm.vmFileInfo.getVmPathName());
+ DatastoreMO existingVmDsMo = new DatastoreMO(dcMo.getContext(), dcMo.findDatastore(fileInDatastore.getDatastoreName()));
+ registerVm(existingVm.vmName, existingVmDsMo, vmwareResource);
+ } catch (Exception ex) {
+ String message = "Failed to register an existing VM: " + existingVm.vmName + " due to " + VmwareHelper.getExceptionMessage(ex);
+ LOGGER.warn(message, ex);
+ }
+ }
+ }
+
+ private void checkIfVmExistsInVcenter(String vmInternalCSName, String vmNameOnVcenter, DatacenterMO dcMo) throws VmAlreadyExistsInVcenter, Exception {
+ // Validate VM name is unique in Datacenter
+ VirtualMachineMO vmInVcenter = dcMo.checkIfVmAlreadyExistsInVcenter(vmNameOnVcenter, vmInternalCSName);
+ if (vmInVcenter != null) {
+ String msg = "VM with name: " + vmNameOnVcenter + " already exists in vCenter.";
+ LOGGER.error(msg);
+ throw new VmAlreadyExistsInVcenter(msg);
+ }
+ }
+
+ private Pair<String, String> getDiskControllerInfo(VirtualMachineTO vmSpec) {
+ String dataDiskController = vmSpec.getDetails().get(VmDetailConstants.DATA_DISK_CONTROLLER);
+ String rootDiskController = vmSpec.getDetails().get(VmDetailConstants.ROOT_DISK_CONTROLLER);
+ // If root disk controller is scsi, then data disk controller would also be scsi instead of using 'osdefault'
+ // This helps avoid mix of different scsi subtype controllers in instance.
+ if (DiskControllerType.osdefault == DiskControllerType.getType(dataDiskController) && DiskControllerType.lsilogic == DiskControllerType.getType(rootDiskController)) {
+ dataDiskController = DiskControllerType.scsi.toString();
+ }
+
+ // Validate the controller types
+ dataDiskController = DiskControllerType.getType(dataDiskController).toString();
+ rootDiskController = DiskControllerType.getType(rootDiskController).toString();
+
+ if (DiskControllerType.getType(rootDiskController) == DiskControllerType.none) {
+ throw new CloudRuntimeException("Invalid root disk controller detected : " + rootDiskController);
+ }
+ if (DiskControllerType.getType(dataDiskController) == DiskControllerType.none) {
+ throw new CloudRuntimeException("Invalid data disk controller detected : " + dataDiskController);
+ }
+
+ return new Pair<>(rootDiskController, dataDiskController);
+ }
+
+ /**
+ * Registers the vm to the inventory given the vmx file.
+ * @param vmName
+ * @param dsMo
+ * @param vmwareResource
+ */
+ private void registerVm(String vmName, DatastoreMO dsMo, VmwareResource vmwareResource) throws Exception {
+
+ //1st param
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(vmwareResource.getServiceContext());
+ ManagedObjectReference dcMor = hyperHost.getHyperHostDatacenter();
+ DatacenterMO dataCenterMo = new DatacenterMO(vmwareResource.getServiceContext(), dcMor);
+ ManagedObjectReference vmFolderMor = dataCenterMo.getVmFolder();
+
+ //2nd param
+ String vmxFilePath = dsMo.searchFileInSubFolders(vmName + ".vmx", false, VmwareManager.s_vmwareSearchExcludeFolder.value());
+
+ // 5th param
+ ManagedObjectReference morPool = hyperHost.getHyperHostOwnerResourcePool();
+
+ ManagedObjectReference morTask = vmwareResource.getServiceContext().getService().registerVMTask(vmFolderMor, vmxFilePath, vmName, false, morPool, hyperHost.getMor());
+ boolean result = vmwareResource.getServiceContext().getVimClient().waitForTask(morTask);
+ if (!result) {
+ throw new Exception("Unable to register vm due to " + TaskMO.getTaskFailureInfo(vmwareResource.getServiceContext(), morTask));
+ } else {
+ vmwareResource.getServiceContext().waitForTaskProgressDone(morTask);
+ }
+
+ }
+
+ // Pair<internal CS name, vCenter display name>
+ private Pair<String, String> composeVmNames(VirtualMachineTO vmSpec) {
+ String vmInternalCSName = vmSpec.getName();
+ String vmNameOnVcenter = vmSpec.getName();
+ if (VmwareResource.instanceNameFlag && vmSpec.getHostName() != null) {
+ vmNameOnVcenter = vmSpec.getHostName();
+ }
+ return new Pair<String, String>(vmInternalCSName, vmNameOnVcenter);
+ }
+
+ private VirtualMachineGuestOsIdentifier translateGuestOsIdentifier(String cpuArchitecture, String guestOs, String cloudGuestOs) {
+ if (cpuArchitecture == null) {
+ LOGGER.warn("CPU arch is not set, default to i386. guest os: " + guestOs);
+ cpuArchitecture = "i386";
+ }
+
+ if (cloudGuestOs == null) {
+ LOGGER.warn("Guest OS mapping name is not set for guest os: " + guestOs);
+ }
+
+ VirtualMachineGuestOsIdentifier identifier = null;
+ try {
+ if (cloudGuestOs != null) {
+ identifier = VirtualMachineGuestOsIdentifier.fromValue(cloudGuestOs);
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Using mapping name : " + identifier.toString());
+ }
+ }
+ } catch (IllegalArgumentException e) {
+ LOGGER.warn("Unable to find Guest OS Identifier in VMware for mapping name: " + cloudGuestOs + ". Continuing with defaults.");
+ }
+ if (identifier != null) {
+ return identifier;
+ }
+
+ if (cpuArchitecture.equalsIgnoreCase("x86_64")) {
+ return VirtualMachineGuestOsIdentifier.OTHER_GUEST_64;
+ }
+ return VirtualMachineGuestOsIdentifier.OTHER_GUEST;
+ }
+
+ private DiskTO[] validateDisks(DiskTO[] disks) {
+ List<DiskTO> validatedDisks = new ArrayList<DiskTO>();
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() != Volume.Type.ISO) {
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ if (primaryStore.getUuid() != null && !primaryStore.getUuid().isEmpty()) {
+ validatedDisks.add(vol);
+ }
+ } else if (vol.getType() == Volume.Type.ISO) {
+ TemplateObjectTO templateTO = (TemplateObjectTO) vol.getData();
+ if (templateTO.getPath() != null && !templateTO.getPath().isEmpty()) {
+ validatedDisks.add(vol);
+ }
+ } else {
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Drop invalid disk option, volumeTO: " + vmwareResource.getGson().toJson(vol));
+ }
+ }
+ }
+ Collections.sort(validatedDisks, (d1, d2) -> d1.getDiskSeq().compareTo(d2.getDiskSeq()));
+ return validatedDisks.toArray(new DiskTO[0]);
+ }
+
+ private HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> inferDatastoreDetailsFromDiskInfo(VmwareHypervisorHost hyperHost, VmwareContext context,
+ DiskTO[] disks, Command cmd) throws Exception {
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> mapIdToMors = new HashMap<>();
+
+ assert (hyperHost != null) && (context != null);
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() != Volume.Type.ISO) {
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ String poolUuid = primaryStore.getUuid();
+
+ if (mapIdToMors.get(poolUuid) == null) {
+ boolean isManaged = false;
+ Map<String, String> details = vol.getDetails();
+
+ if (details != null) {
+ isManaged = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (isManaged) {
+ String iScsiName = details.get(DiskTO.IQN); // details should not be null for managed storage (it may or may not be null for non-managed storage)
+ String datastoreName = VmwareResource.getDatastoreName(iScsiName);
+ ManagedObjectReference morDatastore = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, datastoreName);
+
+ // if the datastore is not present, we need to discover the iSCSI device that will support it,
+ // create the datastore, and create a VMDK file in the datastore
+ if (morDatastore == null) {
+ final String vmdkPath = vmwareResource.getVmdkPath(volumeTO.getPath());
+
+ morDatastore = getStorageProcessor().prepareManagedStorage(context, hyperHost, null, iScsiName,
+ details.get(DiskTO.STORAGE_HOST), Integer.parseInt(details.get(DiskTO.STORAGE_PORT)),
+ vmdkPath,
+ details.get(DiskTO.CHAP_INITIATOR_USERNAME), details.get(DiskTO.CHAP_INITIATOR_SECRET),
+ details.get(DiskTO.CHAP_TARGET_USERNAME), details.get(DiskTO.CHAP_TARGET_SECRET),
+ Long.parseLong(details.get(DiskTO.VOLUME_SIZE)), cmd);
+
+ DatastoreMO dsMo = new DatastoreMO(vmwareResource.getServiceContext(), morDatastore);
+
+ final String datastoreVolumePath;
+
+ if (vmdkPath != null) {
+ datastoreVolumePath = dsMo.getDatastorePath(vmdkPath + VmwareResource.VMDK_EXTENSION);
+ } else {
+ datastoreVolumePath = dsMo.getDatastorePath(dsMo.getName() + VmwareResource.VMDK_EXTENSION);
+ }
+
+ volumeTO.setPath(datastoreVolumePath);
+ vol.setPath(datastoreVolumePath);
+ }
+
+ mapIdToMors.put(datastoreName, new Pair<>(morDatastore, new DatastoreMO(context, morDatastore)));
+ } else {
+ ManagedObjectReference morDatastore = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, poolUuid);
+
+ if (morDatastore == null) {
+ String msg = "Failed to get the mounted datastore for the volume's pool " + poolUuid;
+
+ LOGGER.error(msg);
+
+ throw new Exception(msg);
+ }
+
+ mapIdToMors.put(poolUuid, new Pair<>(morDatastore, new DatastoreMO(context, morDatastore)));
+ }
+ }
+ }
+ }
+
+ return mapIdToMors;
+ }
+
+ private VmwareStorageProcessor getStorageProcessor() {
+ return vmwareResource.getStorageProcessor();
+ }
+
+ private DatastoreMO getDatastoreThatRootDiskIsOn(HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails, DiskTO disks[]) {
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = null;
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ROOT) {
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (managed) {
+ String datastoreName = VmwareResource.getDatastoreName(details.get(DiskTO.IQN));
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(datastoreName);
+
+ break;
+ } else {
+ DataStoreTO primaryStore = vol.getData().getDataStore();
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(primaryStore.getUuid());
+
+ break;
+ }
+ }
+ }
+
+ if (rootDiskDataStoreDetails != null) {
+ return rootDiskDataStoreDetails.second();
+ }
+
+ return null;
+ }
+
+ private void ensureDiskControllers(VirtualMachineMO vmMo, Pair<String, String> controllerInfo) throws Exception {
+ if (vmMo == null) {
+ return;
+ }
+
+ String msg;
+ String rootDiskController = controllerInfo.first();
+ String dataDiskController = controllerInfo.second();
+ String scsiDiskController;
+ String recommendedDiskController = null;
+
+ if (VmwareHelper.isControllerOsRecommended(dataDiskController) || VmwareHelper.isControllerOsRecommended(rootDiskController)) {
+ recommendedDiskController = vmMo.getRecommendedDiskController(null);
+ }
+ scsiDiskController = HypervisorHostHelper.getScsiController(new Pair<String, String>(rootDiskController, dataDiskController), recommendedDiskController);
+ if (scsiDiskController == null) {
+ return;
+ }
+
+ vmMo.getScsiDeviceControllerKeyNoException();
+ // This VM needs SCSI controllers.
+ // Get count of existing scsi controllers. Helps not to attempt to create more than the maximum allowed 4
+ // Get maximum among the bus numbers in use by scsi controllers. Safe to pick maximum, because we always go sequential allocating bus numbers.
+ Ternary<Integer, Integer, DiskControllerType> scsiControllerInfo = vmMo.getScsiControllerInfo();
+ int requiredNumScsiControllers = VmwareHelper.MAX_SCSI_CONTROLLER_COUNT - scsiControllerInfo.first();
+ int availableBusNum = scsiControllerInfo.second() + 1; // method returned current max. bus number
+
+ if (requiredNumScsiControllers == 0) {
+ return;
+ }
+ if (scsiControllerInfo.first() > 0) {
+ // For VMs which already have a SCSI controller, do NOT attempt to add any more SCSI controllers & return the sub type.
+ // For Legacy VMs would have only 1 LsiLogic Parallel SCSI controller, and doesn't require more.
+ // For VMs created post device ordering support, 4 SCSI subtype controllers are ensured during deployment itself. No need to add more.
+ // For fresh VM deployment only, all required controllers should be ensured.
+ return;
+ }
+ ensureScsiDiskControllers(vmMo, scsiDiskController, requiredNumScsiControllers, availableBusNum);
+ }
+
+ private void ensureScsiDiskControllers(VirtualMachineMO vmMo, String scsiDiskController, int requiredNumScsiControllers, int availableBusNum) throws Exception {
+ // Pick the sub type of scsi
+ if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.pvscsi) {
+ if (!vmMo.isPvScsiSupported()) {
+ String msg = "This VM doesn't support Vmware Paravirtual SCSI controller for virtual disks, because the virtual hardware version is less than 7.";
+ throw new Exception(msg);
+ }
+ vmMo.ensurePvScsiDeviceController(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.lsisas1068) {
+ vmMo.ensureLsiLogicSasDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.buslogic) {
+ vmMo.ensureBusLogicDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.lsilogic) {
+ vmMo.ensureLsiLogicDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ }
+ }
+
+ private void tearDownVm(VirtualMachineMO vmMo) throws Exception {
+
+ if (vmMo == null)
+ return;
+
+ boolean hasSnapshot = false;
+ hasSnapshot = vmMo.hasSnapshot();
+ if (!hasSnapshot)
+ vmMo.tearDownDevices(new Class<?>[]{VirtualDisk.class, VirtualEthernetCard.class});
+ else
+ vmMo.tearDownDevices(new Class<?>[]{VirtualEthernetCard.class});
+ vmMo.ensureScsiDeviceController();
+ }
+
+ private static DiskTO getIsoDiskTO(DiskTO[] disks) {
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ISO) {
+ return vol;
+ }
+ }
+ return null;
+ }
+
+ private static DiskTO[] sortVolumesByDeviceId(DiskTO[] volumes) {
+
+ List<DiskTO> listForSort = new ArrayList<DiskTO>();
+ for (DiskTO vol : volumes) {
+ listForSort.add(vol);
+ }
+ Collections.sort(listForSort, new Comparator<DiskTO>() {
+
+ @Override
+ public int compare(DiskTO arg0, DiskTO arg1) {
+ if (arg0.getDiskSeq() < arg1.getDiskSeq()) {
+ return -1;
+ } else if (arg0.getDiskSeq().equals(arg1.getDiskSeq())) {
+ return 0;
+ }
+
+ return 1;
+ }
+ });
+
+ return listForSort.toArray(new DiskTO[0]);
+ }
+
+ private void postDiskConfigBeforeStart(VirtualMachineMO vmMo, VirtualMachineTO vmSpec, DiskTO[] sortedDisks,
+ Map<String, Map<String, String>> iqnToData, VmwareHypervisorHost hyperHost, VmwareContext context, boolean installAsIs)
+ throws Exception {
+ VirtualMachineDiskInfoBuilder diskInfoBuilder = vmMo.getDiskInfoBuilder();
+
+ for (DiskTO vol : sortedDisks) {
+ //TODO: Map existing disks to the ones returned in the answer
+ if (vol.getType() == Volume.Type.ISO)
+ continue;
+
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+
+ VirtualMachineDiskInfo diskInfo = getMatchingExistingDisk(diskInfoBuilder, vol, hyperHost, context);
+ assert (diskInfo != null);
+
+ String[] diskChain = diskInfo.getDiskChain();
+ assert (diskChain.length > 0);
+
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ DatastoreFile file = new DatastoreFile(diskChain[0]);
+
+ if (managed) {
+ DatastoreFile originalFile = new DatastoreFile(volumeTO.getPath());
+
+ if (!file.getFileBaseName().equalsIgnoreCase(originalFile.getFileBaseName())) {
+ if (LOGGER.isInfoEnabled())
+ LOGGER.info("Detected disk-chain top file change on volume: " + volumeTO.getId() + " " + volumeTO.getPath() + " -> " + diskChain[0]);
+ }
+ } else {
+ if (!file.getFileBaseName().equalsIgnoreCase(volumeTO.getPath())) {
+ if (LOGGER.isInfoEnabled())
+ LOGGER.info("Detected disk-chain top file change on volume: " + volumeTO.getId() + " " + volumeTO.getPath() + " -> " + file.getFileBaseName());
+ }
+ }
+
+ VolumeObjectTO volInSpec = getVolumeInSpec(vmSpec, volumeTO);
+
+ if (volInSpec != null) {
+ if (managed) {
+ Map<String, String> data = new HashMap<>();
+
+ String datastoreVolumePath = diskChain[0];
+
+ data.put(StartAnswer.PATH, datastoreVolumePath);
+ data.put(StartAnswer.IMAGE_FORMAT, Storage.ImageFormat.OVA.toString());
+
+ iqnToData.put(details.get(DiskTO.IQN), data);
+
+ vol.setPath(datastoreVolumePath);
+ volumeTO.setPath(datastoreVolumePath);
+ volInSpec.setPath(datastoreVolumePath);
+ } else {
+ volInSpec.setPath(file.getFileBaseName());
+ }
+ volInSpec.setChainInfo(vmwareResource.getGson().toJson(diskInfo));
+ }
+ }
+ }
+
+ private void checkBootOptions(VirtualMachineTO vmSpec, VirtualMachineConfigSpec vmConfigSpec) {
+ String bootMode = null;
+ if (vmSpec.getDetails().containsKey(VmDetailConstants.BOOT_MODE)) {
+ bootMode = vmSpec.getDetails().get(VmDetailConstants.BOOT_MODE);
+ }
+ if (null == bootMode) {
+ bootMode = ApiConstants.BootType.BIOS.toString();
+ }
+
+ setBootOptions(vmSpec, bootMode, vmConfigSpec);
+ }
+
+ private void setBootOptions(VirtualMachineTO vmSpec, String bootMode, VirtualMachineConfigSpec vmConfigSpec) {
+ VirtualMachineBootOptions bootOptions = null;
+ if (StringUtils.isNotBlank(bootMode) && !bootMode.equalsIgnoreCase("bios")) {
+ vmConfigSpec.setFirmware("efi");
+ if (vmSpec.getDetails().containsKey(ApiConstants.BootType.UEFI.toString()) && "secure".equalsIgnoreCase(vmSpec.getDetails().get(ApiConstants.BootType.UEFI.toString()))) {
+ if (bootOptions == null) {
+ bootOptions = new VirtualMachineBootOptions();
+ }
+ bootOptions.setEfiSecureBootEnabled(true);
+ }
+ }
+ if (vmSpec.isEnterHardwareSetup()) {
+ if (bootOptions == null) {
+ bootOptions = new VirtualMachineBootOptions();
+ }
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("configuring VM '%s' to enter hardware setup",vmSpec.getName()));
+ }
+ bootOptions.setEnterBIOSSetup(vmSpec.isEnterHardwareSetup());
+ }
+ if (bootOptions != null) {
+ vmConfigSpec.setBootOptions(bootOptions);
+ }
+ }
+
+ private void resizeRootDiskOnVMStart(VirtualMachineMO vmMo, DiskTO rootDiskTO, VmwareHypervisorHost hyperHost, VmwareContext context) throws Exception {
+ final Pair<VirtualDisk, String> vdisk = vmwareResource.getVirtualDiskInfo(vmMo, VmwareResource.appendFileType(rootDiskTO.getPath(), VmwareResource.VMDK_EXTENSION));
+ assert (vdisk != null);
+
+ Long reqSize = 0L;
+ final VolumeObjectTO volumeTO = ((VolumeObjectTO) rootDiskTO.getData());
+ if (volumeTO != null) {
+ reqSize = volumeTO.getSize() / 1024;
+ }
+ final VirtualDisk disk = vdisk.first();
+ if (reqSize > disk.getCapacityInKB()) {
+ final VirtualMachineDiskInfo diskInfo = getMatchingExistingDisk(vmMo.getDiskInfoBuilder(), rootDiskTO, hyperHost, context);
+ assert (diskInfo != null);
+ final String[] diskChain = diskInfo.getDiskChain();
+
+ if (diskChain != null && diskChain.length > 1) {
+ LOGGER.warn("Disk chain length for the VM is greater than one, this is not supported");
+ throw new CloudRuntimeException("Unsupported VM disk chain length: " + diskChain.length);
+ }
+
+ boolean resizingSupported = false;
+ String deviceBusName = diskInfo.getDiskDeviceBusName();
+ if (deviceBusName != null && (deviceBusName.toLowerCase().contains("scsi") || deviceBusName.toLowerCase().contains("lsi"))) {
+ resizingSupported = true;
+ }
+ if (!resizingSupported) {
+ LOGGER.warn("Resizing of root disk is only support for scsi device/bus, the provide VM's disk device bus name is " + diskInfo.getDiskDeviceBusName());
+ throw new CloudRuntimeException("Unsupported VM root disk device bus: " + diskInfo.getDiskDeviceBusName());
+ }
+
+ disk.setCapacityInKB(reqSize);
+ VirtualMachineConfigSpec vmConfigSpec = new VirtualMachineConfigSpec();
+ VirtualDeviceConfigSpec deviceConfigSpec = new VirtualDeviceConfigSpec();
+ deviceConfigSpec.setDevice(disk);
+ deviceConfigSpec.setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ vmConfigSpec.getDeviceChange().add(deviceConfigSpec);
+ if (!vmMo.configureVm(vmConfigSpec)) {
+ throw new Exception("Failed to configure VM for given root disk size. vmName: " + vmMo.getName());
+ }
+ }
+ }
+
+ private static void postNvpConfigBeforeStart(VirtualMachineMO vmMo, VirtualMachineTO vmSpec) throws Exception {
+ /**
+ * We need to configure the port on the DV switch after the host is
+ * connected. So make this happen between the configure and start of
+ * the VM
+ */
+ int nicIndex = 0;
+ for (NicTO nicTo : sortNicsByDeviceId(vmSpec.getNics())) {
+ if (nicTo.getBroadcastType() == Networks.BroadcastDomainType.Lswitch) {
+ // We need to create a port with a unique vlan and pass the key to the nic device
+ LOGGER.trace("Nic " + nicTo.toString() + " is connected to an NVP logicalswitch");
+ VirtualDevice nicVirtualDevice = vmMo.getNicDeviceByIndex(nicIndex);
+ if (nicVirtualDevice == null) {
+ throw new Exception("Failed to find a VirtualDevice for nic " + nicIndex); //FIXME Generic exceptions are bad
+ }
+ VirtualDeviceBackingInfo backing = nicVirtualDevice.getBacking();
+ if (backing instanceof VirtualEthernetCardDistributedVirtualPortBackingInfo) {
+ // This NIC is connected to a Distributed Virtual Switch
+ VirtualEthernetCardDistributedVirtualPortBackingInfo portInfo = (VirtualEthernetCardDistributedVirtualPortBackingInfo) backing;
+ DistributedVirtualSwitchPortConnection port = portInfo.getPort();
+ String portKey = port.getPortKey();
+ String portGroupKey = port.getPortgroupKey();
+ String dvSwitchUuid = port.getSwitchUuid();
+
+ LOGGER.debug("NIC " + nicTo.toString() + " is connected to dvSwitch " + dvSwitchUuid + " pg " + portGroupKey + " port " + portKey);
+
+ ManagedObjectReference dvSwitchManager = vmMo.getContext().getVimClient().getServiceContent().getDvSwitchManager();
+ ManagedObjectReference dvSwitch = vmMo.getContext().getVimClient().getService().queryDvsByUuid(dvSwitchManager, dvSwitchUuid);
+
+ // Get all ports
+ DistributedVirtualSwitchPortCriteria criteria = new DistributedVirtualSwitchPortCriteria();
+ criteria.setInside(true);
+ criteria.getPortgroupKey().add(portGroupKey);
+ List<DistributedVirtualPort> dvPorts = vmMo.getContext().getVimClient().getService().fetchDVPorts(dvSwitch, criteria);
+
+ DistributedVirtualPort vmDvPort = null;
+ List<Integer> usedVlans = new ArrayList<Integer>();
+ for (DistributedVirtualPort dvPort : dvPorts) {
+ // Find the port for this NIC by portkey
+ if (portKey.equals(dvPort.getKey())) {
+ vmDvPort = dvPort;
+ }
+ VMwareDVSPortSetting settings = (VMwareDVSPortSetting) dvPort.getConfig().getSetting();
+ VmwareDistributedVirtualSwitchVlanIdSpec vlanId = (VmwareDistributedVirtualSwitchVlanIdSpec) settings.getVlan();
+ LOGGER.trace("Found port " + dvPort.getKey() + " with vlan " + vlanId.getVlanId());
+ if (vlanId.getVlanId() > 0 && vlanId.getVlanId() < 4095) {
+ usedVlans.add(vlanId.getVlanId());
+ }
+ }
+
+ if (vmDvPort == null) {
+ throw new Exception("Empty port list from dvSwitch for nic " + nicTo.toString());
+ }
+
+ DVPortConfigInfo dvPortConfigInfo = vmDvPort.getConfig();
+ VMwareDVSPortSetting settings = (VMwareDVSPortSetting) dvPortConfigInfo.getSetting();
+
+ VmwareDistributedVirtualSwitchVlanIdSpec vlanId = (VmwareDistributedVirtualSwitchVlanIdSpec) settings.getVlan();
+ BoolPolicy blocked = settings.getBlocked();
+ if (blocked.isValue() == Boolean.TRUE) {
+ LOGGER.trace("Port is blocked, set a vlanid and unblock");
+ DVPortConfigSpec dvPortConfigSpec = new DVPortConfigSpec();
+ VMwareDVSPortSetting edittedSettings = new VMwareDVSPortSetting();
+ // Unblock
+ blocked.setValue(Boolean.FALSE);
+ blocked.setInherited(Boolean.FALSE);
+ edittedSettings.setBlocked(blocked);
+ // Set vlan
+ int i;
+ for (i = 1; i < 4095; i++) {
+ if (!usedVlans.contains(i))
+ break;
+ }
+ vlanId.setVlanId(i); // FIXME should be a determined
+ // based on usage
+ vlanId.setInherited(false);
+ edittedSettings.setVlan(vlanId);
+
+ dvPortConfigSpec.setSetting(edittedSettings);
+ dvPortConfigSpec.setOperation("edit");
+ dvPortConfigSpec.setKey(portKey);
+ List<DVPortConfigSpec> dvPortConfigSpecs = new ArrayList<DVPortConfigSpec>();
+ dvPortConfigSpecs.add(dvPortConfigSpec);
+ ManagedObjectReference task = vmMo.getContext().getVimClient().getService().reconfigureDVPortTask(dvSwitch, dvPortConfigSpecs);
+ if (!vmMo.getContext().getVimClient().waitForTask(task)) {
+ throw new Exception("Failed to configure the dvSwitch port for nic " + nicTo.toString());
+ }
+ LOGGER.debug("NIC " + nicTo.toString() + " connected to vlan " + i);
+ } else {
+ LOGGER.trace("Port already configured and set to vlan " + vlanId.getVlanId());
+ }
+ } else if (backing instanceof VirtualEthernetCardNetworkBackingInfo) {
+ // This NIC is connected to a Virtual Switch
+ // Nothing to do
+ } else if (backing instanceof VirtualEthernetCardOpaqueNetworkBackingInfo) {
+ //if NSX API VERSION >= 4.2, connect to br-int (nsx.network), do not create portgroup else previous behaviour
+ //OK, connected to OpaqueNetwork
+ } else {
+ LOGGER.error("nic device backing is of type " + backing.getClass().getName());
+ throw new Exception("Incompatible backing for a VirtualDevice for nic " + nicIndex); //FIXME Generic exceptions are bad
+ }
+ }
+ nicIndex++;
+ }
+ }
+
+ private VirtualMachineDiskInfo getMatchingExistingDisk(VirtualMachineDiskInfoBuilder diskInfoBuilder, DiskTO vol, VmwareHypervisorHost hyperHost, VmwareContext context)
+ throws Exception {
+ if (diskInfoBuilder != null) {
+ VolumeObjectTO volume = (VolumeObjectTO) vol.getData();
+
+ String dsName = null;
+ String diskBackingFileBaseName = null;
+
+ Map<String, String> details = vol.getDetails();
+ boolean isManaged = details != null && Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+
+ if (isManaged) {
+ String iScsiName = details.get(DiskTO.IQN);
+
+ // if the storage is managed, iScsiName should not be null
+ dsName = VmwareResource.getDatastoreName(iScsiName);
+
+ diskBackingFileBaseName = new DatastoreFile(volume.getPath()).getFileBaseName();
+ } else {
+ ManagedObjectReference morDs = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, volume.getDataStore().getUuid());
+ DatastoreMO dsMo = new DatastoreMO(context, morDs);
+
+ dsName = dsMo.getName();
+
+ diskBackingFileBaseName = volume.getPath();
+ }
+
+ VirtualMachineDiskInfo diskInfo = diskInfoBuilder.getDiskInfoByBackingFileBaseName(diskBackingFileBaseName, dsName);
+ if (diskInfo != null) {
+ LOGGER.info("Found existing disk info from volume path: " + volume.getPath());
+ return diskInfo;
+ } else {
+ String chainInfo = volume.getChainInfo();
+ if (chainInfo != null) {
+ VirtualMachineDiskInfo infoInChain = vmwareResource.getGson().fromJson(chainInfo, VirtualMachineDiskInfo.class);
+ if (infoInChain != null) {
+ String[] disks = infoInChain.getDiskChain();
+ if (disks.length > 0) {
+ for (String diskPath : disks) {
+ DatastoreFile file = new DatastoreFile(diskPath);
+ diskInfo = diskInfoBuilder.getDiskInfoByBackingFileBaseName(file.getFileBaseName(), dsName);
+ if (diskInfo != null) {
+ LOGGER.info("Found existing disk from chain info: " + diskPath);
+ return diskInfo;
+ }
+ }
+ }
+
+ if (diskInfo == null) {
+ diskInfo = diskInfoBuilder.getDiskInfoByDeviceBusName(infoInChain.getDiskDeviceBusName());
+ if (diskInfo != null) {
+ LOGGER.info("Found existing disk from from chain device bus information: " + infoInChain.getDiskDeviceBusName());
+ return diskInfo;
+ }
+ }
+ }
+ }
+ }
+ }
+
+ return null;
+ }
+
+ private static VolumeObjectTO getVolumeInSpec(VirtualMachineTO vmSpec, VolumeObjectTO srcVol) {
+ for (DiskTO disk : vmSpec.getDisks()) {
+ if (disk.getData() instanceof VolumeObjectTO) {
+ VolumeObjectTO vol = (VolumeObjectTO) disk.getData();
+ if (vol.getId() == srcVol.getId())
+ return vol;
+ }
+ }
+
+ return null;
+ }
+
+ /**
+ * Generate the mac sequence from the nics.
+ */
+ protected String generateMacSequence(NicTO[] nics) {
+ if (nics.length == 0) {
+ return "";
+ }
+
+ StringBuffer sbMacSequence = new StringBuffer();
+ for (NicTO nicTo : sortNicsByDeviceId(nics)) {
+ sbMacSequence.append(nicTo.getMac()).append("|");
+ }
+ if (!sbMacSequence.toString().isEmpty()) {
+ sbMacSequence.deleteCharAt(sbMacSequence.length() - 1); //Remove extra '|' char appended at the end
+ }
+
+ return sbMacSequence.toString();
+ }
+
+ static NicTO[] sortNicsByDeviceId(NicTO[] nics) {
+
+ List<NicTO> listForSort = new ArrayList<NicTO>();
+ for (NicTO nic : nics) {
+ listForSort.add(nic);
+ }
+ Collections.sort(listForSort, new Comparator<NicTO>() {
+
+ @Override
+ public int compare(NicTO arg0, NicTO arg1) {
+ if (arg0.getDeviceId() < arg1.getDeviceId()) {
+ return -1;
+ } else if (arg0.getDeviceId() == arg1.getDeviceId()) {
+ return 0;
+ }
+
+ return 1;
+ }
+ });
+
+ return listForSort.toArray(new NicTO[0]);
+ }
+
+ private static void configBasicExtraOption(List<OptionValue> extraOptions, VirtualMachineTO vmSpec) {
+ OptionValue newVal = new OptionValue();
+ newVal.setKey("machine.id");
+ newVal.setValue(vmSpec.getBootArgs());
+ extraOptions.add(newVal);
+
+ newVal = new OptionValue();
+ newVal.setKey("devices.hotplug");
+ newVal.setValue("true");
+ extraOptions.add(newVal);
+ }
+
+ private static void configNvpExtraOption(List<OptionValue> extraOptions, VirtualMachineTO vmSpec, Map<String, String> nicUuidToDvSwitchUuid) {
+ /**
+ * Extra Config : nvp.vm-uuid = uuid
+ * - Required for Nicira NVP integration
+ */
+ OptionValue newVal = new OptionValue();
+ newVal.setKey("nvp.vm-uuid");
+ newVal.setValue(vmSpec.getUuid());
+ extraOptions.add(newVal);
+
+ /**
+ * Extra Config : nvp.iface-id.<num> = uuid
+ * - Required for Nicira NVP integration
+ */
+ int nicNum = 0;
+ for (NicTO nicTo : sortNicsByDeviceId(vmSpec.getNics())) {
+ if (nicTo.getUuid() != null) {
+ newVal = new OptionValue();
+ newVal.setKey("nvp.iface-id." + nicNum);
+ newVal.setValue(nicTo.getUuid());
+ extraOptions.add(newVal);
+ }
+ nicNum++;
+ }
+ }
+
+ private static void configCustomExtraOption(List<OptionValue> extraOptions, VirtualMachineTO vmSpec) {
+ // we no longer to validation anymore
+ for (Map.Entry<String, String> entry : vmSpec.getDetails().entrySet()) {
+ if (entry.getKey().equalsIgnoreCase(VmDetailConstants.BOOT_MODE)) {
+ continue;
+ }
+ OptionValue newVal = new OptionValue();
+ newVal.setKey(entry.getKey());
+ newVal.setValue(entry.getValue());
+ extraOptions.add(newVal);
+ }
+ }
+
+ private class VmAlreadyExistsInVcenter extends Exception {
+ public VmAlreadyExistsInVcenter(String msg) {
+ }
+ }
+
+ private class VirtualMachineData {
+ String vmName = null;
+ VirtualMachineFileInfo vmFileInfo = null;
+ VirtualMachineFileLayoutEx vmFileLayout = null;
+ List<DatastoreMO> datastores = new ArrayList<>();
+ }
+
+ private class VirtualMachineRecycler {
+ private String vmInternalCSName;
+ private Pair<String, String> controllerInfo;
+ private Boolean systemVm;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private DiskControllerType systemVmScsiControllerType;
+ private int firstScsiControllerBusNum;
+ private int numScsiControllerForSystemVm;
+ private VirtualMachineDiskInfoBuilder diskInfoBuilder;
+ private boolean hasSnapshot;
+ private VirtualDevice[] nicDevices;
+
+ public VirtualMachineRecycler(String vmInternalCSName, Pair<String, String> controllerInfo, Boolean systemVm, VmwareHypervisorHost hyperHost, VirtualMachineMO vmMo,
+ DiskControllerType systemVmScsiControllerType, int firstScsiControllerBusNum, int numScsiControllerForSystemVm) {
+ this.vmInternalCSName = vmInternalCSName;
+ this.controllerInfo = controllerInfo;
+ this.systemVm = systemVm;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.systemVmScsiControllerType = systemVmScsiControllerType;
+ this.firstScsiControllerBusNum = firstScsiControllerBusNum;
+ this.numScsiControllerForSystemVm = numScsiControllerForSystemVm;
+ }
+
+ public VirtualMachineDiskInfoBuilder getDiskInfoBuilder() {
+ return diskInfoBuilder;
+ }
+
+ public boolean isHasSnapshot() {
+ return hasSnapshot;
+ }
+
+ public VirtualMachineRecycler invoke() throws Exception {
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Found vm " + vmInternalCSName + " at other host, relocate to " + hyperHost.getHyperHostName());
+ }
+
+ takeVmFromOtherHyperHost(hyperHost, vmInternalCSName);
+
+ if (VmwareResource.getVmPowerState(vmMo) != VirtualMachine.PowerState.PowerOff)
+ vmMo.safePowerOff(vmwareResource.getShutdownWaitMs());
+
+ diskInfoBuilder = vmMo.getDiskInfoBuilder();
+ hasSnapshot = vmMo.hasSnapshot();
+ nicDevices = vmMo.getNicDevices();
+ if (!hasSnapshot)
+ vmMo.tearDownDevices(new Class<?>[] {VirtualDisk.class, VirtualEthernetCard.class});
+ else
+ vmMo.tearDownDevices(new Class<?>[] {VirtualEthernetCard.class});
+
+ if (systemVm) {
+ // System volumes doesn't require more than 1 SCSI controller as there is no requirement for data volumes.
+ ensureScsiDiskControllers(vmMo, systemVmScsiControllerType.toString(), numScsiControllerForSystemVm, firstScsiControllerBusNum);
+ } else {
+ ensureDiskControllers(vmMo, controllerInfo);
+ }
+ return this;
+ }
+
+ private VirtualMachineMO takeVmFromOtherHyperHost(VmwareHypervisorHost hyperHost, String vmName) throws Exception {
+
+ VirtualMachineMO vmMo = hyperHost.findVmOnPeerHyperHost(vmName);
+ if (vmMo != null) {
+ ManagedObjectReference morTargetPhysicalHost = hyperHost.findMigrationTarget(vmMo);
+ if (morTargetPhysicalHost == null) {
+ String msg = "VM " + vmName + " is on other host and we have no resource available to migrate and start it here";
+ LOGGER.error(msg);
+ throw new Exception(msg);
+ }
+
+ if (!vmMo.relocate(morTargetPhysicalHost)) {
+ String msg = "VM " + vmName + " is on other host and we failed to relocate it here";
+ LOGGER.error(msg);
+ throw new Exception(msg);
+ }
+
+ return vmMo;
+ }
+ return null;
+ }
+
+ public VirtualDevice[] getNicDevices() {
+ return nicDevices;
+ }
+ }
+
+ private class PrepareSytemVMPatchISOMethod {
+ private VmwareManager mgr;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int i;
+ private int ideUnitNumber;
+
+ public PrepareSytemVMPatchISOMethod(VmwareManager mgr, VmwareHypervisorHost hyperHost, VirtualMachineMO vmMo, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int i, int ideUnitNumber) {
+ this.mgr = mgr;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.i = i;
+ this.ideUnitNumber = ideUnitNumber;
+ }
+
+ public int getI() {
+ return i;
+ }
+
+ public int getIdeUnitNumber() {
+ return ideUnitNumber;
+ }
+
+ public PrepareSytemVMPatchISOMethod invoke() throws Exception {
+ // attach ISO (for patching of system VM)
+ DatastoreMO secDsMo = getDatastoreMOForSecStore(mgr, hyperHost);
+
+ deviceConfigSpecArray[i] = new VirtualDeviceConfigSpec();
+ Pair<VirtualDevice, Boolean> isoInfo = VmwareHelper
+ .prepareIsoDevice(vmMo, String.format("[%s] systemvm/%s", secDsMo.getName(), mgr.getSystemVMIsoFileNameOnDatastore()), secDsMo.getMor(), true, true,
+ ideUnitNumber++, i + 1);
+ deviceConfigSpecArray[i].setDevice(isoInfo.first());
+ if (isoInfo.second()) {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at new device " + vmwareResource.getGson().toJson(isoInfo.first()));
+ deviceConfigSpecArray[i].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+ } else {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+ deviceConfigSpecArray[i].setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ }
+ i++;
+ return this;
+ }
+
+ }
+ private DatastoreMO getDatastoreMOForSecStore(VmwareManager mgr, VmwareHypervisorHost hyperHost) throws Exception {
+ Pair<String, Long> secStoreUrlAndId = mgr.getSecondaryStorageStoreUrlAndId(Long.parseLong(vmwareResource.getDcId()));
+ String secStoreUrl = secStoreUrlAndId.first();
+ Long secStoreId = secStoreUrlAndId.second();
+ if (secStoreUrl == null) {
+ String msg = "secondary storage for dc " + vmwareResource.getDcId() + " is not ready yet?";
+ throw new Exception(msg);
+ }
+ mgr.prepareSecondaryStorageStore(secStoreUrl, secStoreId);
+
+ ManagedObjectReference morSecDs = vmwareResource.prepareSecondaryDatastoreOnHost(secStoreUrl);
+ if (morSecDs == null) {
+ String msg = "Failed to prepare secondary storage on host, secondary store url: " + secStoreUrl;
+ throw new Exception(msg);
+ }
+ return new DatastoreMO(hyperHost.getContext(), morSecDs);
+ }
+
+ /**
+ // Setup ROOT/DATA disk devices
+ */
+ private class DiskSetup {
+ private VirtualMachineTO vmSpec;
+ private DiskTO rootDiskTO;
+ private Pair<String, String> controllerInfo;
+ private VmwareContext context;
+ private DatacenterMO dcMo;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private DiskTO[] disks;
+ private HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails;
+ private VirtualMachineDiskInfoBuilder diskInfoBuilder;
+ private boolean hasSnapshot;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int deviceCount;
+ private int ideUnitNumber;
+ private int scsiUnitNumber;
+ private int ideControllerKey;
+ private int scsiControllerKey;
+ private DiskTO[] sortedDisks;
+ private boolean installAsIs;
+
+ public DiskSetup(VirtualMachineTO vmSpec, DiskTO rootDiskTO, Pair<String, String> controllerInfo, VmwareContext context, DatacenterMO dcMo, VmwareHypervisorHost hyperHost,
+ VirtualMachineMO vmMo, DiskTO[] disks, HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails, VirtualMachineDiskInfoBuilder diskInfoBuilder,
+ boolean hasSnapshot, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount, int ideUnitNumber, int scsiUnitNumber, int ideControllerKey, int scsiControllerKey, boolean installAsIs) {
+ this.vmSpec = vmSpec;
+ this.rootDiskTO = rootDiskTO;
+ this.controllerInfo = controllerInfo;
+ this.context = context;
+ this.dcMo = dcMo;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.disks = disks;
+ this.dataStoresDetails = dataStoresDetails;
+ this.diskInfoBuilder = diskInfoBuilder;
+ this.hasSnapshot = hasSnapshot;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.deviceCount = deviceCount;
+ this.ideUnitNumber = ideUnitNumber;
+ this.scsiUnitNumber = scsiUnitNumber;
+ this.ideControllerKey = ideControllerKey;
+ this.scsiControllerKey = scsiControllerKey;
+ this.installAsIs = installAsIs;
+ }
+
+ public DiskTO getRootDiskTO() {
+ return rootDiskTO;
+ }
+
+ public int getDeviceCount() {
+ return deviceCount;
+ }
+
+ public DiskTO[] getSortedDisks() {
+ return sortedDisks;
+ }
+
+ public DiskSetup invoke() throws Exception {
+ int controllerKey;
+ sortedDisks = sortVolumesByDeviceId(disks);
+ for (DiskTO vol : sortedDisks) {
+ if (vol.getType() == Volume.Type.ISO || installAsIs) {
+ continue;
+ }
+
+ VirtualMachineDiskInfo matchingExistingDisk = getMatchingExistingDisk(diskInfoBuilder, vol, hyperHost, context);
+ controllerKey = getDiskController(matchingExistingDisk, vol, vmSpec, ideControllerKey, scsiControllerKey);
+ String diskController = getDiskController(vmMo, matchingExistingDisk, vol, controllerInfo);
+
+ if (DiskControllerType.getType(diskController) == DiskControllerType.osdefault) {
+ diskController = vmMo.getRecommendedDiskController(null);
+ }
+ if (DiskControllerType.getType(diskController) == DiskControllerType.ide) {
+ controllerKey = vmMo.getIDEControllerKey(ideUnitNumber);
+ if (vol.getType() == Volume.Type.DATADISK) {
+ // Could be result of flip due to user configured setting or "osdefault" for data disks
+ // Ensure maximum of 2 data volumes over IDE controller, 3 includeing root volume
+ if (vmMo.getNumberOfVirtualDisks() > 3) {
+ throw new CloudRuntimeException(
+ "Found more than 3 virtual disks attached to this VM [" + vmMo.getVmName() + "]. Unable to implement the disks over " + diskController + " controller, as maximum number of devices supported over IDE controller is 4 includeing CDROM device.");
+ }
+ }
+ } else {
+ if (VmwareHelper.isReservedScsiDeviceNumber(scsiUnitNumber)) {
+ scsiUnitNumber++;
+ }
+
+ controllerKey = vmMo.getScsiDiskControllerKeyNoException(diskController, scsiUnitNumber);
+ if (controllerKey == -1) {
+ // This may happen for ROOT legacy VMs which doesn't have recommended disk controller when global configuration parameter 'vmware.root.disk.controller' is set to "osdefault"
+ // Retrieve existing controller and use.
+ Ternary<Integer, Integer, DiskControllerType> vmScsiControllerInfo = vmMo.getScsiControllerInfo();
+ DiskControllerType existingControllerType = vmScsiControllerInfo.third();
+ controllerKey = vmMo.getScsiDiskControllerKeyNoException(existingControllerType.toString(), scsiUnitNumber);
+ }
+ }
+ if (!hasSnapshot) {
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+
+ VolumeObjectTO volumeTO = (VolumeObjectTO)vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+ String iScsiName = null;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ iScsiName = details.get(DiskTO.IQN);
+ }
+
+ // if the storage is managed, iScsiName should not be null
+ String datastoreName = managed ? VmwareResource.getDatastoreName(iScsiName) : primaryStore.getUuid();
+ Pair<ManagedObjectReference, DatastoreMO> volumeDsDetails = dataStoresDetails.get(datastoreName);
+
+ assert (volumeDsDetails != null);
+
+ String[] diskChain = syncDiskChain(dcMo, vmMo, vmSpec, vol, matchingExistingDisk, dataStoresDetails);
+
+ int deviceNumber = -1;
+ if (controllerKey == vmMo.getIDEControllerKey(ideUnitNumber)) {
+ deviceNumber = ideUnitNumber % VmwareHelper.MAX_ALLOWED_DEVICES_IDE_CONTROLLER;
+ ideUnitNumber++;
+ } else {
+ deviceNumber = scsiUnitNumber % VmwareHelper.MAX_ALLOWED_DEVICES_SCSI_CONTROLLER;
+ scsiUnitNumber++;
+ }
+ VirtualDevice device = VmwareHelper.prepareDiskDevice(vmMo, null, controllerKey, diskChain, volumeDsDetails.first(), deviceNumber, deviceCount + 1);
+ if (vol.getType() == Volume.Type.ROOT)
+ rootDiskTO = vol;
+ deviceConfigSpecArray[deviceCount].setDevice(device);
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare volume at new device " + vmwareResource.getGson().toJson(device));
+
+ deviceCount++;
+ } else {
+ if (controllerKey == vmMo.getIDEControllerKey(ideUnitNumber))
+ ideUnitNumber++;
+ else
+ scsiUnitNumber++;
+ }
+ }
+ return this;
+ }
+
+ private int getDiskController(VirtualMachineDiskInfo matchingExistingDisk, DiskTO vol, VirtualMachineTO vmSpec, int ideControllerKey, int scsiControllerKey) {
+
+ int controllerKey;
+ if (matchingExistingDisk != null) {
+ LOGGER.info("Chose disk controller based on existing information: " + matchingExistingDisk.getDiskDeviceBusName());
+ if (matchingExistingDisk.getDiskDeviceBusName().startsWith("ide"))
+ return ideControllerKey;
+ else
+ return scsiControllerKey;
+ }
+
+ if (vol.getType() == Volume.Type.ROOT) {
+ Map<String, String> vmDetails = vmSpec.getDetails();
+ if (vmDetails != null && vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER) != null) {
+ if (vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER).equalsIgnoreCase("scsi")) {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> scsi, based on root disk controller settings: "
+ + vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER));
+ controllerKey = scsiControllerKey;
+ } else {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> ide, based on root disk controller settings: "
+ + vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER));
+ controllerKey = ideControllerKey;
+ }
+ } else {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> scsi. due to null root disk controller setting");
+ controllerKey = scsiControllerKey;
+ }
+
+ } else {
+ // DATA volume always use SCSI device
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> scsi");
+ controllerKey = scsiControllerKey;
+ }
+
+ return controllerKey;
+ }
+
+ private String getDiskController(VirtualMachineMO vmMo, VirtualMachineDiskInfo matchingExistingDisk, DiskTO vol, Pair<String, String> controllerInfo) throws Exception {
+ int controllerKey;
+ DiskControllerType controllerType = DiskControllerType.none;
+ if (matchingExistingDisk != null) {
+ String currentBusName = matchingExistingDisk.getDiskDeviceBusName();
+ if (currentBusName != null) {
+ LOGGER.info("Chose disk controller based on existing information: " + currentBusName);
+ if (currentBusName.startsWith("ide")) {
+ controllerType = DiskControllerType.ide;
+ } else if (currentBusName.startsWith("scsi")) {
+ controllerType = DiskControllerType.scsi;
+ }
+ }
+ if (controllerType == DiskControllerType.scsi || controllerType == DiskControllerType.none) {
+ Ternary<Integer, Integer, DiskControllerType> vmScsiControllerInfo = vmMo.getScsiControllerInfo();
+ controllerType = vmScsiControllerInfo.third();
+ }
+ return controllerType.toString();
+ }
+
+ if (vol.getType() == Volume.Type.ROOT) {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> " + controllerInfo.first()
+ + ", based on root disk controller settings at global configuration setting.");
+ return controllerInfo.first();
+ } else {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> " + controllerInfo.second()
+ + ", based on default data disk controller setting i.e. Operating system recommended."); // Need to bring in global configuration setting & template level setting.
+ return controllerInfo.second();
+ }
+ }
+
+ // return the finalized disk chain for startup, from top to bottom
+ private String[] syncDiskChain(DatacenterMO dcMo, VirtualMachineMO vmMo, VirtualMachineTO vmSpec, DiskTO vol, VirtualMachineDiskInfo diskInfo,
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails) throws Exception {
+
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ Map<String, String> details = vol.getDetails();
+ boolean isManaged = false;
+ String iScsiName = null;
+
+ if (details != null) {
+ isManaged = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ iScsiName = details.get(DiskTO.IQN);
+ }
+
+ // if the storage is managed, iScsiName should not be null
+ String datastoreName = isManaged ? VmwareResource.getDatastoreName(iScsiName) : primaryStore.getUuid();
+ Pair<ManagedObjectReference, DatastoreMO> volumeDsDetails = dataStoresDetails.get(datastoreName);
+
+ if (volumeDsDetails == null) {
+ throw new Exception("Primary datastore " + primaryStore.getUuid() + " is not mounted on host.");
+ }
+
+ DatastoreMO dsMo = volumeDsDetails.second();
+
+ // we will honor vCenter's meta if it exists
+ if (diskInfo != null) {
+ // to deal with run-time upgrade to maintain the new datastore folder structure
+ String disks[] = diskInfo.getDiskChain();
+ for (int i = 0; i < disks.length; i++) {
+ DatastoreFile file = new DatastoreFile(disks[i]);
+ if (!isManaged && file.getDir() != null && file.getDir().isEmpty()) {
+ LOGGER.info("Perform run-time datastore folder upgrade. sync " + disks[i] + " to VM folder");
+ disks[i] = VmwareStorageLayoutHelper.syncVolumeToVmDefaultFolder(dcMo, vmMo.getName(), dsMo, file.getFileBaseName(), VmwareManager.s_vmwareSearchExcludeFolder.value());
+ }
+ }
+ return disks;
+ }
+
+ final String datastoreDiskPath;
+
+ if (isManaged) {
+ String vmdkPath = new DatastoreFile(volumeTO.getPath()).getFileBaseName();
+
+ if (volumeTO.getVolumeType() == Volume.Type.ROOT) {
+ if (vmdkPath == null) {
+ vmdkPath = volumeTO.getName();
+ }
+
+ datastoreDiskPath = VmwareStorageLayoutHelper.syncVolumeToVmDefaultFolder(dcMo, vmMo.getName(), dsMo, vmdkPath);
+ } else {
+ if (vmdkPath == null) {
+ vmdkPath = dsMo.getName();
+ }
+
+ datastoreDiskPath = dsMo.getDatastorePath(vmdkPath + VmwareResource.VMDK_EXTENSION);
+ }
+ } else {
+ datastoreDiskPath = VmwareStorageLayoutHelper.syncVolumeToVmDefaultFolder(dcMo, vmMo.getName(), dsMo, volumeTO.getPath(), VmwareManager.s_vmwareSearchExcludeFolder.value());
+ }
+
+ if (!dsMo.fileExists(datastoreDiskPath)) {
+ LOGGER.warn("Volume " + volumeTO.getId() + " does not seem to exist on datastore, out of sync? path: " + datastoreDiskPath);
+ }
+
+ return new String[]{datastoreDiskPath};
+ }
+ }
+
+ /**
+ // Setup NIC devices
+ */
+ private class NicSetup {
+ private StartCommand cmd;
+ private VirtualMachineTO vmSpec;
+ private String vmInternalCSName;
+ private VmwareContext context;
+ private VmwareManager mgr;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private NicTO[] nics;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int deviceCount;
+ private int nicMask;
+ private int nicCount;
+ private Map<String, String> nicUuidToDvSwitchUuid;
+ private VirtualDevice[] nicDevices;
+
+ public NicSetup(StartCommand cmd, VirtualMachineTO vmSpec, String vmInternalCSName, VmwareContext context, VmwareManager mgr, VmwareHypervisorHost hyperHost,
+ VirtualMachineMO vmMo, NicTO[] nics, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount, VirtualDevice[] nicDevices) {
+ this.cmd = cmd;
+ this.vmSpec = vmSpec;
+ this.vmInternalCSName = vmInternalCSName;
+ this.context = context;
+ this.mgr = mgr;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.nics = nics;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.deviceCount = deviceCount;
+ this.nicDevices = nicDevices;
+ }
+
+ public int getDeviceCount() {
+ return deviceCount;
+ }
+
+ public int getNicMask() {
+ return nicMask;
+ }
+
+ public int getNicCount() {
+ return nicCount;
+ }
+
+ public Map<String, String> getNicUuidToDvSwitchUuid() {
+ return nicUuidToDvSwitchUuid;
+ }
+
+ public NicSetup invoke() throws Exception {
+ VirtualDevice nic;
+ nicMask = 0;
+ nicCount = 0;
+
+ if (vmSpec.getType() == VirtualMachine.Type.DomainRouter) {
+ doDomainRouterSetup();
+ }
+
+ VirtualEthernetCardType nicDeviceType = VirtualEthernetCardType.valueOf(vmSpec.getDetails().get(VmDetailConstants.NIC_ADAPTER));
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("VM " + vmInternalCSName + " will be started with NIC device type: " + nicDeviceType);
+ }
+
+ NiciraNvpApiVersion.logNiciraApiVersion();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.debug(String.format("deciding for VM '%s' to use orchestrated NICs or as is.", vmInternalCSName));
+ }
+ nicUuidToDvSwitchUuid = new HashMap<>();
+ LOGGER.info(String.format("adding %d nics to VM '%s'", nics.length, vmInternalCSName));
+ for (NicTO nicTo : sortNicsByDeviceId(nics)) {
+ LOGGER.info("Prepare NIC device based on NicTO: " + vmwareResource.getGson().toJson(nicTo));
+
+ boolean configureVServiceInNexus = (nicTo.getType() == Networks.TrafficType.Guest) && (vmSpec.getDetails().containsKey("ConfigureVServiceInNexus"));
+ VirtualMachine.Type vmType = cmd.getVirtualMachine().getType();
+ Pair<ManagedObjectReference, String> networkInfo = vmwareResource.prepareNetworkFromNicInfo(vmMo.getRunningHost(), nicTo, configureVServiceInNexus, vmType);
+ if ((nicTo.getBroadcastType() != Networks.BroadcastDomainType.Lswitch) || (nicTo.getBroadcastType() == Networks.BroadcastDomainType.Lswitch && NiciraNvpApiVersion.isApiVersionLowerThan("4.2"))) {
+ if (VmwareHelper.isDvPortGroup(networkInfo.first())) {
+ String dvSwitchUuid;
+ ManagedObjectReference dcMor = hyperHost.getHyperHostDatacenter();
+ DatacenterMO dataCenterMo = new DatacenterMO(context, dcMor);
+ ManagedObjectReference dvsMor = dataCenterMo.getDvSwitchMor(networkInfo.first());
+ dvSwitchUuid = dataCenterMo.getDvSwitchUuid(dvsMor);
+ LOGGER.info("Preparing NIC device on dvSwitch : " + dvSwitchUuid);
+ nic = VmwareHelper.prepareDvNicDevice(vmMo, networkInfo.first(), nicDeviceType, networkInfo.second(), dvSwitchUuid, nicTo.getMac(), deviceCount + 1, true, true);
+ if (nicTo.getUuid() != null) {
+ nicUuidToDvSwitchUuid.put(nicTo.getUuid(), dvSwitchUuid);
+ }
+ } else {
+ LOGGER.info("Preparing NIC device on network " + networkInfo.second());
+ nic = VmwareHelper.prepareNicDevice(vmMo, networkInfo.first(), nicDeviceType, networkInfo.second(), nicTo.getMac(), deviceCount + 1, true, true);
+ }
+ } else {
+ //if NSX API VERSION >= 4.2, connect to br-int (nsx.network), do not create portgroup else previous behaviour
+ nic = VmwareHelper.prepareNicOpaque(vmMo, nicDeviceType, networkInfo.second(), nicTo.getMac(), deviceCount + 1, true, true);
+ }
+
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ deviceConfigSpecArray[deviceCount].setDevice(nic);
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare NIC at new device " + vmwareResource.getGson().toJson(deviceConfigSpecArray[deviceCount]));
+
+ // this is really a hacking for DomR, upon DomR startup, we will reset all the NIC allocation after eth3
+ if (nicCount < 3)
+ nicMask |= (1 << nicCount);
+
+ deviceCount++;
+ nicCount++;
+ }
+ return this;
+ }
+
+ private void doDomainRouterSetup() throws Exception {
+ int extraPublicNics = mgr.getRouterExtraPublicNics();
+ if (extraPublicNics > 0 && vmSpec.getDetails().containsKey("PeerRouterInstanceName")) {
+ //Set identical MAC address for RvR on extra public interfaces
+ String peerRouterInstanceName = vmSpec.getDetails().get("PeerRouterInstanceName");
+
+ VirtualMachineMO peerVmMo = hyperHost.findVmOnHyperHost(peerRouterInstanceName);
+ if (peerVmMo == null) {
+ peerVmMo = hyperHost.findVmOnPeerHyperHost(peerRouterInstanceName);
+ }
+
+ if (peerVmMo != null) {
+ String oldMacSequence = generateMacSequence(nics);
+
+ for (int nicIndex = nics.length - extraPublicNics; nicIndex < nics.length; nicIndex++) {
+ VirtualDevice nicDevice = peerVmMo.getNicDeviceByIndex(nics[nicIndex].getDeviceId());
+ if (nicDevice != null) {
+ String mac = ((VirtualEthernetCard)nicDevice).getMacAddress();
+ if (mac != null) {
+ LOGGER.info("Use same MAC as previous RvR, the MAC is " + mac + " for extra NIC with device id: " + nics[nicIndex].getDeviceId());
+ nics[nicIndex].setMac(mac);
+ }
+ }
+ }
+
+ if (!StringUtils.isBlank(vmSpec.getBootArgs())) {
+ String newMacSequence = generateMacSequence(nics);
+ vmSpec.setBootArgs(vmwareResource.replaceNicsMacSequenceInBootArgs(oldMacSequence, newMacSequence, vmSpec));
+ }
+ }
+ }
+ }
+ }
+
+ private class IsoSetup {
+ private VirtualMachineTO vmSpec;
+ private VmwareManager mgr;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private DiskTO[] disks;
+ private DiskTO volIso;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int deviceCount;
+ private int ideUnitNumber;
+
+ public IsoSetup(VirtualMachineTO vmSpec, VmwareManager mgr, VmwareHypervisorHost hyperHost, VirtualMachineMO vmMo, DiskTO[] disks, DiskTO volIso,
+ VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount, int ideUnitNumber) {
+ this.vmSpec = vmSpec;
+ this.mgr = mgr;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.disks = disks;
+ this.volIso = volIso;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.deviceCount = deviceCount;
+ this.ideUnitNumber = ideUnitNumber;
+ }
+
+ public int getDeviceCount() {
+ return deviceCount;
+ }
+
+ public int getIdeUnitNumber() {
+ return ideUnitNumber;
+ }
+
+ public IsoSetup invoke() throws Exception {
+ //
+ // Setup ISO device
+ //
+
+ // vAPP ISO
+ // FR37 the native deploy mechs should create this for us
+ if (vmSpec.getOvfProperties() != null) {
+ if (LOGGER.isTraceEnabled()) {
+ // FR37 TODO add more usefull info (if we keep this bit
+ LOGGER.trace("adding iso for properties for 'xxx'");
+ }
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ Pair<VirtualDevice, Boolean> isoInfo = VmwareHelper.prepareIsoDevice(vmMo, null, null, true, true, ideUnitNumber++, deviceCount + 1);
+ deviceConfigSpecArray[deviceCount].setDevice(isoInfo.first());
+ if (isoInfo.second()) {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare vApp ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+ } else {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare vApp ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ }
+ deviceCount++;
+ }
+
+ // prepare systemvm patch ISO
Review comment:
private method name `IsoSetup.prepareSystemVmPatchIso()`
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/StartCommandExecutor.java
##########
@@ -0,0 +1,2247 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.hypervisor.vmware.resource;
+
+import java.io.File;
+import java.rmi.RemoteException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.cloud.agent.api.to.DataTO;
+import com.vmware.vim25.VirtualDiskFlatVer2BackingInfo;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.storage.configdrive.ConfigDrive;
+import org.apache.cloudstack.storage.to.TemplateObjectTO;
+import org.apache.cloudstack.storage.to.VolumeObjectTO;
+import org.apache.cloudstack.utils.volume.VirtualMachineDiskInfo;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.commons.lang.ArrayUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.log4j.Logger;
+
+import com.cloud.agent.api.Command;
+import com.cloud.agent.api.StartAnswer;
+import com.cloud.agent.api.StartCommand;
+import com.cloud.agent.api.storage.OVFPropertyTO;
+import com.cloud.agent.api.to.DataStoreTO;
+import com.cloud.agent.api.to.DiskTO;
+import com.cloud.agent.api.to.NfsTO;
+import com.cloud.agent.api.to.NicTO;
+import com.cloud.agent.api.to.VirtualMachineTO;
+import com.cloud.configuration.Resource;
+import com.cloud.hypervisor.vmware.VmwareResourceException;
+import com.cloud.hypervisor.vmware.manager.VmwareManager;
+import com.cloud.hypervisor.vmware.mo.CustomFieldConstants;
+import com.cloud.hypervisor.vmware.mo.DatacenterMO;
+import com.cloud.hypervisor.vmware.mo.DatastoreFile;
+import com.cloud.hypervisor.vmware.mo.DatastoreMO;
+import com.cloud.hypervisor.vmware.mo.DiskControllerType;
+import com.cloud.hypervisor.vmware.mo.HostMO;
+import com.cloud.hypervisor.vmware.mo.HypervisorHostHelper;
+import com.cloud.hypervisor.vmware.mo.TaskMO;
+import com.cloud.hypervisor.vmware.mo.VirtualEthernetCardType;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineDiskInfoBuilder;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineMO;
+import com.cloud.hypervisor.vmware.mo.VmwareHypervisorHost;
+import com.cloud.hypervisor.vmware.util.VmwareContext;
+import com.cloud.hypervisor.vmware.util.VmwareHelper;
+import com.cloud.network.Networks;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Volume;
+import com.cloud.storage.resource.VmwareStorageLayoutHelper;
+import com.cloud.storage.resource.VmwareStorageProcessor;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nicira.nvp.plugin.NiciraNvpApiVersion;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.VmDetailConstants;
+import com.vmware.vim25.BoolPolicy;
+import com.vmware.vim25.DVPortConfigInfo;
+import com.vmware.vim25.DVPortConfigSpec;
+import com.vmware.vim25.DasVmPriority;
+import com.vmware.vim25.DistributedVirtualPort;
+import com.vmware.vim25.DistributedVirtualSwitchPortConnection;
+import com.vmware.vim25.DistributedVirtualSwitchPortCriteria;
+import com.vmware.vim25.ManagedObjectReference;
+import com.vmware.vim25.OptionValue;
+import com.vmware.vim25.VMwareDVSPortSetting;
+import com.vmware.vim25.VirtualDevice;
+import com.vmware.vim25.VirtualDeviceBackingInfo;
+import com.vmware.vim25.VirtualDeviceConfigSpec;
+import com.vmware.vim25.VirtualDeviceConfigSpecOperation;
+import com.vmware.vim25.VirtualDisk;
+import com.vmware.vim25.VirtualEthernetCard;
+import com.vmware.vim25.VirtualEthernetCardDistributedVirtualPortBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardNetworkBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardOpaqueNetworkBackingInfo;
+import com.vmware.vim25.VirtualMachineBootOptions;
+import com.vmware.vim25.VirtualMachineConfigSpec;
+import com.vmware.vim25.VirtualMachineFileInfo;
+import com.vmware.vim25.VirtualMachineFileLayoutEx;
+import com.vmware.vim25.VirtualMachineGuestOsIdentifier;
+import com.vmware.vim25.VirtualUSBController;
+import com.vmware.vim25.VmConfigInfo;
+import com.vmware.vim25.VmwareDistributedVirtualSwitchVlanIdSpec;
+
+class StartCommandExecutor {
+ private static final Logger LOGGER = Logger.getLogger(StartCommandExecutor.class);
+
+ private final VmwareResource vmwareResource;
+
+ public StartCommandExecutor(VmwareResource vmwareResource) {
+ this.vmwareResource = vmwareResource;
+ }
+
+ // FR37 the monster blob god method: reduce to 320 so far and counting
+ protected StartAnswer execute(StartCommand cmd) {
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Executing resource StartCommand: " + vmwareResource.getGson().toJson(cmd));
+ }
+
+ VirtualMachineTO vmSpec = cmd.getVirtualMachine();
+
+ VirtualMachineData existingVm = null;
+
+ Pair<String, String> names = composeVmNames(vmSpec);
+ String vmInternalCSName = names.first();
+ String vmNameOnVcenter = names.second();
+
+ DiskTO rootDiskTO = null;
+
+ Pair<String, String> controllerInfo = getDiskControllerInfo(vmSpec);
+
+ Boolean systemVm = vmSpec.getType().isUsedBySystem();
+
+ VmwareContext context = vmwareResource.getServiceContext();
+ DatacenterMO dcMo = null;
+ try {
+ VmwareManager mgr = context.getStockObject(VmwareManager.CONTEXT_STOCK_NAME);
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(context);
+ dcMo = new DatacenterMO(hyperHost.getContext(), hyperHost.getHyperHostDatacenter());
+
+ // checkIfVmExistsInVcenter(vmInternalCSName, vmNameOnVcenter, dcMo);
+ // FR37 - We expect VM to be already cloned and available at this point
+ VirtualMachineMO vmMo = dcMo.findVm(vmInternalCSName);
+ if (vmMo == null) {
+ vmMo = dcMo.findVm(vmNameOnVcenter);
+ }
+
+ DiskTO[] specDisks = vmSpec.getDisks();
+ boolean installAsIs = StringUtils.isNotEmpty(vmSpec.getTemplateLocation());
+ // FR37 if startcommand contains enough info: a template url/-location and flag; deploy OVA as is
+ if (vmMo == null && installAsIs) {
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("deploying OVA from %s as is", vmSpec.getTemplateLocation()));
+ }
+ getStorageProcessor().cloneVMFromTemplate(vmSpec.getTemplateName(), vmInternalCSName, vmSpec.getTemplatePrimaryStoreUuid());
+ vmMo = dcMo.findVm(vmInternalCSName);
+ mapSpecDisksToClonedDisks(vmMo, vmInternalCSName, specDisks);
+ }
+
+ // VM may not have been on the same host, relocate to expected host
+ if (vmMo != null) {
+ vmMo.relocate(hyperHost.getMor());
+ // Get updated MO
+ vmMo = hyperHost.findVmOnHyperHost(vmMo.getVmName());
+ }
+
+ String guestOsId = translateGuestOsIdentifier(vmSpec.getArch(), vmSpec.getOs(), vmSpec.getPlatformEmulator()).value();
+ DiskTO[] disks = validateDisks(specDisks);
+ NicTO[] nics = vmSpec.getNics();
+
+ // FIXME: disks logic here, why is disks/volumes during copy not set with pool ID?
+ // FR37 TODO if deployasis a new VM no datastores are known (yet) and we need to get the data store from the tvmspec content library / template location
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails = inferDatastoreDetailsFromDiskInfo(hyperHost, context, disks, cmd);
+ if ((dataStoresDetails == null) || (dataStoresDetails.isEmpty())) {
+ String msg = "Unable to locate datastore details of the volumes to be attached";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ // FR37 - this may not be necessary the cloned VM will have disks and knowledge of datastore paths/location?
+ // throw new Exception(msg);
+ }
+
+ // FR37 - this may need checking, if the first datastore is the right one - ideally it should be datastore where first disk is hosted
+ DatastoreMO dsRootVolumeIsOn = null; //
+ if (! installAsIs) {
+ dsRootVolumeIsOn = getDatastoreThatRootDiskIsOn(dataStoresDetails, disks);
+
+ if (dsRootVolumeIsOn == null) {
+ String msg = "Unable to locate datastore details of root volume";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ throw new VmwareResourceException(msg);
+ }
+ }
+
+ VirtualMachineDiskInfoBuilder diskInfoBuilder = null;
+ VirtualDevice[] nicDevices = null;
+ DiskControllerType systemVmScsiControllerType = DiskControllerType.lsilogic;
+ int firstScsiControllerBusNum = 0;
+ int numScsiControllerForSystemVm = 1;
+ boolean hasSnapshot = false;
+ if (vmMo != null) {
+ PrepareRunningVMForConfiguration prepareRunningVMForConfiguration = new PrepareRunningVMForConfiguration(vmInternalCSName, controllerInfo, systemVm, vmMo,
+ systemVmScsiControllerType, firstScsiControllerBusNum, numScsiControllerForSystemVm);
+ prepareRunningVMForConfiguration.invoke();
+ diskInfoBuilder = prepareRunningVMForConfiguration.getDiskInfoBuilder();
+ nicDevices = prepareRunningVMForConfiguration.getNicDevices();
+ hasSnapshot = prepareRunningVMForConfiguration.isHasSnapshot();
+ } else {
+ ManagedObjectReference morDc = hyperHost.getHyperHostDatacenter();
+ assert (morDc != null);
+
+ vmMo = hyperHost.findVmOnPeerHyperHost(vmInternalCSName);
+ if (vmMo != null) {
+ VirtualMachineRecycler virtualMachineRecycler = new VirtualMachineRecycler(vmInternalCSName, controllerInfo, systemVm, hyperHost, vmMo,
+ systemVmScsiControllerType, firstScsiControllerBusNum, numScsiControllerForSystemVm);
+ virtualMachineRecycler.invoke();
+ diskInfoBuilder = virtualMachineRecycler.getDiskInfoBuilder();
+ hasSnapshot = virtualMachineRecycler.isHasSnapshot();
+ nicDevices = virtualMachineRecycler.getNicDevices();
+ } else {
+ // FR37 we just didn't find a VM by name 'vmInternalCSName', so why is this here?
+ existingVm = unregisterOnOtherClusterButHoldOnToOldVmData(vmInternalCSName, dcMo);
+
+ createNewVm(context, vmSpec, installAsIs, vmInternalCSName, vmNameOnVcenter, controllerInfo, systemVm, mgr, hyperHost, guestOsId, disks, dataStoresDetails,
+ dsRootVolumeIsOn);
+ }
+
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ if (vmMo == null) {
+ throw new Exception("Failed to find the newly create or relocated VM. vmName: " + vmInternalCSName);
+ }
+ }
+ // vmMo should now be a stopped VM on the intended host
+ // The number of disks changed must be 0 for install as is, as the VM is a clone
+ int disksChanges = !installAsIs ? disks.length : 0;
+ int totalChangeDevices = disksChanges + nics.length;
+ int hackDeviceCount = 0;
+ if (diskInfoBuilder != null) {
+ hackDeviceCount += diskInfoBuilder.getDiskCount();
+ }
+ hackDeviceCount += nicDevices == null ? 0 : nicDevices.length;
+ // vApp cdrom device
+ // HACK ALERT: ovf properties might not be the only or defining feature of vApps; needs checking
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("current count(s) desired: %d/ found:%d. now adding device to device count for vApp config ISO", totalChangeDevices, hackDeviceCount));
+ }
+ if (vmSpec.getOvfProperties() != null) {
+ totalChangeDevices++;
+ }
+
+ DiskTO volIso = null;
+ if (vmSpec.getType() != VirtualMachine.Type.User) {
+ // system VM needs a patch ISO
+ totalChangeDevices++;
+ } else {
+ volIso = getIsoDiskTO(disks);
+ if (volIso == null)
+ totalChangeDevices++;
+ }
+
+ VirtualMachineConfigSpec vmConfigSpec = new VirtualMachineConfigSpec();
+
+ VmwareHelper.setBasicVmConfig(vmConfigSpec, vmSpec.getCpus(), vmSpec.getMaxSpeed(), vmwareResource.getReservedCpuMHZ(vmSpec), (int)(vmSpec.getMaxRam() / (1024 * 1024)),
+ vmwareResource.getReservedMemoryMb(vmSpec), guestOsId, vmSpec.getLimitCpuUse());
+
+ int numCoresPerSocket = adjustNumberOfCoresPerSocket(vmSpec, vmMo, vmConfigSpec);
+
+ // Check for hotadd settings
+ vmConfigSpec.setMemoryHotAddEnabled(vmMo.isMemoryHotAddSupported(guestOsId));
+
+ String hostApiVersion = ((HostMO)hyperHost).getHostAboutInfo().getApiVersion();
+ if (numCoresPerSocket > 1 && hostApiVersion.compareTo("5.0") < 0) {
+ LOGGER.warn("Dynamic scaling of CPU is not supported for Virtual Machines with multi-core vCPUs in case of ESXi hosts 4.1 and prior. Hence CpuHotAdd will not be"
+ + " enabled for Virtual Machine: " + vmInternalCSName);
+ vmConfigSpec.setCpuHotAddEnabled(false);
+ } else {
+ vmConfigSpec.setCpuHotAddEnabled(vmMo.isCpuHotAddSupported(guestOsId));
+ }
+
+ vmwareResource.configNestedHVSupport(vmMo, vmSpec, vmConfigSpec);
+
+ VirtualDeviceConfigSpec[] deviceConfigSpecArray = new VirtualDeviceConfigSpec[totalChangeDevices];
+ int deviceCount = 0;
+ int ideUnitNumber = 0;
+ int scsiUnitNumber = 0;
+ int ideControllerKey = vmMo.getIDEDeviceControllerKey();
+ int scsiControllerKey = vmMo.getScsiDeviceControllerKeyNoException();
+
+ IsoSetup isoSetup = new IsoSetup(vmSpec, mgr, hyperHost, vmMo, disks, volIso, deviceConfigSpecArray, deviceCount, ideUnitNumber);
+ isoSetup.invoke();
+ deviceCount = isoSetup.getDeviceCount();
+ ideUnitNumber = isoSetup.getIdeUnitNumber();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("setting up %d disks with root from %s", diskInfoBuilder.getDiskCount(), vmwareResource.getGson().toJson(rootDiskTO)));
+ }
+
+ DiskSetup diskSetup = new DiskSetup(vmSpec, rootDiskTO, controllerInfo, context, dcMo, hyperHost, vmMo, disks, dataStoresDetails, diskInfoBuilder, hasSnapshot,
+ deviceConfigSpecArray, deviceCount, ideUnitNumber, scsiUnitNumber, ideControllerKey, scsiControllerKey, installAsIs);
+ diskSetup.invoke();
+
+ rootDiskTO = diskSetup.getRootDiskTO();
+ deviceCount = diskSetup.getDeviceCount();
+ DiskTO[] sortedDisks = diskSetup.getSortedDisks();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("seting up %d disks with root from %s", sortedDisks.length, vmwareResource.getGson().toJson(rootDiskTO)));
+ }
+
+ deviceCount += setupUsbDevicesAndGetCount(vmInternalCSName, vmMo, guestOsId, deviceConfigSpecArray, deviceCount);
+
+ NicSetup nicSetup = new NicSetup(cmd, vmSpec, vmInternalCSName, context, mgr, hyperHost, vmMo, nics, deviceConfigSpecArray, deviceCount, nicDevices);
+ nicSetup.invoke();
+
+ deviceCount = nicSetup.getDeviceCount();
+ int nicMask = nicSetup.getNicMask();
+ int nicCount = nicSetup.getNicCount();
+ Map<String, String> nicUuidToDvSwitchUuid = nicSetup.getNicUuidToDvSwitchUuid();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("device count: %d; nic count %d", deviceCount, nicCount));
+ }
+ for (int j = 0; j < deviceCount; j++)
+ vmConfigSpec.getDeviceChange().add(deviceConfigSpecArray[j]);
+
+ //
+ // Setup VM options
+ //
+
+ // pass boot arguments through machine.id & perform customized options to VMX
+ ArrayList<OptionValue> extraOptions = new ArrayList<>();
+ configBasicExtraOption(extraOptions, vmSpec);
+ configNvpExtraOption(extraOptions, vmSpec, nicUuidToDvSwitchUuid);
+ configCustomExtraOption(extraOptions, vmSpec);
+
+ // config for NCC
+ VirtualMachine.Type vmType = cmd.getVirtualMachine().getType();
+ if (vmType.equals(VirtualMachine.Type.NetScalerVm)) {
+ NicTO mgmtNic = vmSpec.getNics()[0];
+ OptionValue option = new OptionValue();
+ option.setKey("machine.id");
+ option.setValue("ip=" + mgmtNic.getIp() + "&netmask=" + mgmtNic.getNetmask() + "&gateway=" + mgmtNic.getGateway());
+ extraOptions.add(option);
+ }
+
+ // config VNC
+ String keyboardLayout = null;
+ if (vmSpec.getDetails() != null)
+ keyboardLayout = vmSpec.getDetails().get(VmDetailConstants.KEYBOARD);
+ vmConfigSpec.getExtraConfig().addAll(
+ Arrays.asList(vmwareResource.configureVnc(extraOptions.toArray(new OptionValue[0]), hyperHost, vmInternalCSName, vmSpec.getVncPassword(), keyboardLayout)));
+
+ // config video card
+ vmwareResource.configureVideoCard(vmMo, vmSpec, vmConfigSpec);
+
+ // Set OVF properties (if available)
+ Pair<String, List<OVFPropertyTO>> ovfPropsMap = vmSpec.getOvfProperties();
+ VmConfigInfo templateVappConfig;
+ List<OVFPropertyTO> ovfProperties;
+ if (ovfPropsMap != null) {
+ String vmTemplate = ovfPropsMap.first();
+ LOGGER.info("Find VM template " + vmTemplate);
+ VirtualMachineMO vmTemplateMO = dcMo.findVm(vmTemplate);
+ templateVappConfig = vmTemplateMO.getConfigInfo().getVAppConfig();
+ ovfProperties = ovfPropsMap.second();
+ // Set OVF properties (if available)
+ if (CollectionUtils.isNotEmpty(ovfProperties)) {
+ LOGGER.info("Copying OVF properties from template and setting them to the values the user provided");
+ vmwareResource.copyVAppConfigsFromTemplate(templateVappConfig, ovfProperties, vmConfigSpec);
+ }
+ }
+
+ checkBootOptions(vmSpec, vmConfigSpec);
+
+ //
+ // Configure VM
+ //
+ if (!vmMo.configureVm(vmConfigSpec)) {
+ throw new Exception("Failed to configure VM before start. vmName: " + vmInternalCSName);
+ }
+ // FR37 TODO reconcile disks now!!! they are configured, check and add them to the returning vmTO
+ if (vmSpec.getType() == VirtualMachine.Type.DomainRouter) {
+ hyperHost.setRestartPriorityForVM(vmMo, DasVmPriority.HIGH.value());
+ }
+
+ // Resizing root disk only when explicit requested by user
+ final Map<String, String> vmDetails = cmd.getVirtualMachine().getDetails();
+ if (rootDiskTO != null && !hasSnapshot && (vmDetails != null && vmDetails.containsKey(ApiConstants.ROOT_DISK_SIZE))) {
+ resizeRootDiskOnVMStart(vmMo, rootDiskTO, hyperHost, context);
+ }
+
+ //
+ // Post Configuration
+ //
+
+ vmMo.setCustomFieldValue(CustomFieldConstants.CLOUD_NIC_MASK, String.valueOf(nicMask));
+ postNvpConfigBeforeStart(vmMo, vmSpec);
+
+ Map<String, Map<String, String>> iqnToData = new HashMap<>();
+
+ postDiskConfigBeforeStart(vmMo, vmSpec, sortedDisks, iqnToData, hyperHost, context, installAsIs);
+
+ //
+ // Power-on VM
+ //
+ if (!vmMo.powerOn()) {
+ throw new Exception("Failed to start VM. vmName: " + vmInternalCSName + " with hostname " + vmNameOnVcenter);
+ }
+
+ if (installAsIs) {
+ // Set disks as the disks path and chain is retrieved from the cloned VM disks
+ cmd.getVirtualMachine().setDisks(disks);
+ }
+
+ StartAnswer startAnswer = new StartAnswer(cmd);
+
+ startAnswer.setIqnToData(iqnToData);
+
+ deleteOldVersionOfTheStartedVM(existingVm, dcMo, vmMo);
+
+ return startAnswer;
+ } catch (Throwable e) {
+ return handleStartFailure(cmd, vmSpec, existingVm, dcMo, e);
+ } finally {
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("finally done with %s", vmwareResource.getGson().toJson(cmd)));
+ }
+ }
+ }
+
+ /**
+ * Modify the specDisks information to match the cloned VM's disks (from vmMo VM)
+ */
+ private void mapSpecDisksToClonedDisks(VirtualMachineMO vmMo, String vmInternalCSName, DiskTO[] specDisks) {
+ try {
+ if (vmMo != null && ArrayUtils.isNotEmpty(specDisks)) {
+ List<VirtualDisk> vmDisks = vmMo.getVirtualDisks();
+ List<DiskTO> sortedDisks = Arrays.asList(sortVolumesByDeviceId(specDisks))
+ .stream()
+ .filter(x -> x.getType() == Volume.Type.ROOT)
+ .collect(Collectors.toList());
+ if (sortedDisks.size() != vmDisks.size()) {
+ LOGGER.error("Different number of root disks spec vs cloned deploy-as-is VM disks: " + sortedDisks.size() + " - " + vmDisks.size());
+ return;
+ }
+ for (int i = 0; i < sortedDisks.size(); i++) {
+ DiskTO specDisk = sortedDisks.get(i);
+ VirtualDisk vmDisk = vmDisks.get(i);
+ DataTO dataVolume = specDisk.getData();
+ if (dataVolume instanceof VolumeObjectTO) {
+ VolumeObjectTO volumeObjectTO = (VolumeObjectTO) dataVolume;
+ if (!volumeObjectTO.getSize().equals(vmDisk.getCapacityInBytes())) {
+ LOGGER.info("Mapped disk size is not the same as the cloned VM disk size: " +
+ volumeObjectTO.getSize() + " - " + vmDisk.getCapacityInBytes());
+ }
+ VirtualDeviceBackingInfo backingInfo = vmDisk.getBacking();
+ if (backingInfo instanceof VirtualDiskFlatVer2BackingInfo) {
+ VirtualDiskFlatVer2BackingInfo backing = (VirtualDiskFlatVer2BackingInfo) backingInfo;
+ String fileName = backing.getFileName();
+ if (StringUtils.isNotBlank(fileName)) {
+ String[] fileNameParts = fileName.split(" ");
+ String datastoreUuid = fileNameParts[0].replace("[", "").replace("]", "");
+ String relativePath = fileNameParts[1].split("/")[1].replace(".vmdk", "");
+ String vmSpecDatastoreUuid = volumeObjectTO.getDataStore().getUuid().replaceAll("-", "");
+ if (!datastoreUuid.equals(vmSpecDatastoreUuid)) {
+ LOGGER.info("Mapped disk datastore UUID is not the same as the cloned VM datastore UUID: " +
+ datastoreUuid + " - " + vmSpecDatastoreUuid);
+ }
+ volumeObjectTO.setPath(relativePath);
+ specDisk.setPath(relativePath);
+ }
+ }
+ }
+ }
+ }
+ } catch (Exception e) {
+ String msg = "Error mapping deploy-as-is VM disks from cloned VM " + vmInternalCSName;
+ LOGGER.error(msg, e);
+ throw new CloudRuntimeException(e);
+ }
+ }
+
+ private StartAnswer handleStartFailure(StartCommand cmd, VirtualMachineTO vmSpec, VirtualMachineData existingVm, DatacenterMO dcMo, Throwable e) {
+ if (e instanceof RemoteException) {
+ LOGGER.warn("Encounter remote exception to vCenter, invalidate VMware session context");
+ vmwareResource.invalidateServiceContext();
+ }
+
+ String msg = "StartCommand failed due to " + VmwareHelper.getExceptionMessage(e);
+ LOGGER.warn(msg, e);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("didn't start VM %s", vmSpec.getName()), e);
+ }
+ StartAnswer startAnswer = new StartAnswer(cmd, msg);
+ if ( e instanceof VmAlreadyExistsInVcenter) {
+ startAnswer.setContextParam("stopRetry", "true");
+ }
+ reRegisterExistingVm(existingVm, dcMo);
+
+ return startAnswer;
+ }
+
+ private void deleteOldVersionOfTheStartedVM(VirtualMachineData existingVm, DatacenterMO dcMo, VirtualMachineMO vmMo) throws Exception {
+ // Since VM was successfully powered-on, if there was an existing VM in a different cluster that was unregistered, delete all the files associated with it.
+ if (existingVm != null && existingVm.vmName != null && existingVm.vmFileLayout != null) {
+ List<String> vmDatastoreNames = new ArrayList<>();
+ for (DatastoreMO vmDatastore : vmMo.getAllDatastores()) {
+ vmDatastoreNames.add(vmDatastore.getName());
+ }
+ // Don't delete files that are in a datastore that is being used by the new VM as well (zone-wide datastore).
+ List<String> skipDatastores = new ArrayList<>();
+ for (DatastoreMO existingDatastore : existingVm.datastores) {
+ if (vmDatastoreNames.contains(existingDatastore.getName())) {
+ skipDatastores.add(existingDatastore.getName());
+ }
+ }
+ vmwareResource.deleteUnregisteredVmFiles(existingVm.vmFileLayout, dcMo, true, skipDatastores);
+ }
+ }
+
+ private int adjustNumberOfCoresPerSocket(VirtualMachineTO vmSpec, VirtualMachineMO vmMo, VirtualMachineConfigSpec vmConfigSpec) throws Exception {
+ // Check for multi-cores per socket settings
+ int numCoresPerSocket = 1;
+ String coresPerSocket = vmSpec.getDetails().get(VmDetailConstants.CPU_CORE_PER_SOCKET);
+ if (coresPerSocket != null) {
+ String apiVersion = HypervisorHostHelper.getVcenterApiVersion(vmMo.getContext());
+ // Property 'numCoresPerSocket' is supported since vSphere API 5.0
+ if (apiVersion.compareTo("5.0") >= 0) {
+ numCoresPerSocket = NumbersUtil.parseInt(coresPerSocket, 1);
+ vmConfigSpec.setNumCoresPerSocket(numCoresPerSocket);
+ }
+ }
+ return numCoresPerSocket;
+ }
+
+ /**
+ // Setup USB devices
+ */
+ private int setupUsbDevicesAndGetCount(String vmInternalCSName, VirtualMachineMO vmMo, String guestOsId, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount)
+ throws Exception {
+ int usbDeviceCount = 0;
+ if (guestOsId.startsWith("darwin")) { //Mac OS
+ VirtualDevice[] devices = vmMo.getMatchedDevices(new Class<?>[] {VirtualUSBController.class});
+ if (devices.length == 0) {
+ LOGGER.debug("No USB Controller device on VM Start. Add USB Controller device for Mac OS VM " + vmInternalCSName);
+
+ //For Mac OS X systems, the EHCI+UHCI controller is enabled by default and is required for USB mouse and keyboard access.
+ VirtualDevice usbControllerDevice = VmwareHelper.prepareUSBControllerDevice();
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ deviceConfigSpecArray[deviceCount].setDevice(usbControllerDevice);
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare USB controller at new device " + vmwareResource.getGson().toJson(deviceConfigSpecArray[deviceCount]));
+
+ deviceCount++;
+ usbDeviceCount++;
+ } else {
+ LOGGER.debug("USB Controller device exists on VM Start for Mac OS VM " + vmInternalCSName);
+ }
+ }
+ return usbDeviceCount;
+ }
+
+ private void createNewVm(VmwareContext context, VirtualMachineTO vmSpec, boolean installAsIs, String vmInternalCSName, String vmNameOnVcenter, Pair<String, String> controllerInfo, Boolean systemVm,
+ VmwareManager mgr, VmwareHypervisorHost hyperHost, String guestOsId, DiskTO[] disks, HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails,
+ DatastoreMO dsRootVolumeIsOn) throws Exception {
+ VirtualMachineMO vmMo;
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = getRootDiskDataStoreDetails(disks, dataStoresDetails);
+
+ assert (vmSpec.getMinSpeed() != null) && (rootDiskDataStoreDetails != null);
+
+ boolean vmFolderExists = rootDiskDataStoreDetails.second().folderExists(String.format("[%s]", rootDiskDataStoreDetails.second().getName()), vmNameOnVcenter);
+ String vmxFileFullPath = dsRootVolumeIsOn.searchFileInSubFolders(vmNameOnVcenter + ".vmx", false, VmwareManager.s_vmwareSearchExcludeFolder.value());
+ if (vmFolderExists && vmxFileFullPath != null) { // VM can be registered only if .vmx is present.
+ registerVm(vmNameOnVcenter, dsRootVolumeIsOn, vmwareResource);
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ if (vmMo != null) {
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Found registered vm " + vmInternalCSName + " at host " + hyperHost.getHyperHostName());
+ }
+ }
+ tearDownVm(vmMo);
+ } else if (installAsIs) {
+// first get all the MORs ManagedObjectReference morPool = hyperHost.getHyperHostOwnerResourcePool();
+// get the base VM vmMo = hyperHost.findVmOnHyperHost(vm.template.getPath());
+// do templateVm.createLinkedClone(vmInternalCSName, morBaseSnapshot, dcMo.getVmFolder(), morPool, morDatastore)
+// or hyperHost....createLinkedOrFullClone(templateVm, volume, dcMo, vmMo, morDatastore, dsMo, vmInternalCSName, morPool);
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ // At this point vmMo points to the cloned VM
+ } else {
+ if (!hyperHost
+ .createBlankVm(vmNameOnVcenter, vmInternalCSName, vmSpec.getCpus(), vmSpec.getMaxSpeed(), vmwareResource.getReservedCpuMHZ(vmSpec), vmSpec.getLimitCpuUse(), (int)(vmSpec.getMaxRam() / Resource.ResourceType.bytesToMiB), vmwareResource.getReservedMemoryMb(vmSpec), guestOsId,
+ rootDiskDataStoreDetails.first(), false, controllerInfo, systemVm)) {
+ throw new Exception("Failed to create VM. vmName: " + vmInternalCSName);
+ }
+ }
+ }
+
+ /**
+ * If a VM with the same name is found in a different cluster in the DC, unregister the old VM and configure a new VM (cold-migration).
+ */
+ private VirtualMachineData unregisterOnOtherClusterButHoldOnToOldVmData(String vmInternalCSName, DatacenterMO dcMo) throws Exception {
+ VirtualMachineMO existingVmInDc = dcMo.findVm(vmInternalCSName);
+ VirtualMachineData existingVm = null;
+ if (existingVmInDc != null) {
+ existingVm = new VirtualMachineData();
+ existingVm.vmName = existingVmInDc.getName();
+ existingVm.vmFileInfo = existingVmInDc.getFileInfo();
+ existingVm.vmFileLayout = existingVmInDc.getFileLayout();
+ existingVm.datastores = existingVmInDc.getAllDatastores();
+ LOGGER.info("Found VM: " + vmInternalCSName + " on a host in a different cluster. Unregistering the exisitng VM.");
+ existingVmInDc.unregisterVm();
+ }
+ return existingVm;
+ }
+
+ private Pair<ManagedObjectReference, DatastoreMO> getRootDiskDataStoreDetails(DiskTO[] disks, HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails) {
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = null;
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ROOT) {
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (managed) {
+ String datastoreName = VmwareResource.getDatastoreName(details.get(DiskTO.IQN));
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(datastoreName);
+ } else {
+ DataStoreTO primaryStore = vol.getData().getDataStore();
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(primaryStore.getUuid());
+ }
+ }
+ }
+ return rootDiskDataStoreDetails;
+ }
+
+ /**
+ * Since VM start failed, if there was an existing VM in a different cluster that was unregistered, register it back.
+ *
+ * @param dcMo is guaranteed to be not null since we have noticed there is an existing VM in the dc (using that mo)
+ */
+ private void reRegisterExistingVm(VirtualMachineData existingVm, DatacenterMO dcMo) {
+ if (existingVm != null && existingVm.vmName != null && existingVm.vmFileInfo != null) {
+ LOGGER.debug("Since VM start failed, registering back an existing VM: " + existingVm.vmName + " that was unregistered");
+ try {
+ DatastoreFile fileInDatastore = new DatastoreFile(existingVm.vmFileInfo.getVmPathName());
+ DatastoreMO existingVmDsMo = new DatastoreMO(dcMo.getContext(), dcMo.findDatastore(fileInDatastore.getDatastoreName()));
+ registerVm(existingVm.vmName, existingVmDsMo, vmwareResource);
+ } catch (Exception ex) {
+ String message = "Failed to register an existing VM: " + existingVm.vmName + " due to " + VmwareHelper.getExceptionMessage(ex);
+ LOGGER.warn(message, ex);
+ }
+ }
+ }
+
+ private void checkIfVmExistsInVcenter(String vmInternalCSName, String vmNameOnVcenter, DatacenterMO dcMo) throws VmAlreadyExistsInVcenter, Exception {
+ // Validate VM name is unique in Datacenter
+ VirtualMachineMO vmInVcenter = dcMo.checkIfVmAlreadyExistsInVcenter(vmNameOnVcenter, vmInternalCSName);
+ if (vmInVcenter != null) {
+ String msg = "VM with name: " + vmNameOnVcenter + " already exists in vCenter.";
+ LOGGER.error(msg);
+ throw new VmAlreadyExistsInVcenter(msg);
+ }
+ }
+
+ private Pair<String, String> getDiskControllerInfo(VirtualMachineTO vmSpec) {
+ String dataDiskController = vmSpec.getDetails().get(VmDetailConstants.DATA_DISK_CONTROLLER);
+ String rootDiskController = vmSpec.getDetails().get(VmDetailConstants.ROOT_DISK_CONTROLLER);
+ // If root disk controller is scsi, then data disk controller would also be scsi instead of using 'osdefault'
+ // This helps avoid mix of different scsi subtype controllers in instance.
+ if (DiskControllerType.osdefault == DiskControllerType.getType(dataDiskController) && DiskControllerType.lsilogic == DiskControllerType.getType(rootDiskController)) {
+ dataDiskController = DiskControllerType.scsi.toString();
+ }
+
+ // Validate the controller types
+ dataDiskController = DiskControllerType.getType(dataDiskController).toString();
+ rootDiskController = DiskControllerType.getType(rootDiskController).toString();
+
+ if (DiskControllerType.getType(rootDiskController) == DiskControllerType.none) {
+ throw new CloudRuntimeException("Invalid root disk controller detected : " + rootDiskController);
+ }
+ if (DiskControllerType.getType(dataDiskController) == DiskControllerType.none) {
+ throw new CloudRuntimeException("Invalid data disk controller detected : " + dataDiskController);
+ }
+
+ return new Pair<>(rootDiskController, dataDiskController);
+ }
+
+ /**
+ * Registers the vm to the inventory given the vmx file.
+ * @param vmName
+ * @param dsMo
+ * @param vmwareResource
+ */
+ private void registerVm(String vmName, DatastoreMO dsMo, VmwareResource vmwareResource) throws Exception {
+
+ //1st param
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(vmwareResource.getServiceContext());
+ ManagedObjectReference dcMor = hyperHost.getHyperHostDatacenter();
+ DatacenterMO dataCenterMo = new DatacenterMO(vmwareResource.getServiceContext(), dcMor);
+ ManagedObjectReference vmFolderMor = dataCenterMo.getVmFolder();
+
+ //2nd param
+ String vmxFilePath = dsMo.searchFileInSubFolders(vmName + ".vmx", false, VmwareManager.s_vmwareSearchExcludeFolder.value());
+
+ // 5th param
+ ManagedObjectReference morPool = hyperHost.getHyperHostOwnerResourcePool();
+
+ ManagedObjectReference morTask = vmwareResource.getServiceContext().getService().registerVMTask(vmFolderMor, vmxFilePath, vmName, false, morPool, hyperHost.getMor());
+ boolean result = vmwareResource.getServiceContext().getVimClient().waitForTask(morTask);
+ if (!result) {
+ throw new Exception("Unable to register vm due to " + TaskMO.getTaskFailureInfo(vmwareResource.getServiceContext(), morTask));
+ } else {
+ vmwareResource.getServiceContext().waitForTaskProgressDone(morTask);
+ }
+
+ }
+
+ // Pair<internal CS name, vCenter display name>
+ private Pair<String, String> composeVmNames(VirtualMachineTO vmSpec) {
+ String vmInternalCSName = vmSpec.getName();
+ String vmNameOnVcenter = vmSpec.getName();
+ if (VmwareResource.instanceNameFlag && vmSpec.getHostName() != null) {
+ vmNameOnVcenter = vmSpec.getHostName();
+ }
+ return new Pair<String, String>(vmInternalCSName, vmNameOnVcenter);
+ }
+
+ private VirtualMachineGuestOsIdentifier translateGuestOsIdentifier(String cpuArchitecture, String guestOs, String cloudGuestOs) {
+ if (cpuArchitecture == null) {
+ LOGGER.warn("CPU arch is not set, default to i386. guest os: " + guestOs);
+ cpuArchitecture = "i386";
+ }
+
+ if (cloudGuestOs == null) {
+ LOGGER.warn("Guest OS mapping name is not set for guest os: " + guestOs);
+ }
+
+ VirtualMachineGuestOsIdentifier identifier = null;
+ try {
+ if (cloudGuestOs != null) {
+ identifier = VirtualMachineGuestOsIdentifier.fromValue(cloudGuestOs);
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Using mapping name : " + identifier.toString());
+ }
+ }
+ } catch (IllegalArgumentException e) {
+ LOGGER.warn("Unable to find Guest OS Identifier in VMware for mapping name: " + cloudGuestOs + ". Continuing with defaults.");
+ }
+ if (identifier != null) {
+ return identifier;
+ }
+
+ if (cpuArchitecture.equalsIgnoreCase("x86_64")) {
+ return VirtualMachineGuestOsIdentifier.OTHER_GUEST_64;
+ }
+ return VirtualMachineGuestOsIdentifier.OTHER_GUEST;
+ }
+
+ private DiskTO[] validateDisks(DiskTO[] disks) {
+ List<DiskTO> validatedDisks = new ArrayList<DiskTO>();
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() != Volume.Type.ISO) {
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ if (primaryStore.getUuid() != null && !primaryStore.getUuid().isEmpty()) {
+ validatedDisks.add(vol);
+ }
+ } else if (vol.getType() == Volume.Type.ISO) {
+ TemplateObjectTO templateTO = (TemplateObjectTO) vol.getData();
+ if (templateTO.getPath() != null && !templateTO.getPath().isEmpty()) {
+ validatedDisks.add(vol);
+ }
+ } else {
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Drop invalid disk option, volumeTO: " + vmwareResource.getGson().toJson(vol));
+ }
+ }
+ }
+ Collections.sort(validatedDisks, (d1, d2) -> d1.getDiskSeq().compareTo(d2.getDiskSeq()));
+ return validatedDisks.toArray(new DiskTO[0]);
+ }
+
+ private HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> inferDatastoreDetailsFromDiskInfo(VmwareHypervisorHost hyperHost, VmwareContext context,
+ DiskTO[] disks, Command cmd) throws Exception {
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> mapIdToMors = new HashMap<>();
+
+ assert (hyperHost != null) && (context != null);
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() != Volume.Type.ISO) {
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ String poolUuid = primaryStore.getUuid();
+
+ if (mapIdToMors.get(poolUuid) == null) {
+ boolean isManaged = false;
+ Map<String, String> details = vol.getDetails();
+
+ if (details != null) {
+ isManaged = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (isManaged) {
+ String iScsiName = details.get(DiskTO.IQN); // details should not be null for managed storage (it may or may not be null for non-managed storage)
+ String datastoreName = VmwareResource.getDatastoreName(iScsiName);
+ ManagedObjectReference morDatastore = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, datastoreName);
+
+ // if the datastore is not present, we need to discover the iSCSI device that will support it,
+ // create the datastore, and create a VMDK file in the datastore
+ if (morDatastore == null) {
+ final String vmdkPath = vmwareResource.getVmdkPath(volumeTO.getPath());
+
+ morDatastore = getStorageProcessor().prepareManagedStorage(context, hyperHost, null, iScsiName,
+ details.get(DiskTO.STORAGE_HOST), Integer.parseInt(details.get(DiskTO.STORAGE_PORT)),
+ vmdkPath,
+ details.get(DiskTO.CHAP_INITIATOR_USERNAME), details.get(DiskTO.CHAP_INITIATOR_SECRET),
+ details.get(DiskTO.CHAP_TARGET_USERNAME), details.get(DiskTO.CHAP_TARGET_SECRET),
+ Long.parseLong(details.get(DiskTO.VOLUME_SIZE)), cmd);
+
+ DatastoreMO dsMo = new DatastoreMO(vmwareResource.getServiceContext(), morDatastore);
+
+ final String datastoreVolumePath;
+
+ if (vmdkPath != null) {
+ datastoreVolumePath = dsMo.getDatastorePath(vmdkPath + VmwareResource.VMDK_EXTENSION);
+ } else {
+ datastoreVolumePath = dsMo.getDatastorePath(dsMo.getName() + VmwareResource.VMDK_EXTENSION);
+ }
+
+ volumeTO.setPath(datastoreVolumePath);
+ vol.setPath(datastoreVolumePath);
+ }
+
+ mapIdToMors.put(datastoreName, new Pair<>(morDatastore, new DatastoreMO(context, morDatastore)));
+ } else {
+ ManagedObjectReference morDatastore = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, poolUuid);
+
+ if (morDatastore == null) {
+ String msg = "Failed to get the mounted datastore for the volume's pool " + poolUuid;
+
+ LOGGER.error(msg);
+
+ throw new Exception(msg);
+ }
+
+ mapIdToMors.put(poolUuid, new Pair<>(morDatastore, new DatastoreMO(context, morDatastore)));
+ }
+ }
+ }
+ }
+
+ return mapIdToMors;
+ }
+
+ private VmwareStorageProcessor getStorageProcessor() {
+ return vmwareResource.getStorageProcessor();
+ }
+
+ private DatastoreMO getDatastoreThatRootDiskIsOn(HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails, DiskTO disks[]) {
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = null;
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ROOT) {
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (managed) {
+ String datastoreName = VmwareResource.getDatastoreName(details.get(DiskTO.IQN));
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(datastoreName);
+
+ break;
+ } else {
+ DataStoreTO primaryStore = vol.getData().getDataStore();
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(primaryStore.getUuid());
+
+ break;
+ }
+ }
+ }
+
+ if (rootDiskDataStoreDetails != null) {
+ return rootDiskDataStoreDetails.second();
+ }
+
+ return null;
+ }
+
+ private void ensureDiskControllers(VirtualMachineMO vmMo, Pair<String, String> controllerInfo) throws Exception {
+ if (vmMo == null) {
+ return;
+ }
+
+ String msg;
+ String rootDiskController = controllerInfo.first();
+ String dataDiskController = controllerInfo.second();
+ String scsiDiskController;
+ String recommendedDiskController = null;
+
+ if (VmwareHelper.isControllerOsRecommended(dataDiskController) || VmwareHelper.isControllerOsRecommended(rootDiskController)) {
+ recommendedDiskController = vmMo.getRecommendedDiskController(null);
+ }
+ scsiDiskController = HypervisorHostHelper.getScsiController(new Pair<String, String>(rootDiskController, dataDiskController), recommendedDiskController);
+ if (scsiDiskController == null) {
+ return;
+ }
+
+ vmMo.getScsiDeviceControllerKeyNoException();
+ // This VM needs SCSI controllers.
+ // Get count of existing scsi controllers. Helps not to attempt to create more than the maximum allowed 4
+ // Get maximum among the bus numbers in use by scsi controllers. Safe to pick maximum, because we always go sequential allocating bus numbers.
+ Ternary<Integer, Integer, DiskControllerType> scsiControllerInfo = vmMo.getScsiControllerInfo();
+ int requiredNumScsiControllers = VmwareHelper.MAX_SCSI_CONTROLLER_COUNT - scsiControllerInfo.first();
+ int availableBusNum = scsiControllerInfo.second() + 1; // method returned current max. bus number
+
+ if (requiredNumScsiControllers == 0) {
+ return;
+ }
+ if (scsiControllerInfo.first() > 0) {
+ // For VMs which already have a SCSI controller, do NOT attempt to add any more SCSI controllers & return the sub type.
+ // For Legacy VMs would have only 1 LsiLogic Parallel SCSI controller, and doesn't require more.
+ // For VMs created post device ordering support, 4 SCSI subtype controllers are ensured during deployment itself. No need to add more.
+ // For fresh VM deployment only, all required controllers should be ensured.
+ return;
+ }
+ ensureScsiDiskControllers(vmMo, scsiDiskController, requiredNumScsiControllers, availableBusNum);
+ }
+
+ private void ensureScsiDiskControllers(VirtualMachineMO vmMo, String scsiDiskController, int requiredNumScsiControllers, int availableBusNum) throws Exception {
+ // Pick the sub type of scsi
+ if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.pvscsi) {
+ if (!vmMo.isPvScsiSupported()) {
+ String msg = "This VM doesn't support Vmware Paravirtual SCSI controller for virtual disks, because the virtual hardware version is less than 7.";
+ throw new Exception(msg);
+ }
+ vmMo.ensurePvScsiDeviceController(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.lsisas1068) {
+ vmMo.ensureLsiLogicSasDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.buslogic) {
+ vmMo.ensureBusLogicDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.lsilogic) {
+ vmMo.ensureLsiLogicDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ }
+ }
+
+ private void tearDownVm(VirtualMachineMO vmMo) throws Exception {
+
+ if (vmMo == null)
+ return;
+
+ boolean hasSnapshot = false;
+ hasSnapshot = vmMo.hasSnapshot();
+ if (!hasSnapshot)
+ vmMo.tearDownDevices(new Class<?>[]{VirtualDisk.class, VirtualEthernetCard.class});
+ else
+ vmMo.tearDownDevices(new Class<?>[]{VirtualEthernetCard.class});
+ vmMo.ensureScsiDeviceController();
+ }
+
+ private static DiskTO getIsoDiskTO(DiskTO[] disks) {
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ISO) {
+ return vol;
+ }
+ }
+ return null;
+ }
+
+ private static DiskTO[] sortVolumesByDeviceId(DiskTO[] volumes) {
+
+ List<DiskTO> listForSort = new ArrayList<DiskTO>();
+ for (DiskTO vol : volumes) {
+ listForSort.add(vol);
+ }
+ Collections.sort(listForSort, new Comparator<DiskTO>() {
+
+ @Override
+ public int compare(DiskTO arg0, DiskTO arg1) {
+ if (arg0.getDiskSeq() < arg1.getDiskSeq()) {
+ return -1;
+ } else if (arg0.getDiskSeq().equals(arg1.getDiskSeq())) {
+ return 0;
+ }
+
+ return 1;
+ }
+ });
+
+ return listForSort.toArray(new DiskTO[0]);
+ }
+
+ private void postDiskConfigBeforeStart(VirtualMachineMO vmMo, VirtualMachineTO vmSpec, DiskTO[] sortedDisks,
+ Map<String, Map<String, String>> iqnToData, VmwareHypervisorHost hyperHost, VmwareContext context, boolean installAsIs)
+ throws Exception {
+ VirtualMachineDiskInfoBuilder diskInfoBuilder = vmMo.getDiskInfoBuilder();
+
+ for (DiskTO vol : sortedDisks) {
+ //TODO: Map existing disks to the ones returned in the answer
+ if (vol.getType() == Volume.Type.ISO)
+ continue;
+
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+
+ VirtualMachineDiskInfo diskInfo = getMatchingExistingDisk(diskInfoBuilder, vol, hyperHost, context);
+ assert (diskInfo != null);
+
+ String[] diskChain = diskInfo.getDiskChain();
+ assert (diskChain.length > 0);
+
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ DatastoreFile file = new DatastoreFile(diskChain[0]);
+
+ if (managed) {
+ DatastoreFile originalFile = new DatastoreFile(volumeTO.getPath());
+
+ if (!file.getFileBaseName().equalsIgnoreCase(originalFile.getFileBaseName())) {
+ if (LOGGER.isInfoEnabled())
+ LOGGER.info("Detected disk-chain top file change on volume: " + volumeTO.getId() + " " + volumeTO.getPath() + " -> " + diskChain[0]);
+ }
+ } else {
+ if (!file.getFileBaseName().equalsIgnoreCase(volumeTO.getPath())) {
+ if (LOGGER.isInfoEnabled())
+ LOGGER.info("Detected disk-chain top file change on volume: " + volumeTO.getId() + " " + volumeTO.getPath() + " -> " + file.getFileBaseName());
+ }
+ }
+
+ VolumeObjectTO volInSpec = getVolumeInSpec(vmSpec, volumeTO);
+
+ if (volInSpec != null) {
+ if (managed) {
+ Map<String, String> data = new HashMap<>();
+
+ String datastoreVolumePath = diskChain[0];
+
+ data.put(StartAnswer.PATH, datastoreVolumePath);
+ data.put(StartAnswer.IMAGE_FORMAT, Storage.ImageFormat.OVA.toString());
+
+ iqnToData.put(details.get(DiskTO.IQN), data);
+
+ vol.setPath(datastoreVolumePath);
+ volumeTO.setPath(datastoreVolumePath);
+ volInSpec.setPath(datastoreVolumePath);
+ } else {
+ volInSpec.setPath(file.getFileBaseName());
+ }
+ volInSpec.setChainInfo(vmwareResource.getGson().toJson(diskInfo));
+ }
+ }
+ }
+
+ private void checkBootOptions(VirtualMachineTO vmSpec, VirtualMachineConfigSpec vmConfigSpec) {
+ String bootMode = null;
+ if (vmSpec.getDetails().containsKey(VmDetailConstants.BOOT_MODE)) {
+ bootMode = vmSpec.getDetails().get(VmDetailConstants.BOOT_MODE);
+ }
+ if (null == bootMode) {
+ bootMode = ApiConstants.BootType.BIOS.toString();
+ }
+
+ setBootOptions(vmSpec, bootMode, vmConfigSpec);
+ }
+
+ private void setBootOptions(VirtualMachineTO vmSpec, String bootMode, VirtualMachineConfigSpec vmConfigSpec) {
+ VirtualMachineBootOptions bootOptions = null;
+ if (StringUtils.isNotBlank(bootMode) && !bootMode.equalsIgnoreCase("bios")) {
+ vmConfigSpec.setFirmware("efi");
+ if (vmSpec.getDetails().containsKey(ApiConstants.BootType.UEFI.toString()) && "secure".equalsIgnoreCase(vmSpec.getDetails().get(ApiConstants.BootType.UEFI.toString()))) {
+ if (bootOptions == null) {
+ bootOptions = new VirtualMachineBootOptions();
+ }
+ bootOptions.setEfiSecureBootEnabled(true);
+ }
+ }
+ if (vmSpec.isEnterHardwareSetup()) {
+ if (bootOptions == null) {
+ bootOptions = new VirtualMachineBootOptions();
+ }
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("configuring VM '%s' to enter hardware setup",vmSpec.getName()));
+ }
+ bootOptions.setEnterBIOSSetup(vmSpec.isEnterHardwareSetup());
+ }
+ if (bootOptions != null) {
+ vmConfigSpec.setBootOptions(bootOptions);
+ }
+ }
+
+ private void resizeRootDiskOnVMStart(VirtualMachineMO vmMo, DiskTO rootDiskTO, VmwareHypervisorHost hyperHost, VmwareContext context) throws Exception {
+ final Pair<VirtualDisk, String> vdisk = vmwareResource.getVirtualDiskInfo(vmMo, VmwareResource.appendFileType(rootDiskTO.getPath(), VmwareResource.VMDK_EXTENSION));
+ assert (vdisk != null);
+
+ Long reqSize = 0L;
+ final VolumeObjectTO volumeTO = ((VolumeObjectTO) rootDiskTO.getData());
+ if (volumeTO != null) {
+ reqSize = volumeTO.getSize() / 1024;
+ }
+ final VirtualDisk disk = vdisk.first();
+ if (reqSize > disk.getCapacityInKB()) {
+ final VirtualMachineDiskInfo diskInfo = getMatchingExistingDisk(vmMo.getDiskInfoBuilder(), rootDiskTO, hyperHost, context);
+ assert (diskInfo != null);
+ final String[] diskChain = diskInfo.getDiskChain();
+
+ if (diskChain != null && diskChain.length > 1) {
+ LOGGER.warn("Disk chain length for the VM is greater than one, this is not supported");
+ throw new CloudRuntimeException("Unsupported VM disk chain length: " + diskChain.length);
+ }
+
+ boolean resizingSupported = false;
+ String deviceBusName = diskInfo.getDiskDeviceBusName();
+ if (deviceBusName != null && (deviceBusName.toLowerCase().contains("scsi") || deviceBusName.toLowerCase().contains("lsi"))) {
+ resizingSupported = true;
+ }
+ if (!resizingSupported) {
+ LOGGER.warn("Resizing of root disk is only support for scsi device/bus, the provide VM's disk device bus name is " + diskInfo.getDiskDeviceBusName());
+ throw new CloudRuntimeException("Unsupported VM root disk device bus: " + diskInfo.getDiskDeviceBusName());
+ }
+
+ disk.setCapacityInKB(reqSize);
+ VirtualMachineConfigSpec vmConfigSpec = new VirtualMachineConfigSpec();
+ VirtualDeviceConfigSpec deviceConfigSpec = new VirtualDeviceConfigSpec();
+ deviceConfigSpec.setDevice(disk);
+ deviceConfigSpec.setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ vmConfigSpec.getDeviceChange().add(deviceConfigSpec);
+ if (!vmMo.configureVm(vmConfigSpec)) {
+ throw new Exception("Failed to configure VM for given root disk size. vmName: " + vmMo.getName());
+ }
+ }
+ }
+
+ private static void postNvpConfigBeforeStart(VirtualMachineMO vmMo, VirtualMachineTO vmSpec) throws Exception {
+ /**
+ * We need to configure the port on the DV switch after the host is
+ * connected. So make this happen between the configure and start of
+ * the VM
+ */
+ int nicIndex = 0;
+ for (NicTO nicTo : sortNicsByDeviceId(vmSpec.getNics())) {
+ if (nicTo.getBroadcastType() == Networks.BroadcastDomainType.Lswitch) {
+ // We need to create a port with a unique vlan and pass the key to the nic device
+ LOGGER.trace("Nic " + nicTo.toString() + " is connected to an NVP logicalswitch");
+ VirtualDevice nicVirtualDevice = vmMo.getNicDeviceByIndex(nicIndex);
+ if (nicVirtualDevice == null) {
+ throw new Exception("Failed to find a VirtualDevice for nic " + nicIndex); //FIXME Generic exceptions are bad
+ }
+ VirtualDeviceBackingInfo backing = nicVirtualDevice.getBacking();
+ if (backing instanceof VirtualEthernetCardDistributedVirtualPortBackingInfo) {
+ // This NIC is connected to a Distributed Virtual Switch
+ VirtualEthernetCardDistributedVirtualPortBackingInfo portInfo = (VirtualEthernetCardDistributedVirtualPortBackingInfo) backing;
+ DistributedVirtualSwitchPortConnection port = portInfo.getPort();
+ String portKey = port.getPortKey();
+ String portGroupKey = port.getPortgroupKey();
+ String dvSwitchUuid = port.getSwitchUuid();
+
+ LOGGER.debug("NIC " + nicTo.toString() + " is connected to dvSwitch " + dvSwitchUuid + " pg " + portGroupKey + " port " + portKey);
+
+ ManagedObjectReference dvSwitchManager = vmMo.getContext().getVimClient().getServiceContent().getDvSwitchManager();
+ ManagedObjectReference dvSwitch = vmMo.getContext().getVimClient().getService().queryDvsByUuid(dvSwitchManager, dvSwitchUuid);
+
+ // Get all ports
+ DistributedVirtualSwitchPortCriteria criteria = new DistributedVirtualSwitchPortCriteria();
+ criteria.setInside(true);
+ criteria.getPortgroupKey().add(portGroupKey);
+ List<DistributedVirtualPort> dvPorts = vmMo.getContext().getVimClient().getService().fetchDVPorts(dvSwitch, criteria);
+
+ DistributedVirtualPort vmDvPort = null;
+ List<Integer> usedVlans = new ArrayList<Integer>();
+ for (DistributedVirtualPort dvPort : dvPorts) {
+ // Find the port for this NIC by portkey
+ if (portKey.equals(dvPort.getKey())) {
+ vmDvPort = dvPort;
+ }
+ VMwareDVSPortSetting settings = (VMwareDVSPortSetting) dvPort.getConfig().getSetting();
+ VmwareDistributedVirtualSwitchVlanIdSpec vlanId = (VmwareDistributedVirtualSwitchVlanIdSpec) settings.getVlan();
+ LOGGER.trace("Found port " + dvPort.getKey() + " with vlan " + vlanId.getVlanId());
+ if (vlanId.getVlanId() > 0 && vlanId.getVlanId() < 4095) {
+ usedVlans.add(vlanId.getVlanId());
+ }
+ }
+
+ if (vmDvPort == null) {
+ throw new Exception("Empty port list from dvSwitch for nic " + nicTo.toString());
+ }
+
+ DVPortConfigInfo dvPortConfigInfo = vmDvPort.getConfig();
+ VMwareDVSPortSetting settings = (VMwareDVSPortSetting) dvPortConfigInfo.getSetting();
+
+ VmwareDistributedVirtualSwitchVlanIdSpec vlanId = (VmwareDistributedVirtualSwitchVlanIdSpec) settings.getVlan();
+ BoolPolicy blocked = settings.getBlocked();
+ if (blocked.isValue() == Boolean.TRUE) {
+ LOGGER.trace("Port is blocked, set a vlanid and unblock");
+ DVPortConfigSpec dvPortConfigSpec = new DVPortConfigSpec();
+ VMwareDVSPortSetting edittedSettings = new VMwareDVSPortSetting();
+ // Unblock
+ blocked.setValue(Boolean.FALSE);
+ blocked.setInherited(Boolean.FALSE);
+ edittedSettings.setBlocked(blocked);
+ // Set vlan
+ int i;
+ for (i = 1; i < 4095; i++) {
+ if (!usedVlans.contains(i))
+ break;
+ }
+ vlanId.setVlanId(i); // FIXME should be a determined
+ // based on usage
+ vlanId.setInherited(false);
+ edittedSettings.setVlan(vlanId);
+
+ dvPortConfigSpec.setSetting(edittedSettings);
+ dvPortConfigSpec.setOperation("edit");
+ dvPortConfigSpec.setKey(portKey);
+ List<DVPortConfigSpec> dvPortConfigSpecs = new ArrayList<DVPortConfigSpec>();
+ dvPortConfigSpecs.add(dvPortConfigSpec);
+ ManagedObjectReference task = vmMo.getContext().getVimClient().getService().reconfigureDVPortTask(dvSwitch, dvPortConfigSpecs);
+ if (!vmMo.getContext().getVimClient().waitForTask(task)) {
+ throw new Exception("Failed to configure the dvSwitch port for nic " + nicTo.toString());
+ }
+ LOGGER.debug("NIC " + nicTo.toString() + " connected to vlan " + i);
+ } else {
+ LOGGER.trace("Port already configured and set to vlan " + vlanId.getVlanId());
+ }
+ } else if (backing instanceof VirtualEthernetCardNetworkBackingInfo) {
+ // This NIC is connected to a Virtual Switch
+ // Nothing to do
+ } else if (backing instanceof VirtualEthernetCardOpaqueNetworkBackingInfo) {
+ //if NSX API VERSION >= 4.2, connect to br-int (nsx.network), do not create portgroup else previous behaviour
+ //OK, connected to OpaqueNetwork
+ } else {
+ LOGGER.error("nic device backing is of type " + backing.getClass().getName());
+ throw new Exception("Incompatible backing for a VirtualDevice for nic " + nicIndex); //FIXME Generic exceptions are bad
+ }
+ }
+ nicIndex++;
+ }
+ }
+
+ private VirtualMachineDiskInfo getMatchingExistingDisk(VirtualMachineDiskInfoBuilder diskInfoBuilder, DiskTO vol, VmwareHypervisorHost hyperHost, VmwareContext context)
+ throws Exception {
+ if (diskInfoBuilder != null) {
+ VolumeObjectTO volume = (VolumeObjectTO) vol.getData();
+
+ String dsName = null;
+ String diskBackingFileBaseName = null;
+
+ Map<String, String> details = vol.getDetails();
+ boolean isManaged = details != null && Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+
+ if (isManaged) {
+ String iScsiName = details.get(DiskTO.IQN);
+
+ // if the storage is managed, iScsiName should not be null
+ dsName = VmwareResource.getDatastoreName(iScsiName);
+
+ diskBackingFileBaseName = new DatastoreFile(volume.getPath()).getFileBaseName();
+ } else {
+ ManagedObjectReference morDs = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, volume.getDataStore().getUuid());
+ DatastoreMO dsMo = new DatastoreMO(context, morDs);
+
+ dsName = dsMo.getName();
+
+ diskBackingFileBaseName = volume.getPath();
+ }
+
+ VirtualMachineDiskInfo diskInfo = diskInfoBuilder.getDiskInfoByBackingFileBaseName(diskBackingFileBaseName, dsName);
+ if (diskInfo != null) {
+ LOGGER.info("Found existing disk info from volume path: " + volume.getPath());
+ return diskInfo;
+ } else {
+ String chainInfo = volume.getChainInfo();
+ if (chainInfo != null) {
+ VirtualMachineDiskInfo infoInChain = vmwareResource.getGson().fromJson(chainInfo, VirtualMachineDiskInfo.class);
+ if (infoInChain != null) {
+ String[] disks = infoInChain.getDiskChain();
+ if (disks.length > 0) {
+ for (String diskPath : disks) {
+ DatastoreFile file = new DatastoreFile(diskPath);
+ diskInfo = diskInfoBuilder.getDiskInfoByBackingFileBaseName(file.getFileBaseName(), dsName);
+ if (diskInfo != null) {
+ LOGGER.info("Found existing disk from chain info: " + diskPath);
+ return diskInfo;
+ }
+ }
+ }
+
+ if (diskInfo == null) {
+ diskInfo = diskInfoBuilder.getDiskInfoByDeviceBusName(infoInChain.getDiskDeviceBusName());
+ if (diskInfo != null) {
+ LOGGER.info("Found existing disk from from chain device bus information: " + infoInChain.getDiskDeviceBusName());
+ return diskInfo;
+ }
+ }
+ }
+ }
+ }
+ }
+
+ return null;
+ }
+
+ private static VolumeObjectTO getVolumeInSpec(VirtualMachineTO vmSpec, VolumeObjectTO srcVol) {
+ for (DiskTO disk : vmSpec.getDisks()) {
+ if (disk.getData() instanceof VolumeObjectTO) {
+ VolumeObjectTO vol = (VolumeObjectTO) disk.getData();
+ if (vol.getId() == srcVol.getId())
+ return vol;
+ }
+ }
+
+ return null;
+ }
+
+ /**
+ * Generate the mac sequence from the nics.
+ */
+ protected String generateMacSequence(NicTO[] nics) {
+ if (nics.length == 0) {
+ return "";
+ }
+
+ StringBuffer sbMacSequence = new StringBuffer();
+ for (NicTO nicTo : sortNicsByDeviceId(nics)) {
+ sbMacSequence.append(nicTo.getMac()).append("|");
+ }
+ if (!sbMacSequence.toString().isEmpty()) {
+ sbMacSequence.deleteCharAt(sbMacSequence.length() - 1); //Remove extra '|' char appended at the end
+ }
+
+ return sbMacSequence.toString();
+ }
+
+ static NicTO[] sortNicsByDeviceId(NicTO[] nics) {
+
+ List<NicTO> listForSort = new ArrayList<NicTO>();
+ for (NicTO nic : nics) {
+ listForSort.add(nic);
+ }
+ Collections.sort(listForSort, new Comparator<NicTO>() {
+
+ @Override
+ public int compare(NicTO arg0, NicTO arg1) {
+ if (arg0.getDeviceId() < arg1.getDeviceId()) {
+ return -1;
+ } else if (arg0.getDeviceId() == arg1.getDeviceId()) {
+ return 0;
+ }
+
+ return 1;
+ }
+ });
+
+ return listForSort.toArray(new NicTO[0]);
+ }
+
+ private static void configBasicExtraOption(List<OptionValue> extraOptions, VirtualMachineTO vmSpec) {
+ OptionValue newVal = new OptionValue();
+ newVal.setKey("machine.id");
+ newVal.setValue(vmSpec.getBootArgs());
+ extraOptions.add(newVal);
+
+ newVal = new OptionValue();
+ newVal.setKey("devices.hotplug");
+ newVal.setValue("true");
+ extraOptions.add(newVal);
+ }
+
+ private static void configNvpExtraOption(List<OptionValue> extraOptions, VirtualMachineTO vmSpec, Map<String, String> nicUuidToDvSwitchUuid) {
+ /**
+ * Extra Config : nvp.vm-uuid = uuid
+ * - Required for Nicira NVP integration
+ */
+ OptionValue newVal = new OptionValue();
+ newVal.setKey("nvp.vm-uuid");
+ newVal.setValue(vmSpec.getUuid());
+ extraOptions.add(newVal);
+
+ /**
+ * Extra Config : nvp.iface-id.<num> = uuid
+ * - Required for Nicira NVP integration
+ */
+ int nicNum = 0;
+ for (NicTO nicTo : sortNicsByDeviceId(vmSpec.getNics())) {
+ if (nicTo.getUuid() != null) {
+ newVal = new OptionValue();
+ newVal.setKey("nvp.iface-id." + nicNum);
+ newVal.setValue(nicTo.getUuid());
+ extraOptions.add(newVal);
+ }
+ nicNum++;
+ }
+ }
+
+ private static void configCustomExtraOption(List<OptionValue> extraOptions, VirtualMachineTO vmSpec) {
+ // we no longer to validation anymore
+ for (Map.Entry<String, String> entry : vmSpec.getDetails().entrySet()) {
+ if (entry.getKey().equalsIgnoreCase(VmDetailConstants.BOOT_MODE)) {
+ continue;
+ }
+ OptionValue newVal = new OptionValue();
+ newVal.setKey(entry.getKey());
+ newVal.setValue(entry.getValue());
+ extraOptions.add(newVal);
+ }
+ }
+
+ private class VmAlreadyExistsInVcenter extends Exception {
+ public VmAlreadyExistsInVcenter(String msg) {
+ }
+ }
+
+ private class VirtualMachineData {
+ String vmName = null;
+ VirtualMachineFileInfo vmFileInfo = null;
+ VirtualMachineFileLayoutEx vmFileLayout = null;
+ List<DatastoreMO> datastores = new ArrayList<>();
+ }
+
+ private class VirtualMachineRecycler {
+ private String vmInternalCSName;
+ private Pair<String, String> controllerInfo;
+ private Boolean systemVm;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private DiskControllerType systemVmScsiControllerType;
+ private int firstScsiControllerBusNum;
+ private int numScsiControllerForSystemVm;
+ private VirtualMachineDiskInfoBuilder diskInfoBuilder;
+ private boolean hasSnapshot;
+ private VirtualDevice[] nicDevices;
+
+ public VirtualMachineRecycler(String vmInternalCSName, Pair<String, String> controllerInfo, Boolean systemVm, VmwareHypervisorHost hyperHost, VirtualMachineMO vmMo,
+ DiskControllerType systemVmScsiControllerType, int firstScsiControllerBusNum, int numScsiControllerForSystemVm) {
+ this.vmInternalCSName = vmInternalCSName;
+ this.controllerInfo = controllerInfo;
+ this.systemVm = systemVm;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.systemVmScsiControllerType = systemVmScsiControllerType;
+ this.firstScsiControllerBusNum = firstScsiControllerBusNum;
+ this.numScsiControllerForSystemVm = numScsiControllerForSystemVm;
+ }
+
+ public VirtualMachineDiskInfoBuilder getDiskInfoBuilder() {
+ return diskInfoBuilder;
+ }
+
+ public boolean isHasSnapshot() {
+ return hasSnapshot;
+ }
+
+ public VirtualMachineRecycler invoke() throws Exception {
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Found vm " + vmInternalCSName + " at other host, relocate to " + hyperHost.getHyperHostName());
+ }
+
+ takeVmFromOtherHyperHost(hyperHost, vmInternalCSName);
+
+ if (VmwareResource.getVmPowerState(vmMo) != VirtualMachine.PowerState.PowerOff)
+ vmMo.safePowerOff(vmwareResource.getShutdownWaitMs());
+
+ diskInfoBuilder = vmMo.getDiskInfoBuilder();
+ hasSnapshot = vmMo.hasSnapshot();
+ nicDevices = vmMo.getNicDevices();
+ if (!hasSnapshot)
+ vmMo.tearDownDevices(new Class<?>[] {VirtualDisk.class, VirtualEthernetCard.class});
+ else
+ vmMo.tearDownDevices(new Class<?>[] {VirtualEthernetCard.class});
+
+ if (systemVm) {
+ // System volumes doesn't require more than 1 SCSI controller as there is no requirement for data volumes.
+ ensureScsiDiskControllers(vmMo, systemVmScsiControllerType.toString(), numScsiControllerForSystemVm, firstScsiControllerBusNum);
+ } else {
+ ensureDiskControllers(vmMo, controllerInfo);
+ }
+ return this;
+ }
+
+ private VirtualMachineMO takeVmFromOtherHyperHost(VmwareHypervisorHost hyperHost, String vmName) throws Exception {
+
+ VirtualMachineMO vmMo = hyperHost.findVmOnPeerHyperHost(vmName);
+ if (vmMo != null) {
+ ManagedObjectReference morTargetPhysicalHost = hyperHost.findMigrationTarget(vmMo);
+ if (morTargetPhysicalHost == null) {
+ String msg = "VM " + vmName + " is on other host and we have no resource available to migrate and start it here";
+ LOGGER.error(msg);
+ throw new Exception(msg);
+ }
+
+ if (!vmMo.relocate(morTargetPhysicalHost)) {
+ String msg = "VM " + vmName + " is on other host and we failed to relocate it here";
+ LOGGER.error(msg);
+ throw new Exception(msg);
+ }
+
+ return vmMo;
+ }
+ return null;
+ }
+
+ public VirtualDevice[] getNicDevices() {
+ return nicDevices;
+ }
+ }
+
+ private class PrepareSytemVMPatchISOMethod {
+ private VmwareManager mgr;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int i;
+ private int ideUnitNumber;
+
+ public PrepareSytemVMPatchISOMethod(VmwareManager mgr, VmwareHypervisorHost hyperHost, VirtualMachineMO vmMo, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int i, int ideUnitNumber) {
+ this.mgr = mgr;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.i = i;
+ this.ideUnitNumber = ideUnitNumber;
+ }
+
+ public int getI() {
+ return i;
+ }
+
+ public int getIdeUnitNumber() {
+ return ideUnitNumber;
+ }
+
+ public PrepareSytemVMPatchISOMethod invoke() throws Exception {
+ // attach ISO (for patching of system VM)
+ DatastoreMO secDsMo = getDatastoreMOForSecStore(mgr, hyperHost);
+
+ deviceConfigSpecArray[i] = new VirtualDeviceConfigSpec();
+ Pair<VirtualDevice, Boolean> isoInfo = VmwareHelper
+ .prepareIsoDevice(vmMo, String.format("[%s] systemvm/%s", secDsMo.getName(), mgr.getSystemVMIsoFileNameOnDatastore()), secDsMo.getMor(), true, true,
+ ideUnitNumber++, i + 1);
+ deviceConfigSpecArray[i].setDevice(isoInfo.first());
+ if (isoInfo.second()) {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at new device " + vmwareResource.getGson().toJson(isoInfo.first()));
+ deviceConfigSpecArray[i].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+ } else {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+ deviceConfigSpecArray[i].setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ }
+ i++;
+ return this;
+ }
+
+ }
+ private DatastoreMO getDatastoreMOForSecStore(VmwareManager mgr, VmwareHypervisorHost hyperHost) throws Exception {
+ Pair<String, Long> secStoreUrlAndId = mgr.getSecondaryStorageStoreUrlAndId(Long.parseLong(vmwareResource.getDcId()));
+ String secStoreUrl = secStoreUrlAndId.first();
+ Long secStoreId = secStoreUrlAndId.second();
+ if (secStoreUrl == null) {
+ String msg = "secondary storage for dc " + vmwareResource.getDcId() + " is not ready yet?";
+ throw new Exception(msg);
+ }
+ mgr.prepareSecondaryStorageStore(secStoreUrl, secStoreId);
+
+ ManagedObjectReference morSecDs = vmwareResource.prepareSecondaryDatastoreOnHost(secStoreUrl);
+ if (morSecDs == null) {
+ String msg = "Failed to prepare secondary storage on host, secondary store url: " + secStoreUrl;
+ throw new Exception(msg);
+ }
+ return new DatastoreMO(hyperHost.getContext(), morSecDs);
+ }
+
+ /**
+ // Setup ROOT/DATA disk devices
+ */
+ private class DiskSetup {
+ private VirtualMachineTO vmSpec;
+ private DiskTO rootDiskTO;
+ private Pair<String, String> controllerInfo;
+ private VmwareContext context;
+ private DatacenterMO dcMo;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private DiskTO[] disks;
+ private HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails;
+ private VirtualMachineDiskInfoBuilder diskInfoBuilder;
+ private boolean hasSnapshot;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int deviceCount;
+ private int ideUnitNumber;
+ private int scsiUnitNumber;
+ private int ideControllerKey;
+ private int scsiControllerKey;
+ private DiskTO[] sortedDisks;
+ private boolean installAsIs;
+
+ public DiskSetup(VirtualMachineTO vmSpec, DiskTO rootDiskTO, Pair<String, String> controllerInfo, VmwareContext context, DatacenterMO dcMo, VmwareHypervisorHost hyperHost,
+ VirtualMachineMO vmMo, DiskTO[] disks, HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails, VirtualMachineDiskInfoBuilder diskInfoBuilder,
+ boolean hasSnapshot, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount, int ideUnitNumber, int scsiUnitNumber, int ideControllerKey, int scsiControllerKey, boolean installAsIs) {
+ this.vmSpec = vmSpec;
+ this.rootDiskTO = rootDiskTO;
+ this.controllerInfo = controllerInfo;
+ this.context = context;
+ this.dcMo = dcMo;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.disks = disks;
+ this.dataStoresDetails = dataStoresDetails;
+ this.diskInfoBuilder = diskInfoBuilder;
+ this.hasSnapshot = hasSnapshot;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.deviceCount = deviceCount;
+ this.ideUnitNumber = ideUnitNumber;
+ this.scsiUnitNumber = scsiUnitNumber;
+ this.ideControllerKey = ideControllerKey;
+ this.scsiControllerKey = scsiControllerKey;
+ this.installAsIs = installAsIs;
+ }
+
+ public DiskTO getRootDiskTO() {
+ return rootDiskTO;
+ }
+
+ public int getDeviceCount() {
+ return deviceCount;
+ }
+
+ public DiskTO[] getSortedDisks() {
+ return sortedDisks;
+ }
+
+ public DiskSetup invoke() throws Exception {
+ int controllerKey;
+ sortedDisks = sortVolumesByDeviceId(disks);
+ for (DiskTO vol : sortedDisks) {
+ if (vol.getType() == Volume.Type.ISO || installAsIs) {
+ continue;
+ }
+
+ VirtualMachineDiskInfo matchingExistingDisk = getMatchingExistingDisk(diskInfoBuilder, vol, hyperHost, context);
+ controllerKey = getDiskController(matchingExistingDisk, vol, vmSpec, ideControllerKey, scsiControllerKey);
+ String diskController = getDiskController(vmMo, matchingExistingDisk, vol, controllerInfo);
+
+ if (DiskControllerType.getType(diskController) == DiskControllerType.osdefault) {
+ diskController = vmMo.getRecommendedDiskController(null);
+ }
+ if (DiskControllerType.getType(diskController) == DiskControllerType.ide) {
+ controllerKey = vmMo.getIDEControllerKey(ideUnitNumber);
+ if (vol.getType() == Volume.Type.DATADISK) {
+ // Could be result of flip due to user configured setting or "osdefault" for data disks
+ // Ensure maximum of 2 data volumes over IDE controller, 3 includeing root volume
+ if (vmMo.getNumberOfVirtualDisks() > 3) {
+ throw new CloudRuntimeException(
+ "Found more than 3 virtual disks attached to this VM [" + vmMo.getVmName() + "]. Unable to implement the disks over " + diskController + " controller, as maximum number of devices supported over IDE controller is 4 includeing CDROM device.");
+ }
+ }
+ } else {
+ if (VmwareHelper.isReservedScsiDeviceNumber(scsiUnitNumber)) {
+ scsiUnitNumber++;
+ }
+
+ controllerKey = vmMo.getScsiDiskControllerKeyNoException(diskController, scsiUnitNumber);
+ if (controllerKey == -1) {
+ // This may happen for ROOT legacy VMs which doesn't have recommended disk controller when global configuration parameter 'vmware.root.disk.controller' is set to "osdefault"
+ // Retrieve existing controller and use.
+ Ternary<Integer, Integer, DiskControllerType> vmScsiControllerInfo = vmMo.getScsiControllerInfo();
+ DiskControllerType existingControllerType = vmScsiControllerInfo.third();
+ controllerKey = vmMo.getScsiDiskControllerKeyNoException(existingControllerType.toString(), scsiUnitNumber);
+ }
+ }
+ if (!hasSnapshot) {
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+
+ VolumeObjectTO volumeTO = (VolumeObjectTO)vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+ String iScsiName = null;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ iScsiName = details.get(DiskTO.IQN);
+ }
+
+ // if the storage is managed, iScsiName should not be null
+ String datastoreName = managed ? VmwareResource.getDatastoreName(iScsiName) : primaryStore.getUuid();
+ Pair<ManagedObjectReference, DatastoreMO> volumeDsDetails = dataStoresDetails.get(datastoreName);
+
+ assert (volumeDsDetails != null);
+
+ String[] diskChain = syncDiskChain(dcMo, vmMo, vmSpec, vol, matchingExistingDisk, dataStoresDetails);
+
+ int deviceNumber = -1;
+ if (controllerKey == vmMo.getIDEControllerKey(ideUnitNumber)) {
+ deviceNumber = ideUnitNumber % VmwareHelper.MAX_ALLOWED_DEVICES_IDE_CONTROLLER;
+ ideUnitNumber++;
+ } else {
+ deviceNumber = scsiUnitNumber % VmwareHelper.MAX_ALLOWED_DEVICES_SCSI_CONTROLLER;
+ scsiUnitNumber++;
+ }
+ VirtualDevice device = VmwareHelper.prepareDiskDevice(vmMo, null, controllerKey, diskChain, volumeDsDetails.first(), deviceNumber, deviceCount + 1);
+ if (vol.getType() == Volume.Type.ROOT)
+ rootDiskTO = vol;
+ deviceConfigSpecArray[deviceCount].setDevice(device);
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare volume at new device " + vmwareResource.getGson().toJson(device));
+
+ deviceCount++;
+ } else {
+ if (controllerKey == vmMo.getIDEControllerKey(ideUnitNumber))
+ ideUnitNumber++;
+ else
+ scsiUnitNumber++;
+ }
+ }
+ return this;
+ }
+
+ private int getDiskController(VirtualMachineDiskInfo matchingExistingDisk, DiskTO vol, VirtualMachineTO vmSpec, int ideControllerKey, int scsiControllerKey) {
+
+ int controllerKey;
+ if (matchingExistingDisk != null) {
+ LOGGER.info("Chose disk controller based on existing information: " + matchingExistingDisk.getDiskDeviceBusName());
+ if (matchingExistingDisk.getDiskDeviceBusName().startsWith("ide"))
+ return ideControllerKey;
+ else
+ return scsiControllerKey;
+ }
+
+ if (vol.getType() == Volume.Type.ROOT) {
+ Map<String, String> vmDetails = vmSpec.getDetails();
+ if (vmDetails != null && vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER) != null) {
+ if (vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER).equalsIgnoreCase("scsi")) {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> scsi, based on root disk controller settings: "
+ + vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER));
+ controllerKey = scsiControllerKey;
+ } else {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> ide, based on root disk controller settings: "
+ + vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER));
+ controllerKey = ideControllerKey;
+ }
+ } else {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> scsi. due to null root disk controller setting");
+ controllerKey = scsiControllerKey;
+ }
+
+ } else {
+ // DATA volume always use SCSI device
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> scsi");
+ controllerKey = scsiControllerKey;
+ }
+
+ return controllerKey;
+ }
+
+ private String getDiskController(VirtualMachineMO vmMo, VirtualMachineDiskInfo matchingExistingDisk, DiskTO vol, Pair<String, String> controllerInfo) throws Exception {
+ int controllerKey;
+ DiskControllerType controllerType = DiskControllerType.none;
+ if (matchingExistingDisk != null) {
+ String currentBusName = matchingExistingDisk.getDiskDeviceBusName();
+ if (currentBusName != null) {
+ LOGGER.info("Chose disk controller based on existing information: " + currentBusName);
+ if (currentBusName.startsWith("ide")) {
+ controllerType = DiskControllerType.ide;
+ } else if (currentBusName.startsWith("scsi")) {
+ controllerType = DiskControllerType.scsi;
+ }
+ }
+ if (controllerType == DiskControllerType.scsi || controllerType == DiskControllerType.none) {
+ Ternary<Integer, Integer, DiskControllerType> vmScsiControllerInfo = vmMo.getScsiControllerInfo();
+ controllerType = vmScsiControllerInfo.third();
+ }
+ return controllerType.toString();
+ }
+
+ if (vol.getType() == Volume.Type.ROOT) {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> " + controllerInfo.first()
+ + ", based on root disk controller settings at global configuration setting.");
+ return controllerInfo.first();
+ } else {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> " + controllerInfo.second()
+ + ", based on default data disk controller setting i.e. Operating system recommended."); // Need to bring in global configuration setting & template level setting.
+ return controllerInfo.second();
+ }
+ }
+
+ // return the finalized disk chain for startup, from top to bottom
+ private String[] syncDiskChain(DatacenterMO dcMo, VirtualMachineMO vmMo, VirtualMachineTO vmSpec, DiskTO vol, VirtualMachineDiskInfo diskInfo,
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails) throws Exception {
+
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ Map<String, String> details = vol.getDetails();
+ boolean isManaged = false;
+ String iScsiName = null;
+
+ if (details != null) {
+ isManaged = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ iScsiName = details.get(DiskTO.IQN);
+ }
+
+ // if the storage is managed, iScsiName should not be null
+ String datastoreName = isManaged ? VmwareResource.getDatastoreName(iScsiName) : primaryStore.getUuid();
+ Pair<ManagedObjectReference, DatastoreMO> volumeDsDetails = dataStoresDetails.get(datastoreName);
+
+ if (volumeDsDetails == null) {
+ throw new Exception("Primary datastore " + primaryStore.getUuid() + " is not mounted on host.");
+ }
+
+ DatastoreMO dsMo = volumeDsDetails.second();
+
+ // we will honor vCenter's meta if it exists
+ if (diskInfo != null) {
+ // to deal with run-time upgrade to maintain the new datastore folder structure
+ String disks[] = diskInfo.getDiskChain();
+ for (int i = 0; i < disks.length; i++) {
+ DatastoreFile file = new DatastoreFile(disks[i]);
+ if (!isManaged && file.getDir() != null && file.getDir().isEmpty()) {
+ LOGGER.info("Perform run-time datastore folder upgrade. sync " + disks[i] + " to VM folder");
+ disks[i] = VmwareStorageLayoutHelper.syncVolumeToVmDefaultFolder(dcMo, vmMo.getName(), dsMo, file.getFileBaseName(), VmwareManager.s_vmwareSearchExcludeFolder.value());
+ }
+ }
+ return disks;
+ }
+
+ final String datastoreDiskPath;
+
+ if (isManaged) {
+ String vmdkPath = new DatastoreFile(volumeTO.getPath()).getFileBaseName();
+
+ if (volumeTO.getVolumeType() == Volume.Type.ROOT) {
+ if (vmdkPath == null) {
+ vmdkPath = volumeTO.getName();
+ }
+
+ datastoreDiskPath = VmwareStorageLayoutHelper.syncVolumeToVmDefaultFolder(dcMo, vmMo.getName(), dsMo, vmdkPath);
+ } else {
+ if (vmdkPath == null) {
+ vmdkPath = dsMo.getName();
+ }
+
+ datastoreDiskPath = dsMo.getDatastorePath(vmdkPath + VmwareResource.VMDK_EXTENSION);
+ }
+ } else {
+ datastoreDiskPath = VmwareStorageLayoutHelper.syncVolumeToVmDefaultFolder(dcMo, vmMo.getName(), dsMo, volumeTO.getPath(), VmwareManager.s_vmwareSearchExcludeFolder.value());
+ }
+
+ if (!dsMo.fileExists(datastoreDiskPath)) {
+ LOGGER.warn("Volume " + volumeTO.getId() + " does not seem to exist on datastore, out of sync? path: " + datastoreDiskPath);
+ }
+
+ return new String[]{datastoreDiskPath};
+ }
+ }
+
+ /**
+ // Setup NIC devices
+ */
+ private class NicSetup {
+ private StartCommand cmd;
+ private VirtualMachineTO vmSpec;
+ private String vmInternalCSName;
+ private VmwareContext context;
+ private VmwareManager mgr;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private NicTO[] nics;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int deviceCount;
+ private int nicMask;
+ private int nicCount;
+ private Map<String, String> nicUuidToDvSwitchUuid;
+ private VirtualDevice[] nicDevices;
+
+ public NicSetup(StartCommand cmd, VirtualMachineTO vmSpec, String vmInternalCSName, VmwareContext context, VmwareManager mgr, VmwareHypervisorHost hyperHost,
+ VirtualMachineMO vmMo, NicTO[] nics, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount, VirtualDevice[] nicDevices) {
+ this.cmd = cmd;
+ this.vmSpec = vmSpec;
+ this.vmInternalCSName = vmInternalCSName;
+ this.context = context;
+ this.mgr = mgr;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.nics = nics;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.deviceCount = deviceCount;
+ this.nicDevices = nicDevices;
+ }
+
+ public int getDeviceCount() {
+ return deviceCount;
+ }
+
+ public int getNicMask() {
+ return nicMask;
+ }
+
+ public int getNicCount() {
+ return nicCount;
+ }
+
+ public Map<String, String> getNicUuidToDvSwitchUuid() {
+ return nicUuidToDvSwitchUuid;
+ }
+
+ public NicSetup invoke() throws Exception {
+ VirtualDevice nic;
+ nicMask = 0;
+ nicCount = 0;
+
+ if (vmSpec.getType() == VirtualMachine.Type.DomainRouter) {
+ doDomainRouterSetup();
+ }
+
+ VirtualEthernetCardType nicDeviceType = VirtualEthernetCardType.valueOf(vmSpec.getDetails().get(VmDetailConstants.NIC_ADAPTER));
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("VM " + vmInternalCSName + " will be started with NIC device type: " + nicDeviceType);
+ }
+
+ NiciraNvpApiVersion.logNiciraApiVersion();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.debug(String.format("deciding for VM '%s' to use orchestrated NICs or as is.", vmInternalCSName));
+ }
+ nicUuidToDvSwitchUuid = new HashMap<>();
+ LOGGER.info(String.format("adding %d nics to VM '%s'", nics.length, vmInternalCSName));
+ for (NicTO nicTo : sortNicsByDeviceId(nics)) {
+ LOGGER.info("Prepare NIC device based on NicTO: " + vmwareResource.getGson().toJson(nicTo));
+
+ boolean configureVServiceInNexus = (nicTo.getType() == Networks.TrafficType.Guest) && (vmSpec.getDetails().containsKey("ConfigureVServiceInNexus"));
+ VirtualMachine.Type vmType = cmd.getVirtualMachine().getType();
+ Pair<ManagedObjectReference, String> networkInfo = vmwareResource.prepareNetworkFromNicInfo(vmMo.getRunningHost(), nicTo, configureVServiceInNexus, vmType);
+ if ((nicTo.getBroadcastType() != Networks.BroadcastDomainType.Lswitch) || (nicTo.getBroadcastType() == Networks.BroadcastDomainType.Lswitch && NiciraNvpApiVersion.isApiVersionLowerThan("4.2"))) {
+ if (VmwareHelper.isDvPortGroup(networkInfo.first())) {
+ String dvSwitchUuid;
+ ManagedObjectReference dcMor = hyperHost.getHyperHostDatacenter();
+ DatacenterMO dataCenterMo = new DatacenterMO(context, dcMor);
+ ManagedObjectReference dvsMor = dataCenterMo.getDvSwitchMor(networkInfo.first());
+ dvSwitchUuid = dataCenterMo.getDvSwitchUuid(dvsMor);
+ LOGGER.info("Preparing NIC device on dvSwitch : " + dvSwitchUuid);
+ nic = VmwareHelper.prepareDvNicDevice(vmMo, networkInfo.first(), nicDeviceType, networkInfo.second(), dvSwitchUuid, nicTo.getMac(), deviceCount + 1, true, true);
+ if (nicTo.getUuid() != null) {
+ nicUuidToDvSwitchUuid.put(nicTo.getUuid(), dvSwitchUuid);
+ }
+ } else {
+ LOGGER.info("Preparing NIC device on network " + networkInfo.second());
+ nic = VmwareHelper.prepareNicDevice(vmMo, networkInfo.first(), nicDeviceType, networkInfo.second(), nicTo.getMac(), deviceCount + 1, true, true);
+ }
+ } else {
+ //if NSX API VERSION >= 4.2, connect to br-int (nsx.network), do not create portgroup else previous behaviour
+ nic = VmwareHelper.prepareNicOpaque(vmMo, nicDeviceType, networkInfo.second(), nicTo.getMac(), deviceCount + 1, true, true);
+ }
+
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ deviceConfigSpecArray[deviceCount].setDevice(nic);
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare NIC at new device " + vmwareResource.getGson().toJson(deviceConfigSpecArray[deviceCount]));
+
+ // this is really a hacking for DomR, upon DomR startup, we will reset all the NIC allocation after eth3
+ if (nicCount < 3)
+ nicMask |= (1 << nicCount);
+
+ deviceCount++;
+ nicCount++;
+ }
+ return this;
+ }
+
+ private void doDomainRouterSetup() throws Exception {
+ int extraPublicNics = mgr.getRouterExtraPublicNics();
+ if (extraPublicNics > 0 && vmSpec.getDetails().containsKey("PeerRouterInstanceName")) {
+ //Set identical MAC address for RvR on extra public interfaces
+ String peerRouterInstanceName = vmSpec.getDetails().get("PeerRouterInstanceName");
+
+ VirtualMachineMO peerVmMo = hyperHost.findVmOnHyperHost(peerRouterInstanceName);
+ if (peerVmMo == null) {
+ peerVmMo = hyperHost.findVmOnPeerHyperHost(peerRouterInstanceName);
+ }
+
+ if (peerVmMo != null) {
+ String oldMacSequence = generateMacSequence(nics);
+
+ for (int nicIndex = nics.length - extraPublicNics; nicIndex < nics.length; nicIndex++) {
+ VirtualDevice nicDevice = peerVmMo.getNicDeviceByIndex(nics[nicIndex].getDeviceId());
+ if (nicDevice != null) {
+ String mac = ((VirtualEthernetCard)nicDevice).getMacAddress();
+ if (mac != null) {
+ LOGGER.info("Use same MAC as previous RvR, the MAC is " + mac + " for extra NIC with device id: " + nics[nicIndex].getDeviceId());
+ nics[nicIndex].setMac(mac);
+ }
+ }
+ }
+
+ if (!StringUtils.isBlank(vmSpec.getBootArgs())) {
+ String newMacSequence = generateMacSequence(nics);
+ vmSpec.setBootArgs(vmwareResource.replaceNicsMacSequenceInBootArgs(oldMacSequence, newMacSequence, vmSpec));
+ }
+ }
+ }
+ }
+ }
+
+ private class IsoSetup {
+ private VirtualMachineTO vmSpec;
+ private VmwareManager mgr;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private DiskTO[] disks;
+ private DiskTO volIso;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int deviceCount;
+ private int ideUnitNumber;
+
+ public IsoSetup(VirtualMachineTO vmSpec, VmwareManager mgr, VmwareHypervisorHost hyperHost, VirtualMachineMO vmMo, DiskTO[] disks, DiskTO volIso,
+ VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount, int ideUnitNumber) {
+ this.vmSpec = vmSpec;
+ this.mgr = mgr;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.disks = disks;
+ this.volIso = volIso;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.deviceCount = deviceCount;
+ this.ideUnitNumber = ideUnitNumber;
+ }
+
+ public int getDeviceCount() {
+ return deviceCount;
+ }
+
+ public int getIdeUnitNumber() {
+ return ideUnitNumber;
+ }
+
+ public IsoSetup invoke() throws Exception {
+ //
+ // Setup ISO device
+ //
+
+ // vAPP ISO
+ // FR37 the native deploy mechs should create this for us
+ if (vmSpec.getOvfProperties() != null) {
+ if (LOGGER.isTraceEnabled()) {
+ // FR37 TODO add more usefull info (if we keep this bit
+ LOGGER.trace("adding iso for properties for 'xxx'");
+ }
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ Pair<VirtualDevice, Boolean> isoInfo = VmwareHelper.prepareIsoDevice(vmMo, null, null, true, true, ideUnitNumber++, deviceCount + 1);
+ deviceConfigSpecArray[deviceCount].setDevice(isoInfo.first());
+ if (isoInfo.second()) {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare vApp ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+ } else {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare vApp ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ }
+ deviceCount++;
+ }
+
+ // prepare systemvm patch ISO
+ if (vmSpec.getType() != VirtualMachine.Type.User) {
+ PrepareSytemVMPatchISOMethod prepareSytemVm = new PrepareSytemVMPatchISOMethod(mgr, hyperHost, vmMo, deviceConfigSpecArray, deviceCount, ideUnitNumber);
+ prepareSytemVm.invoke();
+ deviceCount = prepareSytemVm.getI();
+ ideUnitNumber = prepareSytemVm.getIdeUnitNumber();
+ } else {
+ // Note: we will always plug a CDROM device
+ if (volIso != null) {
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ISO) {
+
+ TemplateObjectTO iso = (TemplateObjectTO)vol.getData();
+
+ if (iso.getPath() != null && !iso.getPath().isEmpty()) {
+ DataStoreTO imageStore = iso.getDataStore();
+ if (!(imageStore instanceof NfsTO)) {
+ LOGGER.debug("unsupported protocol");
+ throw new Exception("unsupported protocol");
+ }
+ NfsTO nfsImageStore = (NfsTO)imageStore;
+ String isoPath = nfsImageStore.getUrl() + File.separator + iso.getPath();
+ Pair<String, ManagedObjectReference> isoDatastoreInfo = getIsoDatastoreInfo(hyperHost, isoPath);
+ assert (isoDatastoreInfo != null);
+ assert (isoDatastoreInfo.second() != null);
+
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ Pair<VirtualDevice, Boolean> isoInfo = VmwareHelper
+ .prepareIsoDevice(vmMo, isoDatastoreInfo.first(), isoDatastoreInfo.second(), true, true, ideUnitNumber++, deviceCount + 1);
+ deviceConfigSpecArray[deviceCount].setDevice(isoInfo.first());
+ if (isoInfo.second()) {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at new device " + vmwareResource.getGson().toJson(isoInfo.first()));
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+ } else {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ }
+ }
+ deviceCount++;
+ }
+ }
+ } else {
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ Pair<VirtualDevice, Boolean> isoInfo = VmwareHelper.prepareIsoDevice(vmMo, null, null, true, true, ideUnitNumber++, deviceCount + 1);
+ deviceConfigSpecArray[deviceCount].setDevice(isoInfo.first());
+ if (isoInfo.second()) {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+ } else {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ }
+ deviceCount++;
+ }
+ }
+ return this;
+ }
+
+ // isoUrl sample content :
+ // nfs://192.168.10.231/export/home/kelven/vmware-test/secondary/template/tmpl/2/200//200-2-80f7ee58-6eff-3a2d-bcb0-59663edf6d26.iso
Review comment:
check if this is outdated
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/StartCommandExecutor.java
##########
@@ -0,0 +1,2247 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.hypervisor.vmware.resource;
+
+import java.io.File;
+import java.rmi.RemoteException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.cloud.agent.api.to.DataTO;
+import com.vmware.vim25.VirtualDiskFlatVer2BackingInfo;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.storage.configdrive.ConfigDrive;
+import org.apache.cloudstack.storage.to.TemplateObjectTO;
+import org.apache.cloudstack.storage.to.VolumeObjectTO;
+import org.apache.cloudstack.utils.volume.VirtualMachineDiskInfo;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.commons.lang.ArrayUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.log4j.Logger;
+
+import com.cloud.agent.api.Command;
+import com.cloud.agent.api.StartAnswer;
+import com.cloud.agent.api.StartCommand;
+import com.cloud.agent.api.storage.OVFPropertyTO;
+import com.cloud.agent.api.to.DataStoreTO;
+import com.cloud.agent.api.to.DiskTO;
+import com.cloud.agent.api.to.NfsTO;
+import com.cloud.agent.api.to.NicTO;
+import com.cloud.agent.api.to.VirtualMachineTO;
+import com.cloud.configuration.Resource;
+import com.cloud.hypervisor.vmware.VmwareResourceException;
+import com.cloud.hypervisor.vmware.manager.VmwareManager;
+import com.cloud.hypervisor.vmware.mo.CustomFieldConstants;
+import com.cloud.hypervisor.vmware.mo.DatacenterMO;
+import com.cloud.hypervisor.vmware.mo.DatastoreFile;
+import com.cloud.hypervisor.vmware.mo.DatastoreMO;
+import com.cloud.hypervisor.vmware.mo.DiskControllerType;
+import com.cloud.hypervisor.vmware.mo.HostMO;
+import com.cloud.hypervisor.vmware.mo.HypervisorHostHelper;
+import com.cloud.hypervisor.vmware.mo.TaskMO;
+import com.cloud.hypervisor.vmware.mo.VirtualEthernetCardType;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineDiskInfoBuilder;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineMO;
+import com.cloud.hypervisor.vmware.mo.VmwareHypervisorHost;
+import com.cloud.hypervisor.vmware.util.VmwareContext;
+import com.cloud.hypervisor.vmware.util.VmwareHelper;
+import com.cloud.network.Networks;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Volume;
+import com.cloud.storage.resource.VmwareStorageLayoutHelper;
+import com.cloud.storage.resource.VmwareStorageProcessor;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nicira.nvp.plugin.NiciraNvpApiVersion;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.VmDetailConstants;
+import com.vmware.vim25.BoolPolicy;
+import com.vmware.vim25.DVPortConfigInfo;
+import com.vmware.vim25.DVPortConfigSpec;
+import com.vmware.vim25.DasVmPriority;
+import com.vmware.vim25.DistributedVirtualPort;
+import com.vmware.vim25.DistributedVirtualSwitchPortConnection;
+import com.vmware.vim25.DistributedVirtualSwitchPortCriteria;
+import com.vmware.vim25.ManagedObjectReference;
+import com.vmware.vim25.OptionValue;
+import com.vmware.vim25.VMwareDVSPortSetting;
+import com.vmware.vim25.VirtualDevice;
+import com.vmware.vim25.VirtualDeviceBackingInfo;
+import com.vmware.vim25.VirtualDeviceConfigSpec;
+import com.vmware.vim25.VirtualDeviceConfigSpecOperation;
+import com.vmware.vim25.VirtualDisk;
+import com.vmware.vim25.VirtualEthernetCard;
+import com.vmware.vim25.VirtualEthernetCardDistributedVirtualPortBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardNetworkBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardOpaqueNetworkBackingInfo;
+import com.vmware.vim25.VirtualMachineBootOptions;
+import com.vmware.vim25.VirtualMachineConfigSpec;
+import com.vmware.vim25.VirtualMachineFileInfo;
+import com.vmware.vim25.VirtualMachineFileLayoutEx;
+import com.vmware.vim25.VirtualMachineGuestOsIdentifier;
+import com.vmware.vim25.VirtualUSBController;
+import com.vmware.vim25.VmConfigInfo;
+import com.vmware.vim25.VmwareDistributedVirtualSwitchVlanIdSpec;
+
+class StartCommandExecutor {
+ private static final Logger LOGGER = Logger.getLogger(StartCommandExecutor.class);
+
+ private final VmwareResource vmwareResource;
+
+ public StartCommandExecutor(VmwareResource vmwareResource) {
+ this.vmwareResource = vmwareResource;
+ }
+
+ // FR37 the monster blob god method: reduce to 320 so far and counting
+ protected StartAnswer execute(StartCommand cmd) {
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Executing resource StartCommand: " + vmwareResource.getGson().toJson(cmd));
+ }
+
+ VirtualMachineTO vmSpec = cmd.getVirtualMachine();
+
+ VirtualMachineData existingVm = null;
+
+ Pair<String, String> names = composeVmNames(vmSpec);
+ String vmInternalCSName = names.first();
+ String vmNameOnVcenter = names.second();
+
+ DiskTO rootDiskTO = null;
+
+ Pair<String, String> controllerInfo = getDiskControllerInfo(vmSpec);
+
+ Boolean systemVm = vmSpec.getType().isUsedBySystem();
+
+ VmwareContext context = vmwareResource.getServiceContext();
+ DatacenterMO dcMo = null;
+ try {
+ VmwareManager mgr = context.getStockObject(VmwareManager.CONTEXT_STOCK_NAME);
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(context);
+ dcMo = new DatacenterMO(hyperHost.getContext(), hyperHost.getHyperHostDatacenter());
+
+ // checkIfVmExistsInVcenter(vmInternalCSName, vmNameOnVcenter, dcMo);
+ // FR37 - We expect VM to be already cloned and available at this point
+ VirtualMachineMO vmMo = dcMo.findVm(vmInternalCSName);
+ if (vmMo == null) {
+ vmMo = dcMo.findVm(vmNameOnVcenter);
+ }
+
+ DiskTO[] specDisks = vmSpec.getDisks();
+ boolean installAsIs = StringUtils.isNotEmpty(vmSpec.getTemplateLocation());
+ // FR37 if startcommand contains enough info: a template url/-location and flag; deploy OVA as is
+ if (vmMo == null && installAsIs) {
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("deploying OVA from %s as is", vmSpec.getTemplateLocation()));
+ }
+ getStorageProcessor().cloneVMFromTemplate(vmSpec.getTemplateName(), vmInternalCSName, vmSpec.getTemplatePrimaryStoreUuid());
+ vmMo = dcMo.findVm(vmInternalCSName);
+ mapSpecDisksToClonedDisks(vmMo, vmInternalCSName, specDisks);
+ }
+
+ // VM may not have been on the same host, relocate to expected host
+ if (vmMo != null) {
+ vmMo.relocate(hyperHost.getMor());
+ // Get updated MO
+ vmMo = hyperHost.findVmOnHyperHost(vmMo.getVmName());
+ }
+
+ String guestOsId = translateGuestOsIdentifier(vmSpec.getArch(), vmSpec.getOs(), vmSpec.getPlatformEmulator()).value();
+ DiskTO[] disks = validateDisks(specDisks);
+ NicTO[] nics = vmSpec.getNics();
+
+ // FIXME: disks logic here, why is disks/volumes during copy not set with pool ID?
+ // FR37 TODO if deployasis a new VM no datastores are known (yet) and we need to get the data store from the tvmspec content library / template location
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails = inferDatastoreDetailsFromDiskInfo(hyperHost, context, disks, cmd);
+ if ((dataStoresDetails == null) || (dataStoresDetails.isEmpty())) {
+ String msg = "Unable to locate datastore details of the volumes to be attached";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ // FR37 - this may not be necessary the cloned VM will have disks and knowledge of datastore paths/location?
+ // throw new Exception(msg);
+ }
+
+ // FR37 - this may need checking, if the first datastore is the right one - ideally it should be datastore where first disk is hosted
+ DatastoreMO dsRootVolumeIsOn = null; //
+ if (! installAsIs) {
+ dsRootVolumeIsOn = getDatastoreThatRootDiskIsOn(dataStoresDetails, disks);
+
+ if (dsRootVolumeIsOn == null) {
+ String msg = "Unable to locate datastore details of root volume";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ throw new VmwareResourceException(msg);
+ }
+ }
+
+ VirtualMachineDiskInfoBuilder diskInfoBuilder = null;
+ VirtualDevice[] nicDevices = null;
+ DiskControllerType systemVmScsiControllerType = DiskControllerType.lsilogic;
+ int firstScsiControllerBusNum = 0;
+ int numScsiControllerForSystemVm = 1;
+ boolean hasSnapshot = false;
+ if (vmMo != null) {
+ PrepareRunningVMForConfiguration prepareRunningVMForConfiguration = new PrepareRunningVMForConfiguration(vmInternalCSName, controllerInfo, systemVm, vmMo,
+ systemVmScsiControllerType, firstScsiControllerBusNum, numScsiControllerForSystemVm);
+ prepareRunningVMForConfiguration.invoke();
+ diskInfoBuilder = prepareRunningVMForConfiguration.getDiskInfoBuilder();
+ nicDevices = prepareRunningVMForConfiguration.getNicDevices();
+ hasSnapshot = prepareRunningVMForConfiguration.isHasSnapshot();
+ } else {
+ ManagedObjectReference morDc = hyperHost.getHyperHostDatacenter();
+ assert (morDc != null);
+
+ vmMo = hyperHost.findVmOnPeerHyperHost(vmInternalCSName);
+ if (vmMo != null) {
+ VirtualMachineRecycler virtualMachineRecycler = new VirtualMachineRecycler(vmInternalCSName, controllerInfo, systemVm, hyperHost, vmMo,
+ systemVmScsiControllerType, firstScsiControllerBusNum, numScsiControllerForSystemVm);
+ virtualMachineRecycler.invoke();
+ diskInfoBuilder = virtualMachineRecycler.getDiskInfoBuilder();
+ hasSnapshot = virtualMachineRecycler.isHasSnapshot();
+ nicDevices = virtualMachineRecycler.getNicDevices();
+ } else {
+ // FR37 we just didn't find a VM by name 'vmInternalCSName', so why is this here?
+ existingVm = unregisterOnOtherClusterButHoldOnToOldVmData(vmInternalCSName, dcMo);
+
+ createNewVm(context, vmSpec, installAsIs, vmInternalCSName, vmNameOnVcenter, controllerInfo, systemVm, mgr, hyperHost, guestOsId, disks, dataStoresDetails,
+ dsRootVolumeIsOn);
+ }
+
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ if (vmMo == null) {
+ throw new Exception("Failed to find the newly create or relocated VM. vmName: " + vmInternalCSName);
+ }
+ }
+ // vmMo should now be a stopped VM on the intended host
+ // The number of disks changed must be 0 for install as is, as the VM is a clone
+ int disksChanges = !installAsIs ? disks.length : 0;
+ int totalChangeDevices = disksChanges + nics.length;
+ int hackDeviceCount = 0;
+ if (diskInfoBuilder != null) {
+ hackDeviceCount += diskInfoBuilder.getDiskCount();
+ }
+ hackDeviceCount += nicDevices == null ? 0 : nicDevices.length;
+ // vApp cdrom device
+ // HACK ALERT: ovf properties might not be the only or defining feature of vApps; needs checking
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("current count(s) desired: %d/ found:%d. now adding device to device count for vApp config ISO", totalChangeDevices, hackDeviceCount));
+ }
+ if (vmSpec.getOvfProperties() != null) {
+ totalChangeDevices++;
+ }
+
+ DiskTO volIso = null;
+ if (vmSpec.getType() != VirtualMachine.Type.User) {
+ // system VM needs a patch ISO
+ totalChangeDevices++;
+ } else {
+ volIso = getIsoDiskTO(disks);
+ if (volIso == null)
+ totalChangeDevices++;
+ }
+
+ VirtualMachineConfigSpec vmConfigSpec = new VirtualMachineConfigSpec();
+
+ VmwareHelper.setBasicVmConfig(vmConfigSpec, vmSpec.getCpus(), vmSpec.getMaxSpeed(), vmwareResource.getReservedCpuMHZ(vmSpec), (int)(vmSpec.getMaxRam() / (1024 * 1024)),
+ vmwareResource.getReservedMemoryMb(vmSpec), guestOsId, vmSpec.getLimitCpuUse());
+
+ int numCoresPerSocket = adjustNumberOfCoresPerSocket(vmSpec, vmMo, vmConfigSpec);
+
+ // Check for hotadd settings
+ vmConfigSpec.setMemoryHotAddEnabled(vmMo.isMemoryHotAddSupported(guestOsId));
+
+ String hostApiVersion = ((HostMO)hyperHost).getHostAboutInfo().getApiVersion();
+ if (numCoresPerSocket > 1 && hostApiVersion.compareTo("5.0") < 0) {
+ LOGGER.warn("Dynamic scaling of CPU is not supported for Virtual Machines with multi-core vCPUs in case of ESXi hosts 4.1 and prior. Hence CpuHotAdd will not be"
+ + " enabled for Virtual Machine: " + vmInternalCSName);
+ vmConfigSpec.setCpuHotAddEnabled(false);
+ } else {
+ vmConfigSpec.setCpuHotAddEnabled(vmMo.isCpuHotAddSupported(guestOsId));
+ }
+
+ vmwareResource.configNestedHVSupport(vmMo, vmSpec, vmConfigSpec);
+
+ VirtualDeviceConfigSpec[] deviceConfigSpecArray = new VirtualDeviceConfigSpec[totalChangeDevices];
+ int deviceCount = 0;
+ int ideUnitNumber = 0;
+ int scsiUnitNumber = 0;
+ int ideControllerKey = vmMo.getIDEDeviceControllerKey();
+ int scsiControllerKey = vmMo.getScsiDeviceControllerKeyNoException();
+
+ IsoSetup isoSetup = new IsoSetup(vmSpec, mgr, hyperHost, vmMo, disks, volIso, deviceConfigSpecArray, deviceCount, ideUnitNumber);
+ isoSetup.invoke();
+ deviceCount = isoSetup.getDeviceCount();
+ ideUnitNumber = isoSetup.getIdeUnitNumber();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("setting up %d disks with root from %s", diskInfoBuilder.getDiskCount(), vmwareResource.getGson().toJson(rootDiskTO)));
+ }
+
+ DiskSetup diskSetup = new DiskSetup(vmSpec, rootDiskTO, controllerInfo, context, dcMo, hyperHost, vmMo, disks, dataStoresDetails, diskInfoBuilder, hasSnapshot,
+ deviceConfigSpecArray, deviceCount, ideUnitNumber, scsiUnitNumber, ideControllerKey, scsiControllerKey, installAsIs);
+ diskSetup.invoke();
+
+ rootDiskTO = diskSetup.getRootDiskTO();
+ deviceCount = diskSetup.getDeviceCount();
+ DiskTO[] sortedDisks = diskSetup.getSortedDisks();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("seting up %d disks with root from %s", sortedDisks.length, vmwareResource.getGson().toJson(rootDiskTO)));
+ }
+
+ deviceCount += setupUsbDevicesAndGetCount(vmInternalCSName, vmMo, guestOsId, deviceConfigSpecArray, deviceCount);
+
+ NicSetup nicSetup = new NicSetup(cmd, vmSpec, vmInternalCSName, context, mgr, hyperHost, vmMo, nics, deviceConfigSpecArray, deviceCount, nicDevices);
+ nicSetup.invoke();
+
+ deviceCount = nicSetup.getDeviceCount();
+ int nicMask = nicSetup.getNicMask();
+ int nicCount = nicSetup.getNicCount();
+ Map<String, String> nicUuidToDvSwitchUuid = nicSetup.getNicUuidToDvSwitchUuid();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("device count: %d; nic count %d", deviceCount, nicCount));
+ }
+ for (int j = 0; j < deviceCount; j++)
+ vmConfigSpec.getDeviceChange().add(deviceConfigSpecArray[j]);
+
+ //
+ // Setup VM options
+ //
+
+ // pass boot arguments through machine.id & perform customized options to VMX
+ ArrayList<OptionValue> extraOptions = new ArrayList<>();
+ configBasicExtraOption(extraOptions, vmSpec);
+ configNvpExtraOption(extraOptions, vmSpec, nicUuidToDvSwitchUuid);
+ configCustomExtraOption(extraOptions, vmSpec);
+
+ // config for NCC
+ VirtualMachine.Type vmType = cmd.getVirtualMachine().getType();
+ if (vmType.equals(VirtualMachine.Type.NetScalerVm)) {
+ NicTO mgmtNic = vmSpec.getNics()[0];
+ OptionValue option = new OptionValue();
+ option.setKey("machine.id");
+ option.setValue("ip=" + mgmtNic.getIp() + "&netmask=" + mgmtNic.getNetmask() + "&gateway=" + mgmtNic.getGateway());
+ extraOptions.add(option);
+ }
+
+ // config VNC
+ String keyboardLayout = null;
+ if (vmSpec.getDetails() != null)
+ keyboardLayout = vmSpec.getDetails().get(VmDetailConstants.KEYBOARD);
+ vmConfigSpec.getExtraConfig().addAll(
+ Arrays.asList(vmwareResource.configureVnc(extraOptions.toArray(new OptionValue[0]), hyperHost, vmInternalCSName, vmSpec.getVncPassword(), keyboardLayout)));
+
+ // config video card
+ vmwareResource.configureVideoCard(vmMo, vmSpec, vmConfigSpec);
+
+ // Set OVF properties (if available)
+ Pair<String, List<OVFPropertyTO>> ovfPropsMap = vmSpec.getOvfProperties();
+ VmConfigInfo templateVappConfig;
+ List<OVFPropertyTO> ovfProperties;
+ if (ovfPropsMap != null) {
+ String vmTemplate = ovfPropsMap.first();
+ LOGGER.info("Find VM template " + vmTemplate);
+ VirtualMachineMO vmTemplateMO = dcMo.findVm(vmTemplate);
+ templateVappConfig = vmTemplateMO.getConfigInfo().getVAppConfig();
+ ovfProperties = ovfPropsMap.second();
+ // Set OVF properties (if available)
+ if (CollectionUtils.isNotEmpty(ovfProperties)) {
+ LOGGER.info("Copying OVF properties from template and setting them to the values the user provided");
+ vmwareResource.copyVAppConfigsFromTemplate(templateVappConfig, ovfProperties, vmConfigSpec);
+ }
+ }
+
+ checkBootOptions(vmSpec, vmConfigSpec);
+
+ //
+ // Configure VM
+ //
+ if (!vmMo.configureVm(vmConfigSpec)) {
+ throw new Exception("Failed to configure VM before start. vmName: " + vmInternalCSName);
+ }
+ // FR37 TODO reconcile disks now!!! they are configured, check and add them to the returning vmTO
+ if (vmSpec.getType() == VirtualMachine.Type.DomainRouter) {
+ hyperHost.setRestartPriorityForVM(vmMo, DasVmPriority.HIGH.value());
+ }
+
+ // Resizing root disk only when explicit requested by user
+ final Map<String, String> vmDetails = cmd.getVirtualMachine().getDetails();
+ if (rootDiskTO != null && !hasSnapshot && (vmDetails != null && vmDetails.containsKey(ApiConstants.ROOT_DISK_SIZE))) {
+ resizeRootDiskOnVMStart(vmMo, rootDiskTO, hyperHost, context);
+ }
+
+ //
+ // Post Configuration
+ //
+
+ vmMo.setCustomFieldValue(CustomFieldConstants.CLOUD_NIC_MASK, String.valueOf(nicMask));
+ postNvpConfigBeforeStart(vmMo, vmSpec);
+
+ Map<String, Map<String, String>> iqnToData = new HashMap<>();
+
+ postDiskConfigBeforeStart(vmMo, vmSpec, sortedDisks, iqnToData, hyperHost, context, installAsIs);
+
+ //
+ // Power-on VM
+ //
+ if (!vmMo.powerOn()) {
+ throw new Exception("Failed to start VM. vmName: " + vmInternalCSName + " with hostname " + vmNameOnVcenter);
+ }
+
+ if (installAsIs) {
+ // Set disks as the disks path and chain is retrieved from the cloned VM disks
+ cmd.getVirtualMachine().setDisks(disks);
+ }
+
+ StartAnswer startAnswer = new StartAnswer(cmd);
+
+ startAnswer.setIqnToData(iqnToData);
+
+ deleteOldVersionOfTheStartedVM(existingVm, dcMo, vmMo);
+
+ return startAnswer;
+ } catch (Throwable e) {
+ return handleStartFailure(cmd, vmSpec, existingVm, dcMo, e);
+ } finally {
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("finally done with %s", vmwareResource.getGson().toJson(cmd)));
+ }
+ }
+ }
+
+ /**
+ * Modify the specDisks information to match the cloned VM's disks (from vmMo VM)
+ */
+ private void mapSpecDisksToClonedDisks(VirtualMachineMO vmMo, String vmInternalCSName, DiskTO[] specDisks) {
+ try {
+ if (vmMo != null && ArrayUtils.isNotEmpty(specDisks)) {
+ List<VirtualDisk> vmDisks = vmMo.getVirtualDisks();
+ List<DiskTO> sortedDisks = Arrays.asList(sortVolumesByDeviceId(specDisks))
+ .stream()
+ .filter(x -> x.getType() == Volume.Type.ROOT)
+ .collect(Collectors.toList());
+ if (sortedDisks.size() != vmDisks.size()) {
+ LOGGER.error("Different number of root disks spec vs cloned deploy-as-is VM disks: " + sortedDisks.size() + " - " + vmDisks.size());
+ return;
+ }
+ for (int i = 0; i < sortedDisks.size(); i++) {
+ DiskTO specDisk = sortedDisks.get(i);
+ VirtualDisk vmDisk = vmDisks.get(i);
+ DataTO dataVolume = specDisk.getData();
+ if (dataVolume instanceof VolumeObjectTO) {
+ VolumeObjectTO volumeObjectTO = (VolumeObjectTO) dataVolume;
+ if (!volumeObjectTO.getSize().equals(vmDisk.getCapacityInBytes())) {
+ LOGGER.info("Mapped disk size is not the same as the cloned VM disk size: " +
+ volumeObjectTO.getSize() + " - " + vmDisk.getCapacityInBytes());
+ }
+ VirtualDeviceBackingInfo backingInfo = vmDisk.getBacking();
+ if (backingInfo instanceof VirtualDiskFlatVer2BackingInfo) {
+ VirtualDiskFlatVer2BackingInfo backing = (VirtualDiskFlatVer2BackingInfo) backingInfo;
+ String fileName = backing.getFileName();
+ if (StringUtils.isNotBlank(fileName)) {
+ String[] fileNameParts = fileName.split(" ");
+ String datastoreUuid = fileNameParts[0].replace("[", "").replace("]", "");
+ String relativePath = fileNameParts[1].split("/")[1].replace(".vmdk", "");
+ String vmSpecDatastoreUuid = volumeObjectTO.getDataStore().getUuid().replaceAll("-", "");
+ if (!datastoreUuid.equals(vmSpecDatastoreUuid)) {
+ LOGGER.info("Mapped disk datastore UUID is not the same as the cloned VM datastore UUID: " +
+ datastoreUuid + " - " + vmSpecDatastoreUuid);
+ }
+ volumeObjectTO.setPath(relativePath);
+ specDisk.setPath(relativePath);
+ }
+ }
+ }
+ }
+ }
+ } catch (Exception e) {
+ String msg = "Error mapping deploy-as-is VM disks from cloned VM " + vmInternalCSName;
+ LOGGER.error(msg, e);
+ throw new CloudRuntimeException(e);
+ }
+ }
+
+ private StartAnswer handleStartFailure(StartCommand cmd, VirtualMachineTO vmSpec, VirtualMachineData existingVm, DatacenterMO dcMo, Throwable e) {
+ if (e instanceof RemoteException) {
+ LOGGER.warn("Encounter remote exception to vCenter, invalidate VMware session context");
+ vmwareResource.invalidateServiceContext();
+ }
+
+ String msg = "StartCommand failed due to " + VmwareHelper.getExceptionMessage(e);
+ LOGGER.warn(msg, e);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("didn't start VM %s", vmSpec.getName()), e);
+ }
+ StartAnswer startAnswer = new StartAnswer(cmd, msg);
+ if ( e instanceof VmAlreadyExistsInVcenter) {
+ startAnswer.setContextParam("stopRetry", "true");
+ }
+ reRegisterExistingVm(existingVm, dcMo);
+
+ return startAnswer;
+ }
+
+ private void deleteOldVersionOfTheStartedVM(VirtualMachineData existingVm, DatacenterMO dcMo, VirtualMachineMO vmMo) throws Exception {
+ // Since VM was successfully powered-on, if there was an existing VM in a different cluster that was unregistered, delete all the files associated with it.
+ if (existingVm != null && existingVm.vmName != null && existingVm.vmFileLayout != null) {
+ List<String> vmDatastoreNames = new ArrayList<>();
+ for (DatastoreMO vmDatastore : vmMo.getAllDatastores()) {
+ vmDatastoreNames.add(vmDatastore.getName());
+ }
+ // Don't delete files that are in a datastore that is being used by the new VM as well (zone-wide datastore).
+ List<String> skipDatastores = new ArrayList<>();
+ for (DatastoreMO existingDatastore : existingVm.datastores) {
+ if (vmDatastoreNames.contains(existingDatastore.getName())) {
+ skipDatastores.add(existingDatastore.getName());
+ }
+ }
+ vmwareResource.deleteUnregisteredVmFiles(existingVm.vmFileLayout, dcMo, true, skipDatastores);
+ }
+ }
+
+ private int adjustNumberOfCoresPerSocket(VirtualMachineTO vmSpec, VirtualMachineMO vmMo, VirtualMachineConfigSpec vmConfigSpec) throws Exception {
+ // Check for multi-cores per socket settings
+ int numCoresPerSocket = 1;
+ String coresPerSocket = vmSpec.getDetails().get(VmDetailConstants.CPU_CORE_PER_SOCKET);
+ if (coresPerSocket != null) {
+ String apiVersion = HypervisorHostHelper.getVcenterApiVersion(vmMo.getContext());
+ // Property 'numCoresPerSocket' is supported since vSphere API 5.0
+ if (apiVersion.compareTo("5.0") >= 0) {
+ numCoresPerSocket = NumbersUtil.parseInt(coresPerSocket, 1);
+ vmConfigSpec.setNumCoresPerSocket(numCoresPerSocket);
+ }
+ }
+ return numCoresPerSocket;
+ }
+
+ /**
+ // Setup USB devices
+ */
+ private int setupUsbDevicesAndGetCount(String vmInternalCSName, VirtualMachineMO vmMo, String guestOsId, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount)
+ throws Exception {
+ int usbDeviceCount = 0;
+ if (guestOsId.startsWith("darwin")) { //Mac OS
+ VirtualDevice[] devices = vmMo.getMatchedDevices(new Class<?>[] {VirtualUSBController.class});
+ if (devices.length == 0) {
+ LOGGER.debug("No USB Controller device on VM Start. Add USB Controller device for Mac OS VM " + vmInternalCSName);
+
+ //For Mac OS X systems, the EHCI+UHCI controller is enabled by default and is required for USB mouse and keyboard access.
+ VirtualDevice usbControllerDevice = VmwareHelper.prepareUSBControllerDevice();
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ deviceConfigSpecArray[deviceCount].setDevice(usbControllerDevice);
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare USB controller at new device " + vmwareResource.getGson().toJson(deviceConfigSpecArray[deviceCount]));
+
+ deviceCount++;
+ usbDeviceCount++;
+ } else {
+ LOGGER.debug("USB Controller device exists on VM Start for Mac OS VM " + vmInternalCSName);
+ }
+ }
+ return usbDeviceCount;
+ }
+
+ private void createNewVm(VmwareContext context, VirtualMachineTO vmSpec, boolean installAsIs, String vmInternalCSName, String vmNameOnVcenter, Pair<String, String> controllerInfo, Boolean systemVm,
+ VmwareManager mgr, VmwareHypervisorHost hyperHost, String guestOsId, DiskTO[] disks, HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails,
+ DatastoreMO dsRootVolumeIsOn) throws Exception {
+ VirtualMachineMO vmMo;
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = getRootDiskDataStoreDetails(disks, dataStoresDetails);
+
+ assert (vmSpec.getMinSpeed() != null) && (rootDiskDataStoreDetails != null);
+
+ boolean vmFolderExists = rootDiskDataStoreDetails.second().folderExists(String.format("[%s]", rootDiskDataStoreDetails.second().getName()), vmNameOnVcenter);
+ String vmxFileFullPath = dsRootVolumeIsOn.searchFileInSubFolders(vmNameOnVcenter + ".vmx", false, VmwareManager.s_vmwareSearchExcludeFolder.value());
+ if (vmFolderExists && vmxFileFullPath != null) { // VM can be registered only if .vmx is present.
+ registerVm(vmNameOnVcenter, dsRootVolumeIsOn, vmwareResource);
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ if (vmMo != null) {
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Found registered vm " + vmInternalCSName + " at host " + hyperHost.getHyperHostName());
+ }
+ }
+ tearDownVm(vmMo);
+ } else if (installAsIs) {
+// first get all the MORs ManagedObjectReference morPool = hyperHost.getHyperHostOwnerResourcePool();
+// get the base VM vmMo = hyperHost.findVmOnHyperHost(vm.template.getPath());
+// do templateVm.createLinkedClone(vmInternalCSName, morBaseSnapshot, dcMo.getVmFolder(), morPool, morDatastore)
+// or hyperHost....createLinkedOrFullClone(templateVm, volume, dcMo, vmMo, morDatastore, dsMo, vmInternalCSName, morPool);
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ // At this point vmMo points to the cloned VM
+ } else {
+ if (!hyperHost
+ .createBlankVm(vmNameOnVcenter, vmInternalCSName, vmSpec.getCpus(), vmSpec.getMaxSpeed(), vmwareResource.getReservedCpuMHZ(vmSpec), vmSpec.getLimitCpuUse(), (int)(vmSpec.getMaxRam() / Resource.ResourceType.bytesToMiB), vmwareResource.getReservedMemoryMb(vmSpec), guestOsId,
+ rootDiskDataStoreDetails.first(), false, controllerInfo, systemVm)) {
+ throw new Exception("Failed to create VM. vmName: " + vmInternalCSName);
+ }
+ }
+ }
+
+ /**
+ * If a VM with the same name is found in a different cluster in the DC, unregister the old VM and configure a new VM (cold-migration).
+ */
+ private VirtualMachineData unregisterOnOtherClusterButHoldOnToOldVmData(String vmInternalCSName, DatacenterMO dcMo) throws Exception {
+ VirtualMachineMO existingVmInDc = dcMo.findVm(vmInternalCSName);
+ VirtualMachineData existingVm = null;
+ if (existingVmInDc != null) {
+ existingVm = new VirtualMachineData();
+ existingVm.vmName = existingVmInDc.getName();
+ existingVm.vmFileInfo = existingVmInDc.getFileInfo();
+ existingVm.vmFileLayout = existingVmInDc.getFileLayout();
+ existingVm.datastores = existingVmInDc.getAllDatastores();
+ LOGGER.info("Found VM: " + vmInternalCSName + " on a host in a different cluster. Unregistering the exisitng VM.");
+ existingVmInDc.unregisterVm();
+ }
+ return existingVm;
+ }
+
+ private Pair<ManagedObjectReference, DatastoreMO> getRootDiskDataStoreDetails(DiskTO[] disks, HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails) {
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = null;
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ROOT) {
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (managed) {
+ String datastoreName = VmwareResource.getDatastoreName(details.get(DiskTO.IQN));
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(datastoreName);
+ } else {
+ DataStoreTO primaryStore = vol.getData().getDataStore();
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(primaryStore.getUuid());
+ }
+ }
+ }
+ return rootDiskDataStoreDetails;
+ }
+
+ /**
+ * Since VM start failed, if there was an existing VM in a different cluster that was unregistered, register it back.
+ *
+ * @param dcMo is guaranteed to be not null since we have noticed there is an existing VM in the dc (using that mo)
+ */
+ private void reRegisterExistingVm(VirtualMachineData existingVm, DatacenterMO dcMo) {
+ if (existingVm != null && existingVm.vmName != null && existingVm.vmFileInfo != null) {
+ LOGGER.debug("Since VM start failed, registering back an existing VM: " + existingVm.vmName + " that was unregistered");
+ try {
+ DatastoreFile fileInDatastore = new DatastoreFile(existingVm.vmFileInfo.getVmPathName());
+ DatastoreMO existingVmDsMo = new DatastoreMO(dcMo.getContext(), dcMo.findDatastore(fileInDatastore.getDatastoreName()));
+ registerVm(existingVm.vmName, existingVmDsMo, vmwareResource);
+ } catch (Exception ex) {
+ String message = "Failed to register an existing VM: " + existingVm.vmName + " due to " + VmwareHelper.getExceptionMessage(ex);
+ LOGGER.warn(message, ex);
+ }
+ }
+ }
+
+ private void checkIfVmExistsInVcenter(String vmInternalCSName, String vmNameOnVcenter, DatacenterMO dcMo) throws VmAlreadyExistsInVcenter, Exception {
+ // Validate VM name is unique in Datacenter
+ VirtualMachineMO vmInVcenter = dcMo.checkIfVmAlreadyExistsInVcenter(vmNameOnVcenter, vmInternalCSName);
+ if (vmInVcenter != null) {
+ String msg = "VM with name: " + vmNameOnVcenter + " already exists in vCenter.";
+ LOGGER.error(msg);
+ throw new VmAlreadyExistsInVcenter(msg);
+ }
+ }
+
+ private Pair<String, String> getDiskControllerInfo(VirtualMachineTO vmSpec) {
+ String dataDiskController = vmSpec.getDetails().get(VmDetailConstants.DATA_DISK_CONTROLLER);
+ String rootDiskController = vmSpec.getDetails().get(VmDetailConstants.ROOT_DISK_CONTROLLER);
+ // If root disk controller is scsi, then data disk controller would also be scsi instead of using 'osdefault'
+ // This helps avoid mix of different scsi subtype controllers in instance.
+ if (DiskControllerType.osdefault == DiskControllerType.getType(dataDiskController) && DiskControllerType.lsilogic == DiskControllerType.getType(rootDiskController)) {
+ dataDiskController = DiskControllerType.scsi.toString();
+ }
+
+ // Validate the controller types
+ dataDiskController = DiskControllerType.getType(dataDiskController).toString();
+ rootDiskController = DiskControllerType.getType(rootDiskController).toString();
+
+ if (DiskControllerType.getType(rootDiskController) == DiskControllerType.none) {
+ throw new CloudRuntimeException("Invalid root disk controller detected : " + rootDiskController);
+ }
+ if (DiskControllerType.getType(dataDiskController) == DiskControllerType.none) {
+ throw new CloudRuntimeException("Invalid data disk controller detected : " + dataDiskController);
+ }
+
+ return new Pair<>(rootDiskController, dataDiskController);
+ }
+
+ /**
+ * Registers the vm to the inventory given the vmx file.
+ * @param vmName
+ * @param dsMo
+ * @param vmwareResource
+ */
+ private void registerVm(String vmName, DatastoreMO dsMo, VmwareResource vmwareResource) throws Exception {
+
+ //1st param
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(vmwareResource.getServiceContext());
+ ManagedObjectReference dcMor = hyperHost.getHyperHostDatacenter();
+ DatacenterMO dataCenterMo = new DatacenterMO(vmwareResource.getServiceContext(), dcMor);
+ ManagedObjectReference vmFolderMor = dataCenterMo.getVmFolder();
+
+ //2nd param
+ String vmxFilePath = dsMo.searchFileInSubFolders(vmName + ".vmx", false, VmwareManager.s_vmwareSearchExcludeFolder.value());
+
+ // 5th param
+ ManagedObjectReference morPool = hyperHost.getHyperHostOwnerResourcePool();
+
+ ManagedObjectReference morTask = vmwareResource.getServiceContext().getService().registerVMTask(vmFolderMor, vmxFilePath, vmName, false, morPool, hyperHost.getMor());
+ boolean result = vmwareResource.getServiceContext().getVimClient().waitForTask(morTask);
+ if (!result) {
+ throw new Exception("Unable to register vm due to " + TaskMO.getTaskFailureInfo(vmwareResource.getServiceContext(), morTask));
+ } else {
+ vmwareResource.getServiceContext().waitForTaskProgressDone(morTask);
+ }
+
+ }
+
+ // Pair<internal CS name, vCenter display name>
+ private Pair<String, String> composeVmNames(VirtualMachineTO vmSpec) {
+ String vmInternalCSName = vmSpec.getName();
+ String vmNameOnVcenter = vmSpec.getName();
+ if (VmwareResource.instanceNameFlag && vmSpec.getHostName() != null) {
+ vmNameOnVcenter = vmSpec.getHostName();
+ }
+ return new Pair<String, String>(vmInternalCSName, vmNameOnVcenter);
+ }
+
+ private VirtualMachineGuestOsIdentifier translateGuestOsIdentifier(String cpuArchitecture, String guestOs, String cloudGuestOs) {
+ if (cpuArchitecture == null) {
+ LOGGER.warn("CPU arch is not set, default to i386. guest os: " + guestOs);
+ cpuArchitecture = "i386";
+ }
+
+ if (cloudGuestOs == null) {
+ LOGGER.warn("Guest OS mapping name is not set for guest os: " + guestOs);
+ }
+
+ VirtualMachineGuestOsIdentifier identifier = null;
+ try {
+ if (cloudGuestOs != null) {
+ identifier = VirtualMachineGuestOsIdentifier.fromValue(cloudGuestOs);
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Using mapping name : " + identifier.toString());
+ }
+ }
+ } catch (IllegalArgumentException e) {
+ LOGGER.warn("Unable to find Guest OS Identifier in VMware for mapping name: " + cloudGuestOs + ". Continuing with defaults.");
+ }
+ if (identifier != null) {
+ return identifier;
+ }
+
+ if (cpuArchitecture.equalsIgnoreCase("x86_64")) {
+ return VirtualMachineGuestOsIdentifier.OTHER_GUEST_64;
+ }
+ return VirtualMachineGuestOsIdentifier.OTHER_GUEST;
+ }
+
+ private DiskTO[] validateDisks(DiskTO[] disks) {
+ List<DiskTO> validatedDisks = new ArrayList<DiskTO>();
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() != Volume.Type.ISO) {
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ if (primaryStore.getUuid() != null && !primaryStore.getUuid().isEmpty()) {
+ validatedDisks.add(vol);
+ }
+ } else if (vol.getType() == Volume.Type.ISO) {
+ TemplateObjectTO templateTO = (TemplateObjectTO) vol.getData();
+ if (templateTO.getPath() != null && !templateTO.getPath().isEmpty()) {
+ validatedDisks.add(vol);
+ }
+ } else {
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Drop invalid disk option, volumeTO: " + vmwareResource.getGson().toJson(vol));
+ }
+ }
+ }
+ Collections.sort(validatedDisks, (d1, d2) -> d1.getDiskSeq().compareTo(d2.getDiskSeq()));
+ return validatedDisks.toArray(new DiskTO[0]);
+ }
+
+ private HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> inferDatastoreDetailsFromDiskInfo(VmwareHypervisorHost hyperHost, VmwareContext context,
+ DiskTO[] disks, Command cmd) throws Exception {
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> mapIdToMors = new HashMap<>();
+
+ assert (hyperHost != null) && (context != null);
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() != Volume.Type.ISO) {
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ String poolUuid = primaryStore.getUuid();
+
+ if (mapIdToMors.get(poolUuid) == null) {
+ boolean isManaged = false;
+ Map<String, String> details = vol.getDetails();
+
+ if (details != null) {
+ isManaged = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (isManaged) {
+ String iScsiName = details.get(DiskTO.IQN); // details should not be null for managed storage (it may or may not be null for non-managed storage)
+ String datastoreName = VmwareResource.getDatastoreName(iScsiName);
+ ManagedObjectReference morDatastore = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, datastoreName);
+
+ // if the datastore is not present, we need to discover the iSCSI device that will support it,
+ // create the datastore, and create a VMDK file in the datastore
+ if (morDatastore == null) {
+ final String vmdkPath = vmwareResource.getVmdkPath(volumeTO.getPath());
+
+ morDatastore = getStorageProcessor().prepareManagedStorage(context, hyperHost, null, iScsiName,
+ details.get(DiskTO.STORAGE_HOST), Integer.parseInt(details.get(DiskTO.STORAGE_PORT)),
+ vmdkPath,
+ details.get(DiskTO.CHAP_INITIATOR_USERNAME), details.get(DiskTO.CHAP_INITIATOR_SECRET),
+ details.get(DiskTO.CHAP_TARGET_USERNAME), details.get(DiskTO.CHAP_TARGET_SECRET),
+ Long.parseLong(details.get(DiskTO.VOLUME_SIZE)), cmd);
+
+ DatastoreMO dsMo = new DatastoreMO(vmwareResource.getServiceContext(), morDatastore);
+
+ final String datastoreVolumePath;
+
+ if (vmdkPath != null) {
+ datastoreVolumePath = dsMo.getDatastorePath(vmdkPath + VmwareResource.VMDK_EXTENSION);
+ } else {
+ datastoreVolumePath = dsMo.getDatastorePath(dsMo.getName() + VmwareResource.VMDK_EXTENSION);
+ }
+
+ volumeTO.setPath(datastoreVolumePath);
+ vol.setPath(datastoreVolumePath);
+ }
+
+ mapIdToMors.put(datastoreName, new Pair<>(morDatastore, new DatastoreMO(context, morDatastore)));
+ } else {
+ ManagedObjectReference morDatastore = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, poolUuid);
+
+ if (morDatastore == null) {
+ String msg = "Failed to get the mounted datastore for the volume's pool " + poolUuid;
+
+ LOGGER.error(msg);
+
+ throw new Exception(msg);
+ }
+
+ mapIdToMors.put(poolUuid, new Pair<>(morDatastore, new DatastoreMO(context, morDatastore)));
+ }
+ }
+ }
+ }
+
+ return mapIdToMors;
+ }
+
+ private VmwareStorageProcessor getStorageProcessor() {
+ return vmwareResource.getStorageProcessor();
+ }
+
+ private DatastoreMO getDatastoreThatRootDiskIsOn(HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails, DiskTO disks[]) {
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = null;
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ROOT) {
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (managed) {
+ String datastoreName = VmwareResource.getDatastoreName(details.get(DiskTO.IQN));
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(datastoreName);
+
+ break;
+ } else {
+ DataStoreTO primaryStore = vol.getData().getDataStore();
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(primaryStore.getUuid());
+
+ break;
+ }
+ }
+ }
+
+ if (rootDiskDataStoreDetails != null) {
+ return rootDiskDataStoreDetails.second();
+ }
+
+ return null;
+ }
+
+ private void ensureDiskControllers(VirtualMachineMO vmMo, Pair<String, String> controllerInfo) throws Exception {
+ if (vmMo == null) {
+ return;
+ }
+
+ String msg;
+ String rootDiskController = controllerInfo.first();
+ String dataDiskController = controllerInfo.second();
+ String scsiDiskController;
+ String recommendedDiskController = null;
+
+ if (VmwareHelper.isControllerOsRecommended(dataDiskController) || VmwareHelper.isControllerOsRecommended(rootDiskController)) {
+ recommendedDiskController = vmMo.getRecommendedDiskController(null);
+ }
+ scsiDiskController = HypervisorHostHelper.getScsiController(new Pair<String, String>(rootDiskController, dataDiskController), recommendedDiskController);
+ if (scsiDiskController == null) {
+ return;
+ }
+
+ vmMo.getScsiDeviceControllerKeyNoException();
+ // This VM needs SCSI controllers.
+ // Get count of existing scsi controllers. Helps not to attempt to create more than the maximum allowed 4
+ // Get maximum among the bus numbers in use by scsi controllers. Safe to pick maximum, because we always go sequential allocating bus numbers.
+ Ternary<Integer, Integer, DiskControllerType> scsiControllerInfo = vmMo.getScsiControllerInfo();
+ int requiredNumScsiControllers = VmwareHelper.MAX_SCSI_CONTROLLER_COUNT - scsiControllerInfo.first();
+ int availableBusNum = scsiControllerInfo.second() + 1; // method returned current max. bus number
+
+ if (requiredNumScsiControllers == 0) {
+ return;
+ }
+ if (scsiControllerInfo.first() > 0) {
+ // For VMs which already have a SCSI controller, do NOT attempt to add any more SCSI controllers & return the sub type.
+ // For Legacy VMs would have only 1 LsiLogic Parallel SCSI controller, and doesn't require more.
+ // For VMs created post device ordering support, 4 SCSI subtype controllers are ensured during deployment itself. No need to add more.
+ // For fresh VM deployment only, all required controllers should be ensured.
+ return;
+ }
+ ensureScsiDiskControllers(vmMo, scsiDiskController, requiredNumScsiControllers, availableBusNum);
+ }
+
+ private void ensureScsiDiskControllers(VirtualMachineMO vmMo, String scsiDiskController, int requiredNumScsiControllers, int availableBusNum) throws Exception {
+ // Pick the sub type of scsi
+ if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.pvscsi) {
+ if (!vmMo.isPvScsiSupported()) {
+ String msg = "This VM doesn't support Vmware Paravirtual SCSI controller for virtual disks, because the virtual hardware version is less than 7.";
+ throw new Exception(msg);
+ }
+ vmMo.ensurePvScsiDeviceController(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.lsisas1068) {
+ vmMo.ensureLsiLogicSasDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.buslogic) {
+ vmMo.ensureBusLogicDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.lsilogic) {
+ vmMo.ensureLsiLogicDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ }
+ }
+
+ private void tearDownVm(VirtualMachineMO vmMo) throws Exception {
+
+ if (vmMo == null)
+ return;
+
+ boolean hasSnapshot = false;
+ hasSnapshot = vmMo.hasSnapshot();
+ if (!hasSnapshot)
+ vmMo.tearDownDevices(new Class<?>[]{VirtualDisk.class, VirtualEthernetCard.class});
+ else
+ vmMo.tearDownDevices(new Class<?>[]{VirtualEthernetCard.class});
+ vmMo.ensureScsiDeviceController();
+ }
+
+ private static DiskTO getIsoDiskTO(DiskTO[] disks) {
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ISO) {
+ return vol;
+ }
+ }
+ return null;
+ }
+
+ private static DiskTO[] sortVolumesByDeviceId(DiskTO[] volumes) {
+
+ List<DiskTO> listForSort = new ArrayList<DiskTO>();
+ for (DiskTO vol : volumes) {
+ listForSort.add(vol);
+ }
+ Collections.sort(listForSort, new Comparator<DiskTO>() {
+
+ @Override
+ public int compare(DiskTO arg0, DiskTO arg1) {
+ if (arg0.getDiskSeq() < arg1.getDiskSeq()) {
+ return -1;
+ } else if (arg0.getDiskSeq().equals(arg1.getDiskSeq())) {
+ return 0;
+ }
+
+ return 1;
+ }
+ });
+
+ return listForSort.toArray(new DiskTO[0]);
+ }
+
+ private void postDiskConfigBeforeStart(VirtualMachineMO vmMo, VirtualMachineTO vmSpec, DiskTO[] sortedDisks,
+ Map<String, Map<String, String>> iqnToData, VmwareHypervisorHost hyperHost, VmwareContext context, boolean installAsIs)
+ throws Exception {
+ VirtualMachineDiskInfoBuilder diskInfoBuilder = vmMo.getDiskInfoBuilder();
+
+ for (DiskTO vol : sortedDisks) {
+ //TODO: Map existing disks to the ones returned in the answer
+ if (vol.getType() == Volume.Type.ISO)
+ continue;
+
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+
+ VirtualMachineDiskInfo diskInfo = getMatchingExistingDisk(diskInfoBuilder, vol, hyperHost, context);
+ assert (diskInfo != null);
+
+ String[] diskChain = diskInfo.getDiskChain();
+ assert (diskChain.length > 0);
+
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ DatastoreFile file = new DatastoreFile(diskChain[0]);
+
+ if (managed) {
+ DatastoreFile originalFile = new DatastoreFile(volumeTO.getPath());
+
+ if (!file.getFileBaseName().equalsIgnoreCase(originalFile.getFileBaseName())) {
+ if (LOGGER.isInfoEnabled())
+ LOGGER.info("Detected disk-chain top file change on volume: " + volumeTO.getId() + " " + volumeTO.getPath() + " -> " + diskChain[0]);
+ }
+ } else {
+ if (!file.getFileBaseName().equalsIgnoreCase(volumeTO.getPath())) {
+ if (LOGGER.isInfoEnabled())
+ LOGGER.info("Detected disk-chain top file change on volume: " + volumeTO.getId() + " " + volumeTO.getPath() + " -> " + file.getFileBaseName());
+ }
+ }
+
+ VolumeObjectTO volInSpec = getVolumeInSpec(vmSpec, volumeTO);
+
+ if (volInSpec != null) {
+ if (managed) {
+ Map<String, String> data = new HashMap<>();
+
+ String datastoreVolumePath = diskChain[0];
+
+ data.put(StartAnswer.PATH, datastoreVolumePath);
+ data.put(StartAnswer.IMAGE_FORMAT, Storage.ImageFormat.OVA.toString());
+
+ iqnToData.put(details.get(DiskTO.IQN), data);
+
+ vol.setPath(datastoreVolumePath);
+ volumeTO.setPath(datastoreVolumePath);
+ volInSpec.setPath(datastoreVolumePath);
+ } else {
+ volInSpec.setPath(file.getFileBaseName());
+ }
+ volInSpec.setChainInfo(vmwareResource.getGson().toJson(diskInfo));
+ }
+ }
+ }
+
+ private void checkBootOptions(VirtualMachineTO vmSpec, VirtualMachineConfigSpec vmConfigSpec) {
+ String bootMode = null;
+ if (vmSpec.getDetails().containsKey(VmDetailConstants.BOOT_MODE)) {
+ bootMode = vmSpec.getDetails().get(VmDetailConstants.BOOT_MODE);
+ }
+ if (null == bootMode) {
+ bootMode = ApiConstants.BootType.BIOS.toString();
+ }
+
+ setBootOptions(vmSpec, bootMode, vmConfigSpec);
+ }
+
+ private void setBootOptions(VirtualMachineTO vmSpec, String bootMode, VirtualMachineConfigSpec vmConfigSpec) {
+ VirtualMachineBootOptions bootOptions = null;
+ if (StringUtils.isNotBlank(bootMode) && !bootMode.equalsIgnoreCase("bios")) {
+ vmConfigSpec.setFirmware("efi");
+ if (vmSpec.getDetails().containsKey(ApiConstants.BootType.UEFI.toString()) && "secure".equalsIgnoreCase(vmSpec.getDetails().get(ApiConstants.BootType.UEFI.toString()))) {
+ if (bootOptions == null) {
+ bootOptions = new VirtualMachineBootOptions();
+ }
+ bootOptions.setEfiSecureBootEnabled(true);
+ }
+ }
+ if (vmSpec.isEnterHardwareSetup()) {
+ if (bootOptions == null) {
+ bootOptions = new VirtualMachineBootOptions();
+ }
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("configuring VM '%s' to enter hardware setup",vmSpec.getName()));
+ }
+ bootOptions.setEnterBIOSSetup(vmSpec.isEnterHardwareSetup());
+ }
+ if (bootOptions != null) {
+ vmConfigSpec.setBootOptions(bootOptions);
+ }
+ }
+
+ private void resizeRootDiskOnVMStart(VirtualMachineMO vmMo, DiskTO rootDiskTO, VmwareHypervisorHost hyperHost, VmwareContext context) throws Exception {
+ final Pair<VirtualDisk, String> vdisk = vmwareResource.getVirtualDiskInfo(vmMo, VmwareResource.appendFileType(rootDiskTO.getPath(), VmwareResource.VMDK_EXTENSION));
+ assert (vdisk != null);
+
+ Long reqSize = 0L;
+ final VolumeObjectTO volumeTO = ((VolumeObjectTO) rootDiskTO.getData());
+ if (volumeTO != null) {
+ reqSize = volumeTO.getSize() / 1024;
+ }
+ final VirtualDisk disk = vdisk.first();
+ if (reqSize > disk.getCapacityInKB()) {
+ final VirtualMachineDiskInfo diskInfo = getMatchingExistingDisk(vmMo.getDiskInfoBuilder(), rootDiskTO, hyperHost, context);
+ assert (diskInfo != null);
+ final String[] diskChain = diskInfo.getDiskChain();
+
+ if (diskChain != null && diskChain.length > 1) {
+ LOGGER.warn("Disk chain length for the VM is greater than one, this is not supported");
+ throw new CloudRuntimeException("Unsupported VM disk chain length: " + diskChain.length);
+ }
+
+ boolean resizingSupported = false;
+ String deviceBusName = diskInfo.getDiskDeviceBusName();
+ if (deviceBusName != null && (deviceBusName.toLowerCase().contains("scsi") || deviceBusName.toLowerCase().contains("lsi"))) {
+ resizingSupported = true;
+ }
+ if (!resizingSupported) {
+ LOGGER.warn("Resizing of root disk is only support for scsi device/bus, the provide VM's disk device bus name is " + diskInfo.getDiskDeviceBusName());
+ throw new CloudRuntimeException("Unsupported VM root disk device bus: " + diskInfo.getDiskDeviceBusName());
+ }
+
+ disk.setCapacityInKB(reqSize);
+ VirtualMachineConfigSpec vmConfigSpec = new VirtualMachineConfigSpec();
+ VirtualDeviceConfigSpec deviceConfigSpec = new VirtualDeviceConfigSpec();
+ deviceConfigSpec.setDevice(disk);
+ deviceConfigSpec.setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ vmConfigSpec.getDeviceChange().add(deviceConfigSpec);
+ if (!vmMo.configureVm(vmConfigSpec)) {
+ throw new Exception("Failed to configure VM for given root disk size. vmName: " + vmMo.getName());
+ }
+ }
+ }
+
+ private static void postNvpConfigBeforeStart(VirtualMachineMO vmMo, VirtualMachineTO vmSpec) throws Exception {
+ /**
+ * We need to configure the port on the DV switch after the host is
+ * connected. So make this happen between the configure and start of
+ * the VM
+ */
+ int nicIndex = 0;
+ for (NicTO nicTo : sortNicsByDeviceId(vmSpec.getNics())) {
+ if (nicTo.getBroadcastType() == Networks.BroadcastDomainType.Lswitch) {
+ // We need to create a port with a unique vlan and pass the key to the nic device
+ LOGGER.trace("Nic " + nicTo.toString() + " is connected to an NVP logicalswitch");
+ VirtualDevice nicVirtualDevice = vmMo.getNicDeviceByIndex(nicIndex);
+ if (nicVirtualDevice == null) {
+ throw new Exception("Failed to find a VirtualDevice for nic " + nicIndex); //FIXME Generic exceptions are bad
+ }
+ VirtualDeviceBackingInfo backing = nicVirtualDevice.getBacking();
+ if (backing instanceof VirtualEthernetCardDistributedVirtualPortBackingInfo) {
+ // This NIC is connected to a Distributed Virtual Switch
+ VirtualEthernetCardDistributedVirtualPortBackingInfo portInfo = (VirtualEthernetCardDistributedVirtualPortBackingInfo) backing;
+ DistributedVirtualSwitchPortConnection port = portInfo.getPort();
+ String portKey = port.getPortKey();
+ String portGroupKey = port.getPortgroupKey();
+ String dvSwitchUuid = port.getSwitchUuid();
+
+ LOGGER.debug("NIC " + nicTo.toString() + " is connected to dvSwitch " + dvSwitchUuid + " pg " + portGroupKey + " port " + portKey);
+
+ ManagedObjectReference dvSwitchManager = vmMo.getContext().getVimClient().getServiceContent().getDvSwitchManager();
+ ManagedObjectReference dvSwitch = vmMo.getContext().getVimClient().getService().queryDvsByUuid(dvSwitchManager, dvSwitchUuid);
+
+ // Get all ports
+ DistributedVirtualSwitchPortCriteria criteria = new DistributedVirtualSwitchPortCriteria();
+ criteria.setInside(true);
+ criteria.getPortgroupKey().add(portGroupKey);
+ List<DistributedVirtualPort> dvPorts = vmMo.getContext().getVimClient().getService().fetchDVPorts(dvSwitch, criteria);
+
+ DistributedVirtualPort vmDvPort = null;
+ List<Integer> usedVlans = new ArrayList<Integer>();
+ for (DistributedVirtualPort dvPort : dvPorts) {
+ // Find the port for this NIC by portkey
+ if (portKey.equals(dvPort.getKey())) {
+ vmDvPort = dvPort;
+ }
+ VMwareDVSPortSetting settings = (VMwareDVSPortSetting) dvPort.getConfig().getSetting();
+ VmwareDistributedVirtualSwitchVlanIdSpec vlanId = (VmwareDistributedVirtualSwitchVlanIdSpec) settings.getVlan();
+ LOGGER.trace("Found port " + dvPort.getKey() + " with vlan " + vlanId.getVlanId());
+ if (vlanId.getVlanId() > 0 && vlanId.getVlanId() < 4095) {
+ usedVlans.add(vlanId.getVlanId());
+ }
+ }
+
+ if (vmDvPort == null) {
+ throw new Exception("Empty port list from dvSwitch for nic " + nicTo.toString());
+ }
+
+ DVPortConfigInfo dvPortConfigInfo = vmDvPort.getConfig();
+ VMwareDVSPortSetting settings = (VMwareDVSPortSetting) dvPortConfigInfo.getSetting();
+
+ VmwareDistributedVirtualSwitchVlanIdSpec vlanId = (VmwareDistributedVirtualSwitchVlanIdSpec) settings.getVlan();
+ BoolPolicy blocked = settings.getBlocked();
+ if (blocked.isValue() == Boolean.TRUE) {
+ LOGGER.trace("Port is blocked, set a vlanid and unblock");
+ DVPortConfigSpec dvPortConfigSpec = new DVPortConfigSpec();
+ VMwareDVSPortSetting edittedSettings = new VMwareDVSPortSetting();
+ // Unblock
+ blocked.setValue(Boolean.FALSE);
+ blocked.setInherited(Boolean.FALSE);
+ edittedSettings.setBlocked(blocked);
+ // Set vlan
+ int i;
+ for (i = 1; i < 4095; i++) {
+ if (!usedVlans.contains(i))
+ break;
+ }
+ vlanId.setVlanId(i); // FIXME should be a determined
+ // based on usage
+ vlanId.setInherited(false);
+ edittedSettings.setVlan(vlanId);
+
+ dvPortConfigSpec.setSetting(edittedSettings);
+ dvPortConfigSpec.setOperation("edit");
+ dvPortConfigSpec.setKey(portKey);
+ List<DVPortConfigSpec> dvPortConfigSpecs = new ArrayList<DVPortConfigSpec>();
+ dvPortConfigSpecs.add(dvPortConfigSpec);
+ ManagedObjectReference task = vmMo.getContext().getVimClient().getService().reconfigureDVPortTask(dvSwitch, dvPortConfigSpecs);
+ if (!vmMo.getContext().getVimClient().waitForTask(task)) {
+ throw new Exception("Failed to configure the dvSwitch port for nic " + nicTo.toString());
+ }
+ LOGGER.debug("NIC " + nicTo.toString() + " connected to vlan " + i);
+ } else {
+ LOGGER.trace("Port already configured and set to vlan " + vlanId.getVlanId());
+ }
+ } else if (backing instanceof VirtualEthernetCardNetworkBackingInfo) {
+ // This NIC is connected to a Virtual Switch
+ // Nothing to do
+ } else if (backing instanceof VirtualEthernetCardOpaqueNetworkBackingInfo) {
+ //if NSX API VERSION >= 4.2, connect to br-int (nsx.network), do not create portgroup else previous behaviour
+ //OK, connected to OpaqueNetwork
+ } else {
+ LOGGER.error("nic device backing is of type " + backing.getClass().getName());
+ throw new Exception("Incompatible backing for a VirtualDevice for nic " + nicIndex); //FIXME Generic exceptions are bad
+ }
+ }
+ nicIndex++;
+ }
+ }
+
+ private VirtualMachineDiskInfo getMatchingExistingDisk(VirtualMachineDiskInfoBuilder diskInfoBuilder, DiskTO vol, VmwareHypervisorHost hyperHost, VmwareContext context)
+ throws Exception {
+ if (diskInfoBuilder != null) {
+ VolumeObjectTO volume = (VolumeObjectTO) vol.getData();
+
+ String dsName = null;
+ String diskBackingFileBaseName = null;
+
+ Map<String, String> details = vol.getDetails();
+ boolean isManaged = details != null && Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+
+ if (isManaged) {
+ String iScsiName = details.get(DiskTO.IQN);
+
+ // if the storage is managed, iScsiName should not be null
+ dsName = VmwareResource.getDatastoreName(iScsiName);
+
+ diskBackingFileBaseName = new DatastoreFile(volume.getPath()).getFileBaseName();
+ } else {
+ ManagedObjectReference morDs = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, volume.getDataStore().getUuid());
+ DatastoreMO dsMo = new DatastoreMO(context, morDs);
+
+ dsName = dsMo.getName();
+
+ diskBackingFileBaseName = volume.getPath();
+ }
+
+ VirtualMachineDiskInfo diskInfo = diskInfoBuilder.getDiskInfoByBackingFileBaseName(diskBackingFileBaseName, dsName);
+ if (diskInfo != null) {
+ LOGGER.info("Found existing disk info from volume path: " + volume.getPath());
+ return diskInfo;
+ } else {
+ String chainInfo = volume.getChainInfo();
+ if (chainInfo != null) {
+ VirtualMachineDiskInfo infoInChain = vmwareResource.getGson().fromJson(chainInfo, VirtualMachineDiskInfo.class);
+ if (infoInChain != null) {
+ String[] disks = infoInChain.getDiskChain();
+ if (disks.length > 0) {
+ for (String diskPath : disks) {
+ DatastoreFile file = new DatastoreFile(diskPath);
+ diskInfo = diskInfoBuilder.getDiskInfoByBackingFileBaseName(file.getFileBaseName(), dsName);
+ if (diskInfo != null) {
+ LOGGER.info("Found existing disk from chain info: " + diskPath);
+ return diskInfo;
+ }
+ }
+ }
+
+ if (diskInfo == null) {
+ diskInfo = diskInfoBuilder.getDiskInfoByDeviceBusName(infoInChain.getDiskDeviceBusName());
+ if (diskInfo != null) {
+ LOGGER.info("Found existing disk from from chain device bus information: " + infoInChain.getDiskDeviceBusName());
+ return diskInfo;
+ }
+ }
+ }
+ }
+ }
+ }
+
+ return null;
+ }
+
+ private static VolumeObjectTO getVolumeInSpec(VirtualMachineTO vmSpec, VolumeObjectTO srcVol) {
+ for (DiskTO disk : vmSpec.getDisks()) {
+ if (disk.getData() instanceof VolumeObjectTO) {
+ VolumeObjectTO vol = (VolumeObjectTO) disk.getData();
+ if (vol.getId() == srcVol.getId())
+ return vol;
+ }
+ }
+
+ return null;
+ }
+
+ /**
+ * Generate the mac sequence from the nics.
+ */
+ protected String generateMacSequence(NicTO[] nics) {
+ if (nics.length == 0) {
+ return "";
+ }
+
+ StringBuffer sbMacSequence = new StringBuffer();
+ for (NicTO nicTo : sortNicsByDeviceId(nics)) {
+ sbMacSequence.append(nicTo.getMac()).append("|");
+ }
+ if (!sbMacSequence.toString().isEmpty()) {
+ sbMacSequence.deleteCharAt(sbMacSequence.length() - 1); //Remove extra '|' char appended at the end
+ }
+
+ return sbMacSequence.toString();
+ }
+
+ static NicTO[] sortNicsByDeviceId(NicTO[] nics) {
+
+ List<NicTO> listForSort = new ArrayList<NicTO>();
+ for (NicTO nic : nics) {
+ listForSort.add(nic);
+ }
+ Collections.sort(listForSort, new Comparator<NicTO>() {
+
+ @Override
+ public int compare(NicTO arg0, NicTO arg1) {
+ if (arg0.getDeviceId() < arg1.getDeviceId()) {
+ return -1;
+ } else if (arg0.getDeviceId() == arg1.getDeviceId()) {
+ return 0;
+ }
+
+ return 1;
+ }
+ });
+
+ return listForSort.toArray(new NicTO[0]);
+ }
+
+ private static void configBasicExtraOption(List<OptionValue> extraOptions, VirtualMachineTO vmSpec) {
+ OptionValue newVal = new OptionValue();
+ newVal.setKey("machine.id");
+ newVal.setValue(vmSpec.getBootArgs());
+ extraOptions.add(newVal);
+
+ newVal = new OptionValue();
+ newVal.setKey("devices.hotplug");
+ newVal.setValue("true");
+ extraOptions.add(newVal);
+ }
+
+ private static void configNvpExtraOption(List<OptionValue> extraOptions, VirtualMachineTO vmSpec, Map<String, String> nicUuidToDvSwitchUuid) {
+ /**
+ * Extra Config : nvp.vm-uuid = uuid
+ * - Required for Nicira NVP integration
+ */
+ OptionValue newVal = new OptionValue();
+ newVal.setKey("nvp.vm-uuid");
+ newVal.setValue(vmSpec.getUuid());
+ extraOptions.add(newVal);
+
+ /**
+ * Extra Config : nvp.iface-id.<num> = uuid
+ * - Required for Nicira NVP integration
+ */
+ int nicNum = 0;
+ for (NicTO nicTo : sortNicsByDeviceId(vmSpec.getNics())) {
+ if (nicTo.getUuid() != null) {
+ newVal = new OptionValue();
+ newVal.setKey("nvp.iface-id." + nicNum);
+ newVal.setValue(nicTo.getUuid());
+ extraOptions.add(newVal);
+ }
+ nicNum++;
+ }
+ }
+
+ private static void configCustomExtraOption(List<OptionValue> extraOptions, VirtualMachineTO vmSpec) {
+ // we no longer to validation anymore
+ for (Map.Entry<String, String> entry : vmSpec.getDetails().entrySet()) {
+ if (entry.getKey().equalsIgnoreCase(VmDetailConstants.BOOT_MODE)) {
+ continue;
+ }
+ OptionValue newVal = new OptionValue();
+ newVal.setKey(entry.getKey());
+ newVal.setValue(entry.getValue());
+ extraOptions.add(newVal);
+ }
+ }
+
+ private class VmAlreadyExistsInVcenter extends Exception {
+ public VmAlreadyExistsInVcenter(String msg) {
+ }
+ }
+
+ private class VirtualMachineData {
+ String vmName = null;
+ VirtualMachineFileInfo vmFileInfo = null;
+ VirtualMachineFileLayoutEx vmFileLayout = null;
+ List<DatastoreMO> datastores = new ArrayList<>();
+ }
+
+ private class VirtualMachineRecycler {
+ private String vmInternalCSName;
+ private Pair<String, String> controllerInfo;
+ private Boolean systemVm;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private DiskControllerType systemVmScsiControllerType;
+ private int firstScsiControllerBusNum;
+ private int numScsiControllerForSystemVm;
+ private VirtualMachineDiskInfoBuilder diskInfoBuilder;
+ private boolean hasSnapshot;
+ private VirtualDevice[] nicDevices;
+
+ public VirtualMachineRecycler(String vmInternalCSName, Pair<String, String> controllerInfo, Boolean systemVm, VmwareHypervisorHost hyperHost, VirtualMachineMO vmMo,
+ DiskControllerType systemVmScsiControllerType, int firstScsiControllerBusNum, int numScsiControllerForSystemVm) {
+ this.vmInternalCSName = vmInternalCSName;
+ this.controllerInfo = controllerInfo;
+ this.systemVm = systemVm;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.systemVmScsiControllerType = systemVmScsiControllerType;
+ this.firstScsiControllerBusNum = firstScsiControllerBusNum;
+ this.numScsiControllerForSystemVm = numScsiControllerForSystemVm;
+ }
+
+ public VirtualMachineDiskInfoBuilder getDiskInfoBuilder() {
+ return diskInfoBuilder;
+ }
+
+ public boolean isHasSnapshot() {
+ return hasSnapshot;
+ }
+
+ public VirtualMachineRecycler invoke() throws Exception {
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Found vm " + vmInternalCSName + " at other host, relocate to " + hyperHost.getHyperHostName());
+ }
+
+ takeVmFromOtherHyperHost(hyperHost, vmInternalCSName);
+
+ if (VmwareResource.getVmPowerState(vmMo) != VirtualMachine.PowerState.PowerOff)
+ vmMo.safePowerOff(vmwareResource.getShutdownWaitMs());
+
+ diskInfoBuilder = vmMo.getDiskInfoBuilder();
+ hasSnapshot = vmMo.hasSnapshot();
+ nicDevices = vmMo.getNicDevices();
+ if (!hasSnapshot)
+ vmMo.tearDownDevices(new Class<?>[] {VirtualDisk.class, VirtualEthernetCard.class});
+ else
+ vmMo.tearDownDevices(new Class<?>[] {VirtualEthernetCard.class});
+
+ if (systemVm) {
+ // System volumes doesn't require more than 1 SCSI controller as there is no requirement for data volumes.
+ ensureScsiDiskControllers(vmMo, systemVmScsiControllerType.toString(), numScsiControllerForSystemVm, firstScsiControllerBusNum);
+ } else {
+ ensureDiskControllers(vmMo, controllerInfo);
+ }
+ return this;
+ }
+
+ private VirtualMachineMO takeVmFromOtherHyperHost(VmwareHypervisorHost hyperHost, String vmName) throws Exception {
+
+ VirtualMachineMO vmMo = hyperHost.findVmOnPeerHyperHost(vmName);
+ if (vmMo != null) {
+ ManagedObjectReference morTargetPhysicalHost = hyperHost.findMigrationTarget(vmMo);
+ if (morTargetPhysicalHost == null) {
+ String msg = "VM " + vmName + " is on other host and we have no resource available to migrate and start it here";
+ LOGGER.error(msg);
+ throw new Exception(msg);
+ }
+
+ if (!vmMo.relocate(morTargetPhysicalHost)) {
+ String msg = "VM " + vmName + " is on other host and we failed to relocate it here";
+ LOGGER.error(msg);
+ throw new Exception(msg);
+ }
+
+ return vmMo;
+ }
+ return null;
+ }
+
+ public VirtualDevice[] getNicDevices() {
+ return nicDevices;
+ }
+ }
+
+ private class PrepareSytemVMPatchISOMethod {
+ private VmwareManager mgr;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int i;
+ private int ideUnitNumber;
+
+ public PrepareSytemVMPatchISOMethod(VmwareManager mgr, VmwareHypervisorHost hyperHost, VirtualMachineMO vmMo, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int i, int ideUnitNumber) {
+ this.mgr = mgr;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.i = i;
+ this.ideUnitNumber = ideUnitNumber;
+ }
+
+ public int getI() {
+ return i;
+ }
+
+ public int getIdeUnitNumber() {
+ return ideUnitNumber;
+ }
+
+ public PrepareSytemVMPatchISOMethod invoke() throws Exception {
+ // attach ISO (for patching of system VM)
+ DatastoreMO secDsMo = getDatastoreMOForSecStore(mgr, hyperHost);
+
+ deviceConfigSpecArray[i] = new VirtualDeviceConfigSpec();
+ Pair<VirtualDevice, Boolean> isoInfo = VmwareHelper
+ .prepareIsoDevice(vmMo, String.format("[%s] systemvm/%s", secDsMo.getName(), mgr.getSystemVMIsoFileNameOnDatastore()), secDsMo.getMor(), true, true,
+ ideUnitNumber++, i + 1);
+ deviceConfigSpecArray[i].setDevice(isoInfo.first());
+ if (isoInfo.second()) {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at new device " + vmwareResource.getGson().toJson(isoInfo.first()));
+ deviceConfigSpecArray[i].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+ } else {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+ deviceConfigSpecArray[i].setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ }
+ i++;
+ return this;
+ }
+
+ }
+ private DatastoreMO getDatastoreMOForSecStore(VmwareManager mgr, VmwareHypervisorHost hyperHost) throws Exception {
+ Pair<String, Long> secStoreUrlAndId = mgr.getSecondaryStorageStoreUrlAndId(Long.parseLong(vmwareResource.getDcId()));
+ String secStoreUrl = secStoreUrlAndId.first();
+ Long secStoreId = secStoreUrlAndId.second();
+ if (secStoreUrl == null) {
+ String msg = "secondary storage for dc " + vmwareResource.getDcId() + " is not ready yet?";
+ throw new Exception(msg);
+ }
+ mgr.prepareSecondaryStorageStore(secStoreUrl, secStoreId);
+
+ ManagedObjectReference morSecDs = vmwareResource.prepareSecondaryDatastoreOnHost(secStoreUrl);
+ if (morSecDs == null) {
+ String msg = "Failed to prepare secondary storage on host, secondary store url: " + secStoreUrl;
+ throw new Exception(msg);
+ }
+ return new DatastoreMO(hyperHost.getContext(), morSecDs);
+ }
+
+ /**
+ // Setup ROOT/DATA disk devices
+ */
+ private class DiskSetup {
+ private VirtualMachineTO vmSpec;
+ private DiskTO rootDiskTO;
+ private Pair<String, String> controllerInfo;
+ private VmwareContext context;
+ private DatacenterMO dcMo;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private DiskTO[] disks;
+ private HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails;
+ private VirtualMachineDiskInfoBuilder diskInfoBuilder;
+ private boolean hasSnapshot;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int deviceCount;
+ private int ideUnitNumber;
+ private int scsiUnitNumber;
+ private int ideControllerKey;
+ private int scsiControllerKey;
+ private DiskTO[] sortedDisks;
+ private boolean installAsIs;
+
+ public DiskSetup(VirtualMachineTO vmSpec, DiskTO rootDiskTO, Pair<String, String> controllerInfo, VmwareContext context, DatacenterMO dcMo, VmwareHypervisorHost hyperHost,
+ VirtualMachineMO vmMo, DiskTO[] disks, HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails, VirtualMachineDiskInfoBuilder diskInfoBuilder,
+ boolean hasSnapshot, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount, int ideUnitNumber, int scsiUnitNumber, int ideControllerKey, int scsiControllerKey, boolean installAsIs) {
+ this.vmSpec = vmSpec;
+ this.rootDiskTO = rootDiskTO;
+ this.controllerInfo = controllerInfo;
+ this.context = context;
+ this.dcMo = dcMo;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.disks = disks;
+ this.dataStoresDetails = dataStoresDetails;
+ this.diskInfoBuilder = diskInfoBuilder;
+ this.hasSnapshot = hasSnapshot;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.deviceCount = deviceCount;
+ this.ideUnitNumber = ideUnitNumber;
+ this.scsiUnitNumber = scsiUnitNumber;
+ this.ideControllerKey = ideControllerKey;
+ this.scsiControllerKey = scsiControllerKey;
+ this.installAsIs = installAsIs;
+ }
+
+ public DiskTO getRootDiskTO() {
+ return rootDiskTO;
+ }
+
+ public int getDeviceCount() {
+ return deviceCount;
+ }
+
+ public DiskTO[] getSortedDisks() {
+ return sortedDisks;
+ }
+
+ public DiskSetup invoke() throws Exception {
+ int controllerKey;
+ sortedDisks = sortVolumesByDeviceId(disks);
+ for (DiskTO vol : sortedDisks) {
+ if (vol.getType() == Volume.Type.ISO || installAsIs) {
+ continue;
+ }
+
+ VirtualMachineDiskInfo matchingExistingDisk = getMatchingExistingDisk(diskInfoBuilder, vol, hyperHost, context);
+ controllerKey = getDiskController(matchingExistingDisk, vol, vmSpec, ideControllerKey, scsiControllerKey);
+ String diskController = getDiskController(vmMo, matchingExistingDisk, vol, controllerInfo);
+
+ if (DiskControllerType.getType(diskController) == DiskControllerType.osdefault) {
+ diskController = vmMo.getRecommendedDiskController(null);
+ }
+ if (DiskControllerType.getType(diskController) == DiskControllerType.ide) {
+ controllerKey = vmMo.getIDEControllerKey(ideUnitNumber);
+ if (vol.getType() == Volume.Type.DATADISK) {
+ // Could be result of flip due to user configured setting or "osdefault" for data disks
+ // Ensure maximum of 2 data volumes over IDE controller, 3 includeing root volume
+ if (vmMo.getNumberOfVirtualDisks() > 3) {
+ throw new CloudRuntimeException(
+ "Found more than 3 virtual disks attached to this VM [" + vmMo.getVmName() + "]. Unable to implement the disks over " + diskController + " controller, as maximum number of devices supported over IDE controller is 4 includeing CDROM device.");
+ }
+ }
+ } else {
+ if (VmwareHelper.isReservedScsiDeviceNumber(scsiUnitNumber)) {
+ scsiUnitNumber++;
+ }
+
+ controllerKey = vmMo.getScsiDiskControllerKeyNoException(diskController, scsiUnitNumber);
+ if (controllerKey == -1) {
+ // This may happen for ROOT legacy VMs which doesn't have recommended disk controller when global configuration parameter 'vmware.root.disk.controller' is set to "osdefault"
+ // Retrieve existing controller and use.
+ Ternary<Integer, Integer, DiskControllerType> vmScsiControllerInfo = vmMo.getScsiControllerInfo();
+ DiskControllerType existingControllerType = vmScsiControllerInfo.third();
+ controllerKey = vmMo.getScsiDiskControllerKeyNoException(existingControllerType.toString(), scsiUnitNumber);
+ }
+ }
+ if (!hasSnapshot) {
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+
+ VolumeObjectTO volumeTO = (VolumeObjectTO)vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+ String iScsiName = null;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ iScsiName = details.get(DiskTO.IQN);
+ }
+
+ // if the storage is managed, iScsiName should not be null
+ String datastoreName = managed ? VmwareResource.getDatastoreName(iScsiName) : primaryStore.getUuid();
+ Pair<ManagedObjectReference, DatastoreMO> volumeDsDetails = dataStoresDetails.get(datastoreName);
+
+ assert (volumeDsDetails != null);
+
+ String[] diskChain = syncDiskChain(dcMo, vmMo, vmSpec, vol, matchingExistingDisk, dataStoresDetails);
+
+ int deviceNumber = -1;
+ if (controllerKey == vmMo.getIDEControllerKey(ideUnitNumber)) {
+ deviceNumber = ideUnitNumber % VmwareHelper.MAX_ALLOWED_DEVICES_IDE_CONTROLLER;
+ ideUnitNumber++;
+ } else {
+ deviceNumber = scsiUnitNumber % VmwareHelper.MAX_ALLOWED_DEVICES_SCSI_CONTROLLER;
+ scsiUnitNumber++;
+ }
+ VirtualDevice device = VmwareHelper.prepareDiskDevice(vmMo, null, controllerKey, diskChain, volumeDsDetails.first(), deviceNumber, deviceCount + 1);
+ if (vol.getType() == Volume.Type.ROOT)
+ rootDiskTO = vol;
+ deviceConfigSpecArray[deviceCount].setDevice(device);
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare volume at new device " + vmwareResource.getGson().toJson(device));
+
+ deviceCount++;
+ } else {
+ if (controllerKey == vmMo.getIDEControllerKey(ideUnitNumber))
+ ideUnitNumber++;
+ else
+ scsiUnitNumber++;
+ }
+ }
+ return this;
+ }
+
+ private int getDiskController(VirtualMachineDiskInfo matchingExistingDisk, DiskTO vol, VirtualMachineTO vmSpec, int ideControllerKey, int scsiControllerKey) {
+
+ int controllerKey;
+ if (matchingExistingDisk != null) {
+ LOGGER.info("Chose disk controller based on existing information: " + matchingExistingDisk.getDiskDeviceBusName());
+ if (matchingExistingDisk.getDiskDeviceBusName().startsWith("ide"))
+ return ideControllerKey;
+ else
+ return scsiControllerKey;
+ }
+
+ if (vol.getType() == Volume.Type.ROOT) {
+ Map<String, String> vmDetails = vmSpec.getDetails();
+ if (vmDetails != null && vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER) != null) {
+ if (vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER).equalsIgnoreCase("scsi")) {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> scsi, based on root disk controller settings: "
+ + vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER));
+ controllerKey = scsiControllerKey;
+ } else {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> ide, based on root disk controller settings: "
+ + vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER));
+ controllerKey = ideControllerKey;
+ }
+ } else {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> scsi. due to null root disk controller setting");
+ controllerKey = scsiControllerKey;
+ }
+
+ } else {
+ // DATA volume always use SCSI device
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> scsi");
+ controllerKey = scsiControllerKey;
+ }
+
+ return controllerKey;
+ }
+
+ private String getDiskController(VirtualMachineMO vmMo, VirtualMachineDiskInfo matchingExistingDisk, DiskTO vol, Pair<String, String> controllerInfo) throws Exception {
+ int controllerKey;
+ DiskControllerType controllerType = DiskControllerType.none;
+ if (matchingExistingDisk != null) {
+ String currentBusName = matchingExistingDisk.getDiskDeviceBusName();
+ if (currentBusName != null) {
+ LOGGER.info("Chose disk controller based on existing information: " + currentBusName);
+ if (currentBusName.startsWith("ide")) {
+ controllerType = DiskControllerType.ide;
+ } else if (currentBusName.startsWith("scsi")) {
+ controllerType = DiskControllerType.scsi;
+ }
+ }
+ if (controllerType == DiskControllerType.scsi || controllerType == DiskControllerType.none) {
+ Ternary<Integer, Integer, DiskControllerType> vmScsiControllerInfo = vmMo.getScsiControllerInfo();
+ controllerType = vmScsiControllerInfo.third();
+ }
+ return controllerType.toString();
+ }
+
+ if (vol.getType() == Volume.Type.ROOT) {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> " + controllerInfo.first()
+ + ", based on root disk controller settings at global configuration setting.");
+ return controllerInfo.first();
+ } else {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> " + controllerInfo.second()
+ + ", based on default data disk controller setting i.e. Operating system recommended."); // Need to bring in global configuration setting & template level setting.
+ return controllerInfo.second();
+ }
+ }
+
+ // return the finalized disk chain for startup, from top to bottom
+ private String[] syncDiskChain(DatacenterMO dcMo, VirtualMachineMO vmMo, VirtualMachineTO vmSpec, DiskTO vol, VirtualMachineDiskInfo diskInfo,
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails) throws Exception {
+
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ Map<String, String> details = vol.getDetails();
+ boolean isManaged = false;
+ String iScsiName = null;
+
+ if (details != null) {
+ isManaged = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ iScsiName = details.get(DiskTO.IQN);
+ }
+
+ // if the storage is managed, iScsiName should not be null
+ String datastoreName = isManaged ? VmwareResource.getDatastoreName(iScsiName) : primaryStore.getUuid();
+ Pair<ManagedObjectReference, DatastoreMO> volumeDsDetails = dataStoresDetails.get(datastoreName);
+
+ if (volumeDsDetails == null) {
+ throw new Exception("Primary datastore " + primaryStore.getUuid() + " is not mounted on host.");
+ }
+
+ DatastoreMO dsMo = volumeDsDetails.second();
+
+ // we will honor vCenter's meta if it exists
+ if (diskInfo != null) {
+ // to deal with run-time upgrade to maintain the new datastore folder structure
+ String disks[] = diskInfo.getDiskChain();
+ for (int i = 0; i < disks.length; i++) {
+ DatastoreFile file = new DatastoreFile(disks[i]);
+ if (!isManaged && file.getDir() != null && file.getDir().isEmpty()) {
+ LOGGER.info("Perform run-time datastore folder upgrade. sync " + disks[i] + " to VM folder");
+ disks[i] = VmwareStorageLayoutHelper.syncVolumeToVmDefaultFolder(dcMo, vmMo.getName(), dsMo, file.getFileBaseName(), VmwareManager.s_vmwareSearchExcludeFolder.value());
+ }
+ }
+ return disks;
+ }
+
+ final String datastoreDiskPath;
+
+ if (isManaged) {
+ String vmdkPath = new DatastoreFile(volumeTO.getPath()).getFileBaseName();
+
+ if (volumeTO.getVolumeType() == Volume.Type.ROOT) {
+ if (vmdkPath == null) {
+ vmdkPath = volumeTO.getName();
+ }
+
+ datastoreDiskPath = VmwareStorageLayoutHelper.syncVolumeToVmDefaultFolder(dcMo, vmMo.getName(), dsMo, vmdkPath);
+ } else {
+ if (vmdkPath == null) {
+ vmdkPath = dsMo.getName();
+ }
+
+ datastoreDiskPath = dsMo.getDatastorePath(vmdkPath + VmwareResource.VMDK_EXTENSION);
+ }
+ } else {
+ datastoreDiskPath = VmwareStorageLayoutHelper.syncVolumeToVmDefaultFolder(dcMo, vmMo.getName(), dsMo, volumeTO.getPath(), VmwareManager.s_vmwareSearchExcludeFolder.value());
+ }
+
+ if (!dsMo.fileExists(datastoreDiskPath)) {
+ LOGGER.warn("Volume " + volumeTO.getId() + " does not seem to exist on datastore, out of sync? path: " + datastoreDiskPath);
+ }
+
+ return new String[]{datastoreDiskPath};
+ }
+ }
+
+ /**
+ // Setup NIC devices
+ */
+ private class NicSetup {
+ private StartCommand cmd;
+ private VirtualMachineTO vmSpec;
+ private String vmInternalCSName;
+ private VmwareContext context;
+ private VmwareManager mgr;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private NicTO[] nics;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int deviceCount;
+ private int nicMask;
+ private int nicCount;
+ private Map<String, String> nicUuidToDvSwitchUuid;
+ private VirtualDevice[] nicDevices;
+
+ public NicSetup(StartCommand cmd, VirtualMachineTO vmSpec, String vmInternalCSName, VmwareContext context, VmwareManager mgr, VmwareHypervisorHost hyperHost,
+ VirtualMachineMO vmMo, NicTO[] nics, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount, VirtualDevice[] nicDevices) {
+ this.cmd = cmd;
+ this.vmSpec = vmSpec;
+ this.vmInternalCSName = vmInternalCSName;
+ this.context = context;
+ this.mgr = mgr;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.nics = nics;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.deviceCount = deviceCount;
+ this.nicDevices = nicDevices;
+ }
+
+ public int getDeviceCount() {
+ return deviceCount;
+ }
+
+ public int getNicMask() {
+ return nicMask;
+ }
+
+ public int getNicCount() {
+ return nicCount;
+ }
+
+ public Map<String, String> getNicUuidToDvSwitchUuid() {
+ return nicUuidToDvSwitchUuid;
+ }
+
+ public NicSetup invoke() throws Exception {
+ VirtualDevice nic;
+ nicMask = 0;
+ nicCount = 0;
+
+ if (vmSpec.getType() == VirtualMachine.Type.DomainRouter) {
+ doDomainRouterSetup();
+ }
+
+ VirtualEthernetCardType nicDeviceType = VirtualEthernetCardType.valueOf(vmSpec.getDetails().get(VmDetailConstants.NIC_ADAPTER));
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("VM " + vmInternalCSName + " will be started with NIC device type: " + nicDeviceType);
+ }
+
+ NiciraNvpApiVersion.logNiciraApiVersion();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.debug(String.format("deciding for VM '%s' to use orchestrated NICs or as is.", vmInternalCSName));
+ }
+ nicUuidToDvSwitchUuid = new HashMap<>();
+ LOGGER.info(String.format("adding %d nics to VM '%s'", nics.length, vmInternalCSName));
+ for (NicTO nicTo : sortNicsByDeviceId(nics)) {
+ LOGGER.info("Prepare NIC device based on NicTO: " + vmwareResource.getGson().toJson(nicTo));
+
+ boolean configureVServiceInNexus = (nicTo.getType() == Networks.TrafficType.Guest) && (vmSpec.getDetails().containsKey("ConfigureVServiceInNexus"));
+ VirtualMachine.Type vmType = cmd.getVirtualMachine().getType();
+ Pair<ManagedObjectReference, String> networkInfo = vmwareResource.prepareNetworkFromNicInfo(vmMo.getRunningHost(), nicTo, configureVServiceInNexus, vmType);
+ if ((nicTo.getBroadcastType() != Networks.BroadcastDomainType.Lswitch) || (nicTo.getBroadcastType() == Networks.BroadcastDomainType.Lswitch && NiciraNvpApiVersion.isApiVersionLowerThan("4.2"))) {
+ if (VmwareHelper.isDvPortGroup(networkInfo.first())) {
+ String dvSwitchUuid;
+ ManagedObjectReference dcMor = hyperHost.getHyperHostDatacenter();
+ DatacenterMO dataCenterMo = new DatacenterMO(context, dcMor);
+ ManagedObjectReference dvsMor = dataCenterMo.getDvSwitchMor(networkInfo.first());
+ dvSwitchUuid = dataCenterMo.getDvSwitchUuid(dvsMor);
+ LOGGER.info("Preparing NIC device on dvSwitch : " + dvSwitchUuid);
+ nic = VmwareHelper.prepareDvNicDevice(vmMo, networkInfo.first(), nicDeviceType, networkInfo.second(), dvSwitchUuid, nicTo.getMac(), deviceCount + 1, true, true);
+ if (nicTo.getUuid() != null) {
+ nicUuidToDvSwitchUuid.put(nicTo.getUuid(), dvSwitchUuid);
+ }
+ } else {
+ LOGGER.info("Preparing NIC device on network " + networkInfo.second());
+ nic = VmwareHelper.prepareNicDevice(vmMo, networkInfo.first(), nicDeviceType, networkInfo.second(), nicTo.getMac(), deviceCount + 1, true, true);
+ }
+ } else {
+ //if NSX API VERSION >= 4.2, connect to br-int (nsx.network), do not create portgroup else previous behaviour
+ nic = VmwareHelper.prepareNicOpaque(vmMo, nicDeviceType, networkInfo.second(), nicTo.getMac(), deviceCount + 1, true, true);
+ }
+
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ deviceConfigSpecArray[deviceCount].setDevice(nic);
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare NIC at new device " + vmwareResource.getGson().toJson(deviceConfigSpecArray[deviceCount]));
+
+ // this is really a hacking for DomR, upon DomR startup, we will reset all the NIC allocation after eth3
+ if (nicCount < 3)
+ nicMask |= (1 << nicCount);
+
+ deviceCount++;
+ nicCount++;
+ }
+ return this;
+ }
+
+ private void doDomainRouterSetup() throws Exception {
+ int extraPublicNics = mgr.getRouterExtraPublicNics();
+ if (extraPublicNics > 0 && vmSpec.getDetails().containsKey("PeerRouterInstanceName")) {
+ //Set identical MAC address for RvR on extra public interfaces
+ String peerRouterInstanceName = vmSpec.getDetails().get("PeerRouterInstanceName");
+
+ VirtualMachineMO peerVmMo = hyperHost.findVmOnHyperHost(peerRouterInstanceName);
+ if (peerVmMo == null) {
+ peerVmMo = hyperHost.findVmOnPeerHyperHost(peerRouterInstanceName);
+ }
+
+ if (peerVmMo != null) {
+ String oldMacSequence = generateMacSequence(nics);
+
+ for (int nicIndex = nics.length - extraPublicNics; nicIndex < nics.length; nicIndex++) {
+ VirtualDevice nicDevice = peerVmMo.getNicDeviceByIndex(nics[nicIndex].getDeviceId());
+ if (nicDevice != null) {
+ String mac = ((VirtualEthernetCard)nicDevice).getMacAddress();
+ if (mac != null) {
+ LOGGER.info("Use same MAC as previous RvR, the MAC is " + mac + " for extra NIC with device id: " + nics[nicIndex].getDeviceId());
+ nics[nicIndex].setMac(mac);
+ }
+ }
+ }
+
+ if (!StringUtils.isBlank(vmSpec.getBootArgs())) {
+ String newMacSequence = generateMacSequence(nics);
+ vmSpec.setBootArgs(vmwareResource.replaceNicsMacSequenceInBootArgs(oldMacSequence, newMacSequence, vmSpec));
+ }
+ }
+ }
+ }
+ }
+
+ private class IsoSetup {
+ private VirtualMachineTO vmSpec;
+ private VmwareManager mgr;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private DiskTO[] disks;
+ private DiskTO volIso;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int deviceCount;
+ private int ideUnitNumber;
+
+ public IsoSetup(VirtualMachineTO vmSpec, VmwareManager mgr, VmwareHypervisorHost hyperHost, VirtualMachineMO vmMo, DiskTO[] disks, DiskTO volIso,
+ VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount, int ideUnitNumber) {
+ this.vmSpec = vmSpec;
+ this.mgr = mgr;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.disks = disks;
+ this.volIso = volIso;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.deviceCount = deviceCount;
+ this.ideUnitNumber = ideUnitNumber;
+ }
+
+ public int getDeviceCount() {
+ return deviceCount;
+ }
+
+ public int getIdeUnitNumber() {
+ return ideUnitNumber;
+ }
+
+ public IsoSetup invoke() throws Exception {
+ //
+ // Setup ISO device
+ //
+
+ // vAPP ISO
+ // FR37 the native deploy mechs should create this for us
+ if (vmSpec.getOvfProperties() != null) {
+ if (LOGGER.isTraceEnabled()) {
+ // FR37 TODO add more usefull info (if we keep this bit
+ LOGGER.trace("adding iso for properties for 'xxx'");
+ }
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ Pair<VirtualDevice, Boolean> isoInfo = VmwareHelper.prepareIsoDevice(vmMo, null, null, true, true, ideUnitNumber++, deviceCount + 1);
+ deviceConfigSpecArray[deviceCount].setDevice(isoInfo.first());
+ if (isoInfo.second()) {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare vApp ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+ } else {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare vApp ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ }
+ deviceCount++;
+ }
+
+ // prepare systemvm patch ISO
+ if (vmSpec.getType() != VirtualMachine.Type.User) {
+ PrepareSytemVMPatchISOMethod prepareSytemVm = new PrepareSytemVMPatchISOMethod(mgr, hyperHost, vmMo, deviceConfigSpecArray, deviceCount, ideUnitNumber);
+ prepareSytemVm.invoke();
+ deviceCount = prepareSytemVm.getI();
+ ideUnitNumber = prepareSytemVm.getIdeUnitNumber();
+ } else {
+ // Note: we will always plug a CDROM device
+ if (volIso != null) {
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ISO) {
+
+ TemplateObjectTO iso = (TemplateObjectTO)vol.getData();
+
+ if (iso.getPath() != null && !iso.getPath().isEmpty()) {
+ DataStoreTO imageStore = iso.getDataStore();
+ if (!(imageStore instanceof NfsTO)) {
+ LOGGER.debug("unsupported protocol");
+ throw new Exception("unsupported protocol");
+ }
+ NfsTO nfsImageStore = (NfsTO)imageStore;
+ String isoPath = nfsImageStore.getUrl() + File.separator + iso.getPath();
+ Pair<String, ManagedObjectReference> isoDatastoreInfo = getIsoDatastoreInfo(hyperHost, isoPath);
+ assert (isoDatastoreInfo != null);
+ assert (isoDatastoreInfo.second() != null);
+
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ Pair<VirtualDevice, Boolean> isoInfo = VmwareHelper
+ .prepareIsoDevice(vmMo, isoDatastoreInfo.first(), isoDatastoreInfo.second(), true, true, ideUnitNumber++, deviceCount + 1);
+ deviceConfigSpecArray[deviceCount].setDevice(isoInfo.first());
+ if (isoInfo.second()) {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at new device " + vmwareResource.getGson().toJson(isoInfo.first()));
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+ } else {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ }
+ }
+ deviceCount++;
+ }
+ }
+ } else {
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ Pair<VirtualDevice, Boolean> isoInfo = VmwareHelper.prepareIsoDevice(vmMo, null, null, true, true, ideUnitNumber++, deviceCount + 1);
+ deviceConfigSpecArray[deviceCount].setDevice(isoInfo.first());
+ if (isoInfo.second()) {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+ } else {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ }
+ deviceCount++;
+ }
+ }
+ return this;
+ }
+
+ // isoUrl sample content :
+ // nfs://192.168.10.231/export/home/kelven/vmware-test/secondary/template/tmpl/2/200//200-2-80f7ee58-6eff-3a2d-bcb0-59663edf6d26.iso
+ private Pair<String, ManagedObjectReference> getIsoDatastoreInfo(VmwareHypervisorHost hyperHost, String isoUrl) throws Exception {
+
+ assert (isoUrl != null);
+ int isoFileNameStartPos = isoUrl.lastIndexOf("/");
+ if (isoFileNameStartPos < 0) {
+ throw new Exception("Invalid ISO path info");
+ }
+
+ String isoFileName = isoUrl.substring(isoFileNameStartPos);
+
+ int templateRootPos = isoUrl.indexOf("template/tmpl");
+ templateRootPos = (templateRootPos < 0 ? isoUrl.indexOf(ConfigDrive.CONFIGDRIVEDIR) : templateRootPos);
+ if (templateRootPos < 0) {
+ throw new Exception("Invalid ISO path info");
+ }
+
+ String storeUrl = isoUrl.substring(0, templateRootPos - 1);
+ String isoPath = isoUrl.substring(templateRootPos, isoFileNameStartPos);
+
+ ManagedObjectReference morDs = vmwareResource.prepareSecondaryDatastoreOnHost(storeUrl);
+ DatastoreMO dsMo = new DatastoreMO(vmwareResource.getServiceContext(), morDs);
+
+ return new Pair<String, ManagedObjectReference>(String.format("[%s] %s%s", dsMo.getName(), isoPath, isoFileName), morDs);
+ }
+ }
+
+ private class PrepareRunningVMForConfiguration {
+ private String vmInternalCSName;
+ private Pair<String, String> controllerInfo;
+ private Boolean systemVm;
+ private VirtualMachineMO vmMo;
+ private DiskControllerType systemVmScsiControllerType;
+ private int firstScsiControllerBusNum;
+ private int numScsiControllerForSystemVm;
+ private VirtualMachineDiskInfoBuilder diskInfoBuilder;
+ private VirtualDevice[] nicDevices;
+ private boolean hasSnapshot;
+
+ public PrepareRunningVMForConfiguration(String vmInternalCSName, Pair<String, String> controllerInfo, Boolean systemVm, VirtualMachineMO vmMo,
+ DiskControllerType systemVmScsiControllerType, int firstScsiControllerBusNum, int numScsiControllerForSystemVm) {
+ this.vmInternalCSName = vmInternalCSName;
+ this.controllerInfo = controllerInfo;
+ this.systemVm = systemVm;
+ this.vmMo = vmMo;
+ this.systemVmScsiControllerType = systemVmScsiControllerType;
+ this.firstScsiControllerBusNum = firstScsiControllerBusNum;
+ this.numScsiControllerForSystemVm = numScsiControllerForSystemVm;
+ }
+
+ public VirtualMachineDiskInfoBuilder getDiskInfoBuilder() {
+ return diskInfoBuilder;
+ }
+
+ public VirtualDevice[] getNicDevices() {
+ return nicDevices;
+ }
+
+ public boolean isHasSnapshot() {
+ return hasSnapshot;
+ }
+
+ public PrepareRunningVMForConfiguration invoke() throws Exception {
+ LOGGER.info("VM " + vmInternalCSName + " already exists, tear down devices for reconfiguration");
+ if (VmwareResource.getVmPowerState(vmMo) != VirtualMachine.PowerState.PowerOff) {
+ vmMo.safePowerOff(vmwareResource.getShutdownWaitMs());
+ }
+
+ // retrieve disk information before we tear down
+ diskInfoBuilder = vmMo.getDiskInfoBuilder();
+ hasSnapshot = vmMo.hasSnapshot();
+ nicDevices = vmMo.getNicDevices();
+ // FR37 - only tear nics, and add nics per the provided nics list
+ if (LOGGER.isTraceEnabled()) {
+ String netMsg = "tearing down networks :";
+ for (VirtualDevice nic : vmMo.getNicDevices()) {
+ netMsg += nic.getDeviceInfo().getLabel()+":";
+ }
+ LOGGER.trace(netMsg);
+ }
+ // FR37 save vmMo.getNicDevices() to ensure recreation?
Review comment:
if this is handled otherwise we can delete, else needs addressing
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/StartCommandExecutor.java
##########
@@ -0,0 +1,2247 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.hypervisor.vmware.resource;
+
+import java.io.File;
+import java.rmi.RemoteException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.cloud.agent.api.to.DataTO;
+import com.vmware.vim25.VirtualDiskFlatVer2BackingInfo;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.storage.configdrive.ConfigDrive;
+import org.apache.cloudstack.storage.to.TemplateObjectTO;
+import org.apache.cloudstack.storage.to.VolumeObjectTO;
+import org.apache.cloudstack.utils.volume.VirtualMachineDiskInfo;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.commons.lang.ArrayUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.log4j.Logger;
+
+import com.cloud.agent.api.Command;
+import com.cloud.agent.api.StartAnswer;
+import com.cloud.agent.api.StartCommand;
+import com.cloud.agent.api.storage.OVFPropertyTO;
+import com.cloud.agent.api.to.DataStoreTO;
+import com.cloud.agent.api.to.DiskTO;
+import com.cloud.agent.api.to.NfsTO;
+import com.cloud.agent.api.to.NicTO;
+import com.cloud.agent.api.to.VirtualMachineTO;
+import com.cloud.configuration.Resource;
+import com.cloud.hypervisor.vmware.VmwareResourceException;
+import com.cloud.hypervisor.vmware.manager.VmwareManager;
+import com.cloud.hypervisor.vmware.mo.CustomFieldConstants;
+import com.cloud.hypervisor.vmware.mo.DatacenterMO;
+import com.cloud.hypervisor.vmware.mo.DatastoreFile;
+import com.cloud.hypervisor.vmware.mo.DatastoreMO;
+import com.cloud.hypervisor.vmware.mo.DiskControllerType;
+import com.cloud.hypervisor.vmware.mo.HostMO;
+import com.cloud.hypervisor.vmware.mo.HypervisorHostHelper;
+import com.cloud.hypervisor.vmware.mo.TaskMO;
+import com.cloud.hypervisor.vmware.mo.VirtualEthernetCardType;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineDiskInfoBuilder;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineMO;
+import com.cloud.hypervisor.vmware.mo.VmwareHypervisorHost;
+import com.cloud.hypervisor.vmware.util.VmwareContext;
+import com.cloud.hypervisor.vmware.util.VmwareHelper;
+import com.cloud.network.Networks;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Volume;
+import com.cloud.storage.resource.VmwareStorageLayoutHelper;
+import com.cloud.storage.resource.VmwareStorageProcessor;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nicira.nvp.plugin.NiciraNvpApiVersion;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.VmDetailConstants;
+import com.vmware.vim25.BoolPolicy;
+import com.vmware.vim25.DVPortConfigInfo;
+import com.vmware.vim25.DVPortConfigSpec;
+import com.vmware.vim25.DasVmPriority;
+import com.vmware.vim25.DistributedVirtualPort;
+import com.vmware.vim25.DistributedVirtualSwitchPortConnection;
+import com.vmware.vim25.DistributedVirtualSwitchPortCriteria;
+import com.vmware.vim25.ManagedObjectReference;
+import com.vmware.vim25.OptionValue;
+import com.vmware.vim25.VMwareDVSPortSetting;
+import com.vmware.vim25.VirtualDevice;
+import com.vmware.vim25.VirtualDeviceBackingInfo;
+import com.vmware.vim25.VirtualDeviceConfigSpec;
+import com.vmware.vim25.VirtualDeviceConfigSpecOperation;
+import com.vmware.vim25.VirtualDisk;
+import com.vmware.vim25.VirtualEthernetCard;
+import com.vmware.vim25.VirtualEthernetCardDistributedVirtualPortBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardNetworkBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardOpaqueNetworkBackingInfo;
+import com.vmware.vim25.VirtualMachineBootOptions;
+import com.vmware.vim25.VirtualMachineConfigSpec;
+import com.vmware.vim25.VirtualMachineFileInfo;
+import com.vmware.vim25.VirtualMachineFileLayoutEx;
+import com.vmware.vim25.VirtualMachineGuestOsIdentifier;
+import com.vmware.vim25.VirtualUSBController;
+import com.vmware.vim25.VmConfigInfo;
+import com.vmware.vim25.VmwareDistributedVirtualSwitchVlanIdSpec;
+
+class StartCommandExecutor {
+ private static final Logger LOGGER = Logger.getLogger(StartCommandExecutor.class);
+
+ private final VmwareResource vmwareResource;
+
+ public StartCommandExecutor(VmwareResource vmwareResource) {
+ this.vmwareResource = vmwareResource;
+ }
+
+ // FR37 the monster blob god method: reduce to 320 so far and counting
+ protected StartAnswer execute(StartCommand cmd) {
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Executing resource StartCommand: " + vmwareResource.getGson().toJson(cmd));
+ }
+
+ VirtualMachineTO vmSpec = cmd.getVirtualMachine();
+
+ VirtualMachineData existingVm = null;
+
+ Pair<String, String> names = composeVmNames(vmSpec);
+ String vmInternalCSName = names.first();
+ String vmNameOnVcenter = names.second();
+
+ DiskTO rootDiskTO = null;
+
+ Pair<String, String> controllerInfo = getDiskControllerInfo(vmSpec);
+
+ Boolean systemVm = vmSpec.getType().isUsedBySystem();
+
+ VmwareContext context = vmwareResource.getServiceContext();
+ DatacenterMO dcMo = null;
+ try {
+ VmwareManager mgr = context.getStockObject(VmwareManager.CONTEXT_STOCK_NAME);
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(context);
+ dcMo = new DatacenterMO(hyperHost.getContext(), hyperHost.getHyperHostDatacenter());
+
+ // checkIfVmExistsInVcenter(vmInternalCSName, vmNameOnVcenter, dcMo);
+ // FR37 - We expect VM to be already cloned and available at this point
+ VirtualMachineMO vmMo = dcMo.findVm(vmInternalCSName);
+ if (vmMo == null) {
+ vmMo = dcMo.findVm(vmNameOnVcenter);
+ }
+
+ DiskTO[] specDisks = vmSpec.getDisks();
+ boolean installAsIs = StringUtils.isNotEmpty(vmSpec.getTemplateLocation());
+ // FR37 if startcommand contains enough info: a template url/-location and flag; deploy OVA as is
+ if (vmMo == null && installAsIs) {
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("deploying OVA from %s as is", vmSpec.getTemplateLocation()));
+ }
+ getStorageProcessor().cloneVMFromTemplate(vmSpec.getTemplateName(), vmInternalCSName, vmSpec.getTemplatePrimaryStoreUuid());
+ vmMo = dcMo.findVm(vmInternalCSName);
+ mapSpecDisksToClonedDisks(vmMo, vmInternalCSName, specDisks);
+ }
+
+ // VM may not have been on the same host, relocate to expected host
+ if (vmMo != null) {
+ vmMo.relocate(hyperHost.getMor());
+ // Get updated MO
+ vmMo = hyperHost.findVmOnHyperHost(vmMo.getVmName());
+ }
+
+ String guestOsId = translateGuestOsIdentifier(vmSpec.getArch(), vmSpec.getOs(), vmSpec.getPlatformEmulator()).value();
+ DiskTO[] disks = validateDisks(specDisks);
+ NicTO[] nics = vmSpec.getNics();
+
+ // FIXME: disks logic here, why is disks/volumes during copy not set with pool ID?
+ // FR37 TODO if deployasis a new VM no datastores are known (yet) and we need to get the data store from the tvmspec content library / template location
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails = inferDatastoreDetailsFromDiskInfo(hyperHost, context, disks, cmd);
+ if ((dataStoresDetails == null) || (dataStoresDetails.isEmpty())) {
+ String msg = "Unable to locate datastore details of the volumes to be attached";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ // FR37 - this may not be necessary the cloned VM will have disks and knowledge of datastore paths/location?
+ // throw new Exception(msg);
+ }
+
+ // FR37 - this may need checking, if the first datastore is the right one - ideally it should be datastore where first disk is hosted
+ DatastoreMO dsRootVolumeIsOn = null; //
+ if (! installAsIs) {
+ dsRootVolumeIsOn = getDatastoreThatRootDiskIsOn(dataStoresDetails, disks);
+
+ if (dsRootVolumeIsOn == null) {
+ String msg = "Unable to locate datastore details of root volume";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ throw new VmwareResourceException(msg);
+ }
+ }
+
+ VirtualMachineDiskInfoBuilder diskInfoBuilder = null;
+ VirtualDevice[] nicDevices = null;
+ DiskControllerType systemVmScsiControllerType = DiskControllerType.lsilogic;
+ int firstScsiControllerBusNum = 0;
+ int numScsiControllerForSystemVm = 1;
+ boolean hasSnapshot = false;
+ if (vmMo != null) {
+ PrepareRunningVMForConfiguration prepareRunningVMForConfiguration = new PrepareRunningVMForConfiguration(vmInternalCSName, controllerInfo, systemVm, vmMo,
+ systemVmScsiControllerType, firstScsiControllerBusNum, numScsiControllerForSystemVm);
+ prepareRunningVMForConfiguration.invoke();
+ diskInfoBuilder = prepareRunningVMForConfiguration.getDiskInfoBuilder();
+ nicDevices = prepareRunningVMForConfiguration.getNicDevices();
+ hasSnapshot = prepareRunningVMForConfiguration.isHasSnapshot();
+ } else {
+ ManagedObjectReference morDc = hyperHost.getHyperHostDatacenter();
+ assert (morDc != null);
+
+ vmMo = hyperHost.findVmOnPeerHyperHost(vmInternalCSName);
+ if (vmMo != null) {
+ VirtualMachineRecycler virtualMachineRecycler = new VirtualMachineRecycler(vmInternalCSName, controllerInfo, systemVm, hyperHost, vmMo,
+ systemVmScsiControllerType, firstScsiControllerBusNum, numScsiControllerForSystemVm);
+ virtualMachineRecycler.invoke();
+ diskInfoBuilder = virtualMachineRecycler.getDiskInfoBuilder();
+ hasSnapshot = virtualMachineRecycler.isHasSnapshot();
+ nicDevices = virtualMachineRecycler.getNicDevices();
+ } else {
+ // FR37 we just didn't find a VM by name 'vmInternalCSName', so why is this here?
+ existingVm = unregisterOnOtherClusterButHoldOnToOldVmData(vmInternalCSName, dcMo);
+
+ createNewVm(context, vmSpec, installAsIs, vmInternalCSName, vmNameOnVcenter, controllerInfo, systemVm, mgr, hyperHost, guestOsId, disks, dataStoresDetails,
+ dsRootVolumeIsOn);
+ }
+
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ if (vmMo == null) {
+ throw new Exception("Failed to find the newly create or relocated VM. vmName: " + vmInternalCSName);
+ }
+ }
+ // vmMo should now be a stopped VM on the intended host
+ // The number of disks changed must be 0 for install as is, as the VM is a clone
+ int disksChanges = !installAsIs ? disks.length : 0;
+ int totalChangeDevices = disksChanges + nics.length;
+ int hackDeviceCount = 0;
+ if (diskInfoBuilder != null) {
+ hackDeviceCount += diskInfoBuilder.getDiskCount();
+ }
+ hackDeviceCount += nicDevices == null ? 0 : nicDevices.length;
+ // vApp cdrom device
+ // HACK ALERT: ovf properties might not be the only or defining feature of vApps; needs checking
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("current count(s) desired: %d/ found:%d. now adding device to device count for vApp config ISO", totalChangeDevices, hackDeviceCount));
+ }
+ if (vmSpec.getOvfProperties() != null) {
+ totalChangeDevices++;
+ }
+
+ DiskTO volIso = null;
+ if (vmSpec.getType() != VirtualMachine.Type.User) {
+ // system VM needs a patch ISO
+ totalChangeDevices++;
+ } else {
+ volIso = getIsoDiskTO(disks);
+ if (volIso == null)
+ totalChangeDevices++;
+ }
+
+ VirtualMachineConfigSpec vmConfigSpec = new VirtualMachineConfigSpec();
+
+ VmwareHelper.setBasicVmConfig(vmConfigSpec, vmSpec.getCpus(), vmSpec.getMaxSpeed(), vmwareResource.getReservedCpuMHZ(vmSpec), (int)(vmSpec.getMaxRam() / (1024 * 1024)),
+ vmwareResource.getReservedMemoryMb(vmSpec), guestOsId, vmSpec.getLimitCpuUse());
+
+ int numCoresPerSocket = adjustNumberOfCoresPerSocket(vmSpec, vmMo, vmConfigSpec);
+
+ // Check for hotadd settings
+ vmConfigSpec.setMemoryHotAddEnabled(vmMo.isMemoryHotAddSupported(guestOsId));
+
+ String hostApiVersion = ((HostMO)hyperHost).getHostAboutInfo().getApiVersion();
+ if (numCoresPerSocket > 1 && hostApiVersion.compareTo("5.0") < 0) {
+ LOGGER.warn("Dynamic scaling of CPU is not supported for Virtual Machines with multi-core vCPUs in case of ESXi hosts 4.1 and prior. Hence CpuHotAdd will not be"
+ + " enabled for Virtual Machine: " + vmInternalCSName);
+ vmConfigSpec.setCpuHotAddEnabled(false);
+ } else {
+ vmConfigSpec.setCpuHotAddEnabled(vmMo.isCpuHotAddSupported(guestOsId));
+ }
+
+ vmwareResource.configNestedHVSupport(vmMo, vmSpec, vmConfigSpec);
+
+ VirtualDeviceConfigSpec[] deviceConfigSpecArray = new VirtualDeviceConfigSpec[totalChangeDevices];
+ int deviceCount = 0;
+ int ideUnitNumber = 0;
+ int scsiUnitNumber = 0;
+ int ideControllerKey = vmMo.getIDEDeviceControllerKey();
+ int scsiControllerKey = vmMo.getScsiDeviceControllerKeyNoException();
+
+ IsoSetup isoSetup = new IsoSetup(vmSpec, mgr, hyperHost, vmMo, disks, volIso, deviceConfigSpecArray, deviceCount, ideUnitNumber);
+ isoSetup.invoke();
+ deviceCount = isoSetup.getDeviceCount();
+ ideUnitNumber = isoSetup.getIdeUnitNumber();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("setting up %d disks with root from %s", diskInfoBuilder.getDiskCount(), vmwareResource.getGson().toJson(rootDiskTO)));
+ }
+
+ DiskSetup diskSetup = new DiskSetup(vmSpec, rootDiskTO, controllerInfo, context, dcMo, hyperHost, vmMo, disks, dataStoresDetails, diskInfoBuilder, hasSnapshot,
+ deviceConfigSpecArray, deviceCount, ideUnitNumber, scsiUnitNumber, ideControllerKey, scsiControllerKey, installAsIs);
+ diskSetup.invoke();
+
+ rootDiskTO = diskSetup.getRootDiskTO();
+ deviceCount = diskSetup.getDeviceCount();
+ DiskTO[] sortedDisks = diskSetup.getSortedDisks();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("seting up %d disks with root from %s", sortedDisks.length, vmwareResource.getGson().toJson(rootDiskTO)));
+ }
+
+ deviceCount += setupUsbDevicesAndGetCount(vmInternalCSName, vmMo, guestOsId, deviceConfigSpecArray, deviceCount);
+
+ NicSetup nicSetup = new NicSetup(cmd, vmSpec, vmInternalCSName, context, mgr, hyperHost, vmMo, nics, deviceConfigSpecArray, deviceCount, nicDevices);
+ nicSetup.invoke();
+
+ deviceCount = nicSetup.getDeviceCount();
+ int nicMask = nicSetup.getNicMask();
+ int nicCount = nicSetup.getNicCount();
+ Map<String, String> nicUuidToDvSwitchUuid = nicSetup.getNicUuidToDvSwitchUuid();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("device count: %d; nic count %d", deviceCount, nicCount));
+ }
+ for (int j = 0; j < deviceCount; j++)
+ vmConfigSpec.getDeviceChange().add(deviceConfigSpecArray[j]);
+
+ //
+ // Setup VM options
+ //
+
+ // pass boot arguments through machine.id & perform customized options to VMX
+ ArrayList<OptionValue> extraOptions = new ArrayList<>();
+ configBasicExtraOption(extraOptions, vmSpec);
+ configNvpExtraOption(extraOptions, vmSpec, nicUuidToDvSwitchUuid);
+ configCustomExtraOption(extraOptions, vmSpec);
+
+ // config for NCC
+ VirtualMachine.Type vmType = cmd.getVirtualMachine().getType();
+ if (vmType.equals(VirtualMachine.Type.NetScalerVm)) {
+ NicTO mgmtNic = vmSpec.getNics()[0];
+ OptionValue option = new OptionValue();
+ option.setKey("machine.id");
+ option.setValue("ip=" + mgmtNic.getIp() + "&netmask=" + mgmtNic.getNetmask() + "&gateway=" + mgmtNic.getGateway());
+ extraOptions.add(option);
+ }
+
+ // config VNC
+ String keyboardLayout = null;
+ if (vmSpec.getDetails() != null)
+ keyboardLayout = vmSpec.getDetails().get(VmDetailConstants.KEYBOARD);
+ vmConfigSpec.getExtraConfig().addAll(
+ Arrays.asList(vmwareResource.configureVnc(extraOptions.toArray(new OptionValue[0]), hyperHost, vmInternalCSName, vmSpec.getVncPassword(), keyboardLayout)));
+
+ // config video card
+ vmwareResource.configureVideoCard(vmMo, vmSpec, vmConfigSpec);
+
+ // Set OVF properties (if available)
+ Pair<String, List<OVFPropertyTO>> ovfPropsMap = vmSpec.getOvfProperties();
+ VmConfigInfo templateVappConfig;
+ List<OVFPropertyTO> ovfProperties;
+ if (ovfPropsMap != null) {
+ String vmTemplate = ovfPropsMap.first();
+ LOGGER.info("Find VM template " + vmTemplate);
+ VirtualMachineMO vmTemplateMO = dcMo.findVm(vmTemplate);
+ templateVappConfig = vmTemplateMO.getConfigInfo().getVAppConfig();
+ ovfProperties = ovfPropsMap.second();
+ // Set OVF properties (if available)
+ if (CollectionUtils.isNotEmpty(ovfProperties)) {
+ LOGGER.info("Copying OVF properties from template and setting them to the values the user provided");
+ vmwareResource.copyVAppConfigsFromTemplate(templateVappConfig, ovfProperties, vmConfigSpec);
+ }
+ }
+
+ checkBootOptions(vmSpec, vmConfigSpec);
+
+ //
+ // Configure VM
+ //
+ if (!vmMo.configureVm(vmConfigSpec)) {
+ throw new Exception("Failed to configure VM before start. vmName: " + vmInternalCSName);
+ }
+ // FR37 TODO reconcile disks now!!! they are configured, check and add them to the returning vmTO
+ if (vmSpec.getType() == VirtualMachine.Type.DomainRouter) {
+ hyperHost.setRestartPriorityForVM(vmMo, DasVmPriority.HIGH.value());
+ }
+
+ // Resizing root disk only when explicit requested by user
+ final Map<String, String> vmDetails = cmd.getVirtualMachine().getDetails();
+ if (rootDiskTO != null && !hasSnapshot && (vmDetails != null && vmDetails.containsKey(ApiConstants.ROOT_DISK_SIZE))) {
+ resizeRootDiskOnVMStart(vmMo, rootDiskTO, hyperHost, context);
+ }
+
+ //
+ // Post Configuration
+ //
+
+ vmMo.setCustomFieldValue(CustomFieldConstants.CLOUD_NIC_MASK, String.valueOf(nicMask));
+ postNvpConfigBeforeStart(vmMo, vmSpec);
+
+ Map<String, Map<String, String>> iqnToData = new HashMap<>();
+
+ postDiskConfigBeforeStart(vmMo, vmSpec, sortedDisks, iqnToData, hyperHost, context, installAsIs);
+
+ //
+ // Power-on VM
+ //
+ if (!vmMo.powerOn()) {
+ throw new Exception("Failed to start VM. vmName: " + vmInternalCSName + " with hostname " + vmNameOnVcenter);
+ }
+
+ if (installAsIs) {
+ // Set disks as the disks path and chain is retrieved from the cloned VM disks
+ cmd.getVirtualMachine().setDisks(disks);
+ }
+
+ StartAnswer startAnswer = new StartAnswer(cmd);
+
+ startAnswer.setIqnToData(iqnToData);
+
+ deleteOldVersionOfTheStartedVM(existingVm, dcMo, vmMo);
+
+ return startAnswer;
+ } catch (Throwable e) {
+ return handleStartFailure(cmd, vmSpec, existingVm, dcMo, e);
+ } finally {
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("finally done with %s", vmwareResource.getGson().toJson(cmd)));
+ }
+ }
+ }
+
+ /**
+ * Modify the specDisks information to match the cloned VM's disks (from vmMo VM)
+ */
+ private void mapSpecDisksToClonedDisks(VirtualMachineMO vmMo, String vmInternalCSName, DiskTO[] specDisks) {
+ try {
+ if (vmMo != null && ArrayUtils.isNotEmpty(specDisks)) {
+ List<VirtualDisk> vmDisks = vmMo.getVirtualDisks();
+ List<DiskTO> sortedDisks = Arrays.asList(sortVolumesByDeviceId(specDisks))
+ .stream()
+ .filter(x -> x.getType() == Volume.Type.ROOT)
+ .collect(Collectors.toList());
+ if (sortedDisks.size() != vmDisks.size()) {
+ LOGGER.error("Different number of root disks spec vs cloned deploy-as-is VM disks: " + sortedDisks.size() + " - " + vmDisks.size());
+ return;
+ }
+ for (int i = 0; i < sortedDisks.size(); i++) {
+ DiskTO specDisk = sortedDisks.get(i);
+ VirtualDisk vmDisk = vmDisks.get(i);
+ DataTO dataVolume = specDisk.getData();
+ if (dataVolume instanceof VolumeObjectTO) {
+ VolumeObjectTO volumeObjectTO = (VolumeObjectTO) dataVolume;
+ if (!volumeObjectTO.getSize().equals(vmDisk.getCapacityInBytes())) {
+ LOGGER.info("Mapped disk size is not the same as the cloned VM disk size: " +
+ volumeObjectTO.getSize() + " - " + vmDisk.getCapacityInBytes());
+ }
+ VirtualDeviceBackingInfo backingInfo = vmDisk.getBacking();
+ if (backingInfo instanceof VirtualDiskFlatVer2BackingInfo) {
+ VirtualDiskFlatVer2BackingInfo backing = (VirtualDiskFlatVer2BackingInfo) backingInfo;
+ String fileName = backing.getFileName();
+ if (StringUtils.isNotBlank(fileName)) {
+ String[] fileNameParts = fileName.split(" ");
+ String datastoreUuid = fileNameParts[0].replace("[", "").replace("]", "");
+ String relativePath = fileNameParts[1].split("/")[1].replace(".vmdk", "");
+ String vmSpecDatastoreUuid = volumeObjectTO.getDataStore().getUuid().replaceAll("-", "");
+ if (!datastoreUuid.equals(vmSpecDatastoreUuid)) {
+ LOGGER.info("Mapped disk datastore UUID is not the same as the cloned VM datastore UUID: " +
+ datastoreUuid + " - " + vmSpecDatastoreUuid);
+ }
+ volumeObjectTO.setPath(relativePath);
+ specDisk.setPath(relativePath);
+ }
+ }
+ }
+ }
+ }
+ } catch (Exception e) {
+ String msg = "Error mapping deploy-as-is VM disks from cloned VM " + vmInternalCSName;
+ LOGGER.error(msg, e);
+ throw new CloudRuntimeException(e);
+ }
+ }
+
+ private StartAnswer handleStartFailure(StartCommand cmd, VirtualMachineTO vmSpec, VirtualMachineData existingVm, DatacenterMO dcMo, Throwable e) {
+ if (e instanceof RemoteException) {
+ LOGGER.warn("Encounter remote exception to vCenter, invalidate VMware session context");
+ vmwareResource.invalidateServiceContext();
+ }
+
+ String msg = "StartCommand failed due to " + VmwareHelper.getExceptionMessage(e);
+ LOGGER.warn(msg, e);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("didn't start VM %s", vmSpec.getName()), e);
+ }
+ StartAnswer startAnswer = new StartAnswer(cmd, msg);
+ if ( e instanceof VmAlreadyExistsInVcenter) {
+ startAnswer.setContextParam("stopRetry", "true");
+ }
+ reRegisterExistingVm(existingVm, dcMo);
+
+ return startAnswer;
+ }
+
+ private void deleteOldVersionOfTheStartedVM(VirtualMachineData existingVm, DatacenterMO dcMo, VirtualMachineMO vmMo) throws Exception {
+ // Since VM was successfully powered-on, if there was an existing VM in a different cluster that was unregistered, delete all the files associated with it.
+ if (existingVm != null && existingVm.vmName != null && existingVm.vmFileLayout != null) {
+ List<String> vmDatastoreNames = new ArrayList<>();
+ for (DatastoreMO vmDatastore : vmMo.getAllDatastores()) {
+ vmDatastoreNames.add(vmDatastore.getName());
+ }
+ // Don't delete files that are in a datastore that is being used by the new VM as well (zone-wide datastore).
+ List<String> skipDatastores = new ArrayList<>();
+ for (DatastoreMO existingDatastore : existingVm.datastores) {
+ if (vmDatastoreNames.contains(existingDatastore.getName())) {
+ skipDatastores.add(existingDatastore.getName());
+ }
+ }
+ vmwareResource.deleteUnregisteredVmFiles(existingVm.vmFileLayout, dcMo, true, skipDatastores);
+ }
+ }
+
+ private int adjustNumberOfCoresPerSocket(VirtualMachineTO vmSpec, VirtualMachineMO vmMo, VirtualMachineConfigSpec vmConfigSpec) throws Exception {
+ // Check for multi-cores per socket settings
+ int numCoresPerSocket = 1;
+ String coresPerSocket = vmSpec.getDetails().get(VmDetailConstants.CPU_CORE_PER_SOCKET);
+ if (coresPerSocket != null) {
+ String apiVersion = HypervisorHostHelper.getVcenterApiVersion(vmMo.getContext());
+ // Property 'numCoresPerSocket' is supported since vSphere API 5.0
+ if (apiVersion.compareTo("5.0") >= 0) {
+ numCoresPerSocket = NumbersUtil.parseInt(coresPerSocket, 1);
+ vmConfigSpec.setNumCoresPerSocket(numCoresPerSocket);
+ }
+ }
+ return numCoresPerSocket;
+ }
+
+ /**
+ // Setup USB devices
+ */
+ private int setupUsbDevicesAndGetCount(String vmInternalCSName, VirtualMachineMO vmMo, String guestOsId, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount)
+ throws Exception {
+ int usbDeviceCount = 0;
+ if (guestOsId.startsWith("darwin")) { //Mac OS
+ VirtualDevice[] devices = vmMo.getMatchedDevices(new Class<?>[] {VirtualUSBController.class});
+ if (devices.length == 0) {
+ LOGGER.debug("No USB Controller device on VM Start. Add USB Controller device for Mac OS VM " + vmInternalCSName);
+
+ //For Mac OS X systems, the EHCI+UHCI controller is enabled by default and is required for USB mouse and keyboard access.
+ VirtualDevice usbControllerDevice = VmwareHelper.prepareUSBControllerDevice();
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ deviceConfigSpecArray[deviceCount].setDevice(usbControllerDevice);
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare USB controller at new device " + vmwareResource.getGson().toJson(deviceConfigSpecArray[deviceCount]));
+
+ deviceCount++;
+ usbDeviceCount++;
+ } else {
+ LOGGER.debug("USB Controller device exists on VM Start for Mac OS VM " + vmInternalCSName);
+ }
+ }
+ return usbDeviceCount;
+ }
+
+ private void createNewVm(VmwareContext context, VirtualMachineTO vmSpec, boolean installAsIs, String vmInternalCSName, String vmNameOnVcenter, Pair<String, String> controllerInfo, Boolean systemVm,
+ VmwareManager mgr, VmwareHypervisorHost hyperHost, String guestOsId, DiskTO[] disks, HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails,
+ DatastoreMO dsRootVolumeIsOn) throws Exception {
+ VirtualMachineMO vmMo;
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = getRootDiskDataStoreDetails(disks, dataStoresDetails);
+
+ assert (vmSpec.getMinSpeed() != null) && (rootDiskDataStoreDetails != null);
+
+ boolean vmFolderExists = rootDiskDataStoreDetails.second().folderExists(String.format("[%s]", rootDiskDataStoreDetails.second().getName()), vmNameOnVcenter);
+ String vmxFileFullPath = dsRootVolumeIsOn.searchFileInSubFolders(vmNameOnVcenter + ".vmx", false, VmwareManager.s_vmwareSearchExcludeFolder.value());
+ if (vmFolderExists && vmxFileFullPath != null) { // VM can be registered only if .vmx is present.
+ registerVm(vmNameOnVcenter, dsRootVolumeIsOn, vmwareResource);
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ if (vmMo != null) {
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Found registered vm " + vmInternalCSName + " at host " + hyperHost.getHyperHostName());
+ }
+ }
+ tearDownVm(vmMo);
+ } else if (installAsIs) {
+// first get all the MORs ManagedObjectReference morPool = hyperHost.getHyperHostOwnerResourcePool();
+// get the base VM vmMo = hyperHost.findVmOnHyperHost(vm.template.getPath());
+// do templateVm.createLinkedClone(vmInternalCSName, morBaseSnapshot, dcMo.getVmFolder(), morPool, morDatastore)
+// or hyperHost....createLinkedOrFullClone(templateVm, volume, dcMo, vmMo, morDatastore, dsMo, vmInternalCSName, morPool);
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ // At this point vmMo points to the cloned VM
+ } else {
+ if (!hyperHost
+ .createBlankVm(vmNameOnVcenter, vmInternalCSName, vmSpec.getCpus(), vmSpec.getMaxSpeed(), vmwareResource.getReservedCpuMHZ(vmSpec), vmSpec.getLimitCpuUse(), (int)(vmSpec.getMaxRam() / Resource.ResourceType.bytesToMiB), vmwareResource.getReservedMemoryMb(vmSpec), guestOsId,
+ rootDiskDataStoreDetails.first(), false, controllerInfo, systemVm)) {
+ throw new Exception("Failed to create VM. vmName: " + vmInternalCSName);
+ }
+ }
+ }
+
+ /**
+ * If a VM with the same name is found in a different cluster in the DC, unregister the old VM and configure a new VM (cold-migration).
+ */
+ private VirtualMachineData unregisterOnOtherClusterButHoldOnToOldVmData(String vmInternalCSName, DatacenterMO dcMo) throws Exception {
+ VirtualMachineMO existingVmInDc = dcMo.findVm(vmInternalCSName);
+ VirtualMachineData existingVm = null;
+ if (existingVmInDc != null) {
+ existingVm = new VirtualMachineData();
+ existingVm.vmName = existingVmInDc.getName();
+ existingVm.vmFileInfo = existingVmInDc.getFileInfo();
+ existingVm.vmFileLayout = existingVmInDc.getFileLayout();
+ existingVm.datastores = existingVmInDc.getAllDatastores();
+ LOGGER.info("Found VM: " + vmInternalCSName + " on a host in a different cluster. Unregistering the exisitng VM.");
+ existingVmInDc.unregisterVm();
+ }
+ return existingVm;
+ }
+
+ private Pair<ManagedObjectReference, DatastoreMO> getRootDiskDataStoreDetails(DiskTO[] disks, HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails) {
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = null;
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ROOT) {
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (managed) {
+ String datastoreName = VmwareResource.getDatastoreName(details.get(DiskTO.IQN));
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(datastoreName);
+ } else {
+ DataStoreTO primaryStore = vol.getData().getDataStore();
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(primaryStore.getUuid());
+ }
+ }
+ }
+ return rootDiskDataStoreDetails;
+ }
+
+ /**
+ * Since VM start failed, if there was an existing VM in a different cluster that was unregistered, register it back.
+ *
+ * @param dcMo is guaranteed to be not null since we have noticed there is an existing VM in the dc (using that mo)
+ */
+ private void reRegisterExistingVm(VirtualMachineData existingVm, DatacenterMO dcMo) {
+ if (existingVm != null && existingVm.vmName != null && existingVm.vmFileInfo != null) {
+ LOGGER.debug("Since VM start failed, registering back an existing VM: " + existingVm.vmName + " that was unregistered");
+ try {
+ DatastoreFile fileInDatastore = new DatastoreFile(existingVm.vmFileInfo.getVmPathName());
+ DatastoreMO existingVmDsMo = new DatastoreMO(dcMo.getContext(), dcMo.findDatastore(fileInDatastore.getDatastoreName()));
+ registerVm(existingVm.vmName, existingVmDsMo, vmwareResource);
+ } catch (Exception ex) {
+ String message = "Failed to register an existing VM: " + existingVm.vmName + " due to " + VmwareHelper.getExceptionMessage(ex);
+ LOGGER.warn(message, ex);
+ }
+ }
+ }
+
+ private void checkIfVmExistsInVcenter(String vmInternalCSName, String vmNameOnVcenter, DatacenterMO dcMo) throws VmAlreadyExistsInVcenter, Exception {
+ // Validate VM name is unique in Datacenter
+ VirtualMachineMO vmInVcenter = dcMo.checkIfVmAlreadyExistsInVcenter(vmNameOnVcenter, vmInternalCSName);
+ if (vmInVcenter != null) {
+ String msg = "VM with name: " + vmNameOnVcenter + " already exists in vCenter.";
+ LOGGER.error(msg);
+ throw new VmAlreadyExistsInVcenter(msg);
+ }
+ }
+
+ private Pair<String, String> getDiskControllerInfo(VirtualMachineTO vmSpec) {
+ String dataDiskController = vmSpec.getDetails().get(VmDetailConstants.DATA_DISK_CONTROLLER);
+ String rootDiskController = vmSpec.getDetails().get(VmDetailConstants.ROOT_DISK_CONTROLLER);
+ // If root disk controller is scsi, then data disk controller would also be scsi instead of using 'osdefault'
+ // This helps avoid mix of different scsi subtype controllers in instance.
+ if (DiskControllerType.osdefault == DiskControllerType.getType(dataDiskController) && DiskControllerType.lsilogic == DiskControllerType.getType(rootDiskController)) {
+ dataDiskController = DiskControllerType.scsi.toString();
+ }
+
+ // Validate the controller types
+ dataDiskController = DiskControllerType.getType(dataDiskController).toString();
+ rootDiskController = DiskControllerType.getType(rootDiskController).toString();
+
+ if (DiskControllerType.getType(rootDiskController) == DiskControllerType.none) {
+ throw new CloudRuntimeException("Invalid root disk controller detected : " + rootDiskController);
+ }
+ if (DiskControllerType.getType(dataDiskController) == DiskControllerType.none) {
+ throw new CloudRuntimeException("Invalid data disk controller detected : " + dataDiskController);
+ }
+
+ return new Pair<>(rootDiskController, dataDiskController);
+ }
+
+ /**
+ * Registers the vm to the inventory given the vmx file.
+ * @param vmName
+ * @param dsMo
+ * @param vmwareResource
+ */
+ private void registerVm(String vmName, DatastoreMO dsMo, VmwareResource vmwareResource) throws Exception {
+
+ //1st param
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(vmwareResource.getServiceContext());
+ ManagedObjectReference dcMor = hyperHost.getHyperHostDatacenter();
+ DatacenterMO dataCenterMo = new DatacenterMO(vmwareResource.getServiceContext(), dcMor);
+ ManagedObjectReference vmFolderMor = dataCenterMo.getVmFolder();
+
+ //2nd param
+ String vmxFilePath = dsMo.searchFileInSubFolders(vmName + ".vmx", false, VmwareManager.s_vmwareSearchExcludeFolder.value());
+
+ // 5th param
+ ManagedObjectReference morPool = hyperHost.getHyperHostOwnerResourcePool();
+
+ ManagedObjectReference morTask = vmwareResource.getServiceContext().getService().registerVMTask(vmFolderMor, vmxFilePath, vmName, false, morPool, hyperHost.getMor());
+ boolean result = vmwareResource.getServiceContext().getVimClient().waitForTask(morTask);
+ if (!result) {
+ throw new Exception("Unable to register vm due to " + TaskMO.getTaskFailureInfo(vmwareResource.getServiceContext(), morTask));
+ } else {
+ vmwareResource.getServiceContext().waitForTaskProgressDone(morTask);
+ }
+
+ }
+
+ // Pair<internal CS name, vCenter display name>
+ private Pair<String, String> composeVmNames(VirtualMachineTO vmSpec) {
+ String vmInternalCSName = vmSpec.getName();
+ String vmNameOnVcenter = vmSpec.getName();
+ if (VmwareResource.instanceNameFlag && vmSpec.getHostName() != null) {
+ vmNameOnVcenter = vmSpec.getHostName();
+ }
+ return new Pair<String, String>(vmInternalCSName, vmNameOnVcenter);
+ }
+
+ private VirtualMachineGuestOsIdentifier translateGuestOsIdentifier(String cpuArchitecture, String guestOs, String cloudGuestOs) {
+ if (cpuArchitecture == null) {
+ LOGGER.warn("CPU arch is not set, default to i386. guest os: " + guestOs);
+ cpuArchitecture = "i386";
+ }
+
+ if (cloudGuestOs == null) {
+ LOGGER.warn("Guest OS mapping name is not set for guest os: " + guestOs);
+ }
+
+ VirtualMachineGuestOsIdentifier identifier = null;
+ try {
+ if (cloudGuestOs != null) {
+ identifier = VirtualMachineGuestOsIdentifier.fromValue(cloudGuestOs);
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Using mapping name : " + identifier.toString());
+ }
+ }
+ } catch (IllegalArgumentException e) {
+ LOGGER.warn("Unable to find Guest OS Identifier in VMware for mapping name: " + cloudGuestOs + ". Continuing with defaults.");
+ }
+ if (identifier != null) {
+ return identifier;
+ }
+
+ if (cpuArchitecture.equalsIgnoreCase("x86_64")) {
+ return VirtualMachineGuestOsIdentifier.OTHER_GUEST_64;
+ }
+ return VirtualMachineGuestOsIdentifier.OTHER_GUEST;
+ }
+
+ private DiskTO[] validateDisks(DiskTO[] disks) {
+ List<DiskTO> validatedDisks = new ArrayList<DiskTO>();
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() != Volume.Type.ISO) {
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ if (primaryStore.getUuid() != null && !primaryStore.getUuid().isEmpty()) {
+ validatedDisks.add(vol);
+ }
+ } else if (vol.getType() == Volume.Type.ISO) {
+ TemplateObjectTO templateTO = (TemplateObjectTO) vol.getData();
+ if (templateTO.getPath() != null && !templateTO.getPath().isEmpty()) {
+ validatedDisks.add(vol);
+ }
+ } else {
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Drop invalid disk option, volumeTO: " + vmwareResource.getGson().toJson(vol));
+ }
+ }
+ }
+ Collections.sort(validatedDisks, (d1, d2) -> d1.getDiskSeq().compareTo(d2.getDiskSeq()));
+ return validatedDisks.toArray(new DiskTO[0]);
+ }
+
+ private HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> inferDatastoreDetailsFromDiskInfo(VmwareHypervisorHost hyperHost, VmwareContext context,
+ DiskTO[] disks, Command cmd) throws Exception {
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> mapIdToMors = new HashMap<>();
+
+ assert (hyperHost != null) && (context != null);
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() != Volume.Type.ISO) {
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ String poolUuid = primaryStore.getUuid();
+
+ if (mapIdToMors.get(poolUuid) == null) {
+ boolean isManaged = false;
+ Map<String, String> details = vol.getDetails();
+
+ if (details != null) {
+ isManaged = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (isManaged) {
+ String iScsiName = details.get(DiskTO.IQN); // details should not be null for managed storage (it may or may not be null for non-managed storage)
+ String datastoreName = VmwareResource.getDatastoreName(iScsiName);
+ ManagedObjectReference morDatastore = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, datastoreName);
+
+ // if the datastore is not present, we need to discover the iSCSI device that will support it,
+ // create the datastore, and create a VMDK file in the datastore
+ if (morDatastore == null) {
+ final String vmdkPath = vmwareResource.getVmdkPath(volumeTO.getPath());
+
+ morDatastore = getStorageProcessor().prepareManagedStorage(context, hyperHost, null, iScsiName,
+ details.get(DiskTO.STORAGE_HOST), Integer.parseInt(details.get(DiskTO.STORAGE_PORT)),
+ vmdkPath,
+ details.get(DiskTO.CHAP_INITIATOR_USERNAME), details.get(DiskTO.CHAP_INITIATOR_SECRET),
+ details.get(DiskTO.CHAP_TARGET_USERNAME), details.get(DiskTO.CHAP_TARGET_SECRET),
+ Long.parseLong(details.get(DiskTO.VOLUME_SIZE)), cmd);
+
+ DatastoreMO dsMo = new DatastoreMO(vmwareResource.getServiceContext(), morDatastore);
+
+ final String datastoreVolumePath;
+
+ if (vmdkPath != null) {
+ datastoreVolumePath = dsMo.getDatastorePath(vmdkPath + VmwareResource.VMDK_EXTENSION);
+ } else {
+ datastoreVolumePath = dsMo.getDatastorePath(dsMo.getName() + VmwareResource.VMDK_EXTENSION);
+ }
+
+ volumeTO.setPath(datastoreVolumePath);
+ vol.setPath(datastoreVolumePath);
+ }
+
+ mapIdToMors.put(datastoreName, new Pair<>(morDatastore, new DatastoreMO(context, morDatastore)));
+ } else {
+ ManagedObjectReference morDatastore = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, poolUuid);
+
+ if (morDatastore == null) {
+ String msg = "Failed to get the mounted datastore for the volume's pool " + poolUuid;
+
+ LOGGER.error(msg);
+
+ throw new Exception(msg);
+ }
+
+ mapIdToMors.put(poolUuid, new Pair<>(morDatastore, new DatastoreMO(context, morDatastore)));
+ }
+ }
+ }
+ }
+
+ return mapIdToMors;
+ }
+
+ private VmwareStorageProcessor getStorageProcessor() {
+ return vmwareResource.getStorageProcessor();
+ }
+
+ private DatastoreMO getDatastoreThatRootDiskIsOn(HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails, DiskTO disks[]) {
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = null;
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ROOT) {
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (managed) {
+ String datastoreName = VmwareResource.getDatastoreName(details.get(DiskTO.IQN));
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(datastoreName);
+
+ break;
+ } else {
+ DataStoreTO primaryStore = vol.getData().getDataStore();
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(primaryStore.getUuid());
+
+ break;
+ }
+ }
+ }
+
+ if (rootDiskDataStoreDetails != null) {
+ return rootDiskDataStoreDetails.second();
+ }
+
+ return null;
+ }
+
+ private void ensureDiskControllers(VirtualMachineMO vmMo, Pair<String, String> controllerInfo) throws Exception {
+ if (vmMo == null) {
+ return;
+ }
+
+ String msg;
+ String rootDiskController = controllerInfo.first();
+ String dataDiskController = controllerInfo.second();
+ String scsiDiskController;
+ String recommendedDiskController = null;
+
+ if (VmwareHelper.isControllerOsRecommended(dataDiskController) || VmwareHelper.isControllerOsRecommended(rootDiskController)) {
+ recommendedDiskController = vmMo.getRecommendedDiskController(null);
+ }
+ scsiDiskController = HypervisorHostHelper.getScsiController(new Pair<String, String>(rootDiskController, dataDiskController), recommendedDiskController);
+ if (scsiDiskController == null) {
+ return;
+ }
+
+ vmMo.getScsiDeviceControllerKeyNoException();
+ // This VM needs SCSI controllers.
+ // Get count of existing scsi controllers. Helps not to attempt to create more than the maximum allowed 4
+ // Get maximum among the bus numbers in use by scsi controllers. Safe to pick maximum, because we always go sequential allocating bus numbers.
+ Ternary<Integer, Integer, DiskControllerType> scsiControllerInfo = vmMo.getScsiControllerInfo();
+ int requiredNumScsiControllers = VmwareHelper.MAX_SCSI_CONTROLLER_COUNT - scsiControllerInfo.first();
+ int availableBusNum = scsiControllerInfo.second() + 1; // method returned current max. bus number
+
+ if (requiredNumScsiControllers == 0) {
+ return;
+ }
+ if (scsiControllerInfo.first() > 0) {
+ // For VMs which already have a SCSI controller, do NOT attempt to add any more SCSI controllers & return the sub type.
+ // For Legacy VMs would have only 1 LsiLogic Parallel SCSI controller, and doesn't require more.
+ // For VMs created post device ordering support, 4 SCSI subtype controllers are ensured during deployment itself. No need to add more.
+ // For fresh VM deployment only, all required controllers should be ensured.
+ return;
+ }
+ ensureScsiDiskControllers(vmMo, scsiDiskController, requiredNumScsiControllers, availableBusNum);
+ }
+
+ private void ensureScsiDiskControllers(VirtualMachineMO vmMo, String scsiDiskController, int requiredNumScsiControllers, int availableBusNum) throws Exception {
+ // Pick the sub type of scsi
+ if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.pvscsi) {
+ if (!vmMo.isPvScsiSupported()) {
+ String msg = "This VM doesn't support Vmware Paravirtual SCSI controller for virtual disks, because the virtual hardware version is less than 7.";
+ throw new Exception(msg);
+ }
+ vmMo.ensurePvScsiDeviceController(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.lsisas1068) {
+ vmMo.ensureLsiLogicSasDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.buslogic) {
+ vmMo.ensureBusLogicDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.lsilogic) {
+ vmMo.ensureLsiLogicDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ }
+ }
+
+ private void tearDownVm(VirtualMachineMO vmMo) throws Exception {
+
+ if (vmMo == null)
+ return;
+
+ boolean hasSnapshot = false;
+ hasSnapshot = vmMo.hasSnapshot();
+ if (!hasSnapshot)
+ vmMo.tearDownDevices(new Class<?>[]{VirtualDisk.class, VirtualEthernetCard.class});
+ else
+ vmMo.tearDownDevices(new Class<?>[]{VirtualEthernetCard.class});
+ vmMo.ensureScsiDeviceController();
+ }
+
+ private static DiskTO getIsoDiskTO(DiskTO[] disks) {
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ISO) {
+ return vol;
+ }
+ }
+ return null;
+ }
+
+ private static DiskTO[] sortVolumesByDeviceId(DiskTO[] volumes) {
+
+ List<DiskTO> listForSort = new ArrayList<DiskTO>();
+ for (DiskTO vol : volumes) {
+ listForSort.add(vol);
+ }
+ Collections.sort(listForSort, new Comparator<DiskTO>() {
+
+ @Override
+ public int compare(DiskTO arg0, DiskTO arg1) {
+ if (arg0.getDiskSeq() < arg1.getDiskSeq()) {
+ return -1;
+ } else if (arg0.getDiskSeq().equals(arg1.getDiskSeq())) {
+ return 0;
+ }
+
+ return 1;
+ }
+ });
+
+ return listForSort.toArray(new DiskTO[0]);
+ }
+
+ private void postDiskConfigBeforeStart(VirtualMachineMO vmMo, VirtualMachineTO vmSpec, DiskTO[] sortedDisks,
+ Map<String, Map<String, String>> iqnToData, VmwareHypervisorHost hyperHost, VmwareContext context, boolean installAsIs)
+ throws Exception {
+ VirtualMachineDiskInfoBuilder diskInfoBuilder = vmMo.getDiskInfoBuilder();
+
+ for (DiskTO vol : sortedDisks) {
+ //TODO: Map existing disks to the ones returned in the answer
+ if (vol.getType() == Volume.Type.ISO)
+ continue;
+
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+
+ VirtualMachineDiskInfo diskInfo = getMatchingExistingDisk(diskInfoBuilder, vol, hyperHost, context);
+ assert (diskInfo != null);
+
+ String[] diskChain = diskInfo.getDiskChain();
+ assert (diskChain.length > 0);
+
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ DatastoreFile file = new DatastoreFile(diskChain[0]);
+
+ if (managed) {
+ DatastoreFile originalFile = new DatastoreFile(volumeTO.getPath());
+
+ if (!file.getFileBaseName().equalsIgnoreCase(originalFile.getFileBaseName())) {
+ if (LOGGER.isInfoEnabled())
+ LOGGER.info("Detected disk-chain top file change on volume: " + volumeTO.getId() + " " + volumeTO.getPath() + " -> " + diskChain[0]);
+ }
+ } else {
+ if (!file.getFileBaseName().equalsIgnoreCase(volumeTO.getPath())) {
+ if (LOGGER.isInfoEnabled())
+ LOGGER.info("Detected disk-chain top file change on volume: " + volumeTO.getId() + " " + volumeTO.getPath() + " -> " + file.getFileBaseName());
+ }
+ }
+
+ VolumeObjectTO volInSpec = getVolumeInSpec(vmSpec, volumeTO);
+
+ if (volInSpec != null) {
+ if (managed) {
+ Map<String, String> data = new HashMap<>();
+
+ String datastoreVolumePath = diskChain[0];
+
+ data.put(StartAnswer.PATH, datastoreVolumePath);
+ data.put(StartAnswer.IMAGE_FORMAT, Storage.ImageFormat.OVA.toString());
+
+ iqnToData.put(details.get(DiskTO.IQN), data);
+
+ vol.setPath(datastoreVolumePath);
+ volumeTO.setPath(datastoreVolumePath);
+ volInSpec.setPath(datastoreVolumePath);
+ } else {
+ volInSpec.setPath(file.getFileBaseName());
+ }
+ volInSpec.setChainInfo(vmwareResource.getGson().toJson(diskInfo));
+ }
+ }
+ }
+
+ private void checkBootOptions(VirtualMachineTO vmSpec, VirtualMachineConfigSpec vmConfigSpec) {
+ String bootMode = null;
+ if (vmSpec.getDetails().containsKey(VmDetailConstants.BOOT_MODE)) {
+ bootMode = vmSpec.getDetails().get(VmDetailConstants.BOOT_MODE);
+ }
+ if (null == bootMode) {
+ bootMode = ApiConstants.BootType.BIOS.toString();
+ }
+
+ setBootOptions(vmSpec, bootMode, vmConfigSpec);
+ }
+
+ private void setBootOptions(VirtualMachineTO vmSpec, String bootMode, VirtualMachineConfigSpec vmConfigSpec) {
+ VirtualMachineBootOptions bootOptions = null;
+ if (StringUtils.isNotBlank(bootMode) && !bootMode.equalsIgnoreCase("bios")) {
+ vmConfigSpec.setFirmware("efi");
+ if (vmSpec.getDetails().containsKey(ApiConstants.BootType.UEFI.toString()) && "secure".equalsIgnoreCase(vmSpec.getDetails().get(ApiConstants.BootType.UEFI.toString()))) {
+ if (bootOptions == null) {
+ bootOptions = new VirtualMachineBootOptions();
+ }
+ bootOptions.setEfiSecureBootEnabled(true);
+ }
+ }
+ if (vmSpec.isEnterHardwareSetup()) {
+ if (bootOptions == null) {
+ bootOptions = new VirtualMachineBootOptions();
+ }
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("configuring VM '%s' to enter hardware setup",vmSpec.getName()));
+ }
+ bootOptions.setEnterBIOSSetup(vmSpec.isEnterHardwareSetup());
+ }
+ if (bootOptions != null) {
+ vmConfigSpec.setBootOptions(bootOptions);
+ }
+ }
+
+ private void resizeRootDiskOnVMStart(VirtualMachineMO vmMo, DiskTO rootDiskTO, VmwareHypervisorHost hyperHost, VmwareContext context) throws Exception {
+ final Pair<VirtualDisk, String> vdisk = vmwareResource.getVirtualDiskInfo(vmMo, VmwareResource.appendFileType(rootDiskTO.getPath(), VmwareResource.VMDK_EXTENSION));
+ assert (vdisk != null);
+
+ Long reqSize = 0L;
+ final VolumeObjectTO volumeTO = ((VolumeObjectTO) rootDiskTO.getData());
+ if (volumeTO != null) {
+ reqSize = volumeTO.getSize() / 1024;
+ }
+ final VirtualDisk disk = vdisk.first();
+ if (reqSize > disk.getCapacityInKB()) {
+ final VirtualMachineDiskInfo diskInfo = getMatchingExistingDisk(vmMo.getDiskInfoBuilder(), rootDiskTO, hyperHost, context);
+ assert (diskInfo != null);
+ final String[] diskChain = diskInfo.getDiskChain();
+
+ if (diskChain != null && diskChain.length > 1) {
+ LOGGER.warn("Disk chain length for the VM is greater than one, this is not supported");
+ throw new CloudRuntimeException("Unsupported VM disk chain length: " + diskChain.length);
+ }
+
+ boolean resizingSupported = false;
+ String deviceBusName = diskInfo.getDiskDeviceBusName();
+ if (deviceBusName != null && (deviceBusName.toLowerCase().contains("scsi") || deviceBusName.toLowerCase().contains("lsi"))) {
+ resizingSupported = true;
+ }
+ if (!resizingSupported) {
+ LOGGER.warn("Resizing of root disk is only support for scsi device/bus, the provide VM's disk device bus name is " + diskInfo.getDiskDeviceBusName());
+ throw new CloudRuntimeException("Unsupported VM root disk device bus: " + diskInfo.getDiskDeviceBusName());
+ }
+
+ disk.setCapacityInKB(reqSize);
+ VirtualMachineConfigSpec vmConfigSpec = new VirtualMachineConfigSpec();
+ VirtualDeviceConfigSpec deviceConfigSpec = new VirtualDeviceConfigSpec();
+ deviceConfigSpec.setDevice(disk);
+ deviceConfigSpec.setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ vmConfigSpec.getDeviceChange().add(deviceConfigSpec);
+ if (!vmMo.configureVm(vmConfigSpec)) {
+ throw new Exception("Failed to configure VM for given root disk size. vmName: " + vmMo.getName());
+ }
+ }
+ }
+
+ private static void postNvpConfigBeforeStart(VirtualMachineMO vmMo, VirtualMachineTO vmSpec) throws Exception {
+ /**
+ * We need to configure the port on the DV switch after the host is
+ * connected. So make this happen between the configure and start of
+ * the VM
+ */
+ int nicIndex = 0;
+ for (NicTO nicTo : sortNicsByDeviceId(vmSpec.getNics())) {
+ if (nicTo.getBroadcastType() == Networks.BroadcastDomainType.Lswitch) {
+ // We need to create a port with a unique vlan and pass the key to the nic device
+ LOGGER.trace("Nic " + nicTo.toString() + " is connected to an NVP logicalswitch");
+ VirtualDevice nicVirtualDevice = vmMo.getNicDeviceByIndex(nicIndex);
+ if (nicVirtualDevice == null) {
+ throw new Exception("Failed to find a VirtualDevice for nic " + nicIndex); //FIXME Generic exceptions are bad
+ }
+ VirtualDeviceBackingInfo backing = nicVirtualDevice.getBacking();
+ if (backing instanceof VirtualEthernetCardDistributedVirtualPortBackingInfo) {
+ // This NIC is connected to a Distributed Virtual Switch
+ VirtualEthernetCardDistributedVirtualPortBackingInfo portInfo = (VirtualEthernetCardDistributedVirtualPortBackingInfo) backing;
+ DistributedVirtualSwitchPortConnection port = portInfo.getPort();
+ String portKey = port.getPortKey();
+ String portGroupKey = port.getPortgroupKey();
+ String dvSwitchUuid = port.getSwitchUuid();
+
+ LOGGER.debug("NIC " + nicTo.toString() + " is connected to dvSwitch " + dvSwitchUuid + " pg " + portGroupKey + " port " + portKey);
+
+ ManagedObjectReference dvSwitchManager = vmMo.getContext().getVimClient().getServiceContent().getDvSwitchManager();
+ ManagedObjectReference dvSwitch = vmMo.getContext().getVimClient().getService().queryDvsByUuid(dvSwitchManager, dvSwitchUuid);
+
+ // Get all ports
+ DistributedVirtualSwitchPortCriteria criteria = new DistributedVirtualSwitchPortCriteria();
+ criteria.setInside(true);
+ criteria.getPortgroupKey().add(portGroupKey);
+ List<DistributedVirtualPort> dvPorts = vmMo.getContext().getVimClient().getService().fetchDVPorts(dvSwitch, criteria);
+
+ DistributedVirtualPort vmDvPort = null;
+ List<Integer> usedVlans = new ArrayList<Integer>();
+ for (DistributedVirtualPort dvPort : dvPorts) {
+ // Find the port for this NIC by portkey
+ if (portKey.equals(dvPort.getKey())) {
+ vmDvPort = dvPort;
+ }
+ VMwareDVSPortSetting settings = (VMwareDVSPortSetting) dvPort.getConfig().getSetting();
+ VmwareDistributedVirtualSwitchVlanIdSpec vlanId = (VmwareDistributedVirtualSwitchVlanIdSpec) settings.getVlan();
+ LOGGER.trace("Found port " + dvPort.getKey() + " with vlan " + vlanId.getVlanId());
+ if (vlanId.getVlanId() > 0 && vlanId.getVlanId() < 4095) {
+ usedVlans.add(vlanId.getVlanId());
+ }
+ }
+
+ if (vmDvPort == null) {
+ throw new Exception("Empty port list from dvSwitch for nic " + nicTo.toString());
+ }
+
+ DVPortConfigInfo dvPortConfigInfo = vmDvPort.getConfig();
+ VMwareDVSPortSetting settings = (VMwareDVSPortSetting) dvPortConfigInfo.getSetting();
+
+ VmwareDistributedVirtualSwitchVlanIdSpec vlanId = (VmwareDistributedVirtualSwitchVlanIdSpec) settings.getVlan();
+ BoolPolicy blocked = settings.getBlocked();
+ if (blocked.isValue() == Boolean.TRUE) {
+ LOGGER.trace("Port is blocked, set a vlanid and unblock");
+ DVPortConfigSpec dvPortConfigSpec = new DVPortConfigSpec();
+ VMwareDVSPortSetting edittedSettings = new VMwareDVSPortSetting();
+ // Unblock
+ blocked.setValue(Boolean.FALSE);
+ blocked.setInherited(Boolean.FALSE);
+ edittedSettings.setBlocked(blocked);
+ // Set vlan
+ int i;
+ for (i = 1; i < 4095; i++) {
+ if (!usedVlans.contains(i))
+ break;
+ }
+ vlanId.setVlanId(i); // FIXME should be a determined
+ // based on usage
+ vlanId.setInherited(false);
+ edittedSettings.setVlan(vlanId);
+
+ dvPortConfigSpec.setSetting(edittedSettings);
+ dvPortConfigSpec.setOperation("edit");
+ dvPortConfigSpec.setKey(portKey);
+ List<DVPortConfigSpec> dvPortConfigSpecs = new ArrayList<DVPortConfigSpec>();
+ dvPortConfigSpecs.add(dvPortConfigSpec);
+ ManagedObjectReference task = vmMo.getContext().getVimClient().getService().reconfigureDVPortTask(dvSwitch, dvPortConfigSpecs);
+ if (!vmMo.getContext().getVimClient().waitForTask(task)) {
+ throw new Exception("Failed to configure the dvSwitch port for nic " + nicTo.toString());
+ }
+ LOGGER.debug("NIC " + nicTo.toString() + " connected to vlan " + i);
+ } else {
+ LOGGER.trace("Port already configured and set to vlan " + vlanId.getVlanId());
+ }
+ } else if (backing instanceof VirtualEthernetCardNetworkBackingInfo) {
+ // This NIC is connected to a Virtual Switch
+ // Nothing to do
+ } else if (backing instanceof VirtualEthernetCardOpaqueNetworkBackingInfo) {
+ //if NSX API VERSION >= 4.2, connect to br-int (nsx.network), do not create portgroup else previous behaviour
+ //OK, connected to OpaqueNetwork
+ } else {
+ LOGGER.error("nic device backing is of type " + backing.getClass().getName());
+ throw new Exception("Incompatible backing for a VirtualDevice for nic " + nicIndex); //FIXME Generic exceptions are bad
+ }
+ }
+ nicIndex++;
+ }
+ }
+
+ private VirtualMachineDiskInfo getMatchingExistingDisk(VirtualMachineDiskInfoBuilder diskInfoBuilder, DiskTO vol, VmwareHypervisorHost hyperHost, VmwareContext context)
+ throws Exception {
+ if (diskInfoBuilder != null) {
+ VolumeObjectTO volume = (VolumeObjectTO) vol.getData();
+
+ String dsName = null;
+ String diskBackingFileBaseName = null;
+
+ Map<String, String> details = vol.getDetails();
+ boolean isManaged = details != null && Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+
+ if (isManaged) {
+ String iScsiName = details.get(DiskTO.IQN);
+
+ // if the storage is managed, iScsiName should not be null
+ dsName = VmwareResource.getDatastoreName(iScsiName);
+
+ diskBackingFileBaseName = new DatastoreFile(volume.getPath()).getFileBaseName();
+ } else {
+ ManagedObjectReference morDs = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, volume.getDataStore().getUuid());
+ DatastoreMO dsMo = new DatastoreMO(context, morDs);
+
+ dsName = dsMo.getName();
+
+ diskBackingFileBaseName = volume.getPath();
+ }
+
+ VirtualMachineDiskInfo diskInfo = diskInfoBuilder.getDiskInfoByBackingFileBaseName(diskBackingFileBaseName, dsName);
+ if (diskInfo != null) {
+ LOGGER.info("Found existing disk info from volume path: " + volume.getPath());
+ return diskInfo;
+ } else {
+ String chainInfo = volume.getChainInfo();
+ if (chainInfo != null) {
+ VirtualMachineDiskInfo infoInChain = vmwareResource.getGson().fromJson(chainInfo, VirtualMachineDiskInfo.class);
+ if (infoInChain != null) {
+ String[] disks = infoInChain.getDiskChain();
+ if (disks.length > 0) {
+ for (String diskPath : disks) {
+ DatastoreFile file = new DatastoreFile(diskPath);
+ diskInfo = diskInfoBuilder.getDiskInfoByBackingFileBaseName(file.getFileBaseName(), dsName);
+ if (diskInfo != null) {
+ LOGGER.info("Found existing disk from chain info: " + diskPath);
+ return diskInfo;
+ }
+ }
+ }
+
+ if (diskInfo == null) {
+ diskInfo = diskInfoBuilder.getDiskInfoByDeviceBusName(infoInChain.getDiskDeviceBusName());
+ if (diskInfo != null) {
+ LOGGER.info("Found existing disk from from chain device bus information: " + infoInChain.getDiskDeviceBusName());
+ return diskInfo;
+ }
+ }
+ }
+ }
+ }
+ }
+
+ return null;
+ }
+
+ private static VolumeObjectTO getVolumeInSpec(VirtualMachineTO vmSpec, VolumeObjectTO srcVol) {
+ for (DiskTO disk : vmSpec.getDisks()) {
+ if (disk.getData() instanceof VolumeObjectTO) {
+ VolumeObjectTO vol = (VolumeObjectTO) disk.getData();
+ if (vol.getId() == srcVol.getId())
+ return vol;
+ }
+ }
+
+ return null;
+ }
+
+ /**
+ * Generate the mac sequence from the nics.
+ */
+ protected String generateMacSequence(NicTO[] nics) {
+ if (nics.length == 0) {
+ return "";
+ }
+
+ StringBuffer sbMacSequence = new StringBuffer();
+ for (NicTO nicTo : sortNicsByDeviceId(nics)) {
+ sbMacSequence.append(nicTo.getMac()).append("|");
+ }
+ if (!sbMacSequence.toString().isEmpty()) {
+ sbMacSequence.deleteCharAt(sbMacSequence.length() - 1); //Remove extra '|' char appended at the end
+ }
+
+ return sbMacSequence.toString();
+ }
+
+ static NicTO[] sortNicsByDeviceId(NicTO[] nics) {
+
+ List<NicTO> listForSort = new ArrayList<NicTO>();
+ for (NicTO nic : nics) {
+ listForSort.add(nic);
+ }
+ Collections.sort(listForSort, new Comparator<NicTO>() {
+
+ @Override
+ public int compare(NicTO arg0, NicTO arg1) {
+ if (arg0.getDeviceId() < arg1.getDeviceId()) {
+ return -1;
+ } else if (arg0.getDeviceId() == arg1.getDeviceId()) {
+ return 0;
+ }
+
+ return 1;
+ }
+ });
+
+ return listForSort.toArray(new NicTO[0]);
+ }
+
+ private static void configBasicExtraOption(List<OptionValue> extraOptions, VirtualMachineTO vmSpec) {
+ OptionValue newVal = new OptionValue();
+ newVal.setKey("machine.id");
+ newVal.setValue(vmSpec.getBootArgs());
+ extraOptions.add(newVal);
+
+ newVal = new OptionValue();
+ newVal.setKey("devices.hotplug");
+ newVal.setValue("true");
+ extraOptions.add(newVal);
+ }
+
+ private static void configNvpExtraOption(List<OptionValue> extraOptions, VirtualMachineTO vmSpec, Map<String, String> nicUuidToDvSwitchUuid) {
+ /**
+ * Extra Config : nvp.vm-uuid = uuid
+ * - Required for Nicira NVP integration
+ */
+ OptionValue newVal = new OptionValue();
+ newVal.setKey("nvp.vm-uuid");
+ newVal.setValue(vmSpec.getUuid());
+ extraOptions.add(newVal);
+
+ /**
+ * Extra Config : nvp.iface-id.<num> = uuid
+ * - Required for Nicira NVP integration
+ */
+ int nicNum = 0;
+ for (NicTO nicTo : sortNicsByDeviceId(vmSpec.getNics())) {
+ if (nicTo.getUuid() != null) {
+ newVal = new OptionValue();
+ newVal.setKey("nvp.iface-id." + nicNum);
+ newVal.setValue(nicTo.getUuid());
+ extraOptions.add(newVal);
+ }
+ nicNum++;
+ }
+ }
+
+ private static void configCustomExtraOption(List<OptionValue> extraOptions, VirtualMachineTO vmSpec) {
+ // we no longer to validation anymore
+ for (Map.Entry<String, String> entry : vmSpec.getDetails().entrySet()) {
+ if (entry.getKey().equalsIgnoreCase(VmDetailConstants.BOOT_MODE)) {
+ continue;
+ }
+ OptionValue newVal = new OptionValue();
+ newVal.setKey(entry.getKey());
+ newVal.setValue(entry.getValue());
+ extraOptions.add(newVal);
+ }
+ }
+
+ private class VmAlreadyExistsInVcenter extends Exception {
+ public VmAlreadyExistsInVcenter(String msg) {
+ }
+ }
+
+ private class VirtualMachineData {
+ String vmName = null;
+ VirtualMachineFileInfo vmFileInfo = null;
+ VirtualMachineFileLayoutEx vmFileLayout = null;
+ List<DatastoreMO> datastores = new ArrayList<>();
+ }
+
+ private class VirtualMachineRecycler {
+ private String vmInternalCSName;
+ private Pair<String, String> controllerInfo;
+ private Boolean systemVm;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private DiskControllerType systemVmScsiControllerType;
+ private int firstScsiControllerBusNum;
+ private int numScsiControllerForSystemVm;
+ private VirtualMachineDiskInfoBuilder diskInfoBuilder;
+ private boolean hasSnapshot;
+ private VirtualDevice[] nicDevices;
+
+ public VirtualMachineRecycler(String vmInternalCSName, Pair<String, String> controllerInfo, Boolean systemVm, VmwareHypervisorHost hyperHost, VirtualMachineMO vmMo,
+ DiskControllerType systemVmScsiControllerType, int firstScsiControllerBusNum, int numScsiControllerForSystemVm) {
+ this.vmInternalCSName = vmInternalCSName;
+ this.controllerInfo = controllerInfo;
+ this.systemVm = systemVm;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.systemVmScsiControllerType = systemVmScsiControllerType;
+ this.firstScsiControllerBusNum = firstScsiControllerBusNum;
+ this.numScsiControllerForSystemVm = numScsiControllerForSystemVm;
+ }
+
+ public VirtualMachineDiskInfoBuilder getDiskInfoBuilder() {
+ return diskInfoBuilder;
+ }
+
+ public boolean isHasSnapshot() {
+ return hasSnapshot;
+ }
+
+ public VirtualMachineRecycler invoke() throws Exception {
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Found vm " + vmInternalCSName + " at other host, relocate to " + hyperHost.getHyperHostName());
+ }
+
+ takeVmFromOtherHyperHost(hyperHost, vmInternalCSName);
+
+ if (VmwareResource.getVmPowerState(vmMo) != VirtualMachine.PowerState.PowerOff)
+ vmMo.safePowerOff(vmwareResource.getShutdownWaitMs());
+
+ diskInfoBuilder = vmMo.getDiskInfoBuilder();
+ hasSnapshot = vmMo.hasSnapshot();
+ nicDevices = vmMo.getNicDevices();
+ if (!hasSnapshot)
+ vmMo.tearDownDevices(new Class<?>[] {VirtualDisk.class, VirtualEthernetCard.class});
+ else
+ vmMo.tearDownDevices(new Class<?>[] {VirtualEthernetCard.class});
+
+ if (systemVm) {
+ // System volumes doesn't require more than 1 SCSI controller as there is no requirement for data volumes.
+ ensureScsiDiskControllers(vmMo, systemVmScsiControllerType.toString(), numScsiControllerForSystemVm, firstScsiControllerBusNum);
+ } else {
+ ensureDiskControllers(vmMo, controllerInfo);
+ }
+ return this;
+ }
+
+ private VirtualMachineMO takeVmFromOtherHyperHost(VmwareHypervisorHost hyperHost, String vmName) throws Exception {
+
+ VirtualMachineMO vmMo = hyperHost.findVmOnPeerHyperHost(vmName);
+ if (vmMo != null) {
+ ManagedObjectReference morTargetPhysicalHost = hyperHost.findMigrationTarget(vmMo);
+ if (morTargetPhysicalHost == null) {
+ String msg = "VM " + vmName + " is on other host and we have no resource available to migrate and start it here";
+ LOGGER.error(msg);
+ throw new Exception(msg);
+ }
+
+ if (!vmMo.relocate(morTargetPhysicalHost)) {
+ String msg = "VM " + vmName + " is on other host and we failed to relocate it here";
+ LOGGER.error(msg);
+ throw new Exception(msg);
+ }
+
+ return vmMo;
+ }
+ return null;
+ }
+
+ public VirtualDevice[] getNicDevices() {
+ return nicDevices;
+ }
+ }
+
+ private class PrepareSytemVMPatchISOMethod {
+ private VmwareManager mgr;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int i;
+ private int ideUnitNumber;
+
+ public PrepareSytemVMPatchISOMethod(VmwareManager mgr, VmwareHypervisorHost hyperHost, VirtualMachineMO vmMo, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int i, int ideUnitNumber) {
+ this.mgr = mgr;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.i = i;
+ this.ideUnitNumber = ideUnitNumber;
+ }
+
+ public int getI() {
+ return i;
+ }
+
+ public int getIdeUnitNumber() {
+ return ideUnitNumber;
+ }
+
+ public PrepareSytemVMPatchISOMethod invoke() throws Exception {
+ // attach ISO (for patching of system VM)
+ DatastoreMO secDsMo = getDatastoreMOForSecStore(mgr, hyperHost);
+
+ deviceConfigSpecArray[i] = new VirtualDeviceConfigSpec();
+ Pair<VirtualDevice, Boolean> isoInfo = VmwareHelper
+ .prepareIsoDevice(vmMo, String.format("[%s] systemvm/%s", secDsMo.getName(), mgr.getSystemVMIsoFileNameOnDatastore()), secDsMo.getMor(), true, true,
+ ideUnitNumber++, i + 1);
+ deviceConfigSpecArray[i].setDevice(isoInfo.first());
+ if (isoInfo.second()) {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at new device " + vmwareResource.getGson().toJson(isoInfo.first()));
+ deviceConfigSpecArray[i].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+ } else {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+ deviceConfigSpecArray[i].setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ }
+ i++;
+ return this;
+ }
+
+ }
+ private DatastoreMO getDatastoreMOForSecStore(VmwareManager mgr, VmwareHypervisorHost hyperHost) throws Exception {
+ Pair<String, Long> secStoreUrlAndId = mgr.getSecondaryStorageStoreUrlAndId(Long.parseLong(vmwareResource.getDcId()));
+ String secStoreUrl = secStoreUrlAndId.first();
+ Long secStoreId = secStoreUrlAndId.second();
+ if (secStoreUrl == null) {
+ String msg = "secondary storage for dc " + vmwareResource.getDcId() + " is not ready yet?";
+ throw new Exception(msg);
+ }
+ mgr.prepareSecondaryStorageStore(secStoreUrl, secStoreId);
+
+ ManagedObjectReference morSecDs = vmwareResource.prepareSecondaryDatastoreOnHost(secStoreUrl);
+ if (morSecDs == null) {
+ String msg = "Failed to prepare secondary storage on host, secondary store url: " + secStoreUrl;
+ throw new Exception(msg);
+ }
+ return new DatastoreMO(hyperHost.getContext(), morSecDs);
+ }
+
+ /**
+ // Setup ROOT/DATA disk devices
+ */
+ private class DiskSetup {
+ private VirtualMachineTO vmSpec;
+ private DiskTO rootDiskTO;
+ private Pair<String, String> controllerInfo;
+ private VmwareContext context;
+ private DatacenterMO dcMo;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private DiskTO[] disks;
+ private HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails;
+ private VirtualMachineDiskInfoBuilder diskInfoBuilder;
+ private boolean hasSnapshot;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int deviceCount;
+ private int ideUnitNumber;
+ private int scsiUnitNumber;
+ private int ideControllerKey;
+ private int scsiControllerKey;
+ private DiskTO[] sortedDisks;
+ private boolean installAsIs;
+
+ public DiskSetup(VirtualMachineTO vmSpec, DiskTO rootDiskTO, Pair<String, String> controllerInfo, VmwareContext context, DatacenterMO dcMo, VmwareHypervisorHost hyperHost,
+ VirtualMachineMO vmMo, DiskTO[] disks, HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails, VirtualMachineDiskInfoBuilder diskInfoBuilder,
+ boolean hasSnapshot, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount, int ideUnitNumber, int scsiUnitNumber, int ideControllerKey, int scsiControllerKey, boolean installAsIs) {
+ this.vmSpec = vmSpec;
+ this.rootDiskTO = rootDiskTO;
+ this.controllerInfo = controllerInfo;
+ this.context = context;
+ this.dcMo = dcMo;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.disks = disks;
+ this.dataStoresDetails = dataStoresDetails;
+ this.diskInfoBuilder = diskInfoBuilder;
+ this.hasSnapshot = hasSnapshot;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.deviceCount = deviceCount;
+ this.ideUnitNumber = ideUnitNumber;
+ this.scsiUnitNumber = scsiUnitNumber;
+ this.ideControllerKey = ideControllerKey;
+ this.scsiControllerKey = scsiControllerKey;
+ this.installAsIs = installAsIs;
+ }
+
+ public DiskTO getRootDiskTO() {
+ return rootDiskTO;
+ }
+
+ public int getDeviceCount() {
+ return deviceCount;
+ }
+
+ public DiskTO[] getSortedDisks() {
+ return sortedDisks;
+ }
+
+ public DiskSetup invoke() throws Exception {
+ int controllerKey;
+ sortedDisks = sortVolumesByDeviceId(disks);
+ for (DiskTO vol : sortedDisks) {
+ if (vol.getType() == Volume.Type.ISO || installAsIs) {
+ continue;
+ }
+
+ VirtualMachineDiskInfo matchingExistingDisk = getMatchingExistingDisk(diskInfoBuilder, vol, hyperHost, context);
+ controllerKey = getDiskController(matchingExistingDisk, vol, vmSpec, ideControllerKey, scsiControllerKey);
+ String diskController = getDiskController(vmMo, matchingExistingDisk, vol, controllerInfo);
+
+ if (DiskControllerType.getType(diskController) == DiskControllerType.osdefault) {
+ diskController = vmMo.getRecommendedDiskController(null);
+ }
+ if (DiskControllerType.getType(diskController) == DiskControllerType.ide) {
+ controllerKey = vmMo.getIDEControllerKey(ideUnitNumber);
+ if (vol.getType() == Volume.Type.DATADISK) {
+ // Could be result of flip due to user configured setting or "osdefault" for data disks
+ // Ensure maximum of 2 data volumes over IDE controller, 3 includeing root volume
+ if (vmMo.getNumberOfVirtualDisks() > 3) {
+ throw new CloudRuntimeException(
+ "Found more than 3 virtual disks attached to this VM [" + vmMo.getVmName() + "]. Unable to implement the disks over " + diskController + " controller, as maximum number of devices supported over IDE controller is 4 includeing CDROM device.");
+ }
+ }
+ } else {
+ if (VmwareHelper.isReservedScsiDeviceNumber(scsiUnitNumber)) {
+ scsiUnitNumber++;
+ }
+
+ controllerKey = vmMo.getScsiDiskControllerKeyNoException(diskController, scsiUnitNumber);
+ if (controllerKey == -1) {
+ // This may happen for ROOT legacy VMs which doesn't have recommended disk controller when global configuration parameter 'vmware.root.disk.controller' is set to "osdefault"
+ // Retrieve existing controller and use.
+ Ternary<Integer, Integer, DiskControllerType> vmScsiControllerInfo = vmMo.getScsiControllerInfo();
+ DiskControllerType existingControllerType = vmScsiControllerInfo.third();
+ controllerKey = vmMo.getScsiDiskControllerKeyNoException(existingControllerType.toString(), scsiUnitNumber);
+ }
+ }
+ if (!hasSnapshot) {
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+
+ VolumeObjectTO volumeTO = (VolumeObjectTO)vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+ String iScsiName = null;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ iScsiName = details.get(DiskTO.IQN);
+ }
+
+ // if the storage is managed, iScsiName should not be null
+ String datastoreName = managed ? VmwareResource.getDatastoreName(iScsiName) : primaryStore.getUuid();
+ Pair<ManagedObjectReference, DatastoreMO> volumeDsDetails = dataStoresDetails.get(datastoreName);
+
+ assert (volumeDsDetails != null);
+
+ String[] diskChain = syncDiskChain(dcMo, vmMo, vmSpec, vol, matchingExistingDisk, dataStoresDetails);
+
+ int deviceNumber = -1;
+ if (controllerKey == vmMo.getIDEControllerKey(ideUnitNumber)) {
+ deviceNumber = ideUnitNumber % VmwareHelper.MAX_ALLOWED_DEVICES_IDE_CONTROLLER;
+ ideUnitNumber++;
+ } else {
+ deviceNumber = scsiUnitNumber % VmwareHelper.MAX_ALLOWED_DEVICES_SCSI_CONTROLLER;
+ scsiUnitNumber++;
+ }
+ VirtualDevice device = VmwareHelper.prepareDiskDevice(vmMo, null, controllerKey, diskChain, volumeDsDetails.first(), deviceNumber, deviceCount + 1);
+ if (vol.getType() == Volume.Type.ROOT)
+ rootDiskTO = vol;
+ deviceConfigSpecArray[deviceCount].setDevice(device);
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare volume at new device " + vmwareResource.getGson().toJson(device));
+
+ deviceCount++;
+ } else {
+ if (controllerKey == vmMo.getIDEControllerKey(ideUnitNumber))
+ ideUnitNumber++;
+ else
+ scsiUnitNumber++;
+ }
+ }
+ return this;
+ }
+
+ private int getDiskController(VirtualMachineDiskInfo matchingExistingDisk, DiskTO vol, VirtualMachineTO vmSpec, int ideControllerKey, int scsiControllerKey) {
+
+ int controllerKey;
+ if (matchingExistingDisk != null) {
+ LOGGER.info("Chose disk controller based on existing information: " + matchingExistingDisk.getDiskDeviceBusName());
+ if (matchingExistingDisk.getDiskDeviceBusName().startsWith("ide"))
+ return ideControllerKey;
+ else
+ return scsiControllerKey;
+ }
+
+ if (vol.getType() == Volume.Type.ROOT) {
+ Map<String, String> vmDetails = vmSpec.getDetails();
+ if (vmDetails != null && vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER) != null) {
+ if (vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER).equalsIgnoreCase("scsi")) {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> scsi, based on root disk controller settings: "
+ + vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER));
+ controllerKey = scsiControllerKey;
+ } else {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> ide, based on root disk controller settings: "
+ + vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER));
+ controllerKey = ideControllerKey;
+ }
+ } else {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> scsi. due to null root disk controller setting");
+ controllerKey = scsiControllerKey;
+ }
+
+ } else {
+ // DATA volume always use SCSI device
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> scsi");
+ controllerKey = scsiControllerKey;
+ }
+
+ return controllerKey;
+ }
+
+ private String getDiskController(VirtualMachineMO vmMo, VirtualMachineDiskInfo matchingExistingDisk, DiskTO vol, Pair<String, String> controllerInfo) throws Exception {
+ int controllerKey;
+ DiskControllerType controllerType = DiskControllerType.none;
+ if (matchingExistingDisk != null) {
+ String currentBusName = matchingExistingDisk.getDiskDeviceBusName();
+ if (currentBusName != null) {
+ LOGGER.info("Chose disk controller based on existing information: " + currentBusName);
+ if (currentBusName.startsWith("ide")) {
+ controllerType = DiskControllerType.ide;
+ } else if (currentBusName.startsWith("scsi")) {
+ controllerType = DiskControllerType.scsi;
+ }
+ }
+ if (controllerType == DiskControllerType.scsi || controllerType == DiskControllerType.none) {
+ Ternary<Integer, Integer, DiskControllerType> vmScsiControllerInfo = vmMo.getScsiControllerInfo();
+ controllerType = vmScsiControllerInfo.third();
+ }
+ return controllerType.toString();
+ }
+
+ if (vol.getType() == Volume.Type.ROOT) {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> " + controllerInfo.first()
+ + ", based on root disk controller settings at global configuration setting.");
+ return controllerInfo.first();
+ } else {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> " + controllerInfo.second()
+ + ", based on default data disk controller setting i.e. Operating system recommended."); // Need to bring in global configuration setting & template level setting.
+ return controllerInfo.second();
+ }
+ }
+
+ // return the finalized disk chain for startup, from top to bottom
+ private String[] syncDiskChain(DatacenterMO dcMo, VirtualMachineMO vmMo, VirtualMachineTO vmSpec, DiskTO vol, VirtualMachineDiskInfo diskInfo,
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails) throws Exception {
+
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ Map<String, String> details = vol.getDetails();
+ boolean isManaged = false;
+ String iScsiName = null;
+
+ if (details != null) {
+ isManaged = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ iScsiName = details.get(DiskTO.IQN);
+ }
+
+ // if the storage is managed, iScsiName should not be null
+ String datastoreName = isManaged ? VmwareResource.getDatastoreName(iScsiName) : primaryStore.getUuid();
+ Pair<ManagedObjectReference, DatastoreMO> volumeDsDetails = dataStoresDetails.get(datastoreName);
+
+ if (volumeDsDetails == null) {
+ throw new Exception("Primary datastore " + primaryStore.getUuid() + " is not mounted on host.");
+ }
+
+ DatastoreMO dsMo = volumeDsDetails.second();
+
+ // we will honor vCenter's meta if it exists
+ if (diskInfo != null) {
+ // to deal with run-time upgrade to maintain the new datastore folder structure
+ String disks[] = diskInfo.getDiskChain();
+ for (int i = 0; i < disks.length; i++) {
+ DatastoreFile file = new DatastoreFile(disks[i]);
+ if (!isManaged && file.getDir() != null && file.getDir().isEmpty()) {
+ LOGGER.info("Perform run-time datastore folder upgrade. sync " + disks[i] + " to VM folder");
+ disks[i] = VmwareStorageLayoutHelper.syncVolumeToVmDefaultFolder(dcMo, vmMo.getName(), dsMo, file.getFileBaseName(), VmwareManager.s_vmwareSearchExcludeFolder.value());
+ }
+ }
+ return disks;
+ }
+
+ final String datastoreDiskPath;
+
+ if (isManaged) {
+ String vmdkPath = new DatastoreFile(volumeTO.getPath()).getFileBaseName();
+
+ if (volumeTO.getVolumeType() == Volume.Type.ROOT) {
+ if (vmdkPath == null) {
+ vmdkPath = volumeTO.getName();
+ }
+
+ datastoreDiskPath = VmwareStorageLayoutHelper.syncVolumeToVmDefaultFolder(dcMo, vmMo.getName(), dsMo, vmdkPath);
+ } else {
+ if (vmdkPath == null) {
+ vmdkPath = dsMo.getName();
+ }
+
+ datastoreDiskPath = dsMo.getDatastorePath(vmdkPath + VmwareResource.VMDK_EXTENSION);
+ }
+ } else {
+ datastoreDiskPath = VmwareStorageLayoutHelper.syncVolumeToVmDefaultFolder(dcMo, vmMo.getName(), dsMo, volumeTO.getPath(), VmwareManager.s_vmwareSearchExcludeFolder.value());
+ }
+
+ if (!dsMo.fileExists(datastoreDiskPath)) {
+ LOGGER.warn("Volume " + volumeTO.getId() + " does not seem to exist on datastore, out of sync? path: " + datastoreDiskPath);
+ }
+
+ return new String[]{datastoreDiskPath};
+ }
+ }
+
+ /**
+ // Setup NIC devices
+ */
+ private class NicSetup {
+ private StartCommand cmd;
+ private VirtualMachineTO vmSpec;
+ private String vmInternalCSName;
+ private VmwareContext context;
+ private VmwareManager mgr;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private NicTO[] nics;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int deviceCount;
+ private int nicMask;
+ private int nicCount;
+ private Map<String, String> nicUuidToDvSwitchUuid;
+ private VirtualDevice[] nicDevices;
+
+ public NicSetup(StartCommand cmd, VirtualMachineTO vmSpec, String vmInternalCSName, VmwareContext context, VmwareManager mgr, VmwareHypervisorHost hyperHost,
+ VirtualMachineMO vmMo, NicTO[] nics, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount, VirtualDevice[] nicDevices) {
+ this.cmd = cmd;
+ this.vmSpec = vmSpec;
+ this.vmInternalCSName = vmInternalCSName;
+ this.context = context;
+ this.mgr = mgr;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.nics = nics;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.deviceCount = deviceCount;
+ this.nicDevices = nicDevices;
+ }
+
+ public int getDeviceCount() {
+ return deviceCount;
+ }
+
+ public int getNicMask() {
+ return nicMask;
+ }
+
+ public int getNicCount() {
+ return nicCount;
+ }
+
+ public Map<String, String> getNicUuidToDvSwitchUuid() {
+ return nicUuidToDvSwitchUuid;
+ }
+
+ public NicSetup invoke() throws Exception {
+ VirtualDevice nic;
+ nicMask = 0;
+ nicCount = 0;
+
+ if (vmSpec.getType() == VirtualMachine.Type.DomainRouter) {
+ doDomainRouterSetup();
+ }
+
+ VirtualEthernetCardType nicDeviceType = VirtualEthernetCardType.valueOf(vmSpec.getDetails().get(VmDetailConstants.NIC_ADAPTER));
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("VM " + vmInternalCSName + " will be started with NIC device type: " + nicDeviceType);
+ }
+
+ NiciraNvpApiVersion.logNiciraApiVersion();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.debug(String.format("deciding for VM '%s' to use orchestrated NICs or as is.", vmInternalCSName));
+ }
+ nicUuidToDvSwitchUuid = new HashMap<>();
+ LOGGER.info(String.format("adding %d nics to VM '%s'", nics.length, vmInternalCSName));
+ for (NicTO nicTo : sortNicsByDeviceId(nics)) {
+ LOGGER.info("Prepare NIC device based on NicTO: " + vmwareResource.getGson().toJson(nicTo));
+
+ boolean configureVServiceInNexus = (nicTo.getType() == Networks.TrafficType.Guest) && (vmSpec.getDetails().containsKey("ConfigureVServiceInNexus"));
+ VirtualMachine.Type vmType = cmd.getVirtualMachine().getType();
+ Pair<ManagedObjectReference, String> networkInfo = vmwareResource.prepareNetworkFromNicInfo(vmMo.getRunningHost(), nicTo, configureVServiceInNexus, vmType);
+ if ((nicTo.getBroadcastType() != Networks.BroadcastDomainType.Lswitch) || (nicTo.getBroadcastType() == Networks.BroadcastDomainType.Lswitch && NiciraNvpApiVersion.isApiVersionLowerThan("4.2"))) {
+ if (VmwareHelper.isDvPortGroup(networkInfo.first())) {
+ String dvSwitchUuid;
+ ManagedObjectReference dcMor = hyperHost.getHyperHostDatacenter();
+ DatacenterMO dataCenterMo = new DatacenterMO(context, dcMor);
+ ManagedObjectReference dvsMor = dataCenterMo.getDvSwitchMor(networkInfo.first());
+ dvSwitchUuid = dataCenterMo.getDvSwitchUuid(dvsMor);
+ LOGGER.info("Preparing NIC device on dvSwitch : " + dvSwitchUuid);
+ nic = VmwareHelper.prepareDvNicDevice(vmMo, networkInfo.first(), nicDeviceType, networkInfo.second(), dvSwitchUuid, nicTo.getMac(), deviceCount + 1, true, true);
+ if (nicTo.getUuid() != null) {
+ nicUuidToDvSwitchUuid.put(nicTo.getUuid(), dvSwitchUuid);
+ }
+ } else {
+ LOGGER.info("Preparing NIC device on network " + networkInfo.second());
+ nic = VmwareHelper.prepareNicDevice(vmMo, networkInfo.first(), nicDeviceType, networkInfo.second(), nicTo.getMac(), deviceCount + 1, true, true);
+ }
+ } else {
+ //if NSX API VERSION >= 4.2, connect to br-int (nsx.network), do not create portgroup else previous behaviour
+ nic = VmwareHelper.prepareNicOpaque(vmMo, nicDeviceType, networkInfo.second(), nicTo.getMac(), deviceCount + 1, true, true);
+ }
+
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ deviceConfigSpecArray[deviceCount].setDevice(nic);
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare NIC at new device " + vmwareResource.getGson().toJson(deviceConfigSpecArray[deviceCount]));
+
+ // this is really a hacking for DomR, upon DomR startup, we will reset all the NIC allocation after eth3
+ if (nicCount < 3)
+ nicMask |= (1 << nicCount);
+
+ deviceCount++;
+ nicCount++;
+ }
+ return this;
+ }
+
+ private void doDomainRouterSetup() throws Exception {
+ int extraPublicNics = mgr.getRouterExtraPublicNics();
+ if (extraPublicNics > 0 && vmSpec.getDetails().containsKey("PeerRouterInstanceName")) {
+ //Set identical MAC address for RvR on extra public interfaces
+ String peerRouterInstanceName = vmSpec.getDetails().get("PeerRouterInstanceName");
+
+ VirtualMachineMO peerVmMo = hyperHost.findVmOnHyperHost(peerRouterInstanceName);
+ if (peerVmMo == null) {
+ peerVmMo = hyperHost.findVmOnPeerHyperHost(peerRouterInstanceName);
+ }
+
+ if (peerVmMo != null) {
+ String oldMacSequence = generateMacSequence(nics);
+
+ for (int nicIndex = nics.length - extraPublicNics; nicIndex < nics.length; nicIndex++) {
+ VirtualDevice nicDevice = peerVmMo.getNicDeviceByIndex(nics[nicIndex].getDeviceId());
+ if (nicDevice != null) {
+ String mac = ((VirtualEthernetCard)nicDevice).getMacAddress();
+ if (mac != null) {
+ LOGGER.info("Use same MAC as previous RvR, the MAC is " + mac + " for extra NIC with device id: " + nics[nicIndex].getDeviceId());
+ nics[nicIndex].setMac(mac);
+ }
+ }
+ }
+
+ if (!StringUtils.isBlank(vmSpec.getBootArgs())) {
+ String newMacSequence = generateMacSequence(nics);
+ vmSpec.setBootArgs(vmwareResource.replaceNicsMacSequenceInBootArgs(oldMacSequence, newMacSequence, vmSpec));
+ }
+ }
+ }
+ }
+ }
+
+ private class IsoSetup {
+ private VirtualMachineTO vmSpec;
+ private VmwareManager mgr;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private DiskTO[] disks;
+ private DiskTO volIso;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int deviceCount;
+ private int ideUnitNumber;
+
+ public IsoSetup(VirtualMachineTO vmSpec, VmwareManager mgr, VmwareHypervisorHost hyperHost, VirtualMachineMO vmMo, DiskTO[] disks, DiskTO volIso,
+ VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount, int ideUnitNumber) {
+ this.vmSpec = vmSpec;
+ this.mgr = mgr;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.disks = disks;
+ this.volIso = volIso;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.deviceCount = deviceCount;
+ this.ideUnitNumber = ideUnitNumber;
+ }
+
+ public int getDeviceCount() {
+ return deviceCount;
+ }
+
+ public int getIdeUnitNumber() {
+ return ideUnitNumber;
+ }
+
+ public IsoSetup invoke() throws Exception {
+ //
+ // Setup ISO device
+ //
+
+ // vAPP ISO
+ // FR37 the native deploy mechs should create this for us
+ if (vmSpec.getOvfProperties() != null) {
+ if (LOGGER.isTraceEnabled()) {
+ // FR37 TODO add more usefull info (if we keep this bit
+ LOGGER.trace("adding iso for properties for 'xxx'");
+ }
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ Pair<VirtualDevice, Boolean> isoInfo = VmwareHelper.prepareIsoDevice(vmMo, null, null, true, true, ideUnitNumber++, deviceCount + 1);
+ deviceConfigSpecArray[deviceCount].setDevice(isoInfo.first());
+ if (isoInfo.second()) {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare vApp ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+ } else {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare vApp ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ }
+ deviceCount++;
+ }
+
+ // prepare systemvm patch ISO
+ if (vmSpec.getType() != VirtualMachine.Type.User) {
+ PrepareSytemVMPatchISOMethod prepareSytemVm = new PrepareSytemVMPatchISOMethod(mgr, hyperHost, vmMo, deviceConfigSpecArray, deviceCount, ideUnitNumber);
+ prepareSytemVm.invoke();
+ deviceCount = prepareSytemVm.getI();
+ ideUnitNumber = prepareSytemVm.getIdeUnitNumber();
+ } else {
+ // Note: we will always plug a CDROM device
+ if (volIso != null) {
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ISO) {
+
+ TemplateObjectTO iso = (TemplateObjectTO)vol.getData();
+
+ if (iso.getPath() != null && !iso.getPath().isEmpty()) {
+ DataStoreTO imageStore = iso.getDataStore();
+ if (!(imageStore instanceof NfsTO)) {
+ LOGGER.debug("unsupported protocol");
+ throw new Exception("unsupported protocol");
+ }
+ NfsTO nfsImageStore = (NfsTO)imageStore;
+ String isoPath = nfsImageStore.getUrl() + File.separator + iso.getPath();
+ Pair<String, ManagedObjectReference> isoDatastoreInfo = getIsoDatastoreInfo(hyperHost, isoPath);
+ assert (isoDatastoreInfo != null);
+ assert (isoDatastoreInfo.second() != null);
+
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ Pair<VirtualDevice, Boolean> isoInfo = VmwareHelper
+ .prepareIsoDevice(vmMo, isoDatastoreInfo.first(), isoDatastoreInfo.second(), true, true, ideUnitNumber++, deviceCount + 1);
+ deviceConfigSpecArray[deviceCount].setDevice(isoInfo.first());
+ if (isoInfo.second()) {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at new device " + vmwareResource.getGson().toJson(isoInfo.first()));
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+ } else {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ }
+ }
+ deviceCount++;
+ }
+ }
+ } else {
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ Pair<VirtualDevice, Boolean> isoInfo = VmwareHelper.prepareIsoDevice(vmMo, null, null, true, true, ideUnitNumber++, deviceCount + 1);
+ deviceConfigSpecArray[deviceCount].setDevice(isoInfo.first());
+ if (isoInfo.second()) {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+ } else {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ }
+ deviceCount++;
+ }
+ }
+ return this;
+ }
+
+ // isoUrl sample content :
+ // nfs://192.168.10.231/export/home/kelven/vmware-test/secondary/template/tmpl/2/200//200-2-80f7ee58-6eff-3a2d-bcb0-59663edf6d26.iso
+ private Pair<String, ManagedObjectReference> getIsoDatastoreInfo(VmwareHypervisorHost hyperHost, String isoUrl) throws Exception {
+
+ assert (isoUrl != null);
+ int isoFileNameStartPos = isoUrl.lastIndexOf("/");
+ if (isoFileNameStartPos < 0) {
+ throw new Exception("Invalid ISO path info");
+ }
+
+ String isoFileName = isoUrl.substring(isoFileNameStartPos);
+
+ int templateRootPos = isoUrl.indexOf("template/tmpl");
+ templateRootPos = (templateRootPos < 0 ? isoUrl.indexOf(ConfigDrive.CONFIGDRIVEDIR) : templateRootPos);
+ if (templateRootPos < 0) {
+ throw new Exception("Invalid ISO path info");
+ }
+
+ String storeUrl = isoUrl.substring(0, templateRootPos - 1);
+ String isoPath = isoUrl.substring(templateRootPos, isoFileNameStartPos);
+
+ ManagedObjectReference morDs = vmwareResource.prepareSecondaryDatastoreOnHost(storeUrl);
+ DatastoreMO dsMo = new DatastoreMO(vmwareResource.getServiceContext(), morDs);
+
+ return new Pair<String, ManagedObjectReference>(String.format("[%s] %s%s", dsMo.getName(), isoPath, isoFileName), morDs);
+ }
+ }
+
+ private class PrepareRunningVMForConfiguration {
+ private String vmInternalCSName;
+ private Pair<String, String> controllerInfo;
+ private Boolean systemVm;
+ private VirtualMachineMO vmMo;
+ private DiskControllerType systemVmScsiControllerType;
+ private int firstScsiControllerBusNum;
+ private int numScsiControllerForSystemVm;
+ private VirtualMachineDiskInfoBuilder diskInfoBuilder;
+ private VirtualDevice[] nicDevices;
+ private boolean hasSnapshot;
+
+ public PrepareRunningVMForConfiguration(String vmInternalCSName, Pair<String, String> controllerInfo, Boolean systemVm, VirtualMachineMO vmMo,
+ DiskControllerType systemVmScsiControllerType, int firstScsiControllerBusNum, int numScsiControllerForSystemVm) {
+ this.vmInternalCSName = vmInternalCSName;
+ this.controllerInfo = controllerInfo;
+ this.systemVm = systemVm;
+ this.vmMo = vmMo;
+ this.systemVmScsiControllerType = systemVmScsiControllerType;
+ this.firstScsiControllerBusNum = firstScsiControllerBusNum;
+ this.numScsiControllerForSystemVm = numScsiControllerForSystemVm;
+ }
+
+ public VirtualMachineDiskInfoBuilder getDiskInfoBuilder() {
+ return diskInfoBuilder;
+ }
+
+ public VirtualDevice[] getNicDevices() {
+ return nicDevices;
+ }
+
+ public boolean isHasSnapshot() {
+ return hasSnapshot;
+ }
+
+ public PrepareRunningVMForConfiguration invoke() throws Exception {
+ LOGGER.info("VM " + vmInternalCSName + " already exists, tear down devices for reconfiguration");
+ if (VmwareResource.getVmPowerState(vmMo) != VirtualMachine.PowerState.PowerOff) {
+ vmMo.safePowerOff(vmwareResource.getShutdownWaitMs());
+ }
+
+ // retrieve disk information before we tear down
+ diskInfoBuilder = vmMo.getDiskInfoBuilder();
+ hasSnapshot = vmMo.hasSnapshot();
+ nicDevices = vmMo.getNicDevices();
+ // FR37 - only tear nics, and add nics per the provided nics list
+ if (LOGGER.isTraceEnabled()) {
+ String netMsg = "tearing down networks :";
+ for (VirtualDevice nic : vmMo.getNicDevices()) {
+ netMsg += nic.getDeviceInfo().getLabel()+":";
+ }
+ LOGGER.trace(netMsg);
+ }
+ // FR37 save vmMo.getNicDevices() to ensure recreation?
+ vmMo.tearDownDevices(new Class<?>[] {VirtualEthernetCard.class});
+ /*
+ if (!hasSnapshot) {
+ // FR37 do we need to do this, ever?:
+ vmMo.tearDownDevices(new Class<?>[] {VirtualDisk.class});
+ }
+ */
Review comment:
how to handle this?
##########
File path: plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/StartCommandExecutor.java
##########
@@ -0,0 +1,2247 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.hypervisor.vmware.resource;
+
+import java.io.File;
+import java.rmi.RemoteException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import com.cloud.agent.api.to.DataTO;
+import com.vmware.vim25.VirtualDiskFlatVer2BackingInfo;
+import org.apache.cloudstack.api.ApiConstants;
+import org.apache.cloudstack.storage.configdrive.ConfigDrive;
+import org.apache.cloudstack.storage.to.TemplateObjectTO;
+import org.apache.cloudstack.storage.to.VolumeObjectTO;
+import org.apache.cloudstack.utils.volume.VirtualMachineDiskInfo;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.commons.lang.ArrayUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.log4j.Logger;
+
+import com.cloud.agent.api.Command;
+import com.cloud.agent.api.StartAnswer;
+import com.cloud.agent.api.StartCommand;
+import com.cloud.agent.api.storage.OVFPropertyTO;
+import com.cloud.agent.api.to.DataStoreTO;
+import com.cloud.agent.api.to.DiskTO;
+import com.cloud.agent.api.to.NfsTO;
+import com.cloud.agent.api.to.NicTO;
+import com.cloud.agent.api.to.VirtualMachineTO;
+import com.cloud.configuration.Resource;
+import com.cloud.hypervisor.vmware.VmwareResourceException;
+import com.cloud.hypervisor.vmware.manager.VmwareManager;
+import com.cloud.hypervisor.vmware.mo.CustomFieldConstants;
+import com.cloud.hypervisor.vmware.mo.DatacenterMO;
+import com.cloud.hypervisor.vmware.mo.DatastoreFile;
+import com.cloud.hypervisor.vmware.mo.DatastoreMO;
+import com.cloud.hypervisor.vmware.mo.DiskControllerType;
+import com.cloud.hypervisor.vmware.mo.HostMO;
+import com.cloud.hypervisor.vmware.mo.HypervisorHostHelper;
+import com.cloud.hypervisor.vmware.mo.TaskMO;
+import com.cloud.hypervisor.vmware.mo.VirtualEthernetCardType;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineDiskInfoBuilder;
+import com.cloud.hypervisor.vmware.mo.VirtualMachineMO;
+import com.cloud.hypervisor.vmware.mo.VmwareHypervisorHost;
+import com.cloud.hypervisor.vmware.util.VmwareContext;
+import com.cloud.hypervisor.vmware.util.VmwareHelper;
+import com.cloud.network.Networks;
+import com.cloud.storage.Storage;
+import com.cloud.storage.Volume;
+import com.cloud.storage.resource.VmwareStorageLayoutHelper;
+import com.cloud.storage.resource.VmwareStorageProcessor;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.Pair;
+import com.cloud.utils.Ternary;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.cloud.utils.nicira.nvp.plugin.NiciraNvpApiVersion;
+import com.cloud.vm.VirtualMachine;
+import com.cloud.vm.VmDetailConstants;
+import com.vmware.vim25.BoolPolicy;
+import com.vmware.vim25.DVPortConfigInfo;
+import com.vmware.vim25.DVPortConfigSpec;
+import com.vmware.vim25.DasVmPriority;
+import com.vmware.vim25.DistributedVirtualPort;
+import com.vmware.vim25.DistributedVirtualSwitchPortConnection;
+import com.vmware.vim25.DistributedVirtualSwitchPortCriteria;
+import com.vmware.vim25.ManagedObjectReference;
+import com.vmware.vim25.OptionValue;
+import com.vmware.vim25.VMwareDVSPortSetting;
+import com.vmware.vim25.VirtualDevice;
+import com.vmware.vim25.VirtualDeviceBackingInfo;
+import com.vmware.vim25.VirtualDeviceConfigSpec;
+import com.vmware.vim25.VirtualDeviceConfigSpecOperation;
+import com.vmware.vim25.VirtualDisk;
+import com.vmware.vim25.VirtualEthernetCard;
+import com.vmware.vim25.VirtualEthernetCardDistributedVirtualPortBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardNetworkBackingInfo;
+import com.vmware.vim25.VirtualEthernetCardOpaqueNetworkBackingInfo;
+import com.vmware.vim25.VirtualMachineBootOptions;
+import com.vmware.vim25.VirtualMachineConfigSpec;
+import com.vmware.vim25.VirtualMachineFileInfo;
+import com.vmware.vim25.VirtualMachineFileLayoutEx;
+import com.vmware.vim25.VirtualMachineGuestOsIdentifier;
+import com.vmware.vim25.VirtualUSBController;
+import com.vmware.vim25.VmConfigInfo;
+import com.vmware.vim25.VmwareDistributedVirtualSwitchVlanIdSpec;
+
+class StartCommandExecutor {
+ private static final Logger LOGGER = Logger.getLogger(StartCommandExecutor.class);
+
+ private final VmwareResource vmwareResource;
+
+ public StartCommandExecutor(VmwareResource vmwareResource) {
+ this.vmwareResource = vmwareResource;
+ }
+
+ // FR37 the monster blob god method: reduce to 320 so far and counting
+ protected StartAnswer execute(StartCommand cmd) {
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Executing resource StartCommand: " + vmwareResource.getGson().toJson(cmd));
+ }
+
+ VirtualMachineTO vmSpec = cmd.getVirtualMachine();
+
+ VirtualMachineData existingVm = null;
+
+ Pair<String, String> names = composeVmNames(vmSpec);
+ String vmInternalCSName = names.first();
+ String vmNameOnVcenter = names.second();
+
+ DiskTO rootDiskTO = null;
+
+ Pair<String, String> controllerInfo = getDiskControllerInfo(vmSpec);
+
+ Boolean systemVm = vmSpec.getType().isUsedBySystem();
+
+ VmwareContext context = vmwareResource.getServiceContext();
+ DatacenterMO dcMo = null;
+ try {
+ VmwareManager mgr = context.getStockObject(VmwareManager.CONTEXT_STOCK_NAME);
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(context);
+ dcMo = new DatacenterMO(hyperHost.getContext(), hyperHost.getHyperHostDatacenter());
+
+ // checkIfVmExistsInVcenter(vmInternalCSName, vmNameOnVcenter, dcMo);
+ // FR37 - We expect VM to be already cloned and available at this point
+ VirtualMachineMO vmMo = dcMo.findVm(vmInternalCSName);
+ if (vmMo == null) {
+ vmMo = dcMo.findVm(vmNameOnVcenter);
+ }
+
+ DiskTO[] specDisks = vmSpec.getDisks();
+ boolean installAsIs = StringUtils.isNotEmpty(vmSpec.getTemplateLocation());
+ // FR37 if startcommand contains enough info: a template url/-location and flag; deploy OVA as is
+ if (vmMo == null && installAsIs) {
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("deploying OVA from %s as is", vmSpec.getTemplateLocation()));
+ }
+ getStorageProcessor().cloneVMFromTemplate(vmSpec.getTemplateName(), vmInternalCSName, vmSpec.getTemplatePrimaryStoreUuid());
+ vmMo = dcMo.findVm(vmInternalCSName);
+ mapSpecDisksToClonedDisks(vmMo, vmInternalCSName, specDisks);
+ }
+
+ // VM may not have been on the same host, relocate to expected host
+ if (vmMo != null) {
+ vmMo.relocate(hyperHost.getMor());
+ // Get updated MO
+ vmMo = hyperHost.findVmOnHyperHost(vmMo.getVmName());
+ }
+
+ String guestOsId = translateGuestOsIdentifier(vmSpec.getArch(), vmSpec.getOs(), vmSpec.getPlatformEmulator()).value();
+ DiskTO[] disks = validateDisks(specDisks);
+ NicTO[] nics = vmSpec.getNics();
+
+ // FIXME: disks logic here, why is disks/volumes during copy not set with pool ID?
+ // FR37 TODO if deployasis a new VM no datastores are known (yet) and we need to get the data store from the tvmspec content library / template location
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails = inferDatastoreDetailsFromDiskInfo(hyperHost, context, disks, cmd);
+ if ((dataStoresDetails == null) || (dataStoresDetails.isEmpty())) {
+ String msg = "Unable to locate datastore details of the volumes to be attached";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ // FR37 - this may not be necessary the cloned VM will have disks and knowledge of datastore paths/location?
+ // throw new Exception(msg);
+ }
+
+ // FR37 - this may need checking, if the first datastore is the right one - ideally it should be datastore where first disk is hosted
+ DatastoreMO dsRootVolumeIsOn = null; //
+ if (! installAsIs) {
+ dsRootVolumeIsOn = getDatastoreThatRootDiskIsOn(dataStoresDetails, disks);
+
+ if (dsRootVolumeIsOn == null) {
+ String msg = "Unable to locate datastore details of root volume";
+ LOGGER.error(msg);
+ // throw a more specific Exception
+ throw new VmwareResourceException(msg);
+ }
+ }
+
+ VirtualMachineDiskInfoBuilder diskInfoBuilder = null;
+ VirtualDevice[] nicDevices = null;
+ DiskControllerType systemVmScsiControllerType = DiskControllerType.lsilogic;
+ int firstScsiControllerBusNum = 0;
+ int numScsiControllerForSystemVm = 1;
+ boolean hasSnapshot = false;
+ if (vmMo != null) {
+ PrepareRunningVMForConfiguration prepareRunningVMForConfiguration = new PrepareRunningVMForConfiguration(vmInternalCSName, controllerInfo, systemVm, vmMo,
+ systemVmScsiControllerType, firstScsiControllerBusNum, numScsiControllerForSystemVm);
+ prepareRunningVMForConfiguration.invoke();
+ diskInfoBuilder = prepareRunningVMForConfiguration.getDiskInfoBuilder();
+ nicDevices = prepareRunningVMForConfiguration.getNicDevices();
+ hasSnapshot = prepareRunningVMForConfiguration.isHasSnapshot();
+ } else {
+ ManagedObjectReference morDc = hyperHost.getHyperHostDatacenter();
+ assert (morDc != null);
+
+ vmMo = hyperHost.findVmOnPeerHyperHost(vmInternalCSName);
+ if (vmMo != null) {
+ VirtualMachineRecycler virtualMachineRecycler = new VirtualMachineRecycler(vmInternalCSName, controllerInfo, systemVm, hyperHost, vmMo,
+ systemVmScsiControllerType, firstScsiControllerBusNum, numScsiControllerForSystemVm);
+ virtualMachineRecycler.invoke();
+ diskInfoBuilder = virtualMachineRecycler.getDiskInfoBuilder();
+ hasSnapshot = virtualMachineRecycler.isHasSnapshot();
+ nicDevices = virtualMachineRecycler.getNicDevices();
+ } else {
+ // FR37 we just didn't find a VM by name 'vmInternalCSName', so why is this here?
+ existingVm = unregisterOnOtherClusterButHoldOnToOldVmData(vmInternalCSName, dcMo);
+
+ createNewVm(context, vmSpec, installAsIs, vmInternalCSName, vmNameOnVcenter, controllerInfo, systemVm, mgr, hyperHost, guestOsId, disks, dataStoresDetails,
+ dsRootVolumeIsOn);
+ }
+
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ if (vmMo == null) {
+ throw new Exception("Failed to find the newly create or relocated VM. vmName: " + vmInternalCSName);
+ }
+ }
+ // vmMo should now be a stopped VM on the intended host
+ // The number of disks changed must be 0 for install as is, as the VM is a clone
+ int disksChanges = !installAsIs ? disks.length : 0;
+ int totalChangeDevices = disksChanges + nics.length;
+ int hackDeviceCount = 0;
+ if (diskInfoBuilder != null) {
+ hackDeviceCount += diskInfoBuilder.getDiskCount();
+ }
+ hackDeviceCount += nicDevices == null ? 0 : nicDevices.length;
+ // vApp cdrom device
+ // HACK ALERT: ovf properties might not be the only or defining feature of vApps; needs checking
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("current count(s) desired: %d/ found:%d. now adding device to device count for vApp config ISO", totalChangeDevices, hackDeviceCount));
+ }
+ if (vmSpec.getOvfProperties() != null) {
+ totalChangeDevices++;
+ }
+
+ DiskTO volIso = null;
+ if (vmSpec.getType() != VirtualMachine.Type.User) {
+ // system VM needs a patch ISO
+ totalChangeDevices++;
+ } else {
+ volIso = getIsoDiskTO(disks);
+ if (volIso == null)
+ totalChangeDevices++;
+ }
+
+ VirtualMachineConfigSpec vmConfigSpec = new VirtualMachineConfigSpec();
+
+ VmwareHelper.setBasicVmConfig(vmConfigSpec, vmSpec.getCpus(), vmSpec.getMaxSpeed(), vmwareResource.getReservedCpuMHZ(vmSpec), (int)(vmSpec.getMaxRam() / (1024 * 1024)),
+ vmwareResource.getReservedMemoryMb(vmSpec), guestOsId, vmSpec.getLimitCpuUse());
+
+ int numCoresPerSocket = adjustNumberOfCoresPerSocket(vmSpec, vmMo, vmConfigSpec);
+
+ // Check for hotadd settings
+ vmConfigSpec.setMemoryHotAddEnabled(vmMo.isMemoryHotAddSupported(guestOsId));
+
+ String hostApiVersion = ((HostMO)hyperHost).getHostAboutInfo().getApiVersion();
+ if (numCoresPerSocket > 1 && hostApiVersion.compareTo("5.0") < 0) {
+ LOGGER.warn("Dynamic scaling of CPU is not supported for Virtual Machines with multi-core vCPUs in case of ESXi hosts 4.1 and prior. Hence CpuHotAdd will not be"
+ + " enabled for Virtual Machine: " + vmInternalCSName);
+ vmConfigSpec.setCpuHotAddEnabled(false);
+ } else {
+ vmConfigSpec.setCpuHotAddEnabled(vmMo.isCpuHotAddSupported(guestOsId));
+ }
+
+ vmwareResource.configNestedHVSupport(vmMo, vmSpec, vmConfigSpec);
+
+ VirtualDeviceConfigSpec[] deviceConfigSpecArray = new VirtualDeviceConfigSpec[totalChangeDevices];
+ int deviceCount = 0;
+ int ideUnitNumber = 0;
+ int scsiUnitNumber = 0;
+ int ideControllerKey = vmMo.getIDEDeviceControllerKey();
+ int scsiControllerKey = vmMo.getScsiDeviceControllerKeyNoException();
+
+ IsoSetup isoSetup = new IsoSetup(vmSpec, mgr, hyperHost, vmMo, disks, volIso, deviceConfigSpecArray, deviceCount, ideUnitNumber);
+ isoSetup.invoke();
+ deviceCount = isoSetup.getDeviceCount();
+ ideUnitNumber = isoSetup.getIdeUnitNumber();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("setting up %d disks with root from %s", diskInfoBuilder.getDiskCount(), vmwareResource.getGson().toJson(rootDiskTO)));
+ }
+
+ DiskSetup diskSetup = new DiskSetup(vmSpec, rootDiskTO, controllerInfo, context, dcMo, hyperHost, vmMo, disks, dataStoresDetails, diskInfoBuilder, hasSnapshot,
+ deviceConfigSpecArray, deviceCount, ideUnitNumber, scsiUnitNumber, ideControllerKey, scsiControllerKey, installAsIs);
+ diskSetup.invoke();
+
+ rootDiskTO = diskSetup.getRootDiskTO();
+ deviceCount = diskSetup.getDeviceCount();
+ DiskTO[] sortedDisks = diskSetup.getSortedDisks();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("seting up %d disks with root from %s", sortedDisks.length, vmwareResource.getGson().toJson(rootDiskTO)));
+ }
+
+ deviceCount += setupUsbDevicesAndGetCount(vmInternalCSName, vmMo, guestOsId, deviceConfigSpecArray, deviceCount);
+
+ NicSetup nicSetup = new NicSetup(cmd, vmSpec, vmInternalCSName, context, mgr, hyperHost, vmMo, nics, deviceConfigSpecArray, deviceCount, nicDevices);
+ nicSetup.invoke();
+
+ deviceCount = nicSetup.getDeviceCount();
+ int nicMask = nicSetup.getNicMask();
+ int nicCount = nicSetup.getNicCount();
+ Map<String, String> nicUuidToDvSwitchUuid = nicSetup.getNicUuidToDvSwitchUuid();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("device count: %d; nic count %d", deviceCount, nicCount));
+ }
+ for (int j = 0; j < deviceCount; j++)
+ vmConfigSpec.getDeviceChange().add(deviceConfigSpecArray[j]);
+
+ //
+ // Setup VM options
+ //
+
+ // pass boot arguments through machine.id & perform customized options to VMX
+ ArrayList<OptionValue> extraOptions = new ArrayList<>();
+ configBasicExtraOption(extraOptions, vmSpec);
+ configNvpExtraOption(extraOptions, vmSpec, nicUuidToDvSwitchUuid);
+ configCustomExtraOption(extraOptions, vmSpec);
+
+ // config for NCC
+ VirtualMachine.Type vmType = cmd.getVirtualMachine().getType();
+ if (vmType.equals(VirtualMachine.Type.NetScalerVm)) {
+ NicTO mgmtNic = vmSpec.getNics()[0];
+ OptionValue option = new OptionValue();
+ option.setKey("machine.id");
+ option.setValue("ip=" + mgmtNic.getIp() + "&netmask=" + mgmtNic.getNetmask() + "&gateway=" + mgmtNic.getGateway());
+ extraOptions.add(option);
+ }
+
+ // config VNC
+ String keyboardLayout = null;
+ if (vmSpec.getDetails() != null)
+ keyboardLayout = vmSpec.getDetails().get(VmDetailConstants.KEYBOARD);
+ vmConfigSpec.getExtraConfig().addAll(
+ Arrays.asList(vmwareResource.configureVnc(extraOptions.toArray(new OptionValue[0]), hyperHost, vmInternalCSName, vmSpec.getVncPassword(), keyboardLayout)));
+
+ // config video card
+ vmwareResource.configureVideoCard(vmMo, vmSpec, vmConfigSpec);
+
+ // Set OVF properties (if available)
+ Pair<String, List<OVFPropertyTO>> ovfPropsMap = vmSpec.getOvfProperties();
+ VmConfigInfo templateVappConfig;
+ List<OVFPropertyTO> ovfProperties;
+ if (ovfPropsMap != null) {
+ String vmTemplate = ovfPropsMap.first();
+ LOGGER.info("Find VM template " + vmTemplate);
+ VirtualMachineMO vmTemplateMO = dcMo.findVm(vmTemplate);
+ templateVappConfig = vmTemplateMO.getConfigInfo().getVAppConfig();
+ ovfProperties = ovfPropsMap.second();
+ // Set OVF properties (if available)
+ if (CollectionUtils.isNotEmpty(ovfProperties)) {
+ LOGGER.info("Copying OVF properties from template and setting them to the values the user provided");
+ vmwareResource.copyVAppConfigsFromTemplate(templateVappConfig, ovfProperties, vmConfigSpec);
+ }
+ }
+
+ checkBootOptions(vmSpec, vmConfigSpec);
+
+ //
+ // Configure VM
+ //
+ if (!vmMo.configureVm(vmConfigSpec)) {
+ throw new Exception("Failed to configure VM before start. vmName: " + vmInternalCSName);
+ }
+ // FR37 TODO reconcile disks now!!! they are configured, check and add them to the returning vmTO
+ if (vmSpec.getType() == VirtualMachine.Type.DomainRouter) {
+ hyperHost.setRestartPriorityForVM(vmMo, DasVmPriority.HIGH.value());
+ }
+
+ // Resizing root disk only when explicit requested by user
+ final Map<String, String> vmDetails = cmd.getVirtualMachine().getDetails();
+ if (rootDiskTO != null && !hasSnapshot && (vmDetails != null && vmDetails.containsKey(ApiConstants.ROOT_DISK_SIZE))) {
+ resizeRootDiskOnVMStart(vmMo, rootDiskTO, hyperHost, context);
+ }
+
+ //
+ // Post Configuration
+ //
+
+ vmMo.setCustomFieldValue(CustomFieldConstants.CLOUD_NIC_MASK, String.valueOf(nicMask));
+ postNvpConfigBeforeStart(vmMo, vmSpec);
+
+ Map<String, Map<String, String>> iqnToData = new HashMap<>();
+
+ postDiskConfigBeforeStart(vmMo, vmSpec, sortedDisks, iqnToData, hyperHost, context, installAsIs);
+
+ //
+ // Power-on VM
+ //
+ if (!vmMo.powerOn()) {
+ throw new Exception("Failed to start VM. vmName: " + vmInternalCSName + " with hostname " + vmNameOnVcenter);
+ }
+
+ if (installAsIs) {
+ // Set disks as the disks path and chain is retrieved from the cloned VM disks
+ cmd.getVirtualMachine().setDisks(disks);
+ }
+
+ StartAnswer startAnswer = new StartAnswer(cmd);
+
+ startAnswer.setIqnToData(iqnToData);
+
+ deleteOldVersionOfTheStartedVM(existingVm, dcMo, vmMo);
+
+ return startAnswer;
+ } catch (Throwable e) {
+ return handleStartFailure(cmd, vmSpec, existingVm, dcMo, e);
+ } finally {
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("finally done with %s", vmwareResource.getGson().toJson(cmd)));
+ }
+ }
+ }
+
+ /**
+ * Modify the specDisks information to match the cloned VM's disks (from vmMo VM)
+ */
+ private void mapSpecDisksToClonedDisks(VirtualMachineMO vmMo, String vmInternalCSName, DiskTO[] specDisks) {
+ try {
+ if (vmMo != null && ArrayUtils.isNotEmpty(specDisks)) {
+ List<VirtualDisk> vmDisks = vmMo.getVirtualDisks();
+ List<DiskTO> sortedDisks = Arrays.asList(sortVolumesByDeviceId(specDisks))
+ .stream()
+ .filter(x -> x.getType() == Volume.Type.ROOT)
+ .collect(Collectors.toList());
+ if (sortedDisks.size() != vmDisks.size()) {
+ LOGGER.error("Different number of root disks spec vs cloned deploy-as-is VM disks: " + sortedDisks.size() + " - " + vmDisks.size());
+ return;
+ }
+ for (int i = 0; i < sortedDisks.size(); i++) {
+ DiskTO specDisk = sortedDisks.get(i);
+ VirtualDisk vmDisk = vmDisks.get(i);
+ DataTO dataVolume = specDisk.getData();
+ if (dataVolume instanceof VolumeObjectTO) {
+ VolumeObjectTO volumeObjectTO = (VolumeObjectTO) dataVolume;
+ if (!volumeObjectTO.getSize().equals(vmDisk.getCapacityInBytes())) {
+ LOGGER.info("Mapped disk size is not the same as the cloned VM disk size: " +
+ volumeObjectTO.getSize() + " - " + vmDisk.getCapacityInBytes());
+ }
+ VirtualDeviceBackingInfo backingInfo = vmDisk.getBacking();
+ if (backingInfo instanceof VirtualDiskFlatVer2BackingInfo) {
+ VirtualDiskFlatVer2BackingInfo backing = (VirtualDiskFlatVer2BackingInfo) backingInfo;
+ String fileName = backing.getFileName();
+ if (StringUtils.isNotBlank(fileName)) {
+ String[] fileNameParts = fileName.split(" ");
+ String datastoreUuid = fileNameParts[0].replace("[", "").replace("]", "");
+ String relativePath = fileNameParts[1].split("/")[1].replace(".vmdk", "");
+ String vmSpecDatastoreUuid = volumeObjectTO.getDataStore().getUuid().replaceAll("-", "");
+ if (!datastoreUuid.equals(vmSpecDatastoreUuid)) {
+ LOGGER.info("Mapped disk datastore UUID is not the same as the cloned VM datastore UUID: " +
+ datastoreUuid + " - " + vmSpecDatastoreUuid);
+ }
+ volumeObjectTO.setPath(relativePath);
+ specDisk.setPath(relativePath);
+ }
+ }
+ }
+ }
+ }
+ } catch (Exception e) {
+ String msg = "Error mapping deploy-as-is VM disks from cloned VM " + vmInternalCSName;
+ LOGGER.error(msg, e);
+ throw new CloudRuntimeException(e);
+ }
+ }
+
+ private StartAnswer handleStartFailure(StartCommand cmd, VirtualMachineTO vmSpec, VirtualMachineData existingVm, DatacenterMO dcMo, Throwable e) {
+ if (e instanceof RemoteException) {
+ LOGGER.warn("Encounter remote exception to vCenter, invalidate VMware session context");
+ vmwareResource.invalidateServiceContext();
+ }
+
+ String msg = "StartCommand failed due to " + VmwareHelper.getExceptionMessage(e);
+ LOGGER.warn(msg, e);
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.trace(String.format("didn't start VM %s", vmSpec.getName()), e);
+ }
+ StartAnswer startAnswer = new StartAnswer(cmd, msg);
+ if ( e instanceof VmAlreadyExistsInVcenter) {
+ startAnswer.setContextParam("stopRetry", "true");
+ }
+ reRegisterExistingVm(existingVm, dcMo);
+
+ return startAnswer;
+ }
+
+ private void deleteOldVersionOfTheStartedVM(VirtualMachineData existingVm, DatacenterMO dcMo, VirtualMachineMO vmMo) throws Exception {
+ // Since VM was successfully powered-on, if there was an existing VM in a different cluster that was unregistered, delete all the files associated with it.
+ if (existingVm != null && existingVm.vmName != null && existingVm.vmFileLayout != null) {
+ List<String> vmDatastoreNames = new ArrayList<>();
+ for (DatastoreMO vmDatastore : vmMo.getAllDatastores()) {
+ vmDatastoreNames.add(vmDatastore.getName());
+ }
+ // Don't delete files that are in a datastore that is being used by the new VM as well (zone-wide datastore).
+ List<String> skipDatastores = new ArrayList<>();
+ for (DatastoreMO existingDatastore : existingVm.datastores) {
+ if (vmDatastoreNames.contains(existingDatastore.getName())) {
+ skipDatastores.add(existingDatastore.getName());
+ }
+ }
+ vmwareResource.deleteUnregisteredVmFiles(existingVm.vmFileLayout, dcMo, true, skipDatastores);
+ }
+ }
+
+ private int adjustNumberOfCoresPerSocket(VirtualMachineTO vmSpec, VirtualMachineMO vmMo, VirtualMachineConfigSpec vmConfigSpec) throws Exception {
+ // Check for multi-cores per socket settings
+ int numCoresPerSocket = 1;
+ String coresPerSocket = vmSpec.getDetails().get(VmDetailConstants.CPU_CORE_PER_SOCKET);
+ if (coresPerSocket != null) {
+ String apiVersion = HypervisorHostHelper.getVcenterApiVersion(vmMo.getContext());
+ // Property 'numCoresPerSocket' is supported since vSphere API 5.0
+ if (apiVersion.compareTo("5.0") >= 0) {
+ numCoresPerSocket = NumbersUtil.parseInt(coresPerSocket, 1);
+ vmConfigSpec.setNumCoresPerSocket(numCoresPerSocket);
+ }
+ }
+ return numCoresPerSocket;
+ }
+
+ /**
+ // Setup USB devices
+ */
+ private int setupUsbDevicesAndGetCount(String vmInternalCSName, VirtualMachineMO vmMo, String guestOsId, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount)
+ throws Exception {
+ int usbDeviceCount = 0;
+ if (guestOsId.startsWith("darwin")) { //Mac OS
+ VirtualDevice[] devices = vmMo.getMatchedDevices(new Class<?>[] {VirtualUSBController.class});
+ if (devices.length == 0) {
+ LOGGER.debug("No USB Controller device on VM Start. Add USB Controller device for Mac OS VM " + vmInternalCSName);
+
+ //For Mac OS X systems, the EHCI+UHCI controller is enabled by default and is required for USB mouse and keyboard access.
+ VirtualDevice usbControllerDevice = VmwareHelper.prepareUSBControllerDevice();
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ deviceConfigSpecArray[deviceCount].setDevice(usbControllerDevice);
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare USB controller at new device " + vmwareResource.getGson().toJson(deviceConfigSpecArray[deviceCount]));
+
+ deviceCount++;
+ usbDeviceCount++;
+ } else {
+ LOGGER.debug("USB Controller device exists on VM Start for Mac OS VM " + vmInternalCSName);
+ }
+ }
+ return usbDeviceCount;
+ }
+
+ private void createNewVm(VmwareContext context, VirtualMachineTO vmSpec, boolean installAsIs, String vmInternalCSName, String vmNameOnVcenter, Pair<String, String> controllerInfo, Boolean systemVm,
+ VmwareManager mgr, VmwareHypervisorHost hyperHost, String guestOsId, DiskTO[] disks, HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails,
+ DatastoreMO dsRootVolumeIsOn) throws Exception {
+ VirtualMachineMO vmMo;
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = getRootDiskDataStoreDetails(disks, dataStoresDetails);
+
+ assert (vmSpec.getMinSpeed() != null) && (rootDiskDataStoreDetails != null);
+
+ boolean vmFolderExists = rootDiskDataStoreDetails.second().folderExists(String.format("[%s]", rootDiskDataStoreDetails.second().getName()), vmNameOnVcenter);
+ String vmxFileFullPath = dsRootVolumeIsOn.searchFileInSubFolders(vmNameOnVcenter + ".vmx", false, VmwareManager.s_vmwareSearchExcludeFolder.value());
+ if (vmFolderExists && vmxFileFullPath != null) { // VM can be registered only if .vmx is present.
+ registerVm(vmNameOnVcenter, dsRootVolumeIsOn, vmwareResource);
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ if (vmMo != null) {
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Found registered vm " + vmInternalCSName + " at host " + hyperHost.getHyperHostName());
+ }
+ }
+ tearDownVm(vmMo);
+ } else if (installAsIs) {
+// first get all the MORs ManagedObjectReference morPool = hyperHost.getHyperHostOwnerResourcePool();
+// get the base VM vmMo = hyperHost.findVmOnHyperHost(vm.template.getPath());
+// do templateVm.createLinkedClone(vmInternalCSName, morBaseSnapshot, dcMo.getVmFolder(), morPool, morDatastore)
+// or hyperHost....createLinkedOrFullClone(templateVm, volume, dcMo, vmMo, morDatastore, dsMo, vmInternalCSName, morPool);
+ vmMo = hyperHost.findVmOnHyperHost(vmInternalCSName);
+ // At this point vmMo points to the cloned VM
+ } else {
+ if (!hyperHost
+ .createBlankVm(vmNameOnVcenter, vmInternalCSName, vmSpec.getCpus(), vmSpec.getMaxSpeed(), vmwareResource.getReservedCpuMHZ(vmSpec), vmSpec.getLimitCpuUse(), (int)(vmSpec.getMaxRam() / Resource.ResourceType.bytesToMiB), vmwareResource.getReservedMemoryMb(vmSpec), guestOsId,
+ rootDiskDataStoreDetails.first(), false, controllerInfo, systemVm)) {
+ throw new Exception("Failed to create VM. vmName: " + vmInternalCSName);
+ }
+ }
+ }
+
+ /**
+ * If a VM with the same name is found in a different cluster in the DC, unregister the old VM and configure a new VM (cold-migration).
+ */
+ private VirtualMachineData unregisterOnOtherClusterButHoldOnToOldVmData(String vmInternalCSName, DatacenterMO dcMo) throws Exception {
+ VirtualMachineMO existingVmInDc = dcMo.findVm(vmInternalCSName);
+ VirtualMachineData existingVm = null;
+ if (existingVmInDc != null) {
+ existingVm = new VirtualMachineData();
+ existingVm.vmName = existingVmInDc.getName();
+ existingVm.vmFileInfo = existingVmInDc.getFileInfo();
+ existingVm.vmFileLayout = existingVmInDc.getFileLayout();
+ existingVm.datastores = existingVmInDc.getAllDatastores();
+ LOGGER.info("Found VM: " + vmInternalCSName + " on a host in a different cluster. Unregistering the exisitng VM.");
+ existingVmInDc.unregisterVm();
+ }
+ return existingVm;
+ }
+
+ private Pair<ManagedObjectReference, DatastoreMO> getRootDiskDataStoreDetails(DiskTO[] disks, HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails) {
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = null;
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ROOT) {
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (managed) {
+ String datastoreName = VmwareResource.getDatastoreName(details.get(DiskTO.IQN));
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(datastoreName);
+ } else {
+ DataStoreTO primaryStore = vol.getData().getDataStore();
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(primaryStore.getUuid());
+ }
+ }
+ }
+ return rootDiskDataStoreDetails;
+ }
+
+ /**
+ * Since VM start failed, if there was an existing VM in a different cluster that was unregistered, register it back.
+ *
+ * @param dcMo is guaranteed to be not null since we have noticed there is an existing VM in the dc (using that mo)
+ */
+ private void reRegisterExistingVm(VirtualMachineData existingVm, DatacenterMO dcMo) {
+ if (existingVm != null && existingVm.vmName != null && existingVm.vmFileInfo != null) {
+ LOGGER.debug("Since VM start failed, registering back an existing VM: " + existingVm.vmName + " that was unregistered");
+ try {
+ DatastoreFile fileInDatastore = new DatastoreFile(existingVm.vmFileInfo.getVmPathName());
+ DatastoreMO existingVmDsMo = new DatastoreMO(dcMo.getContext(), dcMo.findDatastore(fileInDatastore.getDatastoreName()));
+ registerVm(existingVm.vmName, existingVmDsMo, vmwareResource);
+ } catch (Exception ex) {
+ String message = "Failed to register an existing VM: " + existingVm.vmName + " due to " + VmwareHelper.getExceptionMessage(ex);
+ LOGGER.warn(message, ex);
+ }
+ }
+ }
+
+ private void checkIfVmExistsInVcenter(String vmInternalCSName, String vmNameOnVcenter, DatacenterMO dcMo) throws VmAlreadyExistsInVcenter, Exception {
+ // Validate VM name is unique in Datacenter
+ VirtualMachineMO vmInVcenter = dcMo.checkIfVmAlreadyExistsInVcenter(vmNameOnVcenter, vmInternalCSName);
+ if (vmInVcenter != null) {
+ String msg = "VM with name: " + vmNameOnVcenter + " already exists in vCenter.";
+ LOGGER.error(msg);
+ throw new VmAlreadyExistsInVcenter(msg);
+ }
+ }
+
+ private Pair<String, String> getDiskControllerInfo(VirtualMachineTO vmSpec) {
+ String dataDiskController = vmSpec.getDetails().get(VmDetailConstants.DATA_DISK_CONTROLLER);
+ String rootDiskController = vmSpec.getDetails().get(VmDetailConstants.ROOT_DISK_CONTROLLER);
+ // If root disk controller is scsi, then data disk controller would also be scsi instead of using 'osdefault'
+ // This helps avoid mix of different scsi subtype controllers in instance.
+ if (DiskControllerType.osdefault == DiskControllerType.getType(dataDiskController) && DiskControllerType.lsilogic == DiskControllerType.getType(rootDiskController)) {
+ dataDiskController = DiskControllerType.scsi.toString();
+ }
+
+ // Validate the controller types
+ dataDiskController = DiskControllerType.getType(dataDiskController).toString();
+ rootDiskController = DiskControllerType.getType(rootDiskController).toString();
+
+ if (DiskControllerType.getType(rootDiskController) == DiskControllerType.none) {
+ throw new CloudRuntimeException("Invalid root disk controller detected : " + rootDiskController);
+ }
+ if (DiskControllerType.getType(dataDiskController) == DiskControllerType.none) {
+ throw new CloudRuntimeException("Invalid data disk controller detected : " + dataDiskController);
+ }
+
+ return new Pair<>(rootDiskController, dataDiskController);
+ }
+
+ /**
+ * Registers the vm to the inventory given the vmx file.
+ * @param vmName
+ * @param dsMo
+ * @param vmwareResource
+ */
+ private void registerVm(String vmName, DatastoreMO dsMo, VmwareResource vmwareResource) throws Exception {
+
+ //1st param
+ VmwareHypervisorHost hyperHost = vmwareResource.getHyperHost(vmwareResource.getServiceContext());
+ ManagedObjectReference dcMor = hyperHost.getHyperHostDatacenter();
+ DatacenterMO dataCenterMo = new DatacenterMO(vmwareResource.getServiceContext(), dcMor);
+ ManagedObjectReference vmFolderMor = dataCenterMo.getVmFolder();
+
+ //2nd param
+ String vmxFilePath = dsMo.searchFileInSubFolders(vmName + ".vmx", false, VmwareManager.s_vmwareSearchExcludeFolder.value());
+
+ // 5th param
+ ManagedObjectReference morPool = hyperHost.getHyperHostOwnerResourcePool();
+
+ ManagedObjectReference morTask = vmwareResource.getServiceContext().getService().registerVMTask(vmFolderMor, vmxFilePath, vmName, false, morPool, hyperHost.getMor());
+ boolean result = vmwareResource.getServiceContext().getVimClient().waitForTask(morTask);
+ if (!result) {
+ throw new Exception("Unable to register vm due to " + TaskMO.getTaskFailureInfo(vmwareResource.getServiceContext(), morTask));
+ } else {
+ vmwareResource.getServiceContext().waitForTaskProgressDone(morTask);
+ }
+
+ }
+
+ // Pair<internal CS name, vCenter display name>
+ private Pair<String, String> composeVmNames(VirtualMachineTO vmSpec) {
+ String vmInternalCSName = vmSpec.getName();
+ String vmNameOnVcenter = vmSpec.getName();
+ if (VmwareResource.instanceNameFlag && vmSpec.getHostName() != null) {
+ vmNameOnVcenter = vmSpec.getHostName();
+ }
+ return new Pair<String, String>(vmInternalCSName, vmNameOnVcenter);
+ }
+
+ private VirtualMachineGuestOsIdentifier translateGuestOsIdentifier(String cpuArchitecture, String guestOs, String cloudGuestOs) {
+ if (cpuArchitecture == null) {
+ LOGGER.warn("CPU arch is not set, default to i386. guest os: " + guestOs);
+ cpuArchitecture = "i386";
+ }
+
+ if (cloudGuestOs == null) {
+ LOGGER.warn("Guest OS mapping name is not set for guest os: " + guestOs);
+ }
+
+ VirtualMachineGuestOsIdentifier identifier = null;
+ try {
+ if (cloudGuestOs != null) {
+ identifier = VirtualMachineGuestOsIdentifier.fromValue(cloudGuestOs);
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Using mapping name : " + identifier.toString());
+ }
+ }
+ } catch (IllegalArgumentException e) {
+ LOGGER.warn("Unable to find Guest OS Identifier in VMware for mapping name: " + cloudGuestOs + ". Continuing with defaults.");
+ }
+ if (identifier != null) {
+ return identifier;
+ }
+
+ if (cpuArchitecture.equalsIgnoreCase("x86_64")) {
+ return VirtualMachineGuestOsIdentifier.OTHER_GUEST_64;
+ }
+ return VirtualMachineGuestOsIdentifier.OTHER_GUEST;
+ }
+
+ private DiskTO[] validateDisks(DiskTO[] disks) {
+ List<DiskTO> validatedDisks = new ArrayList<DiskTO>();
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() != Volume.Type.ISO) {
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ if (primaryStore.getUuid() != null && !primaryStore.getUuid().isEmpty()) {
+ validatedDisks.add(vol);
+ }
+ } else if (vol.getType() == Volume.Type.ISO) {
+ TemplateObjectTO templateTO = (TemplateObjectTO) vol.getData();
+ if (templateTO.getPath() != null && !templateTO.getPath().isEmpty()) {
+ validatedDisks.add(vol);
+ }
+ } else {
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Drop invalid disk option, volumeTO: " + vmwareResource.getGson().toJson(vol));
+ }
+ }
+ }
+ Collections.sort(validatedDisks, (d1, d2) -> d1.getDiskSeq().compareTo(d2.getDiskSeq()));
+ return validatedDisks.toArray(new DiskTO[0]);
+ }
+
+ private HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> inferDatastoreDetailsFromDiskInfo(VmwareHypervisorHost hyperHost, VmwareContext context,
+ DiskTO[] disks, Command cmd) throws Exception {
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> mapIdToMors = new HashMap<>();
+
+ assert (hyperHost != null) && (context != null);
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() != Volume.Type.ISO) {
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ String poolUuid = primaryStore.getUuid();
+
+ if (mapIdToMors.get(poolUuid) == null) {
+ boolean isManaged = false;
+ Map<String, String> details = vol.getDetails();
+
+ if (details != null) {
+ isManaged = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (isManaged) {
+ String iScsiName = details.get(DiskTO.IQN); // details should not be null for managed storage (it may or may not be null for non-managed storage)
+ String datastoreName = VmwareResource.getDatastoreName(iScsiName);
+ ManagedObjectReference morDatastore = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, datastoreName);
+
+ // if the datastore is not present, we need to discover the iSCSI device that will support it,
+ // create the datastore, and create a VMDK file in the datastore
+ if (morDatastore == null) {
+ final String vmdkPath = vmwareResource.getVmdkPath(volumeTO.getPath());
+
+ morDatastore = getStorageProcessor().prepareManagedStorage(context, hyperHost, null, iScsiName,
+ details.get(DiskTO.STORAGE_HOST), Integer.parseInt(details.get(DiskTO.STORAGE_PORT)),
+ vmdkPath,
+ details.get(DiskTO.CHAP_INITIATOR_USERNAME), details.get(DiskTO.CHAP_INITIATOR_SECRET),
+ details.get(DiskTO.CHAP_TARGET_USERNAME), details.get(DiskTO.CHAP_TARGET_SECRET),
+ Long.parseLong(details.get(DiskTO.VOLUME_SIZE)), cmd);
+
+ DatastoreMO dsMo = new DatastoreMO(vmwareResource.getServiceContext(), morDatastore);
+
+ final String datastoreVolumePath;
+
+ if (vmdkPath != null) {
+ datastoreVolumePath = dsMo.getDatastorePath(vmdkPath + VmwareResource.VMDK_EXTENSION);
+ } else {
+ datastoreVolumePath = dsMo.getDatastorePath(dsMo.getName() + VmwareResource.VMDK_EXTENSION);
+ }
+
+ volumeTO.setPath(datastoreVolumePath);
+ vol.setPath(datastoreVolumePath);
+ }
+
+ mapIdToMors.put(datastoreName, new Pair<>(morDatastore, new DatastoreMO(context, morDatastore)));
+ } else {
+ ManagedObjectReference morDatastore = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, poolUuid);
+
+ if (morDatastore == null) {
+ String msg = "Failed to get the mounted datastore for the volume's pool " + poolUuid;
+
+ LOGGER.error(msg);
+
+ throw new Exception(msg);
+ }
+
+ mapIdToMors.put(poolUuid, new Pair<>(morDatastore, new DatastoreMO(context, morDatastore)));
+ }
+ }
+ }
+ }
+
+ return mapIdToMors;
+ }
+
+ private VmwareStorageProcessor getStorageProcessor() {
+ return vmwareResource.getStorageProcessor();
+ }
+
+ private DatastoreMO getDatastoreThatRootDiskIsOn(HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails, DiskTO disks[]) {
+ Pair<ManagedObjectReference, DatastoreMO> rootDiskDataStoreDetails = null;
+
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ROOT) {
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ if (managed) {
+ String datastoreName = VmwareResource.getDatastoreName(details.get(DiskTO.IQN));
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(datastoreName);
+
+ break;
+ } else {
+ DataStoreTO primaryStore = vol.getData().getDataStore();
+
+ rootDiskDataStoreDetails = dataStoresDetails.get(primaryStore.getUuid());
+
+ break;
+ }
+ }
+ }
+
+ if (rootDiskDataStoreDetails != null) {
+ return rootDiskDataStoreDetails.second();
+ }
+
+ return null;
+ }
+
+ private void ensureDiskControllers(VirtualMachineMO vmMo, Pair<String, String> controllerInfo) throws Exception {
+ if (vmMo == null) {
+ return;
+ }
+
+ String msg;
+ String rootDiskController = controllerInfo.first();
+ String dataDiskController = controllerInfo.second();
+ String scsiDiskController;
+ String recommendedDiskController = null;
+
+ if (VmwareHelper.isControllerOsRecommended(dataDiskController) || VmwareHelper.isControllerOsRecommended(rootDiskController)) {
+ recommendedDiskController = vmMo.getRecommendedDiskController(null);
+ }
+ scsiDiskController = HypervisorHostHelper.getScsiController(new Pair<String, String>(rootDiskController, dataDiskController), recommendedDiskController);
+ if (scsiDiskController == null) {
+ return;
+ }
+
+ vmMo.getScsiDeviceControllerKeyNoException();
+ // This VM needs SCSI controllers.
+ // Get count of existing scsi controllers. Helps not to attempt to create more than the maximum allowed 4
+ // Get maximum among the bus numbers in use by scsi controllers. Safe to pick maximum, because we always go sequential allocating bus numbers.
+ Ternary<Integer, Integer, DiskControllerType> scsiControllerInfo = vmMo.getScsiControllerInfo();
+ int requiredNumScsiControllers = VmwareHelper.MAX_SCSI_CONTROLLER_COUNT - scsiControllerInfo.first();
+ int availableBusNum = scsiControllerInfo.second() + 1; // method returned current max. bus number
+
+ if (requiredNumScsiControllers == 0) {
+ return;
+ }
+ if (scsiControllerInfo.first() > 0) {
+ // For VMs which already have a SCSI controller, do NOT attempt to add any more SCSI controllers & return the sub type.
+ // For Legacy VMs would have only 1 LsiLogic Parallel SCSI controller, and doesn't require more.
+ // For VMs created post device ordering support, 4 SCSI subtype controllers are ensured during deployment itself. No need to add more.
+ // For fresh VM deployment only, all required controllers should be ensured.
+ return;
+ }
+ ensureScsiDiskControllers(vmMo, scsiDiskController, requiredNumScsiControllers, availableBusNum);
+ }
+
+ private void ensureScsiDiskControllers(VirtualMachineMO vmMo, String scsiDiskController, int requiredNumScsiControllers, int availableBusNum) throws Exception {
+ // Pick the sub type of scsi
+ if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.pvscsi) {
+ if (!vmMo.isPvScsiSupported()) {
+ String msg = "This VM doesn't support Vmware Paravirtual SCSI controller for virtual disks, because the virtual hardware version is less than 7.";
+ throw new Exception(msg);
+ }
+ vmMo.ensurePvScsiDeviceController(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.lsisas1068) {
+ vmMo.ensureLsiLogicSasDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.buslogic) {
+ vmMo.ensureBusLogicDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ } else if (DiskControllerType.getType(scsiDiskController) == DiskControllerType.lsilogic) {
+ vmMo.ensureLsiLogicDeviceControllers(requiredNumScsiControllers, availableBusNum);
+ }
+ }
+
+ private void tearDownVm(VirtualMachineMO vmMo) throws Exception {
+
+ if (vmMo == null)
+ return;
+
+ boolean hasSnapshot = false;
+ hasSnapshot = vmMo.hasSnapshot();
+ if (!hasSnapshot)
+ vmMo.tearDownDevices(new Class<?>[]{VirtualDisk.class, VirtualEthernetCard.class});
+ else
+ vmMo.tearDownDevices(new Class<?>[]{VirtualEthernetCard.class});
+ vmMo.ensureScsiDeviceController();
+ }
+
+ private static DiskTO getIsoDiskTO(DiskTO[] disks) {
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ISO) {
+ return vol;
+ }
+ }
+ return null;
+ }
+
+ private static DiskTO[] sortVolumesByDeviceId(DiskTO[] volumes) {
+
+ List<DiskTO> listForSort = new ArrayList<DiskTO>();
+ for (DiskTO vol : volumes) {
+ listForSort.add(vol);
+ }
+ Collections.sort(listForSort, new Comparator<DiskTO>() {
+
+ @Override
+ public int compare(DiskTO arg0, DiskTO arg1) {
+ if (arg0.getDiskSeq() < arg1.getDiskSeq()) {
+ return -1;
+ } else if (arg0.getDiskSeq().equals(arg1.getDiskSeq())) {
+ return 0;
+ }
+
+ return 1;
+ }
+ });
+
+ return listForSort.toArray(new DiskTO[0]);
+ }
+
+ private void postDiskConfigBeforeStart(VirtualMachineMO vmMo, VirtualMachineTO vmSpec, DiskTO[] sortedDisks,
+ Map<String, Map<String, String>> iqnToData, VmwareHypervisorHost hyperHost, VmwareContext context, boolean installAsIs)
+ throws Exception {
+ VirtualMachineDiskInfoBuilder diskInfoBuilder = vmMo.getDiskInfoBuilder();
+
+ for (DiskTO vol : sortedDisks) {
+ //TODO: Map existing disks to the ones returned in the answer
+ if (vol.getType() == Volume.Type.ISO)
+ continue;
+
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+
+ VirtualMachineDiskInfo diskInfo = getMatchingExistingDisk(diskInfoBuilder, vol, hyperHost, context);
+ assert (diskInfo != null);
+
+ String[] diskChain = diskInfo.getDiskChain();
+ assert (diskChain.length > 0);
+
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ }
+
+ DatastoreFile file = new DatastoreFile(diskChain[0]);
+
+ if (managed) {
+ DatastoreFile originalFile = new DatastoreFile(volumeTO.getPath());
+
+ if (!file.getFileBaseName().equalsIgnoreCase(originalFile.getFileBaseName())) {
+ if (LOGGER.isInfoEnabled())
+ LOGGER.info("Detected disk-chain top file change on volume: " + volumeTO.getId() + " " + volumeTO.getPath() + " -> " + diskChain[0]);
+ }
+ } else {
+ if (!file.getFileBaseName().equalsIgnoreCase(volumeTO.getPath())) {
+ if (LOGGER.isInfoEnabled())
+ LOGGER.info("Detected disk-chain top file change on volume: " + volumeTO.getId() + " " + volumeTO.getPath() + " -> " + file.getFileBaseName());
+ }
+ }
+
+ VolumeObjectTO volInSpec = getVolumeInSpec(vmSpec, volumeTO);
+
+ if (volInSpec != null) {
+ if (managed) {
+ Map<String, String> data = new HashMap<>();
+
+ String datastoreVolumePath = diskChain[0];
+
+ data.put(StartAnswer.PATH, datastoreVolumePath);
+ data.put(StartAnswer.IMAGE_FORMAT, Storage.ImageFormat.OVA.toString());
+
+ iqnToData.put(details.get(DiskTO.IQN), data);
+
+ vol.setPath(datastoreVolumePath);
+ volumeTO.setPath(datastoreVolumePath);
+ volInSpec.setPath(datastoreVolumePath);
+ } else {
+ volInSpec.setPath(file.getFileBaseName());
+ }
+ volInSpec.setChainInfo(vmwareResource.getGson().toJson(diskInfo));
+ }
+ }
+ }
+
+ private void checkBootOptions(VirtualMachineTO vmSpec, VirtualMachineConfigSpec vmConfigSpec) {
+ String bootMode = null;
+ if (vmSpec.getDetails().containsKey(VmDetailConstants.BOOT_MODE)) {
+ bootMode = vmSpec.getDetails().get(VmDetailConstants.BOOT_MODE);
+ }
+ if (null == bootMode) {
+ bootMode = ApiConstants.BootType.BIOS.toString();
+ }
+
+ setBootOptions(vmSpec, bootMode, vmConfigSpec);
+ }
+
+ private void setBootOptions(VirtualMachineTO vmSpec, String bootMode, VirtualMachineConfigSpec vmConfigSpec) {
+ VirtualMachineBootOptions bootOptions = null;
+ if (StringUtils.isNotBlank(bootMode) && !bootMode.equalsIgnoreCase("bios")) {
+ vmConfigSpec.setFirmware("efi");
+ if (vmSpec.getDetails().containsKey(ApiConstants.BootType.UEFI.toString()) && "secure".equalsIgnoreCase(vmSpec.getDetails().get(ApiConstants.BootType.UEFI.toString()))) {
+ if (bootOptions == null) {
+ bootOptions = new VirtualMachineBootOptions();
+ }
+ bootOptions.setEfiSecureBootEnabled(true);
+ }
+ }
+ if (vmSpec.isEnterHardwareSetup()) {
+ if (bootOptions == null) {
+ bootOptions = new VirtualMachineBootOptions();
+ }
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug(String.format("configuring VM '%s' to enter hardware setup",vmSpec.getName()));
+ }
+ bootOptions.setEnterBIOSSetup(vmSpec.isEnterHardwareSetup());
+ }
+ if (bootOptions != null) {
+ vmConfigSpec.setBootOptions(bootOptions);
+ }
+ }
+
+ private void resizeRootDiskOnVMStart(VirtualMachineMO vmMo, DiskTO rootDiskTO, VmwareHypervisorHost hyperHost, VmwareContext context) throws Exception {
+ final Pair<VirtualDisk, String> vdisk = vmwareResource.getVirtualDiskInfo(vmMo, VmwareResource.appendFileType(rootDiskTO.getPath(), VmwareResource.VMDK_EXTENSION));
+ assert (vdisk != null);
+
+ Long reqSize = 0L;
+ final VolumeObjectTO volumeTO = ((VolumeObjectTO) rootDiskTO.getData());
+ if (volumeTO != null) {
+ reqSize = volumeTO.getSize() / 1024;
+ }
+ final VirtualDisk disk = vdisk.first();
+ if (reqSize > disk.getCapacityInKB()) {
+ final VirtualMachineDiskInfo diskInfo = getMatchingExistingDisk(vmMo.getDiskInfoBuilder(), rootDiskTO, hyperHost, context);
+ assert (diskInfo != null);
+ final String[] diskChain = diskInfo.getDiskChain();
+
+ if (diskChain != null && diskChain.length > 1) {
+ LOGGER.warn("Disk chain length for the VM is greater than one, this is not supported");
+ throw new CloudRuntimeException("Unsupported VM disk chain length: " + diskChain.length);
+ }
+
+ boolean resizingSupported = false;
+ String deviceBusName = diskInfo.getDiskDeviceBusName();
+ if (deviceBusName != null && (deviceBusName.toLowerCase().contains("scsi") || deviceBusName.toLowerCase().contains("lsi"))) {
+ resizingSupported = true;
+ }
+ if (!resizingSupported) {
+ LOGGER.warn("Resizing of root disk is only support for scsi device/bus, the provide VM's disk device bus name is " + diskInfo.getDiskDeviceBusName());
+ throw new CloudRuntimeException("Unsupported VM root disk device bus: " + diskInfo.getDiskDeviceBusName());
+ }
+
+ disk.setCapacityInKB(reqSize);
+ VirtualMachineConfigSpec vmConfigSpec = new VirtualMachineConfigSpec();
+ VirtualDeviceConfigSpec deviceConfigSpec = new VirtualDeviceConfigSpec();
+ deviceConfigSpec.setDevice(disk);
+ deviceConfigSpec.setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ vmConfigSpec.getDeviceChange().add(deviceConfigSpec);
+ if (!vmMo.configureVm(vmConfigSpec)) {
+ throw new Exception("Failed to configure VM for given root disk size. vmName: " + vmMo.getName());
+ }
+ }
+ }
+
+ private static void postNvpConfigBeforeStart(VirtualMachineMO vmMo, VirtualMachineTO vmSpec) throws Exception {
+ /**
+ * We need to configure the port on the DV switch after the host is
+ * connected. So make this happen between the configure and start of
+ * the VM
+ */
+ int nicIndex = 0;
+ for (NicTO nicTo : sortNicsByDeviceId(vmSpec.getNics())) {
+ if (nicTo.getBroadcastType() == Networks.BroadcastDomainType.Lswitch) {
+ // We need to create a port with a unique vlan and pass the key to the nic device
+ LOGGER.trace("Nic " + nicTo.toString() + " is connected to an NVP logicalswitch");
+ VirtualDevice nicVirtualDevice = vmMo.getNicDeviceByIndex(nicIndex);
+ if (nicVirtualDevice == null) {
+ throw new Exception("Failed to find a VirtualDevice for nic " + nicIndex); //FIXME Generic exceptions are bad
+ }
+ VirtualDeviceBackingInfo backing = nicVirtualDevice.getBacking();
+ if (backing instanceof VirtualEthernetCardDistributedVirtualPortBackingInfo) {
+ // This NIC is connected to a Distributed Virtual Switch
+ VirtualEthernetCardDistributedVirtualPortBackingInfo portInfo = (VirtualEthernetCardDistributedVirtualPortBackingInfo) backing;
+ DistributedVirtualSwitchPortConnection port = portInfo.getPort();
+ String portKey = port.getPortKey();
+ String portGroupKey = port.getPortgroupKey();
+ String dvSwitchUuid = port.getSwitchUuid();
+
+ LOGGER.debug("NIC " + nicTo.toString() + " is connected to dvSwitch " + dvSwitchUuid + " pg " + portGroupKey + " port " + portKey);
+
+ ManagedObjectReference dvSwitchManager = vmMo.getContext().getVimClient().getServiceContent().getDvSwitchManager();
+ ManagedObjectReference dvSwitch = vmMo.getContext().getVimClient().getService().queryDvsByUuid(dvSwitchManager, dvSwitchUuid);
+
+ // Get all ports
+ DistributedVirtualSwitchPortCriteria criteria = new DistributedVirtualSwitchPortCriteria();
+ criteria.setInside(true);
+ criteria.getPortgroupKey().add(portGroupKey);
+ List<DistributedVirtualPort> dvPorts = vmMo.getContext().getVimClient().getService().fetchDVPorts(dvSwitch, criteria);
+
+ DistributedVirtualPort vmDvPort = null;
+ List<Integer> usedVlans = new ArrayList<Integer>();
+ for (DistributedVirtualPort dvPort : dvPorts) {
+ // Find the port for this NIC by portkey
+ if (portKey.equals(dvPort.getKey())) {
+ vmDvPort = dvPort;
+ }
+ VMwareDVSPortSetting settings = (VMwareDVSPortSetting) dvPort.getConfig().getSetting();
+ VmwareDistributedVirtualSwitchVlanIdSpec vlanId = (VmwareDistributedVirtualSwitchVlanIdSpec) settings.getVlan();
+ LOGGER.trace("Found port " + dvPort.getKey() + " with vlan " + vlanId.getVlanId());
+ if (vlanId.getVlanId() > 0 && vlanId.getVlanId() < 4095) {
+ usedVlans.add(vlanId.getVlanId());
+ }
+ }
+
+ if (vmDvPort == null) {
+ throw new Exception("Empty port list from dvSwitch for nic " + nicTo.toString());
+ }
+
+ DVPortConfigInfo dvPortConfigInfo = vmDvPort.getConfig();
+ VMwareDVSPortSetting settings = (VMwareDVSPortSetting) dvPortConfigInfo.getSetting();
+
+ VmwareDistributedVirtualSwitchVlanIdSpec vlanId = (VmwareDistributedVirtualSwitchVlanIdSpec) settings.getVlan();
+ BoolPolicy blocked = settings.getBlocked();
+ if (blocked.isValue() == Boolean.TRUE) {
+ LOGGER.trace("Port is blocked, set a vlanid and unblock");
+ DVPortConfigSpec dvPortConfigSpec = new DVPortConfigSpec();
+ VMwareDVSPortSetting edittedSettings = new VMwareDVSPortSetting();
+ // Unblock
+ blocked.setValue(Boolean.FALSE);
+ blocked.setInherited(Boolean.FALSE);
+ edittedSettings.setBlocked(blocked);
+ // Set vlan
+ int i;
+ for (i = 1; i < 4095; i++) {
+ if (!usedVlans.contains(i))
+ break;
+ }
+ vlanId.setVlanId(i); // FIXME should be a determined
+ // based on usage
+ vlanId.setInherited(false);
+ edittedSettings.setVlan(vlanId);
+
+ dvPortConfigSpec.setSetting(edittedSettings);
+ dvPortConfigSpec.setOperation("edit");
+ dvPortConfigSpec.setKey(portKey);
+ List<DVPortConfigSpec> dvPortConfigSpecs = new ArrayList<DVPortConfigSpec>();
+ dvPortConfigSpecs.add(dvPortConfigSpec);
+ ManagedObjectReference task = vmMo.getContext().getVimClient().getService().reconfigureDVPortTask(dvSwitch, dvPortConfigSpecs);
+ if (!vmMo.getContext().getVimClient().waitForTask(task)) {
+ throw new Exception("Failed to configure the dvSwitch port for nic " + nicTo.toString());
+ }
+ LOGGER.debug("NIC " + nicTo.toString() + " connected to vlan " + i);
+ } else {
+ LOGGER.trace("Port already configured and set to vlan " + vlanId.getVlanId());
+ }
+ } else if (backing instanceof VirtualEthernetCardNetworkBackingInfo) {
+ // This NIC is connected to a Virtual Switch
+ // Nothing to do
+ } else if (backing instanceof VirtualEthernetCardOpaqueNetworkBackingInfo) {
+ //if NSX API VERSION >= 4.2, connect to br-int (nsx.network), do not create portgroup else previous behaviour
+ //OK, connected to OpaqueNetwork
+ } else {
+ LOGGER.error("nic device backing is of type " + backing.getClass().getName());
+ throw new Exception("Incompatible backing for a VirtualDevice for nic " + nicIndex); //FIXME Generic exceptions are bad
+ }
+ }
+ nicIndex++;
+ }
+ }
+
+ private VirtualMachineDiskInfo getMatchingExistingDisk(VirtualMachineDiskInfoBuilder diskInfoBuilder, DiskTO vol, VmwareHypervisorHost hyperHost, VmwareContext context)
+ throws Exception {
+ if (diskInfoBuilder != null) {
+ VolumeObjectTO volume = (VolumeObjectTO) vol.getData();
+
+ String dsName = null;
+ String diskBackingFileBaseName = null;
+
+ Map<String, String> details = vol.getDetails();
+ boolean isManaged = details != null && Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+
+ if (isManaged) {
+ String iScsiName = details.get(DiskTO.IQN);
+
+ // if the storage is managed, iScsiName should not be null
+ dsName = VmwareResource.getDatastoreName(iScsiName);
+
+ diskBackingFileBaseName = new DatastoreFile(volume.getPath()).getFileBaseName();
+ } else {
+ ManagedObjectReference morDs = HypervisorHostHelper.findDatastoreWithBackwardsCompatibility(hyperHost, volume.getDataStore().getUuid());
+ DatastoreMO dsMo = new DatastoreMO(context, morDs);
+
+ dsName = dsMo.getName();
+
+ diskBackingFileBaseName = volume.getPath();
+ }
+
+ VirtualMachineDiskInfo diskInfo = diskInfoBuilder.getDiskInfoByBackingFileBaseName(diskBackingFileBaseName, dsName);
+ if (diskInfo != null) {
+ LOGGER.info("Found existing disk info from volume path: " + volume.getPath());
+ return diskInfo;
+ } else {
+ String chainInfo = volume.getChainInfo();
+ if (chainInfo != null) {
+ VirtualMachineDiskInfo infoInChain = vmwareResource.getGson().fromJson(chainInfo, VirtualMachineDiskInfo.class);
+ if (infoInChain != null) {
+ String[] disks = infoInChain.getDiskChain();
+ if (disks.length > 0) {
+ for (String diskPath : disks) {
+ DatastoreFile file = new DatastoreFile(diskPath);
+ diskInfo = diskInfoBuilder.getDiskInfoByBackingFileBaseName(file.getFileBaseName(), dsName);
+ if (diskInfo != null) {
+ LOGGER.info("Found existing disk from chain info: " + diskPath);
+ return diskInfo;
+ }
+ }
+ }
+
+ if (diskInfo == null) {
+ diskInfo = diskInfoBuilder.getDiskInfoByDeviceBusName(infoInChain.getDiskDeviceBusName());
+ if (diskInfo != null) {
+ LOGGER.info("Found existing disk from from chain device bus information: " + infoInChain.getDiskDeviceBusName());
+ return diskInfo;
+ }
+ }
+ }
+ }
+ }
+ }
+
+ return null;
+ }
+
+ private static VolumeObjectTO getVolumeInSpec(VirtualMachineTO vmSpec, VolumeObjectTO srcVol) {
+ for (DiskTO disk : vmSpec.getDisks()) {
+ if (disk.getData() instanceof VolumeObjectTO) {
+ VolumeObjectTO vol = (VolumeObjectTO) disk.getData();
+ if (vol.getId() == srcVol.getId())
+ return vol;
+ }
+ }
+
+ return null;
+ }
+
+ /**
+ * Generate the mac sequence from the nics.
+ */
+ protected String generateMacSequence(NicTO[] nics) {
+ if (nics.length == 0) {
+ return "";
+ }
+
+ StringBuffer sbMacSequence = new StringBuffer();
+ for (NicTO nicTo : sortNicsByDeviceId(nics)) {
+ sbMacSequence.append(nicTo.getMac()).append("|");
+ }
+ if (!sbMacSequence.toString().isEmpty()) {
+ sbMacSequence.deleteCharAt(sbMacSequence.length() - 1); //Remove extra '|' char appended at the end
+ }
+
+ return sbMacSequence.toString();
+ }
+
+ static NicTO[] sortNicsByDeviceId(NicTO[] nics) {
+
+ List<NicTO> listForSort = new ArrayList<NicTO>();
+ for (NicTO nic : nics) {
+ listForSort.add(nic);
+ }
+ Collections.sort(listForSort, new Comparator<NicTO>() {
+
+ @Override
+ public int compare(NicTO arg0, NicTO arg1) {
+ if (arg0.getDeviceId() < arg1.getDeviceId()) {
+ return -1;
+ } else if (arg0.getDeviceId() == arg1.getDeviceId()) {
+ return 0;
+ }
+
+ return 1;
+ }
+ });
+
+ return listForSort.toArray(new NicTO[0]);
+ }
+
+ private static void configBasicExtraOption(List<OptionValue> extraOptions, VirtualMachineTO vmSpec) {
+ OptionValue newVal = new OptionValue();
+ newVal.setKey("machine.id");
+ newVal.setValue(vmSpec.getBootArgs());
+ extraOptions.add(newVal);
+
+ newVal = new OptionValue();
+ newVal.setKey("devices.hotplug");
+ newVal.setValue("true");
+ extraOptions.add(newVal);
+ }
+
+ private static void configNvpExtraOption(List<OptionValue> extraOptions, VirtualMachineTO vmSpec, Map<String, String> nicUuidToDvSwitchUuid) {
+ /**
+ * Extra Config : nvp.vm-uuid = uuid
+ * - Required for Nicira NVP integration
+ */
+ OptionValue newVal = new OptionValue();
+ newVal.setKey("nvp.vm-uuid");
+ newVal.setValue(vmSpec.getUuid());
+ extraOptions.add(newVal);
+
+ /**
+ * Extra Config : nvp.iface-id.<num> = uuid
+ * - Required for Nicira NVP integration
+ */
+ int nicNum = 0;
+ for (NicTO nicTo : sortNicsByDeviceId(vmSpec.getNics())) {
+ if (nicTo.getUuid() != null) {
+ newVal = new OptionValue();
+ newVal.setKey("nvp.iface-id." + nicNum);
+ newVal.setValue(nicTo.getUuid());
+ extraOptions.add(newVal);
+ }
+ nicNum++;
+ }
+ }
+
+ private static void configCustomExtraOption(List<OptionValue> extraOptions, VirtualMachineTO vmSpec) {
+ // we no longer to validation anymore
+ for (Map.Entry<String, String> entry : vmSpec.getDetails().entrySet()) {
+ if (entry.getKey().equalsIgnoreCase(VmDetailConstants.BOOT_MODE)) {
+ continue;
+ }
+ OptionValue newVal = new OptionValue();
+ newVal.setKey(entry.getKey());
+ newVal.setValue(entry.getValue());
+ extraOptions.add(newVal);
+ }
+ }
+
+ private class VmAlreadyExistsInVcenter extends Exception {
+ public VmAlreadyExistsInVcenter(String msg) {
+ }
+ }
+
+ private class VirtualMachineData {
+ String vmName = null;
+ VirtualMachineFileInfo vmFileInfo = null;
+ VirtualMachineFileLayoutEx vmFileLayout = null;
+ List<DatastoreMO> datastores = new ArrayList<>();
+ }
+
+ private class VirtualMachineRecycler {
+ private String vmInternalCSName;
+ private Pair<String, String> controllerInfo;
+ private Boolean systemVm;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private DiskControllerType systemVmScsiControllerType;
+ private int firstScsiControllerBusNum;
+ private int numScsiControllerForSystemVm;
+ private VirtualMachineDiskInfoBuilder diskInfoBuilder;
+ private boolean hasSnapshot;
+ private VirtualDevice[] nicDevices;
+
+ public VirtualMachineRecycler(String vmInternalCSName, Pair<String, String> controllerInfo, Boolean systemVm, VmwareHypervisorHost hyperHost, VirtualMachineMO vmMo,
+ DiskControllerType systemVmScsiControllerType, int firstScsiControllerBusNum, int numScsiControllerForSystemVm) {
+ this.vmInternalCSName = vmInternalCSName;
+ this.controllerInfo = controllerInfo;
+ this.systemVm = systemVm;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.systemVmScsiControllerType = systemVmScsiControllerType;
+ this.firstScsiControllerBusNum = firstScsiControllerBusNum;
+ this.numScsiControllerForSystemVm = numScsiControllerForSystemVm;
+ }
+
+ public VirtualMachineDiskInfoBuilder getDiskInfoBuilder() {
+ return diskInfoBuilder;
+ }
+
+ public boolean isHasSnapshot() {
+ return hasSnapshot;
+ }
+
+ public VirtualMachineRecycler invoke() throws Exception {
+ if (LOGGER.isInfoEnabled()) {
+ LOGGER.info("Found vm " + vmInternalCSName + " at other host, relocate to " + hyperHost.getHyperHostName());
+ }
+
+ takeVmFromOtherHyperHost(hyperHost, vmInternalCSName);
+
+ if (VmwareResource.getVmPowerState(vmMo) != VirtualMachine.PowerState.PowerOff)
+ vmMo.safePowerOff(vmwareResource.getShutdownWaitMs());
+
+ diskInfoBuilder = vmMo.getDiskInfoBuilder();
+ hasSnapshot = vmMo.hasSnapshot();
+ nicDevices = vmMo.getNicDevices();
+ if (!hasSnapshot)
+ vmMo.tearDownDevices(new Class<?>[] {VirtualDisk.class, VirtualEthernetCard.class});
+ else
+ vmMo.tearDownDevices(new Class<?>[] {VirtualEthernetCard.class});
+
+ if (systemVm) {
+ // System volumes doesn't require more than 1 SCSI controller as there is no requirement for data volumes.
+ ensureScsiDiskControllers(vmMo, systemVmScsiControllerType.toString(), numScsiControllerForSystemVm, firstScsiControllerBusNum);
+ } else {
+ ensureDiskControllers(vmMo, controllerInfo);
+ }
+ return this;
+ }
+
+ private VirtualMachineMO takeVmFromOtherHyperHost(VmwareHypervisorHost hyperHost, String vmName) throws Exception {
+
+ VirtualMachineMO vmMo = hyperHost.findVmOnPeerHyperHost(vmName);
+ if (vmMo != null) {
+ ManagedObjectReference morTargetPhysicalHost = hyperHost.findMigrationTarget(vmMo);
+ if (morTargetPhysicalHost == null) {
+ String msg = "VM " + vmName + " is on other host and we have no resource available to migrate and start it here";
+ LOGGER.error(msg);
+ throw new Exception(msg);
+ }
+
+ if (!vmMo.relocate(morTargetPhysicalHost)) {
+ String msg = "VM " + vmName + " is on other host and we failed to relocate it here";
+ LOGGER.error(msg);
+ throw new Exception(msg);
+ }
+
+ return vmMo;
+ }
+ return null;
+ }
+
+ public VirtualDevice[] getNicDevices() {
+ return nicDevices;
+ }
+ }
+
+ private class PrepareSytemVMPatchISOMethod {
+ private VmwareManager mgr;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int i;
+ private int ideUnitNumber;
+
+ public PrepareSytemVMPatchISOMethod(VmwareManager mgr, VmwareHypervisorHost hyperHost, VirtualMachineMO vmMo, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int i, int ideUnitNumber) {
+ this.mgr = mgr;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.i = i;
+ this.ideUnitNumber = ideUnitNumber;
+ }
+
+ public int getI() {
+ return i;
+ }
+
+ public int getIdeUnitNumber() {
+ return ideUnitNumber;
+ }
+
+ public PrepareSytemVMPatchISOMethod invoke() throws Exception {
+ // attach ISO (for patching of system VM)
+ DatastoreMO secDsMo = getDatastoreMOForSecStore(mgr, hyperHost);
+
+ deviceConfigSpecArray[i] = new VirtualDeviceConfigSpec();
+ Pair<VirtualDevice, Boolean> isoInfo = VmwareHelper
+ .prepareIsoDevice(vmMo, String.format("[%s] systemvm/%s", secDsMo.getName(), mgr.getSystemVMIsoFileNameOnDatastore()), secDsMo.getMor(), true, true,
+ ideUnitNumber++, i + 1);
+ deviceConfigSpecArray[i].setDevice(isoInfo.first());
+ if (isoInfo.second()) {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at new device " + vmwareResource.getGson().toJson(isoInfo.first()));
+ deviceConfigSpecArray[i].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+ } else {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+ deviceConfigSpecArray[i].setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ }
+ i++;
+ return this;
+ }
+
+ }
+ private DatastoreMO getDatastoreMOForSecStore(VmwareManager mgr, VmwareHypervisorHost hyperHost) throws Exception {
+ Pair<String, Long> secStoreUrlAndId = mgr.getSecondaryStorageStoreUrlAndId(Long.parseLong(vmwareResource.getDcId()));
+ String secStoreUrl = secStoreUrlAndId.first();
+ Long secStoreId = secStoreUrlAndId.second();
+ if (secStoreUrl == null) {
+ String msg = "secondary storage for dc " + vmwareResource.getDcId() + " is not ready yet?";
+ throw new Exception(msg);
+ }
+ mgr.prepareSecondaryStorageStore(secStoreUrl, secStoreId);
+
+ ManagedObjectReference morSecDs = vmwareResource.prepareSecondaryDatastoreOnHost(secStoreUrl);
+ if (morSecDs == null) {
+ String msg = "Failed to prepare secondary storage on host, secondary store url: " + secStoreUrl;
+ throw new Exception(msg);
+ }
+ return new DatastoreMO(hyperHost.getContext(), morSecDs);
+ }
+
+ /**
+ // Setup ROOT/DATA disk devices
+ */
+ private class DiskSetup {
+ private VirtualMachineTO vmSpec;
+ private DiskTO rootDiskTO;
+ private Pair<String, String> controllerInfo;
+ private VmwareContext context;
+ private DatacenterMO dcMo;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private DiskTO[] disks;
+ private HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails;
+ private VirtualMachineDiskInfoBuilder diskInfoBuilder;
+ private boolean hasSnapshot;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int deviceCount;
+ private int ideUnitNumber;
+ private int scsiUnitNumber;
+ private int ideControllerKey;
+ private int scsiControllerKey;
+ private DiskTO[] sortedDisks;
+ private boolean installAsIs;
+
+ public DiskSetup(VirtualMachineTO vmSpec, DiskTO rootDiskTO, Pair<String, String> controllerInfo, VmwareContext context, DatacenterMO dcMo, VmwareHypervisorHost hyperHost,
+ VirtualMachineMO vmMo, DiskTO[] disks, HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails, VirtualMachineDiskInfoBuilder diskInfoBuilder,
+ boolean hasSnapshot, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount, int ideUnitNumber, int scsiUnitNumber, int ideControllerKey, int scsiControllerKey, boolean installAsIs) {
+ this.vmSpec = vmSpec;
+ this.rootDiskTO = rootDiskTO;
+ this.controllerInfo = controllerInfo;
+ this.context = context;
+ this.dcMo = dcMo;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.disks = disks;
+ this.dataStoresDetails = dataStoresDetails;
+ this.diskInfoBuilder = diskInfoBuilder;
+ this.hasSnapshot = hasSnapshot;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.deviceCount = deviceCount;
+ this.ideUnitNumber = ideUnitNumber;
+ this.scsiUnitNumber = scsiUnitNumber;
+ this.ideControllerKey = ideControllerKey;
+ this.scsiControllerKey = scsiControllerKey;
+ this.installAsIs = installAsIs;
+ }
+
+ public DiskTO getRootDiskTO() {
+ return rootDiskTO;
+ }
+
+ public int getDeviceCount() {
+ return deviceCount;
+ }
+
+ public DiskTO[] getSortedDisks() {
+ return sortedDisks;
+ }
+
+ public DiskSetup invoke() throws Exception {
+ int controllerKey;
+ sortedDisks = sortVolumesByDeviceId(disks);
+ for (DiskTO vol : sortedDisks) {
+ if (vol.getType() == Volume.Type.ISO || installAsIs) {
+ continue;
+ }
+
+ VirtualMachineDiskInfo matchingExistingDisk = getMatchingExistingDisk(diskInfoBuilder, vol, hyperHost, context);
+ controllerKey = getDiskController(matchingExistingDisk, vol, vmSpec, ideControllerKey, scsiControllerKey);
+ String diskController = getDiskController(vmMo, matchingExistingDisk, vol, controllerInfo);
+
+ if (DiskControllerType.getType(diskController) == DiskControllerType.osdefault) {
+ diskController = vmMo.getRecommendedDiskController(null);
+ }
+ if (DiskControllerType.getType(diskController) == DiskControllerType.ide) {
+ controllerKey = vmMo.getIDEControllerKey(ideUnitNumber);
+ if (vol.getType() == Volume.Type.DATADISK) {
+ // Could be result of flip due to user configured setting or "osdefault" for data disks
+ // Ensure maximum of 2 data volumes over IDE controller, 3 includeing root volume
+ if (vmMo.getNumberOfVirtualDisks() > 3) {
+ throw new CloudRuntimeException(
+ "Found more than 3 virtual disks attached to this VM [" + vmMo.getVmName() + "]. Unable to implement the disks over " + diskController + " controller, as maximum number of devices supported over IDE controller is 4 includeing CDROM device.");
+ }
+ }
+ } else {
+ if (VmwareHelper.isReservedScsiDeviceNumber(scsiUnitNumber)) {
+ scsiUnitNumber++;
+ }
+
+ controllerKey = vmMo.getScsiDiskControllerKeyNoException(diskController, scsiUnitNumber);
+ if (controllerKey == -1) {
+ // This may happen for ROOT legacy VMs which doesn't have recommended disk controller when global configuration parameter 'vmware.root.disk.controller' is set to "osdefault"
+ // Retrieve existing controller and use.
+ Ternary<Integer, Integer, DiskControllerType> vmScsiControllerInfo = vmMo.getScsiControllerInfo();
+ DiskControllerType existingControllerType = vmScsiControllerInfo.third();
+ controllerKey = vmMo.getScsiDiskControllerKeyNoException(existingControllerType.toString(), scsiUnitNumber);
+ }
+ }
+ if (!hasSnapshot) {
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+
+ VolumeObjectTO volumeTO = (VolumeObjectTO)vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ Map<String, String> details = vol.getDetails();
+ boolean managed = false;
+ String iScsiName = null;
+
+ if (details != null) {
+ managed = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ iScsiName = details.get(DiskTO.IQN);
+ }
+
+ // if the storage is managed, iScsiName should not be null
+ String datastoreName = managed ? VmwareResource.getDatastoreName(iScsiName) : primaryStore.getUuid();
+ Pair<ManagedObjectReference, DatastoreMO> volumeDsDetails = dataStoresDetails.get(datastoreName);
+
+ assert (volumeDsDetails != null);
+
+ String[] diskChain = syncDiskChain(dcMo, vmMo, vmSpec, vol, matchingExistingDisk, dataStoresDetails);
+
+ int deviceNumber = -1;
+ if (controllerKey == vmMo.getIDEControllerKey(ideUnitNumber)) {
+ deviceNumber = ideUnitNumber % VmwareHelper.MAX_ALLOWED_DEVICES_IDE_CONTROLLER;
+ ideUnitNumber++;
+ } else {
+ deviceNumber = scsiUnitNumber % VmwareHelper.MAX_ALLOWED_DEVICES_SCSI_CONTROLLER;
+ scsiUnitNumber++;
+ }
+ VirtualDevice device = VmwareHelper.prepareDiskDevice(vmMo, null, controllerKey, diskChain, volumeDsDetails.first(), deviceNumber, deviceCount + 1);
+ if (vol.getType() == Volume.Type.ROOT)
+ rootDiskTO = vol;
+ deviceConfigSpecArray[deviceCount].setDevice(device);
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare volume at new device " + vmwareResource.getGson().toJson(device));
+
+ deviceCount++;
+ } else {
+ if (controllerKey == vmMo.getIDEControllerKey(ideUnitNumber))
+ ideUnitNumber++;
+ else
+ scsiUnitNumber++;
+ }
+ }
+ return this;
+ }
+
+ private int getDiskController(VirtualMachineDiskInfo matchingExistingDisk, DiskTO vol, VirtualMachineTO vmSpec, int ideControllerKey, int scsiControllerKey) {
+
+ int controllerKey;
+ if (matchingExistingDisk != null) {
+ LOGGER.info("Chose disk controller based on existing information: " + matchingExistingDisk.getDiskDeviceBusName());
+ if (matchingExistingDisk.getDiskDeviceBusName().startsWith("ide"))
+ return ideControllerKey;
+ else
+ return scsiControllerKey;
+ }
+
+ if (vol.getType() == Volume.Type.ROOT) {
+ Map<String, String> vmDetails = vmSpec.getDetails();
+ if (vmDetails != null && vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER) != null) {
+ if (vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER).equalsIgnoreCase("scsi")) {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> scsi, based on root disk controller settings: "
+ + vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER));
+ controllerKey = scsiControllerKey;
+ } else {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> ide, based on root disk controller settings: "
+ + vmDetails.get(VmDetailConstants.ROOT_DISK_CONTROLLER));
+ controllerKey = ideControllerKey;
+ }
+ } else {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> scsi. due to null root disk controller setting");
+ controllerKey = scsiControllerKey;
+ }
+
+ } else {
+ // DATA volume always use SCSI device
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> scsi");
+ controllerKey = scsiControllerKey;
+ }
+
+ return controllerKey;
+ }
+
+ private String getDiskController(VirtualMachineMO vmMo, VirtualMachineDiskInfo matchingExistingDisk, DiskTO vol, Pair<String, String> controllerInfo) throws Exception {
+ int controllerKey;
+ DiskControllerType controllerType = DiskControllerType.none;
+ if (matchingExistingDisk != null) {
+ String currentBusName = matchingExistingDisk.getDiskDeviceBusName();
+ if (currentBusName != null) {
+ LOGGER.info("Chose disk controller based on existing information: " + currentBusName);
+ if (currentBusName.startsWith("ide")) {
+ controllerType = DiskControllerType.ide;
+ } else if (currentBusName.startsWith("scsi")) {
+ controllerType = DiskControllerType.scsi;
+ }
+ }
+ if (controllerType == DiskControllerType.scsi || controllerType == DiskControllerType.none) {
+ Ternary<Integer, Integer, DiskControllerType> vmScsiControllerInfo = vmMo.getScsiControllerInfo();
+ controllerType = vmScsiControllerInfo.third();
+ }
+ return controllerType.toString();
+ }
+
+ if (vol.getType() == Volume.Type.ROOT) {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> " + controllerInfo.first()
+ + ", based on root disk controller settings at global configuration setting.");
+ return controllerInfo.first();
+ } else {
+ LOGGER.info("Chose disk controller for vol " + vol.getType() + " -> " + controllerInfo.second()
+ + ", based on default data disk controller setting i.e. Operating system recommended."); // Need to bring in global configuration setting & template level setting.
+ return controllerInfo.second();
+ }
+ }
+
+ // return the finalized disk chain for startup, from top to bottom
+ private String[] syncDiskChain(DatacenterMO dcMo, VirtualMachineMO vmMo, VirtualMachineTO vmSpec, DiskTO vol, VirtualMachineDiskInfo diskInfo,
+ HashMap<String, Pair<ManagedObjectReference, DatastoreMO>> dataStoresDetails) throws Exception {
+
+ VolumeObjectTO volumeTO = (VolumeObjectTO) vol.getData();
+ DataStoreTO primaryStore = volumeTO.getDataStore();
+ Map<String, String> details = vol.getDetails();
+ boolean isManaged = false;
+ String iScsiName = null;
+
+ if (details != null) {
+ isManaged = Boolean.parseBoolean(details.get(DiskTO.MANAGED));
+ iScsiName = details.get(DiskTO.IQN);
+ }
+
+ // if the storage is managed, iScsiName should not be null
+ String datastoreName = isManaged ? VmwareResource.getDatastoreName(iScsiName) : primaryStore.getUuid();
+ Pair<ManagedObjectReference, DatastoreMO> volumeDsDetails = dataStoresDetails.get(datastoreName);
+
+ if (volumeDsDetails == null) {
+ throw new Exception("Primary datastore " + primaryStore.getUuid() + " is not mounted on host.");
+ }
+
+ DatastoreMO dsMo = volumeDsDetails.second();
+
+ // we will honor vCenter's meta if it exists
+ if (diskInfo != null) {
+ // to deal with run-time upgrade to maintain the new datastore folder structure
+ String disks[] = diskInfo.getDiskChain();
+ for (int i = 0; i < disks.length; i++) {
+ DatastoreFile file = new DatastoreFile(disks[i]);
+ if (!isManaged && file.getDir() != null && file.getDir().isEmpty()) {
+ LOGGER.info("Perform run-time datastore folder upgrade. sync " + disks[i] + " to VM folder");
+ disks[i] = VmwareStorageLayoutHelper.syncVolumeToVmDefaultFolder(dcMo, vmMo.getName(), dsMo, file.getFileBaseName(), VmwareManager.s_vmwareSearchExcludeFolder.value());
+ }
+ }
+ return disks;
+ }
+
+ final String datastoreDiskPath;
+
+ if (isManaged) {
+ String vmdkPath = new DatastoreFile(volumeTO.getPath()).getFileBaseName();
+
+ if (volumeTO.getVolumeType() == Volume.Type.ROOT) {
+ if (vmdkPath == null) {
+ vmdkPath = volumeTO.getName();
+ }
+
+ datastoreDiskPath = VmwareStorageLayoutHelper.syncVolumeToVmDefaultFolder(dcMo, vmMo.getName(), dsMo, vmdkPath);
+ } else {
+ if (vmdkPath == null) {
+ vmdkPath = dsMo.getName();
+ }
+
+ datastoreDiskPath = dsMo.getDatastorePath(vmdkPath + VmwareResource.VMDK_EXTENSION);
+ }
+ } else {
+ datastoreDiskPath = VmwareStorageLayoutHelper.syncVolumeToVmDefaultFolder(dcMo, vmMo.getName(), dsMo, volumeTO.getPath(), VmwareManager.s_vmwareSearchExcludeFolder.value());
+ }
+
+ if (!dsMo.fileExists(datastoreDiskPath)) {
+ LOGGER.warn("Volume " + volumeTO.getId() + " does not seem to exist on datastore, out of sync? path: " + datastoreDiskPath);
+ }
+
+ return new String[]{datastoreDiskPath};
+ }
+ }
+
+ /**
+ // Setup NIC devices
+ */
+ private class NicSetup {
+ private StartCommand cmd;
+ private VirtualMachineTO vmSpec;
+ private String vmInternalCSName;
+ private VmwareContext context;
+ private VmwareManager mgr;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private NicTO[] nics;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int deviceCount;
+ private int nicMask;
+ private int nicCount;
+ private Map<String, String> nicUuidToDvSwitchUuid;
+ private VirtualDevice[] nicDevices;
+
+ public NicSetup(StartCommand cmd, VirtualMachineTO vmSpec, String vmInternalCSName, VmwareContext context, VmwareManager mgr, VmwareHypervisorHost hyperHost,
+ VirtualMachineMO vmMo, NicTO[] nics, VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount, VirtualDevice[] nicDevices) {
+ this.cmd = cmd;
+ this.vmSpec = vmSpec;
+ this.vmInternalCSName = vmInternalCSName;
+ this.context = context;
+ this.mgr = mgr;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.nics = nics;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.deviceCount = deviceCount;
+ this.nicDevices = nicDevices;
+ }
+
+ public int getDeviceCount() {
+ return deviceCount;
+ }
+
+ public int getNicMask() {
+ return nicMask;
+ }
+
+ public int getNicCount() {
+ return nicCount;
+ }
+
+ public Map<String, String> getNicUuidToDvSwitchUuid() {
+ return nicUuidToDvSwitchUuid;
+ }
+
+ public NicSetup invoke() throws Exception {
+ VirtualDevice nic;
+ nicMask = 0;
+ nicCount = 0;
+
+ if (vmSpec.getType() == VirtualMachine.Type.DomainRouter) {
+ doDomainRouterSetup();
+ }
+
+ VirtualEthernetCardType nicDeviceType = VirtualEthernetCardType.valueOf(vmSpec.getDetails().get(VmDetailConstants.NIC_ADAPTER));
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("VM " + vmInternalCSName + " will be started with NIC device type: " + nicDeviceType);
+ }
+
+ NiciraNvpApiVersion.logNiciraApiVersion();
+
+ if (LOGGER.isTraceEnabled()) {
+ LOGGER.debug(String.format("deciding for VM '%s' to use orchestrated NICs or as is.", vmInternalCSName));
+ }
+ nicUuidToDvSwitchUuid = new HashMap<>();
+ LOGGER.info(String.format("adding %d nics to VM '%s'", nics.length, vmInternalCSName));
+ for (NicTO nicTo : sortNicsByDeviceId(nics)) {
+ LOGGER.info("Prepare NIC device based on NicTO: " + vmwareResource.getGson().toJson(nicTo));
+
+ boolean configureVServiceInNexus = (nicTo.getType() == Networks.TrafficType.Guest) && (vmSpec.getDetails().containsKey("ConfigureVServiceInNexus"));
+ VirtualMachine.Type vmType = cmd.getVirtualMachine().getType();
+ Pair<ManagedObjectReference, String> networkInfo = vmwareResource.prepareNetworkFromNicInfo(vmMo.getRunningHost(), nicTo, configureVServiceInNexus, vmType);
+ if ((nicTo.getBroadcastType() != Networks.BroadcastDomainType.Lswitch) || (nicTo.getBroadcastType() == Networks.BroadcastDomainType.Lswitch && NiciraNvpApiVersion.isApiVersionLowerThan("4.2"))) {
+ if (VmwareHelper.isDvPortGroup(networkInfo.first())) {
+ String dvSwitchUuid;
+ ManagedObjectReference dcMor = hyperHost.getHyperHostDatacenter();
+ DatacenterMO dataCenterMo = new DatacenterMO(context, dcMor);
+ ManagedObjectReference dvsMor = dataCenterMo.getDvSwitchMor(networkInfo.first());
+ dvSwitchUuid = dataCenterMo.getDvSwitchUuid(dvsMor);
+ LOGGER.info("Preparing NIC device on dvSwitch : " + dvSwitchUuid);
+ nic = VmwareHelper.prepareDvNicDevice(vmMo, networkInfo.first(), nicDeviceType, networkInfo.second(), dvSwitchUuid, nicTo.getMac(), deviceCount + 1, true, true);
+ if (nicTo.getUuid() != null) {
+ nicUuidToDvSwitchUuid.put(nicTo.getUuid(), dvSwitchUuid);
+ }
+ } else {
+ LOGGER.info("Preparing NIC device on network " + networkInfo.second());
+ nic = VmwareHelper.prepareNicDevice(vmMo, networkInfo.first(), nicDeviceType, networkInfo.second(), nicTo.getMac(), deviceCount + 1, true, true);
+ }
+ } else {
+ //if NSX API VERSION >= 4.2, connect to br-int (nsx.network), do not create portgroup else previous behaviour
+ nic = VmwareHelper.prepareNicOpaque(vmMo, nicDeviceType, networkInfo.second(), nicTo.getMac(), deviceCount + 1, true, true);
+ }
+
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ deviceConfigSpecArray[deviceCount].setDevice(nic);
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare NIC at new device " + vmwareResource.getGson().toJson(deviceConfigSpecArray[deviceCount]));
+
+ // this is really a hacking for DomR, upon DomR startup, we will reset all the NIC allocation after eth3
+ if (nicCount < 3)
+ nicMask |= (1 << nicCount);
+
+ deviceCount++;
+ nicCount++;
+ }
+ return this;
+ }
+
+ private void doDomainRouterSetup() throws Exception {
+ int extraPublicNics = mgr.getRouterExtraPublicNics();
+ if (extraPublicNics > 0 && vmSpec.getDetails().containsKey("PeerRouterInstanceName")) {
+ //Set identical MAC address for RvR on extra public interfaces
+ String peerRouterInstanceName = vmSpec.getDetails().get("PeerRouterInstanceName");
+
+ VirtualMachineMO peerVmMo = hyperHost.findVmOnHyperHost(peerRouterInstanceName);
+ if (peerVmMo == null) {
+ peerVmMo = hyperHost.findVmOnPeerHyperHost(peerRouterInstanceName);
+ }
+
+ if (peerVmMo != null) {
+ String oldMacSequence = generateMacSequence(nics);
+
+ for (int nicIndex = nics.length - extraPublicNics; nicIndex < nics.length; nicIndex++) {
+ VirtualDevice nicDevice = peerVmMo.getNicDeviceByIndex(nics[nicIndex].getDeviceId());
+ if (nicDevice != null) {
+ String mac = ((VirtualEthernetCard)nicDevice).getMacAddress();
+ if (mac != null) {
+ LOGGER.info("Use same MAC as previous RvR, the MAC is " + mac + " for extra NIC with device id: " + nics[nicIndex].getDeviceId());
+ nics[nicIndex].setMac(mac);
+ }
+ }
+ }
+
+ if (!StringUtils.isBlank(vmSpec.getBootArgs())) {
+ String newMacSequence = generateMacSequence(nics);
+ vmSpec.setBootArgs(vmwareResource.replaceNicsMacSequenceInBootArgs(oldMacSequence, newMacSequence, vmSpec));
+ }
+ }
+ }
+ }
+ }
+
+ private class IsoSetup {
+ private VirtualMachineTO vmSpec;
+ private VmwareManager mgr;
+ private VmwareHypervisorHost hyperHost;
+ private VirtualMachineMO vmMo;
+ private DiskTO[] disks;
+ private DiskTO volIso;
+ private VirtualDeviceConfigSpec[] deviceConfigSpecArray;
+ private int deviceCount;
+ private int ideUnitNumber;
+
+ public IsoSetup(VirtualMachineTO vmSpec, VmwareManager mgr, VmwareHypervisorHost hyperHost, VirtualMachineMO vmMo, DiskTO[] disks, DiskTO volIso,
+ VirtualDeviceConfigSpec[] deviceConfigSpecArray, int deviceCount, int ideUnitNumber) {
+ this.vmSpec = vmSpec;
+ this.mgr = mgr;
+ this.hyperHost = hyperHost;
+ this.vmMo = vmMo;
+ this.disks = disks;
+ this.volIso = volIso;
+ this.deviceConfigSpecArray = deviceConfigSpecArray;
+ this.deviceCount = deviceCount;
+ this.ideUnitNumber = ideUnitNumber;
+ }
+
+ public int getDeviceCount() {
+ return deviceCount;
+ }
+
+ public int getIdeUnitNumber() {
+ return ideUnitNumber;
+ }
+
+ public IsoSetup invoke() throws Exception {
+ //
+ // Setup ISO device
+ //
+
+ // vAPP ISO
+ // FR37 the native deploy mechs should create this for us
+ if (vmSpec.getOvfProperties() != null) {
+ if (LOGGER.isTraceEnabled()) {
+ // FR37 TODO add more usefull info (if we keep this bit
+ LOGGER.trace("adding iso for properties for 'xxx'");
+ }
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ Pair<VirtualDevice, Boolean> isoInfo = VmwareHelper.prepareIsoDevice(vmMo, null, null, true, true, ideUnitNumber++, deviceCount + 1);
+ deviceConfigSpecArray[deviceCount].setDevice(isoInfo.first());
+ if (isoInfo.second()) {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare vApp ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+ } else {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare vApp ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ }
+ deviceCount++;
+ }
+
+ // prepare systemvm patch ISO
+ if (vmSpec.getType() != VirtualMachine.Type.User) {
+ PrepareSytemVMPatchISOMethod prepareSytemVm = new PrepareSytemVMPatchISOMethod(mgr, hyperHost, vmMo, deviceConfigSpecArray, deviceCount, ideUnitNumber);
+ prepareSytemVm.invoke();
+ deviceCount = prepareSytemVm.getI();
+ ideUnitNumber = prepareSytemVm.getIdeUnitNumber();
+ } else {
+ // Note: we will always plug a CDROM device
+ if (volIso != null) {
+ for (DiskTO vol : disks) {
+ if (vol.getType() == Volume.Type.ISO) {
+
+ TemplateObjectTO iso = (TemplateObjectTO)vol.getData();
+
+ if (iso.getPath() != null && !iso.getPath().isEmpty()) {
+ DataStoreTO imageStore = iso.getDataStore();
+ if (!(imageStore instanceof NfsTO)) {
+ LOGGER.debug("unsupported protocol");
+ throw new Exception("unsupported protocol");
+ }
+ NfsTO nfsImageStore = (NfsTO)imageStore;
+ String isoPath = nfsImageStore.getUrl() + File.separator + iso.getPath();
+ Pair<String, ManagedObjectReference> isoDatastoreInfo = getIsoDatastoreInfo(hyperHost, isoPath);
+ assert (isoDatastoreInfo != null);
+ assert (isoDatastoreInfo.second() != null);
+
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ Pair<VirtualDevice, Boolean> isoInfo = VmwareHelper
+ .prepareIsoDevice(vmMo, isoDatastoreInfo.first(), isoDatastoreInfo.second(), true, true, ideUnitNumber++, deviceCount + 1);
+ deviceConfigSpecArray[deviceCount].setDevice(isoInfo.first());
+ if (isoInfo.second()) {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at new device " + vmwareResource.getGson().toJson(isoInfo.first()));
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+ } else {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ }
+ }
+ deviceCount++;
+ }
+ }
+ } else {
+ deviceConfigSpecArray[deviceCount] = new VirtualDeviceConfigSpec();
+ Pair<VirtualDevice, Boolean> isoInfo = VmwareHelper.prepareIsoDevice(vmMo, null, null, true, true, ideUnitNumber++, deviceCount + 1);
+ deviceConfigSpecArray[deviceCount].setDevice(isoInfo.first());
+ if (isoInfo.second()) {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.ADD);
+ } else {
+ if (LOGGER.isDebugEnabled())
+ LOGGER.debug("Prepare ISO volume at existing device " + vmwareResource.getGson().toJson(isoInfo.first()));
+
+ deviceConfigSpecArray[deviceCount].setOperation(VirtualDeviceConfigSpecOperation.EDIT);
+ }
+ deviceCount++;
+ }
+ }
+ return this;
+ }
+
+ // isoUrl sample content :
+ // nfs://192.168.10.231/export/home/kelven/vmware-test/secondary/template/tmpl/2/200//200-2-80f7ee58-6eff-3a2d-bcb0-59663edf6d26.iso
+ private Pair<String, ManagedObjectReference> getIsoDatastoreInfo(VmwareHypervisorHost hyperHost, String isoUrl) throws Exception {
+
+ assert (isoUrl != null);
+ int isoFileNameStartPos = isoUrl.lastIndexOf("/");
+ if (isoFileNameStartPos < 0) {
+ throw new Exception("Invalid ISO path info");
+ }
+
+ String isoFileName = isoUrl.substring(isoFileNameStartPos);
+
+ int templateRootPos = isoUrl.indexOf("template/tmpl");
+ templateRootPos = (templateRootPos < 0 ? isoUrl.indexOf(ConfigDrive.CONFIGDRIVEDIR) : templateRootPos);
+ if (templateRootPos < 0) {
+ throw new Exception("Invalid ISO path info");
+ }
+
+ String storeUrl = isoUrl.substring(0, templateRootPos - 1);
+ String isoPath = isoUrl.substring(templateRootPos, isoFileNameStartPos);
+
+ ManagedObjectReference morDs = vmwareResource.prepareSecondaryDatastoreOnHost(storeUrl);
+ DatastoreMO dsMo = new DatastoreMO(vmwareResource.getServiceContext(), morDs);
+
+ return new Pair<String, ManagedObjectReference>(String.format("[%s] %s%s", dsMo.getName(), isoPath, isoFileName), morDs);
+ }
+ }
+
+ private class PrepareRunningVMForConfiguration {
+ private String vmInternalCSName;
+ private Pair<String, String> controllerInfo;
+ private Boolean systemVm;
+ private VirtualMachineMO vmMo;
+ private DiskControllerType systemVmScsiControllerType;
+ private int firstScsiControllerBusNum;
+ private int numScsiControllerForSystemVm;
+ private VirtualMachineDiskInfoBuilder diskInfoBuilder;
+ private VirtualDevice[] nicDevices;
+ private boolean hasSnapshot;
+
+ public PrepareRunningVMForConfiguration(String vmInternalCSName, Pair<String, String> controllerInfo, Boolean systemVm, VirtualMachineMO vmMo,
+ DiskControllerType systemVmScsiControllerType, int firstScsiControllerBusNum, int numScsiControllerForSystemVm) {
+ this.vmInternalCSName = vmInternalCSName;
+ this.controllerInfo = controllerInfo;
+ this.systemVm = systemVm;
+ this.vmMo = vmMo;
+ this.systemVmScsiControllerType = systemVmScsiControllerType;
+ this.firstScsiControllerBusNum = firstScsiControllerBusNum;
+ this.numScsiControllerForSystemVm = numScsiControllerForSystemVm;
+ }
+
+ public VirtualMachineDiskInfoBuilder getDiskInfoBuilder() {
+ return diskInfoBuilder;
+ }
+
+ public VirtualDevice[] getNicDevices() {
+ return nicDevices;
+ }
+
+ public boolean isHasSnapshot() {
+ return hasSnapshot;
+ }
+
+ public PrepareRunningVMForConfiguration invoke() throws Exception {
+ LOGGER.info("VM " + vmInternalCSName + " already exists, tear down devices for reconfiguration");
+ if (VmwareResource.getVmPowerState(vmMo) != VirtualMachine.PowerState.PowerOff) {
+ vmMo.safePowerOff(vmwareResource.getShutdownWaitMs());
+ }
+
+ // retrieve disk information before we tear down
+ diskInfoBuilder = vmMo.getDiskInfoBuilder();
+ hasSnapshot = vmMo.hasSnapshot();
+ nicDevices = vmMo.getNicDevices();
+ // FR37 - only tear nics, and add nics per the provided nics list
Review comment:
needs addressing
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-669095889
@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-669058448
@blueorangutan package
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-668304723
<b>Trillian test result (tid-2251)</b>
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 43504 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4235-t2251-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_accounts.py
Intermittent failure detected: /marvin/tests/smoke/test_async_job.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vgpu_enabled_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vms_with_varied_deploymentplanners.py
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_list_ids_parameter.py
Intermittent failure detected: /marvin/tests/smoke/test_loadbalance.py
Intermittent failure detected: /marvin/tests/smoke/test_nested_virtualization.py
Intermittent failure detected: /marvin/tests/smoke/test_network_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_network.py
Intermittent failure detected: /marvin/tests/smoke/test_nic.py
Intermittent failure detected: /marvin/tests/smoke/test_password_server.py
Intermittent failure detected: /marvin/tests/smoke/test_portforwardingrules.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_projects.py
Intermittent failure detected: /marvin/tests/smoke/test_reset_vm_on_reboot.py
Intermittent failure detected: /marvin/tests/smoke/test_resource_accounting.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dhcphosts.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dns.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dnsservice.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_iptables_default_policy.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_routers.py
Intermittent failure detected: /marvin/tests/smoke/test_service_offerings.py
Intermittent failure detected: /marvin/tests/smoke/test_snapshots.py
Intermittent failure detected: /marvin/tests/smoke/test_ssvm.py
Smoke tests completed. 44 look OK, 25 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_forceDeleteDomain | `Failure` | 81.74 | test_accounts.py
test_forceDeleteDomain | `Error` | 105.32 | test_accounts.py
test_query_async_job_result | `Error` | 84.76 | test_async_job.py
test_3d_gpu_support | `Error` | 221.26 | test_deploy_vgpu_enabled_vm.py
test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80 | `Error` | 189.77 | test_internal_lb.py
test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80 | `Error` | 191.91 | test_internal_lb.py
test_04_rvpc_internallb_haproxy_stats_on_all_interfaces | `Error` | 174.20 | test_internal_lb.py
test_04_rvpc_internallb_haproxy_stats_on_all_interfaces | `Error` | 175.28 | test_internal_lb.py
test_01_deploy_kubernetes_cluster | `Error` | 163.74 | test_kubernetes_clusters.py
test_02_deploy_kubernetes_ha_cluster | `Error` | 133.22 | test_kubernetes_clusters.py
test_04_deploy_and_upgrade_kubernetes_cluster | `Error` | 81.90 | test_kubernetes_clusters.py
test_05_deploy_and_upgrade_kubernetes_ha_cluster | `Error` | 77.77 | test_kubernetes_clusters.py
test_06_deploy_and_invalid_upgrade_kubernetes_cluster | `Error` | 80.91 | test_kubernetes_clusters.py
test_07_deploy_and_scale_kubernetes_cluster | `Error` | 83.87 | test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown | `Error` | 118.24 | test_kubernetes_clusters.py
ContextSuite context=TestListIdsParams>:setup | `Error` | 0.00 | test_list_ids_parameter.py
test_nested_virtualization_vmware | `Error` | 329.98 | test_nested_virtualization.py
test_network_acl | `Error` | 333.57 | test_network_acl.py
test_delete_account | `Error` | 347.19 | test_network.py
test_delete_network_while_vm_on_it | `Error` | 255.75 | test_network.py
test_delete_network_while_vm_on_it | `Error` | 255.83 | test_network.py
test_deploy_vm_l2network | `Error` | 255.68 | test_network.py
test_deploy_vm_l2network | `Error` | 256.77 | test_network.py
test_l2network_restart | `Error` | 264.04 | test_network.py
test_l2network_restart | `Error` | 265.13 | test_network.py
ContextSuite context=TestL2Networks>:teardown | `Error` | 266.24 | test_network.py
ContextSuite context=TestPortForwarding>:setup | `Error` | 359.69 | test_network.py
ContextSuite context=TestPublicIP>:setup | `Error` | 69.84 | test_network.py
test_reboot_router | `Error` | 84.47 | test_network.py
test_releaseIP | `Error` | 86.48 | test_network.py
ContextSuite context=TestRouterRules>:setup | `Error` | 176.15 | test_network.py
test_03_nic_multiple_vmware | `Error` | 339.89 | test_nic.py
ContextSuite context=TestIsolatedNetworksPasswdServer>:setup | `Error` | 0.00 | test_password_server.py
test_01_create_delete_portforwarding_fornonvpc | `Error` | 333.40 | test_portforwardingrules.py
test_02_vpc_privategw_static_routes | `Failure` | 376.45 | test_privategw_acl.py
test_02_vpc_privategw_static_routes | `Error` | 378.65 | test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup | `Failure` | 143.52 | test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup | `Error` | 145.70 | test_privategw_acl.py
test_04_rvpc_privategw_static_routes | `Failure` | 282.44 | test_privategw_acl.py
test_04_rvpc_privategw_static_routes | `Error` | 284.64 | test_privategw_acl.py
test_09_project_suspend | `Error` | 335.86 | test_projects.py
test_10_project_activation | `Error` | 267.01 | test_projects.py
ContextSuite context=TestResetVmOnReboot>:setup | `Error` | 0.00 | test_reset_vm_on_reboot.py
test_01_so_removal_resource_update | `Error` | 325.34 | test_resource_accounting.py
ContextSuite context=TestRouterDnsService>:setup | `Error` | 0.00 | test_router_dnsservice.py
ContextSuite context=TestRouterDHCPHosts>:setup | `Error` | 0.00 | test_router_dhcphosts.py
ContextSuite context=TestRouterDHCPOpts>:setup | `Error` | 0.00 | test_router_dhcphosts.py
ContextSuite context=TestRouterDns>:setup | `Error` | 0.00 | test_router_dns.py
test_02_routervm_iptables_policies | `Error` | 321.99 | test_routers_iptables_default_policy.py
test_01_single_VPC_iptables_policies | `Error` | 327.12 | test_routers_iptables_default_policy.py
test_01_single_VPC_iptables_policies | `Error` | 341.65 | test_routers_iptables_default_policy.py
test_01_isolate_network_FW_PF_default_routes_egress_true | `Error` | 319.10 | test_routers_network_ops.py
test_02_isolate_network_FW_PF_default_routes_egress_false | `Error` | 322.55 | test_routers_network_ops.py
test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true | `Error` | 421.73 | test_routers_network_ops.py
test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false | `Error` | 431.69 | test_routers_network_ops.py
test_03_RVR_Network_check_router_state | `Error` | 434.94 | test_routers_network_ops.py
ContextSuite context=TestRouterServices>:setup | `Error` | 0.00 | test_routers.py
ContextSuite context=TestServiceOfferings>:setup | `Error` | 338.93 | test_service_offerings.py
ContextSuite context=TestSnapshotRootDisk>:setup | `Error` | 0.00 | test_snapshots.py
test_05_stop_ssvm | `Failure` | 925.30 | test_ssvm.py
test_06_stop_cpvm | `Error` | 75.36 | test_ssvm.py
test_07_reboot_ssvm | `Error` | 0.01 | test_ssvm.py
test_08_reboot_cpvm | `Error` | 0.01 | test_ssvm.py
test_09_destroy_ssvm | `Error` | 0.01 | test_ssvm.py
test_10_destroy_cpvm | `Error` | 0.01 | test_ssvm.py
test_11_ss_nfs_version_on_ssvm | `Error` | 0.01 | test_ssvm.py
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-669095282
> @blueorangutan package
@rhtyd why do you restart this, it has unit test failures? Did i miss a fix for those since they apeared?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-669401822
Packaging result: ✔centos7 ✔debian. JID-1664
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
nvazquez commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-667571254
@blueorangutan package
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez commented on a change in pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
nvazquez commented on a change in pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#discussion_r463571948
##########
File path: core/src/main/java/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java
##########
@@ -89,7 +89,7 @@ protected Answer execute(CopyCommand cmd) {
return processor.copyTemplateToPrimaryStorage(cmd);
} else if (srcData.getObjectType() == DataObjectType.TEMPLATE && srcDataStore.getRole() == DataStoreRole.Primary &&
destDataStore.getRole() == DataStoreRole.Primary) {
- //clone template to a volume
+ // FR37 pretend to clone template to a volume but actually create a cloned vm
Review comment:
Yes - clones the VM
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-668409758
Packaging result: ✔centos7 ✔debian. JID-1634
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-669856135
@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#discussion_r463663995
##########
File path: api/src/main/java/org/apache/cloudstack/api/response/TemplateOVFPropertyResponse.java
##########
@@ -16,13 +16,18 @@
// under the License.
package org.apache.cloudstack.api.response;
-import com.cloud.agent.api.storage.OVFProperty;
-import com.cloud.serializer.Param;
-import com.google.gson.annotations.SerializedName;
import org.apache.cloudstack.api.ApiConstants;
import org.apache.cloudstack.api.BaseResponse;
import org.apache.cloudstack.api.EntityReference;
+import com.cloud.agent.api.storage.OVFProperty;
+import com.cloud.serializer.Param;
+import com.google.gson.annotations.SerializedName;
+
+/**
+ * the placeholder of parameters to fill for deployment
+ // FR37 TODO remname for generic use
Review comment:
you removed the javadoc and not just the FR37 line did you?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez commented on a change in pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
nvazquez commented on a change in pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#discussion_r463551853
##########
File path: api/src/main/java/com/cloud/agent/api/storage/OVFProperty.java
##########
@@ -19,6 +19,7 @@
package com.cloud.agent.api.storage;
+// FR37 rename
Review comment:
It was needed but now the `TemplateOVFPropertyVO` is deprecated. I'll remove it and its related classes. Also, table `template_ovf_properties` is not needed anymore
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
nvazquez commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-669474753
@blueorangutan test centos7 vmware-67u3
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-667660478
<b>Trillian test result (tid-2249)</b>
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 56257 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4235-t2249-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_accounts.py
Intermittent failure detected: /marvin/tests/smoke/test_affinity_groups_projects.py
Intermittent failure detected: /marvin/tests/smoke/test_async_job.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vgpu_enabled_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vms_with_varied_deploymentplanners.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vm_with_userdata.py
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_loadbalance.py
Intermittent failure detected: /marvin/tests/smoke/test_nested_virtualization.py
Intermittent failure detected: /marvin/tests/smoke/test_network_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_network.py
Intermittent failure detected: /marvin/tests/smoke/test_nic.py
Intermittent failure detected: /marvin/tests/smoke/test_password_server.py
Intermittent failure detected: /marvin/tests/smoke/test_portforwardingrules.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_projects.py
Intermittent failure detected: /marvin/tests/smoke/test_reset_vm_on_reboot.py
Intermittent failure detected: /marvin/tests/smoke/test_resource_accounting.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dhcphosts.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dns.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dnsservice.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_iptables_default_policy.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_routers.py
Intermittent failure detected: /marvin/tests/smoke/test_service_offerings.py
Intermittent failure detected: /marvin/tests/smoke/test_snapshots.py
Intermittent failure detected: /marvin/tests/smoke/test_ssvm.py
Intermittent failure detected: /marvin/tests/smoke/test_usage.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 50 look OK, 29 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
ContextSuite context=TestAddVmToSubDomain>:setup | `Error` | 215.14 | test_accounts.py
test_DeleteDomain | `Error` | 202.20 | test_accounts.py
test_forceDeleteDomain | `Failure` | 98.27 | test_accounts.py
test_forceDeleteDomain | `Error` | 118.79 | test_accounts.py
test_01_user_remove_VM_running | `Error` | 84.71 | test_accounts.py
test_query_async_job_result | `Error` | 95.01 | test_async_job.py
test_3d_gpu_support | `Error` | 206.31 | test_deploy_vgpu_enabled_vm.py
test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80 | `Error` | 265.70 | test_internal_lb.py
test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80 | `Error` | 296.49 | test_internal_lb.py
test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80 | `Error` | 322.82 | test_internal_lb.py
test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80 | `Error` | 337.20 | test_internal_lb.py
test_03_vpc_internallb_haproxy_stats_on_all_interfaces | `Error` | 108.00 | test_internal_lb.py
test_03_vpc_internallb_haproxy_stats_on_all_interfaces | `Error` | 109.06 | test_internal_lb.py
test_deployvm_userconcentrated | `Error` | 25.93 | test_deploy_vms_with_varied_deploymentplanners.py
test_deployvm_userdispersing | `Error` | 21.85 | test_deploy_vms_with_varied_deploymentplanners.py
ContextSuite context=TestLoadBalance>:setup | `Error` | 0.00 | test_loadbalance.py
test_01_deploy_kubernetes_cluster | `Error` | 187.75 | test_kubernetes_clusters.py
test_02_deploy_kubernetes_ha_cluster | `Error` | 164.55 | test_kubernetes_clusters.py
test_04_deploy_and_upgrade_kubernetes_cluster | `Error` | 78.66 | test_kubernetes_clusters.py
test_05_deploy_and_upgrade_kubernetes_ha_cluster | `Error` | 77.65 | test_kubernetes_clusters.py
test_06_deploy_and_invalid_upgrade_kubernetes_cluster | `Error` | 80.75 | test_kubernetes_clusters.py
test_07_deploy_and_scale_kubernetes_cluster | `Error` | 83.78 | test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown | `Error` | 119.11 | test_kubernetes_clusters.py
test_nested_virtualization_vmware | `Error` | 333.00 | test_nested_virtualization.py
test_network_acl | `Error` | 335.41 | test_network_acl.py
test_delete_account | `Error` | 344.37 | test_network.py
test_delete_network_while_vm_on_it | `Error` | 18.75 | test_network.py
test_delete_network_while_vm_on_it | `Error` | 19.82 | test_network.py
test_deploy_vm_l2network | `Error` | 256.45 | test_network.py
test_deploy_vm_l2network | `Error` | 257.53 | test_network.py
test_l2network_restart | `Error` | 11.67 | test_network.py
test_l2network_restart | `Error` | 12.73 | test_network.py
ContextSuite context=TestL2Networks>:teardown | `Error` | 13.80 | test_network.py
ContextSuite context=TestPortForwarding>:setup | `Error` | 98.25 | test_network.py
ContextSuite context=TestPublicIP>:setup | `Error` | 81.89 | test_network.py
test_reboot_router | `Error` | 333.19 | test_network.py
test_releaseIP | `Error` | 86.49 | test_network.py
ContextSuite context=TestRouterRules>:setup | `Error` | 423.01 | test_network.py
test_03_nic_multiple_vmware | `Error` | 391.98 | test_nic.py
ContextSuite context=TestRouterDnsService>:setup | `Error` | 0.00 | test_router_dnsservice.py
ContextSuite context=TestIsolatedNetworksPasswdServer>:setup | `Error` | 0.00 | test_password_server.py
test_01_create_delete_portforwarding_fornonvpc | `Error` | 336.79 | test_portforwardingrules.py
test_02_vpc_privategw_static_routes | `Failure` | 401.03 | test_privategw_acl.py
test_02_vpc_privategw_static_routes | `Error` | 403.20 | test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup | `Failure` | 403.28 | test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup | `Error` | 405.46 | test_privategw_acl.py
test_04_rvpc_privategw_static_routes | `Failure` | 563.82 | test_privategw_acl.py
test_04_rvpc_privategw_static_routes | `Error` | 566.02 | test_privategw_acl.py
test_09_project_suspend | `Error` | 330.92 | test_projects.py
test_10_project_activation | `Error` | 273.68 | test_projects.py
ContextSuite context=TestResetVmOnReboot>:setup | `Error` | 0.00 | test_reset_vm_on_reboot.py
test_01_so_removal_resource_update | `Error` | 330.11 | test_resource_accounting.py
ContextSuite context=TestRouterDHCPHosts>:setup | `Error` | 0.00 | test_router_dhcphosts.py
ContextSuite context=TestRouterDHCPOpts>:setup | `Error` | 0.00 | test_router_dhcphosts.py
ContextSuite context=TestRouterDns>:setup | `Error` | 0.00 | test_router_dns.py
test_02_routervm_iptables_policies | `Error` | 331.93 | test_routers_iptables_default_policy.py
test_01_single_VPC_iptables_policies | `Error` | 327.17 | test_routers_iptables_default_policy.py
test_01_single_VPC_iptables_policies | `Error` | 339.57 | test_routers_iptables_default_policy.py
test_01_isolate_network_FW_PF_default_routes_egress_true | `Error` | 327.02 | test_routers_network_ops.py
test_02_isolate_network_FW_PF_default_routes_egress_false | `Error` | 324.90 | test_routers_network_ops.py
test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true | `Error` | 439.27 | test_routers_network_ops.py
test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false | `Error` | 194.01 | test_routers_network_ops.py
test_03_RVR_Network_check_router_state | `Error` | 216.36 | test_routers_network_ops.py
ContextSuite context=TestRouterServices>:setup | `Error` | 0.00 | test_routers.py
ContextSuite context=TestServiceOfferings>:setup | `Error` | 346.54 | test_service_offerings.py
ContextSuite context=TestSnapshotRootDisk>:setup | `Error` | 0.00 | test_snapshots.py
test_05_stop_ssvm | `Failure` | 923.42 | test_ssvm.py
test_06_stop_cpvm | `Failure` | 921.41 | test_ssvm.py
test_07_reboot_ssvm | `Failure` | 41.99 | test_ssvm.py
test_01_volume_usage | `Error` | 88.35 | test_usage.py
test_07_start_vm | `Error` | 22.46 | test_vm_life_cycle.py
test_08_reboot_vm | `Error` | 1.07 | test_vm_life_cycle.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL | `Failure` | 465.92 | test_vpc_redundant.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL | `Error` | 483.05 | test_vpc_redundant.py
test_02_redundant_VPC_default_routes | `Failure` | 404.16 | test_vpc_redundant.py
test_02_redundant_VPC_default_routes | `Error` | 404.19 | test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers | `Error` | 0.01 | test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nics | `Error` | 0.00 | test_vpc_redundant.py
test_05_rvpc_multi_tiers | `Error` | 0.00 | test_vpc_redundant.py
ContextSuite context=TestVPCRedundancy>:teardown | `Error` | 0.01 | test_vpc_redundant.py
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-669035890
<b>Trillian test result (tid-2264)</b>
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 55395 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4235-t2264-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_accounts.py
Intermittent failure detected: /marvin/tests/smoke/test_async_job.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vgpu_enabled_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vms_with_varied_deploymentplanners.py
Intermittent failure detected: /marvin/tests/smoke/test_domain_service_offerings.py
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_list_ids_parameter.py
Intermittent failure detected: /marvin/tests/smoke/test_loadbalance.py
Intermittent failure detected: /marvin/tests/smoke/test_nested_virtualization.py
Intermittent failure detected: /marvin/tests/smoke/test_network_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_network.py
Intermittent failure detected: /marvin/tests/smoke/test_nic.py
Intermittent failure detected: /marvin/tests/smoke/test_password_server.py
Intermittent failure detected: /marvin/tests/smoke/test_portforwardingrules.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_projects.py
Intermittent failure detected: /marvin/tests/smoke/test_reset_vm_on_reboot.py
Intermittent failure detected: /marvin/tests/smoke/test_resource_accounting.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dhcphosts.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dns.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dnsservice.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_iptables_default_policy.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_routers.py
Intermittent failure detected: /marvin/tests/smoke/test_service_offerings.py
Intermittent failure detected: /marvin/tests/smoke/test_snapshots.py
Intermittent failure detected: /marvin/tests/smoke/test_ssvm.py
Intermittent failure detected: /marvin/tests/smoke/test_usage.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 50 look OK, 29 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_forceDeleteDomain | `Failure` | 100.21 | test_accounts.py
test_forceDeleteDomain | `Error` | 120.99 | test_accounts.py
test_01_user_remove_VM_running | `Error` | 57.75 | test_accounts.py
test_query_async_job_result | `Error` | 89.05 | test_async_job.py
test_3d_gpu_support | `Error` | 292.06 | test_deploy_vgpu_enabled_vm.py
test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80 | `Error` | 156.82 | test_internal_lb.py
test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80 | `Error` | 166.23 | test_internal_lb.py
test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80 | `Error` | 265.27 | test_internal_lb.py
test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80 | `Error` | 279.81 | test_internal_lb.py
test_03_deploy_vm_domain_service_offering | `Error` | 188.41 | test_domain_service_offerings.py
test_01_deploy_kubernetes_cluster | `Error` | 163.06 | test_kubernetes_clusters.py
test_02_deploy_kubernetes_ha_cluster | `Error` | 138.11 | test_kubernetes_clusters.py
test_04_deploy_and_upgrade_kubernetes_cluster | `Error` | 94.27 | test_kubernetes_clusters.py
test_05_deploy_and_upgrade_kubernetes_ha_cluster | `Error` | 133.36 | test_kubernetes_clusters.py
test_06_deploy_and_invalid_upgrade_kubernetes_cluster | `Error` | 67.40 | test_kubernetes_clusters.py
test_07_deploy_and_scale_kubernetes_cluster | `Error` | 64.24 | test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown | `Error` | 145.45 | test_kubernetes_clusters.py
ContextSuite context=TestListIdsParams>:setup | `Error` | 0.00 | test_list_ids_parameter.py
test_nested_virtualization_vmware | `Error` | 328.28 | test_nested_virtualization.py
test_network_acl | `Error` | 339.80 | test_network_acl.py
test_delete_account | `Error` | 343.03 | test_network.py
test_delete_network_while_vm_on_it | `Error` | 266.31 | test_network.py
test_delete_network_while_vm_on_it | `Error` | 267.42 | test_network.py
test_deploy_vm_l2network | `Error` | 258.15 | test_network.py
test_deploy_vm_l2network | `Error` | 259.26 | test_network.py
test_l2network_restart | `Error` | 11.99 | test_network.py
test_l2network_restart | `Error` | 13.09 | test_network.py
ContextSuite context=TestL2Networks>:teardown | `Error` | 14.21 | test_network.py
ContextSuite context=TestPortForwarding>:setup | `Error` | 93.80 | test_network.py
ContextSuite context=TestPublicIP>:setup | `Error` | 306.19 | test_network.py
test_reboot_router | `Error` | 59.16 | test_network.py
test_releaseIP | `Error` | 314.34 | test_network.py
ContextSuite context=TestRouterRules>:setup | `Error` | 401.17 | test_network.py
test_03_nic_multiple_vmware | `Error` | 313.55 | test_nic.py
ContextSuite context=TestIsolatedNetworksPasswdServer>:setup | `Error` | 0.00 | test_password_server.py
test_01_create_delete_portforwarding_fornonvpc | `Error` | 315.54 | test_portforwardingrules.py
test_02_vpc_privategw_static_routes | `Failure` | 375.30 | test_privategw_acl.py
test_02_vpc_privategw_static_routes | `Error` | 377.56 | test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup | `Failure` | 116.86 | test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup | `Error` | 119.10 | test_privategw_acl.py
test_04_rvpc_privategw_static_routes | `Failure` | 270.42 | test_privategw_acl.py
test_04_rvpc_privategw_static_routes | `Error` | 272.72 | test_privategw_acl.py
test_09_project_suspend | `Error` | 328.50 | test_projects.py
test_10_project_activation | `Error` | 27.32 | test_projects.py
ContextSuite context=TestResetVmOnReboot>:setup | `Error` | 0.00 | test_reset_vm_on_reboot.py
test_01_so_removal_resource_update | `Error` | 323.16 | test_resource_accounting.py
ContextSuite context=TestRouterDnsService>:setup | `Error` | 0.00 | test_router_dnsservice.py
ContextSuite context=TestRouterDHCPHosts>:setup | `Error` | 0.00 | test_router_dhcphosts.py
ContextSuite context=TestRouterDHCPOpts>:setup | `Error` | 0.00 | test_router_dhcphosts.py
ContextSuite context=TestRouterDns>:setup | `Error` | 0.00 | test_router_dns.py
test_02_routervm_iptables_policies | `Error` | 327.04 | test_routers_iptables_default_policy.py
test_01_single_VPC_iptables_policies | `Error` | 67.17 | test_routers_iptables_default_policy.py
test_01_single_VPC_iptables_policies | `Error` | 87.99 | test_routers_iptables_default_policy.py
test_01_isolate_network_FW_PF_default_routes_egress_true | `Error` | 306.16 | test_routers_network_ops.py
test_02_isolate_network_FW_PF_default_routes_egress_false | `Error` | 320.97 | test_routers_network_ops.py
test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true | `Error` | 150.08 | test_routers_network_ops.py
test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false | `Error` | 156.48 | test_routers_network_ops.py
test_03_RVR_Network_check_router_state | `Error` | 407.15 | test_routers_network_ops.py
ContextSuite context=TestRouterServices>:setup | `Error` | 0.00 | test_routers.py
ContextSuite context=TestServiceOfferings>:setup | `Error` | 341.06 | test_service_offerings.py
ContextSuite context=TestSnapshotRootDisk>:setup | `Error` | 0.00 | test_snapshots.py
test_05_stop_ssvm | `Failure` | 922.14 | test_ssvm.py
test_06_stop_cpvm | `Failure` | 921.21 | test_ssvm.py
test_07_reboot_ssvm | `Failure` | 22.72 | test_ssvm.py
test_01_volume_usage | `Error` | 92.06 | test_usage.py
test_07_start_vm | `Error` | 18.61 | test_vm_life_cycle.py
test_08_reboot_vm | `Error` | 1.11 | test_vm_life_cycle.py
test_11_migrate_vm | `Error` | 57.58 | test_vm_life_cycle.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL | `Failure` | 412.45 | test_vpc_redundant.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL | `Error` | 430.36 | test_vpc_redundant.py
test_02_redundant_VPC_default_routes | `Failure` | 158.64 | test_vpc_redundant.py
test_02_redundant_VPC_default_routes | `Error` | 176.82 | test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers | `Failure` | 157.54 | test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers | `Error` | 175.41 | test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nics | `Failure` | 155.88 | test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nics | `Error` | 174.07 | test_vpc_redundant.py
test_05_rvpc_multi_tiers | `Failure` | 396.67 | test_vpc_redundant.py
test_05_rvpc_multi_tiers | `Error` | 396.70 | test_vpc_redundant.py
ContextSuite context=TestVPCRedundancy>:teardown | `Error` | 396.73 | test_vpc_redundant.py
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
nvazquez commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-670101780
Closing this PR - moving work to PR #4250
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-669058977
@nvazquez @shwstppr unit tests are failing, please review travis or local build and fix. I see:
```
...
[ERROR] testCheckAffinityNullPreferredHosts(com.cloud.vm.DeploymentPlanningManagerImplTest) Time elapsed: 0.005 s <<< ERROR!
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'deploymentPlanningManagerImpl': Unsatisfied dependency expressed through field 'templateDao'; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'com.cloud.storage.dao.VMTemplateDao' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {@javax.inject.Inject()}
at com.cloud.vm.DeploymentPlanningManagerImplTest.testSetUp(DeploymentPlanningManagerImplTest.java:158)
Caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'com.cloud.storage.dao.VMTemplateDao' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {@javax.inject.Inject()}
at com.cloud.vm.DeploymentPlanningManagerImplTest.testSetUp(DeploymentPlanningManagerImplTest.java:158)
[ERROR] Errors:
[ERROR] DeploymentPlanningManagerImplTest.testSetUp:158 » UnsatisfiedDependency Error ...
[ERROR] DeploymentPlanningManagerImplTest.testSetUp:158 » UnsatisfiedDependency Error ...
[ERROR] DeploymentPlanningManagerImplTest.testSetUp:158 » UnsatisfiedDependency Error ...
[ERROR] DeploymentPlanningManagerImplTest.testSetUp:158 » UnsatisfiedDependency Error ...
[ERROR] DeploymentPlanningManagerImplTest.testSetUp:158 » UnsatisfiedDependency Error ...
[ERROR] DeploymentPlanningManagerImplTest.testSetUp:158 » UnsatisfiedDependency Error ...
[ERROR] DeploymentPlanningManagerImplTest.testSetUp:158 » UnsatisfiedDependency Error ...
[ERROR] Tests run: 846, Failures: 0, Errors: 7, Skipped: 5
...
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-667195829
@blueorangutan build
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
nvazquez commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-669358753
@blueorangutan package
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-669094553
@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-667571347
@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-668650863
@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-666950593
@blueorangutan test matrix
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-667574695
@nvazquez a Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-668675144
@nvazquez a Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
nvazquez commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-667980507
@blueorangutan test centos7 vmware-67u3
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-670240937
<b>Trillian test result (tid-2283)</b>
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 43727 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4235-t2283-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 82 look OK, 1 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_04_rvpc_network_garbage_collector_nics | `Error` | 3855.17 | test_vpc_redundant.py
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
nvazquez commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-668674348
@blueorangutan test centos7 vmware-67u3
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] shwstppr commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
shwstppr commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-669096103
working on the unit test fix
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-669361317
@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
nvazquez commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-667574574
@blueorangutan test centos7 vmware-67u3
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-669058662
@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-668668690
Packaging result: ✔centos7 ✔debian. JID-1648
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-669855851
@nvazquez there are also Travis/simulator tests failing, pl check. I'll kick test against KVM for regression checks.
@blueorangutan test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
nvazquez commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-668650130
@blueorangutan package
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-666938505
@blueorangutan package
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez closed pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
nvazquez closed pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on a change in pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on a change in pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#discussion_r463662659
##########
File path: api/src/main/java/com/cloud/agent/api/storage/OVFPropertyTO.java
##########
@@ -23,14 +23,15 @@
/**
* Used to represent travel objects like:
+ // FR37 rename
Review comment:
any line with FR37 must be gone before merge!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-667573878
Packaging result: ✔centos7 ✔debian. JID-1633
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-669119277
@blueorangutan test centos7 vmware-67u3
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-669475301
@nvazquez a Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-667980989
@nvazquez a Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] nvazquez commented on a change in pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
nvazquez commented on a change in pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#discussion_r463567127
##########
File path: api/src/main/java/org/apache/cloudstack/api/response/TemplateOVFPropertyResponse.java
##########
@@ -16,13 +16,18 @@
// under the License.
package org.apache.cloudstack.api.response;
-import com.cloud.agent.api.storage.OVFProperty;
-import com.cloud.serializer.Param;
-import com.google.gson.annotations.SerializedName;
import org.apache.cloudstack.api.ApiConstants;
import org.apache.cloudstack.api.BaseResponse;
import org.apache.cloudstack.api.EntityReference;
+import com.cloud.agent.api.storage.OVFProperty;
+import com.cloud.serializer.Param;
+import com.google.gson.annotations.SerializedName;
+
+/**
+ * the placeholder of parameters to fill for deployment
+ // FR37 TODO remname for generic use
Review comment:
Looks good now?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-669119578
@rhtyd a Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-669114429
Packaging result: ✔centos7 ✔debian. JID-1661
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-669067226
Packaging result: ✖centos7 ✖debian. JID-1654
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-666946836
Packaging result: ✔centos7 ✔debian. JID-1631
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan removed a comment on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan removed a comment on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-669035890
<b>Trillian test result (tid-2264)</b>
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 55395 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4235-t2264-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_accounts.py
Intermittent failure detected: /marvin/tests/smoke/test_async_job.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vgpu_enabled_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vms_with_varied_deploymentplanners.py
Intermittent failure detected: /marvin/tests/smoke/test_domain_service_offerings.py
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_list_ids_parameter.py
Intermittent failure detected: /marvin/tests/smoke/test_loadbalance.py
Intermittent failure detected: /marvin/tests/smoke/test_nested_virtualization.py
Intermittent failure detected: /marvin/tests/smoke/test_network_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_network.py
Intermittent failure detected: /marvin/tests/smoke/test_nic.py
Intermittent failure detected: /marvin/tests/smoke/test_password_server.py
Intermittent failure detected: /marvin/tests/smoke/test_portforwardingrules.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_projects.py
Intermittent failure detected: /marvin/tests/smoke/test_reset_vm_on_reboot.py
Intermittent failure detected: /marvin/tests/smoke/test_resource_accounting.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dhcphosts.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dns.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dnsservice.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_iptables_default_policy.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_routers.py
Intermittent failure detected: /marvin/tests/smoke/test_service_offerings.py
Intermittent failure detected: /marvin/tests/smoke/test_snapshots.py
Intermittent failure detected: /marvin/tests/smoke/test_ssvm.py
Intermittent failure detected: /marvin/tests/smoke/test_usage.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 50 look OK, 29 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_forceDeleteDomain | `Failure` | 100.21 | test_accounts.py
test_forceDeleteDomain | `Error` | 120.99 | test_accounts.py
test_01_user_remove_VM_running | `Error` | 57.75 | test_accounts.py
test_query_async_job_result | `Error` | 89.05 | test_async_job.py
test_3d_gpu_support | `Error` | 292.06 | test_deploy_vgpu_enabled_vm.py
test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80 | `Error` | 156.82 | test_internal_lb.py
test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80 | `Error` | 166.23 | test_internal_lb.py
test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80 | `Error` | 265.27 | test_internal_lb.py
test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80 | `Error` | 279.81 | test_internal_lb.py
test_03_deploy_vm_domain_service_offering | `Error` | 188.41 | test_domain_service_offerings.py
test_01_deploy_kubernetes_cluster | `Error` | 163.06 | test_kubernetes_clusters.py
test_02_deploy_kubernetes_ha_cluster | `Error` | 138.11 | test_kubernetes_clusters.py
test_04_deploy_and_upgrade_kubernetes_cluster | `Error` | 94.27 | test_kubernetes_clusters.py
test_05_deploy_and_upgrade_kubernetes_ha_cluster | `Error` | 133.36 | test_kubernetes_clusters.py
test_06_deploy_and_invalid_upgrade_kubernetes_cluster | `Error` | 67.40 | test_kubernetes_clusters.py
test_07_deploy_and_scale_kubernetes_cluster | `Error` | 64.24 | test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown | `Error` | 145.45 | test_kubernetes_clusters.py
ContextSuite context=TestListIdsParams>:setup | `Error` | 0.00 | test_list_ids_parameter.py
test_nested_virtualization_vmware | `Error` | 328.28 | test_nested_virtualization.py
test_network_acl | `Error` | 339.80 | test_network_acl.py
test_delete_account | `Error` | 343.03 | test_network.py
test_delete_network_while_vm_on_it | `Error` | 266.31 | test_network.py
test_delete_network_while_vm_on_it | `Error` | 267.42 | test_network.py
test_deploy_vm_l2network | `Error` | 258.15 | test_network.py
test_deploy_vm_l2network | `Error` | 259.26 | test_network.py
test_l2network_restart | `Error` | 11.99 | test_network.py
test_l2network_restart | `Error` | 13.09 | test_network.py
ContextSuite context=TestL2Networks>:teardown | `Error` | 14.21 | test_network.py
ContextSuite context=TestPortForwarding>:setup | `Error` | 93.80 | test_network.py
ContextSuite context=TestPublicIP>:setup | `Error` | 306.19 | test_network.py
test_reboot_router | `Error` | 59.16 | test_network.py
test_releaseIP | `Error` | 314.34 | test_network.py
ContextSuite context=TestRouterRules>:setup | `Error` | 401.17 | test_network.py
test_03_nic_multiple_vmware | `Error` | 313.55 | test_nic.py
ContextSuite context=TestIsolatedNetworksPasswdServer>:setup | `Error` | 0.00 | test_password_server.py
test_01_create_delete_portforwarding_fornonvpc | `Error` | 315.54 | test_portforwardingrules.py
test_02_vpc_privategw_static_routes | `Failure` | 375.30 | test_privategw_acl.py
test_02_vpc_privategw_static_routes | `Error` | 377.56 | test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup | `Failure` | 116.86 | test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup | `Error` | 119.10 | test_privategw_acl.py
test_04_rvpc_privategw_static_routes | `Failure` | 270.42 | test_privategw_acl.py
test_04_rvpc_privategw_static_routes | `Error` | 272.72 | test_privategw_acl.py
test_09_project_suspend | `Error` | 328.50 | test_projects.py
test_10_project_activation | `Error` | 27.32 | test_projects.py
ContextSuite context=TestResetVmOnReboot>:setup | `Error` | 0.00 | test_reset_vm_on_reboot.py
test_01_so_removal_resource_update | `Error` | 323.16 | test_resource_accounting.py
ContextSuite context=TestRouterDnsService>:setup | `Error` | 0.00 | test_router_dnsservice.py
ContextSuite context=TestRouterDHCPHosts>:setup | `Error` | 0.00 | test_router_dhcphosts.py
ContextSuite context=TestRouterDHCPOpts>:setup | `Error` | 0.00 | test_router_dhcphosts.py
ContextSuite context=TestRouterDns>:setup | `Error` | 0.00 | test_router_dns.py
test_02_routervm_iptables_policies | `Error` | 327.04 | test_routers_iptables_default_policy.py
test_01_single_VPC_iptables_policies | `Error` | 67.17 | test_routers_iptables_default_policy.py
test_01_single_VPC_iptables_policies | `Error` | 87.99 | test_routers_iptables_default_policy.py
test_01_isolate_network_FW_PF_default_routes_egress_true | `Error` | 306.16 | test_routers_network_ops.py
test_02_isolate_network_FW_PF_default_routes_egress_false | `Error` | 320.97 | test_routers_network_ops.py
test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true | `Error` | 150.08 | test_routers_network_ops.py
test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false | `Error` | 156.48 | test_routers_network_ops.py
test_03_RVR_Network_check_router_state | `Error` | 407.15 | test_routers_network_ops.py
ContextSuite context=TestRouterServices>:setup | `Error` | 0.00 | test_routers.py
ContextSuite context=TestServiceOfferings>:setup | `Error` | 341.06 | test_service_offerings.py
ContextSuite context=TestSnapshotRootDisk>:setup | `Error` | 0.00 | test_snapshots.py
test_05_stop_ssvm | `Failure` | 922.14 | test_ssvm.py
test_06_stop_cpvm | `Failure` | 921.21 | test_ssvm.py
test_07_reboot_ssvm | `Failure` | 22.72 | test_ssvm.py
test_01_volume_usage | `Error` | 92.06 | test_usage.py
test_07_start_vm | `Error` | 18.61 | test_vm_life_cycle.py
test_08_reboot_vm | `Error` | 1.11 | test_vm_life_cycle.py
test_11_migrate_vm | `Error` | 57.58 | test_vm_life_cycle.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL | `Failure` | 412.45 | test_vpc_redundant.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL | `Error` | 430.36 | test_vpc_redundant.py
test_02_redundant_VPC_default_routes | `Failure` | 158.64 | test_vpc_redundant.py
test_02_redundant_VPC_default_routes | `Error` | 176.82 | test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers | `Failure` | 157.54 | test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers | `Error` | 175.41 | test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nics | `Failure` | 155.88 | test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nics | `Error` | 174.07 | test_vpc_redundant.py
test_05_rvpc_multi_tiers | `Failure` | 396.67 | test_vpc_redundant.py
test_05_rvpc_multi_tiers | `Error` | 396.70 | test_vpc_redundant.py
ContextSuite context=TestVPCRedundancy>:teardown | `Error` | 396.73 | test_vpc_redundant.py
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-666950717
@rhtyd a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware67, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-668394615
@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-669843256
<b>Trillian test result (tid-2276)</b>
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 49309 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4235-t2276-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vgpu_enabled_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vm_root_resize.py
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_supported_versions.py
Intermittent failure detected: /marvin/tests/smoke/test_network.py
Intermittent failure detected: /marvin/tests/smoke/test_nic.py
Intermittent failure detected: /marvin/tests/smoke/test_projects.py
Intermittent failure detected: /marvin/tests/smoke/test_reset_vm_on_reboot.py
Intermittent failure detected: /marvin/tests/smoke/test_routers.py
Intermittent failure detected: /marvin/tests/smoke/test_service_offerings.py
Intermittent failure detected: /marvin/tests/smoke/test_ssvm.py
Intermittent failure detected: /marvin/tests/smoke/test_usage.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 66 look OK, 12 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_3d_gpu_support | `Error` | 327.04 | test_deploy_vgpu_enabled_vm.py
test_00_deploy_vm_root_resize | `Failure` | 278.67 | test_deploy_vm_root_resize.py
test_01_add_delete_kubernetes_supported_version | `Error` | 1807.57 | test_kubernetes_supported_versions.py
test_reboot_router | `Error` | 239.07 | test_network.py
test_03_nic_multiple_vmware | `Error` | 360.91 | test_nic.py
test_10_project_activation | `Error` | 27.79 | test_projects.py
test_01_reset_vm_on_reboot | `Error` | 17.48 | test_reset_vm_on_reboot.py
test_08_start_router | `Error` | 13.38 | test_routers.py
test_09_reboot_router | `Error` | 1.10 | test_routers.py
test_04_change_offering_small | `Error` | 20.67 | test_service_offerings.py
test_05_stop_ssvm | `Failure` | 920.82 | test_ssvm.py
test_06_stop_cpvm | `Failure` | 919.71 | test_ssvm.py
test_07_reboot_ssvm | `Failure` | 36.81 | test_ssvm.py
test_01_volume_usage | `Error` | 90.60 | test_usage.py
test_07_start_vm | `Error` | 17.45 | test_vm_life_cycle.py
test_08_reboot_vm | `Error` | 1.10 | test_vm_life_cycle.py
test_11_migrate_vm | `Error` | 52.91 | test_vm_life_cycle.py
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-666938722
@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-668394199
@blueorangutan package
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-669099106
@shwstppr a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] blueorangutan commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
blueorangutan commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-667321744
<b>Trillian test result (tid-2245)</b>
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 46736 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4235-t2245-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_accounts.py
Intermittent failure detected: /marvin/tests/smoke/test_async_job.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vgpu_enabled_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vm_root_resize.py
Intermittent failure detected: /marvin/tests/smoke/test_domain_service_offerings.py
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_list_ids_parameter.py
Intermittent failure detected: /marvin/tests/smoke/test_nested_virtualization.py
Intermittent failure detected: /marvin/tests/smoke/test_network_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_network.py
Intermittent failure detected: /marvin/tests/smoke/test_nic.py
Intermittent failure detected: /marvin/tests/smoke/test_password_server.py
Intermittent failure detected: /marvin/tests/smoke/test_portforwardingrules.py
Intermittent failure detected: /marvin/tests/smoke/test_primary_storage.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_projects.py
Intermittent failure detected: /marvin/tests/smoke/test_reset_vm_on_reboot.py
Intermittent failure detected: /marvin/tests/smoke/test_resource_accounting.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dhcphosts.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dns.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dnsservice.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_iptables_default_policy.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_routers.py
Intermittent failure detected: /marvin/tests/smoke/test_service_offerings.py
Intermittent failure detected: /marvin/tests/smoke/test_snapshots.py
Intermittent failure detected: /marvin/tests/smoke/test_ssvm.py
Intermittent failure detected: /marvin/tests/smoke/test_templates.py
Intermittent failure detected: /marvin/tests/smoke/test_usage.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_deployment_planner.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_snapshots.py
Intermittent failure detected: /marvin/tests/smoke/test_volumes.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_router_nics.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Smoke tests completed. 49 look OK, 34 have error(s)
Only failed tests results shown below:
Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_forceDeleteDomain | `Failure` | 200.83 | test_accounts.py
test_forceDeleteDomain | `Error` | 238.86 | test_accounts.py
test_01_user_remove_VM_running | `Error` | 79.96 | test_accounts.py
test_query_async_job_result | `Error` | 86.81 | test_async_job.py
test_3d_gpu_support | `Error` | 231.79 | test_deploy_vgpu_enabled_vm.py
test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80 | `Error` | 174.10 | test_internal_lb.py
test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80 | `Error` | 176.30 | test_internal_lb.py
test_04_rvpc_internallb_haproxy_stats_on_all_interfaces | `Error` | 177.01 | test_internal_lb.py
test_04_rvpc_internallb_haproxy_stats_on_all_interfaces | `Error` | 178.13 | test_internal_lb.py
test_01_deploy_kubernetes_cluster | `Error` | 165.09 | test_kubernetes_clusters.py
test_02_deploy_kubernetes_ha_cluster | `Error` | 160.30 | test_kubernetes_clusters.py
test_04_deploy_and_upgrade_kubernetes_cluster | `Error` | 78.09 | test_kubernetes_clusters.py
test_05_deploy_and_upgrade_kubernetes_ha_cluster | `Error` | 80.11 | test_kubernetes_clusters.py
test_06_deploy_and_invalid_upgrade_kubernetes_cluster | `Error` | 82.10 | test_kubernetes_clusters.py
test_07_deploy_and_scale_kubernetes_cluster | `Error` | 68.82 | test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown | `Error` | 104.32 | test_kubernetes_clusters.py
test_nested_virtualization_vmware | `Error` | 333.31 | test_nested_virtualization.py
test_network_acl | `Error` | 319.39 | test_network_acl.py
test_delete_account | `Error` | 352.77 | test_network.py
test_delete_network_while_vm_on_it | `Error` | 23.82 | test_network.py
test_delete_network_while_vm_on_it | `Error` | 23.92 | test_network.py
test_deploy_vm_l2network | `Error` | 259.42 | test_network.py
test_deploy_vm_l2network | `Error` | 259.54 | test_network.py
test_l2network_restart | `Error` | 21.04 | test_network.py
test_l2network_restart | `Error` | 22.13 | test_network.py
ContextSuite context=TestL2Networks>:teardown | `Error` | 23.24 | test_network.py
ContextSuite context=TestPortForwarding>:setup | `Error` | 346.43 | test_network.py
ContextSuite context=TestPublicIP>:setup | `Error` | 335.34 | test_network.py
test_reboot_router | `Error` | 75.74 | test_network.py
test_releaseIP | `Error` | 319.58 | test_network.py
ContextSuite context=TestRouterRules>:setup | `Error` | 395.44 | test_network.py
test_03_nic_multiple_vmware | `Error` | 336.57 | test_nic.py
ContextSuite context=TestRouterDnsService>:setup | `Error` | 0.00 | test_router_dnsservice.py
ContextSuite context=TestIsolatedNetworksPasswdServer>:setup | `Error` | 0.00 | test_password_server.py
test_01_create_delete_portforwarding_fornonvpc | `Error` | 335.06 | test_portforwardingrules.py
test_01_add_primary_storage_disabled_host | `Error` | 0.19 | test_primary_storage.py
test_01_primary_storage_nfs | `Error` | 0.14 | test_primary_storage.py
ContextSuite context=TestStorageTags>:setup | `Error` | 0.32 | test_primary_storage.py
test_02_vpc_privategw_static_routes | `Failure` | 356.52 | test_privategw_acl.py
test_02_vpc_privategw_static_routes | `Error` | 358.81 | test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup | `Failure` | 354.41 | test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup | `Error` | 355.68 | test_privategw_acl.py
test_04_rvpc_privategw_static_routes | `Failure` | 242.44 | test_privategw_acl.py
test_04_rvpc_privategw_static_routes | `Error` | 244.69 | test_privategw_acl.py
test_09_project_suspend | `Error` | 311.94 | test_projects.py
test_10_project_activation | `Error` | 17.87 | test_projects.py
ContextSuite context=TestResetVmOnReboot>:setup | `Error` | 0.00 | test_reset_vm_on_reboot.py
test_01_so_removal_resource_update | `Error` | 307.81 | test_resource_accounting.py
ContextSuite context=TestRouterDHCPHosts>:setup | `Error` | 0.00 | test_router_dhcphosts.py
ContextSuite context=TestRouterDHCPOpts>:setup | `Error` | 0.00 | test_router_dhcphosts.py
ContextSuite context=TestRouterDns>:setup | `Error` | 0.00 | test_router_dns.py
test_02_routervm_iptables_policies | `Error` | 307.17 | test_routers_iptables_default_policy.py
test_01_single_VPC_iptables_policies | `Error` | 65.46 | test_routers_iptables_default_policy.py
test_01_single_VPC_iptables_policies | `Error` | 79.98 | test_routers_iptables_default_policy.py
test_01_isolate_network_FW_PF_default_routes_egress_true | `Error` | 304.22 | test_routers_network_ops.py
test_02_isolate_network_FW_PF_default_routes_egress_false | `Error` | 66.06 | test_routers_network_ops.py
test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true | `Error` | 142.71 | test_routers_network_ops.py
test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false | `Error` | 393.99 | test_routers_network_ops.py
test_03_RVR_Network_check_router_state | `Error` | 414.65 | test_routers_network_ops.py
ContextSuite context=TestRouterServices>:setup | `Error` | 0.00 | test_routers.py
ContextSuite context=TestServiceOfferings>:setup | `Error` | 55.29 | test_service_offerings.py
ContextSuite context=TestSnapshotRootDisk>:setup | `Error` | 0.00 | test_snapshots.py
test_05_stop_ssvm | `Failure` | 924.90 | test_ssvm.py
test_06_stop_cpvm | `Failure` | 919.77 | test_ssvm.py
ContextSuite context=TestTemplates>:setup | `Error` | 56.47 | test_templates.py
ContextSuite context=TestLBRuleUsage>:setup | `Error` | 60.22 | test_usage.py
ContextSuite context=TestNatRuleUsage>:setup | `Error` | 69.10 | test_usage.py
ContextSuite context=TestPublicIPUsage>:setup | `Error` | 78.95 | test_usage.py
ContextSuite context=TestSnapshotUsage>:setup | `Error` | 86.74 | test_usage.py
ContextSuite context=TestVmUsage>:setup | `Error` | 108.68 | test_usage.py
ContextSuite context=TestVolumeUsage>:setup | `Error` | 116.53 | test_usage.py
ContextSuite context=TestVpnUsage>:setup | `Error` | 124.31 | test_usage.py
test_02_deploy_vm_on_specific_cluster | `Error` | 7.50 | test_vm_deployment_planner.py
test_03_deploy_vm_on_specific_pod | `Error` | 6.34 | test_vm_deployment_planner.py
test_05_deploy_vm_on_cluster_override_pod | `Error` | 8.35 | test_vm_deployment_planner.py
ContextSuite context=Test01DeployVM>:setup | `Error` | 0.00 | test_vm_life_cycle.py
ContextSuite context=Test02VMLifeCycle>:setup | `Error` | 0.00 | test_vm_life_cycle.py
test_change_service_offering_for_vm_with_snapshots | `Error` | 57.70 | test_vm_snapshots.py
ContextSuite context=TestVmSnapshot>:setup | `Error` | 130.51 | test_vm_snapshots.py
ContextSuite context=TestCreateVolume>:setup | `Error` | 0.00 | test_volumes.py
ContextSuite context=TestVolumes>:setup | `Error` | 0.00 | test_volumes.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL | `Failure` | 57.04 | test_vpc_redundant.py
test_02_redundant_VPC_default_routes | `Failure` | 56.04 | test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers | `Failure` | 56.06 | test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nics | `Failure` | 57.04 | test_vpc_redundant.py
test_05_rvpc_multi_tiers | `Failure` | 53.97 | test_vpc_redundant.py
test_01_VPC_nics_after_destroy | `Failure` | 59.63 | test_vpc_router_nics.py
test_02_VPC_default_routes | `Failure` | 59.64 | test_vpc_router_nics.py
test_01_redundant_vpc_site2site_vpn | `Failure` | 59.94 | test_vpc_vpn.py
test_01_vpc_site2site_vpn_multiple_options | `Failure` | 48.65 | test_vpc_vpn.py
test_01_vpc_remote_access_vpn | `Failure` | 53.85 | test_vpc_vpn.py
test_01_vpc_site2site_vpn | `Failure` | 49.58 | test_vpc_vpn.py
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on pull request #4235: [VMware] Full OVF properties support
Posted by GitBox <gi...@apache.org>.
rhtyd commented on pull request #4235:
URL: https://github.com/apache/cloudstack/pull/4235#issuecomment-669094222
@blueorangutan package
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org