You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@cloudstack.apache.org by "Sateesh Chodapuneedi (JIRA)" <ji...@apache.org> on 2013/06/13 09:24:19 UTC
[jira] [Resolved] (CLOUDSTACK-2962) [SM][VMWare]: migrate a
virtualMachine with volumes is failing
[ https://issues.apache.org/jira/browse/CLOUDSTACK-2962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sateesh Chodapuneedi resolved CLOUDSTACK-2962.
----------------------------------------------
Resolution: Fixed
> [SM][VMWare]: migrate a virtualMachine with volumes is failing
> --------------------------------------------------------------
>
> Key: CLOUDSTACK-2962
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-2962
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the default.)
> Components: Management Server
> Affects Versions: 4.2.0
> Reporter: Srikanteswararao Talluri
> Assignee: Sateesh Chodapuneedi
> Priority: Blocker
> Fix For: 4.2.0
>
> Attachments: Archive.zip
>
>
> Have a vmware deployment with two clusters in a zone and zonewide primary storage
> 1. Now try to migrate a virtual machine with volumes.
> ===START=== 10.252.192.7 -- GET command=findHostsForMigration&VirtualMachineId=88d92066-ca9c-4ecc-b204-7c07edc117a2&response=json&sessionkey=5ez1jBXiPUJzcam0%2Ft9UtTy9L%2BY%3D&_=1371038241533
> 2013-06-12 22:52:48,366 DEBUG [cloud.storage.StorageManagerImpl] (catalina-exec-8:null) Checking pool 2 for storage, totalSize: 5902284800000, usedBytes: 3289819828224, usedPct: 0.5573807330042766, disable threshold: 0.85
> 2013-06-12 22:52:48,447 DEBUG [cloud.server.StatsCollector] (StatsCollector-1:null) VmStatsCollector is running...
> 2013-06-12 22:52:48,464 DEBUG [cloud.storage.StorageManagerImpl] (catalina-exec-8:null) Checking pool: 2 for volume allocation [Vol[14|vm=8|ROOT]], maxSize : 11804569600000, totalAllocatedSize : 2147483648, askingSize : 0, allocated disable threshold: 0.85
> 2013-06-12 22:52:48,484 DEBUG [allocator.impl.FirstFitAllocator] (catalina-exec-8:null) FirstFitAllocator has 1 hosts to check for allocation: [Host[-1-Routing]]
> 2013-06-12 22:52:48,487 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-439:null) Seq 1-715325919: Executing request
> 2013-06-12 22:52:48,507 DEBUG [allocator.impl.FirstFitAllocator] (catalina-exec-8:null) Found 1 hosts for allocation after prioritization: [Host[-1-Routing]]
> 2013-06-12 22:52:48,507 DEBUG [allocator.impl.FirstFitAllocator] (catalina-exec-8:null) Looking for speed=100Mhz, Ram=512
> 2013-06-12 22:52:48,518 DEBUG [cloud.capacity.CapacityManagerImpl] (catalina-exec-8:null) Checking if host: 1 has enough capacity for requested CPU: 100 and requested RAM: 536870912 , cpuOverprovisioningFactor: 1.0
> 2013-06-12 22:52:48,523 DEBUG [cloud.capacity.CapacityManagerImpl] (catalina-exec-8:null) Hosts's actual total CPU: 12368 and CPU after applying overprovisioning: 12368
> 2013-06-12 22:52:48,523 DEBUG [cloud.capacity.CapacityManagerImpl] (catalina-exec-8:null) considerReservedCapacity isfalse , not considering reserved capacity for calculating free capacity
> 2013-06-12 22:52:48,523 DEBUG [cloud.capacity.CapacityManagerImpl] (catalina-exec-8:null) Free CPU: 10768 , Requested CPU: 100
> 2013-06-12 22:52:48,524 DEBUG [cloud.capacity.CapacityManagerImpl] (catalina-exec-8:null) Free RAM: 6565937152 , Requested RAM: 536870912
> 2013-06-12 22:52:48,524 DEBUG [cloud.capacity.CapacityManagerImpl] (catalina-exec-8:null) Host has enough CPU and RAM available
> 2013-06-12 22:52:48,524 DEBUG [cloud.capacity.CapacityManagerImpl] (catalina-exec-8:null) STATS: Can alloc CPU from host: 1, used: 1600, reserved: 0, actual total: 12368, total with overprovisioning: 12368; requested cpu:100,alloc_from_last_host?:false ,considerReservedCapacity?: false
> 2013-06-12 22:52:48,524 DEBUG [cloud.capacity.CapacityManagerImpl] (catalina-exec-8:null) STATS: Can alloc MEM from host: 1, used: 2013265920, reserved: 0, total: 8579203072; requested mem: 536870912,alloc_from_last_host?:false ,considerReservedCapacity?: false
> 2013-06-12 22:52:48,524 DEBUG [allocator.impl.FirstFitAllocator] (catalina-exec-8:null) Found a suitable host, adding to list: 1
> 2013-06-12 22:52:48,524 DEBUG [allocator.impl.FirstFitAllocator] (catalina-exec-8:null) Host Allocator returning 1 suitable hosts
> 2013-06-12 22:52:48,524 DEBUG [cloud.server.ManagementServerImpl] (catalina-exec-8:null) Hosts having capacity and suitable for migration: [Host[-1-Routing]]
> 2013-06-12 22:52:48,561 DEBUG [cloud.api.ApiServlet] (catalina-exec-8:null) ===END=== 10.252.192.7 -- GET command=findHostsForMigration&VirtualMachineId=88d92066-ca9c-4ecc-b204-7c07edc117a2&response=json&sessionkey=5ez1jBXiPUJzcam0%2Ft9UtTy9L%2BY%3D&_=1371038241533
> 2013-06-12 22:52:48,598 DEBUG [vmware.mo.HostMO] (DirectAgent-439:10.147.40.24) find VM i-2-9-VM on host
> 2013-06-12 22:52:48,599 DEBUG [vmware.mo.HostMO] (DirectAgent-439:10.147.40.24) load VM cache on host
> 2013-06-12 22:52:48,608 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-439:null) Seq 1-715325919: Response Received:
> 2013-06-12 22:52:48,609 DEBUG [agent.transport.Request] (StatsCollector-1:null) Seq 1-715325919: Received: { Ans: , MgmtId: 7566222426160, via: 1, Ver: v1, Flags: 10, { GetVmStatsAnswer } }
> 2013-06-12 22:52:48,637 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-5:null) Seq 6-835518759: Executing request
> 2013-06-12 22:52:48,716 DEBUG [vmware.mo.HostMO] (DirectAgent-5:10.147.40.30) find VM i-2-7-VM on host
> 2013-06-12 22:52:48,717 DEBUG [vmware.mo.HostMO] (DirectAgent-5:10.147.40.30) load VM cache on host
> 2013-06-12 22:52:48,733 DEBUG [vmware.mo.HostMO] (DirectAgent-5:10.147.40.30) find VM i-2-8-VM on host
> 2013-06-12 22:52:48,733 DEBUG [vmware.mo.HostMO] (DirectAgent-5:10.147.40.30) VM i-2-8-VM found in host cache
> 2013-06-12 22:52:48,738 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-5:null) Seq 6-835518759: Response Received:
> 2013-06-12 22:52:48,738 DEBUG [agent.transport.Request] (StatsCollector-1:null) Seq 6-835518759: Received: { Ans: , MgmtId: 7566222426160, via: 6, Ver: v1, Flags: 10, { GetVmStatsAnswer } }
> 2013-06-12 22:52:48,926 DEBUG [cloud.api.ApiServlet] (catalina-exec-1:null) ===START=== 10.252.192.7 -- GET command=queryAsyncJobResult&jobId=8c775230-f312-428d-a77e-248d9f0d107c&response=json&sessionkey=5ez1jBXiPUJzcam0%2Ft9UtTy9L%2BY%3D&_=1371038242151
> 2013-06-12 22:52:48,947 DEBUG [cloud.async.AsyncJobManagerImpl] (catalina-exec-1:null) Async job-35 completed
> 2013-06-12 22:52:48,956 DEBUG [cloud.api.ApiServlet] (catalina-exec-1:null) ===END=== 10.252.192.7 -- GET command=queryAsyncJobResult&jobId=8c775230-f312-428d-a77e-248d9f0d107c&response=json&sessionkey=5ez1jBXiPUJzcam0%2Ft9UtTy9L%2BY%3D&_=1371038242151
> 2013-06-12 22:52:49,842 DEBUG [cloud.api.ApiServlet] (catalina-exec-15:null) ===START=== 10.252.192.7 -- GET command=migrateVirtualMachineWithVolume&hostid=2e452c2f-1898-4784-b971-937e357fc781&virtualmachineid=88d92066-ca9c-4ecc-b204-7c07edc117a2&response=json&sessionkey=5ez1jBXiPUJzcam0%2Ft9UtTy9L%2BY%3D&_=1371038243067
> 2013-06-12 22:52:49,892 DEBUG [cloud.async.AsyncJobManagerImpl] (catalina-exec-15:null) submit async job-51, details: AsyncJobVO {id:51, userId: 2, accountId: 2, sessionKey: null, instanceType: None, instanceId: null, cmd: org.apache.cloudstack.api.command.admin.vm.MigrateVirtualMachineWithVolumeCmd, cmdOriginator: null, cmdInfo: {"response":"json","sessionkey":"5ez1jBXiPUJzcam0/t9UtTy9L+Y\u003d","virtualmachineid":"88d92066-ca9c-4ecc-b204-7c07edc117a2","hostid":"2e452c2f-1898-4784-b971-937e357fc781","ctxUserId":"2","httpmethod":"GET","_":"1371038243067","ctxAccountId":"2","ctxStartEventId":"180"}, cmdVersion: 0, callbackType: 0, callbackAddress: null, status: 0, processStatus: 0, resultCode: 0, result: null, initMsid: 7566222426160, completeMsid: null, lastUpdated: null, lastPolled: null, created: null}
> 2013-06-12 22:52:49,896 DEBUG [cloud.api.ApiServlet] (catalina-exec-15:null) ===END=== 10.252.192.7 -- GET command=migrateVirtualMachineWithVolume&hostid=2e452c2f-1898-4784-b971-937e357fc781&virtualmachineid=88d92066-ca9c-4ecc-b204-7c07edc117a2&response=json&sessionkey=5ez1jBXiPUJzcam0%2Ft9UtTy9L%2BY%3D&_=1371038243067
> 2013-06-12 22:52:49,901 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-39:job-51) Executing org.apache.cloudstack.api.command.admin.vm.MigrateVirtualMachineWithVolumeCmd for job-51
> 2013-06-12 22:52:50,002 DEBUG [cloud.storage.StorageManagerImpl] (Job-Executor-39:job-51) Checking pool 2 for storage, totalSize: 5902284800000, usedBytes: 3289819828224, usedPct: 0.5573807330042766, disable threshold: 0.85
> 2013-06-12 22:52:50,037 DEBUG [cloud.storage.StorageManagerImpl] (Job-Executor-39:job-51) Checking pool: 2 for volume allocation [Vol[14|vm=8|ROOT]], maxSize : 11804569600000, totalAllocatedSize : 2147483648, askingSize : 0, allocated disable threshold: 0.85
> 2013-06-12 22:52:50,061 DEBUG [cloud.network.NetworkModelImpl] (Job-Executor-39:job-51) Service SecurityGroup is not supported in the network id=204
> 2013-06-12 22:52:50,069 DEBUG [cloud.storage.VolumeManagerImpl] (Job-Executor-39:job-51) Preparing 1 volumes for VM[User|prim2]
> 2013-06-12 22:52:50,091 DEBUG [hypervisor.guru.VMwareGuru] (Job-Executor-39:job-51) Nested virtualization requested, adding flag to vm configuration
> 2013-06-12 22:52:50,121 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-39:job-51) VM state transitted from :Running to Migrating with event: MigrationRequestedvm's original host id: 6 new host id: 1 host id before state transition: 6
> 2013-06-12 22:52:50,135 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-39:job-51) Hosts's actual total CPU: 12368 and CPU after applying overprovisioning: 12368
> 2013-06-12 22:52:50,136 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-39:job-51) We are allocating VM, increasing the used capacity of this host:1
> 2013-06-12 22:52:50,136 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-39:job-51) Current Used CPU: 1600 , Free CPU:10768 ,Requested CPU: 100
> 2013-06-12 22:52:50,136 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-39:job-51) Current Used RAM: 2013265920 , Free RAM:6565937152 ,Requested RAM: 536870912
> 2013-06-12 22:52:50,136 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-39:job-51) CPU STATS after allocation: for host: 1, old used: 1600, old reserved: 0, actual total: 12368, total with overprovisioning: 12368; new used:1700, reserved:0; requested cpu:100,alloc_from_last:false
> 2013-06-12 22:52:50,136 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-39:job-51) RAM STATS after allocation: for host: 1, old used: 2013265920, old reserved: 0, total: 8579203072; new used: 2550136832, reserved: 0; requested mem: 536870912,alloc_from_last:false
> 2013-06-12 22:52:50,193 DEBUG [agent.transport.Request] (Job-Executor-39:job-51) Seq 6-835518760: Sending { Cmd , MgmtId: 7566222426160, via: 6, Ver: v1, Flags: 100111, [{"MigrateWithStorageCommand":{"vm":{"id":8,"name":"i-2-8-VM","bootloader":"HVM","type":"User","cpus":1,"minSpeed":100,"maxSpeed":100,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"CentOS 5.3 (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"409dc14c63c9fb0f","params":{"rootDiskController":"ide","nicAdapter":"E1000","nestedVirtualizationFlag":"false"},"uuid":"88d92066-ca9c-4ecc-b204-7c07edc117a2","disks":[{"id":14,"name":"ROOT-8","mountPoint":"/export/home/talluri/mysetup_prim2","path":"ROOT-8-14","size":2147483648,"type":"ROOT","storagePoolType":"NetworkFilesystem","storagePoolUuid":"ece502dc-5d06-3043-afec-38e3d54fe995","deviceId":0}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"uuid":"90463686-3e63-4a3c-99f7-5ab36bd8a1a8","ip":"10.1.1.120","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:70:f1:00:05","dns1":"10.103.128.16","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://944","isolationUri":"vlan://944","isSecurityGroupEnabled":false}]},"volumeToFiler":{"Vol[14|ROOT|ROOT-8-14|2147483648]":{"id":2,"uuid":"ece502dc-5d06-3043-afec-38e3d54fe995","host":"10.147.28.7","path":"/export/home/talluri/mysetup_prim2","port":2049,"type":"NetworkFilesystem"}},"tgtHost":"HostSystem:host-7108@10.147.60.15","wait":0}}] }
> 2013-06-12 22:52:50,195 DEBUG [agent.transport.Request] (Job-Executor-39:job-51) Seq 6-835518760: Executing: { Cmd , MgmtId: 7566222426160, via: 6, Ver: v1, Flags: 100111, [{"MigrateWithStorageCommand":{"vm":{"id":8,"name":"i-2-8-VM","bootloader":"HVM","type":"User","cpus":1,"minSpeed":100,"maxSpeed":100,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"CentOS 5.3 (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"409dc14c63c9fb0f","params":{"rootDiskController":"ide","nicAdapter":"E1000","nestedVirtualizationFlag":"false"},"uuid":"88d92066-ca9c-4ecc-b204-7c07edc117a2","disks":[{"id":14,"name":"ROOT-8","mountPoint":"/export/home/talluri/mysetup_prim2","path":"ROOT-8-14","size":2147483648,"type":"ROOT","storagePoolType":"NetworkFilesystem","storagePoolUuid":"ece502dc-5d06-3043-afec-38e3d54fe995","deviceId":0}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"uuid":"90463686-3e63-4a3c-99f7-5ab36bd8a1a8","ip":"10.1.1.120","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:70:f1:00:05","dns1":"10.103.128.16","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://944","isolationUri":"vlan://944","isSecurityGroupEnabled":false}]},"volumeToFiler":{"Vol[14|ROOT|ROOT-8-14|2147483648]":{"id":2,"uuid":"ece502dc-5d06-3043-afec-38e3d54fe995","host":"10.147.28.7","path":"/export/home/talluri/mysetup_prim2","port":2049,"type":"NetworkFilesystem"}},"tgtHost":"HostSystem:host-7108@10.147.60.15","wait":0}}] }
> 2013-06-12 22:52:50,212 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-327:null) Seq 6-835518760: Executing request
> 2013-06-12 22:52:50,224 INFO [vmware.resource.VmwareResource] (DirectAgent-327:10.147.40.30) Executing resource MigrateWithStorageCommand: {"vm":{"id":8,"name":"i-2-8-VM","bootloader":"HVM","type":"User","cpus":1,"minSpeed":100,"maxSpeed":100,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"CentOS 5.3 (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"409dc14c63c9fb0f","params":{"rootDiskController":"ide","nicAdapter":"E1000","nestedVirtualizationFlag":"false"},"uuid":"88d92066-ca9c-4ecc-b204-7c07edc117a2","disks":[{"id":14,"name":"ROOT-8","mountPoint":"/export/home/talluri/mysetup_prim2","path":"ROOT-8-14","size":2147483648,"type":"ROOT","storagePoolType":"NetworkFilesystem","storagePoolUuid":"ece502dc-5d06-3043-afec-38e3d54fe995","deviceId":0}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"uuid":"90463686-3e63-4a3c-99f7-5ab36bd8a1a8","ip":"10.1.1.120","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:70:f1:00:05","dns1":"10.103.128.16","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://944","isolationUri":"vlan://944","isSecurityGroupEnabled":false}]},"volumeToFiler":{"Vol[14|ROOT|ROOT-8-14|2147483648]":{"id":2,"uuid":"ece502dc-5d06-3043-afec-38e3d54fe995","host":"10.147.28.7","path":"/export/home/talluri/mysetup_prim2","port":2049,"type":"NetworkFilesystem"}},"tgtHost":"HostSystem:host-7108@10.147.60.15","wait":0}
> 2013-06-12 22:52:50,237 WARN [vmware.resource.VmwareResource] (DirectAgent-327:10.147.40.30) MigrationCommand failed due to Exception: com.cloud.utils.exception.CloudRuntimeException
> Message: Source host & target host are in different datacentesr
> com.cloud.utils.exception.CloudRuntimeException: Source host & target host are in different datacentesr
> at com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:3494)
> at com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:418)
> at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2013-06-12 22:52:50,241 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-327:null) Seq 6-835518760: Response Received:
> 2013-06-12 22:52:50,241 DEBUG [agent.transport.Request] (DirectAgent-327:null) Seq 6-835518760: Processing: { Ans: , MgmtId: 7566222426160, via: 6, Ver: v1, Flags: 110, [{"MigrateWithStorageAnswer":{"result":false,"details":"Exception: com.cloud.utils.exception.CloudRuntimeException\nMessage: Source host & target host are in different datacentesr\nStack: com.cloud.utils.exception.CloudRuntimeException: Source host & target host are in different datacentesr\n\tat com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:3494)\n\tat com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:418)\n\tat com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)\n\tat java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:166)\n\tat java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)\n\tat java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)\n\tat java.lang.Thread.run(Thread.java:679)\n","wait":0}}] }
> 2013-06-12 22:52:50,242 DEBUG [agent.transport.Request] (Job-Executor-39:job-51) Seq 6-835518760: Received: { Ans: , MgmtId: 7566222426160, via: 6, Ver: v1, Flags: 110, { MigrateWithStorageAnswer } }
> 2013-06-12 22:52:50,242 ERROR [storage.motion.VmwareStorageMotionStrategy] (Job-Executor-39:job-51) Migration with storage of vm VM[User|prim2] failed. Details: Exception: com.cloud.utils.exception.CloudRuntimeException
> Message: Source host & target host are in different datacentesr
> Stack: com.cloud.utils.exception.CloudRuntimeException: Source host & target host are in different datacentesr
> at com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:3494)
> at com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:418)
> at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2013-06-12 22:52:50,243 ERROR [storage.motion.VmwareStorageMotionStrategy] (Job-Executor-39:job-51) copy failed
> com.cloud.utils.exception.CloudRuntimeException: Error while migrating the vm VM[User|prim2] to host Host[-1-Routing]. Exception: com.cloud.utils.exception.CloudRuntimeException
> Message: Source host & target host are in different datacentesr
> Stack: com.cloud.utils.exception.CloudRuntimeException: Source host & target host are in different datacentesr
> at com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:3494)
> at com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:418)
> at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> at org.apache.cloudstack.storage.motion.VmwareStorageMotionStrategy.migrateVmWithVolumesAcrossCluster(VmwareStorageMotionStrategy.java:147)
> at org.apache.cloudstack.storage.motion.VmwareStorageMotionStrategy.copyAsync(VmwareStorageMotionStrategy.java:106)
> at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:71)
> at org.apache.cloudstack.storage.volume.VolumeServiceImpl.migrateVolumes(VolumeServiceImpl.java:693)
> at com.cloud.storage.VolumeManagerImpl.migrateVolumes(VolumeManagerImpl.java:2171)
> at com.cloud.vm.VirtualMachineManagerImpl.migrateWithStorage(VirtualMachineManagerImpl.java:1676)
> at com.cloud.vm.UserVmManagerImpl.migrateVirtualMachineWithVolume(UserVmManagerImpl.java:4056)
> at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> at org.apache.cloudstack.api.command.admin.vm.MigrateVirtualMachineWithVolumeCmd.execute(MigrateVirtualMachineWithVolumeCmd.java:137)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:155)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:437)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2013-06-12 22:52:50,246 DEBUG [agent.manager.AgentAttache] (DirectAgent-327:null) Seq 6-835518760: No more commands found
> 2013-06-12 22:52:50,259 DEBUG [cloud.storage.VolumeManagerImpl] (Job-Executor-39:job-51) Failed to migrated vm VM[User|prim2] along with its volumes. com.cloud.utils.exception.CloudRuntimeException: Error while migrating the vm VM[User|prim2] to host Host[-1-Routing]. Exception: com.cloud.utils.exception.CloudRuntimeException
> Message: Source host & target host are in different datacentesr
> Stack: com.cloud.utils.exception.CloudRuntimeException: Source host & target host are in different datacentesr
> at com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:3494)
> at com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:418)
> at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2013-06-12 22:52:50,259 INFO [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-39:job-51) Migration was unsuccessful. Cleaning up: VM[User|prim2]
> 2013-06-12 22:52:50,260 WARN [apache.cloudstack.alerts] (Job-Executor-39:job-51) alertType:: 17 // dataCenterId:: 1 // podId:: 1 // clusterId:: null // message:: Unable to migrate vm i-2-8-VM from host 10.147.40.30 in zone zwpszone and pod zwpszone
> 2013-06-12 22:52:50,264 DEBUG [cloud.alert.AlertManagerImpl] (Job-Executor-39:job-51) Have already sent: 1 emails for alert type '17' -- skipping send email
> 2013-06-12 22:52:50,267 DEBUG [agent.manager.AgentAttache] (Job-Executor-39:job-51) Seq 1-715325920: Routed from 7566222426160
> 2013-06-12 22:52:50,267 DEBUG [agent.transport.Request] (Job-Executor-39:job-51) Seq 1-715325920: Sending { Cmd , MgmtId: 7566222426160, via: 1, Ver: v1, Flags: 100111, [{"StopCommand":{"isProxy":false,"vmName":"i-2-8-VM","wait":0}}] }
> 2013-06-12 22:52:50,268 DEBUG [agent.transport.Request] (Job-Executor-39:job-51) Seq 1-715325920: Executing: { Cmd , MgmtId: 7566222426160, via: 1, Ver: v1, Flags: 100111, [{"StopCommand":{"isProxy":false,"vmName":"i-2-8-VM","wait":0}}] }
> 2013-06-12 22:52:50,269 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-440:null) Seq 1-715325920: Executing request
> 2013-06-12 22:52:50,270 INFO [vmware.resource.VmwareResource] (DirectAgent-440:10.147.40.24) Executing resource StopCommand: {"isProxy":false,"vmName":"i-2-8-VM","wait":0}
> 2013-06-12 22:52:50,273 DEBUG [vmware.mo.HostMO] (DirectAgent-440:10.147.40.24) find VM i-2-8-VM on host
> 2013-06-12 22:52:50,273 DEBUG [vmware.mo.HostMO] (DirectAgent-440:10.147.40.24) load VM cache on host
> 2013-06-12 22:52:50,291 INFO [vmware.resource.VmwareResource] (DirectAgent-440:10.147.40.24) VM i-2-8-VM is no longer in vSphere
> 2013-06-12 22:52:50,292 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-440:null) Seq 1-715325920: Response Received:
> 2013-06-12 22:52:50,293 DEBUG [agent.transport.Request] (DirectAgent-440:null) Seq 1-715325920: Processing: { Ans: , MgmtId: 7566222426160, via: 1, Ver: v1, Flags: 110, [{"StopAnswer":{"vncPort":0,"result":true,"details":"VM i-2-8-VM is no longer in vSphere","wait":0}}] }
> 2013-06-12 22:52:50,294 DEBUG [agent.manager.AgentAttache] (DirectAgent-440:null) Seq 1-715325920: Unable to find listener.
> 2013-06-12 22:52:50,294 DEBUG [agent.manager.AgentAttache] (DirectAgent-440:null) Seq 1-715325920: No more commands found
> 2013-06-12 22:52:50,297 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-39:job-51) VM state transitted from :Migrating to Running with event: OperationFailedvm's original host id: 6 new host id: 6 host id before state transition: 1
> 2013-06-12 22:52:50,310 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-39:job-51) Hosts's actual total CPU: 12368 and CPU after applying overprovisioning: 12368
> 2013-06-12 22:52:50,310 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-39:job-51) Hosts's actual total RAM: 8579203072 and RAM after applying overprovisioning: 8579203072
> 2013-06-12 22:52:50,310 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-39:job-51) release cpu from host: 1, old used: 1700,reserved: 0, actual total: 12368, total with overprovisioning: 12368; new used: 1600,reserved:0; movedfromreserved: false,moveToReserveredfalse
> 2013-06-12 22:52:50,311 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-39:job-51) release mem from host: 1, old used: 2550136832,reserved: 0, total: 8579203072; new used: 2013265920,reserved:0; movedfromreserved: false,moveToReserveredfalse
> 2013-06-12 22:52:50,336 ERROR [cloud.async.AsyncJobManagerImpl] (Job-Executor-39:job-51) Unexpected exception while executing org.apache.cloudstack.api.command.admin.vm.MigrateVirtualMachineWithVolumeCmd
> com.cloud.utils.exception.CloudRuntimeException: Failed to migrated vm VM[User|prim2] along with its volumes. com.cloud.utils.exception.CloudRuntimeException: Error while migrating the vm VM[User|prim2] to host Host[-1-Routing]. Exception: com.cloud.utils.exception.CloudRuntimeException
> Message: Source host & target host are in different datacentesr
> Stack: com.cloud.utils.exception.CloudRuntimeException: Source host & target host are in different datacentesr
> at com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:3494)
> at com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:418)
> at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> at com.cloud.storage.VolumeManagerImpl.migrateVolumes(VolumeManagerImpl.java:2176)
> at com.cloud.vm.VirtualMachineManagerImpl.migrateWithStorage(VirtualMachineManagerImpl.java:1676)
> at com.cloud.vm.UserVmManagerImpl.migrateVirtualMachineWithVolume(UserVmManagerImpl.java:4056)
> at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> at org.apache.cloudstack.api.command.admin.vm.MigrateVirtualMachineWithVolumeCmd.execute(MigrateVirtualMachineWithVolumeCmd.java:137)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:155)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:437)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2013-06-12 22:52:50,338 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-39:job-51) Complete async job-51, jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to migrated vm VM[User|prim2] along with its volumes. com.cloud.utils.exception.CloudRuntimeException: Error while migrating the vm VM[User|prim2] to host Host[-1-Routing]. Exception: com.cloud.utils.exception.CloudRuntimeException
> Message: Source host & target host are in different datacentesr
> Stack: com.cloud.utils.exception.CloudRuntimeException: Source host & target host are in different datacentesr
> at com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:3494)
> at com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:418)
> at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2013-06-12 22:52:51,583 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-15:null) SeqA 3-882: Processing Seq 3-882: { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11, [{"Con
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira