You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@cloudstack.apache.org by "Sanjay Tripathi (JIRA)" <ji...@apache.org> on 2014/11/04 13:26:34 UTC
[jira] [Resolved] (CLOUDSTACK-7372) [vGPU] When a host is put in
maintenance mode, vGPU enabled VMs failed to migrate to the other host in
the cluster
[ https://issues.apache.org/jira/browse/CLOUDSTACK-7372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sanjay Tripathi resolved CLOUDSTACK-7372.
-----------------------------------------
Resolution: Fixed
> [vGPU] When a host is put in maintenance mode, vGPU enabled VMs failed to migrate to the other host in the cluster
> ------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-7372
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-7372
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the default.)
> Components: Management Server
> Affects Versions: 4.5.0
> Environment: Xenserver 6.2.5 + PV drivers
> CS - 4.5.0
> Reporter: Abhinav Roy
> Assignee: Sanjay Tripathi
> Priority: Critical
> Fix For: 4.5.0
>
>
> Steps :
> ==========================
> 1. Deploy an advanced zone CS setup with 2 XEN hosts in a cluster having same configuration and vGPU cards.
> 2. Create 2 VMs with vGPU offering on Host1
> 3. Put Host1 in maintenance mode
> Expected behavior:
> =========================
> VMs should migrate to Host 2 and Host1 should go to "maintenance" state
> Observed behavior :
> =========================
> VMs fail to migrate to Host2 with the following error and Host1 is stuck in "prepareformaintenance" state.
> 2014-08-18 19:16:00,608 DEBUG [c.c.a.ApiServlet] (catalina-exec-1:ctx-a55c13e9) ===START=== 10.144.7.6 -- GET command=prepareHostForMaintenance&id=f68de59b-3d28-452f-b82b-86fca83f6f16&response=json&sessionkey=XUBDEpbw7%2BbvhRUGB0j2dKXFW08%3D&_=1408369336934
> 2014-08-18 19:16:00,651 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (catalina-exec-1:ctx-a55c13e9 ctx-5208dce9) submit async job-690, details: AsyncJobVO {id:690, userId: 2, accountId: 2, instanceType: Host, instanceId: 8, cmd: org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd, cmdInfo: {"id":"f68de59b-3d28-452f-b82b-86fca83f6f16","response":"json","sessionkey":"XUBDEpbw7+bvhRUGB0j2dKXFW08\u003d","ctxDetails":"{\"com.cloud.host.Host\":\"f68de59b-3d28-452f-b82b-86fca83f6f16\"}","cmdEventType":"MAINT.PREPARE","ctxUserId":"2","httpmethod":"GET","_":"1408369336934","uuid":"f68de59b-3d28-452f-b82b-86fca83f6f16","ctxAccountId":"2","ctxStartEventId":"983"}, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 213737702773493, completeMsid: null, lastUpdated: null, lastPolled: null, created: null}
> 2014-08-18 19:16:00,653 DEBUG [c.c.a.ApiServlet] (catalina-exec-1:ctx-a55c13e9 ctx-5208dce9) ===END=== 10.144.7.6 -- GET command=prepareHostForMaintenance&id=f68de59b-3d28-452f-b82b-86fca83f6f16&response=json&sessionkey=XUBDEpbw7%2BbvhRUGB0j2dKXFW08%3D&_=1408369336934
> 2014-08-18 19:16:00,654 INFO [o.a.c.f.j.i.AsyncJobMonitor] (API-Job-Executor-3:ctx-5d223414 job-690) Add job-690 into job monitoring
> 2014-08-18 19:16:00,655 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-3:ctx-5d223414 job-690) Executing AsyncJobVO {id:690, userId: 2, accountId: 2, instanceType: Host, instanceId: 8, cmd: org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd, cmdInfo: {"id":"f68de59b-3d28-452f-b82b-86fca83f6f16","response":"json","sessionkey":"XUBDEpbw7+bvhRUGB0j2dKXFW08\u003d","ctxDetails":"{\"com.cloud.host.Host\":\"f68de59b-3d28-452f-b82b-86fca83f6f16\"}","cmdEventType":"MAINT.PREPARE","ctxUserId":"2","httpmethod":"GET","_":"1408369336934","uuid":"f68de59b-3d28-452f-b82b-86fca83f6f16","ctxAccountId":"2","ctxStartEventId":"983"}, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 213737702773493, completeMsid: null, lastUpdated: null, lastPolled: null, created: null}
> 2014-08-18 19:16:00,683 DEBUG [c.c.a.t.Request] (API-Job-Executor-3:ctx-5d223414 job-690 ctx-4be20e99) Seq 8-5623307084725485586: Sending { Cmd , MgmtId: 213737702773493, via: 8(cldstk-R720-66), Ver: v1, Flags: 100111, [{"com.cloud.agent.api.MaintainCommand":{"wait":0}}] }
> 2014-08-18 19:16:00,684 DEBUG [c.c.a.t.Request] (API-Job-Executor-3:ctx-5d223414 job-690 ctx-4be20e99) Seq 8-5623307084725485586: Executing: { Cmd , MgmtId: 213737702773493, via: 8(cldstk-R720-66), Ver: v1, Flags: 100111, [{"com.cloud.agent.api.MaintainCommand":{"wait":0}}] }
> 2014-08-18 19:16:00,684 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-30:ctx-d6521262) Seq 8-5623307084725485586: Executing request
> 2014-08-18 19:16:00,713 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-30:ctx-d6521262) Seq 8-5623307084725485586: Response Received:
> 2014-08-18 19:16:00,713 DEBUG [c.c.a.t.Request] (DirectAgent-30:ctx-d6521262) Seq 8-5623307084725485586: Processing: { Ans: , MgmtId: 213737702773493, via: 8, Ver: v1, Flags: 110, [{"com.cloud.agent.api.MaintainAnswer":{"willMigrate":true,"result":true,"wait":0}}] }
> 2014-08-18 19:16:00,713 DEBUG [c.c.a.t.Request] (API-Job-Executor-3:ctx-5d223414 job-690 ctx-4be20e99) Seq 8-5623307084725485586: Received: { Ans: , MgmtId: 213737702773493, via: 8, Ver: v1, Flags: 110, { MaintainAnswer } }
> 2014-08-18 19:16:00,717 DEBUG [c.c.a.m.AgentAttache] (DirectAgent-30:ctx-d6521262) Seq 8-5623307084725485586: No more commands found
> 2014-08-18 19:16:00,725 DEBUG [c.c.r.ResourceState] (API-Job-Executor-3:ctx-5d223414 job-690 ctx-4be20e99) Resource state update: [id = 8; name = cldstk-R720-66; old state = Enabled; event = AdminAskMaintenace; new state = PrepareForMaintenance]
> 2014-08-18 19:16:00,725 DEBUG [c.c.a.m.AgentAttache] (API-Job-Executor-3:ctx-5d223414 job-690 ctx-4be20e99) Seq 8-5623307084725485570: Sending disconnect to class com.cloud.network.security.SecurityGroupListener
> 2014-08-18 19:16:00,750 INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-0:ctx-258603ac work-6) Processing HAWork[6-Migration-44-Running-Scheduled]
> 2014-08-18 19:16:00,758 WARN [o.a.c.f.j.AsyncJobExecutionContext] (HA-Worker-0:ctx-258603ac work-6) Job is executed without a context, setup psudo job for the executing thread
> 2014-08-18 19:16:00,769 INFO [c.c.h.HighAvailabilityManagerImpl] (HA-Worker-1:ctx-aab890b8 work-7) Processing HAWork[7-Migration-45-Running-Scheduled]
> 2014-08-18 19:16:00,780 WARN [o.a.c.f.j.AsyncJobExecutionContext] (HA-Worker-1:ctx-aab890b8 work-7) Job is executed without a context, setup psudo job for the executing thread
> 2014-08-18 19:16:00,785 DEBUG [c.c.r.ResourceManagerImpl] (API-Job-Executor-3:ctx-5d223414 job-690 ctx-4be20e99) Sent resource event EVENT_PREPARE_MAINTENANCE_AFTER to listener CapacityManagerImpl
> 2014-08-18 19:16:00,834 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (HA-Worker-0:ctx-258603ac work-6) Sync job-693 execution on object VmWorkJobQueue.44
> 2014-08-18 19:16:00,837 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (HA-Worker-1:ctx-aab890b8 work-7) Sync job-694 execution on object VmWorkJobQueue.45
> 2014-08-18 19:16:00,839 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-3:ctx-5d223414 job-690 ctx-4be20e99) Complete async job-690, jobStatus: SUCCEEDED, resultCode: 0, result: org.apache.cloudstack.api.response.HostResponse/host/{"id":"f68de59b-3d28-452f-b82b-86fca83f6f16","name":"cldstk-R720-66","state":"Up","disconnected":"2014-08-18T19:11:52+0530","type":"Routing","ipaddress":"10.102.192.66","zoneid":"5c4147e0-8545-454e-8761-191d90fded70","zonename":"vgpu","podid":"25c2e27b-b43b-4fba-a775-304e6dc5ac02","podname":"vgpu","version":"4.5.0-SNAPSHOT","hypervisor":"XenServer","cpusockets":2,"cpunumber":24,"cpuspeed":2300,"cpuallocated":"0%","cpuused":"0%","cpuwithoverprovisioning":"55200.0","networkkbsread":0,"networkkbswrite":0,"memorytotal":15719168512,"memoryallocated":0,"memoryused":0,"gpugroup":[{"gpugroupname":"Group of NVIDIA Corporation GK104GL [GRID K2] GPUs","vgpu":[{"vgputype":"GRID K220Q","videoram":503316480,"maxheads":2,"maxresolutionx":2560,"maxresolutiony":1600,"maxvgpuperpgpu":8,"remainingcapacity":0,"maxcapacity":16},{"vgputype":"GRID K200","videoram":268435456,"maxheads":2,"maxresolutionx":1920,"maxresolutiony":1200,"maxvgpuperpgpu":8,"remainingcapacity":7,"maxcapacity":16},{"vgputype":"passthrough","videoram":0,"maxheads":0,"maxresolutionx":0,"maxresolutiony":0,"maxvgpuperpgpu":1,"remainingcapacity":0,"maxcapacity":2},{"vgputype":"GRID K260Q","videoram":2013265920,"maxheads":4,"maxresolutionx":2560,"maxresolutiony":1600,"maxvgpuperpgpu":2,"remainingcapacity":0,"maxcapacity":4},{"vgputype":"GRID K240Q","videoram":1006632960,"maxheads":2,"maxresolutionx":2560,"maxresolutiony":1600,"maxvgpuperpgpu":4,"remainingcapacity":3,"maxcapacity":8}]}],"capabilities":"xen-3.0-x86_64 , xen-3.0-x86_32p , hvm-3.0-x86_32 , hvm-3.0-x86_32p , hvm-3.0-x86_64","lastpinged":"1970-01-17T03:32:40+0530","managementserverid":213737702773493,"clusterid":"d9404bb3-55f7-4e33-acb8-ba6a447e84d3","clustername":"vgpu1","clustertype":"CloudManaged","islocalstorageactive":false,"created":"2014-08-18T18:33:02+0530","events":"AgentConnected; HostDown; PingTimeout; ManagementServerDown; ShutdownRequested; Remove; StartAgentRebalance; Ping; AgentDisconnected","resourcestate":"PrepareForMaintenance","hypervisorversion":"6.2.0","hahost":false,"jobid":"94713beb-05bc-4dee-becc-c4ea303e86d0","jobstatus":0}
> 2014-08-18 19:16:00,858 WARN [c.c.u.d.Merovingian2] (HA-Worker-1:ctx-aab890b8 work-7) Was unable to find lock for the key vm_instance45 and thread id 1183613611
> 2014-08-18 19:16:00,858 WARN [c.c.u.d.Merovingian2] (HA-Worker-0:ctx-258603ac work-6) Was unable to find lock for the key vm_instance44 and thread id 741663394
> 2014-08-18 19:16:00,873 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-3:ctx-5d223414 job-690) Done executing org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd for job-690
> 2014-08-18 19:16:00,885 INFO [o.a.c.f.j.i.AsyncJobMonitor] (API-Job-Executor-3:ctx-5d223414 job-690) Remove job-690 from job monitoring
> 2014-08-18 19:16:01,259 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-d7897592) Execute sync-queue item: SyncQueueItemVO {id:285, queueId: 261, contentType: AsyncJob, contentId: 693, lastProcessMsid: null, lastprocessNumber: null, lastProcessTime: null, created: Mon Aug 18 19:16:00 IST 2014}
> 2014-08-18 19:16:01,261 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-d7897592) Schedule queued job-693
> 2014-08-18 19:16:01,264 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-d7897592) Execute sync-queue item: SyncQueueItemVO {id:286, queueId: 262, contentType: AsyncJob, contentId: 694, lastProcessMsid: null, lastprocessNumber: null, lastProcessTime: null, created: Mon Aug 18 19:16:00 IST 2014}
> 2014-08-18 19:16:01,266 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-d7897592) Schedule queued job-694
> 2014-08-18 19:16:01,268 INFO [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693) Add job-693 into job monitoring
> 2014-08-18 19:16:01,269 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693) Executing AsyncJobVO {id:693, userId: 1, accountId: 1, instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkMigrateAway, cmdInfo: rO0ABXNyAB5jb20uY2xvdWQudm0uVm1Xb3JrTWlncmF0ZUF3YXmt4MX4jtcEmwIAAUoACXNyY0hvc3RJZHhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAALHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAACA, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 213737702773493, completeMsid: null, lastUpdated: null, lastPolled: null, created: Mon Aug 18 19:16:00 IST 2014}
> 2014-08-18 19:16:01,270 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693) Run VM work job: com.cloud.vm.VmWorkMigrateAway for VM 44, job origin: 691
> 2014-08-18 19:16:01,272 DEBUG [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Execute VM work job: com.cloud.vm.VmWorkMigrateAway{"srcHostId":8,"userId":1,"accountId":1,"vmId":44,"handlerName":"VirtualMachineManagerImpl"}
> 2014-08-18 19:16:01,276 INFO [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694) Add job-694 into job monitoring
> 2014-08-18 19:16:01,278 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694) Executing AsyncJobVO {id:694, userId: 1, accountId: 1, instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkMigrateAway, cmdInfo: rO0ABXNyAB5jb20uY2xvdWQudm0uVm1Xb3JrTWlncmF0ZUF3YXmt4MX4jtcEmwIAAUoACXNyY0hvc3RJZHhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAALXQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAACA, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 213737702773493, completeMsid: null, lastUpdated: null, lastPolled: null, created: Mon Aug 18 19:16:00 IST 2014}
> 2014-08-18 19:16:01,279 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694) Run VM work job: com.cloud.vm.VmWorkMigrateAway for VM 45, job origin: 692
> 2014-08-18 19:16:01,282 DEBUG [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Execute VM work job: com.cloud.vm.VmWorkMigrateAway{"srcHostId":8,"userId":1,"accountId":1,"vmId":45,"handlerName":"VirtualMachineManagerImpl"}
> 2014-08-18 19:16:01,323 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Deploy avoids pods: [], clusters: [], hosts: [8]
> 2014-08-18 19:16:01,323 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Deploy avoids pods: [], clusters: [], hosts: [8]
> 2014-08-18 19:16:01,326 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) DeploymentPlanner allocation algorithm: com.cloud.deploy.FirstFitPlanner@37eb3935
> 2014-08-18 19:16:01,326 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Trying to allocate a host and storage pools from dc:1, pod:1,cluster:1, requested cpu: 3200, requested ram: 3221225472
> 2014-08-18 19:16:01,327 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Is ROOT volume READY (pool already allocated)?: Yes
> 2014-08-18 19:16:01,327 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) This VM has last host_id specified, trying to choose the same host: 8
> 2014-08-18 19:16:01,326 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) DeploymentPlanner allocation algorithm: com.cloud.deploy.FirstFitPlanner@37eb3935
> 2014-08-18 19:16:01,328 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Trying to allocate a host and storage pools from dc:1, pod:1,cluster:1, requested cpu: 3200, requested ram: 3221225472
> 2014-08-18 19:16:01,328 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Is ROOT volume READY (pool already allocated)?: Yes
> 2014-08-18 19:16:01,328 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) This VM has last host_id specified, trying to choose the same host: 8
> 2014-08-18 19:16:01,330 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) The last host of this VM is in avoid set
> 2014-08-18 19:16:01,332 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Cannot choose the last host to deploy this VM
> 2014-08-18 19:16:01,332 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Searching resources only under specified Cluster: 1
> 2014-08-18 19:16:01,332 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) The last host of this VM is in avoid set
> 2014-08-18 19:16:01,333 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Cannot choose the last host to deploy this VM
> 2014-08-18 19:16:01,333 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Searching resources only under specified Cluster: 1
> 2014-08-18 19:16:01,351 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Checking resources in Cluster: 1 under Pod: 1
> 2014-08-18 19:16:01,351 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Checking resources in Cluster: 1 under Pod: 1
> 2014-08-18 19:16:01,357 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d FirstFitRoutingAllocator) Looking for hosts in dc: 1 pod:1 cluster:1
> 2014-08-18 19:16:01,357 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a FirstFitRoutingAllocator) Looking for hosts in dc: 1 pod:1 cluster:1
> 2014-08-18 19:16:01,366 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d FirstFitRoutingAllocator) FirstFitAllocator has 2 hosts to check for allocation: [Host[-5-Routing], Host[-1-Routing]]
> 2014-08-18 19:16:01,368 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a FirstFitRoutingAllocator) FirstFitAllocator has 2 hosts to check for allocation: [Host[-5-Routing], Host[-1-Routing]]
> 2014-08-18 19:16:01,383 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d FirstFitRoutingAllocator) Found 2 hosts for allocation after prioritization: [Host[-5-Routing], Host[-1-Routing]]
> 2014-08-18 19:16:01,383 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d FirstFitRoutingAllocator) Looking for speed=3200Mhz, Ram=3072
> 2014-08-18 19:16:01,383 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a FirstFitRoutingAllocator) Found 2 hosts for allocation after prioritization: [Host[-5-Routing], Host[-1-Routing]]
> 2014-08-18 19:16:01,384 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a FirstFitRoutingAllocator) Looking for speed=3200Mhz, Ram=3072
> 2014-08-18 19:16:01,399 DEBUG [c.c.r.ResourceManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a FirstFitRoutingAllocator) Host ID: 5 does not have GPU device available
> 2014-08-18 19:16:01,399 INFO [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a FirstFitRoutingAllocator) Host name: cldstk-R720-55, hostId: 5 does not have required GPU devices available
> 2014-08-18 19:16:01,399 DEBUG [c.c.r.ResourceManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d FirstFitRoutingAllocator) Host ID: 5 does not have GPU device available
> 2014-08-18 19:16:01,400 INFO [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d FirstFitRoutingAllocator) Host name: cldstk-R720-55, hostId: 5 does not have required GPU devices available
> 2014-08-18 19:16:01,423 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d FirstFitRoutingAllocator) Host: 1 has cpu capability (cpu:24, speed:2300) to support requested CPU: 2 and requested speed: 1600
> 2014-08-18 19:16:01,423 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d FirstFitRoutingAllocator) Checking if host: 1 has enough capacity for requested CPU: 3200 and requested RAM: 3221225472 , cpuOverprovisioningFactor: 1.0
> 2014-08-18 19:16:01,423 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a FirstFitRoutingAllocator) Host: 1 has cpu capability (cpu:24, speed:2300) to support requested CPU: 2 and requested speed: 1600
> 2014-08-18 19:16:01,424 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a FirstFitRoutingAllocator) Checking if host: 1 has enough capacity for requested CPU: 3200 and requested RAM: 3221225472 , cpuOverprovisioningFactor: 1.0
> 2014-08-18 19:16:01,430 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a FirstFitRoutingAllocator) Hosts's actual total CPU: 55200 and CPU after applying overprovisioning: 55200
> 2014-08-18 19:16:01,430 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a FirstFitRoutingAllocator) Free CPU: 53200 , Requested CPU: 3200
> 2014-08-18 19:16:01,430 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a FirstFitRoutingAllocator) Free RAM: 13840119808 , Requested RAM: 3221225472
> 2014-08-18 19:16:01,431 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a FirstFitRoutingAllocator) Host has enough CPU and RAM available
> 2014-08-18 19:16:01,431 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a FirstFitRoutingAllocator) STATS: Can alloc CPU from host: 1, used: 2000, reserved: 0, actual total: 55200, total with overprovisioning: 55200; requested cpu:3200,alloc_from_last_host?:false ,considerReservedCapacity?: true
> 2014-08-18 19:16:01,431 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a FirstFitRoutingAllocator) STATS: Can alloc MEM from host: 1, used: 1879048192, reserved: 0, total: 15719168000; requested mem: 3221225472,alloc_from_last_host?:false ,considerReservedCapacity?: true
> 2014-08-18 19:16:01,431 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a FirstFitRoutingAllocator) Found a suitable host, adding to list: 1
> 2014-08-18 19:16:01,431 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a FirstFitRoutingAllocator) Host Allocator returning 1 suitable hosts
> 2014-08-18 19:16:01,430 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d FirstFitRoutingAllocator) Hosts's actual total CPU: 55200 and CPU after applying overprovisioning: 55200
> 2014-08-18 19:16:01,432 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d FirstFitRoutingAllocator) Free CPU: 53200 , Requested CPU: 3200
> 2014-08-18 19:16:01,432 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d FirstFitRoutingAllocator) Free RAM: 13840119808 , Requested RAM: 3221225472
> 2014-08-18 19:16:01,432 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d FirstFitRoutingAllocator) Host has enough CPU and RAM available
> 2014-08-18 19:16:01,432 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d FirstFitRoutingAllocator) STATS: Can alloc CPU from host: 1, used: 2000, reserved: 0, actual total: 55200, total with overprovisioning: 55200; requested cpu:3200,alloc_from_last_host?:false ,considerReservedCapacity?: true
> 2014-08-18 19:16:01,432 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d FirstFitRoutingAllocator) STATS: Can alloc MEM from host: 1, used: 1879048192, reserved: 0, total: 15719168000; requested mem: 3221225472,alloc_from_last_host?:false ,considerReservedCapacity?: true
> 2014-08-18 19:16:01,432 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d FirstFitRoutingAllocator) Found a suitable host, adding to list: 1
> 2014-08-18 19:16:01,432 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d FirstFitRoutingAllocator) Host Allocator returning 1 suitable hosts
> 2014-08-18 19:16:01,439 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Checking suitable pools for volume (Id, Type): (60,ROOT)
> 2014-08-18 19:16:01,441 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Volume has pool already allocated, checking if pool can be reused, poolId: 1
> 2014-08-18 19:16:01,441 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Checking suitable pools for volume (Id, Type): (59,ROOT)
> 2014-08-18 19:16:01,442 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Volume has pool already allocated, checking if pool can be reused, poolId: 1
> 2014-08-18 19:16:01,443 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Planner need not allocate a pool for this volume since its READY
> 2014-08-18 19:16:01,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Trying to find a potenial host and associated storage pools from the suitable host/pool lists for this VM
> 2014-08-18 19:16:01,444 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Planner need not allocate a pool for this volume since its READY
> 2014-08-18 19:16:01,446 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Trying to find a potenial host and associated storage pools from the suitable host/pool lists for this VM
> 2014-08-18 19:16:01,446 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Checking if host: 1 can access any suitable storage pool for volume: ROOT
> 2014-08-18 19:16:01,446 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Checking if host: 1 can access any suitable storage pool for volume: ROOT
> 2014-08-18 19:16:01,452 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Host: 1 can access pool: 1
> 2014-08-18 19:16:01,455 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Host: 1 can access pool: 1
> 2014-08-18 19:16:01,455 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Found a potential host id: 1 name: cldstk-R720-10 and associated storage pools for this VM
> 2014-08-18 19:16:01,460 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Found a potential host id: 1 name: cldstk-R720-10 and associated storage pools for this VM
> 2014-08-18 19:16:01,461 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Returning Deployment Destination: Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))] : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
> 2014-08-18 19:16:01,463 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Found destination Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))] : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()] for migrating to.
> 2014-08-18 19:16:01,463 INFO [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Migrating VM[User|i-3-44-VM] to Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))] : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
> 2014-08-18 19:16:01,463 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Returning Deployment Destination: Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))] : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
> 2014-08-18 19:16:01,464 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Found destination Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))] : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()] for migrating to.
> 2014-08-18 19:16:01,464 INFO [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Migrating VM[User|i-3-45-VM] to Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))] : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
> 2014-08-18 19:16:01,503 DEBUG [c.c.n.NetworkModelImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Service SecurityGroup is not supported in the network id=204
> 2014-08-18 19:16:01,504 DEBUG [c.c.n.NetworkModelImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Service SecurityGroup is not supported in the network id=204
> 2014-08-18 19:16:01,592 DEBUG [c.c.n.NetworkModelImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Service SecurityGroup is not supported in the network id=204
> 2014-08-18 19:16:01,593 DEBUG [c.c.n.NetworkModelImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Service SecurityGroup is not supported in the network id=204
> 2014-08-18 19:16:01,614 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Preparing 1 volumes for VM[User|i-3-45-VM]
> 2014-08-18 19:16:01,617 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Preparing 1 volumes for VM[User|i-3-44-VM]
> 2014-08-18 19:16:01,792 DEBUG [c.c.a.t.Request] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Seq 1-7894810146780479507: Sending { Cmd , MgmtId: 213737702773493, via: 1(cldstk-R720-10), Ver: v1, Flags: 100111, [{"com.cloud.agent.api.PrepareForMigrationCommand":{"vm":{"id":45,"name":"i-3-45-VM","bootloader":"PyGrub","type":"User","cpus":2,"minSpeed":1600,"maxSpeed":1600,"minRam":3221225472,"maxRam":3221225472,"arch":"x86_64","os":"Windows 8 (64-bit)","platformEmulator":"Windows 8 (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"fJUipR1PM6PLvPSTXWLuKg==","params":{"platform":"vgpu_config:/usr/share/nvidia/vgx/grid_k240q.conf;viridian:true;acpi:1;apic:true;pae:true;vgpu_pci_id:0000:07:00.0;videoram:8;device_id:0002;nx:true;timeoffset:-1;vga:std","hypervisortoolsversion":"xenserver61","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"3aa5d887-38e0-40e4-9c11-d3f27dd80f62","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"ac0b242e-d2cc-40ca-9e1a-4e88c0976b39","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e3feb450-3c8e-380f-bbf5-4e8e2424b5f9","id":1,"poolType":"NetworkFilesystem","host":"10.102.192.101","path":"/Abhinav/vgpu-ps1","port":2049,"url":"NetworkFilesystem://10.102.192.101/Abhinav/vgpu-ps1/?ROLE=Primary&STOREUUID=e3feb450-3c8e-380f-bbf5-4e8e2424b5f9"}},"name":"ROOT-45","size":21474836480,"path":"8343dcb8-fa69-479a-9c6d-5d7d28270b8e","volumeId":60,"vmName":"i-3-45-VM","accountId":3,"format":"VHD","provisioningType":"THIN","id":60,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"8343dcb8-fa69-479a-9c6d-5d7d28270b8e","type":"ROOT","_details":{"managed":"false","storagePort":"2049","storageHost":"10.102.192.101","volumeSize":"21474836480"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"871a3594-2404-4501-bf40-b043240bc737","uuid":"47535f05-4cde-4db0-b899-fcfebd1857c1","ip":"10.1.1.84","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:0d:e0:00:15","dns1":"10.140.50.5","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://700","isolationUri":"vlan://700","isSecurityGroupEnabled":false}],"gpuDevice":{"gpuGroup":"Group of NVIDIA Corporation GK104GL [GRID K2] GPUs","vgpuType":"GRID K240Q"}},"wait":0}}] }
> 2014-08-18 19:16:01,800 DEBUG [c.c.a.t.Request] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Seq 1-7894810146780479507: Executing: { Cmd , MgmtId: 213737702773493, via: 1(cldstk-R720-10), Ver: v1, Flags: 100111, [{"com.cloud.agent.api.PrepareForMigrationCommand":{"vm":{"id":45,"name":"i-3-45-VM","bootloader":"PyGrub","type":"User","cpus":2,"minSpeed":1600,"maxSpeed":1600,"minRam":3221225472,"maxRam":3221225472,"arch":"x86_64","os":"Windows 8 (64-bit)","platformEmulator":"Windows 8 (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"fJUipR1PM6PLvPSTXWLuKg==","params":{"platform":"vgpu_config:/usr/share/nvidia/vgx/grid_k240q.conf;viridian:true;acpi:1;apic:true;pae:true;vgpu_pci_id:0000:07:00.0;videoram:8;device_id:0002;nx:true;timeoffset:-1;vga:std","hypervisortoolsversion":"xenserver61","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"3aa5d887-38e0-40e4-9c11-d3f27dd80f62","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"ac0b242e-d2cc-40ca-9e1a-4e88c0976b39","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e3feb450-3c8e-380f-bbf5-4e8e2424b5f9","id":1,"poolType":"NetworkFilesystem","host":"10.102.192.101","path":"/Abhinav/vgpu-ps1","port":2049,"url":"NetworkFilesystem://10.102.192.101/Abhinav/vgpu-ps1/?ROLE=Primary&STOREUUID=e3feb450-3c8e-380f-bbf5-4e8e2424b5f9"}},"name":"ROOT-45","size":21474836480,"path":"8343dcb8-fa69-479a-9c6d-5d7d28270b8e","volumeId":60,"vmName":"i-3-45-VM","accountId":3,"format":"VHD","provisioningType":"THIN","id":60,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"8343dcb8-fa69-479a-9c6d-5d7d28270b8e","type":"ROOT","_details":{"managed":"false","storagePort":"2049","storageHost":"10.102.192.101","volumeSize":"21474836480"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"871a3594-2404-4501-bf40-b043240bc737","uuid":"47535f05-4cde-4db0-b899-fcfebd1857c1","ip":"10.1.1.84","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:0d:e0:00:15","dns1":"10.140.50.5","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://700","isolationUri":"vlan://700","isSecurityGroupEnabled":false}],"gpuDevice":{"gpuGroup":"Group of NVIDIA Corporation GK104GL [GRID K2] GPUs","vgpuType":"GRID K240Q"}},"wait":0}}] }
> 2014-08-18 19:16:01,805 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-31:ctx-f3087b9e) Seq 1-7894810146780479507: Executing request
> 2014-08-18 19:16:01,813 DEBUG [c.c.a.t.Request] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Seq 1-7894810146780479508: Waiting for Seq 7894810146780479507 Scheduling: { Cmd , MgmtId: 213737702773493, via: 1(cldstk-R720-10), Ver: v1, Flags: 100111, [{"com.cloud.agent.api.PrepareForMigrationCommand":{"vm":{"id":44,"name":"i-3-44-VM","bootloader":"PyGrub","type":"User","cpus":2,"minSpeed":1600,"maxSpeed":1600,"minRam":3221225472,"maxRam":3221225472,"arch":"x86_64","os":"Windows 8 (64-bit)","platformEmulator":"Windows 8 (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"dZkRpXoT4v5TCEO0OP5G9A==","params":{"platform":"vgpu_config:/usr/share/nvidia/vgx/grid_k200.conf;viridian:true;acpi:1;apic:true;pae:true;vgpu_pci_id:0000:08:00.0;videoram:8;device_id:0002;nx:true;timeoffset:-1;vga:std","hypervisortoolsversion":"xenserver61","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"13d4c38e-74e7-43b2-861f-bbb4ad5677b0","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"2bffd799-c085-4aa6-b151-fd8c3a8c36be","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e3feb450-3c8e-380f-bbf5-4e8e2424b5f9","id":1,"poolType":"NetworkFilesystem","host":"10.102.192.101","path":"/Abhinav/vgpu-ps1","port":2049,"url":"NetworkFilesystem://10.102.192.101/Abhinav/vgpu-ps1/?ROLE=Primary&STOREUUID=e3feb450-3c8e-380f-bbf5-4e8e2424b5f9"}},"name":"ROOT-44","size":21474836480,"path":"fde95825-32fb-43ec-a0f9-2eed21dc9623","volumeId":59,"vmName":"i-3-44-VM","accountId":3,"format":"VHD","provisioningType":"THIN","id":59,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"fde95825-32fb-43ec-a0f9-2eed21dc9623","type":"ROOT","_details":{"managed":"false","storagePort":"2049","storageHost":"10.102.192.101","volumeSize":"21474836480"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"6f8b2a4a-45e1-49e5-9772-5ca7c93c5e71","uuid":"47535f05-4cde-4db0-b899-fcfebd1857c1","ip":"10.1.1.177","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:2d:50:00:14","dns1":"10.140.50.5","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://700","isolationUri":"vlan://700","isSecurityGroupEnabled":false}],"gpuDevice":{"gpuGroup":"Group of NVIDIA Corporation GK104GL [GRID K2] GPUs","vgpuType":"GRID K200"}},"wait":0}}] }
> 2014-08-18 19:16:01,815 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-31:ctx-f3087b9e) Preparing host for migrating com.cloud.agent.api.to.VirtualMachineTO@7d4d9b13
> 2014-08-18 19:16:01,890 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-31:ctx-f3087b9e) Found a network called VLAN-7d3eae71-f914-1cf0-fd49-0ba9560014a2-700 on host=10.102.192.10; Network=5ffbcf9d-e864-fc86-551c-1c46cc8eb7f0; pif=f66d3a35-c6d1-c1cb-5082-ca798220ed10
> 2014-08-18 19:16:01,890 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-31:ctx-f3087b9e) 4. The VM i-3-45-VM is in Migrating state
> 2014-08-18 19:16:01,891 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-31:ctx-f3087b9e) Seq 1-7894810146780479507: Response Received:
> 2014-08-18 19:16:01,891 DEBUG [c.c.a.t.Request] (DirectAgent-31:ctx-f3087b9e) Seq 1-7894810146780479507: Processing: { Ans: , MgmtId: 213737702773493, via: 1, Ver: v1, Flags: 110, [{"com.cloud.agent.api.PrepareForMigrationAnswer":{"result":true,"wait":0}}] }
> 2014-08-18 19:16:01,894 DEBUG [c.c.a.t.Request] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Seq 1-7894810146780479507: Received: { Ans: , MgmtId: 213737702773493, via: 1, Ver: v1, Flags: 110, { PrepareForMigrationAnswer } }
> 2014-08-18 19:16:01,899 DEBUG [c.c.a.m.AgentAttache] (DirectAgent-31:ctx-f3087b9e) Seq 1-7894810146780479508: Sending now. is current sequence.
> 2014-08-18 19:16:01,910 DEBUG [c.c.a.t.Request] (DirectAgent-31:ctx-f3087b9e) Seq 1-7894810146780479508: Executing: { Cmd , MgmtId: 213737702773493, via: 1(cldstk-R720-10), Ver: v1, Flags: 100111, [{"com.cloud.agent.api.PrepareForMigrationCommand":{"vm":{"id":44,"name":"i-3-44-VM","bootloader":"PyGrub","type":"User","cpus":2,"minSpeed":1600,"maxSpeed":1600,"minRam":3221225472,"maxRam":3221225472,"arch":"x86_64","os":"Windows 8 (64-bit)","platformEmulator":"Windows 8 (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"dZkRpXoT4v5TCEO0OP5G9A==","params":{"platform":"vgpu_config:/usr/share/nvidia/vgx/grid_k200.conf;viridian:true;acpi:1;apic:true;pae:true;vgpu_pci_id:0000:08:00.0;videoram:8;device_id:0002;nx:true;timeoffset:-1;vga:std","hypervisortoolsversion":"xenserver61","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"13d4c38e-74e7-43b2-861f-bbb4ad5677b0","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"2bffd799-c085-4aa6-b151-fd8c3a8c36be","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e3feb450-3c8e-380f-bbf5-4e8e2424b5f9","id":1,"poolType":"NetworkFilesystem","host":"10.102.192.101","path":"/Abhinav/vgpu-ps1","port":2049,"url":"NetworkFilesystem://10.102.192.101/Abhinav/vgpu-ps1/?ROLE=Primary&STOREUUID=e3feb450-3c8e-380f-bbf5-4e8e2424b5f9"}},"name":"ROOT-44","size":21474836480,"path":"fde95825-32fb-43ec-a0f9-2eed21dc9623","volumeId":59,"vmName":"i-3-44-VM","accountId":3,"format":"VHD","provisioningType":"THIN","id":59,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"fde95825-32fb-43ec-a0f9-2eed21dc9623","type":"ROOT","_details":{"managed":"false","storagePort":"2049","storageHost":"10.102.192.101","volumeSize":"21474836480"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"6f8b2a4a-45e1-49e5-9772-5ca7c93c5e71","uuid":"47535f05-4cde-4db0-b899-fcfebd1857c1","ip":"10.1.1.177","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:2d:50:00:14","dns1":"10.140.50.5","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://700","isolationUri":"vlan://700","isSecurityGroupEnabled":false}],"gpuDevice":{"gpuGroup":"Group of NVIDIA Corporation GK104GL [GRID K2] GPUs","vgpuType":"GRID K200"}},"wait":0}}] }
> 2014-08-18 19:16:01,910 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-10:ctx-afc70703) Seq 1-7894810146780479508: Executing request
> 2014-08-18 19:16:01,916 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-10:ctx-afc70703) Preparing host for migrating com.cloud.agent.api.to.VirtualMachineTO@f5798a8
> 2014-08-18 19:16:01,951 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) VM state transitted from :Running to Migrating with event: MigrationRequestedvm's original host id: 8 new host id: 1 host id before state transition: 8
> 2014-08-18 19:16:01,983 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Hosts's actual total CPU: 55200 and CPU after applying overprovisioning: 55200
> 2014-08-18 19:16:01,983 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) We are allocating VM, increasing the used capacity of this host:1
> 2014-08-18 19:16:01,983 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Current Used CPU: 2000 , Free CPU:53200 ,Requested CPU: 3200
> 2014-08-18 19:16:01,983 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Current Used RAM: 1879048192 , Free RAM:13840119808 ,Requested RAM: 3221225472
> 2014-08-18 19:16:01,986 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) CPU STATS after allocation: for host: 1, old used: 2000, old reserved: 0, actual total: 55200, total with overprovisioning: 55200; new used:5200, reserved:0; requested cpu:3200,alloc_from_last:false
> 2014-08-18 19:16:01,986 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) RAM STATS after allocation: for host: 1, old used: 1879048192, old reserved: 0, total: 15719168000; new used: 5100273664, reserved: 0; requested mem: 3221225472,alloc_from_last:false
> 2014-08-18 19:16:02,014 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-10:ctx-afc70703) Found a network called VLAN-7d3eae71-f914-1cf0-fd49-0ba9560014a2-700 on host=10.102.192.10; Network=5ffbcf9d-e864-fc86-551c-1c46cc8eb7f0; pif=f66d3a35-c6d1-c1cb-5082-ca798220ed10
> 2014-08-18 19:16:02,016 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-10:ctx-afc70703) 4. The VM i-3-44-VM is in Migrating state
> 2014-08-18 19:16:02,017 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-10:ctx-afc70703) Seq 1-7894810146780479508: Response Received:
> 2014-08-18 19:16:02,017 DEBUG [c.c.a.t.Request] (DirectAgent-10:ctx-afc70703) Seq 1-7894810146780479508: Processing: { Ans: , MgmtId: 213737702773493, via: 1, Ver: v1, Flags: 110, [{"com.cloud.agent.api.PrepareForMigrationAnswer":{"result":true,"wait":0}}] }
> 2014-08-18 19:16:02,017 DEBUG [c.c.a.t.Request] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Seq 1-7894810146780479508: Received: { Ans: , MgmtId: 213737702773493, via: 1, Ver: v1, Flags: 110, { PrepareForMigrationAnswer } }
> 2014-08-18 19:16:02,019 DEBUG [c.c.a.m.AgentAttache] (DirectAgent-10:ctx-afc70703) Seq 1-7894810146780479508: No more commands found
> 2014-08-18 19:16:02,028 DEBUG [c.c.a.t.Request] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Seq 8-5623307084725485587: Sending { Cmd , MgmtId: 213737702773493, via: 8(cldstk-R720-66), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.MigrateCommand":{"vmName":"i-3-45-VM","destIp":"10.102.192.10","hostGuid":"964fb4ce-dcaa-4eac-a957-29289cb6d784","isWindows":true,"vmTO":{"id":45,"name":"i-3-45-VM","bootloader":"PyGrub","type":"User","cpus":2,"minSpeed":1600,"maxSpeed":1600,"minRam":3221225472,"maxRam":3221225472,"arch":"x86_64","os":"Windows 8 (64-bit)","platformEmulator":"Windows 8 (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"fJUipR1PM6PLvPSTXWLuKg==","params":{"platform":"vgpu_config:/usr/share/nvidia/vgx/grid_k240q.conf;viridian:true;acpi:1;apic:true;pae:true;vgpu_pci_id:0000:07:00.0;videoram:8;device_id:0002;nx:true;timeoffset:-1;vga:std","hypervisortoolsversion":"xenserver61","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"3aa5d887-38e0-40e4-9c11-d3f27dd80f62","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"ac0b242e-d2cc-40ca-9e1a-4e88c0976b39","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e3feb450-3c8e-380f-bbf5-4e8e2424b5f9","id":1,"poolType":"NetworkFilesystem","host":"10.102.192.101","path":"/Abhinav/vgpu-ps1","port":2049,"url":"NetworkFilesystem://10.102.192.101/Abhinav/vgpu-ps1/?ROLE=Primary&STOREUUID=e3feb450-3c8e-380f-bbf5-4e8e2424b5f9"}},"name":"ROOT-45","size":21474836480,"path":"8343dcb8-fa69-479a-9c6d-5d7d28270b8e","volumeId":60,"vmName":"i-3-45-VM","accountId":3,"format":"VHD","provisioningType":"THIN","id":60,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"8343dcb8-fa69-479a-9c6d-5d7d28270b8e","type":"ROOT","_details":{"managed":"false","storagePort":"2049","storageHost":"10.102.192.101","volumeSize":"21474836480"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"871a3594-2404-4501-bf40-b043240bc737","uuid":"47535f05-4cde-4db0-b899-fcfebd1857c1","ip":"10.1.1.84","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:0d:e0:00:15","dns1":"10.140.50.5","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://700","isolationUri":"vlan://700","isSecurityGroupEnabled":false}],"gpuDevice":{"gpuGroup":"Group of NVIDIA Corporation GK104GL [GRID K2] GPUs","vgpuType":"GRID K240Q"}},"executeInSequence":false,"wait":0}}] }
> 2014-08-18 19:16:02,030 DEBUG [c.c.a.t.Request] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Seq 8-5623307084725485587: Executing: { Cmd , MgmtId: 213737702773493, via: 8(cldstk-R720-66), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.MigrateCommand":{"vmName":"i-3-45-VM","destIp":"10.102.192.10","hostGuid":"964fb4ce-dcaa-4eac-a957-29289cb6d784","isWindows":true,"vmTO":{"id":45,"name":"i-3-45-VM","bootloader":"PyGrub","type":"User","cpus":2,"minSpeed":1600,"maxSpeed":1600,"minRam":3221225472,"maxRam":3221225472,"arch":"x86_64","os":"Windows 8 (64-bit)","platformEmulator":"Windows 8 (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"fJUipR1PM6PLvPSTXWLuKg==","params":{"platform":"vgpu_config:/usr/share/nvidia/vgx/grid_k240q.conf;viridian:true;acpi:1;apic:true;pae:true;vgpu_pci_id:0000:07:00.0;videoram:8;device_id:0002;nx:true;timeoffset:-1;vga:std","hypervisortoolsversion":"xenserver61","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"3aa5d887-38e0-40e4-9c11-d3f27dd80f62","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"ac0b242e-d2cc-40ca-9e1a-4e88c0976b39","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e3feb450-3c8e-380f-bbf5-4e8e2424b5f9","id":1,"poolType":"NetworkFilesystem","host":"10.102.192.101","path":"/Abhinav/vgpu-ps1","port":2049,"url":"NetworkFilesystem://10.102.192.101/Abhinav/vgpu-ps1/?ROLE=Primary&STOREUUID=e3feb450-3c8e-380f-bbf5-4e8e2424b5f9"}},"name":"ROOT-45","size":21474836480,"path":"8343dcb8-fa69-479a-9c6d-5d7d28270b8e","volumeId":60,"vmName":"i-3-45-VM","accountId":3,"format":"VHD","provisioningType":"THIN","id":60,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"8343dcb8-fa69-479a-9c6d-5d7d28270b8e","type":"ROOT","_details":{"managed":"false","storagePort":"2049","storageHost":"10.102.192.101","volumeSize":"21474836480"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"871a3594-2404-4501-bf40-b043240bc737","uuid":"47535f05-4cde-4db0-b899-fcfebd1857c1","ip":"10.1.1.84","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:0d:e0:00:15","dns1":"10.140.50.5","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://700","isolationUri":"vlan://700","isSecurityGroupEnabled":false}],"gpuDevice":{"gpuGroup":"Group of NVIDIA Corporation GK104GL [GRID K2] GPUs","vgpuType":"GRID K240Q"}},"executeInSequence":false,"wait":0}}] }
> 2014-08-18 19:16:02,036 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-32:ctx-02f4f02c) Seq 8-5623307084725485587: Executing request
> 2014-08-18 19:16:02,040 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) VM state transitted from :Running to Migrating with event: MigrationRequestedvm's original host id: 8 new host id: 1 host id before state transition: 8
> 2014-08-18 19:16:02,057 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Hosts's actual total CPU: 55200 and CPU after applying overprovisioning: 55200
> 2014-08-18 19:16:02,057 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) We are allocating VM, increasing the used capacity of this host:1
> 2014-08-18 19:16:02,057 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Current Used CPU: 5200 , Free CPU:50000 ,Requested CPU: 3200
> 2014-08-18 19:16:02,057 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Current Used RAM: 5100273664 , Free RAM:10618894336 ,Requested RAM: 3221225472
> 2014-08-18 19:16:02,057 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) CPU STATS after allocation: for host: 1, old used: 5200, old reserved: 0, actual total: 55200, total with overprovisioning: 55200; new used:8400, reserved:0; requested cpu:3200,alloc_from_last:false
> 2014-08-18 19:16:02,057 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) RAM STATS after allocation: for host: 1, old used: 5100273664, old reserved: 0, total: 15719168000; new used: 8321499136, reserved: 0; requested mem: 3221225472,alloc_from_last:false
> 2014-08-18 19:16:02,081 DEBUG [c.c.a.t.Request] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Seq 8-5623307084725485588: Sending { Cmd , MgmtId: 213737702773493, via: 8(cldstk-R720-66), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.MigrateCommand":{"vmName":"i-3-44-VM","destIp":"10.102.192.10","hostGuid":"964fb4ce-dcaa-4eac-a957-29289cb6d784","isWindows":true,"vmTO":{"id":44,"name":"i-3-44-VM","bootloader":"PyGrub","type":"User","cpus":2,"minSpeed":1600,"maxSpeed":1600,"minRam":3221225472,"maxRam":3221225472,"arch":"x86_64","os":"Windows 8 (64-bit)","platformEmulator":"Windows 8 (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"dZkRpXoT4v5TCEO0OP5G9A==","params":{"platform":"vgpu_config:/usr/share/nvidia/vgx/grid_k200.conf;viridian:true;acpi:1;apic:true;pae:true;vgpu_pci_id:0000:08:00.0;videoram:8;device_id:0002;nx:true;timeoffset:-1;vga:std","hypervisortoolsversion":"xenserver61","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"13d4c38e-74e7-43b2-861f-bbb4ad5677b0","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"2bffd799-c085-4aa6-b151-fd8c3a8c36be","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e3feb450-3c8e-380f-bbf5-4e8e2424b5f9","id":1,"poolType":"NetworkFilesystem","host":"10.102.192.101","path":"/Abhinav/vgpu-ps1","port":2049,"url":"NetworkFilesystem://10.102.192.101/Abhinav/vgpu-ps1/?ROLE=Primary&STOREUUID=e3feb450-3c8e-380f-bbf5-4e8e2424b5f9"}},"name":"ROOT-44","size":21474836480,"path":"fde95825-32fb-43ec-a0f9-2eed21dc9623","volumeId":59,"vmName":"i-3-44-VM","accountId":3,"format":"VHD","provisioningType":"THIN","id":59,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"fde95825-32fb-43ec-a0f9-2eed21dc9623","type":"ROOT","_details":{"managed":"false","storagePort":"2049","storageHost":"10.102.192.101","volumeSize":"21474836480"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"6f8b2a4a-45e1-49e5-9772-5ca7c93c5e71","uuid":"47535f05-4cde-4db0-b899-fcfebd1857c1","ip":"10.1.1.177","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:2d:50:00:14","dns1":"10.140.50.5","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://700","isolationUri":"vlan://700","isSecurityGroupEnabled":false}],"gpuDevice":{"gpuGroup":"Group of NVIDIA Corporation GK104GL [GRID K2] GPUs","vgpuType":"GRID K200"}},"executeInSequence":false,"wait":0}}] }
> 2014-08-18 19:16:02,083 DEBUG [c.c.a.t.Request] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Seq 8-5623307084725485588: Executing: { Cmd , MgmtId: 213737702773493, via: 8(cldstk-R720-66), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.MigrateCommand":{"vmName":"i-3-44-VM","destIp":"10.102.192.10","hostGuid":"964fb4ce-dcaa-4eac-a957-29289cb6d784","isWindows":true,"vmTO":{"id":44,"name":"i-3-44-VM","bootloader":"PyGrub","type":"User","cpus":2,"minSpeed":1600,"maxSpeed":1600,"minRam":3221225472,"maxRam":3221225472,"arch":"x86_64","os":"Windows 8 (64-bit)","platformEmulator":"Windows 8 (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"dZkRpXoT4v5TCEO0OP5G9A==","params":{"platform":"vgpu_config:/usr/share/nvidia/vgx/grid_k200.conf;viridian:true;acpi:1;apic:true;pae:true;vgpu_pci_id:0000:08:00.0;videoram:8;device_id:0002;nx:true;timeoffset:-1;vga:std","hypervisortoolsversion":"xenserver61","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"13d4c38e-74e7-43b2-861f-bbb4ad5677b0","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"2bffd799-c085-4aa6-b151-fd8c3a8c36be","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e3feb450-3c8e-380f-bbf5-4e8e2424b5f9","id":1,"poolType":"NetworkFilesystem","host":"10.102.192.101","path":"/Abhinav/vgpu-ps1","port":2049,"url":"NetworkFilesystem://10.102.192.101/Abhinav/vgpu-ps1/?ROLE=Primary&STOREUUID=e3feb450-3c8e-380f-bbf5-4e8e2424b5f9"}},"name":"ROOT-44","size":21474836480,"path":"fde95825-32fb-43ec-a0f9-2eed21dc9623","volumeId":59,"vmName":"i-3-44-VM","accountId":3,"format":"VHD","provisioningType":"THIN","id":59,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"fde95825-32fb-43ec-a0f9-2eed21dc9623","type":"ROOT","_details":{"managed":"false","storagePort":"2049","storageHost":"10.102.192.101","volumeSize":"21474836480"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"6f8b2a4a-45e1-49e5-9772-5ca7c93c5e71","uuid":"47535f05-4cde-4db0-b899-fcfebd1857c1","ip":"10.1.1.177","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:2d:50:00:14","dns1":"10.140.50.5","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://700","isolationUri":"vlan://700","isSecurityGroupEnabled":false}],"gpuDevice":{"gpuGroup":"Group of NVIDIA Corporation GK104GL [GRID K2] GPUs","vgpuType":"GRID K200"}},"executeInSequence":false,"wait":0}}] }
> 2014-08-18 19:16:02,086 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-11:ctx-84f8fe28) Seq 8-5623307084725485588: Executing request
> 2014-08-18 19:16:03,089 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-32:ctx-02f4f02c) Task failed! Task record: uuid: e4b5c630-e6e1-1abe-88de-0eec313abaf8
> nameLabel: Async.VM.pool_migrate
> nameDescription:
> allowedOperations: []
> currentOperations: {}
> created: Tue Aug 19 00:28:48 IST 2014
> finished: Tue Aug 19 00:28:48 IST 2014
> status: failure
> residentOn: com.xensource.xenapi.Host@534d97dd
> progress: 1.0
> type: <none/>
> result:
> errorInfo: [VM_HAS_VGPU, OpaqueRef:714675bb-47c1-d791-d537-bed377ca3426]
> otherConfig: {}
> subtaskOf: com.xensource.xenapi.Task@aaf13f6f
> subtasks: []
> 2014-08-18 19:16:03,097 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-32:ctx-02f4f02c) Unable to migrate VM(i-3-45-VM) from host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task failed! Task record: uuid: e4b5c630-e6e1-1abe-88de-0eec313abaf8
> nameLabel: Async.VM.pool_migrate
> nameDescription:
> allowedOperations: []
> currentOperations: {}
> created: Tue Aug 19 00:28:48 IST 2014
> finished: Tue Aug 19 00:28:48 IST 2014
> status: failure
> residentOn: com.xensource.xenapi.Host@534d97dd
> progress: 1.0
> type: <none/>
> result:
> errorInfo: [VM_HAS_VGPU, OpaqueRef:714675bb-47c1-d791-d537-bed377ca3426]
> otherConfig: {}
> subtaskOf: com.xensource.xenapi.Task@aaf13f6f
> subtasks: []
> Task failed! Task record: uuid: e4b5c630-e6e1-1abe-88de-0eec313abaf8
> nameLabel: Async.VM.pool_migrate
> nameDescription:
> allowedOperations: []
> currentOperations: {}
> created: Tue Aug 19 00:28:48 IST 2014
> finished: Tue Aug 19 00:28:48 IST 2014
> status: failure
> residentOn: com.xensource.xenapi.Host@534d97dd
> progress: 1.0
> type: <none/>
> result:
> errorInfo: [VM_HAS_VGPU, OpaqueRef:714675bb-47c1-d791-d537-bed377ca3426]
> otherConfig: {}
> subtaskOf: com.xensource.xenapi.Task@aaf13f6f
> subtasks: []
> at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.checkForSuccess(CitrixResourceBase.java:3166)
> at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.migrateVM(CitrixResourceBase.java:3319)
> at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2916)
> at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:455)
> at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
> at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:101)
> at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
> at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:294)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> 2014-08-18 19:16:03,104 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-32:ctx-02f4f02c) unable to destroy task(com.xensource.xenapi.Task@ec446b4c) on host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
> 2014-08-18 19:16:03,105 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-32:ctx-02f4f02c) Catch Exception com.cloud.utils.exception.CloudRuntimeException: Migration failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to migrate VM(i-3-45-VM) from host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task failed! Task record: uuid: e4b5c630-e6e1-1abe-88de-0eec313abaf8
> nameLabel: Async.VM.pool_migrate
> nameDescription:
> allowedOperations: []
> currentOperations: {}
> created: Tue Aug 19 00:28:48 IST 2014
> finished: Tue Aug 19 00:28:48 IST 2014
> status: failure
> residentOn: com.xensource.xenapi.Host@534d97dd
> progress: 1.0
> type: <none/>
> result:
> errorInfo: [VM_HAS_VGPU, OpaqueRef:714675bb-47c1-d791-d537-bed377ca3426]
> otherConfig: {}
> subtaskOf: com.xensource.xenapi.Task@aaf13f6f
> subtasks: []
> com.cloud.utils.exception.CloudRuntimeException: Unable to migrate VM(i-3-45-VM) from host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task failed! Task record: uuid: e4b5c630-e6e1-1abe-88de-0eec313abaf8
> nameLabel: Async.VM.pool_migrate
> nameDescription:
> allowedOperations: []
> currentOperations: {}
> created: Tue Aug 19 00:28:48 IST 2014
> finished: Tue Aug 19 00:28:48 IST 2014
> status: failure
> residentOn: com.xensource.xenapi.Host@534d97dd
> progress: 1.0
> type: <none/>
> result:
> errorInfo: [VM_HAS_VGPU, OpaqueRef:714675bb-47c1-d791-d537-bed377ca3426]
> otherConfig: {}
> subtaskOf: com.xensource.xenapi.Task@aaf13f6f
> subtasks: []
> at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.migrateVM(CitrixResourceBase.java:3330)
> at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2916)
> at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:455)
> at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
> at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:101)
> at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
> at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:294)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> 2014-08-18 19:16:03,106 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-32:ctx-02f4f02c) Seq 8-5623307084725485587: Response Received:
> 2014-08-18 19:16:03,107 DEBUG [c.c.a.t.Request] (DirectAgent-32:ctx-02f4f02c) Seq 8-5623307084725485587: Processing: { Ans: , MgmtId: 213737702773493, via: 8, Ver: v1, Flags: 10, [{"com.cloud.agent.api.MigrateAnswer":{"result":false,"details":"Catch Exception com.cloud.utils.exception.CloudRuntimeException: Migration failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to migrate VM(i-3-45-VM) from host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task failed! Task record: uuid: e4b5c630-e6e1-1abe-88de-0eec313abaf8\n nameLabel: Async.VM.pool_migrate\n nameDescription: \n allowedOperations: []\n currentOperations: {}\n created: Tue Aug 19 00:28:48 IST 2014\n finished: Tue Aug 19 00:28:48 IST 2014\n status: failure\n residentOn: com.xensource.xenapi.Host@534d97dd\n progress: 1.0\n type: <none/>\n result: \n errorInfo: [VM_HAS_VGPU, OpaqueRef:714675bb-47c1-d791-d537-bed377ca3426]\n otherConfig: {}\n subtaskOf: com.xensource.xenapi.Task@aaf13f6f\n subtasks: []\n","wait":0}}] }
> 2014-08-18 19:16:03,107 DEBUG [c.c.a.t.Request] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Seq 8-5623307084725485587: Received: { Ans: , MgmtId: 213737702773493, via: 8, Ver: v1, Flags: 10, { MigrateAnswer } }
> 2014-08-18 19:16:03,107 INFO [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Migration was unsuccessful. Cleaning up: VM[User|i-3-45-VM]
> 2014-08-18 19:16:03,115 WARN [o.a.c.alerts] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) alertType:: 17 // dataCenterId:: 1 // podId:: 1 // clusterId:: null // message:: Unable to migrate vm i-3-45-VM from host cldstk-R720-66 in zone vgpu and pod vgpu
> 2014-08-18 19:16:03,119 DEBUG [c.c.a.AlertManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Have already sent: 1 emails for alert type '17' -- skipping send email
> 2014-08-18 19:16:03,121 DEBUG [c.c.a.m.AgentAttache] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Seq 1-7894810146780479509: Routed from 213737702773493
> 2014-08-18 19:16:03,121 DEBUG [c.c.a.t.Request] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Seq 1-7894810146780479509: Sending { Cmd , MgmtId: 213737702773493, via: 1(cldstk-R720-10), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"i-3-45-VM","wait":0}}] }
> 2014-08-18 19:16:03,122 DEBUG [c.c.a.t.Request] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Seq 1-7894810146780479509: Executing: { Cmd , MgmtId: 213737702773493, via: 1(cldstk-R720-10), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"i-3-45-VM","wait":0}}] }
> 2014-08-18 19:16:03,122 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-17:ctx-d4a3374c) Seq 1-7894810146780479509: Executing request
> 2014-08-18 19:16:03,143 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) VM state transitted from :Migrating to Running with event: OperationFailedvm's original host id: 8 new host id: 8 host id before state transition: 1
> 2014-08-18 19:16:03,146 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-11:ctx-84f8fe28) Task failed! Task record: uuid: f40dee3d-b4fc-d49d-cb2e-4fbdad143171
> nameLabel: Async.VM.pool_migrate
> nameDescription:
> allowedOperations: []
> currentOperations: {}
> created: Tue Aug 19 00:28:48 IST 2014
> finished: Tue Aug 19 00:28:48 IST 2014
> status: failure
> residentOn: com.xensource.xenapi.Host@534d97dd
> progress: 1.0
> type: <none/>
> result:
> errorInfo: [VM_HAS_VGPU, OpaqueRef:87823176-562f-9b52-680c-5a7ef70d8241]
> otherConfig: {}
> subtaskOf: com.xensource.xenapi.Task@aaf13f6f
> subtasks: []
> 2014-08-18 19:16:03,149 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-17:ctx-d4a3374c) Seq 1-7894810146780479509: Response Received:
> 2014-08-18 19:16:03,152 DEBUG [c.c.a.t.Request] (DirectAgent-17:ctx-d4a3374c) Seq 1-7894810146780479509: Processing: { Ans: , MgmtId: 213737702773493, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StopAnswer":{"result":true,"details":"VM does not exist","wait":0}}] }
> 2014-08-18 19:16:03,152 DEBUG [c.c.a.m.AgentAttache] (DirectAgent-17:ctx-d4a3374c) Seq 1-7894810146780479509: Unable to find listener.
> 2014-08-18 19:16:03,159 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Hosts's actual total CPU: 55200 and CPU after applying overprovisioning: 55200
> 2014-08-18 19:16:03,159 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Hosts's actual total RAM: 15719168512 and RAM after applying overprovisioning: 15719168000
> 2014-08-18 19:16:03,160 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-11:ctx-84f8fe28) Unable to migrate VM(i-3-44-VM) from host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task failed! Task record: uuid: f40dee3d-b4fc-d49d-cb2e-4fbdad143171
> nameLabel: Async.VM.pool_migrate
> nameDescription:
> allowedOperations: []
> currentOperations: {}
> created: Tue Aug 19 00:28:48 IST 2014
> finished: Tue Aug 19 00:28:48 IST 2014
> status: failure
> residentOn: com.xensource.xenapi.Host@534d97dd
> progress: 1.0
> type: <none/>
> result:
> errorInfo: [VM_HAS_VGPU, OpaqueRef:87823176-562f-9b52-680c-5a7ef70d8241]
> otherConfig: {}
> subtaskOf: com.xensource.xenapi.Task@aaf13f6f
> subtasks: []
> Task failed! Task record: uuid: f40dee3d-b4fc-d49d-cb2e-4fbdad143171
> nameLabel: Async.VM.pool_migrate
> nameDescription:
> allowedOperations: []
> currentOperations: {}
> created: Tue Aug 19 00:28:48 IST 2014
> finished: Tue Aug 19 00:28:48 IST 2014
> status: failure
> residentOn: com.xensource.xenapi.Host@534d97dd
> progress: 1.0
> type: <none/>
> result:
> errorInfo: [VM_HAS_VGPU, OpaqueRef:87823176-562f-9b52-680c-5a7ef70d8241]
> otherConfig: {}
> subtaskOf: com.xensource.xenapi.Task@aaf13f6f
> subtasks: []
> at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.checkForSuccess(CitrixResourceBase.java:3166)
> at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.migrateVM(CitrixResourceBase.java:3319)
> at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2916)
> at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:455)
> at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
> at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:101)
> at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
> at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:294)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> 2014-08-18 19:16:03,161 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) release cpu from host: 1, old used: 8400,reserved: 0, actual total: 55200, total with overprovisioning: 55200; new used: 5200,reserved:0; movedfromreserved: false,moveToReserveredfalse
> 2014-08-18 19:16:03,162 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) release mem from host: 1, old used: 8321499136,reserved: 0, total: 15719168000; new used: 5100273664,reserved:0; movedfromreserved: false,moveToReserveredfalse
> 2014-08-18 19:16:03,168 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-11:ctx-84f8fe28) unable to destroy task(com.xensource.xenapi.Task@1e06f89f) on host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
> 2014-08-18 19:16:03,171 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-11:ctx-84f8fe28) Catch Exception com.cloud.utils.exception.CloudRuntimeException: Migration failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to migrate VM(i-3-44-VM) from host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task failed! Task record: uuid: f40dee3d-b4fc-d49d-cb2e-4fbdad143171
> nameLabel: Async.VM.pool_migrate
> nameDescription:
> allowedOperations: []
> currentOperations: {}
> created: Tue Aug 19 00:28:48 IST 2014
> finished: Tue Aug 19 00:28:48 IST 2014
> status: failure
> residentOn: com.xensource.xenapi.Host@534d97dd
> progress: 1.0
> type: <none/>
> result:
> errorInfo: [VM_HAS_VGPU, OpaqueRef:87823176-562f-9b52-680c-5a7ef70d8241]
> otherConfig: {}
> subtaskOf: com.xensource.xenapi.Task@aaf13f6f
> subtasks: []
> com.cloud.utils.exception.CloudRuntimeException: Unable to migrate VM(i-3-44-VM) from host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task failed! Task record: uuid: f40dee3d-b4fc-d49d-cb2e-4fbdad143171
> nameLabel: Async.VM.pool_migrate
> nameDescription:
> allowedOperations: []
> currentOperations: {}
> created: Tue Aug 19 00:28:48 IST 2014
> finished: Tue Aug 19 00:28:48 IST 2014
> status: failure
> residentOn: com.xensource.xenapi.Host@534d97dd
> progress: 1.0
> type: <none/>
> result:
> errorInfo: [VM_HAS_VGPU, OpaqueRef:87823176-562f-9b52-680c-5a7ef70d8241]
> otherConfig: {}
> subtaskOf: com.xensource.xenapi.Task@aaf13f6f
> subtasks: []
> at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.migrateVM(CitrixResourceBase.java:3330)
> at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2916)
> at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:455)
> at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
> at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:101)
> at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
> at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:294)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> 2014-08-18 19:16:03,173 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-11:ctx-84f8fe28) Seq 8-5623307084725485588: Response Received:
> 2014-08-18 19:16:03,174 DEBUG [c.c.a.t.Request] (DirectAgent-11:ctx-84f8fe28) Seq 8-5623307084725485588: Processing: { Ans: , MgmtId: 213737702773493, via: 8, Ver: v1, Flags: 10, [{"com.cloud.agent.api.MigrateAnswer":{"result":false,"details":"Catch Exception com.cloud.utils.exception.CloudRuntimeException: Migration failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to migrate VM(i-3-44-VM) from host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task failed! Task record: uuid: f40dee3d-b4fc-d49d-cb2e-4fbdad143171\n nameLabel: Async.VM.pool_migrate\n nameDescription: \n allowedOperations: []\n currentOperations: {}\n created: Tue Aug 19 00:28:48 IST 2014\n finished: Tue Aug 19 00:28:48 IST 2014\n status: failure\n residentOn: com.xensource.xenapi.Host@534d97dd\n progress: 1.0\n type: <none/>\n result: \n errorInfo: [VM_HAS_VGPU, OpaqueRef:87823176-562f-9b52-680c-5a7ef70d8241]\n otherConfig: {}\n subtaskOf: com.xensource.xenapi.Task@aaf13f6f\n subtasks: []\n","wait":0}}] }
> 2014-08-18 19:16:03,175 DEBUG [c.c.a.t.Request] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Seq 8-5623307084725485588: Received: { Ans: , MgmtId: 213737702773493, via: 8, Ver: v1, Flags: 10, { MigrateAnswer } }
> 2014-08-18 19:16:03,175 INFO [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Migration was unsuccessful. Cleaning up: VM[User|i-3-44-VM]
> 2014-08-18 19:16:03,179 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Invocation exception, caused by: com.cloud.utils.exception.CloudRuntimeException: Unable to migrate due to Catch Exception com.cloud.utils.exception.CloudRuntimeException: Migration failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to migrate VM(i-3-45-VM) from host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task failed! Task record: uuid: e4b5c630-e6e1-1abe-88de-0eec313abaf8
> nameLabel: Async.VM.pool_migrate
> nameDescription:
> allowedOperations: []
> currentOperations: {}
> created: Tue Aug 19 00:28:48 IST 2014
> finished: Tue Aug 19 00:28:48 IST 2014
> status: failure
> residentOn: com.xensource.xenapi.Host@534d97dd
> progress: 1.0
> type: <none/>
> result:
> errorInfo: [VM_HAS_VGPU, OpaqueRef:714675bb-47c1-d791-d537-bed377ca3426]
> otherConfig: {}
> subtaskOf: com.xensource.xenapi.Task@aaf13f6f
> subtasks: []
> 2014-08-18 19:16:03,180 INFO [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Rethrow exception com.cloud.utils.exception.CloudRuntimeException: Unable to migrate due to Catch Exception com.cloud.utils.exception.CloudRuntimeException: Migration failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to migrate VM(i-3-45-VM) from host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task failed! Task record: uuid: e4b5c630-e6e1-1abe-88de-0eec313abaf8
> nameLabel: Async.VM.pool_migrate
> nameDescription:
> allowedOperations: []
> currentOperations: {}
> created: Tue Aug 19 00:28:48 IST 2014
> finished: Tue Aug 19 00:28:48 IST 2014
> status: failure
> residentOn: com.xensource.xenapi.Host@534d97dd
> progress: 1.0
> type: <none/>
> result:
> errorInfo: [VM_HAS_VGPU, OpaqueRef:714675bb-47c1-d791-d537-bed377ca3426]
> otherConfig: {}
> subtaskOf: com.xensource.xenapi.Task@aaf13f6f
> subtasks: []
> 2014-08-18 19:16:03,180 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694) Done with run of VM work job: com.cloud.vm.VmWorkMigrateAway for VM 45, job origin: 692
> 2014-08-18 19:16:03,181 ERROR [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694) Unable to complete AsyncJobVO {id:694, userId: 1, accountId: 1, instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkMigrateAway, cmdInfo: rO0ABXNyAB5jb20uY2xvdWQudm0uVm1Xb3JrTWlncmF0ZUF3YXmt4MX4jtcEmwIAAUoACXNyY0hvc3RJZHhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAALXQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAACA, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 213737702773493, completeMsid: null, lastUpdated: null, lastPolled: null, created: Mon Aug 18 19:16:00 IST 2014}, job origin:692
> com.cloud.utils.exception.CloudRuntimeException: Unable to migrate due to Catch Exception com.cloud.utils.exception.CloudRuntimeException: Migration failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to migrate VM(i-3-45-VM) from host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task failed! Task record: uuid: e4b5c630-e6e1-1abe-88de-0eec313abaf8
> nameLabel: Async.VM.pool_migrate
> nameDescription:
> allowedOperations: []
> currentOperations: {}
> created: Tue Aug 19 00:28:48 IST 2014
> finished: Tue Aug 19 00:28:48 IST 2014
> status: failure
> residentOn: com.xensource.xenapi.Host@534d97dd
> progress: 1.0
> type: <none/>
> result:
> errorInfo: [VM_HAS_VGPU, OpaqueRef:714675bb-47c1-d791-d537-bed377ca3426]
> otherConfig: {}
> subtaskOf: com.xensource.xenapi.Task@aaf13f6f
> subtasks: []
> at com.cloud.vm.VirtualMachineManagerImpl.migrate(VirtualMachineManagerImpl.java:1899)
> at com.cloud.vm.VirtualMachineManagerImpl.orchestrateMigrateAway(VirtualMachineManagerImpl.java:2332)
> at com.cloud.vm.VirtualMachineManagerImpl.orchestrateMigrateAway(VirtualMachineManagerImpl.java:4619)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
> at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:4738)
> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
> at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> 2014-08-18 19:16:03,191 WARN [o.a.c.alerts] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) alertType:: 17 // dataCenterId:: 1 // podId:: 1 // clusterId:: null // message:: Unable to migrate vm i-3-44-VM from host cldstk-R720-66 in zone vgpu and pod vgpu
> 2014-08-18 19:16:03,198 DEBUG [c.c.a.AlertManagerImpl] (Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Have already sent: 1 emails for alert type '17' -- skipping send email
> 2014-08-18 19:16:03,200 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-5:ctx-6ba6232b job-692/job-694) Complete async job-694, jobStatus: FAILED, resultCode: 0, result: rO0ABXNyABpqYXZhLmxhbmcuUnVudGltZUV4Y2VwdGlvbp5fBkcKNIPlAgAAeHIAE2phdmEubGFuZy5FeGNlcHRpb27Q_R8-GjscxAIAAHhyABNqYXZhLmxhbmcuVGhyb3dhYmxl1cY1Jzl3uMsDAARMAAVjYXVzZXQAFUxqYXZhL2xhbmcvVGhyb3dhYmxlO0wADWRldGFpbE1lc3NhZ2V0ABJMamF2YS9sYW5nL1N0cmluZztbAApzdGFja1RyYWNldAAeW0xqYXZhL2xhbmcvU3RhY2tUcmFjZUVsZW1lbnQ7TAAUc3VwcHJlc3NlZEV4Y2VwdGlvbnN0ABBMamF2YS91dGlsL0xpc3Q7eHBxAH4AB3QDq0pvYiBmYWlsZWQgZHVlIHRvIGV4Y2VwdGlvbiBVbmFibGUgdG8gbWlncmF0ZSBkdWUgdG8gQ2F0Y2ggRXhjZXB0aW9uI
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)