You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@cloudstack.apache.org by "angeline shen (JIRA)" <ji...@apache.org> on 2013/12/28 02:14:50 UTC
[jira] [Created] (CLOUDSTACK-5665) XEN patch/hotfix certification -
XS 6.0.2 XS602E030 patch installation fail
angeline shen created CLOUDSTACK-5665:
-----------------------------------------
Summary: XEN patch/hotfix certification - XS 6.0.2 XS602E030 patch installation fail
Key: CLOUDSTACK-5665
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5665
Project: CloudStack
Issue Type: Bug
Security Level: Public (Anyone can view this level - this is the default.)
Components: Management Server
Affects Versions: 4.2.1
Environment: MS CloudPlatform-4.2.1-2-rhel6.3.tar.gz
host 10.223.51.3 XS 6.0.2 base + HF XS602E001 to XS602E029 + CSPs
10.223.51.4 XS 6.0.2 base + HF XS602E001 to XS602E029 + CSPs
Reporter: angeline shen
Priority: Critical
Fix For: 4.2.1
http://wiki-ccp.citrix.com/pages/viewpage.action?pageId=11829631 :
XEN hotfix installation procedure
1. Hosts master slave installed with
XS 6.0.2 base + Hotfix XS602E001 to XS602E029 + CSPs
2. Basic zone with cluster of above master & slave hosts
security group default with NO security rules
Deploy two VMs
3. MS: master host put to maintenance mode - VMs migrate to slave host
4. MS: unmanage cluster
5. master host: install hotfix XS602E030 + CSP , reboot master
6. MS: master host cancel maintenance mode
7. MS: manage cluster
8. MS: slave host put to maintenance mode, VMs FAIL migrate to master
Unable to complete hotfix installation
2013-12-26 16:51:25,307 INFO [xen.resource.CitrixResourceBase] (DirectAgent-56:null) VM does not exist on XenServerc6c75483-b35b-4a7c-9e2d-2274e34dd437
2013-12-26 16:51:25,307 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-56:null) Seq 1-265617550: Response Received:
2013-12-26 16:51:25,307 DEBUG [agent.transport.Request] (DirectAgent-56:null) Seq 1-265617550: Processing: { Ans: , MgmtId: 7692017993539, via: 1, Ver: v1, Flags: 110, [{"com.cloud.agent.api.StopAnswer":{"result":true,"details":"VM does not exist","wait":0}}] }
2013-12-26 16:51:25,307 DEBUG [agent.manager.AgentAttache] (DirectAgent-56:null) Seq 1-265617550: Unable to find listener.
2013-12-26 16:51:25,307 DEBUG [agent.manager.AgentAttache] (DirectAgent-56:null) Seq 1-265617550: No more commands found
2013-12-26 16:51:25,313 DEBUG [vm.dao.VMInstanceDaoImpl] (HA-Worker-3:work-19) Unable to update VM[DomainRouter|r-4-VM]: DB Data={Host=4; State=Running; updated=13; time=Thu Dec 26 16:51:25 PST 2013} New Data: {Host=4; State=Stopping; updated=12; time=Thu Dec 26 16:51:25 PST 2013} Stale Data: {Host=4; State=Running; updated=11; time=Thu Dec 26 16:51:22 PST 2013}
2013-12-26 16:51:25,313 DEBUG [cloud.vm.VirtualMachineManagerImpl] (HA-Worker-3:work-19) Unable to stop VM due to VM is being operated on.
2013-12-26 16:51:25,313 WARN [cloud.ha.HighAvailabilityManagerImpl] (HA-Worker-3:work-19) Unable to migrate vm from 4
2013-12-26 16:51:25,314 DEBUG [cloud.resource.ResourceManagerImpl] (HA-Worker-3:work-19) No next resource state for host 4 while current state is ErrorInMaintenance with event UnableToMigrate
com.cloud.utils.fsm.NoTransitionException: No next resource state found for current state =ErrorInMaintenance event =UnableToMigrate
at com.cloud.resource.ResourceManagerImpl.resourceStateTransitTo(ResourceManagerImpl.java:1178)
at com.cloud.resource.ResourceManagerImpl.maintenanceFailed(ResourceManagerImpl.java:2313)
at com.cloud.ha.HighAvailabilityManagerImpl.migrate(HighAvailabilityManagerImpl.java:602)
at com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.run(HighAvailabilityManagerImpl.java:858)
2013-12-26 16:51:25,314 INFO [cloud.ha.HighAvailabilityManagerImpl] (HA-Worker-3:work-19) Completed HAWork[19-Migration-4-Running-Migrating]
2013-12-26 16:51:25,324 WARN [xen.resource.CitrixResourceBase] (DirectAgent-57:null) Task failed! Task record: uuid: 38ce6cd4-9ab2-7e46-4d10-dafe25a83084
nameLabel: Async.VM.pool_migrate
nameDescription:
allowedOperations: []
currentOperations: {}
created: Thu Dec 26 16:55:03 PST 2013
finished: Thu Dec 26 16:55:03 PST 2013
status: failure
residentOn: com.xensource.xenapi.Host@40561b9c
progress: 1.0
type: <none/>
result:
errorInfo: [VM_REQUIRES_SR, OpaqueRef:6e9c09e8-9ed5-1e21-2048-d848a084a15a, OpaqueRef:eec84d31-9691-1ba4-f7bb-39db42ac8628]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
2013-12-26 16:51:25,332 WARN [xen.resource.CitrixResourceBase] (DirectAgent-57:null) Unable to migrate VM(s-1-VM) from host(d044a044-1a30-4c01-ae83-11d2631a4ffe) due to Task failed! Task record: uuid: 38ce6cd4-9ab2-7e46-4d10-dafe25a83084
nameLabel: Async.VM.pool_migrate
nameDescription:
allowedOperations: []
currentOperations: {}
created: Thu Dec 26 16:55:03 PST 2013
finished: Thu Dec 26 16:55:03 PST 2013
status: failure
residentOn: com.xensource.xenapi.Host@40561b9c
progress: 1.0
type: <none/>
result:
errorInfo: [VM_REQUIRES_SR, OpaqueRef:6e9c09e8-9ed5-1e21-2048-d848a084a15a, OpaqueRef:eec84d31-9691-1ba4-f7bb-39db42ac8628]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
Task failed! Task record: uuid: 38ce6cd4-9ab2-7e46-4d10-dafe25a83084
nameLabel: Async.VM.pool_migrate
nameDescription:
:
residentOn: com.xensource.xenapi.Host@40561b9c
progress: 1.0
type: <none/>
result:
errorInfo: [VM_REQUIRES_SR, OpaqueRef:6e9c09e8-9ed5-1e21-2048-d848a084a15a, OpaqueRef:eec84d31-9691-1ba4-f7bb-39db42ac8628]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
at com.cloud.hypervisor.xen.resource.CitrixResourceBase.migrateVM(CitrixResourceBase.java:3789)
at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3346)
at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:511)
at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:59)
at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2013-12-26 16:51:25,342 DEBUG [xen.resource.CitrixResourceBase] (DirectAgent-57:null) 6. The VM s-1-VM is in Migrating state
2013-12-26 16:51:25,342 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-57:null) Seq 4-961806364: Response Received:
2013-12-26 16:51:25,342 DEBUG [agent.transport.Request] (DirectAgent-57:null) Seq 4-961806364: Processing: { Ans: , MgmtId: 7692017993539, via: 4, Ver: v1, Flags: 110, [{"com.cloud.agent.api.MigrateAnswer":{"result":false,"details":"Catch Exception com.cloud.utils.exception.CloudRuntimeException: Migration failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to migrate VM(s-1-VM) from host(d044a044-1a30-4c01-ae83-11d2631a4ffe) due to Task failed! Task record: uuid: 38ce6cd4-9ab2-7e46-4d10-dafe25a83084\n nameLabel: Async.VM.pool_migrate\n nameDescription: \n allowedOperations: []\n currentOperations: {}\n created: Thu Dec 26 16:55:03 PST 2013\n finished: Thu Dec 26 16:55:03 PST 2013\n status: failure\n residentOn: com.xensource.xenapi.Host@40561b9c\n progress: 1.0\n type: <none/>\n result: \n errorInfo: [VM_REQUIRES_SR, OpaqueRef:6e9c09e8-9ed5-1e21-2048-d848a084a15a, OpaqueRef:eec84d31-9691-1ba4-f7bb-39db42ac8628]\n otherConfig: {}\n subtaskOf: com.xensource.xenapi.Task@aaf13f6f\n subtasks: []\n","wait":0}}] }
2013-12-26 16:51:25,342 DEBUG [agent.transport.Request] (HA-Worker-0:work-16) Seq 4-961806364: Received: { Ans: , MgmtId: 7692017993539, via: 4, Ver: v1, Flags: 110, { MigrateAnswer } }
2013-12-26 16:51:25,342 ERROR [cloud.vm.VirtualMachineManagerImpl] (HA-Worker-0:work-16) Unable to migrate due to Catch Exception com.cloud.utils.exception.CloudRuntimeException: Migration failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to migrate VM(s-1-VM) from host(d044a044-1a30-4c01-ae83-11d2631a4ffe) due to Task failed! Task record: uuid: 38ce6cd4-9ab2-7e46-4d10-dafe25a83084
nameLabel: Async.VM.pool_migrate
nameDescription:
allowedOperations: []
currentOperations: {}
created: Thu Dec 26 16:55:03 PST 2013
finished: Thu Dec 26 16:55:03 PST 2013
status: failure
residentOn: com.xensource.xenapi.Host@40561b9c
progress: 1.0
type: <none/>
result:
errorInfo: [VM_REQUIRES_SR, OpaqueRef:6e9c09e8-9ed5-1e21-2048-d848a084a15a, OpaqueRef:eec84d31-9691-1ba4-f7bb-39db42ac8628]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
2013-12-26 16:51:25,342 INFO [cloud.vm.VirtualMachineManagerImpl] (HA-Worker-0:work-16) Migration was unsuccessful. Cleaning up: VM[SecondaryStorageVm|s-1-VM]
2013-12-26 16:51:25,346 DEBUG [agent.manager.AgentAttache] (DirectAgent-57:null) Seq 4-961806364: No more commands found
2013-12-26 16:51:25,355 WARN [apache.cloudstack.alerts] (HA-Worker-0:work-16) alertType:: 17 // dataCenterId:: 1 // podId:: 1 // clusterId:: null // message:: Unable to migrate vm s-1-VM from host Rack2Host19.lab.vmops.com in zone z1 and pod z1p1
2013-12-26 16:51:25,356 DEBUG [cloud.alert.AlertManagerImpl] (HA-Worker-0:work-16) Have already sent: 1 emails for alert type '17' -- skipping send email
2013-12-26 16:51:25,357 DEBUG [agent.manager.AgentAttache] (HA-Worker-0:work-16) Seq 1-265617551: Routed from 7692017993539
2013-12-26 16:51:25,358 DEBUG [agent.transport.Request] (HA-Worker-0:work-16) Seq 1-265617551: Sending { Cmd , MgmtId: 7692017993539, via: 1, Ver: v1, Flags: 100111, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":true,"vmName":"s-1-VM","wait":0}}] }
2013-12-26 16:51:25,358 DEBUG [agent.transport.Request] (HA-Worker-0:work-16) Seq 1-265617551: Executing: { Cmd , MgmtId: 7692017993539, via: 1, Ver: v1, Flags: 100111, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":true,"vmName":"s-1-VM","wait":0}}] }
2013-12-26 16:51:25,358 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-62:null) Seq 1-265617551: Executing request
2013-12-26 16:51:25,368 DEBUG [cloud.capacity.CapacityManagerImpl] (HA-Worker-0:work-16) VM state transitted from :Migrating to Running with event: OperationFailedvm's original host id: 4 new host id: 4 host id before state transition: 1
2013-12-26 16:51:25,375 DEBUG [cloud.capacity.CapacityManagerImpl] (HA-Worker-0:work-16) Hosts's actual total CPU: 9044 and CPU after applying overprovisioning: 9044
2013-12-26 16:51:25,375 DEBUG [cloud.capacity.CapacityManagerImpl] (HA-Worker-0:work-16) Hosts's actual total RAM: 15988871808 and RAM after applying overprovisioning: 15988872192
2013-12-26 16:51:25,375 DEBUG [cloud.capacity.CapacityManagerImpl] (HA-Worker-0:work-16) release cpu from host: 1, old used: 500,reserved: 0, actual total: 9044, total with overprovisioning: 9044; new used: 0,reserved:0; movedfromreserved: false,moveToReserveredfalse
2013-12-26 16:51:25,376 DEBUG [cloud.capacity.CapacityManagerImpl] (HA-Worker-0:work-16) release mem from host: 1, old used: 268435456,reserved: 0, total: 15988872192; new used: 0,reserved:0; movedfromreserved: false,moveToReserveredfalse
2013-12-26 16:51:25,385 INFO [xen.resource.CitrixResourceBase] (DirectAgent-62:null) VM does not exist on XenServerc6c75483-b35b-4a7c-9e2d-2274e34dd437
2013-12-26 16:51:25,385 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-62:null) Seq 1-265617551: Response Received:
2013-12-26 16:51:25,386 DEBUG [agent.transport.Request] (DirectAgent-62:null) Seq 1-265617551: Processing: { Ans: , MgmtId: 7692017993539, via: 1, Ver: v1, Flags: 110, [{"com.cloud.agent.api.StopAnswer":{"result":true,"details":"VM does not exist","wait":0}}] }
2013-12-26 16:51:25,386 DEBUG [agent.manager.AgentAttache] (DirectAgent-62:null) Seq 1-265617551: Unable to find listener.
2013-12-26 16:51:25,386 DEBUG [agent.manager.AgentAttache] (DirectAgent-62:null) Seq 1-265617551: No more commands found
2013-12-26 16:51:25,390 DEBUG [vm.dao.VMInstanceDaoImpl] (HA-Worker-0:work-16) Unable to update VM[SecondaryStorageVm|s-1-VM]: DB Data={Host=4; State=Running; updated=13; time=Thu Dec 26 16:51:25 PST 2013} New Data: {Host=4; State=Stopping; updated=12; time=Thu Dec 26 16:51:25 PST 2013} Stale Data: {Host=4; State=Running; updated=11; time=Thu Dec 26 16:51:23 PST 2013}
2013-12-26 16:51:25,390 DEBUG [cloud.vm.VirtualMachineManagerImpl] (HA-Worker-0:work-16) Unable to stop VM due to VM is being operated on.
2013-12-26 16:51:25,390 WARN [cloud.ha.HighAvailabilityManagerImpl] (HA-Worker-0:work-16) Unable to migrate vm from 4
2013-12-26 16:51:25,391 DEBUG [cloud.resource.ResourceManagerImpl] (HA-Worker-0:work-16) No next resource state for host 4 while current state is ErrorInMaintenance with event UnableToMigrate
com.cloud.utils.fsm.NoTransitionException: No next resource state found for current state =ErrorInMaintenance event =UnableToMigrate
at com.cloud.resource.ResourceManagerImpl.resourceStateTransitTo(ResourceManagerImpl.java:1178)
at com.cloud.resource.ResourceManagerImpl.maintenanceFailed(ResourceManagerImpl.java:2313)
at com.cloud.ha.HighAvailabilityManagerImpl.migrate(HighAvailabilityManagerImpl.java:602)
at com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.run(HighAvailabilityManagerImpl.java:858)
2013-12-26 16:51:25,392 INFO [cloud.ha.HighAvailabilityManagerImpl] (HA-Worker-0:work-16) Completed HAWork[16-Migration-1-Running-Migrating]
2013-12-26 16:51:26,339 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-10:null) SeqA 2-416: Processing Seq 2-416: { Cmd , MgmtId: -1, via: 2, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":2,"_loadInfo":"{\n \"connections\": []\n}","wait":0}}] }
2013-12-26 16:51:26,346 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-10:null) SeqA 2-416: Sending Seq 2-416: { Ans: , MgmtId: 7692017993539, via: 2, Ver: v1, Flags: 100010, [{"com.cloud.agent.ap
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)