You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users-cn@cloudstack.apache.org by tao huang <cl...@gmail.com> on 2013/05/16 09:49:58 UTC
突然不能创建虚机，也不能重启虚机了
各位好，cloudstack 3.0 +xenserver6.0环境，高级网络
突然无法创建虚机，感觉是主存储出了问题，但是看状态是正常的，如何排查呢，谢谢指导
2013-05-16 15:29:40,062 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-369:null) Seq 3-939886253: Response Received:
2013-05-16 15:29:40,062 DEBUG [agent.transport.Request]
(DirectAgent-369:null) Seq 3-939886253: Processing: { Ans: , MgmtId:
33932935549676, via: 3, Ver: v1, Flags: 110,
[{"StopAnswer":{"result":false,"details":"Exception:
com.cloud.utils.exception.CloudRuntimeException\nMessage: *Unable to reset
master of slave 192.168.1.38 to 192.168.1.36 due to
org.apache.xmlrpc.XmlRpcException: Failed to create input stream: Read
timed out\nStack: com.cloud.utils.exception.CloudRuntimeException: Unable
to reset master of slave 192.168.1.38 to 192.168.1.36 due to
org.apache.xmlrpc.*XmlRpcException: Failed to create input stream: Read
timed out\n\tat
com.cloud.hypervisor.xen.resource.XenServerConnectionPool.PoolEmergencyResetMaster(XenServerConnectionPool.java:439)\n\tat
com.cloud.hypervisor.xen.resource.XenServerConnectionPool.connect(XenServerConnectionPool.java:657)\n\tat
com.cloud.hypervisor.xen.resource.CitrixResourceBase.getConnection(CitrixResourceBase.java:5090)\n\tat
com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3284)\n\tat
com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:416)\n\tat
com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:69)\n\tat
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)\n\tat
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)\n\tat
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)\n\tat
java.util.concurrent.FutureTask.run(FutureTask.java:166)\n\tat
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)\n\tat
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)\n\tat
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)\n\tat
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)\n\tat
java.lang.Thread.run(Thread.java:636)\n","wait":0}}] }
2013-05-16 15:29:40,062 DEBUG [agent.manager.AgentAttache]
(DirectAgent-369:null) Seq 3-939886253: No more commands found
2013-05-16 15:29:40,062 DEBUG [agent.transport.Request]
(Job-Executor-6:job-18181) Seq 3-939886253: Received: { Ans: , MgmtId:
33932935549676, via: 3, Ver: v1, Flags: 110, { StopAnswer } }
2013-05-16 15:29:40,062 DEBUG [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-6:job-18181) Unable to stop VM due to Exception:
com.cloud.utils.exception.CloudRuntimeException
Message: Unable to reset master of slave 192.168.1.38 to 192.168.1.36 due
to org.apache.xmlrpc.XmlRpcException: Failed to create input stream: Read
timed out
Stack: com.cloud.utils.exception.CloudRuntimeException: Unable to reset
master of slave 192.168.1.38 to 192.168.1.36 due to
org.apache.xmlrpc.XmlRpcException: Failed to create input stream: Read
timed out
at
com.cloud.hypervisor.xen.resource.XenServerConnectionPool.PoolEmergencyResetMaster(XenServerConnectionPool.java:439)
at
com.cloud.hypervisor.xen.resource.XenServerConnectionPool.connect(XenServerConnectionPool.java:657)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.getConnection(CitrixResourceBase.java:5090)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3284)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:416)
at
com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:69)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
2013-05-16 15:29:40,062 WARN [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-6:job-18181) Failed to stop vm VM[DomainRouter|r-2237-VM] in
Starting state as a part of cleanup process
2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) DeploymentPlanner allocation algorithm: random
2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) Trying to allocate a host and storage pools from
dc:1, pod:null,cluster:null, requested cpu: 500, requested ram: 134217728
2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) Is ROOT volume READY (pool already allocated)?:
No
2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) Searching all possible resources under this
Zone: 1
2013-05-16 15:29:40,068 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) Listing clusters in order of aggregate capacity,
that have (atleast one host with) enough CPU and RAM capacity under this
Zone: 1
2013-05-16 15:29:40,069 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) CPUOverprovisioningFactor considered: 3.0
2013-05-16 15:29:40,077 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) Checking resources in Cluster: 1 under Pod: 1
2013-05-16 15:29:40,077 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) Calling HostAllocators to find suitable hosts
2013-05-16 15:29:40,077 DEBUG [allocator.impl.FirstFitAllocator]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Looking for hosts in
dc: 1 pod:1 cluster:1
2013-05-16 15:29:40,080 DEBUG [allocator.impl.FirstFitAllocator]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) FirstFitAllocator has 7
hosts to check for allocation: [Host[-4-Routing], Host[-2-Routing],
Host[-1-Routing], Host[-3-Routing], Host[-8-Routing], Host[-7-Routing],
Host[-15-Routing]]
2013-05-16 15:29:40,088 DEBUG [allocator.impl.FirstFitAllocator]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found 7 hosts for
allocation after prioritization: [Host[-4-Routing], Host[-2-Routing],
Host[-1-Routing], Host[-3-Routing], Host[-8-Routing], Host[-7-Routing],
Host[-15-Routing]]
2013-05-16 15:29:40,088 DEBUG [allocator.impl.FirstFitAllocator]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Looking for
speed=500Mhz, Ram=128
2013-05-16 15:29:40,091 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Checking if host: 4 has
enough capacity for requested CPU: 500 and requested RAM: 134217728 ,
cpuOverprovisioningFactor: 3.0
2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Hosts's actual total
CPU: 63984 and CPU after applying overprovisioning: 191952
2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free CPU: 161952 ,
Requested CPU: 500
2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free RAM: 71156002944 ,
Requested RAM: 134217728
2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Host has enough CPU and
RAM available
2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc CPU
from host: 4, used: 29500, reserved: 500, actual total: 63984, total with
overprovisioning: 191952; requested cpu:500,alloc_from_last_host?:false
,considerReservedCapacity?: true
2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc MEM
from host: 4, used: 28856811520, reserved: 536870912, total: 100549685376;
requested mem: 134217728,alloc_from_last_host?:false
,considerReservedCapacity?: true
2013-05-16 15:29:40,093 DEBUG [allocator.impl.FirstFitAllocator]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found a suitable host,
adding to list: 4
2013-05-16 15:29:40,095 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Checking if host: 2 has
enough capacity for requested CPU: 500 and requested RAM: 134217728 ,
cpuOverprovisioningFactor: 3.0
2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Hosts's actual total
CPU: 63984 and CPU after applying overprovisioning: 191952
2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free CPU: 151452 ,
Requested CPU: 500
2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free RAM: 58539536512 ,
Requested RAM: 134217728
2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Host has enough CPU and
RAM available
2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc CPU
from host: 2, used: 40500, reserved: 0, actual total: 63984, total with
overprovisioning: 191952; requested cpu:500,alloc_from_last_host?:false
,considerReservedCapacity?: true
2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc MEM
from host: 2, used: 42010148864, reserved: 0, total: 100549685376;
requested mem: 134217728,alloc_from_last_host?:false
,considerReservedCapacity?: true
2013-05-16 15:29:40,097 DEBUG [allocator.impl.FirstFitAllocator]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found a suitable host,
adding to list: 2
2013-05-16 15:29:40,114 DEBUG [allocator.impl.FirstFitAllocator]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found a suitable host,
adding to list: 15
*2013-05-16 15:29:40,114 DEBUG [allocator.impl.FirstFitAllocator]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Host Allocator
returning 6 suitable hosts*
*2013-05-16 15:29:40,116 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) Checking suitable pools for volume (Id, Type):
(2438,ROOT)*
*2013-05-16 15:29:40,116 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) We need to allocate new storagepool for this
volume*
*2013-05-16 15:29:40,116 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) Calling StoragePoolAllocators to find suitable
pools*
*2013-05-16 15:29:40,117 DEBUG
[storage.allocator.FirstFitStoragePoolAllocator] (Job-Executor-6:job-18181)
Looking for pools in dc: 1 pod:1 cluster:1*
*2013-05-16 15:29:40,118 DEBUG
[storage.allocator.FirstFitStoragePoolAllocator] (Job-Executor-6:job-18181)
FirstFitStoragePoolAllocator has 1 pools to check for allocation*
*2013-05-16 15:29:40,118 DEBUG
[storage.allocator.AbstractStoragePoolAllocator] (Job-Executor-6:job-18181)
Checking if storage pool is suitable, name: ps ,poolId: 200*
*2013-05-16 15:29:40,118 DEBUG
[storage.allocator.AbstractStoragePoolAllocator] (Job-Executor-6:job-18181)
StoragePool is in avoid set, skipping this pool*
*2013-05-16 15:29:40,118 DEBUG
[storage.allocator.FirstFitStoragePoolAllocator] (Job-Executor-6:job-18181)
FirstFitStoragePoolAllocator returning 0 suitable storage pools*
*2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) No suitable pools found for volume:
Vol[2438|vm=2237|ROOT] under cluster: 1*
*2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) No suitable pools found*
*2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) No suitable storagePools found under this
Cluster: 1*
*2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) Could not find suitable Deployment Destination
for this VM under any clusters, returning.*
*2013-05-16 15:29:40,135 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181) VM state transitted from :Starting to Stopped
with event: OperationFailedvm's original host id: null new host id: null
host id before state transition: 3*
*2013-05-16 15:29:40,139 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181) Hosts's actual total CPU: 63984 and CPU after
applying overprovisioning: 191952*
*2013-05-16 15:29:40,139 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181) release cpu from host: 3, old used:
29000,reserved: 0, actual total: 63984, total with overprovisioning:
191952; new used: 28500,reserved:0; movedfromreserved:
false,moveToReserveredfalse*
Re: Re: Re: 突然不能创建虚机，也不能重启虚机了
Posted by tao huang <cl...@gmail.com>.
找到原因了，应该是xenserver 的license到期了，更新试试，谢谢组织
2013/5/17 tanthalas <ta...@hotmail.com>
>
> *Cluster: 1 has HyperVisorType that does not match the VM, skipping this
> cluster*
> 你上传ISO或者模版的时候选的是什么OS type？
> 你给的信息太少，不能判断。
>
> 2013-05-17
> 刘宇超 Richard Liu
>
>
> 发件人： tao huang
> 发送时间： 2013-05-17 15:01:11
> 收件人： users-cn
> 抄送：
> 主题： Re: Re: 突然不能创建虚机，也不能重启虚机了
>
> 现 在在创centos5.4时报xenserver不支持VM，这明显是不对的，其他的没有信息，有没有人遇到这个问题？什么导致的呢？
> 2013-05-17 15:37:08,750 DEBUG [cloud.async.AsyncJobManagerImpl]
> (catalina-exec-21:null) submit async job-18223, details: AsyncJobVO
> {id:18223, userId: 2, accountId: 2, sessionKey: null, instanceType:
> VirtualMachine, instanceId: 2256, cmd: com.cloud.api.commands.DeployVMCmd,
> cmdOriginator: null, cmdInfo:
>
> {"sessionkey":"gaoCOyP2aUedht5GqNgomz3sM4M\u003d","ctxUserId":"2","serviceOfferingId":"ecde8005-0a75-4030-ab90-66153343599e","zoneId":"0bbbf7a8-7ef2-4822-b09b-d6be51862736","templateId":"1fec8aa7-571d-478f-a26a-1504937756d0","response":"json","id":"2256","networkIds":"064284bd-320c-4c31-a49d-e4e6169a2052","hypervisor":"XenServer,XenServer","name":"centos-test-ht","diskOfferingId":"fae8914b-a61c-49e7-93de-9be98957d2ed","_":"1368772888423","ctxAccountId":"2","ctxStartEventId":"130334","displayname":"centos-test-ht"},
> cmdVersion: 0, callbackType: 0, callbackAddress: null, status: 0,
> processStatus: 0, resultCode: 0, result: null, initMsid: 33932935549676,
> completeMsid: null, lastUpdated: null, lastPolled: null, created: null}
> 2013-05-17 15:37:08,751 DEBUG [cloud.async.AsyncJobManagerImpl]
> (Job-Executor-4:job-18223) Executing com.cloud.api.commands.DeployVMCmd for
> job-18223
> 2013-05-17 15:37:08,782 DEBUG [cloud.network.NetworkManagerImpl]
> (Job-Executor-4:job-18223) Service SecurityGroup is not supported in the
> network id=370
> 2013-05-17 15:37:08,785 DEBUG [cloud.network.NetworkManagerImpl]
> (Job-Executor-4:job-18223) Service SecurityGroup is not supported in the
> network id=370
> 2013-05-17 15:37:08,791 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-4:job-18223) VM state transitted from :Stopped to Starting
> with event: StartRequestedvm's original host id: null new host id: null
> host id before state transition: null
> 2013-05-17 15:37:08,791 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> (Job-Executor-4:job-18223) Successfully transitioned to start state for
> VM[User|centos-test-ht] reservation id =
> 52ce7276-4346-4cc1-a554-b03b1a57c363
> 2013-05-17 15:37:08,793 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> (Job-Executor-4:job-18223) Trying to deploy VM, vm has dcId: 1 and podId:
> null
> 2013-05-17 15:37:08,793 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> (Job-Executor-4:job-18223) Deploy avoids pods: null, clusters: null, hosts:
> null
> 2013-05-17 15:37:08,796 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-4:job-18223) DeploymentPlanner allocation algorithm: random
> 2013-05-17 15:37:08,796 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-4:job-18223) Trying to allocate a host and storage pools from
> dc:1, pod:null,cluster:null, requested cpu: 2000, requested ram: 2147483648
> 2013-05-17 15:37:08,796 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-4:job-18223) Is ROOT volume READY (pool already allocated)?:
> No
> 2013-05-17 15:37:08,796 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-4:job-18223) Searching all possible resources under this
> Zone: 1
> 2013-05-17 15:37:08,797 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-4:job-18223) Listing clusters in order of aggregate capacity,
> that have (atleast one host with) enough CPU and RAM capacity under this
> Zone: 1
> 2013-05-17 15:37:08,798 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-4:job-18223) CPUOverprovisioningFactor considered: 3.0
> 2013-05-17 15:37:08,805 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-4:job-18223) *Cluster: 1 has HyperVisorType that does not
> match the VM, skipping this cluster*
> *2013-05-17 15:37:08,805 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-4:job-18223) Could not find suitable Deployment Destination
> for this VM under any clusters, returning.*
> 2013-05-17 15:37:08,809 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-4:job-18223) VM state transitted from :Starting to Stopped
> with event: OperationFailedvm's original host id: null new host id: null
> host id before state transition: null
> 2013-05-17 15:37:08,812 DEBUG [cloud.vm.UserVmManagerImpl]
> (Job-Executor-4:job-18223) Destroying vm VM[User|centos-test-ht] as it
> failed to create
> 2013/5/16 tao huang <cl...@gmail.com>
> >
> >
> 这个环境是8个节点，我看了下，我看配置36是master，38是一个salve，但是38的xsconsole中状态异常了，没有ip，但是ifconfig看是正常的。一个slave不正常就会导致不能建虚机么？谢谢
> >
> >
> > 2013/5/16 tanthalas <ta...@hotmail.com>
> >
> >>
> >> 看来是CS找不到谁是pool master了，你要看看你的两个xenserver。
> >> 一般因为2个节点的集群，出现断网或者断电造成的。
> >> 去找xenserver对应的解决办法就好。
> >>
> >> 2013-05-16
> >> 刘宇超 Richard Liu
> >>
> >>
> >>
> >>
> >> 发件人： tao huang
> >> 发送时间： 2013-05-16 16:03:28
> >> 收件人： users-cn
> >> 抄送：
> >> 主题： Re: 突然不能创建虚机，也不能重启虚机了
> >>
> >> 这两个是两台host的IP
> >> 2013/5/16 tanthalas <ta...@hotmail.com>
> >> >
> >> > 先解释一下这俩ip是分配给谁的
> >> > 192.168.1.38
> >> > 192.168.1.36
> >> >
> >> >
> >> > 2013-05-16
> >> > 刘宇超 Richard Liu
> >> >
> >> >
> >> >
> >> >
> >> > 发件人： tao huang
> >> > 发送时间： 2013-05-16 15:50:33
> >> > 收件人： users-cn
> >> > 抄送：
> >> > 主题： 突然不能创建虚机，也不能重启虚机了
> >> >
> >> > 各位好，cloudstack 3.0 +xenserver6.0环境，高级网络
> >> > 突然无法创建虚机，感觉是主存储出了问题，但是看状态是正常的，如何排查呢，谢谢指导
> >> > 2013-05-16 15:29:40,062 DEBUG [agent.manager.DirectAgentAttache]
> >> > (DirectAgent-369:null) Seq 3-939886253: Response Received:
> >> > 2013-05-16 15:29:40,062 DEBUG [agent.transport.Request]
> >> > (DirectAgent-369:null) Seq 3-939886253: Processing: { Ans: , MgmtId:
> >> > 33932935549676, via: 3, Ver: v1, Flags: 110,
> >> > [{"StopAnswer":{"result":false,"details":"Exception:
> >> > com.cloud.utils.exception.CloudRuntimeException\nMessage: *Unable to
> >> reset
> >> > master of slave 192.168.1.38 to 192.168.1.36 due to
> >> > org.apache.xmlrpc.XmlRpcException: Failed to create input stream: Read
> >> > timed out\nStack: com.cloud.utils.exception.CloudRuntimeException:
> >> Unable
> >> > to reset master of slave 192.168.1.38 to 192.168.1.36 due to
> >> > org.apache.xmlrpc.*XmlRpcException: Failed to create input stream:
> Read
> >> > timed out\n\tat
> >> >
> >> >
> >>
> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.PoolEmergencyResetMaster(XenServerConnectionPool.java:439)\n\tat
> >> >
> >> >
> >>
> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.connect(XenServerConnectionPool.java:657)\n\tat
> >> >
> >> >
> >>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getConnection(CitrixResourceBase.java:5090)\n\tat
> >> >
> >> >
> >>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3284)\n\tat
> >> >
> >> >
> >>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:416)\n\tat
> >> >
> >> >
> >>
> com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:69)\n\tat
> >> >
> >> >
> >>
> com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)\n\tat
> >> >
> >> >
> >>
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)\n\tat
> >> >
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)\n\tat
> >> > java.util.concurrent.FutureTask.run(FutureTask.java:166)\n\tat
> >> >
> >> >
> >>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)\n\tat
> >> >
> >> >
> >>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)\n\tat
> >> >
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)\n\tat
> >> >
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)\n\tat
> >> > java.lang.Thread.run(Thread.java:636)\n","wait":0}}] }
> >> > 2013-05-16 15:29:40,062 DEBUG [agent.manager.AgentAttache]
> >> > (DirectAgent-369:null) Seq 3-939886253: No more commands found
> >> > 2013-05-16 15:29:40,062 DEBUG [agent.transport.Request]
> >> > (Job-Executor-6:job-18181) Seq 3-939886253: Received: { Ans: ,
> MgmtId:
> >> > 33932935549676, via: 3, Ver: v1, Flags: 110, { StopAnswer } }
> >> > 2013-05-16 15:29:40,062 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> >> > (Job-Executor-6:job-18181) Unable to stop VM due to Exception:
> >> > com.cloud.utils.exception.CloudRuntimeException
> >> > Message: Unable to reset master of slave 192.168.1.38 to 192.168.1.36
> >> due
> >> > to org.apache.xmlrpc.XmlRpcException: Failed to create input stream:
> >> Read
> >> > timed out
> >> > Stack: com.cloud.utils.exception.CloudRuntimeException: Unable to
> reset
> >> > master of slave 192.168.1.38 to 192.168.1.36 due to
> >> > org.apache.xmlrpc.XmlRpcException: Failed to create input stream: Read
> >> > timed out
> >> > at
> >> >
> >> >
> >>
> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.PoolEmergencyResetMaster(XenServerConnectionPool.java:439)
> >> > at
> >> >
> >> >
> >>
> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.connect(XenServerConnectionPool.java:657)
> >> > at
> >> >
> >> >
> >>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getConnection(CitrixResourceBase.java:5090)
> >> > at
> >> >
> >> >
> >>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3284)
> >> > at
> >> >
> >> >
> >>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:416)
> >> > at
> >> >
> >> >
> >>
> com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:69)
> >> > at
> >> >
> >> >
> >>
> com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)
> >> > at
> >> >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> >> > at
> >> > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> >> > at
> >> >
> >> >
> >>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> >> > at
> >> >
> >> >
> >>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> >> > at
> >> >
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> >> > at
> >> >
> >> >
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> >> > at java.lang.Thread.run(Thread.java:636)
> >> > 2013-05-16 15:29:40,062 WARN [cloud.vm.VirtualMachineManagerImpl]
> >> > (Job-Executor-6:job-18181) Failed to stop vm
> VM[DomainRouter|r-2237-VM]
> >> in
> >> > Starting state as a part of cleanup process
> >> > 2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
> >> > (Job-Executor-6:job-18181) DeploymentPlanner allocation algorithm:
> >> random
> >> > 2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
> >> > (Job-Executor-6:job-18181) Trying to allocate a host and storage pools
> >> from
> >> > dc:1, pod:null,cluster:null, requested cpu: 500, requested ram:
> >> 134217728
> >> > 2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
> >> > (Job-Executor-6:job-18181) Is ROOT volume READY (pool already
> >> allocated)?:
> >> > No
> >> > 2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
> >> > (Job-Executor-6:job-18181) Searching all possible resources under this
> >> > Zone: 1
> >> > 2013-05-16 15:29:40,068 DEBUG [cloud.deploy.FirstFitPlanner]
> >> > (Job-Executor-6:job-18181) Listing clusters in order of aggregate
> >> capacity,
> >> > that have (atleast one host with) enough CPU and RAM capacity under
> this
> >> > Zone: 1
> >> > 2013-05-16 15:29:40,069 DEBUG [cloud.deploy.FirstFitPlanner]
> >> > (Job-Executor-6:job-18181) CPUOverprovisioningFactor considered: 3.0
> >> > 2013-05-16 15:29:40,077 DEBUG [cloud.deploy.FirstFitPlanner]
> >> > (Job-Executor-6:job-18181) Checking resources in Cluster: 1 under
> Pod: 1
> >> > 2013-05-16 15:29:40,077 DEBUG [cloud.deploy.FirstFitPlanner]
> >> > (Job-Executor-6:job-18181) Calling HostAllocators to find suitable
> hosts
> >> > 2013-05-16 15:29:40,077 DEBUG [allocator.impl.FirstFitAllocator]
> >> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Looking for hosts
> in
> >> > dc: 1 pod:1 cluster:1
> >> > 2013-05-16 15:29:40,080 DEBUG [allocator.impl.FirstFitAllocator]
> >> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) FirstFitAllocator
> >> has 7
> >> > hosts to check for allocation: [Host[-4-Routing], Host[-2-Routing],
> >> > Host[-1-Routing], Host[-3-Routing], Host[-8-Routing],
> Host[-7-Routing],
> >> > Host[-15-Routing]]
> >> > 2013-05-16 15:29:40,088 DEBUG [allocator.impl.FirstFitAllocator]
> >> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found 7 hosts for
> >> > allocation after prioritization: [Host[-4-Routing], Host[-2-Routing],
> >> > Host[-1-Routing], Host[-3-Routing], Host[-8-Routing],
> Host[-7-Routing],
> >> > Host[-15-Routing]]
> >> > 2013-05-16 15:29:40,088 DEBUG [allocator.impl.FirstFitAllocator]
> >> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Looking for
> >> > speed=500Mhz, Ram=128
> >> > 2013-05-16 15:29:40,091 DEBUG [cloud.capacity.CapacityManagerImpl]
> >> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Checking if host:
> 4
> >> has
> >> > enough capacity for requested CPU: 500 and requested RAM: 134217728 ,
> >> > cpuOverprovisioningFactor: 3.0
> >> > 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
> >> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Hosts's actual
> total
> >> > CPU: 63984 and CPU after applying overprovisioning: 191952
> >> > 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
> >> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free CPU: 161952 ,
> >> > Requested CPU: 500
> >> > 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
> >> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free RAM:
> >> 71156002944 ,
> >> > Requested RAM: 134217728
> >> > 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
> >> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Host has enough
> CPU
> >> and
> >> > RAM available
> >> > 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
> >> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc
> CPU
> >> > from host: 4, used: 29500, reserved: 500, actual total: 63984, total
> >> with
> >> > overprovisioning: 191952; requested
> cpu:500,alloc_from_last_host?:false
> >> > ,considerReservedCapacity?: true
> >> > 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
> >> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc
> MEM
> >> > from host: 4, used: 28856811520, reserved: 536870912, total:
> >> 100549685376;
> >> > requested mem: 134217728,alloc_from_last_host?:false
> >> > ,considerReservedCapacity?: true
> >> > 2013-05-16 15:29:40,093 DEBUG [allocator.impl.FirstFitAllocator]
> >> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found a suitable
> >> host,
> >> > adding to list: 4
> >> > 2013-05-16 15:29:40,095 DEBUG [cloud.capacity.CapacityManagerImpl]
> >> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Checking if host:
> 2
> >> has
> >> > enough capacity for requested CPU: 500 and requested RAM: 134217728 ,
> >> > cpuOverprovisioningFactor: 3.0
> >> > 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
> >> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Hosts's actual
> total
> >> > CPU: 63984 and CPU after applying overprovisioning: 191952
> >> > 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
> >> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free CPU: 151452 ,
> >> > Requested CPU: 500
> >> > 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
> >> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free RAM:
> >> 58539536512 ,
> >> > Requested RAM: 134217728
> >> > 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
> >> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Host has enough
> CPU
> >> and
> >> > RAM available
> >> > 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
> >> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc
> CPU
> >> > from host: 2, used: 40500, reserved: 0, actual total: 63984, total
> with
> >> > overprovisioning: 191952; requested
> cpu:500,alloc_from_last_host?:false
> >> > ,considerReservedCapacity?: true
> >> > 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
> >> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc
> MEM
> >> > from host: 2, used: 42010148864, reserved: 0, total: 100549685376;
> >> > requested mem: 134217728,alloc_from_last_host?:false
> >> > ,considerReservedCapacity?: true
> >> > 2013-05-16 15:29:40,097 DEBUG [allocator.impl.FirstFitAllocator]
> >> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found a suitable
> >> host,
> >> > adding to list: 2
> >> > 2013-05-16 15:29:40,114 DEBUG [allocator.impl.FirstFitAllocator]
> >> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found a suitable
> >> host,
> >> > adding to list: 15
> >> > *2013-05-16 15:29:40,114 DEBUG [allocator.impl.FirstFitAllocator]
> >> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Host Allocator
> >> > returning 6 suitable hosts*
> >> > *2013-05-16 15:29:40,116 DEBUG [cloud.deploy.FirstFitPlanner]
> >> > (Job-Executor-6:job-18181) Checking suitable pools for volume (Id,
> >> Type):
> >> > (2438,ROOT)*
> >> > *2013-05-16 15:29:40,116 DEBUG [cloud.deploy.FirstFitPlanner]
> >> > (Job-Executor-6:job-18181) We need to allocate new storagepool for
> this
> >> > volume*
> >> > *2013-05-16 15:29:40,116 DEBUG [cloud.deploy.FirstFitPlanner]
> >> > (Job-Executor-6:job-18181) Calling StoragePoolAllocators to find
> >> suitable
> >> > pools*
> >> > *2013-05-16 15:29:40,117 DEBUG
> >> > [storage.allocator.FirstFitStoragePoolAllocator]
> >> (Job-Executor-6:job-18181)
> >> > Looking for pools in dc: 1 pod:1 cluster:1*
> >> > *2013-05-16 15:29:40,118 DEBUG
> >> > [storage.allocator.FirstFitStoragePoolAllocator]
> >> (Job-Executor-6:job-18181)
> >> > FirstFitStoragePoolAllocator has 1 pools to check for allocation*
> >> > *2013-05-16 15:29:40,118 DEBUG
> >> > [storage.allocator.AbstractStoragePoolAllocator]
> >> (Job-Executor-6:job-18181)
> >> > Checking if storage pool is suitable, name: ps ,poolId: 200*
> >> > *2013-05-16 15:29:40,118 DEBUG
> >> > [storage.allocator.AbstractStoragePoolAllocator]
> >> (Job-Executor-6:job-18181)
> >> > StoragePool is in avoid set, skipping this pool*
> >> > *2013-05-16 15:29:40,118 DEBUG
> >> > [storage.allocator.FirstFitStoragePoolAllocator]
> >> (Job-Executor-6:job-18181)
> >> > FirstFitStoragePoolAllocator returning 0 suitable storage pools*
> >> > *2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
> >> > (Job-Executor-6:job-18181) No suitable pools found for volume:
> >> > Vol[2438|vm=2237|ROOT] under cluster: 1*
> >> > *2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
> >> > (Job-Executor-6:job-18181) No suitable pools found*
> >> > *2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
> >> > (Job-Executor-6:job-18181) No suitable storagePools found under this
> >> > Cluster: 1*
> >> > *2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
> >> > (Job-Executor-6:job-18181) Could not find suitable Deployment
> >> Destination
> >> > for this VM under any clusters, returning.*
> >> > *2013-05-16 15:29:40,135 DEBUG [cloud.capacity.CapacityManagerImpl]
> >> > (Job-Executor-6:job-18181) VM state transitted from :Starting to
> Stopped
> >> > with event: OperationFailedvm's original host id: null new host id:
> null
> >> > host id before state transition: 3*
> >> > *2013-05-16 15:29:40,139 DEBUG [cloud.capacity.CapacityManagerImpl]
> >> > (Job-Executor-6:job-18181) Hosts's actual total CPU: 63984 and CPU
> after
> >> > applying overprovisioning: 191952*
> >> > *2013-05-16 15:29:40,139 DEBUG [cloud.capacity.CapacityManagerImpl]
> >> > (Job-Executor-6:job-18181) release cpu from host: 3, old used:
> >> > 29000,reserved: 0, actual total: 63984, total with overprovisioning:
> >> > 191952; new used: 28500,reserved:0; movedfromreserved:
> >> > false,moveToReserveredfalse*
> >> >
> >>
> >
> >
>
Re: Re: Re: 突然不能创建虚机，也不能重启虚机了
Posted by tanthalas <ta...@hotmail.com>.
*Cluster: 1 has HyperVisorType that does not match the VM, skipping this cluster*
你上传ISO或者模版的时候选的是什么OS type？
你给的信息太少，不能判断。
2013-05-17
刘宇超 Richard Liu
发件人： tao huang
发送时间： 2013-05-17 15:01:11
收件人： users-cn
抄送：
主题： Re: Re: 突然不能创建虚机，也不能重启虚机了
现 在在创centos5.4时报xenserver不支持VM，这明显是不对的，其他的没有信息，有没有人遇到这个问题？什么导致的呢？
2013-05-17 15:37:08,750 DEBUG [cloud.async.AsyncJobManagerImpl]
(catalina-exec-21:null) submit async job-18223, details: AsyncJobVO
{id:18223, userId: 2, accountId: 2, sessionKey: null, instanceType:
VirtualMachine, instanceId: 2256, cmd: com.cloud.api.commands.DeployVMCmd,
cmdOriginator: null, cmdInfo:
{"sessionkey":"gaoCOyP2aUedht5GqNgomz3sM4M\u003d","ctxUserId":"2","serviceOfferingId":"ecde8005-0a75-4030-ab90-66153343599e","zoneId":"0bbbf7a8-7ef2-4822-b09b-d6be51862736","templateId":"1fec8aa7-571d-478f-a26a-1504937756d0","response":"json","id":"2256","networkIds":"064284bd-320c-4c31-a49d-e4e6169a2052","hypervisor":"XenServer,XenServer","name":"centos-test-ht","diskOfferingId":"fae8914b-a61c-49e7-93de-9be98957d2ed","_":"1368772888423","ctxAccountId":"2","ctxStartEventId":"130334","displayname":"centos-test-ht"},
cmdVersion: 0, callbackType: 0, callbackAddress: null, status: 0,
processStatus: 0, resultCode: 0, result: null, initMsid: 33932935549676,
completeMsid: null, lastUpdated: null, lastPolled: null, created: null}
2013-05-17 15:37:08,751 DEBUG [cloud.async.AsyncJobManagerImpl]
(Job-Executor-4:job-18223) Executing com.cloud.api.commands.DeployVMCmd for
job-18223
2013-05-17 15:37:08,782 DEBUG [cloud.network.NetworkManagerImpl]
(Job-Executor-4:job-18223) Service SecurityGroup is not supported in the
network id=370
2013-05-17 15:37:08,785 DEBUG [cloud.network.NetworkManagerImpl]
(Job-Executor-4:job-18223) Service SecurityGroup is not supported in the
network id=370
2013-05-17 15:37:08,791 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-4:job-18223) VM state transitted from :Stopped to Starting
with event: StartRequestedvm's original host id: null new host id: null
host id before state transition: null
2013-05-17 15:37:08,791 DEBUG [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-4:job-18223) Successfully transitioned to start state for
VM[User|centos-test-ht] reservation id =
52ce7276-4346-4cc1-a554-b03b1a57c363
2013-05-17 15:37:08,793 DEBUG [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-4:job-18223) Trying to deploy VM, vm has dcId: 1 and podId:
null
2013-05-17 15:37:08,793 DEBUG [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-4:job-18223) Deploy avoids pods: null, clusters: null, hosts:
null
2013-05-17 15:37:08,796 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-4:job-18223) DeploymentPlanner allocation algorithm: random
2013-05-17 15:37:08,796 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-4:job-18223) Trying to allocate a host and storage pools from
dc:1, pod:null,cluster:null, requested cpu: 2000, requested ram: 2147483648
2013-05-17 15:37:08,796 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-4:job-18223) Is ROOT volume READY (pool already allocated)?:
No
2013-05-17 15:37:08,796 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-4:job-18223) Searching all possible resources under this
Zone: 1
2013-05-17 15:37:08,797 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-4:job-18223) Listing clusters in order of aggregate capacity,
that have (atleast one host with) enough CPU and RAM capacity under this
Zone: 1
2013-05-17 15:37:08,798 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-4:job-18223) CPUOverprovisioningFactor considered: 3.0
2013-05-17 15:37:08,805 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-4:job-18223) *Cluster: 1 has HyperVisorType that does not
match the VM, skipping this cluster*
*2013-05-17 15:37:08,805 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-4:job-18223) Could not find suitable Deployment Destination
for this VM under any clusters, returning.*
2013-05-17 15:37:08,809 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-4:job-18223) VM state transitted from :Starting to Stopped
with event: OperationFailedvm's original host id: null new host id: null
host id before state transition: null
2013-05-17 15:37:08,812 DEBUG [cloud.vm.UserVmManagerImpl]
(Job-Executor-4:job-18223) Destroying vm VM[User|centos-test-ht] as it
failed to create
2013/5/16 tao huang <cl...@gmail.com>
>
> 这个环境是8个节点，我看了下，我看配置36是master，38是一个salve，但是38的xsconsole中状态异常了，没有ip，但是ifconfig看是正常的。一个slave不正常就会导致不能建虚机么？谢谢
>
>
> 2013/5/16 tanthalas <ta...@hotmail.com>
>
>>
>> 看来是CS找不到谁是pool master了，你要看看你的两个xenserver。
>> 一般因为2个节点的集群，出现断网或者断电造成的。
>> 去找xenserver对应的解决办法就好。
>>
>> 2013-05-16
>> 刘宇超 Richard Liu
>>
>>
>>
>>
>> 发件人： tao huang
>> 发送时间： 2013-05-16 16:03:28
>> 收件人： users-cn
>> 抄送：
>> 主题： Re: 突然不能创建虚机，也不能重启虚机了
>>
>> 这两个是两台host的IP
>> 2013/5/16 tanthalas <ta...@hotmail.com>
>> >
>> > 先解释一下这俩ip是分配给谁的
>> > 192.168.1.38
>> > 192.168.1.36
>> >
>> >
>> > 2013-05-16
>> > 刘宇超 Richard Liu
>> >
>> >
>> >
>> >
>> > 发件人： tao huang
>> > 发送时间： 2013-05-16 15:50:33
>> > 收件人： users-cn
>> > 抄送：
>> > 主题： 突然不能创建虚机，也不能重启虚机了
>> >
>> > 各位好，cloudstack 3.0 +xenserver6.0环境，高级网络
>> > 突然无法创建虚机，感觉是主存储出了问题，但是看状态是正常的，如何排查呢，谢谢指导
>> > 2013-05-16 15:29:40,062 DEBUG [agent.manager.DirectAgentAttache]
>> > (DirectAgent-369:null) Seq 3-939886253: Response Received:
>> > 2013-05-16 15:29:40,062 DEBUG [agent.transport.Request]
>> > (DirectAgent-369:null) Seq 3-939886253: Processing: { Ans: , MgmtId:
>> > 33932935549676, via: 3, Ver: v1, Flags: 110,
>> > [{"StopAnswer":{"result":false,"details":"Exception:
>> > com.cloud.utils.exception.CloudRuntimeException\nMessage: *Unable to
>> reset
>> > master of slave 192.168.1.38 to 192.168.1.36 due to
>> > org.apache.xmlrpc.XmlRpcException: Failed to create input stream: Read
>> > timed out\nStack: com.cloud.utils.exception.CloudRuntimeException:
>> Unable
>> > to reset master of slave 192.168.1.38 to 192.168.1.36 due to
>> > org.apache.xmlrpc.*XmlRpcException: Failed to create input stream: Read
>> > timed out\n\tat
>> >
>> >
>> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.PoolEmergencyResetMaster(XenServerConnectionPool.java:439)\n\tat
>> >
>> >
>> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.connect(XenServerConnectionPool.java:657)\n\tat
>> >
>> >
>> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getConnection(CitrixResourceBase.java:5090)\n\tat
>> >
>> >
>> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3284)\n\tat
>> >
>> >
>> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:416)\n\tat
>> >
>> >
>> com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:69)\n\tat
>> >
>> >
>> com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)\n\tat
>> >
>> >
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)\n\tat
>> > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)\n\tat
>> > java.util.concurrent.FutureTask.run(FutureTask.java:166)\n\tat
>> >
>> >
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)\n\tat
>> >
>> >
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)\n\tat
>> >
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)\n\tat
>> >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)\n\tat
>> > java.lang.Thread.run(Thread.java:636)\n","wait":0}}] }
>> > 2013-05-16 15:29:40,062 DEBUG [agent.manager.AgentAttache]
>> > (DirectAgent-369:null) Seq 3-939886253: No more commands found
>> > 2013-05-16 15:29:40,062 DEBUG [agent.transport.Request]
>> > (Job-Executor-6:job-18181) Seq 3-939886253: Received: { Ans: , MgmtId:
>> > 33932935549676, via: 3, Ver: v1, Flags: 110, { StopAnswer } }
>> > 2013-05-16 15:29:40,062 DEBUG [cloud.vm.VirtualMachineManagerImpl]
>> > (Job-Executor-6:job-18181) Unable to stop VM due to Exception:
>> > com.cloud.utils.exception.CloudRuntimeException
>> > Message: Unable to reset master of slave 192.168.1.38 to 192.168.1.36
>> due
>> > to org.apache.xmlrpc.XmlRpcException: Failed to create input stream:
>> Read
>> > timed out
>> > Stack: com.cloud.utils.exception.CloudRuntimeException: Unable to reset
>> > master of slave 192.168.1.38 to 192.168.1.36 due to
>> > org.apache.xmlrpc.XmlRpcException: Failed to create input stream: Read
>> > timed out
>> > at
>> >
>> >
>> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.PoolEmergencyResetMaster(XenServerConnectionPool.java:439)
>> > at
>> >
>> >
>> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.connect(XenServerConnectionPool.java:657)
>> > at
>> >
>> >
>> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getConnection(CitrixResourceBase.java:5090)
>> > at
>> >
>> >
>> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3284)
>> > at
>> >
>> >
>> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:416)
>> > at
>> >
>> >
>> com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:69)
>> > at
>> >
>> >
>> com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)
>> > at
>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> > at
>> > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>> > at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>> > at
>> >
>> >
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
>> > at
>> >
>> >
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
>> > at
>> >
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>> > at
>> >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>> > at java.lang.Thread.run(Thread.java:636)
>> > 2013-05-16 15:29:40,062 WARN [cloud.vm.VirtualMachineManagerImpl]
>> > (Job-Executor-6:job-18181) Failed to stop vm VM[DomainRouter|r-2237-VM]
>> in
>> > Starting state as a part of cleanup process
>> > 2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) DeploymentPlanner allocation algorithm:
>> random
>> > 2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) Trying to allocate a host and storage pools
>> from
>> > dc:1, pod:null,cluster:null, requested cpu: 500, requested ram:
>> 134217728
>> > 2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) Is ROOT volume READY (pool already
>> allocated)?:
>> > No
>> > 2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) Searching all possible resources under this
>> > Zone: 1
>> > 2013-05-16 15:29:40,068 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) Listing clusters in order of aggregate
>> capacity,
>> > that have (atleast one host with) enough CPU and RAM capacity under this
>> > Zone: 1
>> > 2013-05-16 15:29:40,069 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) CPUOverprovisioningFactor considered: 3.0
>> > 2013-05-16 15:29:40,077 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) Checking resources in Cluster: 1 under Pod: 1
>> > 2013-05-16 15:29:40,077 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) Calling HostAllocators to find suitable hosts
>> > 2013-05-16 15:29:40,077 DEBUG [allocator.impl.FirstFitAllocator]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Looking for hosts in
>> > dc: 1 pod:1 cluster:1
>> > 2013-05-16 15:29:40,080 DEBUG [allocator.impl.FirstFitAllocator]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) FirstFitAllocator
>> has 7
>> > hosts to check for allocation: [Host[-4-Routing], Host[-2-Routing],
>> > Host[-1-Routing], Host[-3-Routing], Host[-8-Routing], Host[-7-Routing],
>> > Host[-15-Routing]]
>> > 2013-05-16 15:29:40,088 DEBUG [allocator.impl.FirstFitAllocator]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found 7 hosts for
>> > allocation after prioritization: [Host[-4-Routing], Host[-2-Routing],
>> > Host[-1-Routing], Host[-3-Routing], Host[-8-Routing], Host[-7-Routing],
>> > Host[-15-Routing]]
>> > 2013-05-16 15:29:40,088 DEBUG [allocator.impl.FirstFitAllocator]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Looking for
>> > speed=500Mhz, Ram=128
>> > 2013-05-16 15:29:40,091 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Checking if host: 4
>> has
>> > enough capacity for requested CPU: 500 and requested RAM: 134217728 ,
>> > cpuOverprovisioningFactor: 3.0
>> > 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Hosts's actual total
>> > CPU: 63984 and CPU after applying overprovisioning: 191952
>> > 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free CPU: 161952 ,
>> > Requested CPU: 500
>> > 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free RAM:
>> 71156002944 ,
>> > Requested RAM: 134217728
>> > 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Host has enough CPU
>> and
>> > RAM available
>> > 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc CPU
>> > from host: 4, used: 29500, reserved: 500, actual total: 63984, total
>> with
>> > overprovisioning: 191952; requested cpu:500,alloc_from_last_host?:false
>> > ,considerReservedCapacity?: true
>> > 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc MEM
>> > from host: 4, used: 28856811520, reserved: 536870912, total:
>> 100549685376;
>> > requested mem: 134217728,alloc_from_last_host?:false
>> > ,considerReservedCapacity?: true
>> > 2013-05-16 15:29:40,093 DEBUG [allocator.impl.FirstFitAllocator]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found a suitable
>> host,
>> > adding to list: 4
>> > 2013-05-16 15:29:40,095 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Checking if host: 2
>> has
>> > enough capacity for requested CPU: 500 and requested RAM: 134217728 ,
>> > cpuOverprovisioningFactor: 3.0
>> > 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Hosts's actual total
>> > CPU: 63984 and CPU after applying overprovisioning: 191952
>> > 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free CPU: 151452 ,
>> > Requested CPU: 500
>> > 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free RAM:
>> 58539536512 ,
>> > Requested RAM: 134217728
>> > 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Host has enough CPU
>> and
>> > RAM available
>> > 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc CPU
>> > from host: 2, used: 40500, reserved: 0, actual total: 63984, total with
>> > overprovisioning: 191952; requested cpu:500,alloc_from_last_host?:false
>> > ,considerReservedCapacity?: true
>> > 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc MEM
>> > from host: 2, used: 42010148864, reserved: 0, total: 100549685376;
>> > requested mem: 134217728,alloc_from_last_host?:false
>> > ,considerReservedCapacity?: true
>> > 2013-05-16 15:29:40,097 DEBUG [allocator.impl.FirstFitAllocator]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found a suitable
>> host,
>> > adding to list: 2
>> > 2013-05-16 15:29:40,114 DEBUG [allocator.impl.FirstFitAllocator]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found a suitable
>> host,
>> > adding to list: 15
>> > *2013-05-16 15:29:40,114 DEBUG [allocator.impl.FirstFitAllocator]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Host Allocator
>> > returning 6 suitable hosts*
>> > *2013-05-16 15:29:40,116 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) Checking suitable pools for volume (Id,
>> Type):
>> > (2438,ROOT)*
>> > *2013-05-16 15:29:40,116 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) We need to allocate new storagepool for this
>> > volume*
>> > *2013-05-16 15:29:40,116 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) Calling StoragePoolAllocators to find
>> suitable
>> > pools*
>> > *2013-05-16 15:29:40,117 DEBUG
>> > [storage.allocator.FirstFitStoragePoolAllocator]
>> (Job-Executor-6:job-18181)
>> > Looking for pools in dc: 1 pod:1 cluster:1*
>> > *2013-05-16 15:29:40,118 DEBUG
>> > [storage.allocator.FirstFitStoragePoolAllocator]
>> (Job-Executor-6:job-18181)
>> > FirstFitStoragePoolAllocator has 1 pools to check for allocation*
>> > *2013-05-16 15:29:40,118 DEBUG
>> > [storage.allocator.AbstractStoragePoolAllocator]
>> (Job-Executor-6:job-18181)
>> > Checking if storage pool is suitable, name: ps ,poolId: 200*
>> > *2013-05-16 15:29:40,118 DEBUG
>> > [storage.allocator.AbstractStoragePoolAllocator]
>> (Job-Executor-6:job-18181)
>> > StoragePool is in avoid set, skipping this pool*
>> > *2013-05-16 15:29:40,118 DEBUG
>> > [storage.allocator.FirstFitStoragePoolAllocator]
>> (Job-Executor-6:job-18181)
>> > FirstFitStoragePoolAllocator returning 0 suitable storage pools*
>> > *2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) No suitable pools found for volume:
>> > Vol[2438|vm=2237|ROOT] under cluster: 1*
>> > *2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) No suitable pools found*
>> > *2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) No suitable storagePools found under this
>> > Cluster: 1*
>> > *2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) Could not find suitable Deployment
>> Destination
>> > for this VM under any clusters, returning.*
>> > *2013-05-16 15:29:40,135 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181) VM state transitted from :Starting to Stopped
>> > with event: OperationFailedvm's original host id: null new host id: null
>> > host id before state transition: 3*
>> > *2013-05-16 15:29:40,139 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181) Hosts's actual total CPU: 63984 and CPU after
>> > applying overprovisioning: 191952*
>> > *2013-05-16 15:29:40,139 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181) release cpu from host: 3, old used:
>> > 29000,reserved: 0, actual total: 63984, total with overprovisioning:
>> > 191952; new used: 28500,reserved:0; movedfromreserved:
>> > false,moveToReserveredfalse*
>> >
>>
>
>
Re: Re: 突然不能创建虚机，也不能重启虚机了
Posted by tao huang <cl...@gmail.com>.
现 在在创centos5.4时报xenserver不支持VM，这明显是不对的，其他的没有信息，有没有人遇到这个问题？什么导致的呢？
2013-05-17 15:37:08,750 DEBUG [cloud.async.AsyncJobManagerImpl]
(catalina-exec-21:null) submit async job-18223, details: AsyncJobVO
{id:18223, userId: 2, accountId: 2, sessionKey: null, instanceType:
VirtualMachine, instanceId: 2256, cmd: com.cloud.api.commands.DeployVMCmd,
cmdOriginator: null, cmdInfo:
{"sessionkey":"gaoCOyP2aUedht5GqNgomz3sM4M\u003d","ctxUserId":"2","serviceOfferingId":"ecde8005-0a75-4030-ab90-66153343599e","zoneId":"0bbbf7a8-7ef2-4822-b09b-d6be51862736","templateId":"1fec8aa7-571d-478f-a26a-1504937756d0","response":"json","id":"2256","networkIds":"064284bd-320c-4c31-a49d-e4e6169a2052","hypervisor":"XenServer,XenServer","name":"centos-test-ht","diskOfferingId":"fae8914b-a61c-49e7-93de-9be98957d2ed","_":"1368772888423","ctxAccountId":"2","ctxStartEventId":"130334","displayname":"centos-test-ht"},
cmdVersion: 0, callbackType: 0, callbackAddress: null, status: 0,
processStatus: 0, resultCode: 0, result: null, initMsid: 33932935549676,
completeMsid: null, lastUpdated: null, lastPolled: null, created: null}
2013-05-17 15:37:08,751 DEBUG [cloud.async.AsyncJobManagerImpl]
(Job-Executor-4:job-18223) Executing com.cloud.api.commands.DeployVMCmd for
job-18223
2013-05-17 15:37:08,782 DEBUG [cloud.network.NetworkManagerImpl]
(Job-Executor-4:job-18223) Service SecurityGroup is not supported in the
network id=370
2013-05-17 15:37:08,785 DEBUG [cloud.network.NetworkManagerImpl]
(Job-Executor-4:job-18223) Service SecurityGroup is not supported in the
network id=370
2013-05-17 15:37:08,791 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-4:job-18223) VM state transitted from :Stopped to Starting
with event: StartRequestedvm's original host id: null new host id: null
host id before state transition: null
2013-05-17 15:37:08,791 DEBUG [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-4:job-18223) Successfully transitioned to start state for
VM[User|centos-test-ht] reservation id =
52ce7276-4346-4cc1-a554-b03b1a57c363
2013-05-17 15:37:08,793 DEBUG [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-4:job-18223) Trying to deploy VM, vm has dcId: 1 and podId:
null
2013-05-17 15:37:08,793 DEBUG [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-4:job-18223) Deploy avoids pods: null, clusters: null, hosts:
null
2013-05-17 15:37:08,796 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-4:job-18223) DeploymentPlanner allocation algorithm: random
2013-05-17 15:37:08,796 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-4:job-18223) Trying to allocate a host and storage pools from
dc:1, pod:null,cluster:null, requested cpu: 2000, requested ram: 2147483648
2013-05-17 15:37:08,796 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-4:job-18223) Is ROOT volume READY (pool already allocated)?:
No
2013-05-17 15:37:08,796 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-4:job-18223) Searching all possible resources under this
Zone: 1
2013-05-17 15:37:08,797 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-4:job-18223) Listing clusters in order of aggregate capacity,
that have (atleast one host with) enough CPU and RAM capacity under this
Zone: 1
2013-05-17 15:37:08,798 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-4:job-18223) CPUOverprovisioningFactor considered: 3.0
2013-05-17 15:37:08,805 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-4:job-18223) *Cluster: 1 has HyperVisorType that does not
match the VM, skipping this cluster*
*2013-05-17 15:37:08,805 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-4:job-18223) Could not find suitable Deployment Destination
for this VM under any clusters, returning.*
2013-05-17 15:37:08,809 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-4:job-18223) VM state transitted from :Starting to Stopped
with event: OperationFailedvm's original host id: null new host id: null
host id before state transition: null
2013-05-17 15:37:08,812 DEBUG [cloud.vm.UserVmManagerImpl]
(Job-Executor-4:job-18223) Destroying vm VM[User|centos-test-ht] as it
failed to create
2013/5/16 tao huang <cl...@gmail.com>
>
> 这个环境是8个节点，我看了下，我看配置36是master，38是一个salve，但是38的xsconsole中状态异常了，没有ip，但是ifconfig看是正常的。一个slave不正常就会导致不能建虚机么？谢谢
>
>
> 2013/5/16 tanthalas <ta...@hotmail.com>
>
>>
>> 看来是CS找不到谁是pool master了，你要看看你的两个xenserver。
>> 一般因为2个节点的集群，出现断网或者断电造成的。
>> 去找xenserver对应的解决办法就好。
>>
>> 2013-05-16
>> 刘宇超 Richard Liu
>>
>>
>>
>>
>> 发件人： tao huang
>> 发送时间： 2013-05-16 16:03:28
>> 收件人： users-cn
>> 抄送：
>> 主题： Re: 突然不能创建虚机，也不能重启虚机了
>>
>> 这两个是两台host的IP
>> 2013/5/16 tanthalas <ta...@hotmail.com>
>> >
>> > 先解释一下这俩ip是分配给谁的
>> > 192.168.1.38
>> > 192.168.1.36
>> >
>> >
>> > 2013-05-16
>> > 刘宇超 Richard Liu
>> >
>> >
>> >
>> >
>> > 发件人： tao huang
>> > 发送时间： 2013-05-16 15:50:33
>> > 收件人： users-cn
>> > 抄送：
>> > 主题： 突然不能创建虚机，也不能重启虚机了
>> >
>> > 各位好，cloudstack 3.0 +xenserver6.0环境，高级网络
>> > 突然无法创建虚机，感觉是主存储出了问题，但是看状态是正常的，如何排查呢，谢谢指导
>> > 2013-05-16 15:29:40,062 DEBUG [agent.manager.DirectAgentAttache]
>> > (DirectAgent-369:null) Seq 3-939886253: Response Received:
>> > 2013-05-16 15:29:40,062 DEBUG [agent.transport.Request]
>> > (DirectAgent-369:null) Seq 3-939886253: Processing: { Ans: , MgmtId:
>> > 33932935549676, via: 3, Ver: v1, Flags: 110,
>> > [{"StopAnswer":{"result":false,"details":"Exception:
>> > com.cloud.utils.exception.CloudRuntimeException\nMessage: *Unable to
>> reset
>> > master of slave 192.168.1.38 to 192.168.1.36 due to
>> > org.apache.xmlrpc.XmlRpcException: Failed to create input stream: Read
>> > timed out\nStack: com.cloud.utils.exception.CloudRuntimeException:
>> Unable
>> > to reset master of slave 192.168.1.38 to 192.168.1.36 due to
>> > org.apache.xmlrpc.*XmlRpcException: Failed to create input stream: Read
>> > timed out\n\tat
>> >
>> >
>> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.PoolEmergencyResetMaster(XenServerConnectionPool.java:439)\n\tat
>> >
>> >
>> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.connect(XenServerConnectionPool.java:657)\n\tat
>> >
>> >
>> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getConnection(CitrixResourceBase.java:5090)\n\tat
>> >
>> >
>> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3284)\n\tat
>> >
>> >
>> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:416)\n\tat
>> >
>> >
>> com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:69)\n\tat
>> >
>> >
>> com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)\n\tat
>> >
>> >
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)\n\tat
>> > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)\n\tat
>> > java.util.concurrent.FutureTask.run(FutureTask.java:166)\n\tat
>> >
>> >
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)\n\tat
>> >
>> >
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)\n\tat
>> >
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)\n\tat
>> >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)\n\tat
>> > java.lang.Thread.run(Thread.java:636)\n","wait":0}}] }
>> > 2013-05-16 15:29:40,062 DEBUG [agent.manager.AgentAttache]
>> > (DirectAgent-369:null) Seq 3-939886253: No more commands found
>> > 2013-05-16 15:29:40,062 DEBUG [agent.transport.Request]
>> > (Job-Executor-6:job-18181) Seq 3-939886253: Received: { Ans: , MgmtId:
>> > 33932935549676, via: 3, Ver: v1, Flags: 110, { StopAnswer } }
>> > 2013-05-16 15:29:40,062 DEBUG [cloud.vm.VirtualMachineManagerImpl]
>> > (Job-Executor-6:job-18181) Unable to stop VM due to Exception:
>> > com.cloud.utils.exception.CloudRuntimeException
>> > Message: Unable to reset master of slave 192.168.1.38 to 192.168.1.36
>> due
>> > to org.apache.xmlrpc.XmlRpcException: Failed to create input stream:
>> Read
>> > timed out
>> > Stack: com.cloud.utils.exception.CloudRuntimeException: Unable to reset
>> > master of slave 192.168.1.38 to 192.168.1.36 due to
>> > org.apache.xmlrpc.XmlRpcException: Failed to create input stream: Read
>> > timed out
>> > at
>> >
>> >
>> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.PoolEmergencyResetMaster(XenServerConnectionPool.java:439)
>> > at
>> >
>> >
>> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.connect(XenServerConnectionPool.java:657)
>> > at
>> >
>> >
>> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getConnection(CitrixResourceBase.java:5090)
>> > at
>> >
>> >
>> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3284)
>> > at
>> >
>> >
>> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:416)
>> > at
>> >
>> >
>> com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:69)
>> > at
>> >
>> >
>> com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)
>> > at
>> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> > at
>> > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>> > at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>> > at
>> >
>> >
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
>> > at
>> >
>> >
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
>> > at
>> >
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>> > at
>> >
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>> > at java.lang.Thread.run(Thread.java:636)
>> > 2013-05-16 15:29:40,062 WARN [cloud.vm.VirtualMachineManagerImpl]
>> > (Job-Executor-6:job-18181) Failed to stop vm VM[DomainRouter|r-2237-VM]
>> in
>> > Starting state as a part of cleanup process
>> > 2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) DeploymentPlanner allocation algorithm:
>> random
>> > 2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) Trying to allocate a host and storage pools
>> from
>> > dc:1, pod:null,cluster:null, requested cpu: 500, requested ram:
>> 134217728
>> > 2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) Is ROOT volume READY (pool already
>> allocated)?:
>> > No
>> > 2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) Searching all possible resources under this
>> > Zone: 1
>> > 2013-05-16 15:29:40,068 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) Listing clusters in order of aggregate
>> capacity,
>> > that have (atleast one host with) enough CPU and RAM capacity under this
>> > Zone: 1
>> > 2013-05-16 15:29:40,069 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) CPUOverprovisioningFactor considered: 3.0
>> > 2013-05-16 15:29:40,077 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) Checking resources in Cluster: 1 under Pod: 1
>> > 2013-05-16 15:29:40,077 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) Calling HostAllocators to find suitable hosts
>> > 2013-05-16 15:29:40,077 DEBUG [allocator.impl.FirstFitAllocator]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Looking for hosts in
>> > dc: 1 pod:1 cluster:1
>> > 2013-05-16 15:29:40,080 DEBUG [allocator.impl.FirstFitAllocator]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) FirstFitAllocator
>> has 7
>> > hosts to check for allocation: [Host[-4-Routing], Host[-2-Routing],
>> > Host[-1-Routing], Host[-3-Routing], Host[-8-Routing], Host[-7-Routing],
>> > Host[-15-Routing]]
>> > 2013-05-16 15:29:40,088 DEBUG [allocator.impl.FirstFitAllocator]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found 7 hosts for
>> > allocation after prioritization: [Host[-4-Routing], Host[-2-Routing],
>> > Host[-1-Routing], Host[-3-Routing], Host[-8-Routing], Host[-7-Routing],
>> > Host[-15-Routing]]
>> > 2013-05-16 15:29:40,088 DEBUG [allocator.impl.FirstFitAllocator]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Looking for
>> > speed=500Mhz, Ram=128
>> > 2013-05-16 15:29:40,091 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Checking if host: 4
>> has
>> > enough capacity for requested CPU: 500 and requested RAM: 134217728 ,
>> > cpuOverprovisioningFactor: 3.0
>> > 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Hosts's actual total
>> > CPU: 63984 and CPU after applying overprovisioning: 191952
>> > 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free CPU: 161952 ,
>> > Requested CPU: 500
>> > 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free RAM:
>> 71156002944 ,
>> > Requested RAM: 134217728
>> > 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Host has enough CPU
>> and
>> > RAM available
>> > 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc CPU
>> > from host: 4, used: 29500, reserved: 500, actual total: 63984, total
>> with
>> > overprovisioning: 191952; requested cpu:500,alloc_from_last_host?:false
>> > ,considerReservedCapacity?: true
>> > 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc MEM
>> > from host: 4, used: 28856811520, reserved: 536870912, total:
>> 100549685376;
>> > requested mem: 134217728,alloc_from_last_host?:false
>> > ,considerReservedCapacity?: true
>> > 2013-05-16 15:29:40,093 DEBUG [allocator.impl.FirstFitAllocator]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found a suitable
>> host,
>> > adding to list: 4
>> > 2013-05-16 15:29:40,095 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Checking if host: 2
>> has
>> > enough capacity for requested CPU: 500 and requested RAM: 134217728 ,
>> > cpuOverprovisioningFactor: 3.0
>> > 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Hosts's actual total
>> > CPU: 63984 and CPU after applying overprovisioning: 191952
>> > 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free CPU: 151452 ,
>> > Requested CPU: 500
>> > 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free RAM:
>> 58539536512 ,
>> > Requested RAM: 134217728
>> > 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Host has enough CPU
>> and
>> > RAM available
>> > 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc CPU
>> > from host: 2, used: 40500, reserved: 0, actual total: 63984, total with
>> > overprovisioning: 191952; requested cpu:500,alloc_from_last_host?:false
>> > ,considerReservedCapacity?: true
>> > 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc MEM
>> > from host: 2, used: 42010148864, reserved: 0, total: 100549685376;
>> > requested mem: 134217728,alloc_from_last_host?:false
>> > ,considerReservedCapacity?: true
>> > 2013-05-16 15:29:40,097 DEBUG [allocator.impl.FirstFitAllocator]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found a suitable
>> host,
>> > adding to list: 2
>> > 2013-05-16 15:29:40,114 DEBUG [allocator.impl.FirstFitAllocator]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found a suitable
>> host,
>> > adding to list: 15
>> > *2013-05-16 15:29:40,114 DEBUG [allocator.impl.FirstFitAllocator]
>> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Host Allocator
>> > returning 6 suitable hosts*
>> > *2013-05-16 15:29:40,116 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) Checking suitable pools for volume (Id,
>> Type):
>> > (2438,ROOT)*
>> > *2013-05-16 15:29:40,116 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) We need to allocate new storagepool for this
>> > volume*
>> > *2013-05-16 15:29:40,116 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) Calling StoragePoolAllocators to find
>> suitable
>> > pools*
>> > *2013-05-16 15:29:40,117 DEBUG
>> > [storage.allocator.FirstFitStoragePoolAllocator]
>> (Job-Executor-6:job-18181)
>> > Looking for pools in dc: 1 pod:1 cluster:1*
>> > *2013-05-16 15:29:40,118 DEBUG
>> > [storage.allocator.FirstFitStoragePoolAllocator]
>> (Job-Executor-6:job-18181)
>> > FirstFitStoragePoolAllocator has 1 pools to check for allocation*
>> > *2013-05-16 15:29:40,118 DEBUG
>> > [storage.allocator.AbstractStoragePoolAllocator]
>> (Job-Executor-6:job-18181)
>> > Checking if storage pool is suitable, name: ps ,poolId: 200*
>> > *2013-05-16 15:29:40,118 DEBUG
>> > [storage.allocator.AbstractStoragePoolAllocator]
>> (Job-Executor-6:job-18181)
>> > StoragePool is in avoid set, skipping this pool*
>> > *2013-05-16 15:29:40,118 DEBUG
>> > [storage.allocator.FirstFitStoragePoolAllocator]
>> (Job-Executor-6:job-18181)
>> > FirstFitStoragePoolAllocator returning 0 suitable storage pools*
>> > *2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) No suitable pools found for volume:
>> > Vol[2438|vm=2237|ROOT] under cluster: 1*
>> > *2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) No suitable pools found*
>> > *2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) No suitable storagePools found under this
>> > Cluster: 1*
>> > *2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
>> > (Job-Executor-6:job-18181) Could not find suitable Deployment
>> Destination
>> > for this VM under any clusters, returning.*
>> > *2013-05-16 15:29:40,135 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181) VM state transitted from :Starting to Stopped
>> > with event: OperationFailedvm's original host id: null new host id: null
>> > host id before state transition: 3*
>> > *2013-05-16 15:29:40,139 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181) Hosts's actual total CPU: 63984 and CPU after
>> > applying overprovisioning: 191952*
>> > *2013-05-16 15:29:40,139 DEBUG [cloud.capacity.CapacityManagerImpl]
>> > (Job-Executor-6:job-18181) release cpu from host: 3, old used:
>> > 29000,reserved: 0, actual total: 63984, total with overprovisioning:
>> > 191952; new used: 28500,reserved:0; movedfromreserved:
>> > false,moveToReserveredfalse*
>> >
>>
>
>
Re: Re: 突然不能创建虚机，也不能重启虚机了
Posted by tao huang <cl...@gmail.com>.
这个环境是8个节点，我看了下，我看配置36是master，38是一个salve，但是38的xsconsole中状态异常了，没有ip，但是ifconfig看是正常的。一个slave不正常就会导致不能建虚机么？谢谢
2013/5/16 tanthalas <ta...@hotmail.com>
>
> 看来是CS找不到谁是pool master了，你要看看你的两个xenserver。
> 一般因为2个节点的集群，出现断网或者断电造成的。
> 去找xenserver对应的解决办法就好。
>
> 2013-05-16
> 刘宇超 Richard Liu
>
>
>
>
> 发件人： tao huang
> 发送时间： 2013-05-16 16:03:28
> 收件人： users-cn
> 抄送：
> 主题： Re: 突然不能创建虚机，也不能重启虚机了
>
> 这两个是两台host的IP
> 2013/5/16 tanthalas <ta...@hotmail.com>
> >
> > 先解释一下这俩ip是分配给谁的
> > 192.168.1.38
> > 192.168.1.36
> >
> >
> > 2013-05-16
> > 刘宇超 Richard Liu
> >
> >
> >
> >
> > 发件人： tao huang
> > 发送时间： 2013-05-16 15:50:33
> > 收件人： users-cn
> > 抄送：
> > 主题： 突然不能创建虚机，也不能重启虚机了
> >
> > 各位好，cloudstack 3.0 +xenserver6.0环境，高级网络
> > 突然无法创建虚机，感觉是主存储出了问题，但是看状态是正常的，如何排查呢，谢谢指导
> > 2013-05-16 15:29:40,062 DEBUG [agent.manager.DirectAgentAttache]
> > (DirectAgent-369:null) Seq 3-939886253: Response Received:
> > 2013-05-16 15:29:40,062 DEBUG [agent.transport.Request]
> > (DirectAgent-369:null) Seq 3-939886253: Processing: { Ans: , MgmtId:
> > 33932935549676, via: 3, Ver: v1, Flags: 110,
> > [{"StopAnswer":{"result":false,"details":"Exception:
> > com.cloud.utils.exception.CloudRuntimeException\nMessage: *Unable to
> reset
> > master of slave 192.168.1.38 to 192.168.1.36 due to
> > org.apache.xmlrpc.XmlRpcException: Failed to create input stream: Read
> > timed out\nStack: com.cloud.utils.exception.CloudRuntimeException: Unable
> > to reset master of slave 192.168.1.38 to 192.168.1.36 due to
> > org.apache.xmlrpc.*XmlRpcException: Failed to create input stream: Read
> > timed out\n\tat
> >
> >
> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.PoolEmergencyResetMaster(XenServerConnectionPool.java:439)\n\tat
> >
> >
> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.connect(XenServerConnectionPool.java:657)\n\tat
> >
> >
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getConnection(CitrixResourceBase.java:5090)\n\tat
> >
> >
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3284)\n\tat
> >
> >
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:416)\n\tat
> >
> >
> com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:69)\n\tat
> >
> >
> com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)\n\tat
> >
> >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)\n\tat
> > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)\n\tat
> > java.util.concurrent.FutureTask.run(FutureTask.java:166)\n\tat
> >
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)\n\tat
> >
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)\n\tat
> >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)\n\tat
> >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)\n\tat
> > java.lang.Thread.run(Thread.java:636)\n","wait":0}}] }
> > 2013-05-16 15:29:40,062 DEBUG [agent.manager.AgentAttache]
> > (DirectAgent-369:null) Seq 3-939886253: No more commands found
> > 2013-05-16 15:29:40,062 DEBUG [agent.transport.Request]
> > (Job-Executor-6:job-18181) Seq 3-939886253: Received: { Ans: , MgmtId:
> > 33932935549676, via: 3, Ver: v1, Flags: 110, { StopAnswer } }
> > 2013-05-16 15:29:40,062 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> > (Job-Executor-6:job-18181) Unable to stop VM due to Exception:
> > com.cloud.utils.exception.CloudRuntimeException
> > Message: Unable to reset master of slave 192.168.1.38 to 192.168.1.36 due
> > to org.apache.xmlrpc.XmlRpcException: Failed to create input stream: Read
> > timed out
> > Stack: com.cloud.utils.exception.CloudRuntimeException: Unable to reset
> > master of slave 192.168.1.38 to 192.168.1.36 due to
> > org.apache.xmlrpc.XmlRpcException: Failed to create input stream: Read
> > timed out
> > at
> >
> >
> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.PoolEmergencyResetMaster(XenServerConnectionPool.java:439)
> > at
> >
> >
> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.connect(XenServerConnectionPool.java:657)
> > at
> >
> >
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getConnection(CitrixResourceBase.java:5090)
> > at
> >
> >
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3284)
> > at
> >
> >
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:416)
> > at
> >
> >
> com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:69)
> > at
> >
> >
> com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)
> > at
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> > at
> > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> > at
> >
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> > at
> >
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> > at
> >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> > at
> >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> > at java.lang.Thread.run(Thread.java:636)
> > 2013-05-16 15:29:40,062 WARN [cloud.vm.VirtualMachineManagerImpl]
> > (Job-Executor-6:job-18181) Failed to stop vm VM[DomainRouter|r-2237-VM]
> in
> > Starting state as a part of cleanup process
> > 2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
> > (Job-Executor-6:job-18181) DeploymentPlanner allocation algorithm: random
> > 2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
> > (Job-Executor-6:job-18181) Trying to allocate a host and storage pools
> from
> > dc:1, pod:null,cluster:null, requested cpu: 500, requested ram: 134217728
> > 2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
> > (Job-Executor-6:job-18181) Is ROOT volume READY (pool already
> allocated)?:
> > No
> > 2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
> > (Job-Executor-6:job-18181) Searching all possible resources under this
> > Zone: 1
> > 2013-05-16 15:29:40,068 DEBUG [cloud.deploy.FirstFitPlanner]
> > (Job-Executor-6:job-18181) Listing clusters in order of aggregate
> capacity,
> > that have (atleast one host with) enough CPU and RAM capacity under this
> > Zone: 1
> > 2013-05-16 15:29:40,069 DEBUG [cloud.deploy.FirstFitPlanner]
> > (Job-Executor-6:job-18181) CPUOverprovisioningFactor considered: 3.0
> > 2013-05-16 15:29:40,077 DEBUG [cloud.deploy.FirstFitPlanner]
> > (Job-Executor-6:job-18181) Checking resources in Cluster: 1 under Pod: 1
> > 2013-05-16 15:29:40,077 DEBUG [cloud.deploy.FirstFitPlanner]
> > (Job-Executor-6:job-18181) Calling HostAllocators to find suitable hosts
> > 2013-05-16 15:29:40,077 DEBUG [allocator.impl.FirstFitAllocator]
> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Looking for hosts in
> > dc: 1 pod:1 cluster:1
> > 2013-05-16 15:29:40,080 DEBUG [allocator.impl.FirstFitAllocator]
> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) FirstFitAllocator
> has 7
> > hosts to check for allocation: [Host[-4-Routing], Host[-2-Routing],
> > Host[-1-Routing], Host[-3-Routing], Host[-8-Routing], Host[-7-Routing],
> > Host[-15-Routing]]
> > 2013-05-16 15:29:40,088 DEBUG [allocator.impl.FirstFitAllocator]
> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found 7 hosts for
> > allocation after prioritization: [Host[-4-Routing], Host[-2-Routing],
> > Host[-1-Routing], Host[-3-Routing], Host[-8-Routing], Host[-7-Routing],
> > Host[-15-Routing]]
> > 2013-05-16 15:29:40,088 DEBUG [allocator.impl.FirstFitAllocator]
> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Looking for
> > speed=500Mhz, Ram=128
> > 2013-05-16 15:29:40,091 DEBUG [cloud.capacity.CapacityManagerImpl]
> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Checking if host: 4
> has
> > enough capacity for requested CPU: 500 and requested RAM: 134217728 ,
> > cpuOverprovisioningFactor: 3.0
> > 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Hosts's actual total
> > CPU: 63984 and CPU after applying overprovisioning: 191952
> > 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free CPU: 161952 ,
> > Requested CPU: 500
> > 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free RAM:
> 71156002944 ,
> > Requested RAM: 134217728
> > 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Host has enough CPU
> and
> > RAM available
> > 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc CPU
> > from host: 4, used: 29500, reserved: 500, actual total: 63984, total with
> > overprovisioning: 191952; requested cpu:500,alloc_from_last_host?:false
> > ,considerReservedCapacity?: true
> > 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc MEM
> > from host: 4, used: 28856811520, reserved: 536870912, total:
> 100549685376;
> > requested mem: 134217728,alloc_from_last_host?:false
> > ,considerReservedCapacity?: true
> > 2013-05-16 15:29:40,093 DEBUG [allocator.impl.FirstFitAllocator]
> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found a suitable
> host,
> > adding to list: 4
> > 2013-05-16 15:29:40,095 DEBUG [cloud.capacity.CapacityManagerImpl]
> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Checking if host: 2
> has
> > enough capacity for requested CPU: 500 and requested RAM: 134217728 ,
> > cpuOverprovisioningFactor: 3.0
> > 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Hosts's actual total
> > CPU: 63984 and CPU after applying overprovisioning: 191952
> > 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free CPU: 151452 ,
> > Requested CPU: 500
> > 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free RAM:
> 58539536512 ,
> > Requested RAM: 134217728
> > 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Host has enough CPU
> and
> > RAM available
> > 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc CPU
> > from host: 2, used: 40500, reserved: 0, actual total: 63984, total with
> > overprovisioning: 191952; requested cpu:500,alloc_from_last_host?:false
> > ,considerReservedCapacity?: true
> > 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc MEM
> > from host: 2, used: 42010148864, reserved: 0, total: 100549685376;
> > requested mem: 134217728,alloc_from_last_host?:false
> > ,considerReservedCapacity?: true
> > 2013-05-16 15:29:40,097 DEBUG [allocator.impl.FirstFitAllocator]
> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found a suitable
> host,
> > adding to list: 2
> > 2013-05-16 15:29:40,114 DEBUG [allocator.impl.FirstFitAllocator]
> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found a suitable
> host,
> > adding to list: 15
> > *2013-05-16 15:29:40,114 DEBUG [allocator.impl.FirstFitAllocator]
> > (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Host Allocator
> > returning 6 suitable hosts*
> > *2013-05-16 15:29:40,116 DEBUG [cloud.deploy.FirstFitPlanner]
> > (Job-Executor-6:job-18181) Checking suitable pools for volume (Id, Type):
> > (2438,ROOT)*
> > *2013-05-16 15:29:40,116 DEBUG [cloud.deploy.FirstFitPlanner]
> > (Job-Executor-6:job-18181) We need to allocate new storagepool for this
> > volume*
> > *2013-05-16 15:29:40,116 DEBUG [cloud.deploy.FirstFitPlanner]
> > (Job-Executor-6:job-18181) Calling StoragePoolAllocators to find suitable
> > pools*
> > *2013-05-16 15:29:40,117 DEBUG
> > [storage.allocator.FirstFitStoragePoolAllocator]
> (Job-Executor-6:job-18181)
> > Looking for pools in dc: 1 pod:1 cluster:1*
> > *2013-05-16 15:29:40,118 DEBUG
> > [storage.allocator.FirstFitStoragePoolAllocator]
> (Job-Executor-6:job-18181)
> > FirstFitStoragePoolAllocator has 1 pools to check for allocation*
> > *2013-05-16 15:29:40,118 DEBUG
> > [storage.allocator.AbstractStoragePoolAllocator]
> (Job-Executor-6:job-18181)
> > Checking if storage pool is suitable, name: ps ,poolId: 200*
> > *2013-05-16 15:29:40,118 DEBUG
> > [storage.allocator.AbstractStoragePoolAllocator]
> (Job-Executor-6:job-18181)
> > StoragePool is in avoid set, skipping this pool*
> > *2013-05-16 15:29:40,118 DEBUG
> > [storage.allocator.FirstFitStoragePoolAllocator]
> (Job-Executor-6:job-18181)
> > FirstFitStoragePoolAllocator returning 0 suitable storage pools*
> > *2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
> > (Job-Executor-6:job-18181) No suitable pools found for volume:
> > Vol[2438|vm=2237|ROOT] under cluster: 1*
> > *2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
> > (Job-Executor-6:job-18181) No suitable pools found*
> > *2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
> > (Job-Executor-6:job-18181) No suitable storagePools found under this
> > Cluster: 1*
> > *2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
> > (Job-Executor-6:job-18181) Could not find suitable Deployment Destination
> > for this VM under any clusters, returning.*
> > *2013-05-16 15:29:40,135 DEBUG [cloud.capacity.CapacityManagerImpl]
> > (Job-Executor-6:job-18181) VM state transitted from :Starting to Stopped
> > with event: OperationFailedvm's original host id: null new host id: null
> > host id before state transition: 3*
> > *2013-05-16 15:29:40,139 DEBUG [cloud.capacity.CapacityManagerImpl]
> > (Job-Executor-6:job-18181) Hosts's actual total CPU: 63984 and CPU after
> > applying overprovisioning: 191952*
> > *2013-05-16 15:29:40,139 DEBUG [cloud.capacity.CapacityManagerImpl]
> > (Job-Executor-6:job-18181) release cpu from host: 3, old used:
> > 29000,reserved: 0, actual total: 63984, total with overprovisioning:
> > 191952; new used: 28500,reserved:0; movedfromreserved:
> > false,moveToReserveredfalse*
> >
>
Re: Re: 突然不能创建虚机，也不能重启虚机了
Posted by tanthalas <ta...@hotmail.com>.
看来是CS找不到谁是pool master了，你要看看你的两个xenserver。
一般因为2个节点的集群，出现断网或者断电造成的。
去找xenserver对应的解决办法就好。
2013-05-16
刘宇超 Richard Liu
发件人： tao huang
发送时间： 2013-05-16 16:03:28
收件人： users-cn
抄送：
主题： Re: 突然不能创建虚机，也不能重启虚机了
这两个是两台host的IP
2013/5/16 tanthalas <ta...@hotmail.com>
>
> 先解释一下这俩ip是分配给谁的
> 192.168.1.38
> 192.168.1.36
>
>
> 2013-05-16
> 刘宇超 Richard Liu
>
>
>
>
> 发件人： tao huang
> 发送时间： 2013-05-16 15:50:33
> 收件人： users-cn
> 抄送：
> 主题： 突然不能创建虚机，也不能重启虚机了
>
> 各位好，cloudstack 3.0 +xenserver6.0环境，高级网络
> 突然无法创建虚机，感觉是主存储出了问题，但是看状态是正常的，如何排查呢，谢谢指导
> 2013-05-16 15:29:40,062 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-369:null) Seq 3-939886253: Response Received:
> 2013-05-16 15:29:40,062 DEBUG [agent.transport.Request]
> (DirectAgent-369:null) Seq 3-939886253: Processing: { Ans: , MgmtId:
> 33932935549676, via: 3, Ver: v1, Flags: 110,
> [{"StopAnswer":{"result":false,"details":"Exception:
> com.cloud.utils.exception.CloudRuntimeException\nMessage: *Unable to reset
> master of slave 192.168.1.38 to 192.168.1.36 due to
> org.apache.xmlrpc.XmlRpcException: Failed to create input stream: Read
> timed out\nStack: com.cloud.utils.exception.CloudRuntimeException: Unable
> to reset master of slave 192.168.1.38 to 192.168.1.36 due to
> org.apache.xmlrpc.*XmlRpcException: Failed to create input stream: Read
> timed out\n\tat
>
> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.PoolEmergencyResetMaster(XenServerConnectionPool.java:439)\n\tat
>
> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.connect(XenServerConnectionPool.java:657)\n\tat
>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getConnection(CitrixResourceBase.java:5090)\n\tat
>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3284)\n\tat
>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:416)\n\tat
>
> com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:69)\n\tat
>
> com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)\n\tat
>
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)\n\tat
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)\n\tat
> java.util.concurrent.FutureTask.run(FutureTask.java:166)\n\tat
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)\n\tat
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)\n\tat
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)\n\tat
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)\n\tat
> java.lang.Thread.run(Thread.java:636)\n","wait":0}}] }
> 2013-05-16 15:29:40,062 DEBUG [agent.manager.AgentAttache]
> (DirectAgent-369:null) Seq 3-939886253: No more commands found
> 2013-05-16 15:29:40,062 DEBUG [agent.transport.Request]
> (Job-Executor-6:job-18181) Seq 3-939886253: Received: { Ans: , MgmtId:
> 33932935549676, via: 3, Ver: v1, Flags: 110, { StopAnswer } }
> 2013-05-16 15:29:40,062 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> (Job-Executor-6:job-18181) Unable to stop VM due to Exception:
> com.cloud.utils.exception.CloudRuntimeException
> Message: Unable to reset master of slave 192.168.1.38 to 192.168.1.36 due
> to org.apache.xmlrpc.XmlRpcException: Failed to create input stream: Read
> timed out
> Stack: com.cloud.utils.exception.CloudRuntimeException: Unable to reset
> master of slave 192.168.1.38 to 192.168.1.36 due to
> org.apache.xmlrpc.XmlRpcException: Failed to create input stream: Read
> timed out
> at
>
> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.PoolEmergencyResetMaster(XenServerConnectionPool.java:439)
> at
>
> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.connect(XenServerConnectionPool.java:657)
> at
>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getConnection(CitrixResourceBase.java:5090)
> at
>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3284)
> at
>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:416)
> at
>
> com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:69)
> at
>
> com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:636)
> 2013-05-16 15:29:40,062 WARN [cloud.vm.VirtualMachineManagerImpl]
> (Job-Executor-6:job-18181) Failed to stop vm VM[DomainRouter|r-2237-VM] in
> Starting state as a part of cleanup process
> 2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) DeploymentPlanner allocation algorithm: random
> 2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) Trying to allocate a host and storage pools from
> dc:1, pod:null,cluster:null, requested cpu: 500, requested ram: 134217728
> 2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) Is ROOT volume READY (pool already allocated)?:
> No
> 2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) Searching all possible resources under this
> Zone: 1
> 2013-05-16 15:29:40,068 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) Listing clusters in order of aggregate capacity,
> that have (atleast one host with) enough CPU and RAM capacity under this
> Zone: 1
> 2013-05-16 15:29:40,069 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) CPUOverprovisioningFactor considered: 3.0
> 2013-05-16 15:29:40,077 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) Checking resources in Cluster: 1 under Pod: 1
> 2013-05-16 15:29:40,077 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) Calling HostAllocators to find suitable hosts
> 2013-05-16 15:29:40,077 DEBUG [allocator.impl.FirstFitAllocator]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Looking for hosts in
> dc: 1 pod:1 cluster:1
> 2013-05-16 15:29:40,080 DEBUG [allocator.impl.FirstFitAllocator]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) FirstFitAllocator has 7
> hosts to check for allocation: [Host[-4-Routing], Host[-2-Routing],
> Host[-1-Routing], Host[-3-Routing], Host[-8-Routing], Host[-7-Routing],
> Host[-15-Routing]]
> 2013-05-16 15:29:40,088 DEBUG [allocator.impl.FirstFitAllocator]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found 7 hosts for
> allocation after prioritization: [Host[-4-Routing], Host[-2-Routing],
> Host[-1-Routing], Host[-3-Routing], Host[-8-Routing], Host[-7-Routing],
> Host[-15-Routing]]
> 2013-05-16 15:29:40,088 DEBUG [allocator.impl.FirstFitAllocator]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Looking for
> speed=500Mhz, Ram=128
> 2013-05-16 15:29:40,091 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Checking if host: 4 has
> enough capacity for requested CPU: 500 and requested RAM: 134217728 ,
> cpuOverprovisioningFactor: 3.0
> 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Hosts's actual total
> CPU: 63984 and CPU after applying overprovisioning: 191952
> 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free CPU: 161952 ,
> Requested CPU: 500
> 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free RAM: 71156002944 ,
> Requested RAM: 134217728
> 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Host has enough CPU and
> RAM available
> 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc CPU
> from host: 4, used: 29500, reserved: 500, actual total: 63984, total with
> overprovisioning: 191952; requested cpu:500,alloc_from_last_host?:false
> ,considerReservedCapacity?: true
> 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc MEM
> from host: 4, used: 28856811520, reserved: 536870912, total: 100549685376;
> requested mem: 134217728,alloc_from_last_host?:false
> ,considerReservedCapacity?: true
> 2013-05-16 15:29:40,093 DEBUG [allocator.impl.FirstFitAllocator]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found a suitable host,
> adding to list: 4
> 2013-05-16 15:29:40,095 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Checking if host: 2 has
> enough capacity for requested CPU: 500 and requested RAM: 134217728 ,
> cpuOverprovisioningFactor: 3.0
> 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Hosts's actual total
> CPU: 63984 and CPU after applying overprovisioning: 191952
> 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free CPU: 151452 ,
> Requested CPU: 500
> 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free RAM: 58539536512 ,
> Requested RAM: 134217728
> 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Host has enough CPU and
> RAM available
> 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc CPU
> from host: 2, used: 40500, reserved: 0, actual total: 63984, total with
> overprovisioning: 191952; requested cpu:500,alloc_from_last_host?:false
> ,considerReservedCapacity?: true
> 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc MEM
> from host: 2, used: 42010148864, reserved: 0, total: 100549685376;
> requested mem: 134217728,alloc_from_last_host?:false
> ,considerReservedCapacity?: true
> 2013-05-16 15:29:40,097 DEBUG [allocator.impl.FirstFitAllocator]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found a suitable host,
> adding to list: 2
> 2013-05-16 15:29:40,114 DEBUG [allocator.impl.FirstFitAllocator]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found a suitable host,
> adding to list: 15
> *2013-05-16 15:29:40,114 DEBUG [allocator.impl.FirstFitAllocator]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Host Allocator
> returning 6 suitable hosts*
> *2013-05-16 15:29:40,116 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) Checking suitable pools for volume (Id, Type):
> (2438,ROOT)*
> *2013-05-16 15:29:40,116 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) We need to allocate new storagepool for this
> volume*
> *2013-05-16 15:29:40,116 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) Calling StoragePoolAllocators to find suitable
> pools*
> *2013-05-16 15:29:40,117 DEBUG
> [storage.allocator.FirstFitStoragePoolAllocator] (Job-Executor-6:job-18181)
> Looking for pools in dc: 1 pod:1 cluster:1*
> *2013-05-16 15:29:40,118 DEBUG
> [storage.allocator.FirstFitStoragePoolAllocator] (Job-Executor-6:job-18181)
> FirstFitStoragePoolAllocator has 1 pools to check for allocation*
> *2013-05-16 15:29:40,118 DEBUG
> [storage.allocator.AbstractStoragePoolAllocator] (Job-Executor-6:job-18181)
> Checking if storage pool is suitable, name: ps ,poolId: 200*
> *2013-05-16 15:29:40,118 DEBUG
> [storage.allocator.AbstractStoragePoolAllocator] (Job-Executor-6:job-18181)
> StoragePool is in avoid set, skipping this pool*
> *2013-05-16 15:29:40,118 DEBUG
> [storage.allocator.FirstFitStoragePoolAllocator] (Job-Executor-6:job-18181)
> FirstFitStoragePoolAllocator returning 0 suitable storage pools*
> *2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) No suitable pools found for volume:
> Vol[2438|vm=2237|ROOT] under cluster: 1*
> *2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) No suitable pools found*
> *2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) No suitable storagePools found under this
> Cluster: 1*
> *2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) Could not find suitable Deployment Destination
> for this VM under any clusters, returning.*
> *2013-05-16 15:29:40,135 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181) VM state transitted from :Starting to Stopped
> with event: OperationFailedvm's original host id: null new host id: null
> host id before state transition: 3*
> *2013-05-16 15:29:40,139 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181) Hosts's actual total CPU: 63984 and CPU after
> applying overprovisioning: 191952*
> *2013-05-16 15:29:40,139 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181) release cpu from host: 3, old used:
> 29000,reserved: 0, actual total: 63984, total with overprovisioning:
> 191952; new used: 28500,reserved:0; movedfromreserved:
> false,moveToReserveredfalse*
>
Re: 突然不能创建虚机，也不能重启虚机了
Posted by tao huang <cl...@gmail.com>.
这两个是两台host的IP
2013/5/16 tanthalas <ta...@hotmail.com>
>
> 先解释一下这俩ip是分配给谁的
> 192.168.1.38
> 192.168.1.36
>
>
> 2013-05-16
> 刘宇超 Richard Liu
>
>
>
>
> 发件人： tao huang
> 发送时间： 2013-05-16 15:50:33
> 收件人： users-cn
> 抄送：
> 主题： 突然不能创建虚机，也不能重启虚机了
>
> 各位好，cloudstack 3.0 +xenserver6.0环境，高级网络
> 突然无法创建虚机，感觉是主存储出了问题，但是看状态是正常的，如何排查呢，谢谢指导
> 2013-05-16 15:29:40,062 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-369:null) Seq 3-939886253: Response Received:
> 2013-05-16 15:29:40,062 DEBUG [agent.transport.Request]
> (DirectAgent-369:null) Seq 3-939886253: Processing: { Ans: , MgmtId:
> 33932935549676, via: 3, Ver: v1, Flags: 110,
> [{"StopAnswer":{"result":false,"details":"Exception:
> com.cloud.utils.exception.CloudRuntimeException\nMessage: *Unable to reset
> master of slave 192.168.1.38 to 192.168.1.36 due to
> org.apache.xmlrpc.XmlRpcException: Failed to create input stream: Read
> timed out\nStack: com.cloud.utils.exception.CloudRuntimeException: Unable
> to reset master of slave 192.168.1.38 to 192.168.1.36 due to
> org.apache.xmlrpc.*XmlRpcException: Failed to create input stream: Read
> timed out\n\tat
>
> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.PoolEmergencyResetMaster(XenServerConnectionPool.java:439)\n\tat
>
> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.connect(XenServerConnectionPool.java:657)\n\tat
>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getConnection(CitrixResourceBase.java:5090)\n\tat
>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3284)\n\tat
>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:416)\n\tat
>
> com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:69)\n\tat
>
> com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)\n\tat
>
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)\n\tat
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)\n\tat
> java.util.concurrent.FutureTask.run(FutureTask.java:166)\n\tat
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)\n\tat
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)\n\tat
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)\n\tat
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)\n\tat
> java.lang.Thread.run(Thread.java:636)\n","wait":0}}] }
> 2013-05-16 15:29:40,062 DEBUG [agent.manager.AgentAttache]
> (DirectAgent-369:null) Seq 3-939886253: No more commands found
> 2013-05-16 15:29:40,062 DEBUG [agent.transport.Request]
> (Job-Executor-6:job-18181) Seq 3-939886253: Received: { Ans: , MgmtId:
> 33932935549676, via: 3, Ver: v1, Flags: 110, { StopAnswer } }
> 2013-05-16 15:29:40,062 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> (Job-Executor-6:job-18181) Unable to stop VM due to Exception:
> com.cloud.utils.exception.CloudRuntimeException
> Message: Unable to reset master of slave 192.168.1.38 to 192.168.1.36 due
> to org.apache.xmlrpc.XmlRpcException: Failed to create input stream: Read
> timed out
> Stack: com.cloud.utils.exception.CloudRuntimeException: Unable to reset
> master of slave 192.168.1.38 to 192.168.1.36 due to
> org.apache.xmlrpc.XmlRpcException: Failed to create input stream: Read
> timed out
> at
>
> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.PoolEmergencyResetMaster(XenServerConnectionPool.java:439)
> at
>
> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.connect(XenServerConnectionPool.java:657)
> at
>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getConnection(CitrixResourceBase.java:5090)
> at
>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3284)
> at
>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:416)
> at
>
> com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:69)
> at
>
> com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:636)
> 2013-05-16 15:29:40,062 WARN [cloud.vm.VirtualMachineManagerImpl]
> (Job-Executor-6:job-18181) Failed to stop vm VM[DomainRouter|r-2237-VM] in
> Starting state as a part of cleanup process
> 2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) DeploymentPlanner allocation algorithm: random
> 2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) Trying to allocate a host and storage pools from
> dc:1, pod:null,cluster:null, requested cpu: 500, requested ram: 134217728
> 2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) Is ROOT volume READY (pool already allocated)?:
> No
> 2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) Searching all possible resources under this
> Zone: 1
> 2013-05-16 15:29:40,068 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) Listing clusters in order of aggregate capacity,
> that have (atleast one host with) enough CPU and RAM capacity under this
> Zone: 1
> 2013-05-16 15:29:40,069 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) CPUOverprovisioningFactor considered: 3.0
> 2013-05-16 15:29:40,077 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) Checking resources in Cluster: 1 under Pod: 1
> 2013-05-16 15:29:40,077 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) Calling HostAllocators to find suitable hosts
> 2013-05-16 15:29:40,077 DEBUG [allocator.impl.FirstFitAllocator]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Looking for hosts in
> dc: 1 pod:1 cluster:1
> 2013-05-16 15:29:40,080 DEBUG [allocator.impl.FirstFitAllocator]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) FirstFitAllocator has 7
> hosts to check for allocation: [Host[-4-Routing], Host[-2-Routing],
> Host[-1-Routing], Host[-3-Routing], Host[-8-Routing], Host[-7-Routing],
> Host[-15-Routing]]
> 2013-05-16 15:29:40,088 DEBUG [allocator.impl.FirstFitAllocator]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found 7 hosts for
> allocation after prioritization: [Host[-4-Routing], Host[-2-Routing],
> Host[-1-Routing], Host[-3-Routing], Host[-8-Routing], Host[-7-Routing],
> Host[-15-Routing]]
> 2013-05-16 15:29:40,088 DEBUG [allocator.impl.FirstFitAllocator]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Looking for
> speed=500Mhz, Ram=128
> 2013-05-16 15:29:40,091 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Checking if host: 4 has
> enough capacity for requested CPU: 500 and requested RAM: 134217728 ,
> cpuOverprovisioningFactor: 3.0
> 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Hosts's actual total
> CPU: 63984 and CPU after applying overprovisioning: 191952
> 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free CPU: 161952 ,
> Requested CPU: 500
> 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free RAM: 71156002944 ,
> Requested RAM: 134217728
> 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Host has enough CPU and
> RAM available
> 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc CPU
> from host: 4, used: 29500, reserved: 500, actual total: 63984, total with
> overprovisioning: 191952; requested cpu:500,alloc_from_last_host?:false
> ,considerReservedCapacity?: true
> 2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc MEM
> from host: 4, used: 28856811520, reserved: 536870912, total: 100549685376;
> requested mem: 134217728,alloc_from_last_host?:false
> ,considerReservedCapacity?: true
> 2013-05-16 15:29:40,093 DEBUG [allocator.impl.FirstFitAllocator]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found a suitable host,
> adding to list: 4
> 2013-05-16 15:29:40,095 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Checking if host: 2 has
> enough capacity for requested CPU: 500 and requested RAM: 134217728 ,
> cpuOverprovisioningFactor: 3.0
> 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Hosts's actual total
> CPU: 63984 and CPU after applying overprovisioning: 191952
> 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free CPU: 151452 ,
> Requested CPU: 500
> 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free RAM: 58539536512 ,
> Requested RAM: 134217728
> 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Host has enough CPU and
> RAM available
> 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc CPU
> from host: 2, used: 40500, reserved: 0, actual total: 63984, total with
> overprovisioning: 191952; requested cpu:500,alloc_from_last_host?:false
> ,considerReservedCapacity?: true
> 2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc MEM
> from host: 2, used: 42010148864, reserved: 0, total: 100549685376;
> requested mem: 134217728,alloc_from_last_host?:false
> ,considerReservedCapacity?: true
> 2013-05-16 15:29:40,097 DEBUG [allocator.impl.FirstFitAllocator]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found a suitable host,
> adding to list: 2
> 2013-05-16 15:29:40,114 DEBUG [allocator.impl.FirstFitAllocator]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found a suitable host,
> adding to list: 15
> *2013-05-16 15:29:40,114 DEBUG [allocator.impl.FirstFitAllocator]
> (Job-Executor-6:job-18181 FirstFitRoutingAllocator) Host Allocator
> returning 6 suitable hosts*
> *2013-05-16 15:29:40,116 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) Checking suitable pools for volume (Id, Type):
> (2438,ROOT)*
> *2013-05-16 15:29:40,116 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) We need to allocate new storagepool for this
> volume*
> *2013-05-16 15:29:40,116 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) Calling StoragePoolAllocators to find suitable
> pools*
> *2013-05-16 15:29:40,117 DEBUG
> [storage.allocator.FirstFitStoragePoolAllocator] (Job-Executor-6:job-18181)
> Looking for pools in dc: 1 pod:1 cluster:1*
> *2013-05-16 15:29:40,118 DEBUG
> [storage.allocator.FirstFitStoragePoolAllocator] (Job-Executor-6:job-18181)
> FirstFitStoragePoolAllocator has 1 pools to check for allocation*
> *2013-05-16 15:29:40,118 DEBUG
> [storage.allocator.AbstractStoragePoolAllocator] (Job-Executor-6:job-18181)
> Checking if storage pool is suitable, name: ps ,poolId: 200*
> *2013-05-16 15:29:40,118 DEBUG
> [storage.allocator.AbstractStoragePoolAllocator] (Job-Executor-6:job-18181)
> StoragePool is in avoid set, skipping this pool*
> *2013-05-16 15:29:40,118 DEBUG
> [storage.allocator.FirstFitStoragePoolAllocator] (Job-Executor-6:job-18181)
> FirstFitStoragePoolAllocator returning 0 suitable storage pools*
> *2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) No suitable pools found for volume:
> Vol[2438|vm=2237|ROOT] under cluster: 1*
> *2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) No suitable pools found*
> *2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) No suitable storagePools found under this
> Cluster: 1*
> *2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
> (Job-Executor-6:job-18181) Could not find suitable Deployment Destination
> for this VM under any clusters, returning.*
> *2013-05-16 15:29:40,135 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181) VM state transitted from :Starting to Stopped
> with event: OperationFailedvm's original host id: null new host id: null
> host id before state transition: 3*
> *2013-05-16 15:29:40,139 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181) Hosts's actual total CPU: 63984 and CPU after
> applying overprovisioning: 191952*
> *2013-05-16 15:29:40,139 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-18181) release cpu from host: 3, old used:
> 29000,reserved: 0, actual total: 63984, total with overprovisioning:
> 191952; new used: 28500,reserved:0; movedfromreserved:
> false,moveToReserveredfalse*
>
Re: 突然不能创建虚机，也不能重启虚机了
Posted by tanthalas <ta...@hotmail.com>.
先解释一下这俩ip是分配给谁的
192.168.1.38
192.168.1.36
2013-05-16
刘宇超 Richard Liu
发件人： tao huang
发送时间： 2013-05-16 15:50:33
收件人： users-cn
抄送：
主题： 突然不能创建虚机，也不能重启虚机了
各位好，cloudstack 3.0 +xenserver6.0环境，高级网络
突然无法创建虚机，感觉是主存储出了问题，但是看状态是正常的，如何排查呢，谢谢指导
2013-05-16 15:29:40,062 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-369:null) Seq 3-939886253: Response Received:
2013-05-16 15:29:40,062 DEBUG [agent.transport.Request]
(DirectAgent-369:null) Seq 3-939886253: Processing: { Ans: , MgmtId:
33932935549676, via: 3, Ver: v1, Flags: 110,
[{"StopAnswer":{"result":false,"details":"Exception:
com.cloud.utils.exception.CloudRuntimeException\nMessage: *Unable to reset
master of slave 192.168.1.38 to 192.168.1.36 due to
org.apache.xmlrpc.XmlRpcException: Failed to create input stream: Read
timed out\nStack: com.cloud.utils.exception.CloudRuntimeException: Unable
to reset master of slave 192.168.1.38 to 192.168.1.36 due to
org.apache.xmlrpc.*XmlRpcException: Failed to create input stream: Read
timed out\n\tat
com.cloud.hypervisor.xen.resource.XenServerConnectionPool.PoolEmergencyResetMaster(XenServerConnectionPool.java:439)\n\tat
com.cloud.hypervisor.xen.resource.XenServerConnectionPool.connect(XenServerConnectionPool.java:657)\n\tat
com.cloud.hypervisor.xen.resource.CitrixResourceBase.getConnection(CitrixResourceBase.java:5090)\n\tat
com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3284)\n\tat
com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:416)\n\tat
com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:69)\n\tat
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)\n\tat
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)\n\tat
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)\n\tat
java.util.concurrent.FutureTask.run(FutureTask.java:166)\n\tat
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)\n\tat
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)\n\tat
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)\n\tat
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)\n\tat
java.lang.Thread.run(Thread.java:636)\n","wait":0}}] }
2013-05-16 15:29:40,062 DEBUG [agent.manager.AgentAttache]
(DirectAgent-369:null) Seq 3-939886253: No more commands found
2013-05-16 15:29:40,062 DEBUG [agent.transport.Request]
(Job-Executor-6:job-18181) Seq 3-939886253: Received: { Ans: , MgmtId:
33932935549676, via: 3, Ver: v1, Flags: 110, { StopAnswer } }
2013-05-16 15:29:40,062 DEBUG [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-6:job-18181) Unable to stop VM due to Exception:
com.cloud.utils.exception.CloudRuntimeException
Message: Unable to reset master of slave 192.168.1.38 to 192.168.1.36 due
to org.apache.xmlrpc.XmlRpcException: Failed to create input stream: Read
timed out
Stack: com.cloud.utils.exception.CloudRuntimeException: Unable to reset
master of slave 192.168.1.38 to 192.168.1.36 due to
org.apache.xmlrpc.XmlRpcException: Failed to create input stream: Read
timed out
at
com.cloud.hypervisor.xen.resource.XenServerConnectionPool.PoolEmergencyResetMaster(XenServerConnectionPool.java:439)
at
com.cloud.hypervisor.xen.resource.XenServerConnectionPool.connect(XenServerConnectionPool.java:657)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.getConnection(CitrixResourceBase.java:5090)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3284)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:416)
at
com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:69)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
2013-05-16 15:29:40,062 WARN [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-6:job-18181) Failed to stop vm VM[DomainRouter|r-2237-VM] in
Starting state as a part of cleanup process
2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) DeploymentPlanner allocation algorithm: random
2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) Trying to allocate a host and storage pools from
dc:1, pod:null,cluster:null, requested cpu: 500, requested ram: 134217728
2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) Is ROOT volume READY (pool already allocated)?:
No
2013-05-16 15:29:40,066 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) Searching all possible resources under this
Zone: 1
2013-05-16 15:29:40,068 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) Listing clusters in order of aggregate capacity,
that have (atleast one host with) enough CPU and RAM capacity under this
Zone: 1
2013-05-16 15:29:40,069 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) CPUOverprovisioningFactor considered: 3.0
2013-05-16 15:29:40,077 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) Checking resources in Cluster: 1 under Pod: 1
2013-05-16 15:29:40,077 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) Calling HostAllocators to find suitable hosts
2013-05-16 15:29:40,077 DEBUG [allocator.impl.FirstFitAllocator]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Looking for hosts in
dc: 1 pod:1 cluster:1
2013-05-16 15:29:40,080 DEBUG [allocator.impl.FirstFitAllocator]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) FirstFitAllocator has 7
hosts to check for allocation: [Host[-4-Routing], Host[-2-Routing],
Host[-1-Routing], Host[-3-Routing], Host[-8-Routing], Host[-7-Routing],
Host[-15-Routing]]
2013-05-16 15:29:40,088 DEBUG [allocator.impl.FirstFitAllocator]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found 7 hosts for
allocation after prioritization: [Host[-4-Routing], Host[-2-Routing],
Host[-1-Routing], Host[-3-Routing], Host[-8-Routing], Host[-7-Routing],
Host[-15-Routing]]
2013-05-16 15:29:40,088 DEBUG [allocator.impl.FirstFitAllocator]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Looking for
speed=500Mhz, Ram=128
2013-05-16 15:29:40,091 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Checking if host: 4 has
enough capacity for requested CPU: 500 and requested RAM: 134217728 ,
cpuOverprovisioningFactor: 3.0
2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Hosts's actual total
CPU: 63984 and CPU after applying overprovisioning: 191952
2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free CPU: 161952 ,
Requested CPU: 500
2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free RAM: 71156002944 ,
Requested RAM: 134217728
2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Host has enough CPU and
RAM available
2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc CPU
from host: 4, used: 29500, reserved: 500, actual total: 63984, total with
overprovisioning: 191952; requested cpu:500,alloc_from_last_host?:false
,considerReservedCapacity?: true
2013-05-16 15:29:40,093 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc MEM
from host: 4, used: 28856811520, reserved: 536870912, total: 100549685376;
requested mem: 134217728,alloc_from_last_host?:false
,considerReservedCapacity?: true
2013-05-16 15:29:40,093 DEBUG [allocator.impl.FirstFitAllocator]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found a suitable host,
adding to list: 4
2013-05-16 15:29:40,095 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Checking if host: 2 has
enough capacity for requested CPU: 500 and requested RAM: 134217728 ,
cpuOverprovisioningFactor: 3.0
2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Hosts's actual total
CPU: 63984 and CPU after applying overprovisioning: 191952
2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free CPU: 151452 ,
Requested CPU: 500
2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Free RAM: 58539536512 ,
Requested RAM: 134217728
2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Host has enough CPU and
RAM available
2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc CPU
from host: 2, used: 40500, reserved: 0, actual total: 63984, total with
overprovisioning: 191952; requested cpu:500,alloc_from_last_host?:false
,considerReservedCapacity?: true
2013-05-16 15:29:40,097 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) STATS: Can alloc MEM
from host: 2, used: 42010148864, reserved: 0, total: 100549685376;
requested mem: 134217728,alloc_from_last_host?:false
,considerReservedCapacity?: true
2013-05-16 15:29:40,097 DEBUG [allocator.impl.FirstFitAllocator]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found a suitable host,
adding to list: 2
2013-05-16 15:29:40,114 DEBUG [allocator.impl.FirstFitAllocator]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Found a suitable host,
adding to list: 15
*2013-05-16 15:29:40,114 DEBUG [allocator.impl.FirstFitAllocator]
(Job-Executor-6:job-18181 FirstFitRoutingAllocator) Host Allocator
returning 6 suitable hosts*
*2013-05-16 15:29:40,116 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) Checking suitable pools for volume (Id, Type):
(2438,ROOT)*
*2013-05-16 15:29:40,116 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) We need to allocate new storagepool for this
volume*
*2013-05-16 15:29:40,116 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) Calling StoragePoolAllocators to find suitable
pools*
*2013-05-16 15:29:40,117 DEBUG
[storage.allocator.FirstFitStoragePoolAllocator] (Job-Executor-6:job-18181)
Looking for pools in dc: 1 pod:1 cluster:1*
*2013-05-16 15:29:40,118 DEBUG
[storage.allocator.FirstFitStoragePoolAllocator] (Job-Executor-6:job-18181)
FirstFitStoragePoolAllocator has 1 pools to check for allocation*
*2013-05-16 15:29:40,118 DEBUG
[storage.allocator.AbstractStoragePoolAllocator] (Job-Executor-6:job-18181)
Checking if storage pool is suitable, name: ps ,poolId: 200*
*2013-05-16 15:29:40,118 DEBUG
[storage.allocator.AbstractStoragePoolAllocator] (Job-Executor-6:job-18181)
StoragePool is in avoid set, skipping this pool*
*2013-05-16 15:29:40,118 DEBUG
[storage.allocator.FirstFitStoragePoolAllocator] (Job-Executor-6:job-18181)
FirstFitStoragePoolAllocator returning 0 suitable storage pools*
*2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) No suitable pools found for volume:
Vol[2438|vm=2237|ROOT] under cluster: 1*
*2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) No suitable pools found*
*2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) No suitable storagePools found under this
Cluster: 1*
*2013-05-16 15:29:40,118 DEBUG [cloud.deploy.FirstFitPlanner]
(Job-Executor-6:job-18181) Could not find suitable Deployment Destination
for this VM under any clusters, returning.*
*2013-05-16 15:29:40,135 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181) VM state transitted from :Starting to Stopped
with event: OperationFailedvm's original host id: null new host id: null
host id before state transition: 3*
*2013-05-16 15:29:40,139 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181) Hosts's actual total CPU: 63984 and CPU after
applying overprovisioning: 191952*
*2013-05-16 15:29:40,139 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-18181) release cpu from host: 3, old used:
29000,reserved: 0, actual total: 63984, total with overprovisioning:
191952; new used: 28500,reserved:0; movedfromreserved:
false,moveToReserveredfalse*