You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Hieu Le <hi...@gmail.com> on 2012/10/09 03:58:19 UTC
Unable to add more XCP host in cluster
Hi everyone,
I have some troubles while adding more XCP host in cluster. I have 2
running hosts which are homogeneous and another host that is inhomogeneous.
Now I want to add this inhomogeneous host to CS in the same cluster with 2
running hosts but it is always fail. The step I follow is:
- Force the inhomogeneous to join existing pool with xe command (xe
pool-join --force=true --master-address=....)
- Add new host in the same cluster with 2 existing hosts via CS UI.
- Getting the error.
The error log I got here is:
2012-10-09 09:19:04,334 DEBUG [cloud.storage.StorageManagerImpl]
(catalina-exec-2:null) Adding pool XenPrimary to host 14
2012-10-09 09:19:04,336 DEBUG [agent.transport.Request]
(catalina-exec-2:null) Seq 14-976879620: Sending { Cmd , MgmtId:
148045887322015, via: 14, Ver: v1, Flags: 100011, [{"ModifyStoragePoolC$
2012-10-09 09:19:04,337 DEBUG [agent.transport.Request]
(catalina-exec-2:null) Seq 14-976879620: Executing: { Cmd , MgmtId:
148045887322015, via: 14, Ver: v1, Flags: 100011, [{"ModifyStoragePo$
2012-10-09 09:19:04,337 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-20:null) Seq 14-976879620: Executing request
2012-10-09 09:19:04,381 WARN [xen.resource.CitrixResourceBase]
(DirectAgent-20:null) ModifyStoragePoolCommand add XenAPIException:Can not
see storage pool: b43e6f96-d047-3f77-ab5c-5a2abb13a732$
com.cloud.utils.exception.CloudRuntimeException: Can not see storage pool:
b43e6f96-d047-3f77-ab5c-5a2abb13a732 from on
host:1949af01-e15e-467c-8d27-42678a5f826c
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.getStorageRepository(CitrixResourceBase.java:6217)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:4617)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:426)
at
com.cloud.hypervisor.xen.resource.XcpServerResource.executeRequest(XcpServerResource.java:48)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2012-10-09 09:19:04,382 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-20:null) Seq 14-976879620: Response Received:
2012-10-09 09:19:04,382 DEBUG [agent.transport.Request]
(DirectAgent-20:null) Seq 14-976879620: Processing: { Ans: , MgmtId:
148045887322015, via: 14, Ver: v1, Flags: 10, [{"Answer":{"result":$
2012-10-09 09:19:04,382 WARN [cloud.vm.VirtualMachineManagerImpl]
(DirectAgent-20:null) Cleanup failed due to ModifyStoragePoolCommand add
XenAPIException:Can not see storage pool: b43e6f96-d0$
2012-10-09 09:19:04,382 DEBUG [agent.transport.Request]
(catalina-exec-2:null) Seq 14-976879620: Received: { Ans: , MgmtId:
148045887322015, via: 14, Ver: v1, Flags: 10, { Answer } }
2012-10-09 09:19:04,383 WARN [cloud.vm.VirtualMachineManagerImpl]
(catalina-exec-2:null) Cleanup failed due to ModifyStoragePoolCommand add
XenAPIException:Can not see storage pool: b43e6f96-d$
2012-10-09 09:19:04,383 DEBUG [agent.manager.AgentManagerImpl]
(catalina-exec-2:null) Details from executing class
com.cloud.agent.api.ModifyStoragePoolCommand: ModifyStoragePoolCommand add
Xen$
2012-10-09 09:19:04,461 WARN [agent.manager.AgentManagerImpl]
(catalina-exec-2:null) Monitor StoragePoolMonitor says there is an error in
the connect process for 14 due to Unable to connect to$
2012-10-09 09:19:04,461 INFO [agent.manager.AgentManagerImpl]
(catalina-exec-2:null) Host 14 is disconnecting with event AgentDisconnected
2012-10-09 09:19:04,462 DEBUG [agent.manager.AgentManagerImpl]
(catalina-exec-2:null) The next status of agent 14is Alert, current status
is Connecting
2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
(catalina-exec-2:null) Deregistering link for 14 with state Alert
2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
(catalina-exec-2:null) Remove Agent : 14
2012-10-09 09:19:04,463 DEBUG [agent.manager.DirectAgentAttache]
(catalina-exec-2:null) Processing disconnect 14
2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
(catalina-exec-2:null) Sending Disconnect to listener:
com.cloud.hypervisor.xen.discoverer.XcpServerDiscoverer$$EnhancerByCGLIB$$4$
2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
(catalina-exec-2:null) Sending Disconnect to listener:
com.cloud.hypervisor.vmware.VmwareManagerImpl$$EnhancerByCGLIB$$8d05fd47
2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
(catalina-exec-2:null) Sending Disconnect to listener:
com.cloud.vm.ClusteredVirtualMachineManagerImpl$$EnhancerByCGLIB$$4b7c62a0
2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
(catalina-exec-2:null) Sending Disconnect to listener:
com.cloud.network.security.SecurityGroupListener
2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
(catalina-exec-2:null) Sending Disconnect to listener:
com.cloud.storage.listener.StoragePoolMonitor
2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
(catalina-exec-2:null) Sending Disconnect to listener:
com.cloud.storage.secondary.SecondaryStorageListener
2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
(catalina-exec-2:null) Sending Disconnect to listener:
com.cloud.network.NetworkManagerImpl$$EnhancerByCGLIB$$f53c41bc
2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
(catalina-exec-2:null) Sending Disconnect to listener:
com.cloud.agent.manager.AgentMonitor$$EnhancerByCGLIB$$65dcc41c
2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
(catalina-exec-2:null) Sending Disconnect to listener:
com.cloud.storage.download.DownloadListener
2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
(catalina-exec-2:null) Sending Disconnect to listener:
com.cloud.storage.upload.UploadListener
2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
(catalina-exec-2:null) Sending Disconnect to listener:
com.cloud.network.SshKeysDistriMonitor
2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
(catalina-exec-2:null) Sending Disconnect to listener:
com.cloud.network.router.VirtualNetworkApplianceManagerImpl$$EnhancerByCGLI$
2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
(catalina-exec-2:null) Sending Disconnect to listener:
com.cloud.capacity.StorageCapacityListener
2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
(catalina-exec-2:null) Sending Disconnect to listener:
com.cloud.capacity.ComputeCapacityListener
2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
(catalina-exec-2:null) Sending Disconnect to listener:
com.cloud.consoleproxy.ConsoleProxyListener
2012-10-09 09:19:04,464 DEBUG [agent.manager.AgentManagerImpl]
(catalina-exec-2:null) Sending Disconnect to listener:
com.cloud.network.NetworkUsageManagerImpl$DirectNetworkStatsListener
2012-10-09 09:19:04,464 DEBUG [cloud.network.NetworkUsageManagerImpl]
(catalina-exec-2:null) Disconnected called on 14 with status Alert
2012-10-09 09:19:04,464 DEBUG [cloud.host.Status] (catalina-exec-2:null)
Transition:[Resource state = Enabled, Agent event = AgentDisconnected, Host
id = 14, name = node1.viettelrd.vn]
2012-10-09 09:19:04,532 DEBUG [cloud.host.Status] (catalina-exec-2:null)
Agent status update: [id = 14; name = node1.viettelrd.vn; old status =
Connecting; event = AgentDisconnected; new status$
2012-10-09 09:19:04,532 DEBUG [agent.manager.ClusteredAgentManagerImpl]
(catalina-exec-2:null) Notifying other nodes of to disconnect
2012-10-09 09:19:04,534 WARN [cloud.resource.ResourceManagerImpl]
(catalina-exec-2:null) Unable to connect due to
com.cloud.exception.ConnectionException: Unable to connect to pool
Pool[200|NetworkFilesystem]
at
com.cloud.storage.listener.StoragePoolMonitor.processConnect(StoragePoolMonitor.java:91)
at
com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsOfConnection(AgentManagerImpl.java:601)
at
com.cloud.agent.manager.AgentManagerImpl.handleDirectConnectAgent(AgentManagerImpl.java:1496)
at
com.cloud.resource.ResourceManagerImpl.createHostAndAgent(ResourceManagerImpl.java:1574)
at
com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:677)
at
com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:514)
at com.cloud.api.commands.AddHostCmd.execute(AddHostCmd.java:136)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:132)
at com.cloud.api.ApiServer.queueCommand(ApiServer.java:509)
at com.cloud.api.ApiServer.handleRequest(ApiServer.java:416)
at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:300)
at com.cloud.api.ApiServlet.doGet(ApiServlet.java:59)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
at
org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2268)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
Caused by: com.cloud.exception.StorageUnavailableException: Resource
[StoragePool:200] is unreachable: Unable establish connection from storage
head to storage pool 200 due to ModifyStoragePool$
at
com.cloud.storage.StorageManagerImpl.connectHostToSharedPool(StorageManagerImpl.java:1567)
at
com.cloud.storage.listener.StoragePoolMonitor.processConnect(StoragePoolMonitor.java:88)
... 28 more
2012-10-09 09:19:04,535 DEBUG [cloud.host.Status] (catalina-exec-2:null)
Transition:[Resource state = Enabled, Agent event = AgentDisconnected, Host
id = 14, name = node1.viettelrd.vn]
2012-10-09 09:19:04,535 DEBUG [cloud.host.Status] (catalina-exec-2:null)
Cannot transit agent status with event AgentDisconnected for host 14, name=
node1.viettelrd.vn, mangement server id is 14$
2012-10-09 09:19:04,535 ERROR [cloud.api.ApiDispatcher]
(catalina-exec-2:null) Exception while executing AddHostCmd:
com.cloud.utils.exception.CloudRuntimeException: Cannot transit agent
status with event AgentDisconnected for host 14, mangement server id is
148045887322015,Unable to transition to a new state$
at
com.cloud.agent.manager.AgentManagerImpl.agentStatusTransitTo(AgentManagerImpl.java:1428)
at
com.cloud.resource.ResourceManagerImpl.createHostAndAgent(ResourceManagerImpl.java:1588)
at
com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:677)
at
com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:514)
at com.cloud.api.commands.AddHostCmd.execute(AddHostCmd.java:136)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:132)
at com.cloud.api.ApiServer.queueCommand(ApiServer.java:509)
at com.cloud.api.ApiServer.handleRequest(ApiServer.java:416)
at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:300)
at com.cloud.api.ApiServlet.doGet(ApiServlet.java:59)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
at
org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2268)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2012-10-09 09:19:08,143 WARN [xen.resource.CitrixResourceBase]
(DirectAgent-9:null) ModifyStoragePoolCommand add XenAPIException:Can not
see storage pool: b43e6f96-d047-3f77-ab5c-5a2abb13a732 $
com.cloud.utils.exception.CloudRuntimeException: Can not see storage pool:
b43e6f96-d047-3f77-ab5c-5a2abb13a732 from on
host:1949af01-e15e-467c-8d27-42678a5f826c
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.getStorageRepository(CitrixResourceBase.java:6217)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:4617)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:426)
at
com.cloud.hypervisor.xen.resource.XcpServerResource.executeRequest(XcpServerResource.java:48)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
- When force joining the inhomogeneous host, Xen Center throws an error
that it can not see the SR (primary storage) and can not connect to host
via PBD.
Please support !
Best regards !
--
------------------------------------
Le Quang Hieu
Specialist – Core Cloud Computing Dept
Cloud Computing Research Center
Viettel Research and Development Institute
No. 380 Lac Long Quan Str, Tay Ho Dist, Hanoi, Vietnam
Mobile: (84) 974616850
Re: Unable to add more XCP host in cluster
Posted by Hieu Le <hi...@gmail.com>.
Yes I have checked it. The error shown there is still no PBD for connecting
SR with new host.
Regards.
On Tue, Oct 9, 2012 at 9:38 AM, Kirk Kosinski <ki...@gmail.com>wrote:
> Did you check the storage configuration to see why the new host cannot
> access it? There may be some additional details on the error in
> /var/log/xensource.log and SMlog.
>
> Regards,
> Kirk
>
> On 10/08/2012 06:58 PM, Hieu Le wrote:
> > Hi everyone,
> >
> > I have some troubles while adding more XCP host in cluster. I have 2
> > running hosts which are homogeneous and another host that is
> inhomogeneous.
> > Now I want to add this inhomogeneous host to CS in the same cluster with
> 2
> > running hosts but it is always fail. The step I follow is:
> >
> > - Force the inhomogeneous to join existing pool with xe command (xe
> > pool-join --force=true --master-address=....)
> > - Add new host in the same cluster with 2 existing hosts via CS UI.
> > - Getting the error.
> >
> > The error log I got here is:
> >
> > 2012-10-09 09:19:04,334 DEBUG [cloud.storage.StorageManagerImpl]
> > (catalina-exec-2:null) Adding pool XenPrimary to host 14
> > 2012-10-09 09:19:04,336 DEBUG [agent.transport.Request]
> > (catalina-exec-2:null) Seq 14-976879620: Sending { Cmd , MgmtId:
> > 148045887322015, via: 14, Ver: v1, Flags: 100011, [{"ModifyStoragePoolC$
> > 2012-10-09 09:19:04,337 DEBUG [agent.transport.Request]
> > (catalina-exec-2:null) Seq 14-976879620: Executing: { Cmd , MgmtId:
> > 148045887322015, via: 14, Ver: v1, Flags: 100011, [{"ModifyStoragePo$
> > 2012-10-09 09:19:04,337 DEBUG [agent.manager.DirectAgentAttache]
> > (DirectAgent-20:null) Seq 14-976879620: Executing request
> > 2012-10-09 09:19:04,381 WARN [xen.resource.CitrixResourceBase]
> > (DirectAgent-20:null) ModifyStoragePoolCommand add XenAPIException:Can
> not
> > see storage pool: b43e6f96-d047-3f77-ab5c-5a2abb13a732$
> > com.cloud.utils.exception.CloudRuntimeException: Can not see storage
> pool:
> > b43e6f96-d047-3f77-ab5c-5a2abb13a732 from on
> > host:1949af01-e15e-467c-8d27-42678a5f826c
> > at
> >
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getStorageRepository(CitrixResourceBase.java:6217)
> > at
> >
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:4617)
> > at
> >
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:426)
> > at
> >
> com.cloud.hypervisor.xen.resource.XcpServerResource.executeRequest(XcpServerResource.java:48)
> > at
> >
> com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)
> > at
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> > at
> > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> > at
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> > at
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> > at java.lang.Thread.run(Thread.java:679)
> > 2012-10-09 09:19:04,382 DEBUG [agent.manager.DirectAgentAttache]
> > (DirectAgent-20:null) Seq 14-976879620: Response Received:
> > 2012-10-09 09:19:04,382 DEBUG [agent.transport.Request]
> > (DirectAgent-20:null) Seq 14-976879620: Processing: { Ans: , MgmtId:
> > 148045887322015, via: 14, Ver: v1, Flags: 10, [{"Answer":{"result":$
> > 2012-10-09 09:19:04,382 WARN [cloud.vm.VirtualMachineManagerImpl]
> > (DirectAgent-20:null) Cleanup failed due to ModifyStoragePoolCommand add
> > XenAPIException:Can not see storage pool: b43e6f96-d0$
> > 2012-10-09 09:19:04,382 DEBUG [agent.transport.Request]
> > (catalina-exec-2:null) Seq 14-976879620: Received: { Ans: , MgmtId:
> > 148045887322015, via: 14, Ver: v1, Flags: 10, { Answer } }
> > 2012-10-09 09:19:04,383 WARN [cloud.vm.VirtualMachineManagerImpl]
> > (catalina-exec-2:null) Cleanup failed due to ModifyStoragePoolCommand add
> > XenAPIException:Can not see storage pool: b43e6f96-d$
> > 2012-10-09 09:19:04,383 DEBUG [agent.manager.AgentManagerImpl]
> > (catalina-exec-2:null) Details from executing class
> > com.cloud.agent.api.ModifyStoragePoolCommand: ModifyStoragePoolCommand
> add
> > Xen$
> > 2012-10-09 09:19:04,461 WARN [agent.manager.AgentManagerImpl]
> > (catalina-exec-2:null) Monitor StoragePoolMonitor says there is an error
> in
> > the connect process for 14 due to Unable to connect to$
> > 2012-10-09 09:19:04,461 INFO [agent.manager.AgentManagerImpl]
> > (catalina-exec-2:null) Host 14 is disconnecting with event
> AgentDisconnected
> > 2012-10-09 09:19:04,462 DEBUG [agent.manager.AgentManagerImpl]
> > (catalina-exec-2:null) The next status of agent 14is Alert, current
> status
> > is Connecting
> > 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> > (catalina-exec-2:null) Deregistering link for 14 with state Alert
> > 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> > (catalina-exec-2:null) Remove Agent : 14
> > 2012-10-09 09:19:04,463 DEBUG [agent.manager.DirectAgentAttache]
> > (catalina-exec-2:null) Processing disconnect 14
> > 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> > (catalina-exec-2:null) Sending Disconnect to listener:
> >
> com.cloud.hypervisor.xen.discoverer.XcpServerDiscoverer$$EnhancerByCGLIB$$4$
> > 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> > (catalina-exec-2:null) Sending Disconnect to listener:
> > com.cloud.hypervisor.vmware.VmwareManagerImpl$$EnhancerByCGLIB$$8d05fd47
> > 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> > (catalina-exec-2:null) Sending Disconnect to listener:
> >
> com.cloud.vm.ClusteredVirtualMachineManagerImpl$$EnhancerByCGLIB$$4b7c62a0
> > 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> > (catalina-exec-2:null) Sending Disconnect to listener:
> > com.cloud.network.security.SecurityGroupListener
> > 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> > (catalina-exec-2:null) Sending Disconnect to listener:
> > com.cloud.storage.listener.StoragePoolMonitor
> > 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> > (catalina-exec-2:null) Sending Disconnect to listener:
> > com.cloud.storage.secondary.SecondaryStorageListener
> > 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> > (catalina-exec-2:null) Sending Disconnect to listener:
> > com.cloud.network.NetworkManagerImpl$$EnhancerByCGLIB$$f53c41bc
> > 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> > (catalina-exec-2:null) Sending Disconnect to listener:
> > com.cloud.agent.manager.AgentMonitor$$EnhancerByCGLIB$$65dcc41c
> > 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> > (catalina-exec-2:null) Sending Disconnect to listener:
> > com.cloud.storage.download.DownloadListener
> > 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> > (catalina-exec-2:null) Sending Disconnect to listener:
> > com.cloud.storage.upload.UploadListener
> > 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> > (catalina-exec-2:null) Sending Disconnect to listener:
> > com.cloud.network.SshKeysDistriMonitor
> > 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> > (catalina-exec-2:null) Sending Disconnect to listener:
> >
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl$$EnhancerByCGLI$
> > 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> > (catalina-exec-2:null) Sending Disconnect to listener:
> > com.cloud.capacity.StorageCapacityListener
> > 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> > (catalina-exec-2:null) Sending Disconnect to listener:
> > com.cloud.capacity.ComputeCapacityListener
> > 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> > (catalina-exec-2:null) Sending Disconnect to listener:
> > com.cloud.consoleproxy.ConsoleProxyListener
> > 2012-10-09 09:19:04,464 DEBUG [agent.manager.AgentManagerImpl]
> > (catalina-exec-2:null) Sending Disconnect to listener:
> > com.cloud.network.NetworkUsageManagerImpl$DirectNetworkStatsListener
> > 2012-10-09 09:19:04,464 DEBUG [cloud.network.NetworkUsageManagerImpl]
> > (catalina-exec-2:null) Disconnected called on 14 with status Alert
> > 2012-10-09 09:19:04,464 DEBUG [cloud.host.Status] (catalina-exec-2:null)
> > Transition:[Resource state = Enabled, Agent event = AgentDisconnected,
> Host
> > id = 14, name = node1.viettelrd.vn]
> > 2012-10-09 09:19:04,532 DEBUG [cloud.host.Status] (catalina-exec-2:null)
> > Agent status update: [id = 14; name = node1.viettelrd.vn; old status =
> > Connecting; event = AgentDisconnected; new status$
> > 2012-10-09 09:19:04,532 DEBUG [agent.manager.ClusteredAgentManagerImpl]
> > (catalina-exec-2:null) Notifying other nodes of to disconnect
> > 2012-10-09 09:19:04,534 WARN [cloud.resource.ResourceManagerImpl]
> > (catalina-exec-2:null) Unable to connect due to
> > com.cloud.exception.ConnectionException: Unable to connect to pool
> > Pool[200|NetworkFilesystem]
> > at
> >
> com.cloud.storage.listener.StoragePoolMonitor.processConnect(StoragePoolMonitor.java:91)
> > at
> >
> com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsOfConnection(AgentManagerImpl.java:601)
> > at
> >
> com.cloud.agent.manager.AgentManagerImpl.handleDirectConnectAgent(AgentManagerImpl.java:1496)
> > at
> >
> com.cloud.resource.ResourceManagerImpl.createHostAndAgent(ResourceManagerImpl.java:1574)
> > at
> >
> com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:677)
> > at
> >
> com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:514)
> > at com.cloud.api.commands.AddHostCmd.execute(AddHostCmd.java:136)
> > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:132)
> > at com.cloud.api.ApiServer.queueCommand(ApiServer.java:509)
> > at com.cloud.api.ApiServer.handleRequest(ApiServer.java:416)
> > at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:300)
> > at com.cloud.api.ApiServlet.doGet(ApiServlet.java:59)
> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
> > at
> >
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
> > at
> >
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> > at
> >
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> > at
> >
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> > at
> >
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> > at
> >
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> > at
> > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
> > at
> >
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> > at
> >
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
> > at
> >
> org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
> > at
> >
> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
> > at
> >
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2268)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> > at java.lang.Thread.run(Thread.java:679)
> > Caused by: com.cloud.exception.StorageUnavailableException: Resource
> > [StoragePool:200] is unreachable: Unable establish connection from
> storage
> > head to storage pool 200 due to ModifyStoragePool$
> > at
> >
> com.cloud.storage.StorageManagerImpl.connectHostToSharedPool(StorageManagerImpl.java:1567)
> > at
> >
> com.cloud.storage.listener.StoragePoolMonitor.processConnect(StoragePoolMonitor.java:88)
> > ... 28 more
> > 2012-10-09 09:19:04,535 DEBUG [cloud.host.Status] (catalina-exec-2:null)
> > Transition:[Resource state = Enabled, Agent event = AgentDisconnected,
> Host
> > id = 14, name = node1.viettelrd.vn]
> > 2012-10-09 09:19:04,535 DEBUG [cloud.host.Status] (catalina-exec-2:null)
> > Cannot transit agent status with event AgentDisconnected for host 14,
> name=
> > node1.viettelrd.vn, mangement server id is 14$
> > 2012-10-09 09:19:04,535 ERROR [cloud.api.ApiDispatcher]
> > (catalina-exec-2:null) Exception while executing AddHostCmd:
> > com.cloud.utils.exception.CloudRuntimeException: Cannot transit agent
> > status with event AgentDisconnected for host 14, mangement server id is
> > 148045887322015,Unable to transition to a new state$
> > at
> >
> com.cloud.agent.manager.AgentManagerImpl.agentStatusTransitTo(AgentManagerImpl.java:1428)
> > at
> >
> com.cloud.resource.ResourceManagerImpl.createHostAndAgent(ResourceManagerImpl.java:1588)
> > at
> >
> com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:677)
> > at
> >
> com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:514)
> > at com.cloud.api.commands.AddHostCmd.execute(AddHostCmd.java:136)
> > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:132)
> > at com.cloud.api.ApiServer.queueCommand(ApiServer.java:509)
> > at com.cloud.api.ApiServer.handleRequest(ApiServer.java:416)
> > at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:300)
> > at com.cloud.api.ApiServlet.doGet(ApiServlet.java:59)
> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
> > at
> >
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
> > at
> >
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> > at
> >
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> > at
> >
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> > at
> >
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> > at
> >
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> > at
> > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
> > at
> >
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> > at
> >
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
> > at
> >
> org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
> > at
> >
> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
> > at
> >
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2268)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> > at java.lang.Thread.run(Thread.java:679)
> > 2012-10-09 09:19:08,143 WARN [xen.resource.CitrixResourceBase]
> > (DirectAgent-9:null) ModifyStoragePoolCommand add XenAPIException:Can not
> > see storage pool: b43e6f96-d047-3f77-ab5c-5a2abb13a732 $
> > com.cloud.utils.exception.CloudRuntimeException: Can not see storage
> pool:
> > b43e6f96-d047-3f77-ab5c-5a2abb13a732 from on
> > host:1949af01-e15e-467c-8d27-42678a5f826c
> > at
> >
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getStorageRepository(CitrixResourceBase.java:6217)
> > at
> >
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:4617)
> > at
> >
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:426)
> > at
> >
> com.cloud.hypervisor.xen.resource.XcpServerResource.executeRequest(XcpServerResource.java:48)
> > at
> >
> com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)
> > at
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> > at
> > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> > at
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> > at
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> > at java.lang.Thread.run(Thread.java:679)
> >
> > - When force joining the inhomogeneous host, Xen Center throws an error
> > that it can not see the SR (primary storage) and can not connect to host
> > via PBD.
> >
> > Please support !
> >
> > Best regards !
> >
>
--
------------------------------------
Le Quang Hieu
Specialist – Core Cloud Computing Dept
Cloud Computing Research Center
Viettel Research and Development Institute
No. 380 Lac Long Quan Str, Tay Ho Dist, Hanoi, Vietnam
Mobile: (84) 974616850
Re: Unable to add more XCP host in cluster
Posted by Kirk Kosinski <ki...@gmail.com>.
Did you check the storage configuration to see why the new host cannot
access it? There may be some additional details on the error in
/var/log/xensource.log and SMlog.
Regards,
Kirk
On 10/08/2012 06:58 PM, Hieu Le wrote:
> Hi everyone,
>
> I have some troubles while adding more XCP host in cluster. I have 2
> running hosts which are homogeneous and another host that is inhomogeneous.
> Now I want to add this inhomogeneous host to CS in the same cluster with 2
> running hosts but it is always fail. The step I follow is:
>
> - Force the inhomogeneous to join existing pool with xe command (xe
> pool-join --force=true --master-address=....)
> - Add new host in the same cluster with 2 existing hosts via CS UI.
> - Getting the error.
>
> The error log I got here is:
>
> 2012-10-09 09:19:04,334 DEBUG [cloud.storage.StorageManagerImpl]
> (catalina-exec-2:null) Adding pool XenPrimary to host 14
> 2012-10-09 09:19:04,336 DEBUG [agent.transport.Request]
> (catalina-exec-2:null) Seq 14-976879620: Sending { Cmd , MgmtId:
> 148045887322015, via: 14, Ver: v1, Flags: 100011, [{"ModifyStoragePoolC$
> 2012-10-09 09:19:04,337 DEBUG [agent.transport.Request]
> (catalina-exec-2:null) Seq 14-976879620: Executing: { Cmd , MgmtId:
> 148045887322015, via: 14, Ver: v1, Flags: 100011, [{"ModifyStoragePo$
> 2012-10-09 09:19:04,337 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-20:null) Seq 14-976879620: Executing request
> 2012-10-09 09:19:04,381 WARN [xen.resource.CitrixResourceBase]
> (DirectAgent-20:null) ModifyStoragePoolCommand add XenAPIException:Can not
> see storage pool: b43e6f96-d047-3f77-ab5c-5a2abb13a732$
> com.cloud.utils.exception.CloudRuntimeException: Can not see storage pool:
> b43e6f96-d047-3f77-ab5c-5a2abb13a732 from on
> host:1949af01-e15e-467c-8d27-42678a5f826c
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getStorageRepository(CitrixResourceBase.java:6217)
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:4617)
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:426)
> at
> com.cloud.hypervisor.xen.resource.XcpServerResource.executeRequest(XcpServerResource.java:48)
> at
> com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-10-09 09:19:04,382 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-20:null) Seq 14-976879620: Response Received:
> 2012-10-09 09:19:04,382 DEBUG [agent.transport.Request]
> (DirectAgent-20:null) Seq 14-976879620: Processing: { Ans: , MgmtId:
> 148045887322015, via: 14, Ver: v1, Flags: 10, [{"Answer":{"result":$
> 2012-10-09 09:19:04,382 WARN [cloud.vm.VirtualMachineManagerImpl]
> (DirectAgent-20:null) Cleanup failed due to ModifyStoragePoolCommand add
> XenAPIException:Can not see storage pool: b43e6f96-d0$
> 2012-10-09 09:19:04,382 DEBUG [agent.transport.Request]
> (catalina-exec-2:null) Seq 14-976879620: Received: { Ans: , MgmtId:
> 148045887322015, via: 14, Ver: v1, Flags: 10, { Answer } }
> 2012-10-09 09:19:04,383 WARN [cloud.vm.VirtualMachineManagerImpl]
> (catalina-exec-2:null) Cleanup failed due to ModifyStoragePoolCommand add
> XenAPIException:Can not see storage pool: b43e6f96-d$
> 2012-10-09 09:19:04,383 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Details from executing class
> com.cloud.agent.api.ModifyStoragePoolCommand: ModifyStoragePoolCommand add
> Xen$
> 2012-10-09 09:19:04,461 WARN [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Monitor StoragePoolMonitor says there is an error in
> the connect process for 14 due to Unable to connect to$
> 2012-10-09 09:19:04,461 INFO [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Host 14 is disconnecting with event AgentDisconnected
> 2012-10-09 09:19:04,462 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) The next status of agent 14is Alert, current status
> is Connecting
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Deregistering link for 14 with state Alert
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Remove Agent : 14
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.DirectAgentAttache]
> (catalina-exec-2:null) Processing disconnect 14
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.hypervisor.xen.discoverer.XcpServerDiscoverer$$EnhancerByCGLIB$$4$
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.hypervisor.vmware.VmwareManagerImpl$$EnhancerByCGLIB$$8d05fd47
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.vm.ClusteredVirtualMachineManagerImpl$$EnhancerByCGLIB$$4b7c62a0
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.network.security.SecurityGroupListener
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.storage.listener.StoragePoolMonitor
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.storage.secondary.SecondaryStorageListener
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.network.NetworkManagerImpl$$EnhancerByCGLIB$$f53c41bc
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.agent.manager.AgentMonitor$$EnhancerByCGLIB$$65dcc41c
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.storage.download.DownloadListener
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.storage.upload.UploadListener
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.network.SshKeysDistriMonitor
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl$$EnhancerByCGLI$
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.capacity.StorageCapacityListener
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.capacity.ComputeCapacityListener
> 2012-10-09 09:19:04,463 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.consoleproxy.ConsoleProxyListener
> 2012-10-09 09:19:04,464 DEBUG [agent.manager.AgentManagerImpl]
> (catalina-exec-2:null) Sending Disconnect to listener:
> com.cloud.network.NetworkUsageManagerImpl$DirectNetworkStatsListener
> 2012-10-09 09:19:04,464 DEBUG [cloud.network.NetworkUsageManagerImpl]
> (catalina-exec-2:null) Disconnected called on 14 with status Alert
> 2012-10-09 09:19:04,464 DEBUG [cloud.host.Status] (catalina-exec-2:null)
> Transition:[Resource state = Enabled, Agent event = AgentDisconnected, Host
> id = 14, name = node1.viettelrd.vn]
> 2012-10-09 09:19:04,532 DEBUG [cloud.host.Status] (catalina-exec-2:null)
> Agent status update: [id = 14; name = node1.viettelrd.vn; old status =
> Connecting; event = AgentDisconnected; new status$
> 2012-10-09 09:19:04,532 DEBUG [agent.manager.ClusteredAgentManagerImpl]
> (catalina-exec-2:null) Notifying other nodes of to disconnect
> 2012-10-09 09:19:04,534 WARN [cloud.resource.ResourceManagerImpl]
> (catalina-exec-2:null) Unable to connect due to
> com.cloud.exception.ConnectionException: Unable to connect to pool
> Pool[200|NetworkFilesystem]
> at
> com.cloud.storage.listener.StoragePoolMonitor.processConnect(StoragePoolMonitor.java:91)
> at
> com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsOfConnection(AgentManagerImpl.java:601)
> at
> com.cloud.agent.manager.AgentManagerImpl.handleDirectConnectAgent(AgentManagerImpl.java:1496)
> at
> com.cloud.resource.ResourceManagerImpl.createHostAndAgent(ResourceManagerImpl.java:1574)
> at
> com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:677)
> at
> com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:514)
> at com.cloud.api.commands.AddHostCmd.execute(AddHostCmd.java:136)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:132)
> at com.cloud.api.ApiServer.queueCommand(ApiServer.java:509)
> at com.cloud.api.ApiServer.handleRequest(ApiServer.java:416)
> at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:300)
> at com.cloud.api.ApiServlet.doGet(ApiServlet.java:59)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
> at
> org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
> at
> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2268)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> Caused by: com.cloud.exception.StorageUnavailableException: Resource
> [StoragePool:200] is unreachable: Unable establish connection from storage
> head to storage pool 200 due to ModifyStoragePool$
> at
> com.cloud.storage.StorageManagerImpl.connectHostToSharedPool(StorageManagerImpl.java:1567)
> at
> com.cloud.storage.listener.StoragePoolMonitor.processConnect(StoragePoolMonitor.java:88)
> ... 28 more
> 2012-10-09 09:19:04,535 DEBUG [cloud.host.Status] (catalina-exec-2:null)
> Transition:[Resource state = Enabled, Agent event = AgentDisconnected, Host
> id = 14, name = node1.viettelrd.vn]
> 2012-10-09 09:19:04,535 DEBUG [cloud.host.Status] (catalina-exec-2:null)
> Cannot transit agent status with event AgentDisconnected for host 14, name=
> node1.viettelrd.vn, mangement server id is 14$
> 2012-10-09 09:19:04,535 ERROR [cloud.api.ApiDispatcher]
> (catalina-exec-2:null) Exception while executing AddHostCmd:
> com.cloud.utils.exception.CloudRuntimeException: Cannot transit agent
> status with event AgentDisconnected for host 14, mangement server id is
> 148045887322015,Unable to transition to a new state$
> at
> com.cloud.agent.manager.AgentManagerImpl.agentStatusTransitTo(AgentManagerImpl.java:1428)
> at
> com.cloud.resource.ResourceManagerImpl.createHostAndAgent(ResourceManagerImpl.java:1588)
> at
> com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:677)
> at
> com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:514)
> at com.cloud.api.commands.AddHostCmd.execute(AddHostCmd.java:136)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:132)
> at com.cloud.api.ApiServer.queueCommand(ApiServer.java:509)
> at com.cloud.api.ApiServer.handleRequest(ApiServer.java:416)
> at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:300)
> at com.cloud.api.ApiServlet.doGet(ApiServlet.java:59)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
> at
> org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
> at
> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2268)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-10-09 09:19:08,143 WARN [xen.resource.CitrixResourceBase]
> (DirectAgent-9:null) ModifyStoragePoolCommand add XenAPIException:Can not
> see storage pool: b43e6f96-d047-3f77-ab5c-5a2abb13a732 $
> com.cloud.utils.exception.CloudRuntimeException: Can not see storage pool:
> b43e6f96-d047-3f77-ab5c-5a2abb13a732 from on
> host:1949af01-e15e-467c-8d27-42678a5f826c
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getStorageRepository(CitrixResourceBase.java:6217)
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:4617)
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:426)
> at
> com.cloud.hypervisor.xen.resource.XcpServerResource.executeRequest(XcpServerResource.java:48)
> at
> com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:187)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
>
> - When force joining the inhomogeneous host, Xen Center throws an error
> that it can not see the SR (primary storage) and can not connect to host
> via PBD.
>
> Please support !
>
> Best regards !
>