You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@cloudstack.apache.org by "Sangeetha Hariharan (JIRA)" <ji...@apache.org> on 2013/12/05 01:00:37 UTC
[jira] [Commented] (CLOUDSTACK-2428) HA - When the master host is
disconnected , the host status contines to remain in "Up" state because of
com.cloud.utils.exception.CloudRuntimeException: Unable to reset master of
slave
[ https://issues.apache.org/jira/browse/CLOUDSTACK-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839512#comment-13839512 ]
Sangeetha Hariharan commented on CLOUDSTACK-2428:
-------------------------------------------------
Tested with build from 4.3
Set up - Advanced zone set up with 2 Xenserver 6.2 hosts.
I had few hourly snapshots that are scheduled.
Shutdown master host.
Both the host continue to show up as being in "Up" state:
I see the following exception in the logs:
Both hosts show the status as being "UP" in the cloud platform and following exception is seen:
2013-12-04 18:13:15,510 WARN [c.c.a.m.DirectAgentAttache] (DirectAgent-59:ctx-0bb6ad1e) Seq 1-2071592963: Exception Caught while executing command
com.cloud.utils.exception.CloudRuntimeException: Unable to reset master of slave 10.223.59.66 to 10.223.59.67 due to org.apache.xmlrpc.XmlRpcException: Failed to read server's response: connect timed out
at com.cloud.hypervisor.xen.resource.XenServerConnectionPool.PoolEmergencyResetMaster(XenServerConnectionPool.java:443)
at com.cloud.hypervisor.xen.resource.XenServerConnectionPool.connect(XenServerConnectionPool.java:661)
at com.cloud.hypervisor.xen.resource.CitrixResourceBase.getConnection(CitrixResourceBase.java:5985)
at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:8248)
at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:587)
at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:59)
at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:106)
at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216)
at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2013-12-04 18:13:15,511 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-59:ctx-0bb6ad1e) Seq 1-2071592963: Response Received:
2013-12-04 18:13:15,511 DEBUG [c.c.a.t.Request] (DirectAgent-59:ctx-0bb6ad1e) Seq 1-2071592963: Processing: { Ans: , MgmtId: 112516401760401, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.Answer":{"result":false,"details":"com.cloud.utils.exception.CloudRuntimeException: Unable to reset master of slave 10.223.59.66 to 10.223.59.67 due to org.apache.xmlrpc.XmlRpcException: Failed to read server's response: connect timed out","wait":0}}] }
Also all the snapshots that were sent to the host that is currently down fails with following exception:
2013-12-04 18:10:24,856 DEBUG [o.a.c.s.s.SnapshotServiceImpl] (Job-Executor-27:ctx-6cb3f72a ctx-dedf771a) create snapshot TestVM-1_ROOT-3_201
31204231014 failed: com.cloud.utils.exception.CloudRuntimeException: Unable to reset master of slave 10.223.59.66 to 10.223.59.67 due to org.
apache.xmlrpc.XmlRpcException: Failed to read server's response: connect timed out
2013-12-04 18:10:24,867 DEBUG [o.a.c.s.s.XenserverSnapshotStrategy] (Job-Executor-27:ctx-6cb3f72a ctx-dedf771a) Failed to take snapshot: com.
cloud.utils.exception.CloudRuntimeException: Unable to reset master of slave 10.223.59.66 to 10.223.59.67 due to org.apache.xmlrpc.XmlRpcExce
ption: Failed to read server's response: connect timed out
2013-12-04 18:10:24,872 DEBUG [c.c.s.s.SnapshotManagerImpl] (Job-Executor-27:ctx-6cb3f72a ctx-dedf771a) Failed to create snapshot
com.cloud.utils.exception.CloudRuntimeException: com.cloud.utils.exception.CloudRuntimeException: Unable to reset master of slave 10.223.59.66 to 10.223.59.67 due to org.apache.xmlrpc.XmlRpcException: Failed to read server's response: connect timed out
at org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.takeSnapshot(XenserverSnapshotStrategy.java:281)
at com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:951)
at sun.reflect.GeneratedMethodAccessor334.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy160.takeSnapshot(Unknown Source)
at org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(VolumeServiceImpl.java:1342)
at com.cloud.storage.VolumeApiServiceImpl.takeSnapshot(VolumeApiServiceImpl.java:1402)
at sun.reflect.GeneratedMethodAccessor333.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy232.takeSnapshot(Unknown Source)
at org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd.execute(CreateSnapshotCmd.java:181)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161)
at com.cloud.api.ApiAsyncJobDispatcher.runJobInContext(ApiAsyncJobDispatcher.java:109)
at com.cloud.api.ApiAsyncJobDispatcher$1.run(ApiAsyncJobDispatcher.java:66)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:63)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:520)
at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2013-12-04 18:10:24,883 DEBUG [o.a.c.s.v.VolumeServiceImpl] (Job-Executor-27:ctx-6cb3f72a ctx-dedf771a) Take snapshot: 3 failed
> HA - When the master host is disconnected , the host status contines to remain in "Up" state because of com.cloud.utils.exception.CloudRuntimeException: Unable to reset master of slave
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-2428
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-2428
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the default.)
> Components: Management Server
> Affects Versions: 4.2.0, 4.3.0
> Environment: Build from pvaln
> Reporter: Sangeetha Hariharan
> Assignee: Koushik Das
> Priority: Critical
> Fix For: 4.2.0, 4.3.0
>
> Attachments: logs.rar, logs_7_29, management-server.rar
>
>
> 1. Advance zone with 1 cluster with 2 hosts. Create Shared network with private vlan.
> 2. Deploy few HA enabled Vms in this network.
> 3. pull network cable for one of the host.
> When cloudstack detects that the host is disconnected , it is not able to out the host in disconnected state and start HA for Vms that are HA enabeld,
> I see the following exception in the management server logs:
> 2013-05-09 17:15:55,576 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-267:null) Seq 1-1435828229: Executing request
> 2013-05-09 17:15:55,602 DEBUG [xen.resource.XenServerConnectionPool] (DirectAgent-267:null) Catch Exception: com.xensource.xenapi.Types$HostOffline Host is offline 10.223.81.62 due to You attempted an operation which involves a host which could not be contacted.
> 2013-05-09 17:15:55,603 DEBUG [xen.resource.XenServerConnectionPool] (DirectAgent-267:null) Trying to reset master of slave 10.223.81.62 to 10.223.81.61
> 2013-05-09 17:16:02,319 WARN [xen.resource.CitrixResourceBase] (DirectAgent-265:null) can not ping xenserver 520d4994-8b1f-4dda-b51d-2ee63750abf6
> 2013-05-09 17:16:02,319 WARN [agent.manager.DirectAgentAttache] (DirectAgent-265:null) Unable to get current status on 1
> 2013-05-09 17:16:02,321 INFO [agent.manager.AgentManagerImpl] (AgentTaskPool-11:null) Investigating why host 1 has disconnected with event AgentDisconnected
> 2013-05-09 17:16:02,321 DEBUG [agent.manager.AgentManagerImpl] (AgentTaskPool-11:null) checking if agent (1) is alive
> 2013-05-09 17:16:02,323 DEBUG [agent.transport.Request] (AgentTaskPool-11:null) Seq 1-1435828294: Sending { Cmd , MgmtId: 7647994577963, via: 1, Ver: v1, Flags: 100011, [{"CheckHealthCommand":{"wait":50}}] }
> 2013-05-09 17:16:02,323 DEBUG [agent.transport.Request] (AgentTaskPool-11:null) Seq 1-1435828294: Executing: { Cmd , MgmtId: 7647994577963, via: 1, Ver: v1, Flags: 100011, [{"CheckHealthCommand":{"wait":50}}] }
> 2013-05-09 17:16:02,323 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-271:null) Seq 1-1435828294: Executing request
> 2013-05-09 17:16:04,035 DEBUG [agent.manager.AgentAttache] (AgentTaskPool-10:null) Seq 6-474349576: Waiting some more time because this is the current command
> 2013-05-09 17:16:04,040 DEBUG [xen.resource.XenServerConnectionPool] (DirectAgent-268:null) localLogout has problem Failed to read server's response: connect timed out
> 2013-05-09 17:16:04,040 WARN [agent.manager.DirectAgentAttache] (DirectAgent-268:null) Seq 1-1435828292: Exception Caught while executing command
> com.cloud.utils.exception.CloudRuntimeException: Unable to reset master of slave 10.223.81.62 to 10.223.81.61 due to org.apache.xmlrpc.XmlRpcException: Failed to read server's response: connect timed out
> at com.cloud.hypervisor.xen.resource.XenServerConnectionPool.PoolEmergencyResetMaster(XenServerConnectionPool.java:443)
> at com.cloud.hypervisor.xen.resource.XenServerConnectionPool.connect(XenServerConnectionPool.java:661)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.getConnection(CitrixResourceBase.java:5639)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1682)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:524)
> at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
> at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:102)
> at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2013-05-09 17:16:04,041 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-268:null) Seq 1-1435828292: Response Received:
> 2013-05-09 17:16:04,041 DEBUG [agent.transport.Request] (DirectAgent-268:null) Seq 1-1435828292: Processing: { Ans: , MgmtId: 7647994577963, via: 1, Ver: v1, Flags: 10, [{"Answer":{"result":false,"details":"com.cloud.utils.exception.CloudRuntimeException: Unable to reset master of slave 10.223.81.62 to 10.223.81.61 due to org.apache.xmlrpc.XmlRpcException: Failed to read server's response: connect timed out","wait":0}}] }
> 2013-05-09 17:16:04,041 DEBUG [agent.transport.Request] (AgentTaskPool-5:null) Seq 1-1435828292: Received: { Ans: , MgmtId: 7647994577963, via: 1, Ver: v1, Flags: 10, { Answer } }
> 2013-05-09 17:16:04,041 DEBUG [cloud.ha.AbstractInvestigatorImpl] (AgentTaskPool-5:null) host (10.223.81.50) cannot be pinged, returning null ('I don't know')
> 2013-05-09 17:16:04,041 DEBUG [cloud.ha.UserVmDomRInvestigator] (AgentTaskPool-5:null) sending ping from (5) to agent's host ip address (10.223.81.50)
> 2013-05-09 17:16:04,043 DEBUG [agent.transport.Request] (AgentTaskPool-5:null) Seq 5-2082341067: Sending { Cmd , MgmtId: 7647994577963, via: 5, Ver: v1, Flags: 100011, [{"PingTestCommand":{"_computingHostIp":"10.223.81.50","wait":20}}] }
> 2013-05-09 17:16:04,043 DEBUG [agent.transport.Request] (AgentTaskPool-5:null) Seq 5-2082341067: Executing: { Cmd , MgmtId: 7647994577963, via: 5, Ver: v1, Flags: 100011, [{"PingTestCommand":{"_computingHostIp":"10.223.81.50","wait":20}}] }
> 2013-05-09 17:16:04,043 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-272:null) Seq 5-2082341067: Executing request
> 2013-05-09 17:16:04,053 DEBUG [xen.resource.XenServerConnectionPool] (DirectAgent-91:null) localLogout has problem Failed to read server's response: connect timed out
> 2013-05-09 17:16:04,053 WARN [agent.manager.DirectAgentAttache] (DirectAgent-91:null) Seq 1-1435828293: Exception Caught while executing command
> com.cloud.utils.exception.CloudRuntimeException: Unable to reset master of slave 10.223.81.62 to 10.223.81.61 due to org.apache.xmlrpc.XmlRpcException: Failed to read server's response: connect timed out
> at com.cloud.hypervisor.xen.resource.XenServerConnectionPool.PoolEmergencyResetMaster(XenServerConnectionPool.java:443)
> at com.cloud.hypervisor.xen.resource.XenServerConnectionPool.connect(XenServerConnectionPool.java:661)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.getConnection(CitrixResourceBase.java:5639)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1682)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:524)
> at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
> at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:102)
> at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2013-05-09 17:16:04,054 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-91:null) Seq 1-1435828293: Response Received:
> 2013-05-09 17:16:04,054 DEBUG [agent.transport.Request] (DirectAgent-91:null) Seq 1-1435828293: Processing: { Ans: , MgmtId: 7647994577963, via: 1, Ver: v1, Flags: 10, [{"Answer":{"result":false,"details":"com.cloud.utils.exception.CloudRuntimeException: Unable to reset master of slave 10.223.81.62 to 10.223.81.61 due to org.apache.xmlrpc.XmlRpcException: Failed to read server's response: connect timed out","wait":0}}] }
> 2013-05-09 17:16:04,055 DEBUG [agent.transport.Request] (AgentTaskPool-7:null) Seq 1-1435828293: Received: { Ans: , MgmtId: 7647994577963, via: 1, Ver: v1, Flags: 10, { Answer } }
> 2013-05-09 17:16:04,055 DEBUG [cloud.ha.AbstractInvestigatorImpl] (AgentTaskPool-7:null) host (10.223.81.52) cannot be pinged, returning null ('I don't know')
> 2013-05-09 17:16:04,055 DEBUG [cloud.ha.UserVmDomRInvestigator] (AgentTaskPool-7:null) sending ping from (5) to agent's host ip address (10.223.81.52)
> 2013-05-09 17:16:04,057 DEBUG [agent.transport.Request] (AgentTaskPool-7:null) Seq 5-2082341068: Sending { Cmd , MgmtId: 7647994577963, via: 5, Ver: v1, Flags: 100011, [{"PingTestCommand":{"_computingHostIp":"10.223.81.52","wait":20}}] }
> 2013-05-09 17:16:04,057 DEBUG [agent.manager.AgentAttache] (AgentTaskPool-14:null) Seq 3-1752367195: Waiting some more time because this is the current command
> 2013-05-09 17:16:04,057 DEBUG [agent.transport.Request] (AgentTaskPool-7:null) Seq 5-2082341068: Executing: { Cmd , MgmtId: 7647994577963, via: 5, Ver: v1, Flags: 100011, [{"PingTestCommand":{"_computingHostIp":"10.223.81.52","wait":20}}] }
> 2013-05-09 17:16:04,057 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-91:null) Seq 5-2082341068: Executing request
> 2013-05-09 17:16:05,175 DEBUG [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null) Zone 1 is ready to launch secondary storage VM
> 2013-05-09 17:16:05,614 DEBUG [xen.resource.XenServerConnectionPool] (DirectAgent-267:null) localLogout has problem Failed to read server's response: connect timed out
> 2013-05-09 17:16:05,614 WARN [agent.manager.DirectAgentAttache] (DirectAgent-267:null) Seq 1-1435828229: Exception Caught while executing command
> com.cloud.utils.exception.CloudRuntimeException: Unable to reset master of slave 10.223.81.62 to 10.223.81.61 due to org.apache.xmlrpc.XmlRpcException: Failed to read server's response: connect timed out
> at com.cloud.hypervisor.xen.resource.XenServerConnectionPool.PoolEmergencyResetMaster(XenServerConnectionPool.java:443)
> at com.cloud.hypervisor.xen.resource.XenServerConnectionPool.connect(XenServerConnectionPool.java:661)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.getConnection(CitrixResourceBase.java:5639)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:7725)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:570)
> at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
> at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:102)
> at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2013-05-09 17:16:05,615 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-267:null) Seq 1-1435828229: Response Received:
> 2013-05-09 17:16:05,615 DEBUG [agent.transport.Request] (DirectAgent-267:null) Seq 1-1435828229: Processing: { Ans: , MgmtId: 7647994577963, via: 1, Ver: v1, Flags: 10, [{"Answer":{"result":false,"details":"com.cloud.utils.exception.CloudRuntimeException: Unable to reset master of slave 10.223.81.62 to 10.223.81.61 due to org.apache.xmlrpc.XmlRpcException: Failed to read server's response: connect timed out","wait":0}}] }
> 2013-05-09 17:16:05,704 DEB
--
This message was sent by Atlassian JIRA
(v6.1#6144)