You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by "bradh352 (via GitHub)" <gi...@apache.org> on 2025/10/28 15:16:55 UTC
[D] Agent stuck in "Connecting" state, 4.21 [cloudstack]
GitHub user bradh352 created a discussion: Agent stuck in "Connecting" state, 4.21
I have an agent stuck in the "Connecting" state, it seems like its the fault of the management server in some way as the agent just keeps restarting
The interesting log on the management server says it can't acquire a lock on the host, I'm assuming this is something DB related?
```
2025-10-28 15:04:36,040 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentConnectTaskPool-2:[ctx-abe4ba59]) (logid:c6a383ed) Failed to handle host connection: com.cloud.exception.ConnectionException: Unable to acquire lock on host Host {"id":31,"name":"hv102.internal.nexthop.ai","type":"Routing","uuid":"8a8a2f0b-be76-4179-a2b9-c9785a81d061"}, to process agent connection
```
GitHub link: https://github.com/apache/cloudstack/discussions/11920
----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org
Re: [D] Agent stuck in "Connecting" state, 4.21 [cloudstack]
Posted by "bradh352 (via GitHub)" <gi...@apache.org>.
GitHub user bradh352 closed a discussion: Agent stuck in "Connecting" state, 4.21
I have an agent stuck in the "Connecting" state, it seems like its the fault of the management server in some way as the agent just keeps restarting
The interesting log on the management server says it can't acquire a lock on the host, I'm assuming this is something DB related?
```
2025-10-28 15:04:36,040 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentConnectTaskPool-2:[ctx-abe4ba59]) (logid:c6a383ed) Failed to handle host connection: com.cloud.exception.ConnectionException: Unable to acquire lock on host Host {"id":31,"name":"hv102.internal.nexthop.ai","type":"Routing","uuid":"8a8a2f0b-be76-4179-a2b9-c9785a81d061"}, to process agent connection
```
GitHub link: https://github.com/apache/cloudstack/discussions/11920
----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org
Re: [D] Agent stuck in "Connecting" state, 4.21 [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
GitHub user weizhouapache added a comment to the discussion: Agent stuck in "Connecting" state, 4.21
I faced this issue some times in the past, but did not encounter recently.
GitHub link: https://github.com/apache/cloudstack/discussions/11920#discussioncomment-14805722
----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org
Re: [D] Agent stuck in "Connecting" state, 4.21 [cloudstack]
Posted by "bradh352 (via GitHub)" <gi...@apache.org>.
GitHub user bradh352 added a comment to the discussion: Agent stuck in "Connecting" state, 4.21
I restarted by DB nodes and the agent was then able to connect, so seems like some long lived SQL lock? The lock survived restarting of all the management nodes, which seems exceptionally strange. Locks should always die when the connection dies, no?
GitHub link: https://github.com/apache/cloudstack/discussions/11920#discussioncomment-14805565
----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org
Re: [D] Agent stuck in "Connecting" state, 4.21 [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
GitHub user DaanHoogland added a comment to the discussion: Agent stuck in "Connecting" state, 4.21
this is an “interesting” problem. Could be in the connector or the DB itself. bit of an expensive experiment, but I wonder if the lock would get freed on a tcp timeout…?
GitHub link: https://github.com/apache/cloudstack/discussions/11920#discussioncomment-15137257
----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org
Re: [D] Agent stuck in "Connecting" state, 4.21 [cloudstack]
Posted by "bhouse-nexthop (via GitHub)" <gi...@apache.org>.
GitHub user bhouse-nexthop added a comment to the discussion: Agent stuck in "Connecting" state, 4.21
I think ultimately everything is inter-related to the libvirt 10.0.0 issue on ubuntu 24. I haven't yet tried @weizhouapache 10.6.0 deb package, but its on my list. For now restarting libvirt and cloudstack-agent daily has helped a lot.
GitHub link: https://github.com/apache/cloudstack/discussions/11920#discussioncomment-15137306
----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org
Re: [D] Agent stuck in "Connecting" state, 4.21 [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
GitHub user weizhouapache added a comment to the discussion: Agent stuck in "Connecting" state, 4.21
> I think ultimately everything is inter-related to the libvirt 10.0.0 issue on ubuntu 24. I haven't yet tried @weizhouapache 10.6.0 deb package, but its on my list. For now restarting libvirt and cloudstack-agent daily has helped a lot.
yes, could be related.
let's see if this will appear again when libvirt/agent are restarted daily.
GitHub link: https://github.com/apache/cloudstack/discussions/11920#discussioncomment-15137443
----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org