You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Alessandro Caviglione <c....@gmail.com> on 2021/05/12 20:34:58 UTC
Fail to add KVM Host
Hi guys,
a lot of troubles these days...
We've ACS 4.13 installed from source with one zone with a vcenter server
and advanced networking.
Now we need to add another zone to manage some KVM hosts but with basic
networking, we installed libvirt on a Centos7, configured cloudbr0 for
management and cloudbr1 for gest traffic, installed cloudstack-agent, made
all the configuration for tcp and tls, started the services and all it's ok.
So we created a new zone in ACS, with cluster and the above host, all works
fine but the host has not been added.
So I've tried again to add but after some seconds, the blinking row with
the new server disappear and in the log I can't see any error:
2021-05-12 22:18:35,457 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-0b1a9c73) (logid:39eed862) Zone 2 is not ready to launch
secondary storage VM yet
2021-05-12 22:18:37,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-7e5591e8) (logid:442791ae) Begin cleanup
expired async-jobs
2021-05-12 22:18:37,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-7e5591e8) (logid:442791ae) End cleanup expired
async-jobs
2021-05-12 22:18:37,795 DEBUG [c.c.u.s.SSHCmdHelper]
(qtp504527234-17:ctx-dc82de11 ctx-2eb654f7) (logid:201abc34) SSH command:
cloudstack-setup-agent -m 192.168.200.125 -z 2 -p 2 -c 2 -g
92f6da7e-d7f0-3b03-93e8-c588dbf9182a -a -s --pubNic=cloudbr1
--prvNic=cloudbr0 --guestNic=cloudbr1 --hypervisor=kvm
SSH command output:Starting to configure your system:
Configure SElinux ... [OK]
Configure Network ... [OK]
Configure Libvirt ... [OK]
Configure Firewall ... [OK]
Configure Nfs ... [OK]
Configure cloudAgent ... [OK]
CloudStack Agent setup is done!
2021-05-12 22:18:44,977 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-7:null) (logid:) SeqA 6-1063907: Processing Seq
6-1063907: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2021-05-12 22:18:44,979 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-7:null) (logid:) SeqA 6-1063907: Sending Seq
6-1063907: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1, Flags:
100010,
[{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2021-05-12 22:18:47,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-5b168792) (logid:b9c80c90) Begin cleanup
expired async-jobs
2021-05-12 22:18:47,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-5b168792) (logid:b9c80c90) End cleanup expired
async-jobs
2021-05-12 22:18:47,284 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 14 routers to
update status.
2021-05-12 22:18:47,291 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 10 VPC networks
to update Redundant State.
2021-05-12 22:18:47,296 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 0 networks to
update RvR status.
2021-05-12 22:18:49,977 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-2:null) (logid:) SeqA 6-1063908: Processing Seq
6-1063908: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2021-05-12 22:18:49,979 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-2:null) (logid:) SeqA 6-1063908: Sending Seq
6-1063908: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1, Flags:
100010,
[{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2021-05-12 22:18:53,959 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-5:null) (logid:) Ping from 4(s-4-VM)
2021-05-12 22:18:53,997 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-12:null) (logid:) Ping from 6(v-7-VM)
2021-05-12 22:18:57,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-22922b92) (logid:88c162be) Begin cleanup
expired async-jobs
2021-05-12 22:18:57,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-22922b92) (logid:88c162be) End cleanup expired
async-jobs
2021-05-12 22:18:59,978 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-13:null) (logid:) SeqA 6-1063910: Processing Seq
6-1063910: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2021-05-12 22:18:59,980 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-13:null) (logid:) SeqA 6-1063910: Sending Seq
6-1063910: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1, Flags:
100010,
[{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2021-05-12 22:19:02,644 DEBUG [c.c.s.StatsCollector]
(StatsCollector-2:ctx-1237412a) (logid:3c78cd8f) AutoScaling Monitor is
running...
2021-05-12 22:19:02,934 DEBUG [c.c.s.StatsCollector]
(StatsCollector-4:ctx-6c1c6167) (logid:585ce1c2) HostStatsCollector is
running...
2021-05-12 22:19:05,352 DEBUG [c.c.s.StatsCollector]
(StatsCollector-3:ctx-58cb4a3f) (logid:13d34be0) StorageCollector is
running...
Re: Fail to add KVM Host
Posted by Alessandro Caviglione <c....@gmail.com>.
Yep, all working fine!
Thank you!
On Fri, May 14, 2021 at 2:45 AM Nicolas Vazquez <
Nicolas.Vazquez@shapeblue.com> wrote:
> Good thing, confirmed all working good now?
>
> I have created a PR to include this as a note on the documentation:
> https://github.com/apache/cloudstack-documentation/pull/214. Should be
> included on the next releases documentation.
>
>
> Regards,
>
> Nicolas Vazquez
>
> ________________________________
> From: Alessandro Caviglione <c....@gmail.com>
> Sent: Thursday, May 13, 2021 4:36 PM
> To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
> Subject: Re: Fail to add KVM Host
>
> OMG guys... I think you've to specify this requirement in documentation!!
> This row point me in the right direction:
>
> failing to get physical interface from bridge cloudbr1, did not find an
> eth*, bond*, team*, vlan*, em*, p*p*, ens*, eno*, enp*, or enx* in
> /sys/devices/virtual/net/cloudbr1/brif
>
> So the script looks for a bond that starts with: eth*, bond*, team*,
> vlan*, em*, p*p*, ens*, eno*, enp*, or enx*
> I've created a bond named cbdr_bond1 so it was not reachable!!
>
> Thank you for your help and your suggestions!
>
> On Thu, May 13, 2021 at 7:12 PM Alessandro Caviglione <
> c.alessandro@gmail.com> wrote:
>
> > Hi Nicolas,
> > same link for the new logs in folder "LOG2":
> > https://box.beitsa.ch/index.php/s/ZmGMWbQgmMkbzoK
> >
> > Strange thing is that I see in agent.log these lines:
> >
> > 2021-05-13 18:53:08,587 DEBUG [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-5:null) (logid:4b3902b9) matchPifFileInDirectory:
> > file name 'cdbr_bond1'
> > 2021-05-13 18:53:08,587 DEBUG [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-5:null) (logid:4b3902b9) failing to get physical
> > interface from bridge cloudbr1, did not find an eth*, bond*, team*,
> vlan*,
> > em*, p*p*, ens*, eno*, enp*, or enx* in
> > /sys/devices/virtual/net/cloudbr1/brif
> >
> > But as you can see in interfaces.txt , cloudbr1/brif exists as the
> > cloudbr0/brif
> >
> > On Thu, May 13, 2021 at 6:32 PM Nicolas Vazquez <
> > Nicolas.Vazquez@shapeblue.com> wrote:
> >
> >> Still not clear on the issue yet. Could you please increase the log
> level
> >> on /etc/cloudstack/agent/log4j-cloud.xml from INFO to DEBUG and check
> for
> >> any exception trace?
> >>
> >>
> >> Regards,
> >>
> >> Nicolas Vazquez
> >>
> >> ________________________________
> >> From: Alessandro Caviglione <c....@gmail.com>
> >> Sent: Thursday, May 13, 2021 1:05 PM
> >> To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
> >> Subject: Re: Fail to add KVM Host
> >>
> >> Should I install older version of libvirt/qemu or there's another
> problem?
> >>
> >> Il gio 13 mag 2021, 16:05 Alessandro Caviglione <c.alessandro@gmail.com
> >
> >> ha
> >> scritto:
> >>
> >> > Hi,
> >> > yes I saw... but this is a centos 7.9 installed from scratch.
> >> > I've installed with: yum install qemu-kvm libvirt libvirt-python
> >> > libguestfs-tools virt-install
> >> >
> >> > libvirt is 4.5.0
> >> > qemu-kvm is 1.5.3
> >> > cloudstack-agent is 4.13
> >> >
> >> > I've also tried with qemu-kvm-ev 2.12 but nothing changes...
> >> >
> >> >
> >> > On Thu, May 13, 2021 at 3:48 PM Nicolas Vazquez <
> >> > Nicolas.Vazquez@shapeblue.com> wrote:
> >> >
> >> >> Hi Alessandro,
> >> >>
> >> >> From the agent logs the issue comes from this: Connection with
> libvirtd
> >> >> is broken: invalid connection pointer in virConnectGetVersion
> >> >>
> >> >> Which libvirt and qemu versions are you using?
> >> >>
> >> >>
> >> >> Regards,
> >> >>
> >> >> Nicolas Vazquez
> >> >>
> >> >> ________________________________
> >> >> From: Alessandro Caviglione <c....@gmail.com>
> >> >> Sent: Thursday, May 13, 2021 10:12 AM
> >> >> To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
> >> >> Subject: Re: Fail to add KVM Host
> >> >>
> >> >> Hi Andrija,
> >> >> any idea?
> >> >>
> >> >> On Thu, May 13, 2021 at 11:34 AM Alessandro Caviglione <
> >> >> c.alessandro@gmail.com> wrote:
> >> >>
> >> >> > Thank you Andrija!!
> >> >> > https://box.beitsa.ch/index.php/s/ZmGMWbQgmMkbzoK
> >> >> >
> >> >> >
> >> >> > On Thu, May 13, 2021 at 10:04 AM Andrija Panic <
> >> andrija.panic@gmail.com
> >> >> >
> >> >> > wrote:
> >> >> >
> >> >> >> Can you share (on b pastebin. org or somewhere) the content of the
> >> >> >> agent.log and setup. log from the KVM host please?
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> Best,
> >> >> >>
> >> >> >> On Wed, 12 May 2021, 22:45 Alessandro Caviglione, <
> >> >> c.alessandro@gmail.com
> >> >> >> >
> >> >> >> wrote:
> >> >> >>
> >> >> >> > Just a quick news... Now I see the host in hosts list, but it's
> in
> >> >> alert
> >> >> >> > state.
> >> >> >> > In log I see:
> >> >> >> > 2021-05-12 22:40:54,061 DEBUG
> [c.c.a.m.ClusteredAgentManagerImpl]
> >> >> >> > (AgentConnectTaskPool-417:ctx-36dd291a) (logid:7320b6af)
> Notifying
> >> >> other
> >> >> >> > nodes of to disconnect
> >> >> >> > 2021-05-12 22:40:54,062 DEBUG [c.c.a.m.AgentManagerImpl]
> >> >> >> > (AgentConnectTaskPool-417:ctx-36dd291a) (logid:7320b6af) Failed
> to
> >> >> >> handle
> >> >> >> > host connection:
> >> >> >> > com.cloud.exception.ConnectionException: Incorrect Network setup
> >> on
> >> >> >> agent,
> >> >> >> > Reinitialize agent after network names are setup, details : Can
> >> not
> >> >> find
> >> >> >> > network: cloudbr1
> >> >> >> >
> >> >> >> > but I have cloudbr0 and cloudbr1 configured on host...
> >> >> >> > What can I do?
> >> >> >> >
> >> >> >> >
> >> >> >> > On Wed, May 12, 2021 at 10:34 PM Alessandro Caviglione <
> >> >> >> > c.alessandro@gmail.com> wrote:
> >> >> >> >
> >> >> >> > > Hi guys,
> >> >> >> > > a lot of troubles these days...
> >> >> >> > > We've ACS 4.13 installed from source with one zone with a
> >> vcenter
> >> >> >> server
> >> >> >> > > and advanced networking.
> >> >> >> > > Now we need to add another zone to manage some KVM hosts but
> >> with
> >> >> >> basic
> >> >> >> > > networking, we installed libvirt on a Centos7, configured
> >> cloudbr0
> >> >> for
> >> >> >> > > management and cloudbr1 for gest traffic, installed
> >> >> cloudstack-agent,
> >> >> >> > made
> >> >> >> > > all the configuration for tcp and tls, started the services
> and
> >> all
> >> >> >> it's
> >> >> >> > ok.
> >> >> >> > > So we created a new zone in ACS, with cluster and the above
> >> host,
> >> >> all
> >> >> >> > > works fine but the host has not been added.
> >> >> >> > > So I've tried again to add but after some seconds, the
> blinking
> >> row
> >> >> >> with
> >> >> >> > > the new server disappear and in the log I can't see any error:
> >> >> >> > >
> >> >> >> > > 2021-05-12 22:18:35,457 DEBUG
> >> [o.a.c.s.SecondaryStorageManagerImpl]
> >> >> >> > > (secstorage-1:ctx-0b1a9c73) (logid:39eed862) Zone 2 is not
> >> ready to
> >> >> >> > launch
> >> >> >> > > secondary storage VM yet
> >> >> >> > > 2021-05-12 22:18:37,178 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> >> >> > > (AsyncJobMgr-Heartbeat-1:ctx-7e5591e8) (logid:442791ae) Begin
> >> >> cleanup
> >> >> >> > > expired async-jobs
> >> >> >> > > 2021-05-12 22:18:37,181 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> >> >> > > (AsyncJobMgr-Heartbeat-1:ctx-7e5591e8) (logid:442791ae) End
> >> cleanup
> >> >> >> > expired
> >> >> >> > > async-jobs
> >> >> >> > > 2021-05-12 22:18:37,795 DEBUG [c.c.u.s.SSHCmdHelper]
> >> >> >> > > (qtp504527234-17:ctx-dc82de11 ctx-2eb654f7) (logid:201abc34)
> SSH
> >> >> >> command:
> >> >> >> > > cloudstack-setup-agent -m 192.168.200.125 -z 2 -p 2 -c 2 -g
> >> >> >> > > 92f6da7e-d7f0-3b03-93e8-c588dbf9182a -a -s --pubNic=cloudbr1
> >> >> >> > > --prvNic=cloudbr0 --guestNic=cloudbr1 --hypervisor=kvm
> >> >> >> > > SSH command output:Starting to configure your system:
> >> >> >> > > Configure SElinux ... [OK]
> >> >> >> > > Configure Network ... [OK]
> >> >> >> > > Configure Libvirt ... [OK]
> >> >> >> > > Configure Firewall ... [OK]
> >> >> >> > > Configure Nfs ... [OK]
> >> >> >> > > Configure cloudAgent ... [OK]
> >> >> >> > > CloudStack Agent setup is done!
> >> >> >> > >
> >> >> >> > >
> >> >> >> > > 2021-05-12 22:18:44,977 DEBUG [c.c.a.m.AgentManagerImpl]
> >> >> >> > > (AgentManager-Handler-7:null) (logid:) SeqA 6-1063907:
> >> Processing
> >> >> Seq
> >> >> >> > > 6-1063907: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
> >> >> >> > >
> >> >> >> >
> >> >> >>
> >> >>
> >>
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
> >> >> >> > > \"connections\": []\n}","wait":0}}] }
> >> >> >> > > 2021-05-12 22:18:44,979 DEBUG [c.c.a.m.AgentManagerImpl]
> >> >> >> > > (AgentManager-Handler-7:null) (logid:) SeqA 6-1063907: Sending
> >> Seq
> >> >> >> > > 6-1063907: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1,
> >> >> Flags:
> >> >> >> > > 100010,
> >> >> >> > >
> >> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> >> >> >> }
> >> >> >> > > 2021-05-12 22:18:47,178 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> >> >> > > (AsyncJobMgr-Heartbeat-1:ctx-5b168792) (logid:b9c80c90) Begin
> >> >> cleanup
> >> >> >> > > expired async-jobs
> >> >> >> > > 2021-05-12 22:18:47,181 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> >> >> > > (AsyncJobMgr-Heartbeat-1:ctx-5b168792) (logid:b9c80c90) End
> >> cleanup
> >> >> >> > expired
> >> >> >> > > async-jobs
> >> >> >> > > 2021-05-12 22:18:47,284 DEBUG
> >> >> >> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> >> >> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 14
> >> >> >> routers to
> >> >> >> > > update status.
> >> >> >> > > 2021-05-12 22:18:47,291 DEBUG
> >> >> >> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> >> >> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 10
> >> VPC
> >> >> >> > networks
> >> >> >> > > to update Redundant State.
> >> >> >> > > 2021-05-12 22:18:47,296 DEBUG
> >> >> >> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> >> >> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 0
> >> >> >> networks to
> >> >> >> > > update RvR status.
> >> >> >> > > 2021-05-12 22:18:49,977 DEBUG [c.c.a.m.AgentManagerImpl]
> >> >> >> > > (AgentManager-Handler-2:null) (logid:) SeqA 6-1063908:
> >> Processing
> >> >> Seq
> >> >> >> > > 6-1063908: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
> >> >> >> > >
> >> >> >> >
> >> >> >>
> >> >>
> >>
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
> >> >> >> > > \"connections\": []\n}","wait":0}}] }
> >> >> >> > > 2021-05-12 22:18:49,979 DEBUG [c.c.a.m.AgentManagerImpl]
> >> >> >> > > (AgentManager-Handler-2:null) (logid:) SeqA 6-1063908: Sending
> >> Seq
> >> >> >> > > 6-1063908: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1,
> >> >> Flags:
> >> >> >> > > 100010,
> >> >> >> > >
> >> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> >> >> >> }
> >> >> >> > > 2021-05-12 22:18:53,959 DEBUG [c.c.a.m.AgentManagerImpl]
> >> >> >> > > (AgentManager-Handler-5:null) (logid:) Ping from 4(s-4-VM)
> >> >> >> > > 2021-05-12 22:18:53,997 DEBUG [c.c.a.m.AgentManagerImpl]
> >> >> >> > > (AgentManager-Handler-12:null) (logid:) Ping from 6(v-7-VM)
> >> >> >> > > 2021-05-12 22:18:57,178 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> >> >> > > (AsyncJobMgr-Heartbeat-1:ctx-22922b92) (logid:88c162be) Begin
> >> >> cleanup
> >> >> >> > > expired async-jobs
> >> >> >> > > 2021-05-12 22:18:57,181 INFO
> [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> >> >> > > (AsyncJobMgr-Heartbeat-1:ctx-22922b92) (logid:88c162be) End
> >> cleanup
> >> >> >> > expired
> >> >> >> > > async-jobs
> >> >> >> > > 2021-05-12 22:18:59,978 DEBUG [c.c.a.m.AgentManagerImpl]
> >> >> >> > > (AgentManager-Handler-13:null) (logid:) SeqA 6-1063910:
> >> Processing
> >> >> Seq
> >> >> >> > > 6-1063910: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
> >> >> >> > >
> >> >> >> >
> >> >> >>
> >> >>
> >>
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
> >> >> >> > > \"connections\": []\n}","wait":0}}] }
> >> >> >> > > 2021-05-12 22:18:59,980 DEBUG [c.c.a.m.AgentManagerImpl]
> >> >> >> > > (AgentManager-Handler-13:null) (logid:) SeqA 6-1063910:
> Sending
> >> Seq
> >> >> >> > > 6-1063910: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1,
> >> >> Flags:
> >> >> >> > > 100010,
> >> >> >> > >
> >> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> >> >> >> }
> >> >> >> > > 2021-05-12 22:19:02,644 DEBUG [c.c.s.StatsCollector]
> >> >> >> > > (StatsCollector-2:ctx-1237412a) (logid:3c78cd8f) AutoScaling
> >> >> Monitor
> >> >> >> is
> >> >> >> > > running...
> >> >> >> > > 2021-05-12 22:19:02,934 DEBUG [c.c.s.StatsCollector]
> >> >> >> > > (StatsCollector-4:ctx-6c1c6167) (logid:585ce1c2)
> >> >> HostStatsCollector is
> >> >> >> > > running...
> >> >> >> > > 2021-05-12 22:19:05,352 DEBUG [c.c.s.StatsCollector]
> >> >> >> > > (StatsCollector-3:ctx-58cb4a3f) (logid:13d34be0)
> >> StorageCollector
> >> >> is
> >> >> >> > > running...
> >> >> >> > >
> >> >> >> >
> >> >> >>
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >>
> >>
> >>
> >>
> >>
>
>
>
>
Re: Fail to add KVM Host
Posted by Nicolas Vazquez <Ni...@shapeblue.com>.
Good thing, confirmed all working good now?
I have created a PR to include this as a note on the documentation: https://github.com/apache/cloudstack-documentation/pull/214. Should be included on the next releases documentation.
Regards,
Nicolas Vazquez
________________________________
From: Alessandro Caviglione <c....@gmail.com>
Sent: Thursday, May 13, 2021 4:36 PM
To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
Subject: Re: Fail to add KVM Host
OMG guys... I think you've to specify this requirement in documentation!!
This row point me in the right direction:
failing to get physical interface from bridge cloudbr1, did not find an
eth*, bond*, team*, vlan*, em*, p*p*, ens*, eno*, enp*, or enx* in
/sys/devices/virtual/net/cloudbr1/brif
So the script looks for a bond that starts with: eth*, bond*, team*,
vlan*, em*, p*p*, ens*, eno*, enp*, or enx*
I've created a bond named cbdr_bond1 so it was not reachable!!
Thank you for your help and your suggestions!
On Thu, May 13, 2021 at 7:12 PM Alessandro Caviglione <
c.alessandro@gmail.com> wrote:
> Hi Nicolas,
> same link for the new logs in folder "LOG2":
> https://box.beitsa.ch/index.php/s/ZmGMWbQgmMkbzoK
>
> Strange thing is that I see in agent.log these lines:
>
> 2021-05-13 18:53:08,587 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-5:null) (logid:4b3902b9) matchPifFileInDirectory:
> file name 'cdbr_bond1'
> 2021-05-13 18:53:08,587 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-5:null) (logid:4b3902b9) failing to get physical
> interface from bridge cloudbr1, did not find an eth*, bond*, team*, vlan*,
> em*, p*p*, ens*, eno*, enp*, or enx* in
> /sys/devices/virtual/net/cloudbr1/brif
>
> But as you can see in interfaces.txt , cloudbr1/brif exists as the
> cloudbr0/brif
>
> On Thu, May 13, 2021 at 6:32 PM Nicolas Vazquez <
> Nicolas.Vazquez@shapeblue.com> wrote:
>
>> Still not clear on the issue yet. Could you please increase the log level
>> on /etc/cloudstack/agent/log4j-cloud.xml from INFO to DEBUG and check for
>> any exception trace?
>>
>>
>> Regards,
>>
>> Nicolas Vazquez
>>
>> ________________________________
>> From: Alessandro Caviglione <c....@gmail.com>
>> Sent: Thursday, May 13, 2021 1:05 PM
>> To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
>> Subject: Re: Fail to add KVM Host
>>
>> Should I install older version of libvirt/qemu or there's another problem?
>>
>> Il gio 13 mag 2021, 16:05 Alessandro Caviglione <c....@gmail.com>
>> ha
>> scritto:
>>
>> > Hi,
>> > yes I saw... but this is a centos 7.9 installed from scratch.
>> > I've installed with: yum install qemu-kvm libvirt libvirt-python
>> > libguestfs-tools virt-install
>> >
>> > libvirt is 4.5.0
>> > qemu-kvm is 1.5.3
>> > cloudstack-agent is 4.13
>> >
>> > I've also tried with qemu-kvm-ev 2.12 but nothing changes...
>> >
>> >
>> > On Thu, May 13, 2021 at 3:48 PM Nicolas Vazquez <
>> > Nicolas.Vazquez@shapeblue.com> wrote:
>> >
>> >> Hi Alessandro,
>> >>
>> >> From the agent logs the issue comes from this: Connection with libvirtd
>> >> is broken: invalid connection pointer in virConnectGetVersion
>> >>
>> >> Which libvirt and qemu versions are you using?
>> >>
>> >>
>> >> Regards,
>> >>
>> >> Nicolas Vazquez
>> >>
>> >> ________________________________
>> >> From: Alessandro Caviglione <c....@gmail.com>
>> >> Sent: Thursday, May 13, 2021 10:12 AM
>> >> To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
>> >> Subject: Re: Fail to add KVM Host
>> >>
>> >> Hi Andrija,
>> >> any idea?
>> >>
>> >> On Thu, May 13, 2021 at 11:34 AM Alessandro Caviglione <
>> >> c.alessandro@gmail.com> wrote:
>> >>
>> >> > Thank you Andrija!!
>> >> > https://box.beitsa.ch/index.php/s/ZmGMWbQgmMkbzoK
>> >> >
>> >> >
>> >> > On Thu, May 13, 2021 at 10:04 AM Andrija Panic <
>> andrija.panic@gmail.com
>> >> >
>> >> > wrote:
>> >> >
>> >> >> Can you share (on b pastebin. org or somewhere) the content of the
>> >> >> agent.log and setup. log from the KVM host please?
>> >> >>
>> >> >>
>> >> >>
>> >> >> Best,
>> >> >>
>> >> >> On Wed, 12 May 2021, 22:45 Alessandro Caviglione, <
>> >> c.alessandro@gmail.com
>> >> >> >
>> >> >> wrote:
>> >> >>
>> >> >> > Just a quick news... Now I see the host in hosts list, but it's in
>> >> alert
>> >> >> > state.
>> >> >> > In log I see:
>> >> >> > 2021-05-12 22:40:54,061 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
>> >> >> > (AgentConnectTaskPool-417:ctx-36dd291a) (logid:7320b6af) Notifying
>> >> other
>> >> >> > nodes of to disconnect
>> >> >> > 2021-05-12 22:40:54,062 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> >> > (AgentConnectTaskPool-417:ctx-36dd291a) (logid:7320b6af) Failed to
>> >> >> handle
>> >> >> > host connection:
>> >> >> > com.cloud.exception.ConnectionException: Incorrect Network setup
>> on
>> >> >> agent,
>> >> >> > Reinitialize agent after network names are setup, details : Can
>> not
>> >> find
>> >> >> > network: cloudbr1
>> >> >> >
>> >> >> > but I have cloudbr0 and cloudbr1 configured on host...
>> >> >> > What can I do?
>> >> >> >
>> >> >> >
>> >> >> > On Wed, May 12, 2021 at 10:34 PM Alessandro Caviglione <
>> >> >> > c.alessandro@gmail.com> wrote:
>> >> >> >
>> >> >> > > Hi guys,
>> >> >> > > a lot of troubles these days...
>> >> >> > > We've ACS 4.13 installed from source with one zone with a
>> vcenter
>> >> >> server
>> >> >> > > and advanced networking.
>> >> >> > > Now we need to add another zone to manage some KVM hosts but
>> with
>> >> >> basic
>> >> >> > > networking, we installed libvirt on a Centos7, configured
>> cloudbr0
>> >> for
>> >> >> > > management and cloudbr1 for gest traffic, installed
>> >> cloudstack-agent,
>> >> >> > made
>> >> >> > > all the configuration for tcp and tls, started the services and
>> all
>> >> >> it's
>> >> >> > ok.
>> >> >> > > So we created a new zone in ACS, with cluster and the above
>> host,
>> >> all
>> >> >> > > works fine but the host has not been added.
>> >> >> > > So I've tried again to add but after some seconds, the blinking
>> row
>> >> >> with
>> >> >> > > the new server disappear and in the log I can't see any error:
>> >> >> > >
>> >> >> > > 2021-05-12 22:18:35,457 DEBUG
>> [o.a.c.s.SecondaryStorageManagerImpl]
>> >> >> > > (secstorage-1:ctx-0b1a9c73) (logid:39eed862) Zone 2 is not
>> ready to
>> >> >> > launch
>> >> >> > > secondary storage VM yet
>> >> >> > > 2021-05-12 22:18:37,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >> > > (AsyncJobMgr-Heartbeat-1:ctx-7e5591e8) (logid:442791ae) Begin
>> >> cleanup
>> >> >> > > expired async-jobs
>> >> >> > > 2021-05-12 22:18:37,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >> > > (AsyncJobMgr-Heartbeat-1:ctx-7e5591e8) (logid:442791ae) End
>> cleanup
>> >> >> > expired
>> >> >> > > async-jobs
>> >> >> > > 2021-05-12 22:18:37,795 DEBUG [c.c.u.s.SSHCmdHelper]
>> >> >> > > (qtp504527234-17:ctx-dc82de11 ctx-2eb654f7) (logid:201abc34) SSH
>> >> >> command:
>> >> >> > > cloudstack-setup-agent -m 192.168.200.125 -z 2 -p 2 -c 2 -g
>> >> >> > > 92f6da7e-d7f0-3b03-93e8-c588dbf9182a -a -s --pubNic=cloudbr1
>> >> >> > > --prvNic=cloudbr0 --guestNic=cloudbr1 --hypervisor=kvm
>> >> >> > > SSH command output:Starting to configure your system:
>> >> >> > > Configure SElinux ... [OK]
>> >> >> > > Configure Network ... [OK]
>> >> >> > > Configure Libvirt ... [OK]
>> >> >> > > Configure Firewall ... [OK]
>> >> >> > > Configure Nfs ... [OK]
>> >> >> > > Configure cloudAgent ... [OK]
>> >> >> > > CloudStack Agent setup is done!
>> >> >> > >
>> >> >> > >
>> >> >> > > 2021-05-12 22:18:44,977 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> >> > > (AgentManager-Handler-7:null) (logid:) SeqA 6-1063907:
>> Processing
>> >> Seq
>> >> >> > > 6-1063907: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
>> >> >> > > \"connections\": []\n}","wait":0}}] }
>> >> >> > > 2021-05-12 22:18:44,979 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> >> > > (AgentManager-Handler-7:null) (logid:) SeqA 6-1063907: Sending
>> Seq
>> >> >> > > 6-1063907: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1,
>> >> Flags:
>> >> >> > > 100010,
>> >> >> > >
>> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> >> >> }
>> >> >> > > 2021-05-12 22:18:47,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >> > > (AsyncJobMgr-Heartbeat-1:ctx-5b168792) (logid:b9c80c90) Begin
>> >> cleanup
>> >> >> > > expired async-jobs
>> >> >> > > 2021-05-12 22:18:47,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >> > > (AsyncJobMgr-Heartbeat-1:ctx-5b168792) (logid:b9c80c90) End
>> cleanup
>> >> >> > expired
>> >> >> > > async-jobs
>> >> >> > > 2021-05-12 22:18:47,284 DEBUG
>> >> >> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> >> >> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 14
>> >> >> routers to
>> >> >> > > update status.
>> >> >> > > 2021-05-12 22:18:47,291 DEBUG
>> >> >> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> >> >> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 10
>> VPC
>> >> >> > networks
>> >> >> > > to update Redundant State.
>> >> >> > > 2021-05-12 22:18:47,296 DEBUG
>> >> >> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> >> >> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 0
>> >> >> networks to
>> >> >> > > update RvR status.
>> >> >> > > 2021-05-12 22:18:49,977 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> >> > > (AgentManager-Handler-2:null) (logid:) SeqA 6-1063908:
>> Processing
>> >> Seq
>> >> >> > > 6-1063908: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
>> >> >> > > \"connections\": []\n}","wait":0}}] }
>> >> >> > > 2021-05-12 22:18:49,979 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> >> > > (AgentManager-Handler-2:null) (logid:) SeqA 6-1063908: Sending
>> Seq
>> >> >> > > 6-1063908: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1,
>> >> Flags:
>> >> >> > > 100010,
>> >> >> > >
>> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> >> >> }
>> >> >> > > 2021-05-12 22:18:53,959 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> >> > > (AgentManager-Handler-5:null) (logid:) Ping from 4(s-4-VM)
>> >> >> > > 2021-05-12 22:18:53,997 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> >> > > (AgentManager-Handler-12:null) (logid:) Ping from 6(v-7-VM)
>> >> >> > > 2021-05-12 22:18:57,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >> > > (AsyncJobMgr-Heartbeat-1:ctx-22922b92) (logid:88c162be) Begin
>> >> cleanup
>> >> >> > > expired async-jobs
>> >> >> > > 2021-05-12 22:18:57,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >> > > (AsyncJobMgr-Heartbeat-1:ctx-22922b92) (logid:88c162be) End
>> cleanup
>> >> >> > expired
>> >> >> > > async-jobs
>> >> >> > > 2021-05-12 22:18:59,978 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> >> > > (AgentManager-Handler-13:null) (logid:) SeqA 6-1063910:
>> Processing
>> >> Seq
>> >> >> > > 6-1063910: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
>> >> >> > > \"connections\": []\n}","wait":0}}] }
>> >> >> > > 2021-05-12 22:18:59,980 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> >> > > (AgentManager-Handler-13:null) (logid:) SeqA 6-1063910: Sending
>> Seq
>> >> >> > > 6-1063910: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1,
>> >> Flags:
>> >> >> > > 100010,
>> >> >> > >
>> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> >> >> }
>> >> >> > > 2021-05-12 22:19:02,644 DEBUG [c.c.s.StatsCollector]
>> >> >> > > (StatsCollector-2:ctx-1237412a) (logid:3c78cd8f) AutoScaling
>> >> Monitor
>> >> >> is
>> >> >> > > running...
>> >> >> > > 2021-05-12 22:19:02,934 DEBUG [c.c.s.StatsCollector]
>> >> >> > > (StatsCollector-4:ctx-6c1c6167) (logid:585ce1c2)
>> >> HostStatsCollector is
>> >> >> > > running...
>> >> >> > > 2021-05-12 22:19:05,352 DEBUG [c.c.s.StatsCollector]
>> >> >> > > (StatsCollector-3:ctx-58cb4a3f) (logid:13d34be0)
>> StorageCollector
>> >> is
>> >> >> > > running...
>> >> >> > >
>> >> >> >
>> >> >>
>> >> >
>> >>
>> >>
>> >>
>> >>
>>
>>
>>
>>
Re: Fail to add KVM Host
Posted by Alessandro Caviglione <c....@gmail.com>.
OMG guys... I think you've to specify this requirement in documentation!!
This row point me in the right direction:
failing to get physical interface from bridge cloudbr1, did not find an
eth*, bond*, team*, vlan*, em*, p*p*, ens*, eno*, enp*, or enx* in
/sys/devices/virtual/net/cloudbr1/brif
So the script looks for a bond that starts with: eth*, bond*, team*,
vlan*, em*, p*p*, ens*, eno*, enp*, or enx*
I've created a bond named cbdr_bond1 so it was not reachable!!
Thank you for your help and your suggestions!
On Thu, May 13, 2021 at 7:12 PM Alessandro Caviglione <
c.alessandro@gmail.com> wrote:
> Hi Nicolas,
> same link for the new logs in folder "LOG2":
> https://box.beitsa.ch/index.php/s/ZmGMWbQgmMkbzoK
>
> Strange thing is that I see in agent.log these lines:
>
> 2021-05-13 18:53:08,587 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-5:null) (logid:4b3902b9) matchPifFileInDirectory:
> file name 'cdbr_bond1'
> 2021-05-13 18:53:08,587 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-5:null) (logid:4b3902b9) failing to get physical
> interface from bridge cloudbr1, did not find an eth*, bond*, team*, vlan*,
> em*, p*p*, ens*, eno*, enp*, or enx* in
> /sys/devices/virtual/net/cloudbr1/brif
>
> But as you can see in interfaces.txt , cloudbr1/brif exists as the
> cloudbr0/brif
>
> On Thu, May 13, 2021 at 6:32 PM Nicolas Vazquez <
> Nicolas.Vazquez@shapeblue.com> wrote:
>
>> Still not clear on the issue yet. Could you please increase the log level
>> on /etc/cloudstack/agent/log4j-cloud.xml from INFO to DEBUG and check for
>> any exception trace?
>>
>>
>> Regards,
>>
>> Nicolas Vazquez
>>
>> ________________________________
>> From: Alessandro Caviglione <c....@gmail.com>
>> Sent: Thursday, May 13, 2021 1:05 PM
>> To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
>> Subject: Re: Fail to add KVM Host
>>
>> Should I install older version of libvirt/qemu or there's another problem?
>>
>> Il gio 13 mag 2021, 16:05 Alessandro Caviglione <c....@gmail.com>
>> ha
>> scritto:
>>
>> > Hi,
>> > yes I saw... but this is a centos 7.9 installed from scratch.
>> > I've installed with: yum install qemu-kvm libvirt libvirt-python
>> > libguestfs-tools virt-install
>> >
>> > libvirt is 4.5.0
>> > qemu-kvm is 1.5.3
>> > cloudstack-agent is 4.13
>> >
>> > I've also tried with qemu-kvm-ev 2.12 but nothing changes...
>> >
>> >
>> > On Thu, May 13, 2021 at 3:48 PM Nicolas Vazquez <
>> > Nicolas.Vazquez@shapeblue.com> wrote:
>> >
>> >> Hi Alessandro,
>> >>
>> >> From the agent logs the issue comes from this: Connection with libvirtd
>> >> is broken: invalid connection pointer in virConnectGetVersion
>> >>
>> >> Which libvirt and qemu versions are you using?
>> >>
>> >>
>> >> Regards,
>> >>
>> >> Nicolas Vazquez
>> >>
>> >> ________________________________
>> >> From: Alessandro Caviglione <c....@gmail.com>
>> >> Sent: Thursday, May 13, 2021 10:12 AM
>> >> To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
>> >> Subject: Re: Fail to add KVM Host
>> >>
>> >> Hi Andrija,
>> >> any idea?
>> >>
>> >> On Thu, May 13, 2021 at 11:34 AM Alessandro Caviglione <
>> >> c.alessandro@gmail.com> wrote:
>> >>
>> >> > Thank you Andrija!!
>> >> > https://box.beitsa.ch/index.php/s/ZmGMWbQgmMkbzoK
>> >> >
>> >> >
>> >> > On Thu, May 13, 2021 at 10:04 AM Andrija Panic <
>> andrija.panic@gmail.com
>> >> >
>> >> > wrote:
>> >> >
>> >> >> Can you share (on b pastebin. org or somewhere) the content of the
>> >> >> agent.log and setup. log from the KVM host please?
>> >> >>
>> >> >>
>> >> >>
>> >> >> Best,
>> >> >>
>> >> >> On Wed, 12 May 2021, 22:45 Alessandro Caviglione, <
>> >> c.alessandro@gmail.com
>> >> >> >
>> >> >> wrote:
>> >> >>
>> >> >> > Just a quick news... Now I see the host in hosts list, but it's in
>> >> alert
>> >> >> > state.
>> >> >> > In log I see:
>> >> >> > 2021-05-12 22:40:54,061 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
>> >> >> > (AgentConnectTaskPool-417:ctx-36dd291a) (logid:7320b6af) Notifying
>> >> other
>> >> >> > nodes of to disconnect
>> >> >> > 2021-05-12 22:40:54,062 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> >> > (AgentConnectTaskPool-417:ctx-36dd291a) (logid:7320b6af) Failed to
>> >> >> handle
>> >> >> > host connection:
>> >> >> > com.cloud.exception.ConnectionException: Incorrect Network setup
>> on
>> >> >> agent,
>> >> >> > Reinitialize agent after network names are setup, details : Can
>> not
>> >> find
>> >> >> > network: cloudbr1
>> >> >> >
>> >> >> > but I have cloudbr0 and cloudbr1 configured on host...
>> >> >> > What can I do?
>> >> >> >
>> >> >> >
>> >> >> > On Wed, May 12, 2021 at 10:34 PM Alessandro Caviglione <
>> >> >> > c.alessandro@gmail.com> wrote:
>> >> >> >
>> >> >> > > Hi guys,
>> >> >> > > a lot of troubles these days...
>> >> >> > > We've ACS 4.13 installed from source with one zone with a
>> vcenter
>> >> >> server
>> >> >> > > and advanced networking.
>> >> >> > > Now we need to add another zone to manage some KVM hosts but
>> with
>> >> >> basic
>> >> >> > > networking, we installed libvirt on a Centos7, configured
>> cloudbr0
>> >> for
>> >> >> > > management and cloudbr1 for gest traffic, installed
>> >> cloudstack-agent,
>> >> >> > made
>> >> >> > > all the configuration for tcp and tls, started the services and
>> all
>> >> >> it's
>> >> >> > ok.
>> >> >> > > So we created a new zone in ACS, with cluster and the above
>> host,
>> >> all
>> >> >> > > works fine but the host has not been added.
>> >> >> > > So I've tried again to add but after some seconds, the blinking
>> row
>> >> >> with
>> >> >> > > the new server disappear and in the log I can't see any error:
>> >> >> > >
>> >> >> > > 2021-05-12 22:18:35,457 DEBUG
>> [o.a.c.s.SecondaryStorageManagerImpl]
>> >> >> > > (secstorage-1:ctx-0b1a9c73) (logid:39eed862) Zone 2 is not
>> ready to
>> >> >> > launch
>> >> >> > > secondary storage VM yet
>> >> >> > > 2021-05-12 22:18:37,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >> > > (AsyncJobMgr-Heartbeat-1:ctx-7e5591e8) (logid:442791ae) Begin
>> >> cleanup
>> >> >> > > expired async-jobs
>> >> >> > > 2021-05-12 22:18:37,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >> > > (AsyncJobMgr-Heartbeat-1:ctx-7e5591e8) (logid:442791ae) End
>> cleanup
>> >> >> > expired
>> >> >> > > async-jobs
>> >> >> > > 2021-05-12 22:18:37,795 DEBUG [c.c.u.s.SSHCmdHelper]
>> >> >> > > (qtp504527234-17:ctx-dc82de11 ctx-2eb654f7) (logid:201abc34) SSH
>> >> >> command:
>> >> >> > > cloudstack-setup-agent -m 192.168.200.125 -z 2 -p 2 -c 2 -g
>> >> >> > > 92f6da7e-d7f0-3b03-93e8-c588dbf9182a -a -s --pubNic=cloudbr1
>> >> >> > > --prvNic=cloudbr0 --guestNic=cloudbr1 --hypervisor=kvm
>> >> >> > > SSH command output:Starting to configure your system:
>> >> >> > > Configure SElinux ... [OK]
>> >> >> > > Configure Network ... [OK]
>> >> >> > > Configure Libvirt ... [OK]
>> >> >> > > Configure Firewall ... [OK]
>> >> >> > > Configure Nfs ... [OK]
>> >> >> > > Configure cloudAgent ... [OK]
>> >> >> > > CloudStack Agent setup is done!
>> >> >> > >
>> >> >> > >
>> >> >> > > 2021-05-12 22:18:44,977 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> >> > > (AgentManager-Handler-7:null) (logid:) SeqA 6-1063907:
>> Processing
>> >> Seq
>> >> >> > > 6-1063907: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
>> >> >> > > \"connections\": []\n}","wait":0}}] }
>> >> >> > > 2021-05-12 22:18:44,979 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> >> > > (AgentManager-Handler-7:null) (logid:) SeqA 6-1063907: Sending
>> Seq
>> >> >> > > 6-1063907: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1,
>> >> Flags:
>> >> >> > > 100010,
>> >> >> > >
>> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> >> >> }
>> >> >> > > 2021-05-12 22:18:47,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >> > > (AsyncJobMgr-Heartbeat-1:ctx-5b168792) (logid:b9c80c90) Begin
>> >> cleanup
>> >> >> > > expired async-jobs
>> >> >> > > 2021-05-12 22:18:47,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >> > > (AsyncJobMgr-Heartbeat-1:ctx-5b168792) (logid:b9c80c90) End
>> cleanup
>> >> >> > expired
>> >> >> > > async-jobs
>> >> >> > > 2021-05-12 22:18:47,284 DEBUG
>> >> >> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> >> >> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 14
>> >> >> routers to
>> >> >> > > update status.
>> >> >> > > 2021-05-12 22:18:47,291 DEBUG
>> >> >> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> >> >> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 10
>> VPC
>> >> >> > networks
>> >> >> > > to update Redundant State.
>> >> >> > > 2021-05-12 22:18:47,296 DEBUG
>> >> >> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> >> >> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 0
>> >> >> networks to
>> >> >> > > update RvR status.
>> >> >> > > 2021-05-12 22:18:49,977 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> >> > > (AgentManager-Handler-2:null) (logid:) SeqA 6-1063908:
>> Processing
>> >> Seq
>> >> >> > > 6-1063908: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
>> >> >> > > \"connections\": []\n}","wait":0}}] }
>> >> >> > > 2021-05-12 22:18:49,979 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> >> > > (AgentManager-Handler-2:null) (logid:) SeqA 6-1063908: Sending
>> Seq
>> >> >> > > 6-1063908: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1,
>> >> Flags:
>> >> >> > > 100010,
>> >> >> > >
>> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> >> >> }
>> >> >> > > 2021-05-12 22:18:53,959 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> >> > > (AgentManager-Handler-5:null) (logid:) Ping from 4(s-4-VM)
>> >> >> > > 2021-05-12 22:18:53,997 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> >> > > (AgentManager-Handler-12:null) (logid:) Ping from 6(v-7-VM)
>> >> >> > > 2021-05-12 22:18:57,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >> > > (AsyncJobMgr-Heartbeat-1:ctx-22922b92) (logid:88c162be) Begin
>> >> cleanup
>> >> >> > > expired async-jobs
>> >> >> > > 2021-05-12 22:18:57,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> >> > > (AsyncJobMgr-Heartbeat-1:ctx-22922b92) (logid:88c162be) End
>> cleanup
>> >> >> > expired
>> >> >> > > async-jobs
>> >> >> > > 2021-05-12 22:18:59,978 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> >> > > (AgentManager-Handler-13:null) (logid:) SeqA 6-1063910:
>> Processing
>> >> Seq
>> >> >> > > 6-1063910: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
>> >> >> > > \"connections\": []\n}","wait":0}}] }
>> >> >> > > 2021-05-12 22:18:59,980 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> >> > > (AgentManager-Handler-13:null) (logid:) SeqA 6-1063910: Sending
>> Seq
>> >> >> > > 6-1063910: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1,
>> >> Flags:
>> >> >> > > 100010,
>> >> >> > >
>> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> >> >> }
>> >> >> > > 2021-05-12 22:19:02,644 DEBUG [c.c.s.StatsCollector]
>> >> >> > > (StatsCollector-2:ctx-1237412a) (logid:3c78cd8f) AutoScaling
>> >> Monitor
>> >> >> is
>> >> >> > > running...
>> >> >> > > 2021-05-12 22:19:02,934 DEBUG [c.c.s.StatsCollector]
>> >> >> > > (StatsCollector-4:ctx-6c1c6167) (logid:585ce1c2)
>> >> HostStatsCollector is
>> >> >> > > running...
>> >> >> > > 2021-05-12 22:19:05,352 DEBUG [c.c.s.StatsCollector]
>> >> >> > > (StatsCollector-3:ctx-58cb4a3f) (logid:13d34be0)
>> StorageCollector
>> >> is
>> >> >> > > running...
>> >> >> > >
>> >> >> >
>> >> >>
>> >> >
>> >>
>> >>
>> >>
>> >>
>>
>>
>>
>>
Re: Fail to add KVM Host
Posted by Alessandro Caviglione <c....@gmail.com>.
Hi Nicolas,
same link for the new logs in folder "LOG2":
https://box.beitsa.ch/index.php/s/ZmGMWbQgmMkbzoK
Strange thing is that I see in agent.log these lines:
2021-05-13 18:53:08,587 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-5:null) (logid:4b3902b9) matchPifFileInDirectory:
file name 'cdbr_bond1'
2021-05-13 18:53:08,587 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-5:null) (logid:4b3902b9) failing to get physical
interface from bridge cloudbr1, did not find an eth*, bond*, team*, vlan*,
em*, p*p*, ens*, eno*, enp*, or enx* in
/sys/devices/virtual/net/cloudbr1/brif
But as you can see in interfaces.txt , cloudbr1/brif exists as the
cloudbr0/brif
On Thu, May 13, 2021 at 6:32 PM Nicolas Vazquez <
Nicolas.Vazquez@shapeblue.com> wrote:
> Still not clear on the issue yet. Could you please increase the log level
> on /etc/cloudstack/agent/log4j-cloud.xml from INFO to DEBUG and check for
> any exception trace?
>
>
> Regards,
>
> Nicolas Vazquez
>
> ________________________________
> From: Alessandro Caviglione <c....@gmail.com>
> Sent: Thursday, May 13, 2021 1:05 PM
> To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
> Subject: Re: Fail to add KVM Host
>
> Should I install older version of libvirt/qemu or there's another problem?
>
> Il gio 13 mag 2021, 16:05 Alessandro Caviglione <c....@gmail.com>
> ha
> scritto:
>
> > Hi,
> > yes I saw... but this is a centos 7.9 installed from scratch.
> > I've installed with: yum install qemu-kvm libvirt libvirt-python
> > libguestfs-tools virt-install
> >
> > libvirt is 4.5.0
> > qemu-kvm is 1.5.3
> > cloudstack-agent is 4.13
> >
> > I've also tried with qemu-kvm-ev 2.12 but nothing changes...
> >
> >
> > On Thu, May 13, 2021 at 3:48 PM Nicolas Vazquez <
> > Nicolas.Vazquez@shapeblue.com> wrote:
> >
> >> Hi Alessandro,
> >>
> >> From the agent logs the issue comes from this: Connection with libvirtd
> >> is broken: invalid connection pointer in virConnectGetVersion
> >>
> >> Which libvirt and qemu versions are you using?
> >>
> >>
> >> Regards,
> >>
> >> Nicolas Vazquez
> >>
> >> ________________________________
> >> From: Alessandro Caviglione <c....@gmail.com>
> >> Sent: Thursday, May 13, 2021 10:12 AM
> >> To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
> >> Subject: Re: Fail to add KVM Host
> >>
> >> Hi Andrija,
> >> any idea?
> >>
> >> On Thu, May 13, 2021 at 11:34 AM Alessandro Caviglione <
> >> c.alessandro@gmail.com> wrote:
> >>
> >> > Thank you Andrija!!
> >> > https://box.beitsa.ch/index.php/s/ZmGMWbQgmMkbzoK
> >> >
> >> >
> >> > On Thu, May 13, 2021 at 10:04 AM Andrija Panic <
> andrija.panic@gmail.com
> >> >
> >> > wrote:
> >> >
> >> >> Can you share (on b pastebin. org or somewhere) the content of the
> >> >> agent.log and setup. log from the KVM host please?
> >> >>
> >> >>
> >> >>
> >> >> Best,
> >> >>
> >> >> On Wed, 12 May 2021, 22:45 Alessandro Caviglione, <
> >> c.alessandro@gmail.com
> >> >> >
> >> >> wrote:
> >> >>
> >> >> > Just a quick news... Now I see the host in hosts list, but it's in
> >> alert
> >> >> > state.
> >> >> > In log I see:
> >> >> > 2021-05-12 22:40:54,061 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
> >> >> > (AgentConnectTaskPool-417:ctx-36dd291a) (logid:7320b6af) Notifying
> >> other
> >> >> > nodes of to disconnect
> >> >> > 2021-05-12 22:40:54,062 DEBUG [c.c.a.m.AgentManagerImpl]
> >> >> > (AgentConnectTaskPool-417:ctx-36dd291a) (logid:7320b6af) Failed to
> >> >> handle
> >> >> > host connection:
> >> >> > com.cloud.exception.ConnectionException: Incorrect Network setup on
> >> >> agent,
> >> >> > Reinitialize agent after network names are setup, details : Can not
> >> find
> >> >> > network: cloudbr1
> >> >> >
> >> >> > but I have cloudbr0 and cloudbr1 configured on host...
> >> >> > What can I do?
> >> >> >
> >> >> >
> >> >> > On Wed, May 12, 2021 at 10:34 PM Alessandro Caviglione <
> >> >> > c.alessandro@gmail.com> wrote:
> >> >> >
> >> >> > > Hi guys,
> >> >> > > a lot of troubles these days...
> >> >> > > We've ACS 4.13 installed from source with one zone with a vcenter
> >> >> server
> >> >> > > and advanced networking.
> >> >> > > Now we need to add another zone to manage some KVM hosts but with
> >> >> basic
> >> >> > > networking, we installed libvirt on a Centos7, configured
> cloudbr0
> >> for
> >> >> > > management and cloudbr1 for gest traffic, installed
> >> cloudstack-agent,
> >> >> > made
> >> >> > > all the configuration for tcp and tls, started the services and
> all
> >> >> it's
> >> >> > ok.
> >> >> > > So we created a new zone in ACS, with cluster and the above host,
> >> all
> >> >> > > works fine but the host has not been added.
> >> >> > > So I've tried again to add but after some seconds, the blinking
> row
> >> >> with
> >> >> > > the new server disappear and in the log I can't see any error:
> >> >> > >
> >> >> > > 2021-05-12 22:18:35,457 DEBUG
> [o.a.c.s.SecondaryStorageManagerImpl]
> >> >> > > (secstorage-1:ctx-0b1a9c73) (logid:39eed862) Zone 2 is not ready
> to
> >> >> > launch
> >> >> > > secondary storage VM yet
> >> >> > > 2021-05-12 22:18:37,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> >> > > (AsyncJobMgr-Heartbeat-1:ctx-7e5591e8) (logid:442791ae) Begin
> >> cleanup
> >> >> > > expired async-jobs
> >> >> > > 2021-05-12 22:18:37,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> >> > > (AsyncJobMgr-Heartbeat-1:ctx-7e5591e8) (logid:442791ae) End
> cleanup
> >> >> > expired
> >> >> > > async-jobs
> >> >> > > 2021-05-12 22:18:37,795 DEBUG [c.c.u.s.SSHCmdHelper]
> >> >> > > (qtp504527234-17:ctx-dc82de11 ctx-2eb654f7) (logid:201abc34) SSH
> >> >> command:
> >> >> > > cloudstack-setup-agent -m 192.168.200.125 -z 2 -p 2 -c 2 -g
> >> >> > > 92f6da7e-d7f0-3b03-93e8-c588dbf9182a -a -s --pubNic=cloudbr1
> >> >> > > --prvNic=cloudbr0 --guestNic=cloudbr1 --hypervisor=kvm
> >> >> > > SSH command output:Starting to configure your system:
> >> >> > > Configure SElinux ... [OK]
> >> >> > > Configure Network ... [OK]
> >> >> > > Configure Libvirt ... [OK]
> >> >> > > Configure Firewall ... [OK]
> >> >> > > Configure Nfs ... [OK]
> >> >> > > Configure cloudAgent ... [OK]
> >> >> > > CloudStack Agent setup is done!
> >> >> > >
> >> >> > >
> >> >> > > 2021-05-12 22:18:44,977 DEBUG [c.c.a.m.AgentManagerImpl]
> >> >> > > (AgentManager-Handler-7:null) (logid:) SeqA 6-1063907: Processing
> >> Seq
> >> >> > > 6-1063907: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
> >> >> > >
> >> >> >
> >> >>
> >>
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
> >> >> > > \"connections\": []\n}","wait":0}}] }
> >> >> > > 2021-05-12 22:18:44,979 DEBUG [c.c.a.m.AgentManagerImpl]
> >> >> > > (AgentManager-Handler-7:null) (logid:) SeqA 6-1063907: Sending
> Seq
> >> >> > > 6-1063907: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1,
> >> Flags:
> >> >> > > 100010,
> >> >> > >
> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> >> >> }
> >> >> > > 2021-05-12 22:18:47,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> >> > > (AsyncJobMgr-Heartbeat-1:ctx-5b168792) (logid:b9c80c90) Begin
> >> cleanup
> >> >> > > expired async-jobs
> >> >> > > 2021-05-12 22:18:47,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> >> > > (AsyncJobMgr-Heartbeat-1:ctx-5b168792) (logid:b9c80c90) End
> cleanup
> >> >> > expired
> >> >> > > async-jobs
> >> >> > > 2021-05-12 22:18:47,284 DEBUG
> >> >> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> >> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 14
> >> >> routers to
> >> >> > > update status.
> >> >> > > 2021-05-12 22:18:47,291 DEBUG
> >> >> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> >> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 10
> VPC
> >> >> > networks
> >> >> > > to update Redundant State.
> >> >> > > 2021-05-12 22:18:47,296 DEBUG
> >> >> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> >> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 0
> >> >> networks to
> >> >> > > update RvR status.
> >> >> > > 2021-05-12 22:18:49,977 DEBUG [c.c.a.m.AgentManagerImpl]
> >> >> > > (AgentManager-Handler-2:null) (logid:) SeqA 6-1063908: Processing
> >> Seq
> >> >> > > 6-1063908: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
> >> >> > >
> >> >> >
> >> >>
> >>
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
> >> >> > > \"connections\": []\n}","wait":0}}] }
> >> >> > > 2021-05-12 22:18:49,979 DEBUG [c.c.a.m.AgentManagerImpl]
> >> >> > > (AgentManager-Handler-2:null) (logid:) SeqA 6-1063908: Sending
> Seq
> >> >> > > 6-1063908: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1,
> >> Flags:
> >> >> > > 100010,
> >> >> > >
> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> >> >> }
> >> >> > > 2021-05-12 22:18:53,959 DEBUG [c.c.a.m.AgentManagerImpl]
> >> >> > > (AgentManager-Handler-5:null) (logid:) Ping from 4(s-4-VM)
> >> >> > > 2021-05-12 22:18:53,997 DEBUG [c.c.a.m.AgentManagerImpl]
> >> >> > > (AgentManager-Handler-12:null) (logid:) Ping from 6(v-7-VM)
> >> >> > > 2021-05-12 22:18:57,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> >> > > (AsyncJobMgr-Heartbeat-1:ctx-22922b92) (logid:88c162be) Begin
> >> cleanup
> >> >> > > expired async-jobs
> >> >> > > 2021-05-12 22:18:57,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> >> > > (AsyncJobMgr-Heartbeat-1:ctx-22922b92) (logid:88c162be) End
> cleanup
> >> >> > expired
> >> >> > > async-jobs
> >> >> > > 2021-05-12 22:18:59,978 DEBUG [c.c.a.m.AgentManagerImpl]
> >> >> > > (AgentManager-Handler-13:null) (logid:) SeqA 6-1063910:
> Processing
> >> Seq
> >> >> > > 6-1063910: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
> >> >> > >
> >> >> >
> >> >>
> >>
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
> >> >> > > \"connections\": []\n}","wait":0}}] }
> >> >> > > 2021-05-12 22:18:59,980 DEBUG [c.c.a.m.AgentManagerImpl]
> >> >> > > (AgentManager-Handler-13:null) (logid:) SeqA 6-1063910: Sending
> Seq
> >> >> > > 6-1063910: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1,
> >> Flags:
> >> >> > > 100010,
> >> >> > >
> >> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> >> >> }
> >> >> > > 2021-05-12 22:19:02,644 DEBUG [c.c.s.StatsCollector]
> >> >> > > (StatsCollector-2:ctx-1237412a) (logid:3c78cd8f) AutoScaling
> >> Monitor
> >> >> is
> >> >> > > running...
> >> >> > > 2021-05-12 22:19:02,934 DEBUG [c.c.s.StatsCollector]
> >> >> > > (StatsCollector-4:ctx-6c1c6167) (logid:585ce1c2)
> >> HostStatsCollector is
> >> >> > > running...
> >> >> > > 2021-05-12 22:19:05,352 DEBUG [c.c.s.StatsCollector]
> >> >> > > (StatsCollector-3:ctx-58cb4a3f) (logid:13d34be0) StorageCollector
> >> is
> >> >> > > running...
> >> >> > >
> >> >> >
> >> >>
> >> >
> >>
> >>
> >>
> >>
>
>
>
>
Re: Fail to add KVM Host
Posted by Nicolas Vazquez <Ni...@shapeblue.com>.
Still not clear on the issue yet. Could you please increase the log level on /etc/cloudstack/agent/log4j-cloud.xml from INFO to DEBUG and check for any exception trace?
Regards,
Nicolas Vazquez
________________________________
From: Alessandro Caviglione <c....@gmail.com>
Sent: Thursday, May 13, 2021 1:05 PM
To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
Subject: Re: Fail to add KVM Host
Should I install older version of libvirt/qemu or there's another problem?
Il gio 13 mag 2021, 16:05 Alessandro Caviglione <c....@gmail.com> ha
scritto:
> Hi,
> yes I saw... but this is a centos 7.9 installed from scratch.
> I've installed with: yum install qemu-kvm libvirt libvirt-python
> libguestfs-tools virt-install
>
> libvirt is 4.5.0
> qemu-kvm is 1.5.3
> cloudstack-agent is 4.13
>
> I've also tried with qemu-kvm-ev 2.12 but nothing changes...
>
>
> On Thu, May 13, 2021 at 3:48 PM Nicolas Vazquez <
> Nicolas.Vazquez@shapeblue.com> wrote:
>
>> Hi Alessandro,
>>
>> From the agent logs the issue comes from this: Connection with libvirtd
>> is broken: invalid connection pointer in virConnectGetVersion
>>
>> Which libvirt and qemu versions are you using?
>>
>>
>> Regards,
>>
>> Nicolas Vazquez
>>
>> ________________________________
>> From: Alessandro Caviglione <c....@gmail.com>
>> Sent: Thursday, May 13, 2021 10:12 AM
>> To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
>> Subject: Re: Fail to add KVM Host
>>
>> Hi Andrija,
>> any idea?
>>
>> On Thu, May 13, 2021 at 11:34 AM Alessandro Caviglione <
>> c.alessandro@gmail.com> wrote:
>>
>> > Thank you Andrija!!
>> > https://box.beitsa.ch/index.php/s/ZmGMWbQgmMkbzoK
>> >
>> >
>> > On Thu, May 13, 2021 at 10:04 AM Andrija Panic <andrija.panic@gmail.com
>> >
>> > wrote:
>> >
>> >> Can you share (on b pastebin. org or somewhere) the content of the
>> >> agent.log and setup. log from the KVM host please?
>> >>
>> >>
>> >>
>> >> Best,
>> >>
>> >> On Wed, 12 May 2021, 22:45 Alessandro Caviglione, <
>> c.alessandro@gmail.com
>> >> >
>> >> wrote:
>> >>
>> >> > Just a quick news... Now I see the host in hosts list, but it's in
>> alert
>> >> > state.
>> >> > In log I see:
>> >> > 2021-05-12 22:40:54,061 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
>> >> > (AgentConnectTaskPool-417:ctx-36dd291a) (logid:7320b6af) Notifying
>> other
>> >> > nodes of to disconnect
>> >> > 2021-05-12 22:40:54,062 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> > (AgentConnectTaskPool-417:ctx-36dd291a) (logid:7320b6af) Failed to
>> >> handle
>> >> > host connection:
>> >> > com.cloud.exception.ConnectionException: Incorrect Network setup on
>> >> agent,
>> >> > Reinitialize agent after network names are setup, details : Can not
>> find
>> >> > network: cloudbr1
>> >> >
>> >> > but I have cloudbr0 and cloudbr1 configured on host...
>> >> > What can I do?
>> >> >
>> >> >
>> >> > On Wed, May 12, 2021 at 10:34 PM Alessandro Caviglione <
>> >> > c.alessandro@gmail.com> wrote:
>> >> >
>> >> > > Hi guys,
>> >> > > a lot of troubles these days...
>> >> > > We've ACS 4.13 installed from source with one zone with a vcenter
>> >> server
>> >> > > and advanced networking.
>> >> > > Now we need to add another zone to manage some KVM hosts but with
>> >> basic
>> >> > > networking, we installed libvirt on a Centos7, configured cloudbr0
>> for
>> >> > > management and cloudbr1 for gest traffic, installed
>> cloudstack-agent,
>> >> > made
>> >> > > all the configuration for tcp and tls, started the services and all
>> >> it's
>> >> > ok.
>> >> > > So we created a new zone in ACS, with cluster and the above host,
>> all
>> >> > > works fine but the host has not been added.
>> >> > > So I've tried again to add but after some seconds, the blinking row
>> >> with
>> >> > > the new server disappear and in the log I can't see any error:
>> >> > >
>> >> > > 2021-05-12 22:18:35,457 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>> >> > > (secstorage-1:ctx-0b1a9c73) (logid:39eed862) Zone 2 is not ready to
>> >> > launch
>> >> > > secondary storage VM yet
>> >> > > 2021-05-12 22:18:37,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> > > (AsyncJobMgr-Heartbeat-1:ctx-7e5591e8) (logid:442791ae) Begin
>> cleanup
>> >> > > expired async-jobs
>> >> > > 2021-05-12 22:18:37,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> > > (AsyncJobMgr-Heartbeat-1:ctx-7e5591e8) (logid:442791ae) End cleanup
>> >> > expired
>> >> > > async-jobs
>> >> > > 2021-05-12 22:18:37,795 DEBUG [c.c.u.s.SSHCmdHelper]
>> >> > > (qtp504527234-17:ctx-dc82de11 ctx-2eb654f7) (logid:201abc34) SSH
>> >> command:
>> >> > > cloudstack-setup-agent -m 192.168.200.125 -z 2 -p 2 -c 2 -g
>> >> > > 92f6da7e-d7f0-3b03-93e8-c588dbf9182a -a -s --pubNic=cloudbr1
>> >> > > --prvNic=cloudbr0 --guestNic=cloudbr1 --hypervisor=kvm
>> >> > > SSH command output:Starting to configure your system:
>> >> > > Configure SElinux ... [OK]
>> >> > > Configure Network ... [OK]
>> >> > > Configure Libvirt ... [OK]
>> >> > > Configure Firewall ... [OK]
>> >> > > Configure Nfs ... [OK]
>> >> > > Configure cloudAgent ... [OK]
>> >> > > CloudStack Agent setup is done!
>> >> > >
>> >> > >
>> >> > > 2021-05-12 22:18:44,977 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> > > (AgentManager-Handler-7:null) (logid:) SeqA 6-1063907: Processing
>> Seq
>> >> > > 6-1063907: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
>> >> > >
>> >> >
>> >>
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
>> >> > > \"connections\": []\n}","wait":0}}] }
>> >> > > 2021-05-12 22:18:44,979 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> > > (AgentManager-Handler-7:null) (logid:) SeqA 6-1063907: Sending Seq
>> >> > > 6-1063907: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1,
>> Flags:
>> >> > > 100010,
>> >> > >
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> >> }
>> >> > > 2021-05-12 22:18:47,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> > > (AsyncJobMgr-Heartbeat-1:ctx-5b168792) (logid:b9c80c90) Begin
>> cleanup
>> >> > > expired async-jobs
>> >> > > 2021-05-12 22:18:47,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> > > (AsyncJobMgr-Heartbeat-1:ctx-5b168792) (logid:b9c80c90) End cleanup
>> >> > expired
>> >> > > async-jobs
>> >> > > 2021-05-12 22:18:47,284 DEBUG
>> >> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> >> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 14
>> >> routers to
>> >> > > update status.
>> >> > > 2021-05-12 22:18:47,291 DEBUG
>> >> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> >> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 10 VPC
>> >> > networks
>> >> > > to update Redundant State.
>> >> > > 2021-05-12 22:18:47,296 DEBUG
>> >> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> >> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 0
>> >> networks to
>> >> > > update RvR status.
>> >> > > 2021-05-12 22:18:49,977 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> > > (AgentManager-Handler-2:null) (logid:) SeqA 6-1063908: Processing
>> Seq
>> >> > > 6-1063908: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
>> >> > >
>> >> >
>> >>
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
>> >> > > \"connections\": []\n}","wait":0}}] }
>> >> > > 2021-05-12 22:18:49,979 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> > > (AgentManager-Handler-2:null) (logid:) SeqA 6-1063908: Sending Seq
>> >> > > 6-1063908: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1,
>> Flags:
>> >> > > 100010,
>> >> > >
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> >> }
>> >> > > 2021-05-12 22:18:53,959 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> > > (AgentManager-Handler-5:null) (logid:) Ping from 4(s-4-VM)
>> >> > > 2021-05-12 22:18:53,997 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> > > (AgentManager-Handler-12:null) (logid:) Ping from 6(v-7-VM)
>> >> > > 2021-05-12 22:18:57,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> > > (AsyncJobMgr-Heartbeat-1:ctx-22922b92) (logid:88c162be) Begin
>> cleanup
>> >> > > expired async-jobs
>> >> > > 2021-05-12 22:18:57,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> > > (AsyncJobMgr-Heartbeat-1:ctx-22922b92) (logid:88c162be) End cleanup
>> >> > expired
>> >> > > async-jobs
>> >> > > 2021-05-12 22:18:59,978 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> > > (AgentManager-Handler-13:null) (logid:) SeqA 6-1063910: Processing
>> Seq
>> >> > > 6-1063910: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
>> >> > >
>> >> >
>> >>
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
>> >> > > \"connections\": []\n}","wait":0}}] }
>> >> > > 2021-05-12 22:18:59,980 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> > > (AgentManager-Handler-13:null) (logid:) SeqA 6-1063910: Sending Seq
>> >> > > 6-1063910: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1,
>> Flags:
>> >> > > 100010,
>> >> > >
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> >> }
>> >> > > 2021-05-12 22:19:02,644 DEBUG [c.c.s.StatsCollector]
>> >> > > (StatsCollector-2:ctx-1237412a) (logid:3c78cd8f) AutoScaling
>> Monitor
>> >> is
>> >> > > running...
>> >> > > 2021-05-12 22:19:02,934 DEBUG [c.c.s.StatsCollector]
>> >> > > (StatsCollector-4:ctx-6c1c6167) (logid:585ce1c2)
>> HostStatsCollector is
>> >> > > running...
>> >> > > 2021-05-12 22:19:05,352 DEBUG [c.c.s.StatsCollector]
>> >> > > (StatsCollector-3:ctx-58cb4a3f) (logid:13d34be0) StorageCollector
>> is
>> >> > > running...
>> >> > >
>> >> >
>> >>
>> >
>>
>>
>>
>>
Re: Fail to add KVM Host
Posted by Alessandro Caviglione <c....@gmail.com>.
Should I install older version of libvirt/qemu or there's another problem?
Il gio 13 mag 2021, 16:05 Alessandro Caviglione <c....@gmail.com> ha
scritto:
> Hi,
> yes I saw... but this is a centos 7.9 installed from scratch.
> I've installed with: yum install qemu-kvm libvirt libvirt-python
> libguestfs-tools virt-install
>
> libvirt is 4.5.0
> qemu-kvm is 1.5.3
> cloudstack-agent is 4.13
>
> I've also tried with qemu-kvm-ev 2.12 but nothing changes...
>
>
> On Thu, May 13, 2021 at 3:48 PM Nicolas Vazquez <
> Nicolas.Vazquez@shapeblue.com> wrote:
>
>> Hi Alessandro,
>>
>> From the agent logs the issue comes from this: Connection with libvirtd
>> is broken: invalid connection pointer in virConnectGetVersion
>>
>> Which libvirt and qemu versions are you using?
>>
>>
>> Regards,
>>
>> Nicolas Vazquez
>>
>> ________________________________
>> From: Alessandro Caviglione <c....@gmail.com>
>> Sent: Thursday, May 13, 2021 10:12 AM
>> To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
>> Subject: Re: Fail to add KVM Host
>>
>> Hi Andrija,
>> any idea?
>>
>> On Thu, May 13, 2021 at 11:34 AM Alessandro Caviglione <
>> c.alessandro@gmail.com> wrote:
>>
>> > Thank you Andrija!!
>> > https://box.beitsa.ch/index.php/s/ZmGMWbQgmMkbzoK
>> >
>> >
>> > On Thu, May 13, 2021 at 10:04 AM Andrija Panic <andrija.panic@gmail.com
>> >
>> > wrote:
>> >
>> >> Can you share (on b pastebin. org or somewhere) the content of the
>> >> agent.log and setup. log from the KVM host please?
>> >>
>> >>
>> >>
>> >> Best,
>> >>
>> >> On Wed, 12 May 2021, 22:45 Alessandro Caviglione, <
>> c.alessandro@gmail.com
>> >> >
>> >> wrote:
>> >>
>> >> > Just a quick news... Now I see the host in hosts list, but it's in
>> alert
>> >> > state.
>> >> > In log I see:
>> >> > 2021-05-12 22:40:54,061 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
>> >> > (AgentConnectTaskPool-417:ctx-36dd291a) (logid:7320b6af) Notifying
>> other
>> >> > nodes of to disconnect
>> >> > 2021-05-12 22:40:54,062 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> > (AgentConnectTaskPool-417:ctx-36dd291a) (logid:7320b6af) Failed to
>> >> handle
>> >> > host connection:
>> >> > com.cloud.exception.ConnectionException: Incorrect Network setup on
>> >> agent,
>> >> > Reinitialize agent after network names are setup, details : Can not
>> find
>> >> > network: cloudbr1
>> >> >
>> >> > but I have cloudbr0 and cloudbr1 configured on host...
>> >> > What can I do?
>> >> >
>> >> >
>> >> > On Wed, May 12, 2021 at 10:34 PM Alessandro Caviglione <
>> >> > c.alessandro@gmail.com> wrote:
>> >> >
>> >> > > Hi guys,
>> >> > > a lot of troubles these days...
>> >> > > We've ACS 4.13 installed from source with one zone with a vcenter
>> >> server
>> >> > > and advanced networking.
>> >> > > Now we need to add another zone to manage some KVM hosts but with
>> >> basic
>> >> > > networking, we installed libvirt on a Centos7, configured cloudbr0
>> for
>> >> > > management and cloudbr1 for gest traffic, installed
>> cloudstack-agent,
>> >> > made
>> >> > > all the configuration for tcp and tls, started the services and all
>> >> it's
>> >> > ok.
>> >> > > So we created a new zone in ACS, with cluster and the above host,
>> all
>> >> > > works fine but the host has not been added.
>> >> > > So I've tried again to add but after some seconds, the blinking row
>> >> with
>> >> > > the new server disappear and in the log I can't see any error:
>> >> > >
>> >> > > 2021-05-12 22:18:35,457 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>> >> > > (secstorage-1:ctx-0b1a9c73) (logid:39eed862) Zone 2 is not ready to
>> >> > launch
>> >> > > secondary storage VM yet
>> >> > > 2021-05-12 22:18:37,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> > > (AsyncJobMgr-Heartbeat-1:ctx-7e5591e8) (logid:442791ae) Begin
>> cleanup
>> >> > > expired async-jobs
>> >> > > 2021-05-12 22:18:37,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> > > (AsyncJobMgr-Heartbeat-1:ctx-7e5591e8) (logid:442791ae) End cleanup
>> >> > expired
>> >> > > async-jobs
>> >> > > 2021-05-12 22:18:37,795 DEBUG [c.c.u.s.SSHCmdHelper]
>> >> > > (qtp504527234-17:ctx-dc82de11 ctx-2eb654f7) (logid:201abc34) SSH
>> >> command:
>> >> > > cloudstack-setup-agent -m 192.168.200.125 -z 2 -p 2 -c 2 -g
>> >> > > 92f6da7e-d7f0-3b03-93e8-c588dbf9182a -a -s --pubNic=cloudbr1
>> >> > > --prvNic=cloudbr0 --guestNic=cloudbr1 --hypervisor=kvm
>> >> > > SSH command output:Starting to configure your system:
>> >> > > Configure SElinux ... [OK]
>> >> > > Configure Network ... [OK]
>> >> > > Configure Libvirt ... [OK]
>> >> > > Configure Firewall ... [OK]
>> >> > > Configure Nfs ... [OK]
>> >> > > Configure cloudAgent ... [OK]
>> >> > > CloudStack Agent setup is done!
>> >> > >
>> >> > >
>> >> > > 2021-05-12 22:18:44,977 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> > > (AgentManager-Handler-7:null) (logid:) SeqA 6-1063907: Processing
>> Seq
>> >> > > 6-1063907: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
>> >> > >
>> >> >
>> >>
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
>> >> > > \"connections\": []\n}","wait":0}}] }
>> >> > > 2021-05-12 22:18:44,979 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> > > (AgentManager-Handler-7:null) (logid:) SeqA 6-1063907: Sending Seq
>> >> > > 6-1063907: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1,
>> Flags:
>> >> > > 100010,
>> >> > >
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> >> }
>> >> > > 2021-05-12 22:18:47,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> > > (AsyncJobMgr-Heartbeat-1:ctx-5b168792) (logid:b9c80c90) Begin
>> cleanup
>> >> > > expired async-jobs
>> >> > > 2021-05-12 22:18:47,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> > > (AsyncJobMgr-Heartbeat-1:ctx-5b168792) (logid:b9c80c90) End cleanup
>> >> > expired
>> >> > > async-jobs
>> >> > > 2021-05-12 22:18:47,284 DEBUG
>> >> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> >> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 14
>> >> routers to
>> >> > > update status.
>> >> > > 2021-05-12 22:18:47,291 DEBUG
>> >> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> >> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 10 VPC
>> >> > networks
>> >> > > to update Redundant State.
>> >> > > 2021-05-12 22:18:47,296 DEBUG
>> >> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> >> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 0
>> >> networks to
>> >> > > update RvR status.
>> >> > > 2021-05-12 22:18:49,977 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> > > (AgentManager-Handler-2:null) (logid:) SeqA 6-1063908: Processing
>> Seq
>> >> > > 6-1063908: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
>> >> > >
>> >> >
>> >>
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
>> >> > > \"connections\": []\n}","wait":0}}] }
>> >> > > 2021-05-12 22:18:49,979 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> > > (AgentManager-Handler-2:null) (logid:) SeqA 6-1063908: Sending Seq
>> >> > > 6-1063908: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1,
>> Flags:
>> >> > > 100010,
>> >> > >
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> >> }
>> >> > > 2021-05-12 22:18:53,959 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> > > (AgentManager-Handler-5:null) (logid:) Ping from 4(s-4-VM)
>> >> > > 2021-05-12 22:18:53,997 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> > > (AgentManager-Handler-12:null) (logid:) Ping from 6(v-7-VM)
>> >> > > 2021-05-12 22:18:57,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> > > (AsyncJobMgr-Heartbeat-1:ctx-22922b92) (logid:88c162be) Begin
>> cleanup
>> >> > > expired async-jobs
>> >> > > 2021-05-12 22:18:57,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> > > (AsyncJobMgr-Heartbeat-1:ctx-22922b92) (logid:88c162be) End cleanup
>> >> > expired
>> >> > > async-jobs
>> >> > > 2021-05-12 22:18:59,978 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> > > (AgentManager-Handler-13:null) (logid:) SeqA 6-1063910: Processing
>> Seq
>> >> > > 6-1063910: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
>> >> > >
>> >> >
>> >>
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
>> >> > > \"connections\": []\n}","wait":0}}] }
>> >> > > 2021-05-12 22:18:59,980 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> > > (AgentManager-Handler-13:null) (logid:) SeqA 6-1063910: Sending Seq
>> >> > > 6-1063910: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1,
>> Flags:
>> >> > > 100010,
>> >> > >
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> >> }
>> >> > > 2021-05-12 22:19:02,644 DEBUG [c.c.s.StatsCollector]
>> >> > > (StatsCollector-2:ctx-1237412a) (logid:3c78cd8f) AutoScaling
>> Monitor
>> >> is
>> >> > > running...
>> >> > > 2021-05-12 22:19:02,934 DEBUG [c.c.s.StatsCollector]
>> >> > > (StatsCollector-4:ctx-6c1c6167) (logid:585ce1c2)
>> HostStatsCollector is
>> >> > > running...
>> >> > > 2021-05-12 22:19:05,352 DEBUG [c.c.s.StatsCollector]
>> >> > > (StatsCollector-3:ctx-58cb4a3f) (logid:13d34be0) StorageCollector
>> is
>> >> > > running...
>> >> > >
>> >> >
>> >>
>> >
>>
>>
>>
>>
Re: Fail to add KVM Host
Posted by Alessandro Caviglione <c....@gmail.com>.
Hi,
yes I saw... but this is a centos 7.9 installed from scratch.
I've installed with: yum install qemu-kvm libvirt libvirt-python
libguestfs-tools virt-install
libvirt is 4.5.0
qemu-kvm is 1.5.3
cloudstack-agent is 4.13
I've also tried with qemu-kvm-ev 2.12 but nothing changes...
On Thu, May 13, 2021 at 3:48 PM Nicolas Vazquez <
Nicolas.Vazquez@shapeblue.com> wrote:
> Hi Alessandro,
>
> From the agent logs the issue comes from this: Connection with libvirtd is
> broken: invalid connection pointer in virConnectGetVersion
>
> Which libvirt and qemu versions are you using?
>
>
> Regards,
>
> Nicolas Vazquez
>
> ________________________________
> From: Alessandro Caviglione <c....@gmail.com>
> Sent: Thursday, May 13, 2021 10:12 AM
> To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
> Subject: Re: Fail to add KVM Host
>
> Hi Andrija,
> any idea?
>
> On Thu, May 13, 2021 at 11:34 AM Alessandro Caviglione <
> c.alessandro@gmail.com> wrote:
>
> > Thank you Andrija!!
> > https://box.beitsa.ch/index.php/s/ZmGMWbQgmMkbzoK
> >
> >
> > On Thu, May 13, 2021 at 10:04 AM Andrija Panic <an...@gmail.com>
> > wrote:
> >
> >> Can you share (on b pastebin. org or somewhere) the content of the
> >> agent.log and setup. log from the KVM host please?
> >>
> >>
> >>
> >> Best,
> >>
> >> On Wed, 12 May 2021, 22:45 Alessandro Caviglione, <
> c.alessandro@gmail.com
> >> >
> >> wrote:
> >>
> >> > Just a quick news... Now I see the host in hosts list, but it's in
> alert
> >> > state.
> >> > In log I see:
> >> > 2021-05-12 22:40:54,061 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
> >> > (AgentConnectTaskPool-417:ctx-36dd291a) (logid:7320b6af) Notifying
> other
> >> > nodes of to disconnect
> >> > 2021-05-12 22:40:54,062 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > (AgentConnectTaskPool-417:ctx-36dd291a) (logid:7320b6af) Failed to
> >> handle
> >> > host connection:
> >> > com.cloud.exception.ConnectionException: Incorrect Network setup on
> >> agent,
> >> > Reinitialize agent after network names are setup, details : Can not
> find
> >> > network: cloudbr1
> >> >
> >> > but I have cloudbr0 and cloudbr1 configured on host...
> >> > What can I do?
> >> >
> >> >
> >> > On Wed, May 12, 2021 at 10:34 PM Alessandro Caviglione <
> >> > c.alessandro@gmail.com> wrote:
> >> >
> >> > > Hi guys,
> >> > > a lot of troubles these days...
> >> > > We've ACS 4.13 installed from source with one zone with a vcenter
> >> server
> >> > > and advanced networking.
> >> > > Now we need to add another zone to manage some KVM hosts but with
> >> basic
> >> > > networking, we installed libvirt on a Centos7, configured cloudbr0
> for
> >> > > management and cloudbr1 for gest traffic, installed
> cloudstack-agent,
> >> > made
> >> > > all the configuration for tcp and tls, started the services and all
> >> it's
> >> > ok.
> >> > > So we created a new zone in ACS, with cluster and the above host,
> all
> >> > > works fine but the host has not been added.
> >> > > So I've tried again to add but after some seconds, the blinking row
> >> with
> >> > > the new server disappear and in the log I can't see any error:
> >> > >
> >> > > 2021-05-12 22:18:35,457 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
> >> > > (secstorage-1:ctx-0b1a9c73) (logid:39eed862) Zone 2 is not ready to
> >> > launch
> >> > > secondary storage VM yet
> >> > > 2021-05-12 22:18:37,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > > (AsyncJobMgr-Heartbeat-1:ctx-7e5591e8) (logid:442791ae) Begin
> cleanup
> >> > > expired async-jobs
> >> > > 2021-05-12 22:18:37,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > > (AsyncJobMgr-Heartbeat-1:ctx-7e5591e8) (logid:442791ae) End cleanup
> >> > expired
> >> > > async-jobs
> >> > > 2021-05-12 22:18:37,795 DEBUG [c.c.u.s.SSHCmdHelper]
> >> > > (qtp504527234-17:ctx-dc82de11 ctx-2eb654f7) (logid:201abc34) SSH
> >> command:
> >> > > cloudstack-setup-agent -m 192.168.200.125 -z 2 -p 2 -c 2 -g
> >> > > 92f6da7e-d7f0-3b03-93e8-c588dbf9182a -a -s --pubNic=cloudbr1
> >> > > --prvNic=cloudbr0 --guestNic=cloudbr1 --hypervisor=kvm
> >> > > SSH command output:Starting to configure your system:
> >> > > Configure SElinux ... [OK]
> >> > > Configure Network ... [OK]
> >> > > Configure Libvirt ... [OK]
> >> > > Configure Firewall ... [OK]
> >> > > Configure Nfs ... [OK]
> >> > > Configure cloudAgent ... [OK]
> >> > > CloudStack Agent setup is done!
> >> > >
> >> > >
> >> > > 2021-05-12 22:18:44,977 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > > (AgentManager-Handler-7:null) (logid:) SeqA 6-1063907: Processing
> Seq
> >> > > 6-1063907: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
> >> > >
> >> >
> >>
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
> >> > > \"connections\": []\n}","wait":0}}] }
> >> > > 2021-05-12 22:18:44,979 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > > (AgentManager-Handler-7:null) (logid:) SeqA 6-1063907: Sending Seq
> >> > > 6-1063907: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1,
> Flags:
> >> > > 100010,
> >> > >
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> >> }
> >> > > 2021-05-12 22:18:47,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > > (AsyncJobMgr-Heartbeat-1:ctx-5b168792) (logid:b9c80c90) Begin
> cleanup
> >> > > expired async-jobs
> >> > > 2021-05-12 22:18:47,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > > (AsyncJobMgr-Heartbeat-1:ctx-5b168792) (logid:b9c80c90) End cleanup
> >> > expired
> >> > > async-jobs
> >> > > 2021-05-12 22:18:47,284 DEBUG
> >> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 14
> >> routers to
> >> > > update status.
> >> > > 2021-05-12 22:18:47,291 DEBUG
> >> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 10 VPC
> >> > networks
> >> > > to update Redundant State.
> >> > > 2021-05-12 22:18:47,296 DEBUG
> >> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> >> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 0
> >> networks to
> >> > > update RvR status.
> >> > > 2021-05-12 22:18:49,977 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > > (AgentManager-Handler-2:null) (logid:) SeqA 6-1063908: Processing
> Seq
> >> > > 6-1063908: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
> >> > >
> >> >
> >>
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
> >> > > \"connections\": []\n}","wait":0}}] }
> >> > > 2021-05-12 22:18:49,979 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > > (AgentManager-Handler-2:null) (logid:) SeqA 6-1063908: Sending Seq
> >> > > 6-1063908: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1,
> Flags:
> >> > > 100010,
> >> > >
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> >> }
> >> > > 2021-05-12 22:18:53,959 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > > (AgentManager-Handler-5:null) (logid:) Ping from 4(s-4-VM)
> >> > > 2021-05-12 22:18:53,997 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > > (AgentManager-Handler-12:null) (logid:) Ping from 6(v-7-VM)
> >> > > 2021-05-12 22:18:57,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > > (AsyncJobMgr-Heartbeat-1:ctx-22922b92) (logid:88c162be) Begin
> cleanup
> >> > > expired async-jobs
> >> > > 2021-05-12 22:18:57,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> >> > > (AsyncJobMgr-Heartbeat-1:ctx-22922b92) (logid:88c162be) End cleanup
> >> > expired
> >> > > async-jobs
> >> > > 2021-05-12 22:18:59,978 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > > (AgentManager-Handler-13:null) (logid:) SeqA 6-1063910: Processing
> Seq
> >> > > 6-1063910: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
> >> > >
> >> >
> >>
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
> >> > > \"connections\": []\n}","wait":0}}] }
> >> > > 2021-05-12 22:18:59,980 DEBUG [c.c.a.m.AgentManagerImpl]
> >> > > (AgentManager-Handler-13:null) (logid:) SeqA 6-1063910: Sending Seq
> >> > > 6-1063910: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1,
> Flags:
> >> > > 100010,
> >> > >
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> >> }
> >> > > 2021-05-12 22:19:02,644 DEBUG [c.c.s.StatsCollector]
> >> > > (StatsCollector-2:ctx-1237412a) (logid:3c78cd8f) AutoScaling Monitor
> >> is
> >> > > running...
> >> > > 2021-05-12 22:19:02,934 DEBUG [c.c.s.StatsCollector]
> >> > > (StatsCollector-4:ctx-6c1c6167) (logid:585ce1c2) HostStatsCollector
> is
> >> > > running...
> >> > > 2021-05-12 22:19:05,352 DEBUG [c.c.s.StatsCollector]
> >> > > (StatsCollector-3:ctx-58cb4a3f) (logid:13d34be0) StorageCollector is
> >> > > running...
> >> > >
> >> >
> >>
> >
>
>
>
>
Re: Fail to add KVM Host
Posted by Nicolas Vazquez <Ni...@shapeblue.com>.
Hi Alessandro,
From the agent logs the issue comes from this: Connection with libvirtd is broken: invalid connection pointer in virConnectGetVersion
Which libvirt and qemu versions are you using?
Regards,
Nicolas Vazquez
________________________________
From: Alessandro Caviglione <c....@gmail.com>
Sent: Thursday, May 13, 2021 10:12 AM
To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
Subject: Re: Fail to add KVM Host
Hi Andrija,
any idea?
On Thu, May 13, 2021 at 11:34 AM Alessandro Caviglione <
c.alessandro@gmail.com> wrote:
> Thank you Andrija!!
> https://box.beitsa.ch/index.php/s/ZmGMWbQgmMkbzoK
>
>
> On Thu, May 13, 2021 at 10:04 AM Andrija Panic <an...@gmail.com>
> wrote:
>
>> Can you share (on b pastebin. org or somewhere) the content of the
>> agent.log and setup. log from the KVM host please?
>>
>>
>>
>> Best,
>>
>> On Wed, 12 May 2021, 22:45 Alessandro Caviglione, <c.alessandro@gmail.com
>> >
>> wrote:
>>
>> > Just a quick news... Now I see the host in hosts list, but it's in alert
>> > state.
>> > In log I see:
>> > 2021-05-12 22:40:54,061 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
>> > (AgentConnectTaskPool-417:ctx-36dd291a) (logid:7320b6af) Notifying other
>> > nodes of to disconnect
>> > 2021-05-12 22:40:54,062 DEBUG [c.c.a.m.AgentManagerImpl]
>> > (AgentConnectTaskPool-417:ctx-36dd291a) (logid:7320b6af) Failed to
>> handle
>> > host connection:
>> > com.cloud.exception.ConnectionException: Incorrect Network setup on
>> agent,
>> > Reinitialize agent after network names are setup, details : Can not find
>> > network: cloudbr1
>> >
>> > but I have cloudbr0 and cloudbr1 configured on host...
>> > What can I do?
>> >
>> >
>> > On Wed, May 12, 2021 at 10:34 PM Alessandro Caviglione <
>> > c.alessandro@gmail.com> wrote:
>> >
>> > > Hi guys,
>> > > a lot of troubles these days...
>> > > We've ACS 4.13 installed from source with one zone with a vcenter
>> server
>> > > and advanced networking.
>> > > Now we need to add another zone to manage some KVM hosts but with
>> basic
>> > > networking, we installed libvirt on a Centos7, configured cloudbr0 for
>> > > management and cloudbr1 for gest traffic, installed cloudstack-agent,
>> > made
>> > > all the configuration for tcp and tls, started the services and all
>> it's
>> > ok.
>> > > So we created a new zone in ACS, with cluster and the above host, all
>> > > works fine but the host has not been added.
>> > > So I've tried again to add but after some seconds, the blinking row
>> with
>> > > the new server disappear and in the log I can't see any error:
>> > >
>> > > 2021-05-12 22:18:35,457 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>> > > (secstorage-1:ctx-0b1a9c73) (logid:39eed862) Zone 2 is not ready to
>> > launch
>> > > secondary storage VM yet
>> > > 2021-05-12 22:18:37,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > > (AsyncJobMgr-Heartbeat-1:ctx-7e5591e8) (logid:442791ae) Begin cleanup
>> > > expired async-jobs
>> > > 2021-05-12 22:18:37,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > > (AsyncJobMgr-Heartbeat-1:ctx-7e5591e8) (logid:442791ae) End cleanup
>> > expired
>> > > async-jobs
>> > > 2021-05-12 22:18:37,795 DEBUG [c.c.u.s.SSHCmdHelper]
>> > > (qtp504527234-17:ctx-dc82de11 ctx-2eb654f7) (logid:201abc34) SSH
>> command:
>> > > cloudstack-setup-agent -m 192.168.200.125 -z 2 -p 2 -c 2 -g
>> > > 92f6da7e-d7f0-3b03-93e8-c588dbf9182a -a -s --pubNic=cloudbr1
>> > > --prvNic=cloudbr0 --guestNic=cloudbr1 --hypervisor=kvm
>> > > SSH command output:Starting to configure your system:
>> > > Configure SElinux ... [OK]
>> > > Configure Network ... [OK]
>> > > Configure Libvirt ... [OK]
>> > > Configure Firewall ... [OK]
>> > > Configure Nfs ... [OK]
>> > > Configure cloudAgent ... [OK]
>> > > CloudStack Agent setup is done!
>> > >
>> > >
>> > > 2021-05-12 22:18:44,977 DEBUG [c.c.a.m.AgentManagerImpl]
>> > > (AgentManager-Handler-7:null) (logid:) SeqA 6-1063907: Processing Seq
>> > > 6-1063907: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
>> > >
>> >
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
>> > > \"connections\": []\n}","wait":0}}] }
>> > > 2021-05-12 22:18:44,979 DEBUG [c.c.a.m.AgentManagerImpl]
>> > > (AgentManager-Handler-7:null) (logid:) SeqA 6-1063907: Sending Seq
>> > > 6-1063907: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1, Flags:
>> > > 100010,
>> > > [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> > > 2021-05-12 22:18:47,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > > (AsyncJobMgr-Heartbeat-1:ctx-5b168792) (logid:b9c80c90) Begin cleanup
>> > > expired async-jobs
>> > > 2021-05-12 22:18:47,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > > (AsyncJobMgr-Heartbeat-1:ctx-5b168792) (logid:b9c80c90) End cleanup
>> > expired
>> > > async-jobs
>> > > 2021-05-12 22:18:47,284 DEBUG
>> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 14
>> routers to
>> > > update status.
>> > > 2021-05-12 22:18:47,291 DEBUG
>> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 10 VPC
>> > networks
>> > > to update Redundant State.
>> > > 2021-05-12 22:18:47,296 DEBUG
>> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 0
>> networks to
>> > > update RvR status.
>> > > 2021-05-12 22:18:49,977 DEBUG [c.c.a.m.AgentManagerImpl]
>> > > (AgentManager-Handler-2:null) (logid:) SeqA 6-1063908: Processing Seq
>> > > 6-1063908: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
>> > >
>> >
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
>> > > \"connections\": []\n}","wait":0}}] }
>> > > 2021-05-12 22:18:49,979 DEBUG [c.c.a.m.AgentManagerImpl]
>> > > (AgentManager-Handler-2:null) (logid:) SeqA 6-1063908: Sending Seq
>> > > 6-1063908: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1, Flags:
>> > > 100010,
>> > > [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> > > 2021-05-12 22:18:53,959 DEBUG [c.c.a.m.AgentManagerImpl]
>> > > (AgentManager-Handler-5:null) (logid:) Ping from 4(s-4-VM)
>> > > 2021-05-12 22:18:53,997 DEBUG [c.c.a.m.AgentManagerImpl]
>> > > (AgentManager-Handler-12:null) (logid:) Ping from 6(v-7-VM)
>> > > 2021-05-12 22:18:57,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > > (AsyncJobMgr-Heartbeat-1:ctx-22922b92) (logid:88c162be) Begin cleanup
>> > > expired async-jobs
>> > > 2021-05-12 22:18:57,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > > (AsyncJobMgr-Heartbeat-1:ctx-22922b92) (logid:88c162be) End cleanup
>> > expired
>> > > async-jobs
>> > > 2021-05-12 22:18:59,978 DEBUG [c.c.a.m.AgentManagerImpl]
>> > > (AgentManager-Handler-13:null) (logid:) SeqA 6-1063910: Processing Seq
>> > > 6-1063910: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
>> > >
>> >
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
>> > > \"connections\": []\n}","wait":0}}] }
>> > > 2021-05-12 22:18:59,980 DEBUG [c.c.a.m.AgentManagerImpl]
>> > > (AgentManager-Handler-13:null) (logid:) SeqA 6-1063910: Sending Seq
>> > > 6-1063910: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1, Flags:
>> > > 100010,
>> > > [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> > > 2021-05-12 22:19:02,644 DEBUG [c.c.s.StatsCollector]
>> > > (StatsCollector-2:ctx-1237412a) (logid:3c78cd8f) AutoScaling Monitor
>> is
>> > > running...
>> > > 2021-05-12 22:19:02,934 DEBUG [c.c.s.StatsCollector]
>> > > (StatsCollector-4:ctx-6c1c6167) (logid:585ce1c2) HostStatsCollector is
>> > > running...
>> > > 2021-05-12 22:19:05,352 DEBUG [c.c.s.StatsCollector]
>> > > (StatsCollector-3:ctx-58cb4a3f) (logid:13d34be0) StorageCollector is
>> > > running...
>> > >
>> >
>>
>
Re: Fail to add KVM Host
Posted by Alessandro Caviglione <c....@gmail.com>.
Hi Andrija,
any idea?
On Thu, May 13, 2021 at 11:34 AM Alessandro Caviglione <
c.alessandro@gmail.com> wrote:
> Thank you Andrija!!
> https://box.beitsa.ch/index.php/s/ZmGMWbQgmMkbzoK
>
>
> On Thu, May 13, 2021 at 10:04 AM Andrija Panic <an...@gmail.com>
> wrote:
>
>> Can you share (on b pastebin. org or somewhere) the content of the
>> agent.log and setup. log from the KVM host please?
>>
>>
>>
>> Best,
>>
>> On Wed, 12 May 2021, 22:45 Alessandro Caviglione, <c.alessandro@gmail.com
>> >
>> wrote:
>>
>> > Just a quick news... Now I see the host in hosts list, but it's in alert
>> > state.
>> > In log I see:
>> > 2021-05-12 22:40:54,061 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
>> > (AgentConnectTaskPool-417:ctx-36dd291a) (logid:7320b6af) Notifying other
>> > nodes of to disconnect
>> > 2021-05-12 22:40:54,062 DEBUG [c.c.a.m.AgentManagerImpl]
>> > (AgentConnectTaskPool-417:ctx-36dd291a) (logid:7320b6af) Failed to
>> handle
>> > host connection:
>> > com.cloud.exception.ConnectionException: Incorrect Network setup on
>> agent,
>> > Reinitialize agent after network names are setup, details : Can not find
>> > network: cloudbr1
>> >
>> > but I have cloudbr0 and cloudbr1 configured on host...
>> > What can I do?
>> >
>> >
>> > On Wed, May 12, 2021 at 10:34 PM Alessandro Caviglione <
>> > c.alessandro@gmail.com> wrote:
>> >
>> > > Hi guys,
>> > > a lot of troubles these days...
>> > > We've ACS 4.13 installed from source with one zone with a vcenter
>> server
>> > > and advanced networking.
>> > > Now we need to add another zone to manage some KVM hosts but with
>> basic
>> > > networking, we installed libvirt on a Centos7, configured cloudbr0 for
>> > > management and cloudbr1 for gest traffic, installed cloudstack-agent,
>> > made
>> > > all the configuration for tcp and tls, started the services and all
>> it's
>> > ok.
>> > > So we created a new zone in ACS, with cluster and the above host, all
>> > > works fine but the host has not been added.
>> > > So I've tried again to add but after some seconds, the blinking row
>> with
>> > > the new server disappear and in the log I can't see any error:
>> > >
>> > > 2021-05-12 22:18:35,457 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>> > > (secstorage-1:ctx-0b1a9c73) (logid:39eed862) Zone 2 is not ready to
>> > launch
>> > > secondary storage VM yet
>> > > 2021-05-12 22:18:37,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > > (AsyncJobMgr-Heartbeat-1:ctx-7e5591e8) (logid:442791ae) Begin cleanup
>> > > expired async-jobs
>> > > 2021-05-12 22:18:37,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > > (AsyncJobMgr-Heartbeat-1:ctx-7e5591e8) (logid:442791ae) End cleanup
>> > expired
>> > > async-jobs
>> > > 2021-05-12 22:18:37,795 DEBUG [c.c.u.s.SSHCmdHelper]
>> > > (qtp504527234-17:ctx-dc82de11 ctx-2eb654f7) (logid:201abc34) SSH
>> command:
>> > > cloudstack-setup-agent -m 192.168.200.125 -z 2 -p 2 -c 2 -g
>> > > 92f6da7e-d7f0-3b03-93e8-c588dbf9182a -a -s --pubNic=cloudbr1
>> > > --prvNic=cloudbr0 --guestNic=cloudbr1 --hypervisor=kvm
>> > > SSH command output:Starting to configure your system:
>> > > Configure SElinux ... [OK]
>> > > Configure Network ... [OK]
>> > > Configure Libvirt ... [OK]
>> > > Configure Firewall ... [OK]
>> > > Configure Nfs ... [OK]
>> > > Configure cloudAgent ... [OK]
>> > > CloudStack Agent setup is done!
>> > >
>> > >
>> > > 2021-05-12 22:18:44,977 DEBUG [c.c.a.m.AgentManagerImpl]
>> > > (AgentManager-Handler-7:null) (logid:) SeqA 6-1063907: Processing Seq
>> > > 6-1063907: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
>> > >
>> >
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
>> > > \"connections\": []\n}","wait":0}}] }
>> > > 2021-05-12 22:18:44,979 DEBUG [c.c.a.m.AgentManagerImpl]
>> > > (AgentManager-Handler-7:null) (logid:) SeqA 6-1063907: Sending Seq
>> > > 6-1063907: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1, Flags:
>> > > 100010,
>> > > [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> > > 2021-05-12 22:18:47,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > > (AsyncJobMgr-Heartbeat-1:ctx-5b168792) (logid:b9c80c90) Begin cleanup
>> > > expired async-jobs
>> > > 2021-05-12 22:18:47,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > > (AsyncJobMgr-Heartbeat-1:ctx-5b168792) (logid:b9c80c90) End cleanup
>> > expired
>> > > async-jobs
>> > > 2021-05-12 22:18:47,284 DEBUG
>> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 14
>> routers to
>> > > update status.
>> > > 2021-05-12 22:18:47,291 DEBUG
>> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 10 VPC
>> > networks
>> > > to update Redundant State.
>> > > 2021-05-12 22:18:47,296 DEBUG
>> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 0
>> networks to
>> > > update RvR status.
>> > > 2021-05-12 22:18:49,977 DEBUG [c.c.a.m.AgentManagerImpl]
>> > > (AgentManager-Handler-2:null) (logid:) SeqA 6-1063908: Processing Seq
>> > > 6-1063908: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
>> > >
>> >
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
>> > > \"connections\": []\n}","wait":0}}] }
>> > > 2021-05-12 22:18:49,979 DEBUG [c.c.a.m.AgentManagerImpl]
>> > > (AgentManager-Handler-2:null) (logid:) SeqA 6-1063908: Sending Seq
>> > > 6-1063908: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1, Flags:
>> > > 100010,
>> > > [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> > > 2021-05-12 22:18:53,959 DEBUG [c.c.a.m.AgentManagerImpl]
>> > > (AgentManager-Handler-5:null) (logid:) Ping from 4(s-4-VM)
>> > > 2021-05-12 22:18:53,997 DEBUG [c.c.a.m.AgentManagerImpl]
>> > > (AgentManager-Handler-12:null) (logid:) Ping from 6(v-7-VM)
>> > > 2021-05-12 22:18:57,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > > (AsyncJobMgr-Heartbeat-1:ctx-22922b92) (logid:88c162be) Begin cleanup
>> > > expired async-jobs
>> > > 2021-05-12 22:18:57,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
>> > > (AsyncJobMgr-Heartbeat-1:ctx-22922b92) (logid:88c162be) End cleanup
>> > expired
>> > > async-jobs
>> > > 2021-05-12 22:18:59,978 DEBUG [c.c.a.m.AgentManagerImpl]
>> > > (AgentManager-Handler-13:null) (logid:) SeqA 6-1063910: Processing Seq
>> > > 6-1063910: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
>> > >
>> >
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
>> > > \"connections\": []\n}","wait":0}}] }
>> > > 2021-05-12 22:18:59,980 DEBUG [c.c.a.m.AgentManagerImpl]
>> > > (AgentManager-Handler-13:null) (logid:) SeqA 6-1063910: Sending Seq
>> > > 6-1063910: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1, Flags:
>> > > 100010,
>> > > [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
>> }
>> > > 2021-05-12 22:19:02,644 DEBUG [c.c.s.StatsCollector]
>> > > (StatsCollector-2:ctx-1237412a) (logid:3c78cd8f) AutoScaling Monitor
>> is
>> > > running...
>> > > 2021-05-12 22:19:02,934 DEBUG [c.c.s.StatsCollector]
>> > > (StatsCollector-4:ctx-6c1c6167) (logid:585ce1c2) HostStatsCollector is
>> > > running...
>> > > 2021-05-12 22:19:05,352 DEBUG [c.c.s.StatsCollector]
>> > > (StatsCollector-3:ctx-58cb4a3f) (logid:13d34be0) StorageCollector is
>> > > running...
>> > >
>> >
>>
>
Re: Fail to add KVM Host
Posted by Alessandro Caviglione <c....@gmail.com>.
Thank you Andrija!!
https://box.beitsa.ch/index.php/s/ZmGMWbQgmMkbzoK
On Thu, May 13, 2021 at 10:04 AM Andrija Panic <an...@gmail.com>
wrote:
> Can you share (on b pastebin. org or somewhere) the content of the
> agent.log and setup. log from the KVM host please?
>
>
>
> Best,
>
> On Wed, 12 May 2021, 22:45 Alessandro Caviglione, <c....@gmail.com>
> wrote:
>
> > Just a quick news... Now I see the host in hosts list, but it's in alert
> > state.
> > In log I see:
> > 2021-05-12 22:40:54,061 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
> > (AgentConnectTaskPool-417:ctx-36dd291a) (logid:7320b6af) Notifying other
> > nodes of to disconnect
> > 2021-05-12 22:40:54,062 DEBUG [c.c.a.m.AgentManagerImpl]
> > (AgentConnectTaskPool-417:ctx-36dd291a) (logid:7320b6af) Failed to handle
> > host connection:
> > com.cloud.exception.ConnectionException: Incorrect Network setup on
> agent,
> > Reinitialize agent after network names are setup, details : Can not find
> > network: cloudbr1
> >
> > but I have cloudbr0 and cloudbr1 configured on host...
> > What can I do?
> >
> >
> > On Wed, May 12, 2021 at 10:34 PM Alessandro Caviglione <
> > c.alessandro@gmail.com> wrote:
> >
> > > Hi guys,
> > > a lot of troubles these days...
> > > We've ACS 4.13 installed from source with one zone with a vcenter
> server
> > > and advanced networking.
> > > Now we need to add another zone to manage some KVM hosts but with basic
> > > networking, we installed libvirt on a Centos7, configured cloudbr0 for
> > > management and cloudbr1 for gest traffic, installed cloudstack-agent,
> > made
> > > all the configuration for tcp and tls, started the services and all
> it's
> > ok.
> > > So we created a new zone in ACS, with cluster and the above host, all
> > > works fine but the host has not been added.
> > > So I've tried again to add but after some seconds, the blinking row
> with
> > > the new server disappear and in the log I can't see any error:
> > >
> > > 2021-05-12 22:18:35,457 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
> > > (secstorage-1:ctx-0b1a9c73) (logid:39eed862) Zone 2 is not ready to
> > launch
> > > secondary storage VM yet
> > > 2021-05-12 22:18:37,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (AsyncJobMgr-Heartbeat-1:ctx-7e5591e8) (logid:442791ae) Begin cleanup
> > > expired async-jobs
> > > 2021-05-12 22:18:37,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (AsyncJobMgr-Heartbeat-1:ctx-7e5591e8) (logid:442791ae) End cleanup
> > expired
> > > async-jobs
> > > 2021-05-12 22:18:37,795 DEBUG [c.c.u.s.SSHCmdHelper]
> > > (qtp504527234-17:ctx-dc82de11 ctx-2eb654f7) (logid:201abc34) SSH
> command:
> > > cloudstack-setup-agent -m 192.168.200.125 -z 2 -p 2 -c 2 -g
> > > 92f6da7e-d7f0-3b03-93e8-c588dbf9182a -a -s --pubNic=cloudbr1
> > > --prvNic=cloudbr0 --guestNic=cloudbr1 --hypervisor=kvm
> > > SSH command output:Starting to configure your system:
> > > Configure SElinux ... [OK]
> > > Configure Network ... [OK]
> > > Configure Libvirt ... [OK]
> > > Configure Firewall ... [OK]
> > > Configure Nfs ... [OK]
> > > Configure cloudAgent ... [OK]
> > > CloudStack Agent setup is done!
> > >
> > >
> > > 2021-05-12 22:18:44,977 DEBUG [c.c.a.m.AgentManagerImpl]
> > > (AgentManager-Handler-7:null) (logid:) SeqA 6-1063907: Processing Seq
> > > 6-1063907: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
> > >
> >
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
> > > \"connections\": []\n}","wait":0}}] }
> > > 2021-05-12 22:18:44,979 DEBUG [c.c.a.m.AgentManagerImpl]
> > > (AgentManager-Handler-7:null) (logid:) SeqA 6-1063907: Sending Seq
> > > 6-1063907: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1, Flags:
> > > 100010,
> > > [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> > > 2021-05-12 22:18:47,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (AsyncJobMgr-Heartbeat-1:ctx-5b168792) (logid:b9c80c90) Begin cleanup
> > > expired async-jobs
> > > 2021-05-12 22:18:47,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (AsyncJobMgr-Heartbeat-1:ctx-5b168792) (logid:b9c80c90) End cleanup
> > expired
> > > async-jobs
> > > 2021-05-12 22:18:47,284 DEBUG
> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 14 routers
> to
> > > update status.
> > > 2021-05-12 22:18:47,291 DEBUG
> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 10 VPC
> > networks
> > > to update Redundant State.
> > > 2021-05-12 22:18:47,296 DEBUG
> > [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 0 networks
> to
> > > update RvR status.
> > > 2021-05-12 22:18:49,977 DEBUG [c.c.a.m.AgentManagerImpl]
> > > (AgentManager-Handler-2:null) (logid:) SeqA 6-1063908: Processing Seq
> > > 6-1063908: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
> > >
> >
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
> > > \"connections\": []\n}","wait":0}}] }
> > > 2021-05-12 22:18:49,979 DEBUG [c.c.a.m.AgentManagerImpl]
> > > (AgentManager-Handler-2:null) (logid:) SeqA 6-1063908: Sending Seq
> > > 6-1063908: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1, Flags:
> > > 100010,
> > > [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> > > 2021-05-12 22:18:53,959 DEBUG [c.c.a.m.AgentManagerImpl]
> > > (AgentManager-Handler-5:null) (logid:) Ping from 4(s-4-VM)
> > > 2021-05-12 22:18:53,997 DEBUG [c.c.a.m.AgentManagerImpl]
> > > (AgentManager-Handler-12:null) (logid:) Ping from 6(v-7-VM)
> > > 2021-05-12 22:18:57,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (AsyncJobMgr-Heartbeat-1:ctx-22922b92) (logid:88c162be) Begin cleanup
> > > expired async-jobs
> > > 2021-05-12 22:18:57,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (AsyncJobMgr-Heartbeat-1:ctx-22922b92) (logid:88c162be) End cleanup
> > expired
> > > async-jobs
> > > 2021-05-12 22:18:59,978 DEBUG [c.c.a.m.AgentManagerImpl]
> > > (AgentManager-Handler-13:null) (logid:) SeqA 6-1063910: Processing Seq
> > > 6-1063910: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
> > >
> >
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
> > > \"connections\": []\n}","wait":0}}] }
> > > 2021-05-12 22:18:59,980 DEBUG [c.c.a.m.AgentManagerImpl]
> > > (AgentManager-Handler-13:null) (logid:) SeqA 6-1063910: Sending Seq
> > > 6-1063910: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1, Flags:
> > > 100010,
> > > [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> > > 2021-05-12 22:19:02,644 DEBUG [c.c.s.StatsCollector]
> > > (StatsCollector-2:ctx-1237412a) (logid:3c78cd8f) AutoScaling Monitor is
> > > running...
> > > 2021-05-12 22:19:02,934 DEBUG [c.c.s.StatsCollector]
> > > (StatsCollector-4:ctx-6c1c6167) (logid:585ce1c2) HostStatsCollector is
> > > running...
> > > 2021-05-12 22:19:05,352 DEBUG [c.c.s.StatsCollector]
> > > (StatsCollector-3:ctx-58cb4a3f) (logid:13d34be0) StorageCollector is
> > > running...
> > >
> >
>
Re: Fail to add KVM Host
Posted by Andrija Panic <an...@gmail.com>.
Can you share (on b pastebin. org or somewhere) the content of the
agent.log and setup. log from the KVM host please?
Best,
On Wed, 12 May 2021, 22:45 Alessandro Caviglione, <c....@gmail.com>
wrote:
> Just a quick news... Now I see the host in hosts list, but it's in alert
> state.
> In log I see:
> 2021-05-12 22:40:54,061 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
> (AgentConnectTaskPool-417:ctx-36dd291a) (logid:7320b6af) Notifying other
> nodes of to disconnect
> 2021-05-12 22:40:54,062 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentConnectTaskPool-417:ctx-36dd291a) (logid:7320b6af) Failed to handle
> host connection:
> com.cloud.exception.ConnectionException: Incorrect Network setup on agent,
> Reinitialize agent after network names are setup, details : Can not find
> network: cloudbr1
>
> but I have cloudbr0 and cloudbr1 configured on host...
> What can I do?
>
>
> On Wed, May 12, 2021 at 10:34 PM Alessandro Caviglione <
> c.alessandro@gmail.com> wrote:
>
> > Hi guys,
> > a lot of troubles these days...
> > We've ACS 4.13 installed from source with one zone with a vcenter server
> > and advanced networking.
> > Now we need to add another zone to manage some KVM hosts but with basic
> > networking, we installed libvirt on a Centos7, configured cloudbr0 for
> > management and cloudbr1 for gest traffic, installed cloudstack-agent,
> made
> > all the configuration for tcp and tls, started the services and all it's
> ok.
> > So we created a new zone in ACS, with cluster and the above host, all
> > works fine but the host has not been added.
> > So I've tried again to add but after some seconds, the blinking row with
> > the new server disappear and in the log I can't see any error:
> >
> > 2021-05-12 22:18:35,457 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
> > (secstorage-1:ctx-0b1a9c73) (logid:39eed862) Zone 2 is not ready to
> launch
> > secondary storage VM yet
> > 2021-05-12 22:18:37,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (AsyncJobMgr-Heartbeat-1:ctx-7e5591e8) (logid:442791ae) Begin cleanup
> > expired async-jobs
> > 2021-05-12 22:18:37,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (AsyncJobMgr-Heartbeat-1:ctx-7e5591e8) (logid:442791ae) End cleanup
> expired
> > async-jobs
> > 2021-05-12 22:18:37,795 DEBUG [c.c.u.s.SSHCmdHelper]
> > (qtp504527234-17:ctx-dc82de11 ctx-2eb654f7) (logid:201abc34) SSH command:
> > cloudstack-setup-agent -m 192.168.200.125 -z 2 -p 2 -c 2 -g
> > 92f6da7e-d7f0-3b03-93e8-c588dbf9182a -a -s --pubNic=cloudbr1
> > --prvNic=cloudbr0 --guestNic=cloudbr1 --hypervisor=kvm
> > SSH command output:Starting to configure your system:
> > Configure SElinux ... [OK]
> > Configure Network ... [OK]
> > Configure Libvirt ... [OK]
> > Configure Firewall ... [OK]
> > Configure Nfs ... [OK]
> > Configure cloudAgent ... [OK]
> > CloudStack Agent setup is done!
> >
> >
> > 2021-05-12 22:18:44,977 DEBUG [c.c.a.m.AgentManagerImpl]
> > (AgentManager-Handler-7:null) (logid:) SeqA 6-1063907: Processing Seq
> > 6-1063907: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
> >
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
> > \"connections\": []\n}","wait":0}}] }
> > 2021-05-12 22:18:44,979 DEBUG [c.c.a.m.AgentManagerImpl]
> > (AgentManager-Handler-7:null) (logid:) SeqA 6-1063907: Sending Seq
> > 6-1063907: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1, Flags:
> > 100010,
> > [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> > 2021-05-12 22:18:47,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (AsyncJobMgr-Heartbeat-1:ctx-5b168792) (logid:b9c80c90) Begin cleanup
> > expired async-jobs
> > 2021-05-12 22:18:47,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (AsyncJobMgr-Heartbeat-1:ctx-5b168792) (logid:b9c80c90) End cleanup
> expired
> > async-jobs
> > 2021-05-12 22:18:47,284 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 14 routers to
> > update status.
> > 2021-05-12 22:18:47,291 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 10 VPC
> networks
> > to update Redundant State.
> > 2021-05-12 22:18:47,296 DEBUG
> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> > (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 0 networks to
> > update RvR status.
> > 2021-05-12 22:18:49,977 DEBUG [c.c.a.m.AgentManagerImpl]
> > (AgentManager-Handler-2:null) (logid:) SeqA 6-1063908: Processing Seq
> > 6-1063908: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
> >
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
> > \"connections\": []\n}","wait":0}}] }
> > 2021-05-12 22:18:49,979 DEBUG [c.c.a.m.AgentManagerImpl]
> > (AgentManager-Handler-2:null) (logid:) SeqA 6-1063908: Sending Seq
> > 6-1063908: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1, Flags:
> > 100010,
> > [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> > 2021-05-12 22:18:53,959 DEBUG [c.c.a.m.AgentManagerImpl]
> > (AgentManager-Handler-5:null) (logid:) Ping from 4(s-4-VM)
> > 2021-05-12 22:18:53,997 DEBUG [c.c.a.m.AgentManagerImpl]
> > (AgentManager-Handler-12:null) (logid:) Ping from 6(v-7-VM)
> > 2021-05-12 22:18:57,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (AsyncJobMgr-Heartbeat-1:ctx-22922b92) (logid:88c162be) Begin cleanup
> > expired async-jobs
> > 2021-05-12 22:18:57,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (AsyncJobMgr-Heartbeat-1:ctx-22922b92) (logid:88c162be) End cleanup
> expired
> > async-jobs
> > 2021-05-12 22:18:59,978 DEBUG [c.c.a.m.AgentManagerImpl]
> > (AgentManager-Handler-13:null) (logid:) SeqA 6-1063910: Processing Seq
> > 6-1063910: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
> >
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
> > \"connections\": []\n}","wait":0}}] }
> > 2021-05-12 22:18:59,980 DEBUG [c.c.a.m.AgentManagerImpl]
> > (AgentManager-Handler-13:null) (logid:) SeqA 6-1063910: Sending Seq
> > 6-1063910: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1, Flags:
> > 100010,
> > [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> > 2021-05-12 22:19:02,644 DEBUG [c.c.s.StatsCollector]
> > (StatsCollector-2:ctx-1237412a) (logid:3c78cd8f) AutoScaling Monitor is
> > running...
> > 2021-05-12 22:19:02,934 DEBUG [c.c.s.StatsCollector]
> > (StatsCollector-4:ctx-6c1c6167) (logid:585ce1c2) HostStatsCollector is
> > running...
> > 2021-05-12 22:19:05,352 DEBUG [c.c.s.StatsCollector]
> > (StatsCollector-3:ctx-58cb4a3f) (logid:13d34be0) StorageCollector is
> > running...
> >
>
Re: Fail to add KVM Host
Posted by Alessandro Caviglione <c....@gmail.com>.
Just a quick news... Now I see the host in hosts list, but it's in alert
state.
In log I see:
2021-05-12 22:40:54,061 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
(AgentConnectTaskPool-417:ctx-36dd291a) (logid:7320b6af) Notifying other
nodes of to disconnect
2021-05-12 22:40:54,062 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-417:ctx-36dd291a) (logid:7320b6af) Failed to handle
host connection:
com.cloud.exception.ConnectionException: Incorrect Network setup on agent,
Reinitialize agent after network names are setup, details : Can not find
network: cloudbr1
but I have cloudbr0 and cloudbr1 configured on host...
What can I do?
On Wed, May 12, 2021 at 10:34 PM Alessandro Caviglione <
c.alessandro@gmail.com> wrote:
> Hi guys,
> a lot of troubles these days...
> We've ACS 4.13 installed from source with one zone with a vcenter server
> and advanced networking.
> Now we need to add another zone to manage some KVM hosts but with basic
> networking, we installed libvirt on a Centos7, configured cloudbr0 for
> management and cloudbr1 for gest traffic, installed cloudstack-agent, made
> all the configuration for tcp and tls, started the services and all it's ok.
> So we created a new zone in ACS, with cluster and the above host, all
> works fine but the host has not been added.
> So I've tried again to add but after some seconds, the blinking row with
> the new server disappear and in the log I can't see any error:
>
> 2021-05-12 22:18:35,457 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
> (secstorage-1:ctx-0b1a9c73) (logid:39eed862) Zone 2 is not ready to launch
> secondary storage VM yet
> 2021-05-12 22:18:37,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-7e5591e8) (logid:442791ae) Begin cleanup
> expired async-jobs
> 2021-05-12 22:18:37,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-7e5591e8) (logid:442791ae) End cleanup expired
> async-jobs
> 2021-05-12 22:18:37,795 DEBUG [c.c.u.s.SSHCmdHelper]
> (qtp504527234-17:ctx-dc82de11 ctx-2eb654f7) (logid:201abc34) SSH command:
> cloudstack-setup-agent -m 192.168.200.125 -z 2 -p 2 -c 2 -g
> 92f6da7e-d7f0-3b03-93e8-c588dbf9182a -a -s --pubNic=cloudbr1
> --prvNic=cloudbr0 --guestNic=cloudbr1 --hypervisor=kvm
> SSH command output:Starting to configure your system:
> Configure SElinux ... [OK]
> Configure Network ... [OK]
> Configure Libvirt ... [OK]
> Configure Firewall ... [OK]
> Configure Nfs ... [OK]
> Configure cloudAgent ... [OK]
> CloudStack Agent setup is done!
>
>
> 2021-05-12 22:18:44,977 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-7:null) (logid:) SeqA 6-1063907: Processing Seq
> 6-1063907: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2021-05-12 22:18:44,979 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-7:null) (logid:) SeqA 6-1063907: Sending Seq
> 6-1063907: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> 2021-05-12 22:18:47,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-5b168792) (logid:b9c80c90) Begin cleanup
> expired async-jobs
> 2021-05-12 22:18:47,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-5b168792) (logid:b9c80c90) End cleanup expired
> async-jobs
> 2021-05-12 22:18:47,284 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 14 routers to
> update status.
> 2021-05-12 22:18:47,291 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 10 VPC networks
> to update Redundant State.
> 2021-05-12 22:18:47,296 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-0865edb1) (logid:c141e973) Found 0 networks to
> update RvR status.
> 2021-05-12 22:18:49,977 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-2:null) (logid:) SeqA 6-1063908: Processing Seq
> 6-1063908: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2021-05-12 22:18:49,979 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-2:null) (logid:) SeqA 6-1063908: Sending Seq
> 6-1063908: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> 2021-05-12 22:18:53,959 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-5:null) (logid:) Ping from 4(s-4-VM)
> 2021-05-12 22:18:53,997 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-12:null) (logid:) Ping from 6(v-7-VM)
> 2021-05-12 22:18:57,178 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-22922b92) (logid:88c162be) Begin cleanup
> expired async-jobs
> 2021-05-12 22:18:57,181 INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-22922b92) (logid:88c162be) End cleanup expired
> async-jobs
> 2021-05-12 22:18:59,978 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-13:null) (logid:) SeqA 6-1063910: Processing Seq
> 6-1063910: { Cmd , MgmtId: -1, via: 6, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":7,"_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2021-05-12 22:18:59,980 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-13:null) (logid:) SeqA 6-1063910: Sending Seq
> 6-1063910: { Ans: , MgmtId: 213426611879184, via: 6, Ver: v1, Flags:
> 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> 2021-05-12 22:19:02,644 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-2:ctx-1237412a) (logid:3c78cd8f) AutoScaling Monitor is
> running...
> 2021-05-12 22:19:02,934 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-4:ctx-6c1c6167) (logid:585ce1c2) HostStatsCollector is
> running...
> 2021-05-12 22:19:05,352 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-3:ctx-58cb4a3f) (logid:13d34be0) StorageCollector is
> running...
>