You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by jordan j <yo...@gmail.com> on 2022/09/13 06:11:24 UTC

Virtual Router cannot start

Dear community,

 I have a fresh environment 4.17.0.1 with 1 XenServer.
- 1 Zone - both System VMs and User VMs on Local storage options selected.
- server nics and network design - 2 bonds with VIrtual Router and VPC
enabled.
   -> Public (BOND01 label) - x.x.x.21-29
   -> mgmt (BOND01 label)  - x.x.x.100-199
   -> guest (BOND02 label)

Network offerings and zone networks created from them:
- SHARED_LOCAL - Shared network offering created (BOND02) with Virtual
Router for Local storage. It will provide DHCP, DNS, USERDATA.
- SHARED_NO_VR - shared network with no VR or services

USER VM creation on different networks:
(1) starting a VM on SHARED_LOCAL network - creation fails.
(2) starting a VM on SHARED_NO_VR network - it works without a problem.

However during (1) VR is created in infrastructure -> VR  tab but failed
to start. An attempt to start it manually fails with error.
Unable to create a deployment for VM instance {id: "175", name: "r-175-VM",
uuid: "47a5b579-8aab-4df0-b305-07de30ae7ab0",
type="DomainRouter"}Scope=interface com.cloud.dc.DataCenter; id=47

I see the problem is in interface creation (in GUI i see one halfway
created interface of traffic Type: Control) but i am not really sure how to
take it from here.

What else needs to be enabled for the VR to work? How to further continue
with troubleshooting from here?

Best regards,
Jordan

Re: Virtual Router cannot start

Posted by jordan j <yo...@gmail.com>.
The issue is now fixed.
It seems the vlan 500 was already present on that bond so the XCP-NG
refused to create one on demand for the VR.
It's still a mystery why the error reports trying to create an interface on
device bond0 which is not present anywhere.

Best regards,
Jordan

On Wed, Sep 14, 2022 at 9:42 AM jordan j <yo...@gmail.com> wrote:

> I did some tests, if i make a zone with 1 physical interface (on bond with
> label bond02) and all networks are on it - management, public, guest - all
> is working fine.
> But when i have 2 physical interfaces (labels bond01 for guest and bond02
> for mgmt and public) the the following error appears when VR is being
> created:
>
> 2022-09-14 08:34:44,009 DEBUG [c.c.h.x.r.CitrixResourceBase]
> (DirectAgent-265:ctx-cc426eab) (logid:74d57e46) Failed to find SR by name
> 'XenServer Tools', will try to find 'XCP-ng Tools' SR
> 2022-09-14 08:34:44,508 DEBUG [c.c.h.x.r.CitrixResourceBase]
> (DirectAgent-265:ctx-cc426eab) (logid:74d57e46) VBD
> d9d2a315-cb27-8f20-7c61-695f14d62259 created for
> com.cloud.agent.api.to.DiskTO@27636827
> 2022-09-14 08:34:44,508 DEBUG [c.c.h.x.r.CitrixResourceBase]
> (DirectAgent-265:ctx-cc426eab) (logid:74d57e46) Creating VIF for r-186-VM
> on nic [Nic:Guest-172.128.0.20-vlan://500]
> 2022-09-14 08:34:44,509 DEBUG [c.c.h.x.r.CitrixResourceBase]
> (DirectAgent-265:ctx-cc426eab) (logid:74d57e46) Looking for network named
> bond01
> 2022-09-14 08:34:44,515 DEBUG [c.c.h.x.r.XsLocalNetwork]
> (DirectAgent-265:ctx-cc426eab) (logid:74d57e46) Found a network called
> bond01 on host=172.16.14.105;
>  Network=c9fbc161-e7ee-72cf-295c-1653da3ba596;
> pif=714fe823-dcd5-8115-305c-4a243ffcd396
> 2022-09-14 08:34:44,517 DEBUG [c.c.h.x.r.CitrixResourceBase]
> (DirectAgent-265:ctx-cc426eab) (logid:74d57e46) Creating VLAN 500 on host
> 172.16.14.105 on device bond0
> 2022-09-14 08:34:44,522 WARN  [c.c.h.x.r.w.x.CitrixStartCommandWrapper]
> (DirectAgent-265:ctx-cc426eab) (logid:74d57e46) Catch Exception: class
> com.xensource.xenapi.Types$PifVlanExists due to You tried to create a PIF,
> but it already exists.
> You tried to create a PIF, but it already exists.
>
>
> I don't understand why it says "Creating VLAN 500 on host 172.16.14.105 on
> device bond0" where i dont have such a device or label?
> Could it be a bug?  (v4.17.0.1 + xcp-ng 8.2.1 latest updates).
>
> Regards,
> Jordan
>
> On Tue, Sep 13, 2022 at 10:49 AM jordan j <yo...@gmail.com> wrote:
>
>> Here you go!
>> https://filebin.net/fw07uwmlmjd0f1yp/management-server.log
>>
>> Snippet is generated during the start of a new user vm which also try to
>> start VR.
>>
>> Alright I see one more issue I did not catch before:
>> - You tried to create a PIF, but it already exists. (for the NIC of the
>> VR) regards,
>> Jordan
>>
>> On Tue, Sep 13, 2022 at 10:10 AM Wei ZHOU <us...@gmail.com> wrote:
>>
>>> Hi Jordan,
>>>
>>> Can you share a full log of the api call
>>> in /var/log/cloudstack/management/management-server.log ?
>>>
>>> -Wei
>>>
>>> On Tue, 13 Sept 2022 at 08:11, jordan j <yo...@gmail.com> wrote:
>>>
>>> > Dear community,
>>> >
>>> >  I have a fresh environment 4.17.0.1 with 1 XenServer.
>>> > - 1 Zone - both System VMs and User VMs on Local storage options
>>> selected.
>>> > - server nics and network design - 2 bonds with VIrtual Router and VPC
>>> > enabled.
>>> >    -> Public (BOND01 label) - x.x.x.21-29
>>> >    -> mgmt (BOND01 label)  - x.x.x.100-199
>>> >    -> guest (BOND02 label)
>>> >
>>> > Network offerings and zone networks created from them:
>>> > - SHARED_LOCAL - Shared network offering created (BOND02) with Virtual
>>> > Router for Local storage. It will provide DHCP, DNS, USERDATA.
>>> > - SHARED_NO_VR - shared network with no VR or services
>>> >
>>> > USER VM creation on different networks:
>>> > (1) starting a VM on SHARED_LOCAL network - creation fails.
>>> > (2) starting a VM on SHARED_NO_VR network - it works without a problem.
>>> >
>>> > However during (1) VR is created in infrastructure -> VR  tab but
>>> failed
>>> > to start. An attempt to start it manually fails with error.
>>> > Unable to create a deployment for VM instance {id: "175", name:
>>> "r-175-VM",
>>> > uuid: "47a5b579-8aab-4df0-b305-07de30ae7ab0",
>>> > type="DomainRouter"}Scope=interface com.cloud.dc.DataCenter; id=47
>>> >
>>> > I see the problem is in interface creation (in GUI i see one halfway
>>> > created interface of traffic Type: Control) but i am not really sure
>>> how to
>>> > take it from here.
>>> >
>>> > What else needs to be enabled for the VR to work? How to further
>>> continue
>>> > with troubleshooting from here?
>>> >
>>> > Best regards,
>>> > Jordan
>>> >
>>>
>>

Re: Virtual Router cannot start

Posted by jordan j <yo...@gmail.com>.
I did some tests, if i make a zone with 1 physical interface (on bond with
label bond02) and all networks are on it - management, public, guest - all
is working fine.
But when i have 2 physical interfaces (labels bond01 for guest and bond02
for mgmt and public) the the following error appears when VR is being
created:

2022-09-14 08:34:44,009 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-265:ctx-cc426eab) (logid:74d57e46) Failed to find SR by name
'XenServer Tools', will try to find 'XCP-ng Tools' SR
2022-09-14 08:34:44,508 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-265:ctx-cc426eab) (logid:74d57e46) VBD
d9d2a315-cb27-8f20-7c61-695f14d62259 created for
com.cloud.agent.api.to.DiskTO@27636827
2022-09-14 08:34:44,508 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-265:ctx-cc426eab) (logid:74d57e46) Creating VIF for r-186-VM
on nic [Nic:Guest-172.128.0.20-vlan://500]
2022-09-14 08:34:44,509 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-265:ctx-cc426eab) (logid:74d57e46) Looking for network named
bond01
2022-09-14 08:34:44,515 DEBUG [c.c.h.x.r.XsLocalNetwork]
(DirectAgent-265:ctx-cc426eab) (logid:74d57e46) Found a network called
bond01 on host=172.16.14.105;
 Network=c9fbc161-e7ee-72cf-295c-1653da3ba596;
pif=714fe823-dcd5-8115-305c-4a243ffcd396
2022-09-14 08:34:44,517 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-265:ctx-cc426eab) (logid:74d57e46) Creating VLAN 500 on host
172.16.14.105 on device bond0
2022-09-14 08:34:44,522 WARN  [c.c.h.x.r.w.x.CitrixStartCommandWrapper]
(DirectAgent-265:ctx-cc426eab) (logid:74d57e46) Catch Exception: class
com.xensource.xenapi.Types$PifVlanExists due to You tried to create a PIF,
but it already exists.
You tried to create a PIF, but it already exists.


I don't understand why it says "Creating VLAN 500 on host 172.16.14.105 on
device bond0" where i dont have such a device or label?
Could it be a bug?  (v4.17.0.1 + xcp-ng 8.2.1 latest updates).

Regards,
Jordan

On Tue, Sep 13, 2022 at 10:49 AM jordan j <yo...@gmail.com> wrote:

> Here you go!
> https://filebin.net/fw07uwmlmjd0f1yp/management-server.log
>
> Snippet is generated during the start of a new user vm which also try to
> start VR.
>
> Alright I see one more issue I did not catch before:
> - You tried to create a PIF, but it already exists. (for the NIC of the
> VR) regards,
> Jordan
>
> On Tue, Sep 13, 2022 at 10:10 AM Wei ZHOU <us...@gmail.com> wrote:
>
>> Hi Jordan,
>>
>> Can you share a full log of the api call
>> in /var/log/cloudstack/management/management-server.log ?
>>
>> -Wei
>>
>> On Tue, 13 Sept 2022 at 08:11, jordan j <yo...@gmail.com> wrote:
>>
>> > Dear community,
>> >
>> >  I have a fresh environment 4.17.0.1 with 1 XenServer.
>> > - 1 Zone - both System VMs and User VMs on Local storage options
>> selected.
>> > - server nics and network design - 2 bonds with VIrtual Router and VPC
>> > enabled.
>> >    -> Public (BOND01 label) - x.x.x.21-29
>> >    -> mgmt (BOND01 label)  - x.x.x.100-199
>> >    -> guest (BOND02 label)
>> >
>> > Network offerings and zone networks created from them:
>> > - SHARED_LOCAL - Shared network offering created (BOND02) with Virtual
>> > Router for Local storage. It will provide DHCP, DNS, USERDATA.
>> > - SHARED_NO_VR - shared network with no VR or services
>> >
>> > USER VM creation on different networks:
>> > (1) starting a VM on SHARED_LOCAL network - creation fails.
>> > (2) starting a VM on SHARED_NO_VR network - it works without a problem.
>> >
>> > However during (1) VR is created in infrastructure -> VR  tab but failed
>> > to start. An attempt to start it manually fails with error.
>> > Unable to create a deployment for VM instance {id: "175", name:
>> "r-175-VM",
>> > uuid: "47a5b579-8aab-4df0-b305-07de30ae7ab0",
>> > type="DomainRouter"}Scope=interface com.cloud.dc.DataCenter; id=47
>> >
>> > I see the problem is in interface creation (in GUI i see one halfway
>> > created interface of traffic Type: Control) but i am not really sure
>> how to
>> > take it from here.
>> >
>> > What else needs to be enabled for the VR to work? How to further
>> continue
>> > with troubleshooting from here?
>> >
>> > Best regards,
>> > Jordan
>> >
>>
>

Re: Virtual Router cannot start

Posted by jordan j <yo...@gmail.com>.
Here you go!
https://filebin.net/fw07uwmlmjd0f1yp/management-server.log

Snippet is generated during the start of a new user vm which also try to
start VR.

Alright I see one more issue I did not catch before:
- You tried to create a PIF, but it already exists. (for the NIC of the VR)
regards,
Jordan

On Tue, Sep 13, 2022 at 10:10 AM Wei ZHOU <us...@gmail.com> wrote:

> Hi Jordan,
>
> Can you share a full log of the api call
> in /var/log/cloudstack/management/management-server.log ?
>
> -Wei
>
> On Tue, 13 Sept 2022 at 08:11, jordan j <yo...@gmail.com> wrote:
>
> > Dear community,
> >
> >  I have a fresh environment 4.17.0.1 with 1 XenServer.
> > - 1 Zone - both System VMs and User VMs on Local storage options
> selected.
> > - server nics and network design - 2 bonds with VIrtual Router and VPC
> > enabled.
> >    -> Public (BOND01 label) - x.x.x.21-29
> >    -> mgmt (BOND01 label)  - x.x.x.100-199
> >    -> guest (BOND02 label)
> >
> > Network offerings and zone networks created from them:
> > - SHARED_LOCAL - Shared network offering created (BOND02) with Virtual
> > Router for Local storage. It will provide DHCP, DNS, USERDATA.
> > - SHARED_NO_VR - shared network with no VR or services
> >
> > USER VM creation on different networks:
> > (1) starting a VM on SHARED_LOCAL network - creation fails.
> > (2) starting a VM on SHARED_NO_VR network - it works without a problem.
> >
> > However during (1) VR is created in infrastructure -> VR  tab but failed
> > to start. An attempt to start it manually fails with error.
> > Unable to create a deployment for VM instance {id: "175", name:
> "r-175-VM",
> > uuid: "47a5b579-8aab-4df0-b305-07de30ae7ab0",
> > type="DomainRouter"}Scope=interface com.cloud.dc.DataCenter; id=47
> >
> > I see the problem is in interface creation (in GUI i see one halfway
> > created interface of traffic Type: Control) but i am not really sure how
> to
> > take it from here.
> >
> > What else needs to be enabled for the VR to work? How to further continue
> > with troubleshooting from here?
> >
> > Best regards,
> > Jordan
> >
>

Re: Virtual Router cannot start

Posted by Wei ZHOU <us...@gmail.com>.
Hi Jordan,

Can you share a full log of the api call
in /var/log/cloudstack/management/management-server.log ?

-Wei

On Tue, 13 Sept 2022 at 08:11, jordan j <yo...@gmail.com> wrote:

> Dear community,
>
>  I have a fresh environment 4.17.0.1 with 1 XenServer.
> - 1 Zone - both System VMs and User VMs on Local storage options selected.
> - server nics and network design - 2 bonds with VIrtual Router and VPC
> enabled.
>    -> Public (BOND01 label) - x.x.x.21-29
>    -> mgmt (BOND01 label)  - x.x.x.100-199
>    -> guest (BOND02 label)
>
> Network offerings and zone networks created from them:
> - SHARED_LOCAL - Shared network offering created (BOND02) with Virtual
> Router for Local storage. It will provide DHCP, DNS, USERDATA.
> - SHARED_NO_VR - shared network with no VR or services
>
> USER VM creation on different networks:
> (1) starting a VM on SHARED_LOCAL network - creation fails.
> (2) starting a VM on SHARED_NO_VR network - it works without a problem.
>
> However during (1) VR is created in infrastructure -> VR  tab but failed
> to start. An attempt to start it manually fails with error.
> Unable to create a deployment for VM instance {id: "175", name: "r-175-VM",
> uuid: "47a5b579-8aab-4df0-b305-07de30ae7ab0",
> type="DomainRouter"}Scope=interface com.cloud.dc.DataCenter; id=47
>
> I see the problem is in interface creation (in GUI i see one halfway
> created interface of traffic Type: Control) but i am not really sure how to
> take it from here.
>
> What else needs to be enabled for the VR to work? How to further continue
> with troubleshooting from here?
>
> Best regards,
> Jordan
>