You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Dan Dong <do...@gmail.com> on 2014/11/11 17:29:09 UTC
Confusable default routing table on SSVM.
Hi, All,
When debugging why my SSVM could not access outside, I see the following
route table on SSVM:
root@s-1-VM:~# ip route show
169.254.0.0/16 dev eth0 proto kernel scope link src 169.254.1.47
192.168.0.0/24 dev eth1 proto kernel scope link src 192.168.0.22
192.168.0.0/24 dev eth2 proto kernel scope link src 192.168.0.134
192.168.0.0/24 dev eth3 proto kernel scope link src 192.168.0.25
There are 3 rules to reach the same subnet of 192.168.0.0/24 through
different NICs.
The problem is that SSVM could not ping both the KVM host(192.168.0.101) on
which the SSVM is running and the Management Server(192.168.0.100, it is
also set to be the gateway of the 192.168.0.0/24 network), although they
are all on the same subnet. I suspect it is the bridge setings of the KVM,
but no clues yet. Any hints?
root@comp11:/etc# cat /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
auto em1
iface em1 inet static
address 192.168.0.101
netmask 255.255.255.0
network 192.168.0.0
broadcast 192.168.0.255
gateway 192.168.0.100
dns-nameservers 192.168.0.100
#=============CloudStack Network Bridges============
# Public network
auto cloudbr0
iface cloudbr0 inet manual
bridge_ports em1.200
bridge_fd 5
bridge_stp off
bridge_maxwait 1
# Private network
auto cloudbr1
iface cloudbr1 inet manual
bridge_ports em1.300
bridge_fd 5
bridge_stp off
bridge_maxwait 1
Cheers,
Dan
RE: Confusable default routing table on SSVM.
Posted by Vadim Kimlaychuk <Va...@Elion.ee>.
Hi Dan,
You can't blindly follow the instructions to be able to set up CS. Unfortunately even official documents are far from being perfect. In this particular case you should understand that different bridges defined at KVM host have different VLAN tags. And you must switch/untag them externally. Otherwise nets will not communicate with each other. For this you should use L2/L3 switch or some kind of virtual appliance. We use VyOS installed on PC with several NICs. Make sure that different network segments are visible at network level.
I am sure your problem is not host/CS configuration mismatch, but physical network setup. Try to understand CS networking first. This may help you: http://shapeblue.com/cloudstack/understanding-cloudstacks-physical-networking-architecture/ . It is also useful for basic zone setup because you need to be aware of tags anyway.
With XenServer you can set-up basic zone without L2/L3 switch because there is no such a requirement for public/private networks isolation. We were able to set-up everything inside 192.168.1.0/24 subnet. But this will not work for KVM hypervisor I believe.
Regards,
Vadim.
-----Original Message-----
From: Dan Dong [mailto:dongdan39@gmail.com]
Sent: Thursday, November 13, 2014 6:36 PM
To: users@cloudstack.apache.org
Subject: Re: Confusable default routing table on SSVM.
Thanks Vadim, I'm just following the basic zone setup guide, so I put the Management Server, the KVM host and all guest VMs in the same subnet of 192.168.0.0/24, but when system VMs are up and running on the KVM host and I logged into SSVM, it just could not ping outside and even the Management Server. One thing is that if I follow the instructions of:
http://docs.cloudstack.apache.org/projects/cloudstack-installation/en/latest/hypervisor/kvm.html
then the KVM network is down and so the KVM host could not be logged in by ssh anymore. Did anyone ever succeed with the following official settings for basic zone setups? Without settings for eth0 how could the KVM host be accessible(it only set eth0.100, cloudbr0, cloudbr1)? Did I miss anything?
auto lo
iface lo inet loopback
# The primary network interface
auto eth0.100
iface eth0.100 inet static
address 192.168.42.11
netmask 255.255.255.240
gateway 192.168.42.1
dns-nameservers 8.8.8.8 8.8.4.4
dns-domain lab.example.org
# Public network
auto cloudbr0
iface cloudbr0 inet manual
bridge_ports eth0.200
bridge_fd 5
bridge_stp off
bridge_maxwait 1
# Private network
auto cloudbr1
iface cloudbr1 inet manual
bridge_ports eth0.300
bridge_fd 5
bridge_stp off
bridge_maxwait 1
2014-11-12 1:46 GMT-06:00 Vadim Kimlaychuk <Va...@elion.ee>:
> Hello,
>
> It might happen if you have basic zone setup. We have
> implemented such configuration with XenServer, but it is confusable
> indeed. It is better to have differen LAN-s (VLANs) and partition the network properly.
> External L2/L3 switch is also a requirement.
>
> Vadim.
>
> -----Original Message-----
> From: Dan Dong [mailto:dongdan39@gmail.com]
> Sent: Wednesday, November 12, 2014 1:24 AM
> To: users@cloudstack.apache.org
> Subject: Re: Confusable default routing table on SSVM.
>
> Hi, All,
> Can experts examine the settings above if they are problematic?
> Could Management Server, KVM host and VMs all sit in the same
> 192.168.0.0/24 network? I checked a lot but have no idea, all
> firewalls have been turned off but still the SSVM could not contact Management server and the outside.
> Thanks a lot!
>
> Cheers,
> Dan
>
> 2014-11-11 10:29 GMT-06:00 Dan Dong <do...@gmail.com>:
>
> > Hi, All,
> > When debugging why my SSVM could not access outside, I see the
> > following route table on SSVM:
> > root@s-1-VM:~# ip route show
> > 169.254.0.0/16 dev eth0 proto kernel scope link src 169.254.1.47
> > 192.168.0.0/24 dev eth1 proto kernel scope link src 192.168.0.22
> > 192.168.0.0/24 dev eth2 proto kernel scope link src 192.168.0.134
> > 192.168.0.0/24 dev eth3 proto kernel scope link src 192.168.0.25
> >
> > There are 3 rules to reach the same subnet of 192.168.0.0/24 through
> > different NICs.
> > The problem is that SSVM could not ping both the KVM
> > host(192.168.0.101) on which the SSVM is running and the Management
> > Server(192.168.0.100, it is also set to be the gateway of the
> > 192.168.0.0/24 network), although they are all on the same subnet. I
> > suspect it is the bridge setings of the KVM, but no clues yet. Any hints?
> >
> > root@comp11:/etc# cat /etc/network/interfaces # This file describes
> > the network interfaces available on your system # and how to
> > activate them. For more information, see interfaces(5).
> >
> > # The loopback network interface
> > auto lo
> > iface lo inet loopback
> >
> > # The primary network interface
> > auto em1
> > iface em1 inet static
> > address 192.168.0.101
> > netmask 255.255.255.0
> > network 192.168.0.0
> > broadcast 192.168.0.255
> > gateway 192.168.0.100
> > dns-nameservers 192.168.0.100
> >
> > #=============CloudStack Network Bridges============ # Public
> > network auto cloudbr0 iface cloudbr0 inet manual
> > bridge_ports em1.200
> > bridge_fd 5
> > bridge_stp off
> > bridge_maxwait 1
> >
> > # Private network
> > auto cloudbr1
> > iface cloudbr1 inet manual
> > bridge_ports em1.300
> > bridge_fd 5
> > bridge_stp off
> > bridge_maxwait 1
> >
> >
> > Cheers,
> > Dan
> >
>
Re: Confusable default routing table on SSVM.
Posted by Dan Dong <do...@gmail.com>.
Thanks Vadim, I'm just following the basic zone setup guide, so I put the
Management Server, the KVM host and all guest VMs in the same subnet of
192.168.0.0/24, but when system VMs are up and running on the KVM host and
I logged into SSVM, it just could not ping outside and even the Management
Server. One thing is that if I follow the instructions of:
http://docs.cloudstack.apache.org/projects/cloudstack-installation/en/latest/hypervisor/kvm.html
then the KVM network is down and so the KVM host could not be logged in by
ssh anymore. Did anyone ever succeed with the following official settings
for basic zone setups? Without settings for eth0 how could the KVM host be
accessible(it only set eth0.100, cloudbr0, cloudbr1)? Did I miss anything?
auto lo
iface lo inet loopback
# The primary network interface
auto eth0.100
iface eth0.100 inet static
address 192.168.42.11
netmask 255.255.255.240
gateway 192.168.42.1
dns-nameservers 8.8.8.8 8.8.4.4
dns-domain lab.example.org
# Public network
auto cloudbr0
iface cloudbr0 inet manual
bridge_ports eth0.200
bridge_fd 5
bridge_stp off
bridge_maxwait 1
# Private network
auto cloudbr1
iface cloudbr1 inet manual
bridge_ports eth0.300
bridge_fd 5
bridge_stp off
bridge_maxwait 1
2014-11-12 1:46 GMT-06:00 Vadim Kimlaychuk <Va...@elion.ee>:
> Hello,
>
> It might happen if you have basic zone setup. We have implemented
> such configuration with XenServer, but it is confusable indeed. It is
> better to have differen LAN-s (VLANs) and partition the network properly.
> External L2/L3 switch is also a requirement.
>
> Vadim.
>
> -----Original Message-----
> From: Dan Dong [mailto:dongdan39@gmail.com]
> Sent: Wednesday, November 12, 2014 1:24 AM
> To: users@cloudstack.apache.org
> Subject: Re: Confusable default routing table on SSVM.
>
> Hi, All,
> Can experts examine the settings above if they are problematic? Could
> Management Server, KVM host and VMs all sit in the same 192.168.0.0/24
> network? I checked a lot but have no idea, all firewalls have been turned
> off but still the SSVM could not contact Management server and the outside.
> Thanks a lot!
>
> Cheers,
> Dan
>
> 2014-11-11 10:29 GMT-06:00 Dan Dong <do...@gmail.com>:
>
> > Hi, All,
> > When debugging why my SSVM could not access outside, I see the
> > following route table on SSVM:
> > root@s-1-VM:~# ip route show
> > 169.254.0.0/16 dev eth0 proto kernel scope link src 169.254.1.47
> > 192.168.0.0/24 dev eth1 proto kernel scope link src 192.168.0.22
> > 192.168.0.0/24 dev eth2 proto kernel scope link src 192.168.0.134
> > 192.168.0.0/24 dev eth3 proto kernel scope link src 192.168.0.25
> >
> > There are 3 rules to reach the same subnet of 192.168.0.0/24 through
> > different NICs.
> > The problem is that SSVM could not ping both the KVM
> > host(192.168.0.101) on which the SSVM is running and the Management
> > Server(192.168.0.100, it is also set to be the gateway of the
> > 192.168.0.0/24 network), although they are all on the same subnet. I
> > suspect it is the bridge setings of the KVM, but no clues yet. Any hints?
> >
> > root@comp11:/etc# cat /etc/network/interfaces # This file describes
> > the network interfaces available on your system # and how to activate
> > them. For more information, see interfaces(5).
> >
> > # The loopback network interface
> > auto lo
> > iface lo inet loopback
> >
> > # The primary network interface
> > auto em1
> > iface em1 inet static
> > address 192.168.0.101
> > netmask 255.255.255.0
> > network 192.168.0.0
> > broadcast 192.168.0.255
> > gateway 192.168.0.100
> > dns-nameservers 192.168.0.100
> >
> > #=============CloudStack Network Bridges============ # Public network
> > auto cloudbr0 iface cloudbr0 inet manual
> > bridge_ports em1.200
> > bridge_fd 5
> > bridge_stp off
> > bridge_maxwait 1
> >
> > # Private network
> > auto cloudbr1
> > iface cloudbr1 inet manual
> > bridge_ports em1.300
> > bridge_fd 5
> > bridge_stp off
> > bridge_maxwait 1
> >
> >
> > Cheers,
> > Dan
> >
>
RE: Confusable default routing table on SSVM.
Posted by Vadim Kimlaychuk <Va...@Elion.ee>.
Hello,
It might happen if you have basic zone setup. We have implemented such configuration with XenServer, but it is confusable indeed. It is better to have differen LAN-s (VLANs) and partition the network properly. External L2/L3 switch is also a requirement.
Vadim.
-----Original Message-----
From: Dan Dong [mailto:dongdan39@gmail.com]
Sent: Wednesday, November 12, 2014 1:24 AM
To: users@cloudstack.apache.org
Subject: Re: Confusable default routing table on SSVM.
Hi, All,
Can experts examine the settings above if they are problematic? Could Management Server, KVM host and VMs all sit in the same 192.168.0.0/24 network? I checked a lot but have no idea, all firewalls have been turned off but still the SSVM could not contact Management server and the outside.
Thanks a lot!
Cheers,
Dan
2014-11-11 10:29 GMT-06:00 Dan Dong <do...@gmail.com>:
> Hi, All,
> When debugging why my SSVM could not access outside, I see the
> following route table on SSVM:
> root@s-1-VM:~# ip route show
> 169.254.0.0/16 dev eth0 proto kernel scope link src 169.254.1.47
> 192.168.0.0/24 dev eth1 proto kernel scope link src 192.168.0.22
> 192.168.0.0/24 dev eth2 proto kernel scope link src 192.168.0.134
> 192.168.0.0/24 dev eth3 proto kernel scope link src 192.168.0.25
>
> There are 3 rules to reach the same subnet of 192.168.0.0/24 through
> different NICs.
> The problem is that SSVM could not ping both the KVM
> host(192.168.0.101) on which the SSVM is running and the Management
> Server(192.168.0.100, it is also set to be the gateway of the
> 192.168.0.0/24 network), although they are all on the same subnet. I
> suspect it is the bridge setings of the KVM, but no clues yet. Any hints?
>
> root@comp11:/etc# cat /etc/network/interfaces # This file describes
> the network interfaces available on your system # and how to activate
> them. For more information, see interfaces(5).
>
> # The loopback network interface
> auto lo
> iface lo inet loopback
>
> # The primary network interface
> auto em1
> iface em1 inet static
> address 192.168.0.101
> netmask 255.255.255.0
> network 192.168.0.0
> broadcast 192.168.0.255
> gateway 192.168.0.100
> dns-nameservers 192.168.0.100
>
> #=============CloudStack Network Bridges============ # Public network
> auto cloudbr0 iface cloudbr0 inet manual
> bridge_ports em1.200
> bridge_fd 5
> bridge_stp off
> bridge_maxwait 1
>
> # Private network
> auto cloudbr1
> iface cloudbr1 inet manual
> bridge_ports em1.300
> bridge_fd 5
> bridge_stp off
> bridge_maxwait 1
>
>
> Cheers,
> Dan
>
Re: Confusable default routing table on SSVM.
Posted by Dan Dong <do...@gmail.com>.
Hi, All,
Can experts examine the settings above if they are problematic? Could
Management Server, KVM host and VMs all sit in the same 192.168.0.0/24
network? I checked a lot but have no idea, all firewalls have been turned
off but still the SSVM could not contact Management server and the outside.
Thanks a lot!
Cheers,
Dan
2014-11-11 10:29 GMT-06:00 Dan Dong <do...@gmail.com>:
> Hi, All,
> When debugging why my SSVM could not access outside, I see the following
> route table on SSVM:
> root@s-1-VM:~# ip route show
> 169.254.0.0/16 dev eth0 proto kernel scope link src 169.254.1.47
> 192.168.0.0/24 dev eth1 proto kernel scope link src 192.168.0.22
> 192.168.0.0/24 dev eth2 proto kernel scope link src 192.168.0.134
> 192.168.0.0/24 dev eth3 proto kernel scope link src 192.168.0.25
>
> There are 3 rules to reach the same subnet of 192.168.0.0/24 through
> different NICs.
> The problem is that SSVM could not ping both the KVM host(192.168.0.101)
> on which the SSVM is running and the Management Server(192.168.0.100, it is
> also set to be the gateway of the 192.168.0.0/24 network), although they
> are all on the same subnet. I suspect it is the bridge setings of the KVM,
> but no clues yet. Any hints?
>
> root@comp11:/etc# cat /etc/network/interfaces
> # This file describes the network interfaces available on your system
> # and how to activate them. For more information, see interfaces(5).
>
> # The loopback network interface
> auto lo
> iface lo inet loopback
>
> # The primary network interface
> auto em1
> iface em1 inet static
> address 192.168.0.101
> netmask 255.255.255.0
> network 192.168.0.0
> broadcast 192.168.0.255
> gateway 192.168.0.100
> dns-nameservers 192.168.0.100
>
> #=============CloudStack Network Bridges============
> # Public network
> auto cloudbr0
> iface cloudbr0 inet manual
> bridge_ports em1.200
> bridge_fd 5
> bridge_stp off
> bridge_maxwait 1
>
> # Private network
> auto cloudbr1
> iface cloudbr1 inet manual
> bridge_ports em1.300
> bridge_fd 5
> bridge_stp off
> bridge_maxwait 1
>
>
> Cheers,
> Dan
>
Re: Confusable default routing table on SSVM.
Posted by Gopalakrishnan S <go...@assistanz.com>.
Hi Dan,
It may caused your KVM host bridge settings. Please make sure its working
fine using this command.
[root@kvm ~]# brctl show
Also check the KVM host /etc/cloudstack/agent/agent.properties file network
segregation settings.
guest.network.device=cloudbr0
private.network.device=cloudbr1
Thank You.
Gopalakrishnan.S
----- Original Message -----
From: "Dan Dong" <do...@gmail.com>
To: <us...@cloudstack.apache.org>
Sent: Tuesday, November 11, 2014 9:59 PM
Subject: Confusable default routing table on SSVM.
> Hi, All,
> When debugging why my SSVM could not access outside, I see the following
> route table on SSVM:
> root@s-1-VM:~# ip route show
> 169.254.0.0/16 dev eth0 proto kernel scope link src 169.254.1.47
> 192.168.0.0/24 dev eth1 proto kernel scope link src 192.168.0.22
> 192.168.0.0/24 dev eth2 proto kernel scope link src 192.168.0.134
> 192.168.0.0/24 dev eth3 proto kernel scope link src 192.168.0.25
>
> There are 3 rules to reach the same subnet of 192.168.0.0/24 through
> different NICs.
> The problem is that SSVM could not ping both the KVM host(192.168.0.101)
> on
> which the SSVM is running and the Management Server(192.168.0.100, it is
> also set to be the gateway of the 192.168.0.0/24 network), although they
> are all on the same subnet. I suspect it is the bridge setings of the KVM,
> but no clues yet. Any hints?
>
> root@comp11:/etc# cat /etc/network/interfaces
> # This file describes the network interfaces available on your system
> # and how to activate them. For more information, see interfaces(5).
>
> # The loopback network interface
> auto lo
> iface lo inet loopback
>
> # The primary network interface
> auto em1
> iface em1 inet static
> address 192.168.0.101
> netmask 255.255.255.0
> network 192.168.0.0
> broadcast 192.168.0.255
> gateway 192.168.0.100
> dns-nameservers 192.168.0.100
>
> #=============CloudStack Network Bridges============
> # Public network
> auto cloudbr0
> iface cloudbr0 inet manual
> bridge_ports em1.200
> bridge_fd 5
> bridge_stp off
> bridge_maxwait 1
>
> # Private network
> auto cloudbr1
> iface cloudbr1 inet manual
> bridge_ports em1.300
> bridge_fd 5
> bridge_stp off
> bridge_maxwait 1
>
>
> Cheers,
> Dan
>