You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Dan Dong <do...@gmail.com> on 2014/11/17 18:02:08 UTC
To let SSVM reach outside network.
Hi, All,
I found I could not register my ISO image to the cloudstack( through URL
of http://releases.ubuntu.com/14.04.1/ubuntu-14.04.1-server-amd64.iso). The
SSVM is running and health check show no problems on it. But it just could
not see the outside network, although SSVM is on the same network as KVM
host, i.e. 192.168.0.*. My setup is as following:
Management Server external NIC:10.*
Management Server internal NIC: 192.168.0.100
KVM host NIC: 192.168.0.101
guest VMs are all in 192.168.0.*/24 network.
>From SSVM I can ping the internal IP of the Management Server at
192.168.0.100, but could not ping the external IP of it at 10.*.
>From KVM host itself I can reach the outside internet as NAT is configured
on the Management Server to let the 192.168.0.0 traffic through. Any hints
how to let SSVM to reach the internet?
Cheers,
Dan
RE: To let SSVM reach outside network.
Posted by Vadim Kimlaychuk <Va...@Elion.ee>.
Hello Dan,
I have done simple networking for XenServer and it worked well. All guest and management IP-s belonged to 192.168.1.0/24 network and public net in my case was 10.0.0.0/24. If you don't have external switch/router that can be programmed to handle different nets on different ports than you probably need to configure tagging and trunk-ing at your single interface. Not good option indeed. Try to get something similar to Juniper SRX 100 or get another PC with at least 2 NICs to set up VyOS. We have such router for advanced networking and were satisfied with it.
What do you need is properly define your subnets where guest and public traffic do not overlap. I think it is better to assign them different network classes, but this is not a requirement. Look closer here: https://cwiki.apache.org/confluence/display/CLOUDSTACK/SSVM,+templates,+Secondary+storage+troubleshooting Each interface at SSVM has specific puropose. You always know wich NIC should connect to wich network and this information SSVM gets from the database. Read the SSVM troubleshooting guide carefully. Re-create your zone with proper subnets. You may have to do it several times, so you can backup you database before test and restore in a case of failure.
Of course before configuring zone at Cloudstack you should physically separate nets on network level and it must work before configuration.
Regards,
Vadim.
-----Original Message-----
From: Dan Dong [mailto:dongdan39@gmail.com]
Sent: Tuesday, November 25, 2014 5:54 PM
To: users@cloudstack.apache.org
Subject: Re: To let SSVM reach outside network.
Hi, Vadim,
Thanks a lot and that's a bit clear now, but still confused about how to partition the network, the four NICs of SSVM are generated automatically by cloudstack, so how do you know which subnet is matched to which NIC?
According to what principle should the partition be made( 4 subnet altogether?)? There's only one NIC used on the KVM host with IP addreess
192.168.0.101， do you mean to create eth0:0, eth0:1 etc and assign different IPs to them? The setup of my cloudstack is quite simple, I'm not testing any advanced or complicated experiment on it, so I'd just like to find a direct and simple way to make cloudstack up and running.
root@s-1-VM:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
0.0.0.0 192.168.0.100 0.0.0.0 UG 0 0 0
eth2
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth3
Cheers,
Dan
2014-11-25 2:01 GMT-06:00 Vadim Kimlaychuk <Va...@elion.ee>:
> Dan,
>
> Cloudstack guides are not perfect indeed and general rule of
> thumb
> - not to blind copy instructions. They are not wrong, but there are
> sometimes assumptions hided behind the scene. One of them -- you
> can't use plain network for everything. You must partition it.
>
> If you wish to stick to nearly the same configuration try to
> split you 192.168.0.0/24 network into smaller pieces like /27 or /28 subnets.
> Which size and how - decide by yourself. Your final routing table at
> SSVM should contain different interfaces connected to different net-s
> based on network mask. Of course you should enable inter-net
> connections, but since they are children of the same super-CIDR it should work automatically.
> Re-partition your network and copy-paste new routing table if you
> still have difficulties with it.
>
> Regards,
>
> Vadim.
>
>
> -----Original Message-----
> From: Dan Dong [mailto:dongdan39@gmail.com]
> Sent: Tuesday, November 25, 2014 12:00 AM
> To: users@cloudstack.apache.org
> Subject: Re: To let SSVM reach outside network.
>
> Hi, Vadim,
> Thanks for the hints. Yes, the route table looks weird, but this
> comes from the official guide of network set up of cloudstack:
>
> http://docs.cloudstack.apache.org/projects/cloudstack-installation/en/
> latest/hypervisor/kvm.html
>
> So I confused if the network pattern from the official guide is
> misleading or one should create VLANs one by one manually and set
> different IP address for each of them explicitly(eth0.100,eth0.200 and
> eth0.300)? Note that my network is nothing special, just put the
> Management Server, the KVM hypervisor and guest VMs all in the
> 192.168.0.* network. Could you suggest an simple example of how the
> file "/etc/network/interfaces" on KVM host looks like? The official pattern is as below but certainly does not work:
> ===================
>
> auto lo
> iface lo inet loopback
> # The primary network interface
> auto eth0.100
> iface eth0.100 inet static
> address 192.168.42.11
> netmask 255.255.255.240
> gateway 192.168.42.1
> dns-nameservers 8.8.8.8 8.8.4.4
> dns-domain lab.example.org
> # Public network
> auto cloudbr0
> iface cloudbr0 inet manual
> bridge_ports eth0.200
> bridge_fd 5
> bridge_stp off
> bridge_maxwait 1
> # Private network
> auto cloudbr1
> iface cloudbr1 inet manual
> bridge_ports eth0.300
> bridge_fd 5
> bridge_stp off
> bridge_maxwait 1
>
> ====================
>
> Cheers,
> Dan
>
>
>
>
>
> 2014-11-23 7:51 GMT-06:00 Vadim Kimlaychuk <Va...@elion.ee>:
>
> > Hello Dan,
> >
> > Sorry for the late reply, have to be out from internet
> > sometimes
> > :)
> > You have problem with routing. According to arp table
> > default interface -- eth2 it is not connected to you 192.168.0.*
> > network and can't see gateway (192.168.0.100) what is not ture. Arp
> > table looks ugly -- you have 3 interfaces connected to the same
> > network with same Metric. How do you think VM will choose the
> > correct one?? I suppose it will take the first one in line -- eth1
> > and that is why you see
> > eth1 with complete MAC for 192.168.0.100. It always uses eth1 to
> > reach other networks, but default gw is eth2.
> > For me, network layout looks awkward. It is better to
> > re-partition and re-think it. If you don't want to -- try to fix
> > routing table manually. Put metrics to your routes and eth2 should
> > be the highest (i.e. have lowest number). This will temporarty
> > solve the problem, but you still have a chance to loose your changes
> > after reboot, because routing table is created dynamically.
> >
> > Fix you network and I believe everything will work out-of-the box.
> >
> > Regards,
> >
> > Vadim.
> >
> > ________________________________________
> > From: Dan Dong [dongdan39@gmail.com]
> > Sent: Wednesday, November 19, 2014 17:45
> > To: users@cloudstack.apache.org
> > Subject: Re: To let SSVM reach outside network.
> >
> > Hi, Vadim,
> > 1. route -n
> > root@s-1-VM:~# route -n
> > Kernel IP routing table
> > Destination Gateway Genmask Flags Metric Ref Use
> > Iface
> > 0.0.0.0 192.168.0.100 0.0.0.0 UG 0 0 0
> > eth2
> > 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0
> > eth0
> > 192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0
> > eth1
> > 192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0
> > eth2
> > 192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0
> > eth3
> >
> > Noted that to the same 192.168.0.0 network, eth1 is searched before
> > eth2, while eth2 is supposed to be the public NIC. Should one change
> > the
> order?
> >
> > 2. root@s-1-VM:~# arp -n
> > cloud (192.168.0.100) at 84:2b:2b:01:c3:d0 [ether] on eth1 cloud
> > (192.168.0.100) at <incomplete> on eth2 ? (169.254.0.1) at
> > fe:00:a9:fe:01:7a [ether] on eth0
> >
> > Noted that MAC is incomplete on eth2, while MAC detected on eth1 is
> > the correct address of the internal NIC of the Management Server(the
> > gateway for 192.168.0.0 subnet).
> >
> > 3. The network layout is quite simple here( basic network):
> > Management Server external NIC:10.*
> > Management Server internal NIC: 192.168.0.100
> >
> > KVM host NIC: 192.168.0.101
> > SSVM and guest VMs are all in 192.168.0.*/24 network.
> >
> > 4. NAT is configured on the Management Server so 192.168.0.0/24
> > subnet could reach the internet(download packages from internet
> > etc., e.g, from KVM host itself)
> >
> > Cheers,
> > Dan
> >
> >
> > 2014-11-19 1:23 GMT-06:00 Vadim Kimlaychuk <Va...@elion.ee>:
> >
> > > Dan,
> > >
> > > I would suggest you to use external proxy/GW to hide you network.
> > > Not the same host that contains management server. Because if you
> > > would like to scale it up - how do you do it?? Management server
> > > should be rather
> > simple
> > > component of infrastructure and to put more functions on it is a
> > > bad
> > idea.
> > >
> > > Anyway
> > > 1. what does "route -n" says on your SSVM ??
> > > 2. If you use KVM your bridges tag the traffic. Are you sure that
> > > it is properly handled on your switch?? Run "arp -a" at your SSVM
> > > and see if VM "knowns" MAC of the gateway.
> > >
> > > I do expect you put the output of route and arp here otherwise we
> > > will continue guessing.
> > >
> > > It would be even better if you describe your network layout like
> > > on the first picture here:
> > >
> > http://cloudstack-administration.readthedocs.org/en/latest/networkin
> > g_
> > and_traffic.html
> > >
> > > Vadim.
> > >
> > > -----Original Message-----
> > > From: Dan Dong [mailto:dongdan39@gmail.com]
> > > Sent: Tuesday, November 18, 2014 5:23 PM
> > > To: users@cloudstack.apache.org
> > > Subject: Re: To let SSVM reach outside network.
> > >
> > > Hi, Vadim,
> > > We have to use 2 NICs on the management server as we want to
> > > hide the cloudstack cluster behind the 10.* network, so all KVM
> > > hosts and guest
> > VMs
> > > are in the 192.168.0.* subnet, and they connect to the management
> > server's
> > > internal NIC(192.168.0.100). Is it a rule that the management
> > > server can only use one NIC? And the KVM hypersior host can reach
> > > the internet and download packages, but SSVM running on it could
> > > not see
> the internet.
> > >
> > > Cheers,
> > > Dan
> > >
> > >
> > > 2014-11-18 3:20 GMT-06:00 Vadim Kimlaychuk
> > > <Vadim.Kimlaychuk@elion.ee
> >:
> > >
> > > > Hello Dan,
> > > >
> > > > It seems there is something wrong with your network
> > > > setup and here are some places to search:
> > > > 1. Why your management server has 2 NICs? It should not work as
> > > > NAT, proxy or any kind of switch - keep this in mind.
> > > > 2. SSVM normally has to have 3 interfaces (at least). One --
> > > > with public IP, one -- management network IP and one -- link local IP.
> > > > If you have separate storage network -- it may have one more,
> > > > but this is not your case. Check routing table for your SSVM
> > > > with "route
> -n"
> > > > command. Find your default gateway. It should be public interface.
> > > > 3. Did you set up KVM hypervisor network correctly? Does it have
> > > > 2 interfaces like in the setup guide? Does your hypervisor have
> > > > access to internet to be able to download template?
> > > >
> > > > Vadim.
> > > >
> > > > -----Original Message-----
> > > > From: Dan Dong [mailto:dongdan39@gmail.com]
> > > > Sent: Monday, November 17, 2014 7:02 PM
> > > > To: users@cloudstack.apache.org
> > > > Subject: To let SSVM reach outside network.
> > > >
> > > > Hi, All,
> > > > I found I could not register my ISO image to the cloudstack(
> > > > through URL of
> > > http://releases.ubuntu.com/14.04.1/ubuntu-14.04.1-server-amd64.iso).
> > > > The SSVM is running and health check show no problems on it. But
> > > > it just could not see the outside network, although SSVM is on
> > > > the same network as KVM host, i.e. 192.168.0.*. My setup is as following:
> > > >
> > > > Management Server external NIC:10.* Management Server internal
> > > > NIC: 192.168.0.100
> > > >
> > > > KVM host NIC: 192.168.0.101
> > > > guest VMs are all in 192.168.0.*/24 network.
> > > >
> > > > From SSVM I can ping the internal IP of the Management Server at
> > > > 192.168.0.100, but could not ping the external IP of it at 10.*.
> > > >
> > > > From KVM host itself I can reach the outside internet as NAT is
> > > > configured on the Management Server to let the 192.168.0.0
> > > > traffic through. Any hints how to let SSVM to reach the internet?
> > > >
> > > >
> > > > Cheers,
> > > > Dan
> > > >
> > >
> >
>
Re: To let SSVM reach outside network.
Posted by Dan Dong <do...@gmail.com>.
Hi, Vadim,
Thanks a lot and that's a bit clear now, but still confused about how to
partition the network, the four NICs of SSVM are generated automatically by
cloudstack, so how do you know which subnet is matched to which NIC?
According to what principle should the partition be made( 4 subnet
altogether?)? There's only one NIC used on the KVM host with IP addreess
192.168.0.101， do you mean to create eth0:0, eth0:1 etc and assign
different IPs to them? The setup of my cloudstack is quite simple, I'm not
testing any advanced or complicated experiment on it, so I'd just like to
find a direct and simple way to make cloudstack up and running.
root@s-1-VM:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
0.0.0.0 192.168.0.100 0.0.0.0 UG 0 0 0
eth2
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth3
Cheers,
Dan
2014-11-25 2:01 GMT-06:00 Vadim Kimlaychuk <Va...@elion.ee>:
> Dan,
>
> Cloudstack guides are not perfect indeed and general rule of thumb
> - not to blind copy instructions. They are not wrong, but there are
> sometimes assumptions hided behind the scene. One of them -- you can't use
> plain network for everything. You must partition it.
>
> If you wish to stick to nearly the same configuration try to split
> you 192.168.0.0/24 network into smaller pieces like /27 or /28 subnets.
> Which size and how - decide by yourself. Your final routing table at SSVM
> should contain different interfaces connected to different net-s based on
> network mask. Of course you should enable inter-net connections, but since
> they are children of the same super-CIDR it should work automatically.
> Re-partition your network and copy-paste new routing table if you still
> have difficulties with it.
>
> Regards,
>
> Vadim.
>
>
> -----Original Message-----
> From: Dan Dong [mailto:dongdan39@gmail.com]
> Sent: Tuesday, November 25, 2014 12:00 AM
> To: users@cloudstack.apache.org
> Subject: Re: To let SSVM reach outside network.
>
> Hi, Vadim,
> Thanks for the hints. Yes, the route table looks weird, but this comes
> from the official guide of network set up of cloudstack:
>
> http://docs.cloudstack.apache.org/projects/cloudstack-installation/en/latest/hypervisor/kvm.html
>
> So I confused if the network pattern from the official guide is misleading
> or one should create VLANs one by one manually and set different IP address
> for each of them explicitly(eth0.100,eth0.200 and eth0.300)? Note that my
> network is nothing special, just put the Management Server, the KVM
> hypervisor and guest VMs all in the 192.168.0.* network. Could you suggest
> an simple example of how the file "/etc/network/interfaces" on KVM host
> looks like? The official pattern is as below but certainly does not work:
> ===================
>
> auto lo
> iface lo inet loopback
> # The primary network interface
> auto eth0.100
> iface eth0.100 inet static
> address 192.168.42.11
> netmask 255.255.255.240
> gateway 192.168.42.1
> dns-nameservers 8.8.8.8 8.8.4.4
> dns-domain lab.example.org
> # Public network
> auto cloudbr0
> iface cloudbr0 inet manual
> bridge_ports eth0.200
> bridge_fd 5
> bridge_stp off
> bridge_maxwait 1
> # Private network
> auto cloudbr1
> iface cloudbr1 inet manual
> bridge_ports eth0.300
> bridge_fd 5
> bridge_stp off
> bridge_maxwait 1
>
> ====================
>
> Cheers,
> Dan
>
>
>
>
>
> 2014-11-23 7:51 GMT-06:00 Vadim Kimlaychuk <Va...@elion.ee>:
>
> > Hello Dan,
> >
> > Sorry for the late reply, have to be out from internet
> > sometimes
> > :)
> > You have problem with routing. According to arp table default
> > interface -- eth2 it is not connected to you 192.168.0.* network and
> > can't see gateway (192.168.0.100) what is not ture. Arp table looks
> > ugly -- you have 3 interfaces connected to the same network with same
> > Metric. How do you think VM will choose the correct one?? I suppose
> > it will take the first one in line -- eth1 and that is why you see
> > eth1 with complete MAC for 192.168.0.100. It always uses eth1 to reach
> > other networks, but default gw is eth2.
> > For me, network layout looks awkward. It is better to
> > re-partition and re-think it. If you don't want to -- try to fix
> > routing table manually. Put metrics to your routes and eth2 should be
> > the highest (i.e. have lowest number). This will temporarty solve the
> > problem, but you still have a chance to loose your changes after
> > reboot, because routing table is created dynamically.
> >
> > Fix you network and I believe everything will work out-of-the box.
> >
> > Regards,
> >
> > Vadim.
> >
> > ________________________________________
> > From: Dan Dong [dongdan39@gmail.com]
> > Sent: Wednesday, November 19, 2014 17:45
> > To: users@cloudstack.apache.org
> > Subject: Re: To let SSVM reach outside network.
> >
> > Hi, Vadim,
> > 1. route -n
> > root@s-1-VM:~# route -n
> > Kernel IP routing table
> > Destination Gateway Genmask Flags Metric Ref Use
> > Iface
> > 0.0.0.0 192.168.0.100 0.0.0.0 UG 0 0 0
> > eth2
> > 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0
> > eth0
> > 192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0
> > eth1
> > 192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0
> > eth2
> > 192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0
> > eth3
> >
> > Noted that to the same 192.168.0.0 network, eth1 is searched before
> > eth2, while eth2 is supposed to be the public NIC. Should one change the
> order?
> >
> > 2. root@s-1-VM:~# arp -n
> > cloud (192.168.0.100) at 84:2b:2b:01:c3:d0 [ether] on eth1 cloud
> > (192.168.0.100) at <incomplete> on eth2 ? (169.254.0.1) at
> > fe:00:a9:fe:01:7a [ether] on eth0
> >
> > Noted that MAC is incomplete on eth2, while MAC detected on eth1 is
> > the correct address of the internal NIC of the Management Server(the
> > gateway for 192.168.0.0 subnet).
> >
> > 3. The network layout is quite simple here( basic network):
> > Management Server external NIC:10.*
> > Management Server internal NIC: 192.168.0.100
> >
> > KVM host NIC: 192.168.0.101
> > SSVM and guest VMs are all in 192.168.0.*/24 network.
> >
> > 4. NAT is configured on the Management Server so 192.168.0.0/24 subnet
> > could reach the internet(download packages from internet etc., e.g,
> > from KVM host itself)
> >
> > Cheers,
> > Dan
> >
> >
> > 2014-11-19 1:23 GMT-06:00 Vadim Kimlaychuk <Va...@elion.ee>:
> >
> > > Dan,
> > >
> > > I would suggest you to use external proxy/GW to hide you network.
> > > Not the same host that contains management server. Because if you
> > > would like to scale it up - how do you do it?? Management server
> > > should be rather
> > simple
> > > component of infrastructure and to put more functions on it is a bad
> > idea.
> > >
> > > Anyway
> > > 1. what does "route -n" says on your SSVM ??
> > > 2. If you use KVM your bridges tag the traffic. Are you sure that it
> > > is properly handled on your switch?? Run "arp -a" at your SSVM and
> > > see if VM "knowns" MAC of the gateway.
> > >
> > > I do expect you put the output of route and arp here otherwise we
> > > will continue guessing.
> > >
> > > It would be even better if you describe your network layout like on
> > > the first picture here:
> > >
> > http://cloudstack-administration.readthedocs.org/en/latest/networking_
> > and_traffic.html
> > >
> > > Vadim.
> > >
> > > -----Original Message-----
> > > From: Dan Dong [mailto:dongdan39@gmail.com]
> > > Sent: Tuesday, November 18, 2014 5:23 PM
> > > To: users@cloudstack.apache.org
> > > Subject: Re: To let SSVM reach outside network.
> > >
> > > Hi, Vadim,
> > > We have to use 2 NICs on the management server as we want to hide
> > > the cloudstack cluster behind the 10.* network, so all KVM hosts and
> > > guest
> > VMs
> > > are in the 192.168.0.* subnet, and they connect to the management
> > server's
> > > internal NIC(192.168.0.100). Is it a rule that the management server
> > > can only use one NIC? And the KVM hypersior host can reach the
> > > internet and download packages, but SSVM running on it could not see
> the internet.
> > >
> > > Cheers,
> > > Dan
> > >
> > >
> > > 2014-11-18 3:20 GMT-06:00 Vadim Kimlaychuk <Vadim.Kimlaychuk@elion.ee
> >:
> > >
> > > > Hello Dan,
> > > >
> > > > It seems there is something wrong with your network setup
> > > > and here are some places to search:
> > > > 1. Why your management server has 2 NICs? It should not work as
> > > > NAT, proxy or any kind of switch - keep this in mind.
> > > > 2. SSVM normally has to have 3 interfaces (at least). One -- with
> > > > public IP, one -- management network IP and one -- link local IP.
> > > > If you have separate storage network -- it may have one more, but
> > > > this is not your case. Check routing table for your SSVM with "route
> -n"
> > > > command. Find your default gateway. It should be public interface.
> > > > 3. Did you set up KVM hypervisor network correctly? Does it have 2
> > > > interfaces like in the setup guide? Does your hypervisor have
> > > > access to internet to be able to download template?
> > > >
> > > > Vadim.
> > > >
> > > > -----Original Message-----
> > > > From: Dan Dong [mailto:dongdan39@gmail.com]
> > > > Sent: Monday, November 17, 2014 7:02 PM
> > > > To: users@cloudstack.apache.org
> > > > Subject: To let SSVM reach outside network.
> > > >
> > > > Hi, All,
> > > > I found I could not register my ISO image to the cloudstack(
> > > > through URL of
> > > http://releases.ubuntu.com/14.04.1/ubuntu-14.04.1-server-amd64.iso).
> > > > The SSVM is running and health check show no problems on it. But
> > > > it just could not see the outside network, although SSVM is on the
> > > > same network as KVM host, i.e. 192.168.0.*. My setup is as following:
> > > >
> > > > Management Server external NIC:10.* Management Server internal
> > > > NIC: 192.168.0.100
> > > >
> > > > KVM host NIC: 192.168.0.101
> > > > guest VMs are all in 192.168.0.*/24 network.
> > > >
> > > > From SSVM I can ping the internal IP of the Management Server at
> > > > 192.168.0.100, but could not ping the external IP of it at 10.*.
> > > >
> > > > From KVM host itself I can reach the outside internet as NAT is
> > > > configured on the Management Server to let the 192.168.0.0 traffic
> > > > through. Any hints how to let SSVM to reach the internet?
> > > >
> > > >
> > > > Cheers,
> > > > Dan
> > > >
> > >
> >
>
RE: To let SSVM reach outside network.
Posted by Vadim Kimlaychuk <Va...@Elion.ee>.
Dan,
Cloudstack guides are not perfect indeed and general rule of thumb - not to blind copy instructions. They are not wrong, but there are sometimes assumptions hided behind the scene. One of them -- you can't use plain network for everything. You must partition it.
If you wish to stick to nearly the same configuration try to split you 192.168.0.0/24 network into smaller pieces like /27 or /28 subnets. Which size and how - decide by yourself. Your final routing table at SSVM should contain different interfaces connected to different net-s based on network mask. Of course you should enable inter-net connections, but since they are children of the same super-CIDR it should work automatically. Re-partition your network and copy-paste new routing table if you still have difficulties with it.
Regards,
Vadim.
-----Original Message-----
From: Dan Dong [mailto:dongdan39@gmail.com]
Sent: Tuesday, November 25, 2014 12:00 AM
To: users@cloudstack.apache.org
Subject: Re: To let SSVM reach outside network.
Hi, Vadim,
Thanks for the hints. Yes, the route table looks weird, but this comes from the official guide of network set up of cloudstack:
http://docs.cloudstack.apache.org/projects/cloudstack-installation/en/latest/hypervisor/kvm.html
So I confused if the network pattern from the official guide is misleading or one should create VLANs one by one manually and set different IP address for each of them explicitly(eth0.100,eth0.200 and eth0.300)? Note that my network is nothing special, just put the Management Server, the KVM hypervisor and guest VMs all in the 192.168.0.* network. Could you suggest an simple example of how the file "/etc/network/interfaces" on KVM host looks like? The official pattern is as below but certainly does not work:
===================
auto lo
iface lo inet loopback
# The primary network interface
auto eth0.100
iface eth0.100 inet static
address 192.168.42.11
netmask 255.255.255.240
gateway 192.168.42.1
dns-nameservers 8.8.8.8 8.8.4.4
dns-domain lab.example.org
# Public network
auto cloudbr0
iface cloudbr0 inet manual
bridge_ports eth0.200
bridge_fd 5
bridge_stp off
bridge_maxwait 1
# Private network
auto cloudbr1
iface cloudbr1 inet manual
bridge_ports eth0.300
bridge_fd 5
bridge_stp off
bridge_maxwait 1
====================
Cheers,
Dan
2014-11-23 7:51 GMT-06:00 Vadim Kimlaychuk <Va...@elion.ee>:
> Hello Dan,
>
> Sorry for the late reply, have to be out from internet
> sometimes
> :)
> You have problem with routing. According to arp table default
> interface -- eth2 it is not connected to you 192.168.0.* network and
> can't see gateway (192.168.0.100) what is not ture. Arp table looks
> ugly -- you have 3 interfaces connected to the same network with same
> Metric. How do you think VM will choose the correct one?? I suppose
> it will take the first one in line -- eth1 and that is why you see
> eth1 with complete MAC for 192.168.0.100. It always uses eth1 to reach
> other networks, but default gw is eth2.
> For me, network layout looks awkward. It is better to
> re-partition and re-think it. If you don't want to -- try to fix
> routing table manually. Put metrics to your routes and eth2 should be
> the highest (i.e. have lowest number). This will temporarty solve the
> problem, but you still have a chance to loose your changes after
> reboot, because routing table is created dynamically.
>
> Fix you network and I believe everything will work out-of-the box.
>
> Regards,
>
> Vadim.
>
> ________________________________________
> From: Dan Dong [dongdan39@gmail.com]
> Sent: Wednesday, November 19, 2014 17:45
> To: users@cloudstack.apache.org
> Subject: Re: To let SSVM reach outside network.
>
> Hi, Vadim,
> 1. route -n
> root@s-1-VM:~# route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use
> Iface
> 0.0.0.0 192.168.0.100 0.0.0.0 UG 0 0 0
> eth2
> 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0
> eth0
> 192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0
> eth1
> 192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0
> eth2
> 192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0
> eth3
>
> Noted that to the same 192.168.0.0 network, eth1 is searched before
> eth2, while eth2 is supposed to be the public NIC. Should one change the order?
>
> 2. root@s-1-VM:~# arp -n
> cloud (192.168.0.100) at 84:2b:2b:01:c3:d0 [ether] on eth1 cloud
> (192.168.0.100) at <incomplete> on eth2 ? (169.254.0.1) at
> fe:00:a9:fe:01:7a [ether] on eth0
>
> Noted that MAC is incomplete on eth2, while MAC detected on eth1 is
> the correct address of the internal NIC of the Management Server(the
> gateway for 192.168.0.0 subnet).
>
> 3. The network layout is quite simple here( basic network):
> Management Server external NIC:10.*
> Management Server internal NIC: 192.168.0.100
>
> KVM host NIC: 192.168.0.101
> SSVM and guest VMs are all in 192.168.0.*/24 network.
>
> 4. NAT is configured on the Management Server so 192.168.0.0/24 subnet
> could reach the internet(download packages from internet etc., e.g,
> from KVM host itself)
>
> Cheers,
> Dan
>
>
> 2014-11-19 1:23 GMT-06:00 Vadim Kimlaychuk <Va...@elion.ee>:
>
> > Dan,
> >
> > I would suggest you to use external proxy/GW to hide you network.
> > Not the same host that contains management server. Because if you
> > would like to scale it up - how do you do it?? Management server
> > should be rather
> simple
> > component of infrastructure and to put more functions on it is a bad
> idea.
> >
> > Anyway
> > 1. what does "route -n" says on your SSVM ??
> > 2. If you use KVM your bridges tag the traffic. Are you sure that it
> > is properly handled on your switch?? Run "arp -a" at your SSVM and
> > see if VM "knowns" MAC of the gateway.
> >
> > I do expect you put the output of route and arp here otherwise we
> > will continue guessing.
> >
> > It would be even better if you describe your network layout like on
> > the first picture here:
> >
> http://cloudstack-administration.readthedocs.org/en/latest/networking_
> and_traffic.html
> >
> > Vadim.
> >
> > -----Original Message-----
> > From: Dan Dong [mailto:dongdan39@gmail.com]
> > Sent: Tuesday, November 18, 2014 5:23 PM
> > To: users@cloudstack.apache.org
> > Subject: Re: To let SSVM reach outside network.
> >
> > Hi, Vadim,
> > We have to use 2 NICs on the management server as we want to hide
> > the cloudstack cluster behind the 10.* network, so all KVM hosts and
> > guest
> VMs
> > are in the 192.168.0.* subnet, and they connect to the management
> server's
> > internal NIC(192.168.0.100). Is it a rule that the management server
> > can only use one NIC? And the KVM hypersior host can reach the
> > internet and download packages, but SSVM running on it could not see the internet.
> >
> > Cheers,
> > Dan
> >
> >
> > 2014-11-18 3:20 GMT-06:00 Vadim Kimlaychuk <Va...@elion.ee>:
> >
> > > Hello Dan,
> > >
> > > It seems there is something wrong with your network setup
> > > and here are some places to search:
> > > 1. Why your management server has 2 NICs? It should not work as
> > > NAT, proxy or any kind of switch - keep this in mind.
> > > 2. SSVM normally has to have 3 interfaces (at least). One -- with
> > > public IP, one -- management network IP and one -- link local IP.
> > > If you have separate storage network -- it may have one more, but
> > > this is not your case. Check routing table for your SSVM with "route -n"
> > > command. Find your default gateway. It should be public interface.
> > > 3. Did you set up KVM hypervisor network correctly? Does it have 2
> > > interfaces like in the setup guide? Does your hypervisor have
> > > access to internet to be able to download template?
> > >
> > > Vadim.
> > >
> > > -----Original Message-----
> > > From: Dan Dong [mailto:dongdan39@gmail.com]
> > > Sent: Monday, November 17, 2014 7:02 PM
> > > To: users@cloudstack.apache.org
> > > Subject: To let SSVM reach outside network.
> > >
> > > Hi, All,
> > > I found I could not register my ISO image to the cloudstack(
> > > through URL of
> > http://releases.ubuntu.com/14.04.1/ubuntu-14.04.1-server-amd64.iso).
> > > The SSVM is running and health check show no problems on it. But
> > > it just could not see the outside network, although SSVM is on the
> > > same network as KVM host, i.e. 192.168.0.*. My setup is as following:
> > >
> > > Management Server external NIC:10.* Management Server internal
> > > NIC: 192.168.0.100
> > >
> > > KVM host NIC: 192.168.0.101
> > > guest VMs are all in 192.168.0.*/24 network.
> > >
> > > From SSVM I can ping the internal IP of the Management Server at
> > > 192.168.0.100, but could not ping the external IP of it at 10.*.
> > >
> > > From KVM host itself I can reach the outside internet as NAT is
> > > configured on the Management Server to let the 192.168.0.0 traffic
> > > through. Any hints how to let SSVM to reach the internet?
> > >
> > >
> > > Cheers,
> > > Dan
> > >
> >
>
Re: To let SSVM reach outside network.
Posted by Dan Dong <do...@gmail.com>.
Hi, Vadim,
Thanks for the hints. Yes, the route table looks weird, but this comes
from the official guide of network set up of cloudstack:
http://docs.cloudstack.apache.org/projects/cloudstack-installation/en/latest/hypervisor/kvm.html
So I confused if the network pattern from the official guide is misleading
or one should create VLANs one by one manually and set different IP address
for each of them explicitly(eth0.100,eth0.200 and eth0.300)? Note that my
network is nothing special, just put the Management Server, the KVM
hypervisor and guest VMs all in the 192.168.0.* network. Could you suggest
an simple example of how the file "/etc/network/interfaces" on KVM host
looks like? The official pattern is as below but certainly does not work:
===================
auto lo
iface lo inet loopback
# The primary network interface
auto eth0.100
iface eth0.100 inet static
address 192.168.42.11
netmask 255.255.255.240
gateway 192.168.42.1
dns-nameservers 8.8.8.8 8.8.4.4
dns-domain lab.example.org
# Public network
auto cloudbr0
iface cloudbr0 inet manual
bridge_ports eth0.200
bridge_fd 5
bridge_stp off
bridge_maxwait 1
# Private network
auto cloudbr1
iface cloudbr1 inet manual
bridge_ports eth0.300
bridge_fd 5
bridge_stp off
bridge_maxwait 1
====================
Cheers,
Dan
2014-11-23 7:51 GMT-06:00 Vadim Kimlaychuk <Va...@elion.ee>:
> Hello Dan,
>
> Sorry for the late reply, have to be out from internet sometimes
> :)
> You have problem with routing. According to arp table default
> interface -- eth2 it is not connected to you 192.168.0.* network and can't
> see gateway (192.168.0.100) what is not ture. Arp table looks ugly -- you
> have 3 interfaces connected to the same network with same Metric. How do
> you think VM will choose the correct one?? I suppose it will take the
> first one in line -- eth1 and that is why you see eth1 with complete MAC
> for 192.168.0.100. It always uses eth1 to reach other networks, but default
> gw is eth2.
> For me, network layout looks awkward. It is better to
> re-partition and re-think it. If you don't want to -- try to fix routing
> table manually. Put metrics to your routes and eth2 should be the highest
> (i.e. have lowest number). This will temporarty solve the problem, but you
> still have a chance to loose your changes after reboot, because routing
> table is created dynamically.
>
> Fix you network and I believe everything will work out-of-the box.
>
> Regards,
>
> Vadim.
>
> ________________________________________
> From: Dan Dong [dongdan39@gmail.com]
> Sent: Wednesday, November 19, 2014 17:45
> To: users@cloudstack.apache.org
> Subject: Re: To let SSVM reach outside network.
>
> Hi, Vadim,
> 1. route -n
> root@s-1-VM:~# route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use
> Iface
> 0.0.0.0 192.168.0.100 0.0.0.0 UG 0 0 0
> eth2
> 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0
> eth0
> 192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0
> eth1
> 192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0
> eth2
> 192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0
> eth3
>
> Noted that to the same 192.168.0.0 network, eth1 is searched before eth2,
> while eth2 is supposed to be the public NIC. Should one change the order?
>
> 2. root@s-1-VM:~# arp -n
> cloud (192.168.0.100) at 84:2b:2b:01:c3:d0 [ether] on eth1
> cloud (192.168.0.100) at <incomplete> on eth2
> ? (169.254.0.1) at fe:00:a9:fe:01:7a [ether] on eth0
>
> Noted that MAC is incomplete on eth2, while MAC detected on eth1 is the
> correct address of the internal NIC of the Management Server(the gateway
> for 192.168.0.0 subnet).
>
> 3. The network layout is quite simple here( basic network):
> Management Server external NIC:10.*
> Management Server internal NIC: 192.168.0.100
>
> KVM host NIC: 192.168.0.101
> SSVM and guest VMs are all in 192.168.0.*/24 network.
>
> 4. NAT is configured on the Management Server so 192.168.0.0/24 subnet
> could reach the internet(download packages from internet etc., e.g, from
> KVM host itself)
>
> Cheers,
> Dan
>
>
> 2014-11-19 1:23 GMT-06:00 Vadim Kimlaychuk <Va...@elion.ee>:
>
> > Dan,
> >
> > I would suggest you to use external proxy/GW to hide you network. Not the
> > same host that contains management server. Because if you would like to
> > scale it up - how do you do it?? Management server should be rather
> simple
> > component of infrastructure and to put more functions on it is a bad
> idea.
> >
> > Anyway
> > 1. what does "route -n" says on your SSVM ??
> > 2. If you use KVM your bridges tag the traffic. Are you sure that it is
> > properly handled on your switch?? Run "arp -a" at your SSVM and see if VM
> > "knowns" MAC of the gateway.
> >
> > I do expect you put the output of route and arp here otherwise we will
> > continue guessing.
> >
> > It would be even better if you describe your network layout like on the
> > first picture here:
> >
> http://cloudstack-administration.readthedocs.org/en/latest/networking_and_traffic.html
> >
> > Vadim.
> >
> > -----Original Message-----
> > From: Dan Dong [mailto:dongdan39@gmail.com]
> > Sent: Tuesday, November 18, 2014 5:23 PM
> > To: users@cloudstack.apache.org
> > Subject: Re: To let SSVM reach outside network.
> >
> > Hi, Vadim,
> > We have to use 2 NICs on the management server as we want to hide the
> > cloudstack cluster behind the 10.* network, so all KVM hosts and guest
> VMs
> > are in the 192.168.0.* subnet, and they connect to the management
> server's
> > internal NIC(192.168.0.100). Is it a rule that the management server can
> > only use one NIC? And the KVM hypersior host can reach the internet and
> > download packages, but SSVM running on it could not see the internet.
> >
> > Cheers,
> > Dan
> >
> >
> > 2014-11-18 3:20 GMT-06:00 Vadim Kimlaychuk <Va...@elion.ee>:
> >
> > > Hello Dan,
> > >
> > > It seems there is something wrong with your network setup and
> > > here are some places to search:
> > > 1. Why your management server has 2 NICs? It should not work as NAT,
> > > proxy or any kind of switch - keep this in mind.
> > > 2. SSVM normally has to have 3 interfaces (at least). One -- with
> > > public IP, one -- management network IP and one -- link local IP. If
> > > you have separate storage network -- it may have one more, but this is
> > > not your case. Check routing table for your SSVM with "route -n"
> > > command. Find your default gateway. It should be public interface.
> > > 3. Did you set up KVM hypervisor network correctly? Does it have 2
> > > interfaces like in the setup guide? Does your hypervisor have access
> > > to internet to be able to download template?
> > >
> > > Vadim.
> > >
> > > -----Original Message-----
> > > From: Dan Dong [mailto:dongdan39@gmail.com]
> > > Sent: Monday, November 17, 2014 7:02 PM
> > > To: users@cloudstack.apache.org
> > > Subject: To let SSVM reach outside network.
> > >
> > > Hi, All,
> > > I found I could not register my ISO image to the cloudstack( through
> > > URL of
> > http://releases.ubuntu.com/14.04.1/ubuntu-14.04.1-server-amd64.iso).
> > > The SSVM is running and health check show no problems on it. But it
> > > just could not see the outside network, although SSVM is on the same
> > > network as KVM host, i.e. 192.168.0.*. My setup is as following:
> > >
> > > Management Server external NIC:10.*
> > > Management Server internal NIC: 192.168.0.100
> > >
> > > KVM host NIC: 192.168.0.101
> > > guest VMs are all in 192.168.0.*/24 network.
> > >
> > > From SSVM I can ping the internal IP of the Management Server at
> > > 192.168.0.100, but could not ping the external IP of it at 10.*.
> > >
> > > From KVM host itself I can reach the outside internet as NAT is
> > > configured on the Management Server to let the 192.168.0.0 traffic
> > > through. Any hints how to let SSVM to reach the internet?
> > >
> > >
> > > Cheers,
> > > Dan
> > >
> >
>
RE: To let SSVM reach outside network.
Posted by Vadim Kimlaychuk <Va...@Elion.ee>.
Hello Dan,
Sorry for the late reply, have to be out from internet sometimes :)
You have problem with routing. According to arp table default interface -- eth2 it is not connected to you 192.168.0.* network and can't see gateway (192.168.0.100) what is not ture. Arp table looks ugly -- you have 3 interfaces connected to the same network with same Metric. How do you think VM will choose the correct one?? I suppose it will take the first one in line -- eth1 and that is why you see eth1 with complete MAC for 192.168.0.100. It always uses eth1 to reach other networks, but default gw is eth2.
For me, network layout looks awkward. It is better to re-partition and re-think it. If you don't want to -- try to fix routing table manually. Put metrics to your routes and eth2 should be the highest (i.e. have lowest number). This will temporarty solve the problem, but you still have a chance to loose your changes after reboot, because routing table is created dynamically.
Fix you network and I believe everything will work out-of-the box.
Regards,
Vadim.
________________________________________
From: Dan Dong [dongdan39@gmail.com]
Sent: Wednesday, November 19, 2014 17:45
To: users@cloudstack.apache.org
Subject: Re: To let SSVM reach outside network.
Hi, Vadim,
1. route -n
root@s-1-VM:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
0.0.0.0 192.168.0.100 0.0.0.0 UG 0 0 0
eth2
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth3
Noted that to the same 192.168.0.0 network, eth1 is searched before eth2,
while eth2 is supposed to be the public NIC. Should one change the order?
2. root@s-1-VM:~# arp -n
cloud (192.168.0.100) at 84:2b:2b:01:c3:d0 [ether] on eth1
cloud (192.168.0.100) at <incomplete> on eth2
? (169.254.0.1) at fe:00:a9:fe:01:7a [ether] on eth0
Noted that MAC is incomplete on eth2, while MAC detected on eth1 is the
correct address of the internal NIC of the Management Server(the gateway
for 192.168.0.0 subnet).
3. The network layout is quite simple here( basic network):
Management Server external NIC:10.*
Management Server internal NIC: 192.168.0.100
KVM host NIC: 192.168.0.101
SSVM and guest VMs are all in 192.168.0.*/24 network.
4. NAT is configured on the Management Server so 192.168.0.0/24 subnet
could reach the internet(download packages from internet etc., e.g, from
KVM host itself)
Cheers,
Dan
2014-11-19 1:23 GMT-06:00 Vadim Kimlaychuk <Va...@elion.ee>:
> Dan,
>
> I would suggest you to use external proxy/GW to hide you network. Not the
> same host that contains management server. Because if you would like to
> scale it up - how do you do it?? Management server should be rather simple
> component of infrastructure and to put more functions on it is a bad idea.
>
> Anyway
> 1. what does "route -n" says on your SSVM ??
> 2. If you use KVM your bridges tag the traffic. Are you sure that it is
> properly handled on your switch?? Run "arp -a" at your SSVM and see if VM
> "knowns" MAC of the gateway.
>
> I do expect you put the output of route and arp here otherwise we will
> continue guessing.
>
> It would be even better if you describe your network layout like on the
> first picture here:
> http://cloudstack-administration.readthedocs.org/en/latest/networking_and_traffic.html
>
> Vadim.
>
> -----Original Message-----
> From: Dan Dong [mailto:dongdan39@gmail.com]
> Sent: Tuesday, November 18, 2014 5:23 PM
> To: users@cloudstack.apache.org
> Subject: Re: To let SSVM reach outside network.
>
> Hi, Vadim,
> We have to use 2 NICs on the management server as we want to hide the
> cloudstack cluster behind the 10.* network, so all KVM hosts and guest VMs
> are in the 192.168.0.* subnet, and they connect to the management server's
> internal NIC(192.168.0.100). Is it a rule that the management server can
> only use one NIC? And the KVM hypersior host can reach the internet and
> download packages, but SSVM running on it could not see the internet.
>
> Cheers,
> Dan
>
>
> 2014-11-18 3:20 GMT-06:00 Vadim Kimlaychuk <Va...@elion.ee>:
>
> > Hello Dan,
> >
> > It seems there is something wrong with your network setup and
> > here are some places to search:
> > 1. Why your management server has 2 NICs? It should not work as NAT,
> > proxy or any kind of switch - keep this in mind.
> > 2. SSVM normally has to have 3 interfaces (at least). One -- with
> > public IP, one -- management network IP and one -- link local IP. If
> > you have separate storage network -- it may have one more, but this is
> > not your case. Check routing table for your SSVM with "route -n"
> > command. Find your default gateway. It should be public interface.
> > 3. Did you set up KVM hypervisor network correctly? Does it have 2
> > interfaces like in the setup guide? Does your hypervisor have access
> > to internet to be able to download template?
> >
> > Vadim.
> >
> > -----Original Message-----
> > From: Dan Dong [mailto:dongdan39@gmail.com]
> > Sent: Monday, November 17, 2014 7:02 PM
> > To: users@cloudstack.apache.org
> > Subject: To let SSVM reach outside network.
> >
> > Hi, All,
> > I found I could not register my ISO image to the cloudstack( through
> > URL of
> http://releases.ubuntu.com/14.04.1/ubuntu-14.04.1-server-amd64.iso).
> > The SSVM is running and health check show no problems on it. But it
> > just could not see the outside network, although SSVM is on the same
> > network as KVM host, i.e. 192.168.0.*. My setup is as following:
> >
> > Management Server external NIC:10.*
> > Management Server internal NIC: 192.168.0.100
> >
> > KVM host NIC: 192.168.0.101
> > guest VMs are all in 192.168.0.*/24 network.
> >
> > From SSVM I can ping the internal IP of the Management Server at
> > 192.168.0.100, but could not ping the external IP of it at 10.*.
> >
> > From KVM host itself I can reach the outside internet as NAT is
> > configured on the Management Server to let the 192.168.0.0 traffic
> > through. Any hints how to let SSVM to reach the internet?
> >
> >
> > Cheers,
> > Dan
> >
>
Re: To let SSVM reach outside network.
Posted by Dan Dong <do...@gmail.com>.
Hi, Vadim,
1. route -n
root@s-1-VM:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
0.0.0.0 192.168.0.100 0.0.0.0 UG 0 0 0
eth2
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth3
Noted that to the same 192.168.0.0 network, eth1 is searched before eth2,
while eth2 is supposed to be the public NIC. Should one change the order?
2. root@s-1-VM:~# arp -n
cloud (192.168.0.100) at 84:2b:2b:01:c3:d0 [ether] on eth1
cloud (192.168.0.100) at <incomplete> on eth2
? (169.254.0.1) at fe:00:a9:fe:01:7a [ether] on eth0
Noted that MAC is incomplete on eth2, while MAC detected on eth1 is the
correct address of the internal NIC of the Management Server(the gateway
for 192.168.0.0 subnet).
3. The network layout is quite simple here( basic network):
Management Server external NIC:10.*
Management Server internal NIC: 192.168.0.100
KVM host NIC: 192.168.0.101
SSVM and guest VMs are all in 192.168.0.*/24 network.
4. NAT is configured on the Management Server so 192.168.0.0/24 subnet
could reach the internet(download packages from internet etc., e.g, from
KVM host itself)
Cheers,
Dan
2014-11-19 1:23 GMT-06:00 Vadim Kimlaychuk <Va...@elion.ee>:
> Dan,
>
> I would suggest you to use external proxy/GW to hide you network. Not the
> same host that contains management server. Because if you would like to
> scale it up - how do you do it?? Management server should be rather simple
> component of infrastructure and to put more functions on it is a bad idea.
>
> Anyway
> 1. what does "route -n" says on your SSVM ??
> 2. If you use KVM your bridges tag the traffic. Are you sure that it is
> properly handled on your switch?? Run "arp -a" at your SSVM and see if VM
> "knowns" MAC of the gateway.
>
> I do expect you put the output of route and arp here otherwise we will
> continue guessing.
>
> It would be even better if you describe your network layout like on the
> first picture here:
> http://cloudstack-administration.readthedocs.org/en/latest/networking_and_traffic.html
>
> Vadim.
>
> -----Original Message-----
> From: Dan Dong [mailto:dongdan39@gmail.com]
> Sent: Tuesday, November 18, 2014 5:23 PM
> To: users@cloudstack.apache.org
> Subject: Re: To let SSVM reach outside network.
>
> Hi, Vadim,
> We have to use 2 NICs on the management server as we want to hide the
> cloudstack cluster behind the 10.* network, so all KVM hosts and guest VMs
> are in the 192.168.0.* subnet, and they connect to the management server's
> internal NIC(192.168.0.100). Is it a rule that the management server can
> only use one NIC? And the KVM hypersior host can reach the internet and
> download packages, but SSVM running on it could not see the internet.
>
> Cheers,
> Dan
>
>
> 2014-11-18 3:20 GMT-06:00 Vadim Kimlaychuk <Va...@elion.ee>:
>
> > Hello Dan,
> >
> > It seems there is something wrong with your network setup and
> > here are some places to search:
> > 1. Why your management server has 2 NICs? It should not work as NAT,
> > proxy or any kind of switch - keep this in mind.
> > 2. SSVM normally has to have 3 interfaces (at least). One -- with
> > public IP, one -- management network IP and one -- link local IP. If
> > you have separate storage network -- it may have one more, but this is
> > not your case. Check routing table for your SSVM with "route -n"
> > command. Find your default gateway. It should be public interface.
> > 3. Did you set up KVM hypervisor network correctly? Does it have 2
> > interfaces like in the setup guide? Does your hypervisor have access
> > to internet to be able to download template?
> >
> > Vadim.
> >
> > -----Original Message-----
> > From: Dan Dong [mailto:dongdan39@gmail.com]
> > Sent: Monday, November 17, 2014 7:02 PM
> > To: users@cloudstack.apache.org
> > Subject: To let SSVM reach outside network.
> >
> > Hi, All,
> > I found I could not register my ISO image to the cloudstack( through
> > URL of
> http://releases.ubuntu.com/14.04.1/ubuntu-14.04.1-server-amd64.iso).
> > The SSVM is running and health check show no problems on it. But it
> > just could not see the outside network, although SSVM is on the same
> > network as KVM host, i.e. 192.168.0.*. My setup is as following:
> >
> > Management Server external NIC:10.*
> > Management Server internal NIC: 192.168.0.100
> >
> > KVM host NIC: 192.168.0.101
> > guest VMs are all in 192.168.0.*/24 network.
> >
> > From SSVM I can ping the internal IP of the Management Server at
> > 192.168.0.100, but could not ping the external IP of it at 10.*.
> >
> > From KVM host itself I can reach the outside internet as NAT is
> > configured on the Management Server to let the 192.168.0.0 traffic
> > through. Any hints how to let SSVM to reach the internet?
> >
> >
> > Cheers,
> > Dan
> >
>
RE: To let SSVM reach outside network.
Posted by Vadim Kimlaychuk <Va...@Elion.ee>.
Dan,
I would suggest you to use external proxy/GW to hide you network. Not the same host that contains management server. Because if you would like to scale it up - how do you do it?? Management server should be rather simple component of infrastructure and to put more functions on it is a bad idea.
Anyway
1. what does "route -n" says on your SSVM ??
2. If you use KVM your bridges tag the traffic. Are you sure that it is properly handled on your switch?? Run "arp -a" at your SSVM and see if VM "knowns" MAC of the gateway.
I do expect you put the output of route and arp here otherwise we will continue guessing.
It would be even better if you describe your network layout like on the first picture here: http://cloudstack-administration.readthedocs.org/en/latest/networking_and_traffic.html
Vadim.
-----Original Message-----
From: Dan Dong [mailto:dongdan39@gmail.com]
Sent: Tuesday, November 18, 2014 5:23 PM
To: users@cloudstack.apache.org
Subject: Re: To let SSVM reach outside network.
Hi, Vadim,
We have to use 2 NICs on the management server as we want to hide the cloudstack cluster behind the 10.* network, so all KVM hosts and guest VMs are in the 192.168.0.* subnet, and they connect to the management server's internal NIC(192.168.0.100). Is it a rule that the management server can only use one NIC? And the KVM hypersior host can reach the internet and download packages, but SSVM running on it could not see the internet.
Cheers,
Dan
2014-11-18 3:20 GMT-06:00 Vadim Kimlaychuk <Va...@elion.ee>:
> Hello Dan,
>
> It seems there is something wrong with your network setup and
> here are some places to search:
> 1. Why your management server has 2 NICs? It should not work as NAT,
> proxy or any kind of switch - keep this in mind.
> 2. SSVM normally has to have 3 interfaces (at least). One -- with
> public IP, one -- management network IP and one -- link local IP. If
> you have separate storage network -- it may have one more, but this is
> not your case. Check routing table for your SSVM with "route -n"
> command. Find your default gateway. It should be public interface.
> 3. Did you set up KVM hypervisor network correctly? Does it have 2
> interfaces like in the setup guide? Does your hypervisor have access
> to internet to be able to download template?
>
> Vadim.
>
> -----Original Message-----
> From: Dan Dong [mailto:dongdan39@gmail.com]
> Sent: Monday, November 17, 2014 7:02 PM
> To: users@cloudstack.apache.org
> Subject: To let SSVM reach outside network.
>
> Hi, All,
> I found I could not register my ISO image to the cloudstack( through
> URL of http://releases.ubuntu.com/14.04.1/ubuntu-14.04.1-server-amd64.iso).
> The SSVM is running and health check show no problems on it. But it
> just could not see the outside network, although SSVM is on the same
> network as KVM host, i.e. 192.168.0.*. My setup is as following:
>
> Management Server external NIC:10.*
> Management Server internal NIC: 192.168.0.100
>
> KVM host NIC: 192.168.0.101
> guest VMs are all in 192.168.0.*/24 network.
>
> From SSVM I can ping the internal IP of the Management Server at
> 192.168.0.100, but could not ping the external IP of it at 10.*.
>
> From KVM host itself I can reach the outside internet as NAT is
> configured on the Management Server to let the 192.168.0.0 traffic
> through. Any hints how to let SSVM to reach the internet?
>
>
> Cheers,
> Dan
>
Re: To let SSVM reach outside network.
Posted by Dan Dong <do...@gmail.com>.
Hi, Vadim,
We have to use 2 NICs on the management server as we want to hide the
cloudstack cluster behind the 10.* network, so all KVM hosts and guest VMs
are in the 192.168.0.* subnet, and they connect to the management server's
internal NIC(192.168.0.100). Is it a rule that the management server can
only use one NIC? And the KVM hypersior host can reach the internet and
download packages, but SSVM running on it could not see the internet.
Cheers,
Dan
2014-11-18 3:20 GMT-06:00 Vadim Kimlaychuk <Va...@elion.ee>:
> Hello Dan,
>
> It seems there is something wrong with your network setup and here
> are some places to search:
> 1. Why your management server has 2 NICs? It should not work as NAT,
> proxy or any kind of switch - keep this in mind.
> 2. SSVM normally has to have 3 interfaces (at least). One -- with public
> IP, one -- management network IP and one -- link local IP. If you have
> separate storage network -- it may have one more, but this is not your
> case. Check routing table for your SSVM with "route -n" command. Find your
> default gateway. It should be public interface.
> 3. Did you set up KVM hypervisor network correctly? Does it have 2
> interfaces like in the setup guide? Does your hypervisor have access to
> internet to be able to download template?
>
> Vadim.
>
> -----Original Message-----
> From: Dan Dong [mailto:dongdan39@gmail.com]
> Sent: Monday, November 17, 2014 7:02 PM
> To: users@cloudstack.apache.org
> Subject: To let SSVM reach outside network.
>
> Hi, All,
> I found I could not register my ISO image to the cloudstack( through URL
> of http://releases.ubuntu.com/14.04.1/ubuntu-14.04.1-server-amd64.iso).
> The SSVM is running and health check show no problems on it. But it just
> could not see the outside network, although SSVM is on the same network as
> KVM host, i.e. 192.168.0.*. My setup is as following:
>
> Management Server external NIC:10.*
> Management Server internal NIC: 192.168.0.100
>
> KVM host NIC: 192.168.0.101
> guest VMs are all in 192.168.0.*/24 network.
>
> From SSVM I can ping the internal IP of the Management Server at
> 192.168.0.100, but could not ping the external IP of it at 10.*.
>
> From KVM host itself I can reach the outside internet as NAT is configured
> on the Management Server to let the 192.168.0.0 traffic through. Any hints
> how to let SSVM to reach the internet?
>
>
> Cheers,
> Dan
>
Re: To let SSVM reach outside network.
Posted by Tilak Raj Singh <ti...@gmail.com>.
if you are behind a authenticated proxy or cannot access the outside
network a little trick that worked for me was downloading the iso on the
management server. Then running *python -m SimpleHTTPServer <port number>*
in the download directory to turn it into a sort of mirror and then using
that to upload the iso using the path like *http://localhost
<http://localhost>:<port number>/<name of the iso>*
For doing this you would also need to change the
*secstorage.allowed.internal.sites* value in the global settings to the ip
of ur management server or to its complete subnet...like my server ip is in
subnet 172.31.100.0/22 so i did put that value for the listed global
setting parameter
Regards
On Tue, Nov 18, 2014 at 2:58 PM, Vadim Kimlaychuk <Vadim.Kimlaychuk@elion.ee
> wrote:
> Will fix my last point - KVM should have at least 2 bridges, not
> necessarily interfaces - public/private.
>
> Vadim
>
> -----Original Message-----
> From: Vadim Kimlaychuk [mailto:Vadim.Kimlaychuk@Elion.ee]
> Sent: Tuesday, November 18, 2014 11:21 AM
> To: users@cloudstack.apache.org
> Subject: RE: To let SSVM reach outside network.
>
> Hello Dan,
>
> It seems there is something wrong with your network setup and here
> are some places to search:
> 1. Why your management server has 2 NICs? It should not work as NAT,
> proxy or any kind of switch - keep this in mind.
> 2. SSVM normally has to have 3 interfaces (at least). One -- with public
> IP, one -- management network IP and one -- link local IP. If you have
> separate storage network -- it may have one more, but this is not your
> case. Check routing table for your SSVM with "route -n" command. Find your
> default gateway. It should be public interface.
> 3. Did you set up KVM hypervisor network correctly? Does it have 2
> interfaces like in the setup guide? Does your hypervisor have access to
> internet to be able to download template?
>
> Vadim.
>
> -----Original Message-----
> From: Dan Dong [mailto:dongdan39@gmail.com]
> Sent: Monday, November 17, 2014 7:02 PM
> To: users@cloudstack.apache.org
> Subject: To let SSVM reach outside network.
>
> Hi, All,
> I found I could not register my ISO image to the cloudstack( through URL
> of http://releases.ubuntu.com/14.04.1/ubuntu-14.04.1-server-amd64.iso).
> The SSVM is running and health check show no problems on it. But it just
> could not see the outside network, although SSVM is on the same network as
> KVM host, i.e. 192.168.0.*. My setup is as following:
>
> Management Server external NIC:10.*
> Management Server internal NIC: 192.168.0.100
>
> KVM host NIC: 192.168.0.101
> guest VMs are all in 192.168.0.*/24 network.
>
> From SSVM I can ping the internal IP of the Management Server at
> 192.168.0.100, but could not ping the external IP of it at 10.*.
>
> From KVM host itself I can reach the outside internet as NAT is configured
> on the Management Server to let the 192.168.0.0 traffic through. Any hints
> how to let SSVM to reach the internet?
>
>
> Cheers,
> Dan
>
RE: To let SSVM reach outside network.
Posted by Vadim Kimlaychuk <Va...@Elion.ee>.
Will fix my last point - KVM should have at least 2 bridges, not necessarily interfaces - public/private.
Vadim
-----Original Message-----
From: Vadim Kimlaychuk [mailto:Vadim.Kimlaychuk@Elion.ee]
Sent: Tuesday, November 18, 2014 11:21 AM
To: users@cloudstack.apache.org
Subject: RE: To let SSVM reach outside network.
Hello Dan,
It seems there is something wrong with your network setup and here are some places to search:
1. Why your management server has 2 NICs? It should not work as NAT, proxy or any kind of switch - keep this in mind.
2. SSVM normally has to have 3 interfaces (at least). One -- with public IP, one -- management network IP and one -- link local IP. If you have separate storage network -- it may have one more, but this is not your case. Check routing table for your SSVM with "route -n" command. Find your default gateway. It should be public interface.
3. Did you set up KVM hypervisor network correctly? Does it have 2 interfaces like in the setup guide? Does your hypervisor have access to internet to be able to download template?
Vadim.
-----Original Message-----
From: Dan Dong [mailto:dongdan39@gmail.com]
Sent: Monday, November 17, 2014 7:02 PM
To: users@cloudstack.apache.org
Subject: To let SSVM reach outside network.
Hi, All,
I found I could not register my ISO image to the cloudstack( through URL of http://releases.ubuntu.com/14.04.1/ubuntu-14.04.1-server-amd64.iso). The SSVM is running and health check show no problems on it. But it just could not see the outside network, although SSVM is on the same network as KVM host, i.e. 192.168.0.*. My setup is as following:
Management Server external NIC:10.*
Management Server internal NIC: 192.168.0.100
KVM host NIC: 192.168.0.101
guest VMs are all in 192.168.0.*/24 network.
From SSVM I can ping the internal IP of the Management Server at 192.168.0.100, but could not ping the external IP of it at 10.*.
From KVM host itself I can reach the outside internet as NAT is configured on the Management Server to let the 192.168.0.0 traffic through. Any hints how to let SSVM to reach the internet?
Cheers,
Dan
RE: To let SSVM reach outside network.
Posted by Vadim Kimlaychuk <Va...@Elion.ee>.
Hello Dan,
It seems there is something wrong with your network setup and here are some places to search:
1. Why your management server has 2 NICs? It should not work as NAT, proxy or any kind of switch - keep this in mind.
2. SSVM normally has to have 3 interfaces (at least). One -- with public IP, one -- management network IP and one -- link local IP. If you have separate storage network -- it may have one more, but this is not your case. Check routing table for your SSVM with "route -n" command. Find your default gateway. It should be public interface.
3. Did you set up KVM hypervisor network correctly? Does it have 2 interfaces like in the setup guide? Does your hypervisor have access to internet to be able to download template?
Vadim.
-----Original Message-----
From: Dan Dong [mailto:dongdan39@gmail.com]
Sent: Monday, November 17, 2014 7:02 PM
To: users@cloudstack.apache.org
Subject: To let SSVM reach outside network.
Hi, All,
I found I could not register my ISO image to the cloudstack( through URL of http://releases.ubuntu.com/14.04.1/ubuntu-14.04.1-server-amd64.iso). The SSVM is running and health check show no problems on it. But it just could not see the outside network, although SSVM is on the same network as KVM host, i.e. 192.168.0.*. My setup is as following:
Management Server external NIC:10.*
Management Server internal NIC: 192.168.0.100
KVM host NIC: 192.168.0.101
guest VMs are all in 192.168.0.*/24 network.
From SSVM I can ping the internal IP of the Management Server at 192.168.0.100, but could not ping the external IP of it at 10.*.
From KVM host itself I can reach the outside internet as NAT is configured on the Management Server to let the 192.168.0.0 traffic through. Any hints how to let SSVM to reach the internet?
Cheers,
Dan