You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Brian Smith <bm...@gmail.com> on 2016/04/30 21:44:37 UTC
New Setup, CloudStack 4.8 - Networking Problems
Hello,
I am new to ACS and just deployed a proof of concept environment. My goal
is to run in Advanced Network configuration.
So far, I have 2 servers which have 2 NICs each. 1 server is a management
and the other is a KVM hypervisor
What I am experiencing:
I can ping the Management Server and the KVM Host without any issue. I can
also ping to the two System VMs (Console and Secondary Storage VM) on the,
what I call Internal network. (10.32.0.0/24).
I cannot ping the Public IP for the System VMs and therefore, it won't
download templates, etc.
I am connecting this all to a Cisco 6509 Switch
Important VLANs:
VLAN 80 - 10.32.0.0/24 All Hypervisors, management and storage nodes live
here
VLAN 81 - 104.x.x.128/27 This is going to be the public IPs for the system
VMs.
On the KVM Hypervisor, eth0 is connected to Gi4/39 and eth1 is connected to
Gi3/3 on the switch.
##SWITCH CONFIGURATION
#show run int gi4/39
interface GigabitEthernet4/39
description mgmt01.cs -eth0
switchport
switchport trunk native vlan 80
switchport trunk allowed vlan 80
switchport mode trunk
end
#show run int gi3/3
interface GigabitEthernet3/3
description mgmt01.cs -eth0
switchport
switchport trunk native vlan 81
switchport trunk allowed vlan 80,81
switchport mode trunk
end
$$node01 - KVM HOST NETWORK CONFIGURATION
[root@node01 ~]# ifconfig
breth1-81 Link encap:Ethernet HWaddr 00:23:8B:CE:5E:C6
inet6 addr: fe80::223:8bff:fece:5ec6/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:22 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:616 (616.0 b) TX bytes:468 (468.0 b)
cloud0 Link encap:Ethernet HWaddr FE:00:A9:FE:00:48
inet addr:169.254.0.1 Bcast:0.0.0.0 Mask:255.255.0.0
inet6 addr: fe80::6c50:45ff:fe33:a6e8/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:289 errors:0 dropped:0 overruns:0 frame:0
TX packets:413 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:29487 (28.7 KiB) TX bytes:29895 (29.1 KiB)
cloudbr0 Link encap:Ethernet HWaddr 00:23:8B:CE:5E:C5
inet addr:10.32.0.98 Bcast:10.32.0.255 Mask:255.255.255.0
inet6 addr: fe80::223:8bff:fece:5ec5/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:54188 errors:0 dropped:0 overruns:0 frame:0
TX packets:59313 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:216790382 (206.7 MiB) TX bytes:181559895 (173.1 MiB)
cloudbr1 Link encap:Ethernet HWaddr 00:23:8B:CE:5E:C6
inet6 addr: fe80::223:8bff:fece:5ec6/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:56636 errors:0 dropped:0 overruns:0 frame:0
TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:3383292 (3.2 MiB) TX bytes:578 (578.0 b)
eth0.80 Link encap:Ethernet HWaddr 00:23:8B:CE:5E:C5
inet6 addr: fe80::223:8bff:fece:5ec5/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:171389 errors:0 dropped:0 overruns:0 frame:0
TX packets:176284 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:226117957 (215.6 MiB) TX bytes:190098763 (181.2 MiB)
Interrupt:20 Memory:df5c0000-df5e0000
eth1 Link encap:Ethernet HWaddr 00:23:8B:CE:5E:C6
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:57894 errors:0 dropped:0 overruns:0 frame:0
TX packets:49 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:4748864 (4.5 MiB) TX bytes:3726 (3.6 KiB)
Interrupt:16 Memory:df6e0000-df700000
eth1.81 Link encap:Ethernet HWaddr 00:23:8B:CE:5E:C6
inet6 addr: fe80::223:8bff:fece:5ec6/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:34 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:1860 (1.8 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:13 errors:0 dropped:0 overruns:0 frame:0
TX packets:13 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1144 (1.1 KiB) TX bytes:1144 (1.1 KiB)
virbr0 Link encap:Ethernet HWaddr 52:54:00:D7:F9:D6
inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
vnet0 Link encap:Ethernet HWaddr FE:00:A9:FE:00:48
inet6 addr: fe80::fc00:a9ff:fefe:48/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:278 errors:0 dropped:0 overruns:0 frame:0
TX packets:401 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:32861 (32.0 KiB) TX bytes:29159 (28.4 KiB)
vnet1 Link encap:Ethernet HWaddr FE:1F:96:00:00:01
inet6 addr: fe80::fc1f:96ff:fe00:1/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1653 errors:0 dropped:0 overruns:0 frame:0
TX packets:1958 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:142088 (138.7 KiB) TX bytes:207540 (202.6 KiB)
vnet2 Link encap:Ethernet HWaddr FE:C0:EE:00:00:08
inet6 addr: fe80::fcc0:eeff:fe00:8/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:17 errors:0 dropped:0 overruns:0 frame:0
TX packets:19 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:762 (762.0 b) TX bytes:1230 (1.2 KiB)
vnet3 Link encap:Ethernet HWaddr FE:00:A9:FE:03:42
inet6 addr: fe80::fc00:a9ff:fefe:342/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:11 errors:0 dropped:0 overruns:0 frame:0
TX packets:45 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:672 (672.0 b) TX bytes:2430 (2.3 KiB)
vnet4 Link encap:Ethernet HWaddr FE:5D:7C:00:00:05
inet6 addr: fe80::fc5d:7cff:fe00:5/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:962 errors:0 dropped:0 overruns:0 frame:0
TX packets:1261 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:139736 (136.4 KiB) TX bytes:253449 (247.5 KiB)
vnet5 Link encap:Ethernet HWaddr FE:46:D2:00:00:07
inet6 addr: fe80::fc46:d2ff:fe00:7/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:8 errors:0 dropped:0 overruns:0 frame:0
TX packets:27 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:384 (384.0 b) TX bytes:1530 (1.4 KiB)
vnet6 Link encap:Ethernet HWaddr FE:7A:72:00:00:19
inet6 addr: fe80::fc7a:72ff:fe00:19/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:8 errors:0 dropped:0 overruns:0 frame:0
TX packets:601 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:624 (624.0 b) TX bytes:90661 (88.5 KiB)
## ifcfg-eth0.80
DEVICE=eth0.80
TYPE=Ethernet
HWADDR=00:23:8B:CE:5E:C5
TYPE=Ethernet
UUID=a2f0a042-3d0f-4325-ab13-c9ed60cb9a3e
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=none
BRIDGE=cloudbr0
DNS1=8.8.8.8
## ifcfg-eth1.81
DEVICE=eth1
HWADDR=00:23:8B:CE:5E:C6
TYPE=Ethernet
UUID=367654c0-d0c5-4bed-8191-d3f65c8a315f
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=none
BRIDGE=cloudbr1
## ifcfg-cloudbr0
DEVICE=cloudbr0
TYPE=Bridge
ONBOOT=yes
BOOTPROTO=none
IPADDR=10.32.0.98
NETMASK=255.255.255.0
GATEWAY=10.32.0.2
DELAY=0
STP=no
DNS1=8.8.8.8
## ifcfg-cloudbr1
DEVICE=cloudbr1
TYPE=Bridge
ONBOOT=yes
BOOTPROTO=none
STP=no
DELAY=0
Screenshots:
Home > Infrastructure > Zones >Zone1 > Physical Network -
http://prntscr.com/aysnia
Home > Infrastructure > Zones >Zone1 > Physical Network 1
http://prntscr.com/aysnta
Home > Infrastructure > Zones >Zone1 > Physical Network 1 > Management >
Details http://prntscr.com/ayso8d
Home > Infrastructure > Zones >Zone1 > Physical Network 1 > Management >IP
Ranges http://prntscr.com/aysol9
Note: Storage config for Physical Network 1 looks the same (cloudbr0 as
well)
Home > Infrastructure > Zones >Zone1 > Physical Network 2
http://prntscr.com/aysp50
Home > Infrastructure > Zones >Zone1 > Physical Network 2 > Guest > Details
http://prntscr.com/ayspic
Note: Nothing under "Network" or "Dedicated VLAN/VNI Ranges" tabs.
Home > Infrastructure > Zones >Zone1 > Physical Network 2 > Public >
Details http://prntscr.com/aysq5o
Home > Infrastructure > Zones >Zone1 > Physical Network 2 > Public > IP
Ranges > <http://prntscr.com/aysqz3>http://prntscr.com/aysqz3
System VMs:
s-1-vm - Secondary Storage VM
Public IP: 104.x.x.130
Private IP: 10.32.0.9
Screenshot: http://prntscr.com/ayss7l
v-2-VM - Console VM
Public IP: 104.x.x.131
Private IP: 10.32.0.5
Screenshot: http://prntscr.com/ayst0y
When I SSH'd into the VM (v-2-VM) to run the check script here is the
outpout:
root@v-2-VM:~# /usr/local/cloud/systemvm/ssvm-check.sh
================================================
First DNS server is 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 48 data bytes
--- 8.8.8.8 ping statistics ---
2 packets transmitted, 0 packets received, 100% packet loss
WARNING: cannot ping DNS server
route follows
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
0.0.0.0 104.x.x.129 0.0.0.0 UG 0 0 0 eth2
8.8.8.8 10.32.0.2 255.255.255.255 UGH 0 0 0 eth1
10.0.0.0 10.32.0.2 255.0.0.0 UG 0 0 0 eth1
10.32.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
104.x.x.128 0.0.0.0 255.255.255.224 U 0 0 0 eth2
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
172.16.0.0 10.32.0.2 255.240.0.0 UG 0 0 0 eth1
192.168.0.0 10.32.0.2 255.255.0.0 UG 0 0 0 eth1
================================================
ERROR: DNS not resolving download.cloud.com
resolv.conf follows
nameserver 8.8.8.8
nameserver 8.8.8.8
root@v-2-VM:~# ping 10.32.0.97
PING 10.32.0.97 (10.32.0.97): 48 data bytes
56 bytes from 10.32.0.97: icmp_seq=0 ttl=64 time=0.249 ms
10.32.0.97 is the management server
What portion of my configuration appears to be wrong?
Re: New Setup, CloudStack 4.8 - Networking Problems
Posted by ilya <il...@gmail.com>.
I'm a bit late to the party, but thanks for posting the details of your
problem and solution! This at some point will be searchable by search
engines and help another fellow CloudStack'er.
On 4/30/16 9:43 PM, Brian Smith wrote:
> Hello,
>
> I got a lot of help from Simon in the IRC channel.
>
> I took the Native VLAN 81 off of the switchport config on the 6509. Once
> that was completed, the public IP was pingable externally. We did some
> further digging and 8.8.8.8 was being routed out via the private network
> because I do not have internal DNS servers setup and I filled in 8.8.8.8 as
> the internal DNS servers. So, in that sense, it was thinking 8.8.8.8 wasn't
> going anywhere. I added a few rules and routed that 8.8.8.8 traffic it was
> trying to send out through to the Internet and everything began working.
>
> Again Simon and Rafael (sp) thanks for your help over the past two days.
>
> I certainly hope I was able to help someone else out in the meantime. :)
>
Re: New Setup, CloudStack 4.8 - Networking Problems
Posted by Brian Smith <bm...@gmail.com>.
Hello,
I got a lot of help from Simon in the IRC channel.
I took the Native VLAN 81 off of the switchport config on the 6509. Once
that was completed, the public IP was pingable externally. We did some
further digging and 8.8.8.8 was being routed out via the private network
because I do not have internal DNS servers setup and I filled in 8.8.8.8 as
the internal DNS servers. So, in that sense, it was thinking 8.8.8.8 wasn't
going anywhere. I added a few rules and routed that 8.8.8.8 traffic it was
trying to send out through to the Internet and everything began working.
Again Simon and Rafael (sp) thanks for your help over the past two days.
I certainly hope I was able to help someone else out in the meantime. :)