You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Marc Leeman <ma...@gmail.com> on 2014/10/15 16:11:58 UTC
firewall problems and proxy problems
I've started out with cloudstack a couple days ago and I've hit a bit of a
brick wall.
The installation is running on a system that has two network interfaces, a
public one and private one (connecting to a local network).
auto eth0
iface eth0 inet manual
auto eth1
iface eth1 inet static
address 10.158.231.29
netmask 255.255.255.0
network 10.158.231.0
broadcast 10.158.231.255
gateway 10.158.231.1
auto cloudbr0
iface cloudbr0 inet static
address 172.16.8.7
netmask 255.255.255.0
network 172.16.8.0
broadcast 172.16.8.255
up route add -net 172.16.0.0 netmask 255.255.0.0 gw 172.16.8.1
bridge_ports eth0
bridge_fd 5
bridge_stp off
bridge_maxwait 1
The idea is to connect all the guests to the 172.16 network because access
to multicasting devices is needed.
The systems are running on Debian wheezy, and got the system up after
fixing the /etc/legal problem (echo Ubuntu) and after finding out in the
logs that there is a package dependency problem on the cloudstack-manager
package for 4.3
The biggest problem seems to be that host is running in a corporate network
and cloudstack is auto configuring some things by downloading over http.
This does not work since a corporate proxy is required.
I downloaded the system images manually; but now it seems I need to do the
same in the secondary storage vm.
The vm is running and I can get the login prompt with virsh (kvm), however
there is no password I can find.
The trick with the connection to the local-link address does not work
because I fear I have a networking/firewall issue. It does not matter if I
disable the firewall or not, I cannot access the 169.254.x.x network.
pinging the device returns
ping 169.254.3.236
PING 169.254.3.236 (169.254.3.236) 56(84) bytes of data.
>From 169.254.0.1 icmp_seq=1 Destination Host Unreachable
>From 169.254.0.1 icmp_seq=2 Destination Host Unreachable
>From 169.254.0.1 icmp_seq=3 Destination Host Unreachable
The firewall configuration is firehol
cat /etc/firehol/firehol.conf
version 5
# Accept all client traffic on any interface
FIREHOL_LOG_MODE="ULOG"
server_cloudstackweb_ports="tcp/8080"
client_cloudstackweb_ports="default"
server_buildbot_ports="tcp/8010"
client_buildbot_ports="default"
server_git_ports="tcp/9418"
client_git_ports="default"
labo_ips="172.16.0.0/16 169.254.0.0/16"
server_cloudstack_ports="tcp/1798"
client_cloudstack_ports="default"
server_libvirt_ports="tcp/16509"
client_libvirt_ports="default"
server_vnc_ports="tcp/5900:6100"
client_vnc_ports="default"
server_libvirtlive_ports="tcp/49152:49216"
client_libvirtlive_ports="default"
# local bridge address
interface cloudbr0 LAN src "${labo_ips}"
server all accept
client all accept
interface eth1 WAN src not "${labo_ips}"
protection strong
policy reject
server ssh accept
server cloudstack accept
server cloudstackweb accept
server libvirt accept
server vnc accept
server libvirtlive accept
server buildbot accept
client all accept
# zeroconf bridge address
interface cloud0 LLBR0
client all accept
server all accept
router LAN2WAN inface cloudbr0 outface eth1
masquerade
route all accept
I got a bit further by adding the interface cloud0 to the definition, but
still no joy.
I need to access the vm to run some script with a http_proxy environment
variable...
I am currently starting to run out of ideas; installing cloudstack was
fine; but I entered hell afterwards; ...
/sbin/ifconfig
cloud0 Link encap:Ethernet HWaddr fe:00:a9:fe:01:f8
inet addr:169.254.0.1 Bcast:169.254.255.255 Mask:255.255.0.0
inet6 addr: fe80::4061:7aff:fe05:f240/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:4278 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:193752 (189.2 KiB)
cloudbr0 Link encap:Ethernet HWaddr 9c:8e:99:26:6d:e4
inet addr:172.16.8.7 Bcast:172.16.8.255 Mask:255.255.255.0
inet6 addr: fe80::9e8e:99ff:fe26:6de4/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:123565 errors:0 dropped:0 overruns:0 frame:0
TX packets:58009 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:42680449 (40.7 MiB) TX bytes:236280976 (225.3 MiB)
eth0 Link encap:Ethernet HWaddr 9c:8e:99:26:6d:e4
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:930187 errors:0 dropped:871 overruns:0 frame:0
TX packets:699239 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:314733647 (300.1 MiB) TX bytes:565773302 (539.5 MiB)
eth1 Link encap:Ethernet HWaddr 9c:8e:99:26:6d:e6
inet addr:10.158.231.29 Bcast:150.158.231.255 Mask:255.255.255.0
inet6 addr: fe80::9e8e:99ff:fe26:6de6/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:676481 errors:0 dropped:0 overruns:0 frame:0
TX packets:471296 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:681352598 (649.7 MiB) TX bytes:121936012 (116.2 MiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:8578260 errors:0 dropped:0 overruns:0 frame:0
TX packets:8578260 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:4369297693 (4.0 GiB) TX bytes:4369297693 (4.0 GiB)
vnet0 Link encap:Ethernet HWaddr fe:00:a9:fe:01:f8
inet6 addr: fe80::fc00:a9ff:fefe:1f8/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:281 errors:0 dropped:3656 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:0 (0.0 B) TX bytes:14269 (13.9 KiB)
vnet1 Link encap:Ethernet HWaddr fe:30:ac:00:00:03
inet6 addr: fe80::fc30:acff:fe00:3/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:284 errors:0 dropped:47331 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:0 (0.0 B) TX bytes:23713 (23.1 KiB)
vnet2 Link encap:Ethernet HWaddr fe:c8:d4:00:00:08
inet6 addr: fe80::fcc8:d4ff:fe00:8/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:284 errors:0 dropped:47331 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:0 (0.0 B) TX bytes:23713 (23.1 KiB)
vnet3 Link encap:Ethernet HWaddr fe:00:a9:fe:03:ec
inet6 addr: fe80::fc00:a9ff:fefe:3ec/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:256 errors:0 dropped:3175 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:0 (0.0 B) TX bytes:25346 (24.7 KiB)
vnet4 Link encap:Ethernet HWaddr fe:32:ac:00:00:04
inet6 addr: fe80::fc32:acff:fe00:4/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:256 errors:0 dropped:9386 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:0 (0.0 B) TX bytes:20313 (19.8 KiB)
vnet5 Link encap:Ethernet HWaddr fe:26:52:00:00:18
inet6 addr: fe80::fc26:52ff:fe00:18/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:256 errors:0 dropped:9386 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:0 (0.0 B) TX bytes:20313 (19.8 KiB)
vnet6 Link encap:Ethernet HWaddr fe:e4:38:00:00:05
inet6 addr: fe80::fce4:38ff:fe00:5/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:256 errors:0 dropped:9386 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:0 (0.0 B) TX bytes:20313 (19.8 KiB)
/sbin/route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
0.0.0.0 150.158.231.1 0.0.0.0 UG 0 0 0 eth1
10.158.231.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0
cloud0
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0
cloudbr0
172.16.8.0 0.0.0.0 255.255.255.0 U 0 0 0
cloudbr0
Re: firewall problems and proxy problems
Posted by Marc Leeman <ma...@gmail.com>.
I've managed to get link local working my modifying the routing (disable
the fw atm too).
/sbin/route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
0.0.0.0 10.158.231.1 0.0.0.0 UG 0 0 0 eth1
150.158.231.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0
cloudbr0
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0
cloudbr0
172.16.8.0 0.0.0.0 255.255.255.0 U 0 0 0
cloudbr0
However, only to OTHER machines, I still cannot access the Secondary
storage VM by the ll address.
So LL addresses work to other physical machines, not to the VM on the same
host I am connecting from (or from another machine for that matter).
RE: firewall problems and proxy problems
Posted by Paul Angus <pa...@shapeblue.com>.
Hi Marc,
I'm not sure if this is adding to your issues, but the section of your network config where you send traffic for 172.16.0.0/16 to 172.16.8.1
doesn't look right to my eye, as traffic for your local subnet 172.16.8.0/24 would also get sent to that gateway.
address 172.16.8.7
netmask 255.255.255.0
network 172.16.8.0
broadcast 172.16.8.255
up route add -net 172.16.0.0 netmask 255.255.0.0 gw 172.16.8.1
Regards
Paul Angus
Cloud Architect
S: +44 20 3603 0540 | M: +447711418784 | T: CloudyAngus
paul.angus@shapeblue.com
-----Original Message-----
From: Marc Leeman [mailto:marc.leeman@gmail.com]
Sent: 16 October 2014 08:25
To: users@cloudstack.apache.org
Subject: Fwd: firewall problems and proxy problems
I think my mail did not get through, I can't find it in the archives; I've probably sent it too fast after subscribing
I've started out with cloudstack a couple days ago and I've hit a bit of a brick wall.
The installation is running on a system that has two network interfaces, a public one and private one (connecting to a local network).
auto eth0
iface eth0 inet manual
auto eth1
iface eth1 inet static
address 10.158.231.29
netmask 255.255.255.0
network 10.158.231.0
broadcast 10.158.231.255
gateway 10.158.231.1
auto cloudbr0
iface cloudbr0 inet static
address 172.16.8.7
netmask 255.255.255.0
network 172.16.8.0
broadcast 172.16.8.255
up route add -net 172.16.0.0 netmask 255.255.0.0 gw 172.16.8.1
bridge_ports eth0
bridge_fd 5
bridge_stp off
bridge_maxwait 1
The idea is to connect all the guests to the 172.16 network because access to multicasting devices is needed.
The systems are running on Debian wheezy, and got the system up after fixing the /etc/legal problem (echo Ubuntu) and after finding out in the logs that there is a package dependency problem on the cloudstack-manager package for 4.3
The biggest problem seems to be that host is running in a corporate network and cloudstack is auto configuring some things by downloading over http.
This does not work since a corporate proxy is required.
I downloaded the system images manually; but now it seems I need to do the same in the secondary storage vm.
The vm is running and I can get the login prompt with virsh (kvm), however there is no password I can find.
The trick with the connection to the local-link address does not work because I fear I have a networking/firewall issue. It does not matter if I disable the firewall or not, I cannot access the 169.254.x.x network.
pinging the device returns
ping 169.254.3.236
PING 169.254.3.236 (169.254.3.236) 56(84) bytes of data.
From 169.254.0.1 icmp_seq=1 Destination Host Unreachable From 169.254.0.1 icmp_seq=2 Destination Host Unreachable From 169.254.0.1 icmp_seq=3 Destination Host Unreachable
The firewall configuration is firehol
cat /etc/firehol/firehol.conf
version 5
# Accept all client traffic on any interface
FIREHOL_LOG_MODE="ULOG"
server_cloudstackweb_ports="tcp/8080"
client_cloudstackweb_ports="default"
server_buildbot_ports="tcp/8010"
client_buildbot_ports="default"
server_git_ports="tcp/9418"
client_git_ports="default"
labo_ips="172.16.0.0/16 169.254.0.0/16"
server_cloudstack_ports="tcp/1798"
client_cloudstack_ports="default"
server_libvirt_ports="tcp/16509"
client_libvirt_ports="default"
server_vnc_ports="tcp/5900:6100"
client_vnc_ports="default"
server_libvirtlive_ports="tcp/49152:49216"
client_libvirtlive_ports="default"
# local bridge address
interface cloudbr0 LAN src "${labo_ips}"
server all accept
client all accept
interface eth1 WAN src not "${labo_ips}"
protection strong
policy reject
server ssh accept
server cloudstack accept
server cloudstackweb accept
server libvirt accept
server vnc accept
server libvirtlive accept
server buildbot accept
client all accept
# zeroconf bridge address
interface cloud0 LLBR0
client all accept
server all accept
router LAN2WAN inface cloudbr0 outface eth1
masquerade
route all accept
I got a bit further by adding the interface cloud0 to the definition, but still no joy.
I need to access the vm to run some script with a http_proxy environment variable...
I am currently starting to run out of ideas; installing cloudstack was fine; but I entered hell afterwards; ...
/sbin/ifconfig
cloud0 Link encap:Ethernet HWaddr fe:00:a9:fe:01:f8
inet addr:169.254.0.1 Bcast:169.254.255.255 Mask:255.255.0.0
inet6 addr: fe80::4061:7aff:fe05:f240/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:4278 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:193752 (189.2 KiB)
cloudbr0 Link encap:Ethernet HWaddr 9c:8e:99:26:6d:e4
inet addr:172.16.8.7 Bcast:172.16.8.255 Mask:255.255.255.0
inet6 addr: fe80::9e8e:99ff:fe26:6de4/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:123565 errors:0 dropped:0 overruns:0 frame:0
TX packets:58009 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:42680449 (40.7 MiB) TX bytes:236280976 (225.3 MiB)
eth0 Link encap:Ethernet HWaddr 9c:8e:99:26:6d:e4
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:930187 errors:0 dropped:871 overruns:0 frame:0
TX packets:699239 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:314733647 (300.1 MiB) TX bytes:565773302 (539.5 MiB)
eth1 Link encap:Ethernet HWaddr 9c:8e:99:26:6d:e6
inet addr:10.158.231.29 Bcast:150.158.231.255 Mask:255.255.255.0
inet6 addr: fe80::9e8e:99ff:fe26:6de6/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:676481 errors:0 dropped:0 overruns:0 frame:0
TX packets:471296 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:681352598 (649.7 MiB) TX bytes:121936012 (116.2 MiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:8578260 errors:0 dropped:0 overruns:0 frame:0
TX packets:8578260 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:4369297693 (4.0 GiB) TX bytes:4369297693 (4.0 GiB)
vnet0 Link encap:Ethernet HWaddr fe:00:a9:fe:01:f8
inet6 addr: fe80::fc00:a9ff:fefe:1f8/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:281 errors:0 dropped:3656 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:0 (0.0 B) TX bytes:14269 (13.9 KiB)
vnet1 Link encap:Ethernet HWaddr fe:30:ac:00:00:03
inet6 addr: fe80::fc30:acff:fe00:3/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:284 errors:0 dropped:47331 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:0 (0.0 B) TX bytes:23713 (23.1 KiB)
vnet2 Link encap:Ethernet HWaddr fe:c8:d4:00:00:08
inet6 addr: fe80::fcc8:d4ff:fe00:8/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:284 errors:0 dropped:47331 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:0 (0.0 B) TX bytes:23713 (23.1 KiB)
vnet3 Link encap:Ethernet HWaddr fe:00:a9:fe:03:ec
inet6 addr: fe80::fc00:a9ff:fefe:3ec/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:256 errors:0 dropped:3175 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:0 (0.0 B) TX bytes:25346 (24.7 KiB)
vnet4 Link encap:Ethernet HWaddr fe:32:ac:00:00:04
inet6 addr: fe80::fc32:acff:fe00:4/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:256 errors:0 dropped:9386 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:0 (0.0 B) TX bytes:20313 (19.8 KiB)
vnet5 Link encap:Ethernet HWaddr fe:26:52:00:00:18
inet6 addr: fe80::fc26:52ff:fe00:18/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:256 errors:0 dropped:9386 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:0 (0.0 B) TX bytes:20313 (19.8 KiB)
vnet6 Link encap:Ethernet HWaddr fe:e4:38:00:00:05
inet6 addr: fe80::fce4:38ff:fe00:5/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:256 errors:0 dropped:9386 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:0 (0.0 B) TX bytes:20313 (19.8 KiB)
/sbin/route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
0.0.0.0 150.158.231.1 0.0.0.0 UG 0 0 0 eth1
10.158.231.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0
cloud0
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0
cloudbr0
172.16.8.0 0.0.0.0 255.255.255.0 U 0 0 0
cloudbr0
UPDATE:
I've managed to get link local working my modifying the routing (disable the fw atm too).
/sbin/route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
0.0.0.0 10.158.231.1 0.0.0.0 UG 0 0 0 eth1
150.158.231.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0
cloudbr0
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0
cloudbr0
172.16.8.0 0.0.0.0 255.255.255.0 U 0 0 0
cloudbr0
However, only to OTHER machines, I still cannot access the Secondary storage VM by the ll address.
So LL addresses work to other physical machines, not to the VM on the same host I am connecting from (or from another machine for that matter).
Find out more about ShapeBlue and our range of CloudStack related services
IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//>
CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: firewall problems and proxy problems
Posted by Marc Leeman <ma...@gmail.com>.
Just some final feedback on the setup I had.
I have an installation up and running since about a week ago. A number of
remarks I got from Sebastien allowed me to put things in perspective
(especially when trying to get the network configured). All in all, a
pretty standard configuration seems to be working for me (environment where
the VMs are directly accessible on the same network as the host), mainly
for streaming (multicast) testing.
The main problems I had were:
1. Running it on GNU/Debian is not really a problem, but due to an in
complete dependency in the package (entered a bug for that one); a binary
was not present and that caused a bit of a problem. I assume that the
binary gets pulled in in Ubuntu in some other way if no-one had a problem
with that one. Adding a dependency should fix the problem just fine with no
side effects. Even though the package is to my knowledge not in Debian, it
is pretty easy enough to pull it in from Ubuntu and install it (at the very
least, a user will get a heads up about the missing program (script btw)).
2. A reset of the installation is tricky to do and is not offered (to my
knowledge) in the system itself. This caused me to use a purge script to
clear the data bases and reset the system. On top of that (and that one is
on me); I forgot to change a password in the script that made my system
even more corrupt than it was. Finally; it was a remark of Daan that
pointed out the problem to me (not the root cause, but once I noticed the
inconsistency in the DB Daan pointed out, it was pretty clear what could
have been the culprit).
mleeman@zee:~$ cat bin/cloudstack-reset.sh
#!/bin/sh
/etc/init.d/cloudstack-management stop
mysql -p<my-sql-password> -e 'drop database cloud'
mysql -p<my-sql-password> -e 'drop database cloud_usage'
cloudstack-setup-databases cloud:<another-password>@localhost
--deploy-as=root:<server-root-password>
rm -rf /var/log/cloud/management/*
cloudstack-setup-management
/etc/init.d/cloudstack-management start
3. What really caused a problem and confusion is that I was not aware that,
during the installation part of the install is downloaded. That in itself
is not a problem; but the combination that there is no feedback on this
process AND that my system is in an environment where the use of a proxy is
mandatory; let the install fail silently...
After someone dropped a remark on the 'download during install'; I pieced
it together and started logging in to the systems and by setting the
http_proxy environment variable, I got everything up and running.
Thanks for all the pointers, the help and the patience!
Re: firewall problems and proxy problems
Posted by Marc Leeman <ma...@gmail.com>.
> I cleared the database and re-did the installation and all seems to be
> > working now.
> >
> > Thanks for the help.
> >
> > It seems that something must have been off during the initial install.
>
> Hi Marc, I was at the guy at the booth :)
>
Yeah, thank you; you were very patient. Thank you for helping me out in
Cloudstack; it was pretty new to me :-)
> In my angst to get things working I probably messed up the installed when
> upgrading rather quickly to 4.3.
>
> In any case, glad it's working now, please confirm that you can launch
> instances.
>
Truth be said, the config I am running now is probably not what I was
initially looking but I thought go for the out-of-the-box installation and
take it from there.
I have assigned the internal addresses to be a range of the external
network, but I noticed that though I see the connections from the VMs
coming from the correct IP (when e.g. ssh-ing to an external server); but I
cannot connect back (ingress rules are very strict).
Is there any resource you'd recommend to convert the basic guest network to
a configuration where all the VMs are exposed to the network as if they
were machines sitting on that network?
One thing I've also noticed when deploying test software on the cloud is
that I cannot use DHCP:
The machines are installed by default with a static IP (software needs to
be predictable and on the network). When changing over the static IP to a
DHCP address; I see that the system gets a DHCP address that seems (need to
verify this, but I would need to check the DHCP server on the switch to
which I do not have access atm) to come from the network. However the
system is muted on the network.
Only when I set the IP to the IP it got assigned by cloudstack can I access
the network.
> I have kept track of your bugs for debian packaging and we will try to
> address those.
>
>
Re: firewall problems and proxy problems
Posted by sebgoa <ru...@gmail.com>.
On Oct 17, 2014, at 11:01 AM, Marc Leeman <ma...@gmail.com> wrote:
> I cleared the database and re-did the installation and all seems to be
> working now.
>
> Thanks for the help.
>
> It seems that something must have been off during the initial install.
Hi Marc, I was at the guy at the booth :)
In my angst to get things working I probably messed up the installed when upgrading rather quickly to 4.3.
In any case, glad it's working now, please confirm that you can launch instances.
I have kept track of your bugs for debian packaging and we will try to address those.
-sebastien
Re: firewall problems and proxy problems
Posted by Marc Leeman <ma...@gmail.com>.
I cleared the database and re-did the installation and all seems to be
working now.
Thanks for the help.
It seems that something must have been off during the initial install.
Re: firewall problems and proxy problems
Posted by Daan Hoogland <da...@gmail.com>.
ok, then I'd re-init the db and try again, Keeping Pauls remark in mind. I
usually mess up my ip space and find myself stuck in a mess. Paul didn't
you post a sample layout somewhere?
On Fri, Oct 17, 2014 at 10:38 AM, Marc Leeman <ma...@gmail.com> wrote:
> On 16 October 2014 17:53, Daan Hoogland <da...@gmail.com> wrote:
>
> > On Thu, Oct 16, 2014 at 5:33 PM, Marc Leeman <ma...@gmail.com>
> > wrote:
> >
> > > Is there some inconsistency in the database?
> > >
> > This might well be. To check this look at mshost and mshost_peer.
> >
> > Are you upgrading? Is this a test setup?
> >
> >
> >
> It is my first setup :-)
>
> mysql> select * from mshost_peer;
>
> +----+--------------+-------------+---------------+------------+---------------------+
> | id | owner_mshost | peer_mshost | peer_runid | peer_state |
> last_update |
>
> +----+--------------+-------------+---------------+------------+---------------------+
> | 1 | 1 | 1 | 1413300325989 | Up | 2014-10-14
> 15:25:55 |
> | 2 | 2 | 2 | 1413301475290 | Up | 2014-10-14
> 15:44:48 |
> | 16 | 3 | 3 | 1413473252587 | Up | 2014-10-16
> 15:27:45 |
>
> +----+--------------+-------------+---------------+------------+---------------------+
> 3 rows in set (0.00 sec)
>
> mysql> select * from mshost;
>
> +----+-----------------+---------------+------+-------+---------+------------+--------------+---------------------+---------+-------------+
> | id | msid | runid | name | state | version |
> service_ip | service_port | last_update | removed | alert_count |
>
> +----+-----------------+---------------+------+-------+---------+------------+--------------+---------------------+---------+-------------+
> | 1 | 172136268721636 | 1413300325989 | zee | Up | 4.3.1 |
> 172.16.8.7 | 9090 | 2014-10-14 15:41:40 | NULL | 0 |
> | 2 | 72986426470976 | 1413301475290 | zee | Up | 4.3.1 |
> 172.16.8.7 | 9090 | 2014-10-14 16:18:24 | NULL | 0 |
> | 3 | 279278805451256 | 1413473252587 | zee | Up | 4.3.1 |
> 172.16.8.7 | 9090 | 2014-10-17 08:37:41 | NULL | 0 |
>
> +----+-----------------+---------------+------+-------+---------+------------+--------------+---------------------+---------+-------------+
> 3 rows in set (0.00 sec)
>
--
Daan
Re: firewall problems and proxy problems
Posted by Marc Leeman <ma...@gmail.com>.
On 16 October 2014 17:53, Daan Hoogland <da...@gmail.com> wrote:
> On Thu, Oct 16, 2014 at 5:33 PM, Marc Leeman <ma...@gmail.com>
> wrote:
>
> > Is there some inconsistency in the database?
> >
> This might well be. To check this look at mshost and mshost_peer.
>
> Are you upgrading? Is this a test setup?
>
>
>
It is my first setup :-)
mysql> select * from mshost_peer;
+----+--------------+-------------+---------------+------------+---------------------+
| id | owner_mshost | peer_mshost | peer_runid | peer_state |
last_update |
+----+--------------+-------------+---------------+------------+---------------------+
| 1 | 1 | 1 | 1413300325989 | Up | 2014-10-14
15:25:55 |
| 2 | 2 | 2 | 1413301475290 | Up | 2014-10-14
15:44:48 |
| 16 | 3 | 3 | 1413473252587 | Up | 2014-10-16
15:27:45 |
+----+--------------+-------------+---------------+------------+---------------------+
3 rows in set (0.00 sec)
mysql> select * from mshost;
+----+-----------------+---------------+------+-------+---------+------------+--------------+---------------------+---------+-------------+
| id | msid | runid | name | state | version |
service_ip | service_port | last_update | removed | alert_count |
+----+-----------------+---------------+------+-------+---------+------------+--------------+---------------------+---------+-------------+
| 1 | 172136268721636 | 1413300325989 | zee | Up | 4.3.1 |
172.16.8.7 | 9090 | 2014-10-14 15:41:40 | NULL | 0 |
| 2 | 72986426470976 | 1413301475290 | zee | Up | 4.3.1 |
172.16.8.7 | 9090 | 2014-10-14 16:18:24 | NULL | 0 |
| 3 | 279278805451256 | 1413473252587 | zee | Up | 4.3.1 |
172.16.8.7 | 9090 | 2014-10-17 08:37:41 | NULL | 0 |
+----+-----------------+---------------+------+-------+---------+------------+--------------+---------------------+---------+-------------+
3 rows in set (0.00 sec)
Re: firewall problems and proxy problems
Posted by Daan Hoogland <da...@gmail.com>.
On Thu, Oct 16, 2014 at 5:33 PM, Marc Leeman <ma...@gmail.com> wrote:
> Is there some inconsistency in the database?
>
This might well be. To check this look at mshost and mshost_peer.
Are you upgrading? Is this a test setup?
Daan
Re: firewall problems and proxy problems
Posted by Marc Leeman <ma...@gmail.com>.
Is there some inconsistency in the database?
At the end of the stack dump:
2014-10-16 17:27:36,912 INFO [c.c.c.ClusterManagerImpl] (main:null)
Detected that another management node with the same IP 172.16.8.7 is
considered as running in DB, however it is not pingable, we will continue
cluster initialization with this management server node
Re: firewall problems and proxy problems
Posted by Marc Leeman <ma...@gmail.com>.
The port is being started by the manager. The error is happening during the
restart of the management server
2014-10-16 17:27:36,909 ERROR [c.c.c.ClusterManagerImpl] (main:null) Unable
to ping management server at 172.16.8.7:9090 due to ConnectException
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:525)
at
com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1122)
at
com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1091)
at
com.cloud.cluster.ClusterManagerImpl.checkConflicts(ClusterManagerImpl.java:1169)
at
com.cloud.cluster.ClusterManagerImpl.configure(ClusterManagerImpl.java:1058)
.....
However, connecting afterwards with telnet is fine:
telnet 172.16.8.7 9090
Trying 172.16.8.7...
Connected to 172.16.8.7.
Escape character is '^]'.
^]
root@zee:/var/log/cloudstack/management# ps -ef |grep 32427
Non-standard uts for running kernel:
release Linux version 3.14-0.bpo.1-amd64 (debian-kernel@lists.debian.org)
(gcc version 4.6.3 (Debian 4.6.3-14) ) #1 SMP Debian 3.14.4-1~bpo70+1
(2014-05-14)
=3.14.0 gives version code 200192
root 4553 4514 0 17:26 pts/6 00:00:00 grep 32427
cloud 32427 32424 1 15:26 ? 00:01:43 jsvc.exec -user cloud -cp
/usr/share/java/commons-daemon.jar:/usr/share/cloudstack-management/bin/bootstrap.jar:/etc/cloudstack/management:/usr/share/cloudstack-management/setup
-outfile SYSLOG -errfile SYSLOG -pidfile /var/run/cloudstack-management.pid
-Djava.awt.headless=true -Dcom.sun.management.jmxremote=false -Xmx2g
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/var/log/cloudstack/management/ -XX:PermSize=512M
-XX:MaxPermSize=800m
-Djava.endorsed.dirs=/usr/share/cloudstack-management/endorsed
-Dcatalina.base=/usr/share/cloudstack-management
-Dcatalina.home=/usr/share/cloudstack-management
-Djava.io.tmpdir=/tmp/cloudstack-management-temp
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
-Djava.util.logging.config.file=/usr/share/cloudstack-management/conf/logging.properties
org.apache.catalina.startup.Bootstrap
root@zee:/var/log/cloudstack/management# /etc/init.d/cloudstack-management
restart
[ ok ] Stopping CloudStack-specific Tomcat servlet engine:
cloudstack-management.
[ ok ] Starting CloudStack-specific Tomcat servlet engine:
cloudstack-management.
root@zee:/var/log/cloudstack/management# netstat -tulpn |grep 9090
root@zee:/var/log/cloudstack/management# netstat -tulpn |grep 9090
root@zee:/var/log/cloudstack/management# netstat -tulpn |grep 9090
root@zee:/var/log/cloudstack/management# netstat -tulpn |grep 9090
root@zee:/var/log/cloudstack/management# netstat -tulpn |grep 9090
tcp6 0 0 :::9090 :::*
LISTEN 4643/jsvc.exec
root@zee:/var/log/cloudstack/management# ps -ef |grep 4643
Non-standard uts for running kernel:
release Linux version 3.14-0.bpo.1-amd64 (debian-kernel@lists.debian.org)
(gcc version 4.6.3 (Debian 4.6.3-14) ) #1 SMP Debian 3.14.4-1~bpo70+1
(2014-05-14)
=3.14.0 gives version code 200192
cloud 4643 4640 99 17:27 ? 00:00:52 jsvc.exec -user cloud -cp
/usr/share/java/commons-daemon.jar:/usr/share/cloudstack-management/bin/bootstrap.jar:/etc/cloudstack/management:/usr/share/cloudstack-management/setup
-outfile SYSLOG -errfile SYSLOG -pidfile /var/run/cloudstack-management.pid
-Djava.awt.headless=true -Dcom.sun.management.jmxremote=false -Xmx2g
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/var/log/cloudstack/management/ -XX:PermSize=512M
-XX:MaxPermSize=800m
-Djava.endorsed.dirs=/usr/share/cloudstack-management/endorsed
-Dcatalina.base=/usr/share/cloudstack-management
-Dcatalina.home=/usr/share/cloudstack-management
-Djava.io.tmpdir=/tmp/cloudstack-management-temp
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
-Djava.util.logging.config.file=/usr/share/cloudstack-management/conf/logging.properties
org.apache.catalina.startup.Bootstrap
root 4999 4514 0 17:27 pts/6 00:00:00 grep 4643
On 16 October 2014 17:16, Marc Leeman <ma...@gmail.com> wrote:
>
> No:
>
> I killed the process at the boot the day before yesterday (thanks to the
> cloudstack guys that I made to miss the boot crawl): there was indeed a
> problem with that; NO vsm were starting.
>
> After killing the old stuck programs; we got further and after downloading
> the svm images manually with the http_proxy var; the svms started.
>
> However then we hit the bridging problem.
>
>
>
>
Re: firewall problems and proxy problems
Posted by Daan Hoogland <da...@gmail.com>.
btw
On Thu, Oct 16, 2014 at 5:46 PM, Daan Hoogland <da...@gmail.com>
wrote:
> no, this is restoring an instance AFAICT
>
> On Thu, Oct 16, 2014 at 5:43 PM, Marc Leeman <ma...@gmail.com>
> wrote:
>
>> > Try to restore your system:
>
> this was part of the log!
Not an advice
--
Daan
Re: firewall problems and proxy problems
Posted by Daan Hoogland <da...@gmail.com>.
no, this is restoring an instance AFAICT
On Thu, Oct 16, 2014 at 5:43 PM, Marc Leeman <ma...@gmail.com> wrote:
> > Try to restore your system:
> >
>
>
>
> You are talking about this procedure right?
>
>
> https://support.getcloudservices.com/entries/21982811-CloudStack-Restore-Cloud-Server
>
--
Daan
Re: firewall problems and proxy problems
Posted by Marc Leeman <ma...@gmail.com>.
> Try to restore your system:
>
You are talking about this procedure right?
https://support.getcloudservices.com/entries/21982811-CloudStack-Restore-Cloud-Server
Re: firewall problems and proxy problems
Posted by Marc Leeman <ma...@gmail.com>.
> configure your system:
> Checking KVM...[Failed]
> Please enable KVM on this machine
I opened a bug on this one: there is a packaging problem: a dependency is
missing on cpu_utils. We got that one figured out too :-) The script is
using kvm-ok at some point.
Re: firewall problems and proxy problems
Posted by Daan Hoogland <da...@gmail.com>.
I don't see a mention of cloud0 in the log. It might come from within the
agent itself trying to restore some defaults after a reset or delete.
I do note some other perculiar thing:
Around the following log fragment a lot of nulls are in the file. Also the
meassage is not filling one with hope: note that the nuls are output from
'lsmod|grep kvm'. Is this the problem you tackeled in Dusseldorf?
2014-10-14 17:59:05,664 DEBUG [c.c.u.s.SSHCmdHelper]
(catalina-exec-11:ctx-ba00ae78 ctx-b1389dca) Executing cmd:
cloudstack-setup-agent -m 10.158.231.29 -z 1 -p 1 -c 1 -g
4a9064dc-4e61-3c36-949f-d474d64f5e56 -a --pubNic=cloudbr0 --prvNic=cloudbr0
--guestNic=cloudbr0
2014-10-14 17:59:06,774 DEBUG [c.c.u.s.SSHCmdHelper]
(catalina-exec-11:ctx-ba00ae78 ctx-b1389dca) cloudstack-setup-agent -m
10.158.231.29 -z 1 -p 1 -c 1 -g 4a9064dc-4e61-3c36-949f-d474d64f5e56 -a
--pubNic=cloudbr0 --prvNic=cloudbr0 --guestNic=cloudbr0 output:Starting to
configure your system:
Checking KVM...[Failed]
Please enable KVM on this machine
Try to restore your system:
On Thu, Oct 16, 2014 at 5:16 PM, Marc Leeman <ma...@gmail.com> wrote:
> No:
>
> I killed the process at the boot the day before yesterday (thanks to the
> cloudstack guys that I made to miss the boot crawl): there was indeed a
> problem with that; NO vsm were starting.
>
> After killing the old stuck programs; we got further and after downloading
> the svm images manually with the http_proxy var; the svms started.
>
> However then we hit the bridging problem.
>
--
Daan
Re: firewall problems and proxy problems
Posted by Marc Leeman <ma...@gmail.com>.
No:
I killed the process at the boot the day before yesterday (thanks to the
cloudstack guys that I made to miss the boot crawl): there was indeed a
problem with that; NO vsm were starting.
After killing the old stuck programs; we got further and after downloading
the svm images manually with the http_proxy var; the svms started.
However then we hit the bridging problem.
Re: firewall problems and proxy problems
Posted by Daan Hoogland <da...@gmail.com>.
Ok, let us know if this solves your issue.
Op 16 okt. 2014 17:07 schreef "Marc Leeman" <ma...@gmail.com>:
> $ sudo netstat -tulpn |grep 9090
> tcp6 0 0 :::9090 :::*
> LISTEN 32427/jsvc.exec
>
> That is cloudstack.
>
> But then I thought of it; I asked someone on the cloudstack stand at
> Dusseldorf; and the 4.0 was upgraded to 4.3. It seems that there was
> something wrong with the packages there: the old 4.0 manager/agents were
> not properly shutdown before they were replaced by the cloudstack version
> of the packages (the previous were called cloud).
>
> It might be this that you are noticing.
>
> I killed the processes manually and restarted the cloudstack application.
>
>
Re: firewall problems and proxy problems
Posted by Marc Leeman <ma...@gmail.com>.
$ sudo netstat -tulpn |grep 9090
tcp6 0 0 :::9090 :::*
LISTEN 32427/jsvc.exec
That is cloudstack.
But then I thought of it; I asked someone on the cloudstack stand at
Dusseldorf; and the 4.0 was upgraded to 4.3. It seems that there was
something wrong with the packages there: the old 4.0 manager/agents were
not properly shutdown before they were replaced by the cloudstack version
of the packages (the previous were called cloud).
It might be this that you are noticing.
I killed the processes manually and restarted the cloudstack application.
Re: firewall problems and proxy problems
Posted by Daan Hoogland <da...@gmail.com>.
Marc from your logs it seems like there is a port conflict on your host.
Can you check on that? port 9090
first I see
2014-10-14 17:25:54,181 INFO [c.c.c.ClusterManagerImpl] (main:null)
Management server (host id : 1) is being started at 172.16.8.7:9090
2014-10-14 17:25:54,186 INFO [c.c.c.ClusterManagerImpl] (main:null)
Cluster manager was started successfully
2014-10-14 17:25:54,205 ERROR [c.c.u.n.NioConnection]
(AgentManager-Selector:null) Unable to initialize the threads.
java.net.BindException: Address already in use
at sun.nio.ch.Net.bind(Native Method)
at
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:137)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:77)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:70)
at com.cloud.utils.nio.NioServer.init(NioServer.java:50)
at com.cloud.utils.nio.NioConnection.run(NioConnection.java:108)
at java.lang.Thread.run(Thread.java:636)
and then
2014-10-14 18:18:46,695 ERROR [c.c.c.ClusterManagerImpl] (main:null) Unable
to ping management server at 172.16.8.7:9090 due to ConnectException
are you installing a clustered set of management servers on a single host?
(still looking)
On Thu, Oct 16, 2014 at 4:33 PM, Marc Leeman <ma...@gmail.com> wrote:
> On 16 October 2014 16:10, Daan Hoogland <da...@gmail.com> wrote:
> >
> >> My eye is not much more trained at this table but I'll have a peek in
> the
> >> code. It seems to e that cloudstack should be configuring the bridge
> >> instead of the cloud0 device. I have no idea if this is a config err of
> >> yours or a bug.
> >>
> >> My assuptions:
> >> you are using kvm as hypervisor
> >>
> >
> > ack.
> >
> >
> >> you are running version 4.3.1
> >>
> >
> > dpkg -l |grep cloudstack
> > ii cloudstack-agent 4.3.1
> > all CloudStack agent
> > ii cloudstack-cli 4.3.1
> > all The CloudStack CLI called CloudMonkey
> > ii cloudstack-common 4.3.1
> > all A common package which contains files which are shared by
> > several CloudStack packages
> > ii cloudstack-management 4.3.1
> > all CloudStack server library
> >
> >
> >
> >>
> >> any other info? I might want to see your logs during install and zone
> >> definition later, you still have them?
> >>
> >
> >
> > I still have all the log files on the server.
> >
> > The VMs should be connected to the same 172.16.x.x network as the host
> > because of access to multicast devices is needed.
> >
> > For the rest, we've been looking around on the system until it started
> > dawning on us that there were some scripts trying to download things and
> > got blocked by not using the proxy.
> >
> > We could not figure out whay the SVMs said starting; but got stuck there.
> >
> > I downloaded the SVM images with the script and the VMS started.
> >
>
> The log file is
>
>
> http://crichton.supercomputerrobot.com/~marc/downloads/management-server.log.2014-10-14.gz
>
> and the firewall has been disabled already; so it should not be dependent
> on this.
>
--
Daan
Re: firewall problems and proxy problems
Posted by Marc Leeman <ma...@gmail.com>.
On 16 October 2014 16:10, Daan Hoogland <da...@gmail.com> wrote:
>
>> My eye is not much more trained at this table but I'll have a peek in the
>> code. It seems to e that cloudstack should be configuring the bridge
>> instead of the cloud0 device. I have no idea if this is a config err of
>> yours or a bug.
>>
>> My assuptions:
>> you are using kvm as hypervisor
>>
>
> ack.
>
>
>> you are running version 4.3.1
>>
>
> dpkg -l |grep cloudstack
> ii cloudstack-agent 4.3.1
> all CloudStack agent
> ii cloudstack-cli 4.3.1
> all The CloudStack CLI called CloudMonkey
> ii cloudstack-common 4.3.1
> all A common package which contains files which are shared by
> several CloudStack packages
> ii cloudstack-management 4.3.1
> all CloudStack server library
>
>
>
>>
>> any other info? I might want to see your logs during install and zone
>> definition later, you still have them?
>>
>
>
> I still have all the log files on the server.
>
> The VMs should be connected to the same 172.16.x.x network as the host
> because of access to multicast devices is needed.
>
> For the rest, we've been looking around on the system until it started
> dawning on us that there were some scripts trying to download things and
> got blocked by not using the proxy.
>
> We could not figure out whay the SVMs said starting; but got stuck there.
>
> I downloaded the SVM images with the script and the VMS started.
>
The log file is
http://crichton.supercomputerrobot.com/~marc/downloads/management-server.log.2014-10-14.gz
and the firewall has been disabled already; so it should not be dependent
on this.
Re: firewall problems and proxy problems
Posted by Daan Hoogland <da...@gmail.com>.
My eye is not much more trained at this table but I'll have a peek in the
code. It seems to e that cloudstack should be configuring the bridge
instead of the cloud0 device. I have no idea if this is a config err of
yours or a bug.
My assuptions:
you are using kvm as hypervisor
you are running version 4.3.1
any other info? I might want to see your logs during install and zone
definition later, you still have them?
On Thu, Oct 16, 2014 at 3:55 PM, Marc Leeman <ma...@gmail.com> wrote:
> yes.
>
> But there must be something in the configuration that causes problems. I
> haven't played around too much with bridging.
>
> If I delete the route to cloud0 and replace it to the cloudbr0, I can
> access the LL devices on the network; after resetting the cloudstack
> manager; I cannot. It seems to my (untrained and unexpertised eye) that
> there is some link or something missing between cloud0 and cloudbr0
>
--
Daan
Re: firewall problems and proxy problems
Posted by Marc Leeman <ma...@gmail.com>.
yes.
But there must be something in the configuration that causes problems. I
haven't played around too much with bridging.
If I delete the route to cloud0 and replace it to the cloudbr0, I can
access the LL devices on the network; after resetting the cloudstack
manager; I cannot. It seems to my (untrained and unexpertised eye) that
there is some link or something missing between cloud0 and cloudbr0
Re: firewall problems and proxy problems
Posted by Daan Hoogland <da...@gmail.com>.
Marc,
The only difference I see is de vice cloudbr0 becoming cloud0, in the third
line. Am I correct?
On Thu, Oct 16, 2014 at 3:28 PM, Marc Leeman <ma...@gmail.com> wrote:
>
> OK, there is something off here:
>
> This is my routing table where I can ping another physical host with an
> link-local address
>
> route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use
> Iface
> 0.0.0.0 10.158.231.1 0.0.0.0 UG 0 0 0 eth1
> 10.158.231.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
> 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0
> cloudbr0
> 172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0
> cloudbr0
> 172.16.8.0 0.0.0.0 255.255.255.0 U 0 0 0
> cloudbr0
>
>
> If I restart the cloudstack-agent; the routing table has changed to:
>
> route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use
> Iface
> 0.0.0.0 10.158.231.1 0.0.0.0 UG 0 0 0 eth1
> 10.158.231.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
> 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0
> cloud0
> 172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0
> cloudbr0
> 172.16.8.0 0.0.0.0 255.255.255.0 U 0 0 0
> cloudbr0
>
> and link-local devices cannot be accessed.
>
> In both cases, I cannot access the system VMs with ssh.
>
--
Daan
Re: firewall problems and proxy problems
Posted by Marc Leeman <ma...@gmail.com>.
OK, there is something off here:
This is my routing table where I can ping another physical host with an
link-local address
route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
0.0.0.0 10.158.231.1 0.0.0.0 UG 0 0 0 eth1
10.158.231.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0
cloudbr0
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0
cloudbr0
172.16.8.0 0.0.0.0 255.255.255.0 U 0 0 0
cloudbr0
If I restart the cloudstack-agent; the routing table has changed to:
route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
0.0.0.0 10.158.231.1 0.0.0.0 UG 0 0 0 eth1
10.158.231.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0
cloud0
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0
cloudbr0
172.16.8.0 0.0.0.0 255.255.255.0 U 0 0 0
cloudbr0
and link-local devices cannot be accessed.
In both cases, I cannot access the system VMs with ssh.
Fwd: firewall problems and proxy problems
Posted by Marc Leeman <ma...@gmail.com>.
I think my mail did not get through, I can't find it in the archives; I've
probably sent it too fast after subscribing
I've started out with cloudstack a couple days ago and I've hit a bit of a
brick wall.
The installation is running on a system that has two network interfaces, a
public one and private one (connecting to a local network).
auto eth0
iface eth0 inet manual
auto eth1
iface eth1 inet static
address 10.158.231.29
netmask 255.255.255.0
network 10.158.231.0
broadcast 10.158.231.255
gateway 10.158.231.1
auto cloudbr0
iface cloudbr0 inet static
address 172.16.8.7
netmask 255.255.255.0
network 172.16.8.0
broadcast 172.16.8.255
up route add -net 172.16.0.0 netmask 255.255.0.0 gw 172.16.8.1
bridge_ports eth0
bridge_fd 5
bridge_stp off
bridge_maxwait 1
The idea is to connect all the guests to the 172.16 network because access
to multicasting devices is needed.
The systems are running on Debian wheezy, and got the system up after
fixing the /etc/legal problem (echo Ubuntu) and after finding out in the
logs that there is a package dependency problem on the cloudstack-manager
package for 4.3
The biggest problem seems to be that host is running in a corporate network
and cloudstack is auto configuring some things by downloading over http.
This does not work since a corporate proxy is required.
I downloaded the system images manually; but now it seems I need to do the
same in the secondary storage vm.
The vm is running and I can get the login prompt with virsh (kvm), however
there is no password I can find.
The trick with the connection to the local-link address does not work
because I fear I have a networking/firewall issue. It does not matter if I
disable the firewall or not, I cannot access the 169.254.x.x network.
pinging the device returns
ping 169.254.3.236
PING 169.254.3.236 (169.254.3.236) 56(84) bytes of data.
>From 169.254.0.1 icmp_seq=1 Destination Host Unreachable
>From 169.254.0.1 icmp_seq=2 Destination Host Unreachable
>From 169.254.0.1 icmp_seq=3 Destination Host Unreachable
The firewall configuration is firehol
cat /etc/firehol/firehol.conf
version 5
# Accept all client traffic on any interface
FIREHOL_LOG_MODE="ULOG"
server_cloudstackweb_ports="tcp/8080"
client_cloudstackweb_ports="default"
server_buildbot_ports="tcp/8010"
client_buildbot_ports="default"
server_git_ports="tcp/9418"
client_git_ports="default"
labo_ips="172.16.0.0/16 169.254.0.0/16"
server_cloudstack_ports="tcp/1798"
client_cloudstack_ports="default"
server_libvirt_ports="tcp/16509"
client_libvirt_ports="default"
server_vnc_ports="tcp/5900:6100"
client_vnc_ports="default"
server_libvirtlive_ports="tcp/49152:49216"
client_libvirtlive_ports="default"
# local bridge address
interface cloudbr0 LAN src "${labo_ips}"
server all accept
client all accept
interface eth1 WAN src not "${labo_ips}"
protection strong
policy reject
server ssh accept
server cloudstack accept
server cloudstackweb accept
server libvirt accept
server vnc accept
server libvirtlive accept
server buildbot accept
client all accept
# zeroconf bridge address
interface cloud0 LLBR0
client all accept
server all accept
router LAN2WAN inface cloudbr0 outface eth1
masquerade
route all accept
I got a bit further by adding the interface cloud0 to the definition, but
still no joy.
I need to access the vm to run some script with a http_proxy environment
variable...
I am currently starting to run out of ideas; installing cloudstack was
fine; but I entered hell afterwards; ...
/sbin/ifconfig
cloud0 Link encap:Ethernet HWaddr fe:00:a9:fe:01:f8
inet addr:169.254.0.1 Bcast:169.254.255.255 Mask:255.255.0.0
inet6 addr: fe80::4061:7aff:fe05:f240/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:4278 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:193752 (189.2 KiB)
cloudbr0 Link encap:Ethernet HWaddr 9c:8e:99:26:6d:e4
inet addr:172.16.8.7 Bcast:172.16.8.255 Mask:255.255.255.0
inet6 addr: fe80::9e8e:99ff:fe26:6de4/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:123565 errors:0 dropped:0 overruns:0 frame:0
TX packets:58009 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:42680449 (40.7 MiB) TX bytes:236280976 (225.3 MiB)
eth0 Link encap:Ethernet HWaddr 9c:8e:99:26:6d:e4
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:930187 errors:0 dropped:871 overruns:0 frame:0
TX packets:699239 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:314733647 (300.1 MiB) TX bytes:565773302 (539.5 MiB)
eth1 Link encap:Ethernet HWaddr 9c:8e:99:26:6d:e6
inet addr:10.158.231.29 Bcast:150.158.231.255 Mask:255.255.255.0
inet6 addr: fe80::9e8e:99ff:fe26:6de6/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:676481 errors:0 dropped:0 overruns:0 frame:0
TX packets:471296 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:681352598 (649.7 MiB) TX bytes:121936012 (116.2 MiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:8578260 errors:0 dropped:0 overruns:0 frame:0
TX packets:8578260 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:4369297693 (4.0 GiB) TX bytes:4369297693 (4.0 GiB)
vnet0 Link encap:Ethernet HWaddr fe:00:a9:fe:01:f8
inet6 addr: fe80::fc00:a9ff:fefe:1f8/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:281 errors:0 dropped:3656 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:0 (0.0 B) TX bytes:14269 (13.9 KiB)
vnet1 Link encap:Ethernet HWaddr fe:30:ac:00:00:03
inet6 addr: fe80::fc30:acff:fe00:3/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:284 errors:0 dropped:47331 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:0 (0.0 B) TX bytes:23713 (23.1 KiB)
vnet2 Link encap:Ethernet HWaddr fe:c8:d4:00:00:08
inet6 addr: fe80::fcc8:d4ff:fe00:8/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:284 errors:0 dropped:47331 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:0 (0.0 B) TX bytes:23713 (23.1 KiB)
vnet3 Link encap:Ethernet HWaddr fe:00:a9:fe:03:ec
inet6 addr: fe80::fc00:a9ff:fefe:3ec/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:256 errors:0 dropped:3175 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:0 (0.0 B) TX bytes:25346 (24.7 KiB)
vnet4 Link encap:Ethernet HWaddr fe:32:ac:00:00:04
inet6 addr: fe80::fc32:acff:fe00:4/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:256 errors:0 dropped:9386 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:0 (0.0 B) TX bytes:20313 (19.8 KiB)
vnet5 Link encap:Ethernet HWaddr fe:26:52:00:00:18
inet6 addr: fe80::fc26:52ff:fe00:18/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:256 errors:0 dropped:9386 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:0 (0.0 B) TX bytes:20313 (19.8 KiB)
vnet6 Link encap:Ethernet HWaddr fe:e4:38:00:00:05
inet6 addr: fe80::fce4:38ff:fe00:5/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:256 errors:0 dropped:9386 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:0 (0.0 B) TX bytes:20313 (19.8 KiB)
/sbin/route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
0.0.0.0 150.158.231.1 0.0.0.0 UG 0 0 0 eth1
10.158.231.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0
cloud0
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0
cloudbr0
172.16.8.0 0.0.0.0 255.255.255.0 U 0 0 0
cloudbr0
UPDATE:
I've managed to get link local working my modifying the routing (disable
the fw atm too).
/sbin/route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
0.0.0.0 10.158.231.1 0.0.0.0 UG 0 0 0 eth1
150.158.231.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0
cloudbr0
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0
cloudbr0
172.16.8.0 0.0.0.0 255.255.255.0 U 0 0 0
cloudbr0
However, only to OTHER machines, I still cannot access the Secondary
storage VM by the ll address.
So LL addresses work to other physical machines, not to the VM on the same
host I am connecting from (or from another machine for that matter).