You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Francesco Maria Magnini <fm...@gmail.com> on 2013/12/04 16:53:15 UTC
Storage/Console SSVM loose connectivity (can't ping them anymore)
after creating the first guest instance
I'm experiencing problems in Cloudstack 4.2 installation on both Ubuntu
Server 12.04 and 13.10.
- Installed Cloudstack Management Controller and a KVM host in two
separate boxes
- Cloudstack installations went fine
- Created a basic networking zone, all is green, storage ok, vvms ok
At this point, I'm able to ping physical hosts, Storage and Proxy SSVM
public IP Address.
- Created instance with basic template of CentOS, fine
- Automatically created System Router
At this point I'm still able to ping physical hosts, but no longer Storage
SSMV, Proxy SSVM and the CentOS instance created a while ago.
No errors, all is green, all processes running fine, just connectivity
issue on the public network side of Proxy/Storage VVMS and Instances.
*IMPORTANT* I've configured Cloudstack in CentOS enviroment before testing
Ubuntu Server, with both Basic and Advanced Networking (VLAN separation)
without problems.
Security group is already configured with ICMP/SSH rules for inbound.
Any ideas? Thanks
Re: Storage/Console SSVM loose connectivity (can't ping them anymore)
after creating the first guest instance
Posted by Francesco Maria Magnini <fm...@gmail.com>.
Below you can find the network scenario
- Basic Networking Zone
- Management Controller (Ubuntu Server 12.04 LTS): 10.77.0.11
- KVM Host (Ubuntu Server 13.10, same issue in Ubuntu Server 12.04 LTS):
10.77.0.21
- POD IP Range: 10.77.0.41 - 10.77.0.60
- Guest Network: 10.77.0.61 - 10.77.0.80
- Router serving Cloudstack LAN: 10.77.0.1
- Console Proxy VM: public 10.77.0.62, private 10.77.0.57, link-local
169.254.1.168
- Secondary Storage VM: public 10.77.0.61, private 10.77.0.42, link-local
169.254.2.233
- Instance01 VM: 10.77.0.63
- Virtual Router VM: 10.77.0.64, link-local 169.254.2.165
KVM Host Networking (one NIC only, as tested with CentOS)
lo inet:127.0.0.1 Maschera:255.0.0.0
cloud0 inet:169.254.0.1 Bcast:169.254.255.255 Maschera:255.255.0.0
cloudbr0 inet:10.77.0.21 Bcast:10.77.0.255 Maschera:255.255.255.0
virbr0 inet:192.168.122.1 Bcast:192.168.122.255 Maschera:255.255.255.0
eth0 -
vnet0 -
vnet1 -
vnet2 -
vnet3 -
vnet4 -
vnet5 -
vnet6 -
TEST 1
Pinging from router (10.77.0.1) the Console Proxy VM Public IP 10.77.0.62:
*** KO ***
- TCPUDUMP on KVM Host, ICMP reaching KVM HOST, seeing ICMP requests
passing through physical eth0 and bridge cloudbr0
- TCPDUMP on Console Proxy VM (connecting with virsh console from KVM Host)
shows no packets coming on any interface
Pinging from KVM Host (10.77.0.21) the Console Proxy VM Public IP 10.77.0.62:
*** OK ***
Basically tests show that ICMP coming from outside KVM Host are blocked,
pinging the SSVM from inside the KVM Host is ok.
On Thu, Dec 5, 2013 at 4:37 PM, Shanker Balan
<sh...@shapeblue.com>wrote:
> Comments inline.
>
> On 05-Dec-2013, at 6:34 pm, Francesco Maria Magnini <fm...@gmail.com>
> wrote:
>
> > Cloud0 is created dynamically by Cloudstack, in CentOS too.
>
> Yes, of course its created by cloudstack. I am trying to recall what
> I was thinking while I was typing. :D
>
> > I think it's not related to security groups, since I'm not able to ping
> > anymore from outside the Console VM and Storage VM after creating
> instances.
> > So it's definitely something wrong with the scripts that are responsible
> to
> > create instances (involving the creation of the Virtual Router, and so
> on).
>
> Can you do tcpdumps also?
>
> - tcpdump on the physical NIC thats assigned for public traffic
> - tcpdump on the bridge interface that connects to the public NIC
> - tcpdump on the VIF thats connected to the bridge
> - tcpdump on the VM’s interface
>
> Additionally, can you share your network schema?
>
>
>
>
> >
> >
> >
> >
> > On Thu, Dec 5, 2013 at 1:34 PM, Shanker Balan
> > <sh...@shapeblue.com>wrote:
> >
> >> Comments inline.
> >>
> >> On 05-Dec-2013, at 5:35 pm, Francesco Maria Magnini <fm...@gmail.com>
> >> wrote:
> >>
> >>> I know.
> >>
> >> My reply was inline to the comment:
> >>
> >>>>>
> >>>>> I think icmp is disabled by default on SSVM and CPVM
> >>>>> on control IP address, but should be allowed on public IP address.
> >>>>
> >>>> FWIW, ICMP works on both the public and private addresses on my lab
> >> setup:
> >>
> >> :)
> >>
> >>
> >>> As I said on top of the discussion, I tested Cloudstack 4.2 on a CentOS
> >> 6.4
> >>> deployment (Controller, KVM Host) and never encountered problems on
> >>> network. I even tested Advancend networking with VLANS, GRE Tunnels in
> a
> >>> very complicated scenarios.
> >>>
> >>> Switching to Ubuntu (because I need to interact with CEPH), SSVM and
> KVM
> >>> Guest have no connectivity, in a very basic scenario consisting in
> basic
> >>> network zone.
> >>
> >> Am looking at your brctl output:
> >>
> >> root@kvm01:~# brctl show
> >> bridge name bridge id STP enabled interfaces
> >> cloud0 8000.fe00a9fe01a8 no vnet0
> >> vnet4
> >> cloudbr0 8000.0019995a73ac no eth0
> >> vnet1
> >> vnet2
> >> vnet3
> >> vnet5
> >> vnet6
> >> virbr0 8000.000000000000 yes
> >>
> >> What’s cloud0 interface? Does the brctl output match with your working
> >> CentOS setup?
> >>
> >>> After debugging, watching iptables counters, I see that all the
> incoming
> >>> public traffic is dropped by iptables on the KVM host, and is not
> passed
> >> to
> >>> KVM Guests (including SSVM and Guest VMs).
> >>>
> >>>
> >>> On Thu, Dec 5, 2013 at 12:52 PM, Shanker Balan
> >>> <sh...@shapeblue.com>wrote:
> >>>
> >>>> On 05-Dec-2013, at 10:53 am, Sanjeev Neelarapu <
> >>>> sanjeev.neelarapu@citrix.com> wrote:
> >>>>
> >>>>> Hi,
> >>>>>
> >>>>> Make sure that iptable rules are configured properly for icmp and ssh
> >>>>> traffic on kvm host.
> >>>>
> >>>>
> >>>>> I think icmp is disabled by default on SSVM and CPVM
> >>>>> on control IP address, but should be allowed on public IP address.
> >>>>
> >>>> FWIW, ICMP works on both the public and private addresses on my lab
> >> setup:
> >>>>
> >>>> [root@csman1-1 cloudmonkey]# cloudmonkey list systemvms|grep ip
> >>>> linklocalip = 169.254.3.16
> >>>> privateip = 192.168.44.62
> >>>> publicip = 192.168.64.100
> >>>> linklocalip = 169.254.3.98
> >>>> privateip = 192.168.44.61
> >>>> publicip = 192.168.64.101
> >>>> [root@csman1-1 cloudmonkey]# fping 192.168.44.62
> >>>> 192.168.44.62 is alive
> >>>> [root@csman1-1 cloudmonkey]# fping 192.168.64.100
> >>>> 192.168.64.100 is alive
> >>>> [root@csman1-1 cloudmonkey]# fping 192.168.44.61
> >>>> 192.168.44.61 is alive
> >>>> [root@csman1-1 cloudmonkey]# fping 192.168.64.101
> >>>> 192.168.64.101 is alive
> >>>> [root@csman1-1 cloudmonkey]#
> >>>>
> >>>>
> >>>> --
> >>>> @shankerbalan
> >>>>
> >>>> M: +91 98860 60539 | O: +91 (80) 67935867
> >>>> shanker.balan@shapeblue.com | www.shapeblue.com | Twitter:@shapeblue
> >>>> ShapeBlue Services India LLP, 22nd floor, Unit 2201A, World Trade
> >> Centre,
> >>>> Bangalore - 560 055
> >>>>
> >>>> This email and any attachments to it may be confidential and are
> >> intended
> >>>> solely for the use of the individual to whom it is addressed. Any
> views
> >> or
> >>>> opinions expressed are solely those of the author and do not
> necessarily
> >>>> represent those of Shape Blue Ltd or related companies. If you are not
> >> the
> >>>> intended recipient of this email, you must neither take any action
> based
> >>>> upon its contents, nor copy or show it to anyone. Please contact the
> >> sender
> >>>> if you believe you have received this email in error. Shape Blue Ltd
> is
> >> a
> >>>> company incorporated in England & Wales. ShapeBlue Services India LLP
> >> is a
> >>>> company incorporated in India and is operated under license from Shape
> >> Blue
> >>>> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in
> >> Brasil
> >>>> and is operated under license from Shape Blue Ltd. ShapeBlue is a
> >>>> registered trademark.
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> “I videogiochi non influenzano i bambini.
> >>> Voglio dire, se pac-man avesse influenzato la nostra generazione,
> >>> staremmo tutti saltando in sale scure,
> >>> masticando pillole magiche e ascoltando musica elettronica
> >>> ripetitiva...”
> >>>
> >>> (Kristian Wilson, Nintendo Inc, 1989)
> >>
> >> --
> >> @shankerbalan
> >>
> >> M: +91 98860 60539 | O: +91 (80) 67935867
> >> shanker.balan@shapeblue.com | www.shapeblue.com | Twitter:@shapeblue
> >> ShapeBlue Services India LLP, 22nd floor, Unit 2201A, World Trade
> Centre,
> >> Bangalore - 560 055
> >>
> >> This email and any attachments to it may be confidential and are
> intended
> >> solely for the use of the individual to whom it is addressed. Any views
> or
> >> opinions expressed are solely those of the author and do not necessarily
> >> represent those of Shape Blue Ltd or related companies. If you are not
> the
> >> intended recipient of this email, you must neither take any action based
> >> upon its contents, nor copy or show it to anyone. Please contact the
> sender
> >> if you believe you have received this email in error. Shape Blue Ltd is
> a
> >> company incorporated in England & Wales. ShapeBlue Services India LLP
> is a
> >> company incorporated in India and is operated under license from Shape
> Blue
> >> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in
> Brasil
> >> and is operated under license from Shape Blue Ltd. ShapeBlue is a
> >> registered trademark.
> >>
> >
> >
> >
> > --
> > “I videogiochi non influenzano i bambini.
> > Voglio dire, se pac-man avesse influenzato la nostra generazione,
> > staremmo tutti saltando in sale scure,
> > masticando pillole magiche e ascoltando musica elettronica
> > ripetitiva...”
> >
> > (Kristian Wilson, Nintendo Inc, 1989)
>
> --
> @shankerbalan
>
> M: +91 98860 60539 | O: +91 (80) 67935867
> shanker.balan@shapeblue.com | www.shapeblue.com | Twitter:@shapeblue
> ShapeBlue Services India LLP, 22nd floor, Unit 2201A, World Trade Centre,
> Bangalore - 560 055
>
> This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views or
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd or related companies. If you are not the
> intended recipient of this email, you must neither take any action based
> upon its contents, nor copy or show it to anyone. Please contact the sender
> if you believe you have received this email in error. Shape Blue Ltd is a
> company incorporated in England & Wales. ShapeBlue Services India LLP is a
> company incorporated in India and is operated under license from Shape Blue
> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
> and is operated under license from Shape Blue Ltd. ShapeBlue is a
> registered trademark.
>
--
“I videogiochi non influenzano i bambini.
Voglio dire, se pac-man avesse influenzato la nostra generazione,
staremmo tutti saltando in sale scure,
masticando pillole magiche e ascoltando musica elettronica
ripetitiva...”
(Kristian Wilson, Nintendo Inc, 1989)
Re: Storage/Console SSVM loose connectivity (can't ping them
anymore) after creating the first guest instance
Posted by Shanker Balan <sh...@shapeblue.com>.
Comments inline.
On 05-Dec-2013, at 6:34 pm, Francesco Maria Magnini <fm...@gmail.com> wrote:
> Cloud0 is created dynamically by Cloudstack, in CentOS too.
Yes, of course its created by cloudstack. I am trying to recall what
I was thinking while I was typing. :D
> I think it's not related to security groups, since I'm not able to ping
> anymore from outside the Console VM and Storage VM after creating instances.
> So it's definitely something wrong with the scripts that are responsible to
> create instances (involving the creation of the Virtual Router, and so on).
Can you do tcpdumps also?
- tcpdump on the physical NIC thats assigned for public traffic
- tcpdump on the bridge interface that connects to the public NIC
- tcpdump on the VIF thats connected to the bridge
- tcpdump on the VM’s interface
Additionally, can you share your network schema?
>
>
>
>
> On Thu, Dec 5, 2013 at 1:34 PM, Shanker Balan
> <sh...@shapeblue.com>wrote:
>
>> Comments inline.
>>
>> On 05-Dec-2013, at 5:35 pm, Francesco Maria Magnini <fm...@gmail.com>
>> wrote:
>>
>>> I know.
>>
>> My reply was inline to the comment:
>>
>>>>>
>>>>> I think icmp is disabled by default on SSVM and CPVM
>>>>> on control IP address, but should be allowed on public IP address.
>>>>
>>>> FWIW, ICMP works on both the public and private addresses on my lab
>> setup:
>>
>> :)
>>
>>
>>> As I said on top of the discussion, I tested Cloudstack 4.2 on a CentOS
>> 6.4
>>> deployment (Controller, KVM Host) and never encountered problems on
>>> network. I even tested Advancend networking with VLANS, GRE Tunnels in a
>>> very complicated scenarios.
>>>
>>> Switching to Ubuntu (because I need to interact with CEPH), SSVM and KVM
>>> Guest have no connectivity, in a very basic scenario consisting in basic
>>> network zone.
>>
>> Am looking at your brctl output:
>>
>> root@kvm01:~# brctl show
>> bridge name bridge id STP enabled interfaces
>> cloud0 8000.fe00a9fe01a8 no vnet0
>> vnet4
>> cloudbr0 8000.0019995a73ac no eth0
>> vnet1
>> vnet2
>> vnet3
>> vnet5
>> vnet6
>> virbr0 8000.000000000000 yes
>>
>> What’s cloud0 interface? Does the brctl output match with your working
>> CentOS setup?
>>
>>> After debugging, watching iptables counters, I see that all the incoming
>>> public traffic is dropped by iptables on the KVM host, and is not passed
>> to
>>> KVM Guests (including SSVM and Guest VMs).
>>>
>>>
>>> On Thu, Dec 5, 2013 at 12:52 PM, Shanker Balan
>>> <sh...@shapeblue.com>wrote:
>>>
>>>> On 05-Dec-2013, at 10:53 am, Sanjeev Neelarapu <
>>>> sanjeev.neelarapu@citrix.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Make sure that iptable rules are configured properly for icmp and ssh
>>>>> traffic on kvm host.
>>>>
>>>>
>>>>> I think icmp is disabled by default on SSVM and CPVM
>>>>> on control IP address, but should be allowed on public IP address.
>>>>
>>>> FWIW, ICMP works on both the public and private addresses on my lab
>> setup:
>>>>
>>>> [root@csman1-1 cloudmonkey]# cloudmonkey list systemvms|grep ip
>>>> linklocalip = 169.254.3.16
>>>> privateip = 192.168.44.62
>>>> publicip = 192.168.64.100
>>>> linklocalip = 169.254.3.98
>>>> privateip = 192.168.44.61
>>>> publicip = 192.168.64.101
>>>> [root@csman1-1 cloudmonkey]# fping 192.168.44.62
>>>> 192.168.44.62 is alive
>>>> [root@csman1-1 cloudmonkey]# fping 192.168.64.100
>>>> 192.168.64.100 is alive
>>>> [root@csman1-1 cloudmonkey]# fping 192.168.44.61
>>>> 192.168.44.61 is alive
>>>> [root@csman1-1 cloudmonkey]# fping 192.168.64.101
>>>> 192.168.64.101 is alive
>>>> [root@csman1-1 cloudmonkey]#
>>>>
>>>>
>>>> --
>>>> @shankerbalan
>>>>
>>>> M: +91 98860 60539 | O: +91 (80) 67935867
>>>> shanker.balan@shapeblue.com | www.shapeblue.com | Twitter:@shapeblue
>>>> ShapeBlue Services India LLP, 22nd floor, Unit 2201A, World Trade
>> Centre,
>>>> Bangalore - 560 055
>>>>
>>>> This email and any attachments to it may be confidential and are
>> intended
>>>> solely for the use of the individual to whom it is addressed. Any views
>> or
>>>> opinions expressed are solely those of the author and do not necessarily
>>>> represent those of Shape Blue Ltd or related companies. If you are not
>> the
>>>> intended recipient of this email, you must neither take any action based
>>>> upon its contents, nor copy or show it to anyone. Please contact the
>> sender
>>>> if you believe you have received this email in error. Shape Blue Ltd is
>> a
>>>> company incorporated in England & Wales. ShapeBlue Services India LLP
>> is a
>>>> company incorporated in India and is operated under license from Shape
>> Blue
>>>> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in
>> Brasil
>>>> and is operated under license from Shape Blue Ltd. ShapeBlue is a
>>>> registered trademark.
>>>>
>>>
>>>
>>>
>>> --
>>> “I videogiochi non influenzano i bambini.
>>> Voglio dire, se pac-man avesse influenzato la nostra generazione,
>>> staremmo tutti saltando in sale scure,
>>> masticando pillole magiche e ascoltando musica elettronica
>>> ripetitiva...”
>>>
>>> (Kristian Wilson, Nintendo Inc, 1989)
>>
>> --
>> @shankerbalan
>>
>> M: +91 98860 60539 | O: +91 (80) 67935867
>> shanker.balan@shapeblue.com | www.shapeblue.com | Twitter:@shapeblue
>> ShapeBlue Services India LLP, 22nd floor, Unit 2201A, World Trade Centre,
>> Bangalore - 560 055
>>
>> This email and any attachments to it may be confidential and are intended
>> solely for the use of the individual to whom it is addressed. Any views or
>> opinions expressed are solely those of the author and do not necessarily
>> represent those of Shape Blue Ltd or related companies. If you are not the
>> intended recipient of this email, you must neither take any action based
>> upon its contents, nor copy or show it to anyone. Please contact the sender
>> if you believe you have received this email in error. Shape Blue Ltd is a
>> company incorporated in England & Wales. ShapeBlue Services India LLP is a
>> company incorporated in India and is operated under license from Shape Blue
>> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
>> and is operated under license from Shape Blue Ltd. ShapeBlue is a
>> registered trademark.
>>
>
>
>
> --
> “I videogiochi non influenzano i bambini.
> Voglio dire, se pac-man avesse influenzato la nostra generazione,
> staremmo tutti saltando in sale scure,
> masticando pillole magiche e ascoltando musica elettronica
> ripetitiva...”
>
> (Kristian Wilson, Nintendo Inc, 1989)
--
@shankerbalan
M: +91 98860 60539 | O: +91 (80) 67935867
shanker.balan@shapeblue.com | www.shapeblue.com | Twitter:@shapeblue
ShapeBlue Services India LLP, 22nd floor, Unit 2201A, World Trade Centre, Bangalore - 560 055
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: Storage/Console SSVM loose connectivity (can't ping them anymore)
after creating the first guest instance
Posted by Francesco Maria Magnini <fm...@gmail.com>.
Cloud0 is created dynamically by Cloudstack, in CentOS too.
I think it's not related to security groups, since I'm not able to ping
anymore from outside the Console VM and Storage VM after creating instances.
So it's definitely something wrong with the scripts that are responsible to
create instances (involving the creation of the Virtual Router, and so on).
On Thu, Dec 5, 2013 at 1:34 PM, Shanker Balan
<sh...@shapeblue.com>wrote:
> Comments inline.
>
> On 05-Dec-2013, at 5:35 pm, Francesco Maria Magnini <fm...@gmail.com>
> wrote:
>
> > I know.
>
> My reply was inline to the comment:
>
> >>>
> >>> I think icmp is disabled by default on SSVM and CPVM
> >>> on control IP address, but should be allowed on public IP address.
> >>
> >> FWIW, ICMP works on both the public and private addresses on my lab
> setup:
>
> :)
>
>
> > As I said on top of the discussion, I tested Cloudstack 4.2 on a CentOS
> 6.4
> > deployment (Controller, KVM Host) and never encountered problems on
> > network. I even tested Advancend networking with VLANS, GRE Tunnels in a
> > very complicated scenarios.
> >
> > Switching to Ubuntu (because I need to interact with CEPH), SSVM and KVM
> > Guest have no connectivity, in a very basic scenario consisting in basic
> > network zone.
>
> Am looking at your brctl output:
>
> root@kvm01:~# brctl show
> bridge name bridge id STP enabled interfaces
> cloud0 8000.fe00a9fe01a8 no vnet0
> vnet4
> cloudbr0 8000.0019995a73ac no eth0
> vnet1
> vnet2
> vnet3
> vnet5
> vnet6
> virbr0 8000.000000000000 yes
>
> What’s cloud0 interface? Does the brctl output match with your working
> CentOS setup?
>
> > After debugging, watching iptables counters, I see that all the incoming
> > public traffic is dropped by iptables on the KVM host, and is not passed
> to
> > KVM Guests (including SSVM and Guest VMs).
> >
> >
> > On Thu, Dec 5, 2013 at 12:52 PM, Shanker Balan
> > <sh...@shapeblue.com>wrote:
> >
> >> On 05-Dec-2013, at 10:53 am, Sanjeev Neelarapu <
> >> sanjeev.neelarapu@citrix.com> wrote:
> >>
> >>> Hi,
> >>>
> >>> Make sure that iptable rules are configured properly for icmp and ssh
> >>> traffic on kvm host.
> >>
> >>
> >>> I think icmp is disabled by default on SSVM and CPVM
> >>> on control IP address, but should be allowed on public IP address.
> >>
> >> FWIW, ICMP works on both the public and private addresses on my lab
> setup:
> >>
> >> [root@csman1-1 cloudmonkey]# cloudmonkey list systemvms|grep ip
> >> linklocalip = 169.254.3.16
> >> privateip = 192.168.44.62
> >> publicip = 192.168.64.100
> >> linklocalip = 169.254.3.98
> >> privateip = 192.168.44.61
> >> publicip = 192.168.64.101
> >> [root@csman1-1 cloudmonkey]# fping 192.168.44.62
> >> 192.168.44.62 is alive
> >> [root@csman1-1 cloudmonkey]# fping 192.168.64.100
> >> 192.168.64.100 is alive
> >> [root@csman1-1 cloudmonkey]# fping 192.168.44.61
> >> 192.168.44.61 is alive
> >> [root@csman1-1 cloudmonkey]# fping 192.168.64.101
> >> 192.168.64.101 is alive
> >> [root@csman1-1 cloudmonkey]#
> >>
> >>
> >> --
> >> @shankerbalan
> >>
> >> M: +91 98860 60539 | O: +91 (80) 67935867
> >> shanker.balan@shapeblue.com | www.shapeblue.com | Twitter:@shapeblue
> >> ShapeBlue Services India LLP, 22nd floor, Unit 2201A, World Trade
> Centre,
> >> Bangalore - 560 055
> >>
> >> This email and any attachments to it may be confidential and are
> intended
> >> solely for the use of the individual to whom it is addressed. Any views
> or
> >> opinions expressed are solely those of the author and do not necessarily
> >> represent those of Shape Blue Ltd or related companies. If you are not
> the
> >> intended recipient of this email, you must neither take any action based
> >> upon its contents, nor copy or show it to anyone. Please contact the
> sender
> >> if you believe you have received this email in error. Shape Blue Ltd is
> a
> >> company incorporated in England & Wales. ShapeBlue Services India LLP
> is a
> >> company incorporated in India and is operated under license from Shape
> Blue
> >> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in
> Brasil
> >> and is operated under license from Shape Blue Ltd. ShapeBlue is a
> >> registered trademark.
> >>
> >
> >
> >
> > --
> > “I videogiochi non influenzano i bambini.
> > Voglio dire, se pac-man avesse influenzato la nostra generazione,
> > staremmo tutti saltando in sale scure,
> > masticando pillole magiche e ascoltando musica elettronica
> > ripetitiva...”
> >
> > (Kristian Wilson, Nintendo Inc, 1989)
>
> --
> @shankerbalan
>
> M: +91 98860 60539 | O: +91 (80) 67935867
> shanker.balan@shapeblue.com | www.shapeblue.com | Twitter:@shapeblue
> ShapeBlue Services India LLP, 22nd floor, Unit 2201A, World Trade Centre,
> Bangalore - 560 055
>
> This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views or
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd or related companies. If you are not the
> intended recipient of this email, you must neither take any action based
> upon its contents, nor copy or show it to anyone. Please contact the sender
> if you believe you have received this email in error. Shape Blue Ltd is a
> company incorporated in England & Wales. ShapeBlue Services India LLP is a
> company incorporated in India and is operated under license from Shape Blue
> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
> and is operated under license from Shape Blue Ltd. ShapeBlue is a
> registered trademark.
>
--
“I videogiochi non influenzano i bambini.
Voglio dire, se pac-man avesse influenzato la nostra generazione,
staremmo tutti saltando in sale scure,
masticando pillole magiche e ascoltando musica elettronica
ripetitiva...”
(Kristian Wilson, Nintendo Inc, 1989)
Re: Storage/Console SSVM loose connectivity (can't ping them
anymore) after creating the first guest instance
Posted by Shanker Balan <sh...@shapeblue.com>.
Comments inline.
On 05-Dec-2013, at 5:35 pm, Francesco Maria Magnini <fm...@gmail.com> wrote:
> I know.
My reply was inline to the comment:
>>>
>>> I think icmp is disabled by default on SSVM and CPVM
>>> on control IP address, but should be allowed on public IP address.
>>
>> FWIW, ICMP works on both the public and private addresses on my lab setup:
:)
> As I said on top of the discussion, I tested Cloudstack 4.2 on a CentOS 6.4
> deployment (Controller, KVM Host) and never encountered problems on
> network. I even tested Advancend networking with VLANS, GRE Tunnels in a
> very complicated scenarios.
>
> Switching to Ubuntu (because I need to interact with CEPH), SSVM and KVM
> Guest have no connectivity, in a very basic scenario consisting in basic
> network zone.
Am looking at your brctl output:
root@kvm01:~# brctl show
bridge name bridge id STP enabled interfaces
cloud0 8000.fe00a9fe01a8 no vnet0
vnet4
cloudbr0 8000.0019995a73ac no eth0
vnet1
vnet2
vnet3
vnet5
vnet6
virbr0 8000.000000000000 yes
What’s cloud0 interface? Does the brctl output match with your working CentOS setup?
> After debugging, watching iptables counters, I see that all the incoming
> public traffic is dropped by iptables on the KVM host, and is not passed to
> KVM Guests (including SSVM and Guest VMs).
>
>
> On Thu, Dec 5, 2013 at 12:52 PM, Shanker Balan
> <sh...@shapeblue.com>wrote:
>
>> On 05-Dec-2013, at 10:53 am, Sanjeev Neelarapu <
>> sanjeev.neelarapu@citrix.com> wrote:
>>
>>> Hi,
>>>
>>> Make sure that iptable rules are configured properly for icmp and ssh
>>> traffic on kvm host.
>>
>>
>>> I think icmp is disabled by default on SSVM and CPVM
>>> on control IP address, but should be allowed on public IP address.
>>
>> FWIW, ICMP works on both the public and private addresses on my lab setup:
>>
>> [root@csman1-1 cloudmonkey]# cloudmonkey list systemvms|grep ip
>> linklocalip = 169.254.3.16
>> privateip = 192.168.44.62
>> publicip = 192.168.64.100
>> linklocalip = 169.254.3.98
>> privateip = 192.168.44.61
>> publicip = 192.168.64.101
>> [root@csman1-1 cloudmonkey]# fping 192.168.44.62
>> 192.168.44.62 is alive
>> [root@csman1-1 cloudmonkey]# fping 192.168.64.100
>> 192.168.64.100 is alive
>> [root@csman1-1 cloudmonkey]# fping 192.168.44.61
>> 192.168.44.61 is alive
>> [root@csman1-1 cloudmonkey]# fping 192.168.64.101
>> 192.168.64.101 is alive
>> [root@csman1-1 cloudmonkey]#
>>
>>
>> --
>> @shankerbalan
>>
>> M: +91 98860 60539 | O: +91 (80) 67935867
>> shanker.balan@shapeblue.com | www.shapeblue.com | Twitter:@shapeblue
>> ShapeBlue Services India LLP, 22nd floor, Unit 2201A, World Trade Centre,
>> Bangalore - 560 055
>>
>> This email and any attachments to it may be confidential and are intended
>> solely for the use of the individual to whom it is addressed. Any views or
>> opinions expressed are solely those of the author and do not necessarily
>> represent those of Shape Blue Ltd or related companies. If you are not the
>> intended recipient of this email, you must neither take any action based
>> upon its contents, nor copy or show it to anyone. Please contact the sender
>> if you believe you have received this email in error. Shape Blue Ltd is a
>> company incorporated in England & Wales. ShapeBlue Services India LLP is a
>> company incorporated in India and is operated under license from Shape Blue
>> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
>> and is operated under license from Shape Blue Ltd. ShapeBlue is a
>> registered trademark.
>>
>
>
>
> --
> “I videogiochi non influenzano i bambini.
> Voglio dire, se pac-man avesse influenzato la nostra generazione,
> staremmo tutti saltando in sale scure,
> masticando pillole magiche e ascoltando musica elettronica
> ripetitiva...”
>
> (Kristian Wilson, Nintendo Inc, 1989)
--
@shankerbalan
M: +91 98860 60539 | O: +91 (80) 67935867
shanker.balan@shapeblue.com | www.shapeblue.com | Twitter:@shapeblue
ShapeBlue Services India LLP, 22nd floor, Unit 2201A, World Trade Centre, Bangalore - 560 055
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: Storage/Console SSVM loose connectivity (can't ping them anymore)
after creating the first guest instance
Posted by Francesco Maria Magnini <fm...@gmail.com>.
I know.
As I said on top of the discussion, I tested Cloudstack 4.2 on a CentOS 6.4
deployment (Controller, KVM Host) and never encountered problems on
network. I even tested Advancend networking with VLANS, GRE Tunnels in a
very complicated scenarios.
Switching to Ubuntu (because I need to interact with CEPH), SSVM and KVM
Guest have no connectivity, in a very basic scenario consisting in basic
network zone.
After debugging, watching iptables counters, I see that all the incoming
public traffic is dropped by iptables on the KVM host, and is not passed to
KVM Guests (including SSVM and Guest VMs).
On Thu, Dec 5, 2013 at 12:52 PM, Shanker Balan
<sh...@shapeblue.com>wrote:
> On 05-Dec-2013, at 10:53 am, Sanjeev Neelarapu <
> sanjeev.neelarapu@citrix.com> wrote:
>
> > Hi,
> >
> > Make sure that iptable rules are configured properly for icmp and ssh
> > traffic on kvm host.
>
>
> > I think icmp is disabled by default on SSVM and CPVM
> > on control IP address, but should be allowed on public IP address.
>
> FWIW, ICMP works on both the public and private addresses on my lab setup:
>
> [root@csman1-1 cloudmonkey]# cloudmonkey list systemvms|grep ip
> linklocalip = 169.254.3.16
> privateip = 192.168.44.62
> publicip = 192.168.64.100
> linklocalip = 169.254.3.98
> privateip = 192.168.44.61
> publicip = 192.168.64.101
> [root@csman1-1 cloudmonkey]# fping 192.168.44.62
> 192.168.44.62 is alive
> [root@csman1-1 cloudmonkey]# fping 192.168.64.100
> 192.168.64.100 is alive
> [root@csman1-1 cloudmonkey]# fping 192.168.44.61
> 192.168.44.61 is alive
> [root@csman1-1 cloudmonkey]# fping 192.168.64.101
> 192.168.64.101 is alive
> [root@csman1-1 cloudmonkey]#
>
>
> --
> @shankerbalan
>
> M: +91 98860 60539 | O: +91 (80) 67935867
> shanker.balan@shapeblue.com | www.shapeblue.com | Twitter:@shapeblue
> ShapeBlue Services India LLP, 22nd floor, Unit 2201A, World Trade Centre,
> Bangalore - 560 055
>
> This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views or
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd or related companies. If you are not the
> intended recipient of this email, you must neither take any action based
> upon its contents, nor copy or show it to anyone. Please contact the sender
> if you believe you have received this email in error. Shape Blue Ltd is a
> company incorporated in England & Wales. ShapeBlue Services India LLP is a
> company incorporated in India and is operated under license from Shape Blue
> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
> and is operated under license from Shape Blue Ltd. ShapeBlue is a
> registered trademark.
>
--
“I videogiochi non influenzano i bambini.
Voglio dire, se pac-man avesse influenzato la nostra generazione,
staremmo tutti saltando in sale scure,
masticando pillole magiche e ascoltando musica elettronica
ripetitiva...”
(Kristian Wilson, Nintendo Inc, 1989)
Re: Storage/Console SSVM loose connectivity (can't ping them
anymore) after creating the first guest instance
Posted by Shanker Balan <sh...@shapeblue.com>.
On 05-Dec-2013, at 10:53 am, Sanjeev Neelarapu <sa...@citrix.com> wrote:
> Hi,
>
> Make sure that iptable rules are configured properly for icmp and ssh
> traffic on kvm host.
> I think icmp is disabled by default on SSVM and CPVM
> on control IP address, but should be allowed on public IP address.
FWIW, ICMP works on both the public and private addresses on my lab setup:
[root@csman1-1 cloudmonkey]# cloudmonkey list systemvms|grep ip
linklocalip = 169.254.3.16
privateip = 192.168.44.62
publicip = 192.168.64.100
linklocalip = 169.254.3.98
privateip = 192.168.44.61
publicip = 192.168.64.101
[root@csman1-1 cloudmonkey]# fping 192.168.44.62
192.168.44.62 is alive
[root@csman1-1 cloudmonkey]# fping 192.168.64.100
192.168.64.100 is alive
[root@csman1-1 cloudmonkey]# fping 192.168.44.61
192.168.44.61 is alive
[root@csman1-1 cloudmonkey]# fping 192.168.64.101
192.168.64.101 is alive
[root@csman1-1 cloudmonkey]#
--
@shankerbalan
M: +91 98860 60539 | O: +91 (80) 67935867
shanker.balan@shapeblue.com | www.shapeblue.com | Twitter:@shapeblue
ShapeBlue Services India LLP, 22nd floor, Unit 2201A, World Trade Centre, Bangalore - 560 055
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: Storage/Console SSVM loose connectivity (can't ping them anymore)
after creating the first guest instance
Posted by Francesco Maria Magnini <fm...@gmail.com>.
Hi,
below you can find the iptables rules and the bridge configuration.
Anyway, what I see is that ICMP request are reaching the KVM host, but
opening a virsh console to the guest shows no ICMP packets coming from
public network.
root@kvm01:~# ufw status
Status: inactive
root@kvm01:~#
root@kvm01:~#
root@kvm01:~#
root@kvm01:~#
root@kvm01:~# brctl show
bridge name bridge id STP enabled interfaces
cloud0 8000.fe00a9fe01a8 no vnet0
vnet4
cloudbr0 8000.0019995a73ac no eth0
vnet1
vnet2
vnet3
vnet5
vnet6
virbr0 8000.000000000000 yes
root@kvm01:~# iptables -L -n
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:53
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:53
ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:67
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:67
Chain FORWARD (policy ACCEPT)
target prot opt source destination
BF-cloudbr0 all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV
match --physdev-is-bridged
BF-cloudbr0 all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV
match --physdev-is-bridged
DROP all -- 0.0.0.0/0 0.0.0.0/0
DROP all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 192.168.122.0/24 ctstate
RELATED,ESTABLISHED
ACCEPT all -- 192.168.122.0/24 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with
icmp-port-unreachable
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with
icmp-port-unreachable
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain BF-cloudbr0 (2 references)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 state
RELATED,ESTABLISHED
BF-cloudbr0-IN all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV
match --physdev-is-in --physdev-is-bridged
BF-cloudbr0-OUT all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV
match --physdev-is-out --physdev-is-bridged
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match
--physdev-out eth0 --physdev-is-bridged
Chain BF-cloudbr0-IN (1 references)
target prot opt source destination
Chain BF-cloudbr0-OUT (1 references)
target prot opt source destination
root@kvm01:~#
On Thu, Dec 5, 2013 at 6:23 AM, Sanjeev Neelarapu <
sanjeev.neelarapu@citrix.com> wrote:
> Hi,
>
> Make sure that iptable rules are configured properly for icmp and ssh
> traffic on kvm host.
> I think icmp is disabled by default on SSVM and CPVM on control IP
> address, but should be allowed on public IP address.
>
> Thanks,
> Sanjeev
>
> -----Original Message-----
> From: Francesco Maria Magnini [mailto:fmm1982@gmail.com]
> Sent: Wednesday, December 04, 2013 9:23 PM
> To: users@cloudstack.apache.org
> Subject: Storage/Console SSVM loose connectivity (can't ping them anymore)
> after creating the first guest instance
>
> I'm experiencing problems in Cloudstack 4.2 installation on both Ubuntu
> Server 12.04 and 13.10.
>
> - Installed Cloudstack Management Controller and a KVM host in two
> separate boxes
> - Cloudstack installations went fine
> - Created a basic networking zone, all is green, storage ok, vvms ok
>
> At this point, I'm able to ping physical hosts, Storage and Proxy SSVM
> public IP Address.
>
> - Created instance with basic template of CentOS, fine
> - Automatically created System Router
>
> At this point I'm still able to ping physical hosts, but no longer Storage
> SSMV, Proxy SSVM and the CentOS instance created a while ago.
>
> No errors, all is green, all processes running fine, just connectivity
> issue on the public network side of Proxy/Storage VVMS and Instances.
>
> *IMPORTANT* I've configured Cloudstack in CentOS enviroment before testing
> Ubuntu Server, with both Basic and Advanced Networking (VLAN separation)
> without problems.
>
> Security group is already configured with ICMP/SSH rules for inbound.
>
> Any ideas? Thanks
>
--
“I videogiochi non influenzano i bambini.
Voglio dire, se pac-man avesse influenzato la nostra generazione,
staremmo tutti saltando in sale scure,
masticando pillole magiche e ascoltando musica elettronica
ripetitiva...”
(Kristian Wilson, Nintendo Inc, 1989)
RE: Storage/Console SSVM loose connectivity (can't ping them
anymore) after creating the first guest instance
Posted by Sanjeev Neelarapu <sa...@citrix.com>.
Hi,
Make sure that iptable rules are configured properly for icmp and ssh traffic on kvm host.
I think icmp is disabled by default on SSVM and CPVM on control IP address, but should be allowed on public IP address.
Thanks,
Sanjeev
-----Original Message-----
From: Francesco Maria Magnini [mailto:fmm1982@gmail.com]
Sent: Wednesday, December 04, 2013 9:23 PM
To: users@cloudstack.apache.org
Subject: Storage/Console SSVM loose connectivity (can't ping them anymore) after creating the first guest instance
I'm experiencing problems in Cloudstack 4.2 installation on both Ubuntu Server 12.04 and 13.10.
- Installed Cloudstack Management Controller and a KVM host in two
separate boxes
- Cloudstack installations went fine
- Created a basic networking zone, all is green, storage ok, vvms ok
At this point, I'm able to ping physical hosts, Storage and Proxy SSVM public IP Address.
- Created instance with basic template of CentOS, fine
- Automatically created System Router
At this point I'm still able to ping physical hosts, but no longer Storage SSMV, Proxy SSVM and the CentOS instance created a while ago.
No errors, all is green, all processes running fine, just connectivity issue on the public network side of Proxy/Storage VVMS and Instances.
*IMPORTANT* I've configured Cloudstack in CentOS enviroment before testing Ubuntu Server, with both Basic and Advanced Networking (VLAN separation) without problems.
Security group is already configured with ICMP/SSH rules for inbound.
Any ideas? Thanks