You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Andrija Panic <an...@gmail.com> on 2017/10/02 08:21:42 UTC
Advise on multiple PODs network design
Hi guys,
Sorry for long post below...
I was wondering if someone could bring some light for me for multiple PODs
networking design (L2 vs L3) - idea is to make smaller L2 broadcast domains
(any other reason?)
We might decide to transition from current single pod, single cluster
(single zone) to multiple PODs design (or not...) - we will eventually grow
to over 50 racks worth of KVM hosts (1000+ hosts) so Im trying to
understand best options to avoid having insanely huge L2 broadcast
domains...
Mgmt network is routed between pods, that is clear.
We have dedicated primary storage network and Secondary Storage networks
(vlan interfaces configured locally on all KVM hosts, providing direct L2
connection obviously, not shared with mgmt.network), and same for Public
and Guest networks... (Advanced networking in zone, Vxlan used as isolation)
Now with multiple PODs, since Public Network and Guest network is defined
per Zone level (not POD level), and currently same zone-wide setup for
Primary Storage... what would be the best way to make this traffic stay
inside PODs as much as possible and is this possible at all? Perhaps I
would need to look into multiple zones, not PODs.
My humble conclusion, based on having all dedicated networks, is that I
need to strech (L2 attach as vlan interface) primary and secondary storage
network across all racks/PODs, and also need to strech Guest vlan (that
carry all Guest VXLAN tunnels), and again same for Public Network...and
this again makes huge broadcast domains and doesn't solve my issue...
Don't see other option in my head to make networking work across PODs.
Any suggestion is most welcome (and if of any use as info - we dont plan
for any Xen, VmWare etc, will stay purely with KVM).
Thanks
Andrija
Re: Advise on multiple PODs network design
Posted by Andrija Panic <an...@gmail.com>.
Thanks a lot guys, will report eventually on steps we choose.
Thanks
On 5 October 2017 at 14:40, Voloshanenko Igor <ig...@gmail.com>
wrote:
> Tnx Remi!
>
> Brilliant advice!
>
> чт, 5 окт. 2017 г. в 15:38, Remi Bergsma <RB...@schubergphilis.com>:
>
> > Hi,
> >
> > We solved this problem by splitting the network in an underlay and
> overlay
> > network. The underlay is the physical network, including the management
> > traffic and storage from hypervisors and such. The simpler, the better.
> In
> > the overlay there’s your services layer, for example the guest networks
> for
> > your clients. Since you’re already using vxlan that shouldn’t be hard. In
> > our setup, each POD is a rack with TOR (Top-of-rack routing) which means
> a
> > L2 domain stays within a rack. If that goes wrong, only one rack aka POD
> > has issues and not the rest. We’ve many PODs in several zones. The
> overlay
> > makes tunnels (we use Nicira but vxlan, ovn or nuage can do the same) and
> > those can be created over L3 interconnected PODs. Effectively, this gives
> > you L2 (overlay) over L3 (underlay). Storage wise we have both
> cluster-wide
> > (in the POD) and zone-wide (basically a POD with just storage).
> >
> > VMs in a given guest network can run in any of the PODs and still have a
> > L2 connection between then, even though the actual physical network is
> L3.
> > This is one of the great benefits of SDN (Software defined networking).
> > It’s pretty much the best of both worlds. We scaled this quite a bit and
> > it’s rock solid.
> >
> > Regards,
> > Remi
> >
> >
> > On 05/10/2017, 13:57, "Rafael Weingärtner" <ra...@autonomiccs.com.br>
> > wrote:
> >
> > Exactly; the management IPs is defined per POD already, the public
> you
> > could work out dedicating domains per POD, and then you can dedicate
> a
> > pool of IPs for each domain. The guest networking problem is solved
> if
> > you force users from let´s say same domain to say in the same Pod.
> >
> > The other approach as you said would be a zone per POD.
> >
> > Please keep us posted with your tests, your findings may be valuable
> to
> > spot improvement in ACS design and help others with more complex
> > deployments.
> >
> >
> > On 10/5/2017 6:51 AM, Andrija Panic wrote:
> > > Thanks Rafael,
> > >
> > > yes that is my expectation also (same broadcast domain for Guest
> > network),
> > > so it doesn't really solve my problem (identical thing is expected
> > for
> > > Public Network, at least, if not other networks also)
> > > Other options seems to be zones per each X racks...
> > >
> > > Will see.
> > >
> > > Thanks
> > >
> > > On 4 October 2017 at 22:25, Rafael Weingärtner <
> > rafael@autonomiccs.com.br>
> > > wrote:
> > >
> > >> I think this can cause problems, if not properly managed. Unless
> you
> > >> concentrate Domains/Users in Pods. Otherwise, you might end up
> with
> > some
> > >> VMs of the same user/domain/project in different pods, and if they
> > are all
> > >> in the same VPC for instance, we would expect them to be in the
> same
> > >> broadcast domain.
> > >>
> > >> I think to apply what you want, it may require some designing and
> > testing,
> > >> but it feels feasible with ACS.
> > >>
> > >>
> > >> On 10/4/2017 5:19 PM, Andrija Panic wrote:
> > >>
> > >>> Anyone? I know I'm trying to squeeze some free paid consulting
> > here :),
> > >>> but trying to understand if PODs makes sense in this
> situation....
> > >>>
> > >>> Thx
> > >>>
> > >>> On 2 October 2017 at 10:21, Andrija Panic <
> andrija.panic@gmail.com
> > >
> > >>> wrote:
> > >>>
> > >>> Hi guys,
> > >>>> Sorry for long post below...
> > >>>>
> > >>>> I was wondering if someone could bring some light for me for
> > multiple
> > >>>> PODs
> > >>>> networking design (L2 vs L3) - idea is to make smaller L2
> > broadcast
> > >>>> domains
> > >>>> (any other reason?)
> > >>>>
> > >>>> We might decide to transition from current single pod, single
> > cluster
> > >>>> (single zone) to multiple PODs design (or not...) - we will
> > eventually
> > >>>> grow
> > >>>> to over 50 racks worth of KVM hosts (1000+ hosts) so Im trying
> to
> > >>>> understand best options to avoid having insanely huge L2
> broadcast
> > >>>> domains...
> > >>>>
> > >>>> Mgmt network is routed between pods, that is clear.
> > >>>>
> > >>>> We have dedicated primary storage network and Secondary Storage
> > networks
> > >>>> (vlan interfaces configured locally on all KVM hosts, providing
> > direct L2
> > >>>> connection obviously, not shared with mgmt.network), and same
> for
> > Public
> > >>>> and Guest networks... (Advanced networking in zone, Vxlan used
> as
> > >>>> isolation)
> > >>>>
> > >>>> Now with multiple PODs, since Public Network and Guest network
> is
> > defined
> > >>>> per Zone level (not POD level), and currently same zone-wide
> > setup for
> > >>>> Primary Storage... what would be the best way to make this
> > traffic stay
> > >>>> inside PODs as much as possible and is this possible at all?
> > Perhaps I
> > >>>> would need to look into multiple zones, not PODs.
> > >>>>
> > >>>> My humble conclusion, based on having all dedicated networks, is
> > that I
> > >>>> need to strech (L2 attach as vlan interface) primary and
> secondary
> > >>>> storage
> > >>>> network across all racks/PODs, and also need to strech Guest
> vlan
> > (that
> > >>>> carry all Guest VXLAN tunnels), and again same for Public
> > Network...and
> > >>>> this again makes huge broadcast domains and doesn't solve my
> > issue...
> > >>>> Don't see other option in my head to make networking work across
> > PODs.
> > >>>>
> > >>>> Any suggestion is most welcome (and if of any use as info - we
> > dont plan
> > >>>> for any Xen, VmWare etc, will stay purely with KVM).
> > >>>>
> > >>>> Thanks
> > >>>> Andrija
> > >>>>
> > >>>>
> > >>>
> > >> --
> > >> Rafael Weingärtner
> > >>
> > >>
> > >
> >
> > --
> > Rafael Weingärtner
> >
> >
> >
> >
>
--
Andrija Panić
Re: Advise on multiple PODs network design
Posted by Voloshanenko Igor <ig...@gmail.com>.
Tnx Remi!
Brilliant advice!
чт, 5 окт. 2017 г. в 15:38, Remi Bergsma <RB...@schubergphilis.com>:
> Hi,
>
> We solved this problem by splitting the network in an underlay and overlay
> network. The underlay is the physical network, including the management
> traffic and storage from hypervisors and such. The simpler, the better. In
> the overlay there’s your services layer, for example the guest networks for
> your clients. Since you’re already using vxlan that shouldn’t be hard. In
> our setup, each POD is a rack with TOR (Top-of-rack routing) which means a
> L2 domain stays within a rack. If that goes wrong, only one rack aka POD
> has issues and not the rest. We’ve many PODs in several zones. The overlay
> makes tunnels (we use Nicira but vxlan, ovn or nuage can do the same) and
> those can be created over L3 interconnected PODs. Effectively, this gives
> you L2 (overlay) over L3 (underlay). Storage wise we have both cluster-wide
> (in the POD) and zone-wide (basically a POD with just storage).
>
> VMs in a given guest network can run in any of the PODs and still have a
> L2 connection between then, even though the actual physical network is L3.
> This is one of the great benefits of SDN (Software defined networking).
> It’s pretty much the best of both worlds. We scaled this quite a bit and
> it’s rock solid.
>
> Regards,
> Remi
>
>
> On 05/10/2017, 13:57, "Rafael Weingärtner" <ra...@autonomiccs.com.br>
> wrote:
>
> Exactly; the management IPs is defined per POD already, the public you
> could work out dedicating domains per POD, and then you can dedicate a
> pool of IPs for each domain. The guest networking problem is solved if
> you force users from let´s say same domain to say in the same Pod.
>
> The other approach as you said would be a zone per POD.
>
> Please keep us posted with your tests, your findings may be valuable to
> spot improvement in ACS design and help others with more complex
> deployments.
>
>
> On 10/5/2017 6:51 AM, Andrija Panic wrote:
> > Thanks Rafael,
> >
> > yes that is my expectation also (same broadcast domain for Guest
> network),
> > so it doesn't really solve my problem (identical thing is expected
> for
> > Public Network, at least, if not other networks also)
> > Other options seems to be zones per each X racks...
> >
> > Will see.
> >
> > Thanks
> >
> > On 4 October 2017 at 22:25, Rafael Weingärtner <
> rafael@autonomiccs.com.br>
> > wrote:
> >
> >> I think this can cause problems, if not properly managed. Unless you
> >> concentrate Domains/Users in Pods. Otherwise, you might end up with
> some
> >> VMs of the same user/domain/project in different pods, and if they
> are all
> >> in the same VPC for instance, we would expect them to be in the same
> >> broadcast domain.
> >>
> >> I think to apply what you want, it may require some designing and
> testing,
> >> but it feels feasible with ACS.
> >>
> >>
> >> On 10/4/2017 5:19 PM, Andrija Panic wrote:
> >>
> >>> Anyone? I know I'm trying to squeeze some free paid consulting
> here :),
> >>> but trying to understand if PODs makes sense in this situation....
> >>>
> >>> Thx
> >>>
> >>> On 2 October 2017 at 10:21, Andrija Panic <andrija.panic@gmail.com
> >
> >>> wrote:
> >>>
> >>> Hi guys,
> >>>> Sorry for long post below...
> >>>>
> >>>> I was wondering if someone could bring some light for me for
> multiple
> >>>> PODs
> >>>> networking design (L2 vs L3) - idea is to make smaller L2
> broadcast
> >>>> domains
> >>>> (any other reason?)
> >>>>
> >>>> We might decide to transition from current single pod, single
> cluster
> >>>> (single zone) to multiple PODs design (or not...) - we will
> eventually
> >>>> grow
> >>>> to over 50 racks worth of KVM hosts (1000+ hosts) so Im trying to
> >>>> understand best options to avoid having insanely huge L2 broadcast
> >>>> domains...
> >>>>
> >>>> Mgmt network is routed between pods, that is clear.
> >>>>
> >>>> We have dedicated primary storage network and Secondary Storage
> networks
> >>>> (vlan interfaces configured locally on all KVM hosts, providing
> direct L2
> >>>> connection obviously, not shared with mgmt.network), and same for
> Public
> >>>> and Guest networks... (Advanced networking in zone, Vxlan used as
> >>>> isolation)
> >>>>
> >>>> Now with multiple PODs, since Public Network and Guest network is
> defined
> >>>> per Zone level (not POD level), and currently same zone-wide
> setup for
> >>>> Primary Storage... what would be the best way to make this
> traffic stay
> >>>> inside PODs as much as possible and is this possible at all?
> Perhaps I
> >>>> would need to look into multiple zones, not PODs.
> >>>>
> >>>> My humble conclusion, based on having all dedicated networks, is
> that I
> >>>> need to strech (L2 attach as vlan interface) primary and secondary
> >>>> storage
> >>>> network across all racks/PODs, and also need to strech Guest vlan
> (that
> >>>> carry all Guest VXLAN tunnels), and again same for Public
> Network...and
> >>>> this again makes huge broadcast domains and doesn't solve my
> issue...
> >>>> Don't see other option in my head to make networking work across
> PODs.
> >>>>
> >>>> Any suggestion is most welcome (and if of any use as info - we
> dont plan
> >>>> for any Xen, VmWare etc, will stay purely with KVM).
> >>>>
> >>>> Thanks
> >>>> Andrija
> >>>>
> >>>>
> >>>
> >> --
> >> Rafael Weingärtner
> >>
> >>
> >
>
> --
> Rafael Weingärtner
>
>
>
>
Re: Advise on multiple PODs network design
Posted by Remi Bergsma <RB...@schubergphilis.com>.
Hi,
We solved this problem by splitting the network in an underlay and overlay network. The underlay is the physical network, including the management traffic and storage from hypervisors and such. The simpler, the better. In the overlay there’s your services layer, for example the guest networks for your clients. Since you’re already using vxlan that shouldn’t be hard. In our setup, each POD is a rack with TOR (Top-of-rack routing) which means a L2 domain stays within a rack. If that goes wrong, only one rack aka POD has issues and not the rest. We’ve many PODs in several zones. The overlay makes tunnels (we use Nicira but vxlan, ovn or nuage can do the same) and those can be created over L3 interconnected PODs. Effectively, this gives you L2 (overlay) over L3 (underlay). Storage wise we have both cluster-wide (in the POD) and zone-wide (basically a POD with just storage).
VMs in a given guest network can run in any of the PODs and still have a L2 connection between then, even though the actual physical network is L3. This is one of the great benefits of SDN (Software defined networking). It’s pretty much the best of both worlds. We scaled this quite a bit and it’s rock solid.
Regards,
Remi
On 05/10/2017, 13:57, "Rafael Weingärtner" <ra...@autonomiccs.com.br> wrote:
Exactly; the management IPs is defined per POD already, the public you
could work out dedicating domains per POD, and then you can dedicate a
pool of IPs for each domain. The guest networking problem is solved if
you force users from let´s say same domain to say in the same Pod.
The other approach as you said would be a zone per POD.
Please keep us posted with your tests, your findings may be valuable to
spot improvement in ACS design and help others with more complex
deployments.
On 10/5/2017 6:51 AM, Andrija Panic wrote:
> Thanks Rafael,
>
> yes that is my expectation also (same broadcast domain for Guest network),
> so it doesn't really solve my problem (identical thing is expected for
> Public Network, at least, if not other networks also)
> Other options seems to be zones per each X racks...
>
> Will see.
>
> Thanks
>
> On 4 October 2017 at 22:25, Rafael Weingärtner <ra...@autonomiccs.com.br>
> wrote:
>
>> I think this can cause problems, if not properly managed. Unless you
>> concentrate Domains/Users in Pods. Otherwise, you might end up with some
>> VMs of the same user/domain/project in different pods, and if they are all
>> in the same VPC for instance, we would expect them to be in the same
>> broadcast domain.
>>
>> I think to apply what you want, it may require some designing and testing,
>> but it feels feasible with ACS.
>>
>>
>> On 10/4/2017 5:19 PM, Andrija Panic wrote:
>>
>>> Anyone? I know I'm trying to squeeze some free paid consulting here :),
>>> but trying to understand if PODs makes sense in this situation....
>>>
>>> Thx
>>>
>>> On 2 October 2017 at 10:21, Andrija Panic <an...@gmail.com>
>>> wrote:
>>>
>>> Hi guys,
>>>> Sorry for long post below...
>>>>
>>>> I was wondering if someone could bring some light for me for multiple
>>>> PODs
>>>> networking design (L2 vs L3) - idea is to make smaller L2 broadcast
>>>> domains
>>>> (any other reason?)
>>>>
>>>> We might decide to transition from current single pod, single cluster
>>>> (single zone) to multiple PODs design (or not...) - we will eventually
>>>> grow
>>>> to over 50 racks worth of KVM hosts (1000+ hosts) so Im trying to
>>>> understand best options to avoid having insanely huge L2 broadcast
>>>> domains...
>>>>
>>>> Mgmt network is routed between pods, that is clear.
>>>>
>>>> We have dedicated primary storage network and Secondary Storage networks
>>>> (vlan interfaces configured locally on all KVM hosts, providing direct L2
>>>> connection obviously, not shared with mgmt.network), and same for Public
>>>> and Guest networks... (Advanced networking in zone, Vxlan used as
>>>> isolation)
>>>>
>>>> Now with multiple PODs, since Public Network and Guest network is defined
>>>> per Zone level (not POD level), and currently same zone-wide setup for
>>>> Primary Storage... what would be the best way to make this traffic stay
>>>> inside PODs as much as possible and is this possible at all? Perhaps I
>>>> would need to look into multiple zones, not PODs.
>>>>
>>>> My humble conclusion, based on having all dedicated networks, is that I
>>>> need to strech (L2 attach as vlan interface) primary and secondary
>>>> storage
>>>> network across all racks/PODs, and also need to strech Guest vlan (that
>>>> carry all Guest VXLAN tunnels), and again same for Public Network...and
>>>> this again makes huge broadcast domains and doesn't solve my issue...
>>>> Don't see other option in my head to make networking work across PODs.
>>>>
>>>> Any suggestion is most welcome (and if of any use as info - we dont plan
>>>> for any Xen, VmWare etc, will stay purely with KVM).
>>>>
>>>> Thanks
>>>> Andrija
>>>>
>>>>
>>>
>> --
>> Rafael Weingärtner
>>
>>
>
--
Rafael Weingärtner
Re: Advise on multiple PODs network design
Posted by Rafael Weingärtner <ra...@autonomiccs.com.br>.
Exactly; the management IPs is defined per POD already, the public you
could work out dedicating domains per POD, and then you can dedicate a
pool of IPs for each domain. The guest networking problem is solved if
you force users from let´s say same domain to say in the same Pod.
The other approach as you said would be a zone per POD.
Please keep us posted with your tests, your findings may be valuable to
spot improvement in ACS design and help others with more complex
deployments.
On 10/5/2017 6:51 AM, Andrija Panic wrote:
> Thanks Rafael,
>
> yes that is my expectation also (same broadcast domain for Guest network),
> so it doesn't really solve my problem (identical thing is expected for
> Public Network, at least, if not other networks also)
> Other options seems to be zones per each X racks...
>
> Will see.
>
> Thanks
>
> On 4 October 2017 at 22:25, Rafael Weingärtner <ra...@autonomiccs.com.br>
> wrote:
>
>> I think this can cause problems, if not properly managed. Unless you
>> concentrate Domains/Users in Pods. Otherwise, you might end up with some
>> VMs of the same user/domain/project in different pods, and if they are all
>> in the same VPC for instance, we would expect them to be in the same
>> broadcast domain.
>>
>> I think to apply what you want, it may require some designing and testing,
>> but it feels feasible with ACS.
>>
>>
>> On 10/4/2017 5:19 PM, Andrija Panic wrote:
>>
>>> Anyone? I know I'm trying to squeeze some free paid consulting here :),
>>> but trying to understand if PODs makes sense in this situation....
>>>
>>> Thx
>>>
>>> On 2 October 2017 at 10:21, Andrija Panic <an...@gmail.com>
>>> wrote:
>>>
>>> Hi guys,
>>>> Sorry for long post below...
>>>>
>>>> I was wondering if someone could bring some light for me for multiple
>>>> PODs
>>>> networking design (L2 vs L3) - idea is to make smaller L2 broadcast
>>>> domains
>>>> (any other reason?)
>>>>
>>>> We might decide to transition from current single pod, single cluster
>>>> (single zone) to multiple PODs design (or not...) - we will eventually
>>>> grow
>>>> to over 50 racks worth of KVM hosts (1000+ hosts) so Im trying to
>>>> understand best options to avoid having insanely huge L2 broadcast
>>>> domains...
>>>>
>>>> Mgmt network is routed between pods, that is clear.
>>>>
>>>> We have dedicated primary storage network and Secondary Storage networks
>>>> (vlan interfaces configured locally on all KVM hosts, providing direct L2
>>>> connection obviously, not shared with mgmt.network), and same for Public
>>>> and Guest networks... (Advanced networking in zone, Vxlan used as
>>>> isolation)
>>>>
>>>> Now with multiple PODs, since Public Network and Guest network is defined
>>>> per Zone level (not POD level), and currently same zone-wide setup for
>>>> Primary Storage... what would be the best way to make this traffic stay
>>>> inside PODs as much as possible and is this possible at all? Perhaps I
>>>> would need to look into multiple zones, not PODs.
>>>>
>>>> My humble conclusion, based on having all dedicated networks, is that I
>>>> need to strech (L2 attach as vlan interface) primary and secondary
>>>> storage
>>>> network across all racks/PODs, and also need to strech Guest vlan (that
>>>> carry all Guest VXLAN tunnels), and again same for Public Network...and
>>>> this again makes huge broadcast domains and doesn't solve my issue...
>>>> Don't see other option in my head to make networking work across PODs.
>>>>
>>>> Any suggestion is most welcome (and if of any use as info - we dont plan
>>>> for any Xen, VmWare etc, will stay purely with KVM).
>>>>
>>>> Thanks
>>>> Andrija
>>>>
>>>>
>>>
>> --
>> Rafael Weingärtner
>>
>>
>
--
Rafael Weingärtner
Re: Advise on multiple PODs network design
Posted by Andrija Panic <an...@gmail.com>.
Thanks Rafael,
yes that is my expectation also (same broadcast domain for Guest network),
so it doesn't really solve my problem (identical thing is expected for
Public Network, at least, if not other networks also)
Other options seems to be zones per each X racks...
Will see.
Thanks
On 4 October 2017 at 22:25, Rafael Weingärtner <ra...@autonomiccs.com.br>
wrote:
> I think this can cause problems, if not properly managed. Unless you
> concentrate Domains/Users in Pods. Otherwise, you might end up with some
> VMs of the same user/domain/project in different pods, and if they are all
> in the same VPC for instance, we would expect them to be in the same
> broadcast domain.
>
> I think to apply what you want, it may require some designing and testing,
> but it feels feasible with ACS.
>
>
> On 10/4/2017 5:19 PM, Andrija Panic wrote:
>
>> Anyone? I know I'm trying to squeeze some free paid consulting here :),
>> but trying to understand if PODs makes sense in this situation....
>>
>> Thx
>>
>> On 2 October 2017 at 10:21, Andrija Panic <an...@gmail.com>
>> wrote:
>>
>> Hi guys,
>>>
>>> Sorry for long post below...
>>>
>>> I was wondering if someone could bring some light for me for multiple
>>> PODs
>>> networking design (L2 vs L3) - idea is to make smaller L2 broadcast
>>> domains
>>> (any other reason?)
>>>
>>> We might decide to transition from current single pod, single cluster
>>> (single zone) to multiple PODs design (or not...) - we will eventually
>>> grow
>>> to over 50 racks worth of KVM hosts (1000+ hosts) so Im trying to
>>> understand best options to avoid having insanely huge L2 broadcast
>>> domains...
>>>
>>> Mgmt network is routed between pods, that is clear.
>>>
>>> We have dedicated primary storage network and Secondary Storage networks
>>> (vlan interfaces configured locally on all KVM hosts, providing direct L2
>>> connection obviously, not shared with mgmt.network), and same for Public
>>> and Guest networks... (Advanced networking in zone, Vxlan used as
>>> isolation)
>>>
>>> Now with multiple PODs, since Public Network and Guest network is defined
>>> per Zone level (not POD level), and currently same zone-wide setup for
>>> Primary Storage... what would be the best way to make this traffic stay
>>> inside PODs as much as possible and is this possible at all? Perhaps I
>>> would need to look into multiple zones, not PODs.
>>>
>>> My humble conclusion, based on having all dedicated networks, is that I
>>> need to strech (L2 attach as vlan interface) primary and secondary
>>> storage
>>> network across all racks/PODs, and also need to strech Guest vlan (that
>>> carry all Guest VXLAN tunnels), and again same for Public Network...and
>>> this again makes huge broadcast domains and doesn't solve my issue...
>>> Don't see other option in my head to make networking work across PODs.
>>>
>>> Any suggestion is most welcome (and if of any use as info - we dont plan
>>> for any Xen, VmWare etc, will stay purely with KVM).
>>>
>>> Thanks
>>> Andrija
>>>
>>>
>>
>>
> --
> Rafael Weingärtner
>
>
--
Andrija Panić
Re: Advise on multiple PODs network design
Posted by Rafael Weingärtner <ra...@autonomiccs.com.br>.
I think this can cause problems, if not properly managed. Unless you
concentrate Domains/Users in Pods. Otherwise, you might end up with some
VMs of the same user/domain/project in different pods, and if they are
all in the same VPC for instance, we would expect them to be in the same
broadcast domain.
I think to apply what you want, it may require some designing and
testing, but it feels feasible with ACS.
On 10/4/2017 5:19 PM, Andrija Panic wrote:
> Anyone? I know I'm trying to squeeze some free paid consulting here :),
> but trying to understand if PODs makes sense in this situation....
>
> Thx
>
> On 2 October 2017 at 10:21, Andrija Panic <an...@gmail.com> wrote:
>
>> Hi guys,
>>
>> Sorry for long post below...
>>
>> I was wondering if someone could bring some light for me for multiple PODs
>> networking design (L2 vs L3) - idea is to make smaller L2 broadcast domains
>> (any other reason?)
>>
>> We might decide to transition from current single pod, single cluster
>> (single zone) to multiple PODs design (or not...) - we will eventually grow
>> to over 50 racks worth of KVM hosts (1000+ hosts) so Im trying to
>> understand best options to avoid having insanely huge L2 broadcast
>> domains...
>>
>> Mgmt network is routed between pods, that is clear.
>>
>> We have dedicated primary storage network and Secondary Storage networks
>> (vlan interfaces configured locally on all KVM hosts, providing direct L2
>> connection obviously, not shared with mgmt.network), and same for Public
>> and Guest networks... (Advanced networking in zone, Vxlan used as isolation)
>>
>> Now with multiple PODs, since Public Network and Guest network is defined
>> per Zone level (not POD level), and currently same zone-wide setup for
>> Primary Storage... what would be the best way to make this traffic stay
>> inside PODs as much as possible and is this possible at all? Perhaps I
>> would need to look into multiple zones, not PODs.
>>
>> My humble conclusion, based on having all dedicated networks, is that I
>> need to strech (L2 attach as vlan interface) primary and secondary storage
>> network across all racks/PODs, and also need to strech Guest vlan (that
>> carry all Guest VXLAN tunnels), and again same for Public Network...and
>> this again makes huge broadcast domains and doesn't solve my issue...
>> Don't see other option in my head to make networking work across PODs.
>>
>> Any suggestion is most welcome (and if of any use as info - we dont plan
>> for any Xen, VmWare etc, will stay purely with KVM).
>>
>> Thanks
>> Andrija
>>
>
>
--
Rafael Weingärtner
Re: Advise on multiple PODs network design
Posted by Andrija Panic <an...@gmail.com>.
Anyone? I know I'm trying to squeeze some free paid consulting here :),
but trying to understand if PODs makes sense in this situation....
Thx
On 2 October 2017 at 10:21, Andrija Panic <an...@gmail.com> wrote:
> Hi guys,
>
> Sorry for long post below...
>
> I was wondering if someone could bring some light for me for multiple PODs
> networking design (L2 vs L3) - idea is to make smaller L2 broadcast domains
> (any other reason?)
>
> We might decide to transition from current single pod, single cluster
> (single zone) to multiple PODs design (or not...) - we will eventually grow
> to over 50 racks worth of KVM hosts (1000+ hosts) so Im trying to
> understand best options to avoid having insanely huge L2 broadcast
> domains...
>
> Mgmt network is routed between pods, that is clear.
>
> We have dedicated primary storage network and Secondary Storage networks
> (vlan interfaces configured locally on all KVM hosts, providing direct L2
> connection obviously, not shared with mgmt.network), and same for Public
> and Guest networks... (Advanced networking in zone, Vxlan used as isolation)
>
> Now with multiple PODs, since Public Network and Guest network is defined
> per Zone level (not POD level), and currently same zone-wide setup for
> Primary Storage... what would be the best way to make this traffic stay
> inside PODs as much as possible and is this possible at all? Perhaps I
> would need to look into multiple zones, not PODs.
>
> My humble conclusion, based on having all dedicated networks, is that I
> need to strech (L2 attach as vlan interface) primary and secondary storage
> network across all racks/PODs, and also need to strech Guest vlan (that
> carry all Guest VXLAN tunnels), and again same for Public Network...and
> this again makes huge broadcast domains and doesn't solve my issue...
> Don't see other option in my head to make networking work across PODs.
>
> Any suggestion is most welcome (and if of any use as info - we dont plan
> for any Xen, VmWare etc, will stay purely with KVM).
>
> Thanks
> Andrija
>
--
Andrija Panić
Re: Advise on multiple PODs network design
Posted by Andrija Panic <an...@gmail.com>.
Anyone? I know I'm trying to squeeze some free paid consulting here :),
but trying to understand if PODs makes sense in this situation....
Thx
On 2 October 2017 at 10:21, Andrija Panic <an...@gmail.com> wrote:
> Hi guys,
>
> Sorry for long post below...
>
> I was wondering if someone could bring some light for me for multiple PODs
> networking design (L2 vs L3) - idea is to make smaller L2 broadcast domains
> (any other reason?)
>
> We might decide to transition from current single pod, single cluster
> (single zone) to multiple PODs design (or not...) - we will eventually grow
> to over 50 racks worth of KVM hosts (1000+ hosts) so Im trying to
> understand best options to avoid having insanely huge L2 broadcast
> domains...
>
> Mgmt network is routed between pods, that is clear.
>
> We have dedicated primary storage network and Secondary Storage networks
> (vlan interfaces configured locally on all KVM hosts, providing direct L2
> connection obviously, not shared with mgmt.network), and same for Public
> and Guest networks... (Advanced networking in zone, Vxlan used as isolation)
>
> Now with multiple PODs, since Public Network and Guest network is defined
> per Zone level (not POD level), and currently same zone-wide setup for
> Primary Storage... what would be the best way to make this traffic stay
> inside PODs as much as possible and is this possible at all? Perhaps I
> would need to look into multiple zones, not PODs.
>
> My humble conclusion, based on having all dedicated networks, is that I
> need to strech (L2 attach as vlan interface) primary and secondary storage
> network across all racks/PODs, and also need to strech Guest vlan (that
> carry all Guest VXLAN tunnels), and again same for Public Network...and
> this again makes huge broadcast domains and doesn't solve my issue...
> Don't see other option in my head to make networking work across PODs.
>
> Any suggestion is most welcome (and if of any use as info - we dont plan
> for any Xen, VmWare etc, will stay purely with KVM).
>
> Thanks
> Andrija
>
--
Andrija Panić