[MERGE] network-guru-orchestration into master

[Hugo Trippaers <hu...@trippaers.nl>]

Hey guys,

This is something i had on my wish list for some time. The current way network gurus are handled is that each guru is asked to design a network and will decide by itself to either do it or don’t. Most gurus have sane checks on which types of networks it can handle, but we have seen issues there in the past.

With these changes the network orchestrator will check the capabilities of a guru and based on that ask one or more gurus to design the network. With this the power is where is should new, the network orchestrator. 

This also means that new networking plugins with gurus will have an easier integration, just list the capabilities. It will also save some database calls (once i clean out all canHandle functions) as gurus will have to lookup less to decide if they should to their job.

What do you guys think?

Current branch is tested with devcloud in a manual test, so there is more work to do there. I wanted to get some opinions while performing more tests.

Cheers,

Hugo

Re: [MERGE] network-guru-orchestration into master

[Marcus Sorensen <sh...@gmail.com>]

On Fri, Nov 1, 2013 at 10:16 AM, Pedro Roque Marques
<pe...@gmail.com> wrote:
> Darren,
>
> On Oct 31, 2013, at 10:05 AM, Darren Shepherd <da...@gmail.com> wrote:
>
>> Yeah I think it would be great to talk about this at CCC.  I'm
>> hesitant to further narrow down the definition of the network.  For
>> example, I think OpenStack's Neutron is fundamentally flawed because
>> they defined a network as a L2 segment.
>
> OpenContrail implements a Neutron plugin. It uses the Neutron API to provide the concept of a virtual-network. The virtual-network can be a collection of IP subnets that work as a closed user group; by configuring a network-policy between virtual-networks the user/admin can define additional connectivity for the network. The same functionality can be achieved using the AWS VPC API. We have extended the Neutron API with the concept of network-policy but have not changed the underlying concept of network; the 1.00 release of the software provides an IP service to the guest-only (the latest release provides fallback bridging for non-IP traffic also). While i don't have a firm opinion on the Neutron API, it does not limit the network to be an L2 segment.
>
>> In the world of SDN, I think its even more important to keep the
>> definition of the a network loose.  SDN has the capability of
>> completely changing the way we look at L2 and L3.  Currently in
>> networking we group things by L3 and L2 concepts as that is how
>> routers and switches are laid out today.  As SDN matures and you see
>> more flow oriented design it won't make sense to group things using L2
>> and L3 concepts (as those become more a physical fabric technology),
>> the groups becomes more loose and thus the definition of a network
>> should be loose.
>
> I don't believe there is an accepted definition of SDN. My perspective and the goal for OpenContrail is to decouple the physical network from the service provided to the "edge" (the virtual-machines in this case). The goal is to allow the physical underlay to be designed for throughput and high inter-connectivity (e.g. CLOS topology); while implementing the functionality traditionally found in an aggregation switch (the L2/L3 boundary) in the host.
>
> The logic is that to get the highest server utilization one needs to be able to schedule a VM (or LXC) anywhere in the cluster; this implies much greater data throughput requirements. The standard operating procedure used to be to aim for I/O locality by placing multiple components of an application stack in the same rack. In the traditional design you can easily find a 20:1 over-subscription between server ports and the actual throughput of the network core.
>
> Once you spread the server load around, the network requirements go up to design points like 2:1 oversub. This requires a different physical design for the network and makes it so that there isn't a pair of aggregation switches nicely positioned above the rack in order to implement policies that control network-to-network traffic. This is the reason that OpenContrail tries to implement network-to-network traffic policies in the ingress hypervisor switch and forward traffic directly without requiring a VirtualRouter appliance.
>
> Just to provide one less fluffy definition of what is the problem we are trying to solve...
>
>>
>> Now that's not to say that a network can't provide L2 and L3
>> information.  You should be able to create a network in CloudStack and
>> based on the configuration you know that it is a single L2 or L3.  It
>> is just that the core orchestration system can't make that fundamental
>> assumption.  I'd be interested in furthering the model and maybe
>> adding a concept of a L2 network such that a network guru when
>> designing a network, can define multiple l2networks and associate them
>> with the generic network that was created.  That idea I'm still
>> toiling with.
>
> I'd encourage you to not thing about L2 networks. I've yet to see an application that is "cloud-ready" that needs anything but IP connectivity. For IP it doesn't matter what the underlying data layer looks like... emulating ethernet is a rat-hole. There is no point in doing so.

May be true in the sense that 'cloud-ready' applications are generally
just web/application servers that are ephemeral, but I'd just like to
point out that many folks aren't using CloudStack to provide cloud
servers, they're using it to provide traditional or hybrid
infrastructure.  Throwing out layer 2 to me seems like throwing away
the whole concept of a VPC.  Or perhaps you're just saying that it can
be emulated by managing ACLs on a per-VM basis, like security groups,
and that no applications actually need to be on the same subnet or
broadcast domain. I'm not sure that can be assumed, for example
DSR-style load balancing requires a real layer 2.

>
>>
>> For example, when configuring DHCP on the systemvm.  DHCP is a L2
>> based serviced.
>
> DHCP is an IP service. Typically provided via a DHCP relay service in the aggregation switch. For instance in OpenContrail this is provided in the hypervisor switch (aka vrouter linux kernel module).
>
>>  So to configure DHCP you really need to know for each
>> nic, what is the L2 its attached to and what are the VMs associated
>> with that L2.  Today, since there is no first class concept of a L2
>> network, you have to look at the implied definition of L2.  For basic
>> networks, the L2 is the Pod, so you need to list all VMs in that Pod.
>> For guest/VPC networks, the L2 is the network object, so you need to
>> list all VMs associated with the network.  It would be nice if when
>> the guru designed the network, it also defined the l2networks, and
>> then when a VM starts the guru the reserve() method could associate
>> the l2network to the nic.  So the nic object would have a network_id
>> and a l2_network_id.
>
> With OpenContrail, DHCP is quite simple. The Nic uuid is known by the vrouter kernel module on the compute-node. When the DHCP request comes from the tap/vif interface the vrouter answers locally (it known the relationship between Nic, its properties and virtual-network). Please do not try to bring L2 into the picture. It would be very unhelpful to do so.
>
> For most data-centers, the main networking objective is to get rid of L2 and its limitations. Ethernet is really complex. It has a nice zero config deployment for very simple networks but at the cost of high complexity if you are trying to do redundancy, use multiple links, interoperate with other network devices, scale.... not to mention that all state is data-driven which makes it really really hard to debug. Ethernet as a layer 1 point to point link is great; not as a network.
>
>   Pedro.

Re: [MERGE] network-guru-orchestration into master

[Pedro Roque Marques <pe...@gmail.com>]

Darren,

On Oct 31, 2013, at 10:05 AM, Darren Shepherd <da...@gmail.com> wrote:

> Yeah I think it would be great to talk about this at CCC.  I'm
> hesitant to further narrow down the definition of the network.  For
> example, I think OpenStack's Neutron is fundamentally flawed because
> they defined a network as a L2 segment.

OpenContrail implements a Neutron plugin. It uses the Neutron API to provide the concept of a virtual-network. The virtual-network can be a collection of IP subnets that work as a closed user group; by configuring a network-policy between virtual-networks the user/admin can define additional connectivity for the network. The same functionality can be achieved using the AWS VPC API. We have extended the Neutron API with the concept of network-policy but have not changed the underlying concept of network; the 1.00 release of the software provides an IP service to the guest-only (the latest release provides fallback bridging for non-IP traffic also). While i don't have a firm opinion on the Neutron API, it does not limit the network to be an L2 segment.

> In the world of SDN, I think its even more important to keep the
> definition of the a network loose.  SDN has the capability of
> completely changing the way we look at L2 and L3.  Currently in
> networking we group things by L3 and L2 concepts as that is how
> routers and switches are laid out today.  As SDN matures and you see
> more flow oriented design it won't make sense to group things using L2
> and L3 concepts (as those become more a physical fabric technology),
> the groups becomes more loose and thus the definition of a network
> should be loose.

I don't believe there is an accepted definition of SDN. My perspective and the goal for OpenContrail is to decouple the physical network from the service provided to the "edge" (the virtual-machines in this case). The goal is to allow the physical underlay to be designed for throughput and high inter-connectivity (e.g. CLOS topology); while implementing the functionality traditionally found in an aggregation switch (the L2/L3 boundary) in the host.

The logic is that to get the highest server utilization one needs to be able to schedule a VM (or LXC) anywhere in the cluster; this implies much greater data throughput requirements. The standard operating procedure used to be to aim for I/O locality by placing multiple components of an application stack in the same rack. In the traditional design you can easily find a 20:1 over-subscription between server ports and the actual throughput of the network core.

Once you spread the server load around, the network requirements go up to design points like 2:1 oversub. This requires a different physical design for the network and makes it so that there isn't a pair of aggregation switches nicely positioned above the rack in order to implement policies that control network-to-network traffic. This is the reason that OpenContrail tries to implement network-to-network traffic policies in the ingress hypervisor switch and forward traffic directly without requiring a VirtualRouter appliance.

Just to provide one less fluffy definition of what is the problem we are trying to solve...

> 
> Now that's not to say that a network can't provide L2 and L3
> information.  You should be able to create a network in CloudStack and
> based on the configuration you know that it is a single L2 or L3.  It
> is just that the core orchestration system can't make that fundamental
> assumption.  I'd be interested in furthering the model and maybe
> adding a concept of a L2 network such that a network guru when
> designing a network, can define multiple l2networks and associate them
> with the generic network that was created.  That idea I'm still
> toiling with.

I'd encourage you to not thing about L2 networks. I've yet to see an application that is "cloud-ready" that needs anything but IP connectivity. For IP it doesn't matter what the underlying data layer looks like... emulating ethernet is a rat-hole. There is no point in doing so.

> 
> For example, when configuring DHCP on the systemvm.  DHCP is a L2
> based serviced.

DHCP is an IP service. Typically provided via a DHCP relay service in the aggregation switch. For instance in OpenContrail this is provided in the hypervisor switch (aka vrouter linux kernel module).

>  So to configure DHCP you really need to know for each
> nic, what is the L2 its attached to and what are the VMs associated
> with that L2.  Today, since there is no first class concept of a L2
> network, you have to look at the implied definition of L2.  For basic
> networks, the L2 is the Pod, so you need to list all VMs in that Pod.
> For guest/VPC networks, the L2 is the network object, so you need to
> list all VMs associated with the network.  It would be nice if when
> the guru designed the network, it also defined the l2networks, and
> then when a VM starts the guru the reserve() method could associate
> the l2network to the nic.  So the nic object would have a network_id
> and a l2_network_id.

With OpenContrail, DHCP is quite simple. The Nic uuid is known by the vrouter kernel module on the compute-node. When the DHCP request comes from the tap/vif interface the vrouter answers locally (it known the relationship between Nic, its properties and virtual-network). Please do not try to bring L2 into the picture. It would be very unhelpful to do so.

For most data-centers, the main networking objective is to get rid of L2 and its limitations. Ethernet is really complex. It has a nice zero config deployment for very simple networks but at the cost of high complexity if you are trying to do redundancy, use multiple links, interoperate with other network devices, scale.... not to mention that all state is data-driven which makes it really really hard to debug. Ethernet as a layer 1 point to point link is great; not as a network.

  Pedro.

Re: [MERGE] network-guru-orchestration into master

[Darren Shepherd <da...@gmail.com>]

Yeah I think it would be great to talk about this at CCC.  I'm
hesitant to further narrow down the definition of the network.  For
example, I think OpenStack's Neutron is fundamentally flawed because
they defined a network as a L2 segment.  During my time that I was
away from CloudStack the physical network concept was added as a first
class object in CloudStack.  I don't know if I fully agree with it.  I
have to dig deeper and just haven't had the time to.  My inclination
is that is was one step too far and that it isn't really needed.  I
know you need to associate traffic with essentially a bridge on the
hypervisor, but that can be done with just labelling the network
offering that created the network.  So I need to dig into this
further.

In the world of SDN, I think its even more important to keep the
definition of the a network loose.  SDN has the capability of
completely changing the way we look at L2 and L3.  Currently in
networking we group things by L3 and L2 concepts as that is how
routers and switches are laid out today.  As SDN matures and you see
more flow oriented design it won't make sense to group things using L2
and L3 concepts (as those become more a physical fabric technology),
the groups becomes more loose and thus the definition of a network
should be loose.

Now that's not to say that a network can't provide L2 and L3
information.  You should be able to create a network in CloudStack and
based on the configuration you know that it is a single L2 or L3.  It
is just that the core orchestration system can't make that fundamental
assumption.  I'd be interested in furthering the model and maybe
adding a concept of a L2 network such that a network guru when
designing a network, can define multiple l2networks and associate them
with the generic network that was created.  That idea I'm still
toiling with.

For example, when configuring DHCP on the systemvm.  DHCP is a L2
based serviced.  So to configure DHCP you really need to know for each
nic, what is the L2 its attached to and what are the VMs associated
with that L2.  Today, since there is no first class concept of a L2
network, you have to look at the implied definition of L2.  For basic
networks, the L2 is the Pod, so you need to list all VMs in that Pod.
For guest/VPC networks, the L2 is the network object, so you need to
list all VMs associated with the network.  It would be nice if when
the guru designed the network, it also defined the l2networks, and
then when a VM starts the guru the reserve() method could associate
the l2network to the nic.  So the nic object would have a network_id
and a l2_network_id.

I've gone off on a total tangent, but yeah, we should talk.

Darren

On Thu, Oct 31, 2013 at 2:53 AM, Hugo Trippaers <hu...@trippaers.nl> wrote:
>
> On 31 okt. 2013, at 03:48, Darren Shepherd <da...@gmail.com> wrote:
>
>> Its important to remember that the definition of a Network in
>> CloudStack is quite purposefully very loose.  A network in the
>> simplest terms is just a group of VMs.  A guru creates and manages
>> that network.  A network is not strictly defined as a union of
>> traffictype/isolationtype/networktype/guesttype.  So, for example,
>> there need not be a 1-to-1 mapping between Guru and IsolationType.
>> You should be able to have multiple gurus per any context.  That's the
>> theory and vision.
>
> I think that vision was put down somewhere before we entered the era of software defined networking. At the moment it is simply not true as long as the guru is responsible for creating the overlay networks there is a 1 on 1 relation with the guru and the supported isolation types in several cases.
>
>>
>> Now the reality is that the current selection logic of the guru in the
>> orchestrator doesn't fully back up that vision because the selection
>> logic too easily allows two Guru's to implement the same network.
>>
>> So I think the current behavior of setupNetwork is flawed.  This
>> change, I feel, muddies to a certain degree the definition of a
>> network in CloudStack and attempts to say a network is strict union of
>> traffic/isolation/network/guest-type.  If we merge this change, does
>> it hurt anything?  No, its more just conceptual disagreement.  When I
>> get to the point that I feel setupNetwork is limiting what I want to
>> do, I'll propose a change.  Right now, I just accept that its not
>> right in my mind.
>
> I’ve fixed the definition of setupNetwork so the function returns a single network when called. This was already the effective behavior, but now its put down in code as well. I’ll push that change to the branch today.
>
>
>>
>> So I'd have to say I'm neutral on this change.  I don't care too much
>> either way as I think down the line this all needs to change.
>
> Let’s find a moment to sit down at CCCEU and discuss this further. I have some ideas of where i want to go with the networking bit as well and it would be nice if we are more or less on the same page before we start working on it. Saying that you don’t care because you think its going to change it anyway isn’t really the best way to approach this.
>
>>
>> Darren
>>
>> On Wed, Oct 30, 2013 at 11:36 AM, Pedro Roque Marques
>> <pe...@gmail.com> wrote:
>>>
>>> On Oct 30, 2013, at 11:08 AM, Darren Shepherd <da...@gmail.com> wrote:
>>>
>>>> I definitely don't want the setup to be done at a zone level.
>>>
>>> Why ? That seems to me the most obvious way to do it.
>>> There are different networking solutions: e.g. VLANs and overlays such as OpenContrail that assume an L3 switching topology.  For a deployment one would tend to choose a solution associated with the physical network.
>>>
>>>> Specifically, I'd like to get rid of the notion of basic and advanced
>>>> zones which would mean that you could be mixing gurus in the same
>>>> zone.  If anything, I'd say the simplest would be to just specify the
>>>> Guru in the network offering.
>>>
>>> Basic and advanced zones is rather unhelpful, i agree. But you still can't Mix Guru's unless they can interoperate... The way for them to interop (i.e. minimum common denominator) is to connect at L3 in an router between zones.
>>>
>>>>
>>>> I like the work that Hugo has done, but the implications of that
>>>> change are 2 things.  First, we only want one guru to match.  If that
>>>> is the case, then lets change the NetworkOrchestrator.setupNetwork()
>>>> to just return one network.  I have a hard time understanding why you
>>>> would want to return two or more.  Even if the guru created two
>>>> networks internally, only one should be return to the caller.  There's
>>>> not a good way to invoke setupNetwork() and handle cases where more
>>>> than one network comes back.
>>>>
>>>> Second, we are saying the only way to choose a guru is based on
>>>> networkType/trafficType/guestType/isolationmethods.  I think the core
>>>> logic should be more flexible than that.  We can first check those 4
>>>> items, and then if we find more than one guru that matches, use the
>>>> best one (before what I said about adding canHandle to the interface).
>>>> But the check shouldn't be only those four criteria.  Hugo's change
>>>> does not strictly enforce that, but it is essentially implied.
>>>>
>>>> After saying all that, if we just put the Guru in the network
>>>> offering.  Would that work for everything?  That would be a really
>>>> simple and explicit way to configure this all.
>>>
>>> My preference would be for a NetworkManager per zone.
>>>
>>>>
>>>> Darren
>>>>
>>>>
>>>> On Wed, Oct 30, 2013 at 10:31 AM, Alex Huang <Al...@citrix.com> wrote:
>>>>> I agree with Hugo that the current way of ask NetworkGuru to decide itself is not desirable.  It makes absolute sense to change that but I wonder why do it through capabilities.  Why not just ask the system admin to choose when they setup the zone?   When I originally designed NetworkGuru, I figured there's a difference in the L2-L3 technology deployed and the network services (L4-L7) provided on top of it.  So I separated out NetworkGuru and NetworkElement.  However, I didn't think it's likely that there would be two or three different type of L2-L3 technologies deployed within the same zone.  I figured we can leave it to the system admin to just pick which set of NetworkGurus should be deployed in a certain zone and that's good enough.
>>>>>
>>>>> I do think there should be more tie in between the NetworkElements and NetworkGurus.  For example, whether a certain NetworkElement can work with the L2-L3 technology deployed.
>>>>>
>>>>> --Alex
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Darren Shepherd [mailto:darren.s.shepherd@gmail.com]
>>>>>> Sent: Wednesday, October 30, 2013 10:11 AM
>>>>>> To: dev@cloudstack.apache.org; Alex Huang; Chiradeep Vittal
>>>>>> Subject: Re: [MERGE] network-guru-orchestration into master
>>>>>>
>>>>>> I don't particular like saying that picking a Guru is based solely on that criteria.
>>>>>> It should be much looser than that.  If the problem you are trying to fix is two
>>>>>> Gurus implement the same network then we need to set back a bit.  I'd like
>>>>>> Alex or Chiradeep to weigh in on this.  Currently setupNetwork returns a list
>>>>>> of networks that were created.  I've looked at the code and every single
>>>>>> invocation to setupNetwork expects (and mostly hard code) a response of
>>>>>> one network.
>>>>>>
>>>>>> So for a more proper fix I'd say that the API of setupNetwork should just
>>>>>> return Network.  Additionally NetworkGuru should have a
>>>>>> canHandle/canDesign method on the interface that returns a priority/score
>>>>>> (similar to what was just done in the storage strategies).  Then the
>>>>>> orchestrator can find the best match and then use the correct guru.  Now the
>>>>>> consolidated logic that you've done should be in a abstract base class that
>>>>>> handles the basic cases of traffic type, network type, etc.
>>>>>>
>>>>>> Darren
>>>>>>
>>>>>> On Wed, Oct 30, 2013 at 6:10 AM, Hugo Trippaers <hu...@trippaers.nl> wrote:
>>>>>>>
>>>>>>> On 30 okt. 2013, at 02:09, Darren Shepherd <da...@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>> First, I like the idea of consolidating logic.  I could see also
>>>>>>>> implementing this as just an abstract base class that handles this
>>>>>>>> common logic.  I'm not sure which approach I prefer.  Exactly what is
>>>>>>>> the problem that you are trying to solve?  Without more details, I'd
>>>>>>>> lean towards implementing this logic in an abstract base class that
>>>>>>>> all NetworkGurus can choose to extend.
>>>>>>>>
>>>>>>>
>>>>>>> Not as much a problem as a design choice. It is my intention to use
>>>>>>> capabilities to select the NetworkGuru instead of asking each network
>>>>>>> guru in turn if it wants to design the network. The idea here is that
>>>>>>> new network gurus only need to supply a list of capabilities to be
>>>>>>> integrated into cloudstack. Like i can handle isolation type X in
>>>>>>> advanced networks for traffic type Guest. The network orchestrator can
>>>>>>> make an informed decision on who to give the task (and warn if there
>>>>>>> is more than one capable)
>>>>>>>
>>>>>>>
>>>>>>>> Darren
>>>>>>>>
>>>>>>>> On Tue, Oct 29, 2013 at 9:42 AM, Hugo Trippaers <hu...@trippaers.nl>
>>>>>> wrote:
>>>>>>>>> Hey guys,
>>>>>>>>>
>>>>>>>>> This is something i had on my wish list for some time. The current way
>>>>>> network gurus are handled is that each guru is asked to design a network and
>>>>>> will decide by itself to either do it or don't. Most gurus have sane checks on
>>>>>> which types of networks it can handle, but we have seen issues there in the
>>>>>> past.
>>>>>>>>>
>>>>>>>>> With these changes the network orchestrator will check the capabilities
>>>>>> of a guru and based on that ask one or more gurus to design the network.
>>>>>> With this the power is where is should new, the network orchestrator.
>>>>>>>>>
>>>>>>>>> This also means that new networking plugins with gurus will have an
>>>>>> easier integration, just list the capabilities. It will also save some database
>>>>>> calls (once i clean out all canHandle functions) as gurus will have to lookup
>>>>>> less to decide if they should to their job.
>>>>>>>>>
>>>>>>>>> What do you guys think?
>>>>>>>>>
>>>>>>>>> Current branch is tested with devcloud in a manual test, so there is more
>>>>>> work to do there. I wanted to get some opinions while performing more
>>>>>> tests.
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>>
>>>>>>>>> Hugo
>>>>>>>
>>>
>

Re: [MERGE] network-guru-orchestration into master

[Hugo Trippaers <hu...@trippaers.nl>]

On 31 okt. 2013, at 03:48, Darren Shepherd <da...@gmail.com> wrote:

> Its important to remember that the definition of a Network in
> CloudStack is quite purposefully very loose.  A network in the
> simplest terms is just a group of VMs.  A guru creates and manages
> that network.  A network is not strictly defined as a union of
> traffictype/isolationtype/networktype/guesttype.  So, for example,
> there need not be a 1-to-1 mapping between Guru and IsolationType.
> You should be able to have multiple gurus per any context.  That's the
> theory and vision.

I think that vision was put down somewhere before we entered the era of software defined networking. At the moment it is simply not true as long as the guru is responsible for creating the overlay networks there is a 1 on 1 relation with the guru and the supported isolation types in several cases.

> 
> Now the reality is that the current selection logic of the guru in the
> orchestrator doesn't fully back up that vision because the selection
> logic too easily allows two Guru's to implement the same network.
> 
> So I think the current behavior of setupNetwork is flawed.  This
> change, I feel, muddies to a certain degree the definition of a
> network in CloudStack and attempts to say a network is strict union of
> traffic/isolation/network/guest-type.  If we merge this change, does
> it hurt anything?  No, its more just conceptual disagreement.  When I
> get to the point that I feel setupNetwork is limiting what I want to
> do, I'll propose a change.  Right now, I just accept that its not
> right in my mind.

I’ve fixed the definition of setupNetwork so the function returns a single network when called. This was already the effective behavior, but now its put down in code as well. I’ll push that change to the branch today.


> 
> So I'd have to say I'm neutral on this change.  I don't care too much
> either way as I think down the line this all needs to change.

Let’s find a moment to sit down at CCCEU and discuss this further. I have some ideas of where i want to go with the networking bit as well and it would be nice if we are more or less on the same page before we start working on it. Saying that you don’t care because you think its going to change it anyway isn’t really the best way to approach this.

> 
> Darren
> 
> On Wed, Oct 30, 2013 at 11:36 AM, Pedro Roque Marques
> <pe...@gmail.com> wrote:
>> 
>> On Oct 30, 2013, at 11:08 AM, Darren Shepherd <da...@gmail.com> wrote:
>> 
>>> I definitely don't want the setup to be done at a zone level.
>> 
>> Why ? That seems to me the most obvious way to do it.
>> There are different networking solutions: e.g. VLANs and overlays such as OpenContrail that assume an L3 switching topology.  For a deployment one would tend to choose a solution associated with the physical network.
>> 
>>> Specifically, I'd like to get rid of the notion of basic and advanced
>>> zones which would mean that you could be mixing gurus in the same
>>> zone.  If anything, I'd say the simplest would be to just specify the
>>> Guru in the network offering.
>> 
>> Basic and advanced zones is rather unhelpful, i agree. But you still can't Mix Guru's unless they can interoperate... The way for them to interop (i.e. minimum common denominator) is to connect at L3 in an router between zones.
>> 
>>> 
>>> I like the work that Hugo has done, but the implications of that
>>> change are 2 things.  First, we only want one guru to match.  If that
>>> is the case, then lets change the NetworkOrchestrator.setupNetwork()
>>> to just return one network.  I have a hard time understanding why you
>>> would want to return two or more.  Even if the guru created two
>>> networks internally, only one should be return to the caller.  There's
>>> not a good way to invoke setupNetwork() and handle cases where more
>>> than one network comes back.
>>> 
>>> Second, we are saying the only way to choose a guru is based on
>>> networkType/trafficType/guestType/isolationmethods.  I think the core
>>> logic should be more flexible than that.  We can first check those 4
>>> items, and then if we find more than one guru that matches, use the
>>> best one (before what I said about adding canHandle to the interface).
>>> But the check shouldn't be only those four criteria.  Hugo's change
>>> does not strictly enforce that, but it is essentially implied.
>>> 
>>> After saying all that, if we just put the Guru in the network
>>> offering.  Would that work for everything?  That would be a really
>>> simple and explicit way to configure this all.
>> 
>> My preference would be for a NetworkManager per zone.
>> 
>>> 
>>> Darren
>>> 
>>> 
>>> On Wed, Oct 30, 2013 at 10:31 AM, Alex Huang <Al...@citrix.com> wrote:
>>>> I agree with Hugo that the current way of ask NetworkGuru to decide itself is not desirable.  It makes absolute sense to change that but I wonder why do it through capabilities.  Why not just ask the system admin to choose when they setup the zone?   When I originally designed NetworkGuru, I figured there's a difference in the L2-L3 technology deployed and the network services (L4-L7) provided on top of it.  So I separated out NetworkGuru and NetworkElement.  However, I didn't think it's likely that there would be two or three different type of L2-L3 technologies deployed within the same zone.  I figured we can leave it to the system admin to just pick which set of NetworkGurus should be deployed in a certain zone and that's good enough.
>>>> 
>>>> I do think there should be more tie in between the NetworkElements and NetworkGurus.  For example, whether a certain NetworkElement can work with the L2-L3 technology deployed.
>>>> 
>>>> --Alex
>>>> 
>>>>> -----Original Message-----
>>>>> From: Darren Shepherd [mailto:darren.s.shepherd@gmail.com]
>>>>> Sent: Wednesday, October 30, 2013 10:11 AM
>>>>> To: dev@cloudstack.apache.org; Alex Huang; Chiradeep Vittal
>>>>> Subject: Re: [MERGE] network-guru-orchestration into master
>>>>> 
>>>>> I don't particular like saying that picking a Guru is based solely on that criteria.
>>>>> It should be much looser than that.  If the problem you are trying to fix is two
>>>>> Gurus implement the same network then we need to set back a bit.  I'd like
>>>>> Alex or Chiradeep to weigh in on this.  Currently setupNetwork returns a list
>>>>> of networks that were created.  I've looked at the code and every single
>>>>> invocation to setupNetwork expects (and mostly hard code) a response of
>>>>> one network.
>>>>> 
>>>>> So for a more proper fix I'd say that the API of setupNetwork should just
>>>>> return Network.  Additionally NetworkGuru should have a
>>>>> canHandle/canDesign method on the interface that returns a priority/score
>>>>> (similar to what was just done in the storage strategies).  Then the
>>>>> orchestrator can find the best match and then use the correct guru.  Now the
>>>>> consolidated logic that you've done should be in a abstract base class that
>>>>> handles the basic cases of traffic type, network type, etc.
>>>>> 
>>>>> Darren
>>>>> 
>>>>> On Wed, Oct 30, 2013 at 6:10 AM, Hugo Trippaers <hu...@trippaers.nl> wrote:
>>>>>> 
>>>>>> On 30 okt. 2013, at 02:09, Darren Shepherd <da...@gmail.com>
>>>>> wrote:
>>>>>> 
>>>>>>> First, I like the idea of consolidating logic.  I could see also
>>>>>>> implementing this as just an abstract base class that handles this
>>>>>>> common logic.  I'm not sure which approach I prefer.  Exactly what is
>>>>>>> the problem that you are trying to solve?  Without more details, I'd
>>>>>>> lean towards implementing this logic in an abstract base class that
>>>>>>> all NetworkGurus can choose to extend.
>>>>>>> 
>>>>>> 
>>>>>> Not as much a problem as a design choice. It is my intention to use
>>>>>> capabilities to select the NetworkGuru instead of asking each network
>>>>>> guru in turn if it wants to design the network. The idea here is that
>>>>>> new network gurus only need to supply a list of capabilities to be
>>>>>> integrated into cloudstack. Like i can handle isolation type X in
>>>>>> advanced networks for traffic type Guest. The network orchestrator can
>>>>>> make an informed decision on who to give the task (and warn if there
>>>>>> is more than one capable)
>>>>>> 
>>>>>> 
>>>>>>> Darren
>>>>>>> 
>>>>>>> On Tue, Oct 29, 2013 at 9:42 AM, Hugo Trippaers <hu...@trippaers.nl>
>>>>> wrote:
>>>>>>>> Hey guys,
>>>>>>>> 
>>>>>>>> This is something i had on my wish list for some time. The current way
>>>>> network gurus are handled is that each guru is asked to design a network and
>>>>> will decide by itself to either do it or don't. Most gurus have sane checks on
>>>>> which types of networks it can handle, but we have seen issues there in the
>>>>> past.
>>>>>>>> 
>>>>>>>> With these changes the network orchestrator will check the capabilities
>>>>> of a guru and based on that ask one or more gurus to design the network.
>>>>> With this the power is where is should new, the network orchestrator.
>>>>>>>> 
>>>>>>>> This also means that new networking plugins with gurus will have an
>>>>> easier integration, just list the capabilities. It will also save some database
>>>>> calls (once i clean out all canHandle functions) as gurus will have to lookup
>>>>> less to decide if they should to their job.
>>>>>>>> 
>>>>>>>> What do you guys think?
>>>>>>>> 
>>>>>>>> Current branch is tested with devcloud in a manual test, so there is more
>>>>> work to do there. I wanted to get some opinions while performing more
>>>>> tests.
>>>>>>>> 
>>>>>>>> Cheers,
>>>>>>>> 
>>>>>>>> Hugo
>>>>>> 
>> 

Re: [MERGE] network-guru-orchestration into master

[Darren Shepherd <da...@gmail.com>]

Its important to remember that the definition of a Network in
CloudStack is quite purposefully very loose.  A network in the
simplest terms is just a group of VMs.  A guru creates and manages
that network.  A network is not strictly defined as a union of
traffictype/isolationtype/networktype/guesttype.  So, for example,
there need not be a 1-to-1 mapping between Guru and IsolationType.
You should be able to have multiple gurus per any context.  That's the
theory and vision.

Now the reality is that the current selection logic of the guru in the
orchestrator doesn't fully back up that vision because the selection
logic too easily allows two Guru's to implement the same network.

So I think the current behavior of setupNetwork is flawed.  This
change, I feel, muddies to a certain degree the definition of a
network in CloudStack and attempts to say a network is strict union of
traffic/isolation/network/guest-type.  If we merge this change, does
it hurt anything?  No, its more just conceptual disagreement.  When I
get to the point that I feel setupNetwork is limiting what I want to
do, I'll propose a change.  Right now, I just accept that its not
right in my mind.

So I'd have to say I'm neutral on this change.  I don't care too much
either way as I think down the line this all needs to change.

Darren

On Wed, Oct 30, 2013 at 11:36 AM, Pedro Roque Marques
<pe...@gmail.com> wrote:
>
> On Oct 30, 2013, at 11:08 AM, Darren Shepherd <da...@gmail.com> wrote:
>
>> I definitely don't want the setup to be done at a zone level.
>
> Why ? That seems to me the most obvious way to do it.
> There are different networking solutions: e.g. VLANs and overlays such as OpenContrail that assume an L3 switching topology.  For a deployment one would tend to choose a solution associated with the physical network.
>
>> Specifically, I'd like to get rid of the notion of basic and advanced
>> zones which would mean that you could be mixing gurus in the same
>> zone.  If anything, I'd say the simplest would be to just specify the
>> Guru in the network offering.
>
> Basic and advanced zones is rather unhelpful, i agree. But you still can't Mix Guru's unless they can interoperate... The way for them to interop (i.e. minimum common denominator) is to connect at L3 in an router between zones.
>
>>
>> I like the work that Hugo has done, but the implications of that
>> change are 2 things.  First, we only want one guru to match.  If that
>> is the case, then lets change the NetworkOrchestrator.setupNetwork()
>> to just return one network.  I have a hard time understanding why you
>> would want to return two or more.  Even if the guru created two
>> networks internally, only one should be return to the caller.  There's
>> not a good way to invoke setupNetwork() and handle cases where more
>> than one network comes back.
>>
>> Second, we are saying the only way to choose a guru is based on
>> networkType/trafficType/guestType/isolationmethods.  I think the core
>> logic should be more flexible than that.  We can first check those 4
>> items, and then if we find more than one guru that matches, use the
>> best one (before what I said about adding canHandle to the interface).
>> But the check shouldn't be only those four criteria.  Hugo's change
>> does not strictly enforce that, but it is essentially implied.
>>
>> After saying all that, if we just put the Guru in the network
>> offering.  Would that work for everything?  That would be a really
>> simple and explicit way to configure this all.
>
> My preference would be for a NetworkManager per zone.
>
>>
>> Darren
>>
>>
>> On Wed, Oct 30, 2013 at 10:31 AM, Alex Huang <Al...@citrix.com> wrote:
>>> I agree with Hugo that the current way of ask NetworkGuru to decide itself is not desirable.  It makes absolute sense to change that but I wonder why do it through capabilities.  Why not just ask the system admin to choose when they setup the zone?   When I originally designed NetworkGuru, I figured there's a difference in the L2-L3 technology deployed and the network services (L4-L7) provided on top of it.  So I separated out NetworkGuru and NetworkElement.  However, I didn't think it's likely that there would be two or three different type of L2-L3 technologies deployed within the same zone.  I figured we can leave it to the system admin to just pick which set of NetworkGurus should be deployed in a certain zone and that's good enough.
>>>
>>> I do think there should be more tie in between the NetworkElements and NetworkGurus.  For example, whether a certain NetworkElement can work with the L2-L3 technology deployed.
>>>
>>> --Alex
>>>
>>>> -----Original Message-----
>>>> From: Darren Shepherd [mailto:darren.s.shepherd@gmail.com]
>>>> Sent: Wednesday, October 30, 2013 10:11 AM
>>>> To: dev@cloudstack.apache.org; Alex Huang; Chiradeep Vittal
>>>> Subject: Re: [MERGE] network-guru-orchestration into master
>>>>
>>>> I don't particular like saying that picking a Guru is based solely on that criteria.
>>>> It should be much looser than that.  If the problem you are trying to fix is two
>>>> Gurus implement the same network then we need to set back a bit.  I'd like
>>>> Alex or Chiradeep to weigh in on this.  Currently setupNetwork returns a list
>>>> of networks that were created.  I've looked at the code and every single
>>>> invocation to setupNetwork expects (and mostly hard code) a response of
>>>> one network.
>>>>
>>>> So for a more proper fix I'd say that the API of setupNetwork should just
>>>> return Network.  Additionally NetworkGuru should have a
>>>> canHandle/canDesign method on the interface that returns a priority/score
>>>> (similar to what was just done in the storage strategies).  Then the
>>>> orchestrator can find the best match and then use the correct guru.  Now the
>>>> consolidated logic that you've done should be in a abstract base class that
>>>> handles the basic cases of traffic type, network type, etc.
>>>>
>>>> Darren
>>>>
>>>> On Wed, Oct 30, 2013 at 6:10 AM, Hugo Trippaers <hu...@trippaers.nl> wrote:
>>>>>
>>>>> On 30 okt. 2013, at 02:09, Darren Shepherd <da...@gmail.com>
>>>> wrote:
>>>>>
>>>>>> First, I like the idea of consolidating logic.  I could see also
>>>>>> implementing this as just an abstract base class that handles this
>>>>>> common logic.  I'm not sure which approach I prefer.  Exactly what is
>>>>>> the problem that you are trying to solve?  Without more details, I'd
>>>>>> lean towards implementing this logic in an abstract base class that
>>>>>> all NetworkGurus can choose to extend.
>>>>>>
>>>>>
>>>>> Not as much a problem as a design choice. It is my intention to use
>>>>> capabilities to select the NetworkGuru instead of asking each network
>>>>> guru in turn if it wants to design the network. The idea here is that
>>>>> new network gurus only need to supply a list of capabilities to be
>>>>> integrated into cloudstack. Like i can handle isolation type X in
>>>>> advanced networks for traffic type Guest. The network orchestrator can
>>>>> make an informed decision on who to give the task (and warn if there
>>>>> is more than one capable)
>>>>>
>>>>>
>>>>>> Darren
>>>>>>
>>>>>> On Tue, Oct 29, 2013 at 9:42 AM, Hugo Trippaers <hu...@trippaers.nl>
>>>> wrote:
>>>>>>> Hey guys,
>>>>>>>
>>>>>>> This is something i had on my wish list for some time. The current way
>>>> network gurus are handled is that each guru is asked to design a network and
>>>> will decide by itself to either do it or don't. Most gurus have sane checks on
>>>> which types of networks it can handle, but we have seen issues there in the
>>>> past.
>>>>>>>
>>>>>>> With these changes the network orchestrator will check the capabilities
>>>> of a guru and based on that ask one or more gurus to design the network.
>>>> With this the power is where is should new, the network orchestrator.
>>>>>>>
>>>>>>> This also means that new networking plugins with gurus will have an
>>>> easier integration, just list the capabilities. It will also save some database
>>>> calls (once i clean out all canHandle functions) as gurus will have to lookup
>>>> less to decide if they should to their job.
>>>>>>>
>>>>>>> What do you guys think?
>>>>>>>
>>>>>>> Current branch is tested with devcloud in a manual test, so there is more
>>>> work to do there. I wanted to get some opinions while performing more
>>>> tests.
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Hugo
>>>>>
>

Re: [MERGE] network-guru-orchestration into master

[Hugo Trippaers <hu...@trippaers.nl>]

On 31 okt. 2013, at 04:34, Marcus Sorensen <sh...@gmail.com> wrote:

> Here's a scenario... Lets say we currently have a zone deployed with Vlan
> isolation. There's nothing stopping us from running a vxlan isolation on
> the same hosts, even on the same physical interface. This is actually done
> in devcloud-kvm where public and mgmt traffic are selected by traffic label
> (which are tagged vlans), and guest traffic is handled by vxlan.
> 
> If one wanted to deprecate Vlan in favor of vxlan without a big zone
> shufflr it would be simple if isolation method were a property of network
> offering. I imagine other isolation offerings could also be deployed in
> tandem, even if it required multiple physical interfaces in the host.

Currently the system we have in place is indeed capable of running different isolation types on a host, it uses the physical network concept in cloudstack to determine which isolation type to use. This is mainly because the physical network also defines which tag or network interface label to use on the hypervisors.

I understand the case that it might be possible to combine different isolation types on physical networks, like the case you describe with vlan and vxlan. However in other cases it should not happen, like when nicira nvp is used which takes exclusive control of a traffic label or bridge on the hypervisor. As far as i understood the contrail implementation it actually takes control of the entire networking of a hypervisor, so its exclusive with any other isolation provider.

Within the database and (most of) the code it is already possible to set multiple isolation types on a physical network. That gives the admin the ability to link a certain physical interface/traffic label or bridge to a set of isolation types. For example cloudbr0 with NONE, VLAN and VXLAN and cloudbr1 with STT. The offering system can be used to indicate which isolation provider to use for which particular virtual network.

Cheers,

Hugo


> On Wed, Oct 30, 2013 at 11:36 AM, Pedro Roque Marques
> <pe...@gmail.com> wrote:
>> 
>> Why ? That seems to me the most obvious way to do it.
>> There are different networking solutions: e.g. VLANs and overlays such as
> OpenContrail that assume an L3 switching topology.  For a deployment one
> would tend to choose a solution associated with the physical network.
> 
> Sure that would be simple.  The problem is that Zone also implies
> other things.  Secondary/Primary storage for example.  Zone puts other
> limitations on things.  So by tying networking to a zone you can get
> into the situation where somebody has a CloudStack installation, and
> they want to add OpenContrail.  If we tie it to the zone, that may
> mean that in order for them to use OpenContrail they may need to get
> another NFS secondary storage, primary storage, compute nodes, etc and
> then copy all the templates from one zone to another, etc, etc.
> 
> So currently you can't have Basic and Advanced networking in the same
> zone.  I think it should be possible.  Imagine I had a smaller
> dev/test cloud.  Today if I want basic and advance networking (because
> those are two distinict work loads), I have to have servers dedicated
> to each type.  So 10 for basic, 10 or advanced.  Now the networking
> under the hood for basic and advance can mix quite easily but just
> because we decided to implement it as it is today, I'm suddenly forced
> to managed two pools of physical resources, when it would have been
> must easier and more cost effective to have one pool.
> 
> I think this is general thing that should change with CloudStack over
> time.  Zone and Pod should not be so closely tied to network concepts.
> CloudStack should be capable of being looser for people who want more
> hybrid work loads.  We too easily say that a service provider is just
> going to choose one or the other so why support mixed configurations.
> But there are more people that could be using CloudStack but are
> turned off by the strict models and concepts that CloudStack enforces.
> I have to say I was one of those people.  CloudStack forced me to
> manage my infrastructure in a certain way that I didn't particularly
> care for.
> 
> Darren

Re: [MERGE] network-guru-orchestration into master

[Marcus Sorensen <sh...@gmail.com>]

Here's a scenario... Lets say we currently have a zone deployed with Vlan
isolation. There's nothing stopping us from running a vxlan isolation on
the same hosts, even on the same physical interface. This is actually done
in devcloud-kvm where public and mgmt traffic are selected by traffic label
(which are tagged vlans), and guest traffic is handled by vxlan.

If one wanted to deprecate Vlan in favor of vxlan without a big zone
shufflr it would be simple if isolation method were a property of network
offering. I imagine other isolation offerings could also be deployed in
tandem, even if it required multiple physical interfaces in the host.
On Wed, Oct 30, 2013 at 11:36 AM, Pedro Roque Marques
<pe...@gmail.com> wrote:
>
> Why ? That seems to me the most obvious way to do it.
> There are different networking solutions: e.g. VLANs and overlays such as
OpenContrail that assume an L3 switching topology.  For a deployment one
would tend to choose a solution associated with the physical network.

Sure that would be simple.  The problem is that Zone also implies
other things.  Secondary/Primary storage for example.  Zone puts other
limitations on things.  So by tying networking to a zone you can get
into the situation where somebody has a CloudStack installation, and
they want to add OpenContrail.  If we tie it to the zone, that may
mean that in order for them to use OpenContrail they may need to get
another NFS secondary storage, primary storage, compute nodes, etc and
then copy all the templates from one zone to another, etc, etc.

So currently you can't have Basic and Advanced networking in the same
zone.  I think it should be possible.  Imagine I had a smaller
dev/test cloud.  Today if I want basic and advance networking (because
those are two distinict work loads), I have to have servers dedicated
to each type.  So 10 for basic, 10 or advanced.  Now the networking
under the hood for basic and advance can mix quite easily but just
because we decided to implement it as it is today, I'm suddenly forced
to managed two pools of physical resources, when it would have been
must easier and more cost effective to have one pool.

I think this is general thing that should change with CloudStack over
time.  Zone and Pod should not be so closely tied to network concepts.
 CloudStack should be capable of being looser for people who want more
hybrid work loads.  We too easily say that a service provider is just
going to choose one or the other so why support mixed configurations.
But there are more people that could be using CloudStack but are
turned off by the strict models and concepts that CloudStack enforces.
 I have to say I was one of those people.  CloudStack forced me to
manage my infrastructure in a certain way that I didn't particularly
care for.

Darren

Re: [MERGE] network-guru-orchestration into master

[Darren Shepherd <da...@gmail.com>]

On Wed, Oct 30, 2013 at 11:36 AM, Pedro Roque Marques
<pe...@gmail.com> wrote:
>
> Why ? That seems to me the most obvious way to do it.
> There are different networking solutions: e.g. VLANs and overlays such as OpenContrail that assume an L3 switching topology.  For a deployment one would tend to choose a solution associated with the physical network.

Sure that would be simple.  The problem is that Zone also implies
other things.  Secondary/Primary storage for example.  Zone puts other
limitations on things.  So by tying networking to a zone you can get
into the situation where somebody has a CloudStack installation, and
they want to add OpenContrail.  If we tie it to the zone, that may
mean that in order for them to use OpenContrail they may need to get
another NFS secondary storage, primary storage, compute nodes, etc and
then copy all the templates from one zone to another, etc, etc.

So currently you can't have Basic and Advanced networking in the same
zone.  I think it should be possible.  Imagine I had a smaller
dev/test cloud.  Today if I want basic and advance networking (because
those are two distinict work loads), I have to have servers dedicated
to each type.  So 10 for basic, 10 or advanced.  Now the networking
under the hood for basic and advance can mix quite easily but just
because we decided to implement it as it is today, I'm suddenly forced
to managed two pools of physical resources, when it would have been
must easier and more cost effective to have one pool.

I think this is general thing that should change with CloudStack over
time.  Zone and Pod should not be so closely tied to network concepts.
 CloudStack should be capable of being looser for people who want more
hybrid work loads.  We too easily say that a service provider is just
going to choose one or the other so why support mixed configurations.
But there are more people that could be using CloudStack but are
turned off by the strict models and concepts that CloudStack enforces.
 I have to say I was one of those people.  CloudStack forced me to
manage my infrastructure in a certain way that I didn't particularly
care for.

Darren

Re: [MERGE] network-guru-orchestration into master

[Pedro Roque Marques <pe...@gmail.com>]

On Oct 30, 2013, at 11:08 AM, Darren Shepherd <da...@gmail.com> wrote:

> I definitely don't want the setup to be done at a zone level.

Why ? That seems to me the most obvious way to do it.
There are different networking solutions: e.g. VLANs and overlays such as OpenContrail that assume an L3 switching topology.  For a deployment one would tend to choose a solution associated with the physical network.

> Specifically, I'd like to get rid of the notion of basic and advanced
> zones which would mean that you could be mixing gurus in the same
> zone.  If anything, I'd say the simplest would be to just specify the
> Guru in the network offering.

Basic and advanced zones is rather unhelpful, i agree. But you still can't Mix Guru's unless they can interoperate... The way for them to interop (i.e. minimum common denominator) is to connect at L3 in an router between zones.

> 
> I like the work that Hugo has done, but the implications of that
> change are 2 things.  First, we only want one guru to match.  If that
> is the case, then lets change the NetworkOrchestrator.setupNetwork()
> to just return one network.  I have a hard time understanding why you
> would want to return two or more.  Even if the guru created two
> networks internally, only one should be return to the caller.  There's
> not a good way to invoke setupNetwork() and handle cases where more
> than one network comes back.
> 
> Second, we are saying the only way to choose a guru is based on
> networkType/trafficType/guestType/isolationmethods.  I think the core
> logic should be more flexible than that.  We can first check those 4
> items, and then if we find more than one guru that matches, use the
> best one (before what I said about adding canHandle to the interface).
> But the check shouldn't be only those four criteria.  Hugo's change
> does not strictly enforce that, but it is essentially implied.
> 
> After saying all that, if we just put the Guru in the network
> offering.  Would that work for everything?  That would be a really
> simple and explicit way to configure this all.

My preference would be for a NetworkManager per zone.

> 
> Darren
> 
> 
> On Wed, Oct 30, 2013 at 10:31 AM, Alex Huang <Al...@citrix.com> wrote:
>> I agree with Hugo that the current way of ask NetworkGuru to decide itself is not desirable.  It makes absolute sense to change that but I wonder why do it through capabilities.  Why not just ask the system admin to choose when they setup the zone?   When I originally designed NetworkGuru, I figured there's a difference in the L2-L3 technology deployed and the network services (L4-L7) provided on top of it.  So I separated out NetworkGuru and NetworkElement.  However, I didn't think it's likely that there would be two or three different type of L2-L3 technologies deployed within the same zone.  I figured we can leave it to the system admin to just pick which set of NetworkGurus should be deployed in a certain zone and that's good enough.
>> 
>> I do think there should be more tie in between the NetworkElements and NetworkGurus.  For example, whether a certain NetworkElement can work with the L2-L3 technology deployed.
>> 
>> --Alex
>> 
>>> -----Original Message-----
>>> From: Darren Shepherd [mailto:darren.s.shepherd@gmail.com]
>>> Sent: Wednesday, October 30, 2013 10:11 AM
>>> To: dev@cloudstack.apache.org; Alex Huang; Chiradeep Vittal
>>> Subject: Re: [MERGE] network-guru-orchestration into master
>>> 
>>> I don't particular like saying that picking a Guru is based solely on that criteria.
>>> It should be much looser than that.  If the problem you are trying to fix is two
>>> Gurus implement the same network then we need to set back a bit.  I'd like
>>> Alex or Chiradeep to weigh in on this.  Currently setupNetwork returns a list
>>> of networks that were created.  I've looked at the code and every single
>>> invocation to setupNetwork expects (and mostly hard code) a response of
>>> one network.
>>> 
>>> So for a more proper fix I'd say that the API of setupNetwork should just
>>> return Network.  Additionally NetworkGuru should have a
>>> canHandle/canDesign method on the interface that returns a priority/score
>>> (similar to what was just done in the storage strategies).  Then the
>>> orchestrator can find the best match and then use the correct guru.  Now the
>>> consolidated logic that you've done should be in a abstract base class that
>>> handles the basic cases of traffic type, network type, etc.
>>> 
>>> Darren
>>> 
>>> On Wed, Oct 30, 2013 at 6:10 AM, Hugo Trippaers <hu...@trippaers.nl> wrote:
>>>> 
>>>> On 30 okt. 2013, at 02:09, Darren Shepherd <da...@gmail.com>
>>> wrote:
>>>> 
>>>>> First, I like the idea of consolidating logic.  I could see also
>>>>> implementing this as just an abstract base class that handles this
>>>>> common logic.  I'm not sure which approach I prefer.  Exactly what is
>>>>> the problem that you are trying to solve?  Without more details, I'd
>>>>> lean towards implementing this logic in an abstract base class that
>>>>> all NetworkGurus can choose to extend.
>>>>> 
>>>> 
>>>> Not as much a problem as a design choice. It is my intention to use
>>>> capabilities to select the NetworkGuru instead of asking each network
>>>> guru in turn if it wants to design the network. The idea here is that
>>>> new network gurus only need to supply a list of capabilities to be
>>>> integrated into cloudstack. Like i can handle isolation type X in
>>>> advanced networks for traffic type Guest. The network orchestrator can
>>>> make an informed decision on who to give the task (and warn if there
>>>> is more than one capable)
>>>> 
>>>> 
>>>>> Darren
>>>>> 
>>>>> On Tue, Oct 29, 2013 at 9:42 AM, Hugo Trippaers <hu...@trippaers.nl>
>>> wrote:
>>>>>> Hey guys,
>>>>>> 
>>>>>> This is something i had on my wish list for some time. The current way
>>> network gurus are handled is that each guru is asked to design a network and
>>> will decide by itself to either do it or don't. Most gurus have sane checks on
>>> which types of networks it can handle, but we have seen issues there in the
>>> past.
>>>>>> 
>>>>>> With these changes the network orchestrator will check the capabilities
>>> of a guru and based on that ask one or more gurus to design the network.
>>> With this the power is where is should new, the network orchestrator.
>>>>>> 
>>>>>> This also means that new networking plugins with gurus will have an
>>> easier integration, just list the capabilities. It will also save some database
>>> calls (once i clean out all canHandle functions) as gurus will have to lookup
>>> less to decide if they should to their job.
>>>>>> 
>>>>>> What do you guys think?
>>>>>> 
>>>>>> Current branch is tested with devcloud in a manual test, so there is more
>>> work to do there. I wanted to get some opinions while performing more
>>> tests.
>>>>>> 
>>>>>> Cheers,
>>>>>> 
>>>>>> Hugo
>>>> 

Re: [MERGE] network-guru-orchestration into master

[Hugo Trippaers <hu...@trippaers.nl>]

On 30 okt. 2013, at 19:08, Darren Shepherd <da...@gmail.com> wrote:

> I definitely don't want the setup to be done at a zone level.
Pretty strong words, i’m considering this a veto on any further discussion around this topic?

> Specifically, I'd like to get rid of the notion of basic and advanced
> zones which would mean that you could be mixing gurus in the same
> zone.  If anything, I'd say the simplest would be to just specify the
> Guru in the network offering.
> 
> I like the work that Hugo has done, but the implications of that
> change are 2 things.  First, we only want one guru to match.  If that
> is the case, then lets change the NetworkOrchestrator.setupNetwork()
> to just return one network.  I have a hard time understanding why you
> would want to return two or more.

Lets say that is a historical left over, there is no reason to return more than one network. The current method signature is still there, because nobody changed it yet. Feel free to submit a patch.

>  Even if the guru created two
> networks internally, only one should be return to the caller.  There's
> not a good way to invoke setupNetwork() and handle cases where more
> than one network comes back.

> 
> Second, we are saying the only way to choose a guru is based on
> networkType/trafficType/guestType/isolationmethods.  I think the core
> logic should be more flexible than that.  We can first check those 4
> items, and then if we find more than one guru that matches, use the
> best one (before what I said about adding canHandle to the interface).
> But the check shouldn't be only those four criteria.  Hugo's change
> does not strictly enforce that, but it is essentially implied.
Thats how it is currenty implemented, with the exception that we don’t know which is the best one. So we loop through the remaining gurus and the first one that responds with a network wins. It would be very hard to give a score to a guru.


> 
> After saying all that, if we just put the Guru in the network
> offering.  Would that work for everything?  That would be a really
> simple and explicit way to configure this all.

We need it to link to the physical network. The physical network is configured for a particular technology, usually out side cloudstack. Like configuring your hypervisors for Midonet or VMware NSX. You should not be able to mix different gurus on a physical network. The gurus are there to deal with the low-level network creation stuff. If anything, the gurus should be split into two different things. The first that deals with the isolation method (L2 networking) set on the physical network and the second that deals with the L3 stuff like how to assign IPs and such.

> 
> Darren

Hugo
> 
> 
> On Wed, Oct 30, 2013 at 10:31 AM, Alex Huang <Al...@citrix.com> wrote:
>> I agree with Hugo that the current way of ask NetworkGuru to decide itself is not desirable.  It makes absolute sense to change that but I wonder why do it through capabilities.  Why not just ask the system admin to choose when they setup the zone?   When I originally designed NetworkGuru, I figured there's a difference in the L2-L3 technology deployed and the network services (L4-L7) provided on top of it.  So I separated out NetworkGuru and NetworkElement.  However, I didn't think it's likely that there would be two or three different type of L2-L3 technologies deployed within the same zone.  I figured we can leave it to the system admin to just pick which set of NetworkGurus should be deployed in a certain zone and that's good enough.
>> 
>> I do think there should be more tie in between the NetworkElements and NetworkGurus.  For example, whether a certain NetworkElement can work with the L2-L3 technology deployed.
>> 
>> --Alex
>> 
>>> -----Original Message-----
>>> From: Darren Shepherd [mailto:darren.s.shepherd@gmail.com]
>>> Sent: Wednesday, October 30, 2013 10:11 AM
>>> To: dev@cloudstack.apache.org; Alex Huang; Chiradeep Vittal
>>> Subject: Re: [MERGE] network-guru-orchestration into master
>>> 
>>> I don't particular like saying that picking a Guru is based solely on that criteria.
>>> It should be much looser than that.  If the problem you are trying to fix is two
>>> Gurus implement the same network then we need to set back a bit.  I'd like
>>> Alex or Chiradeep to weigh in on this.  Currently setupNetwork returns a list
>>> of networks that were created.  I've looked at the code and every single
>>> invocation to setupNetwork expects (and mostly hard code) a response of
>>> one network.
>>> 
>>> So for a more proper fix I'd say that the API of setupNetwork should just
>>> return Network.  Additionally NetworkGuru should have a
>>> canHandle/canDesign method on the interface that returns a priority/score
>>> (similar to what was just done in the storage strategies).  Then the
>>> orchestrator can find the best match and then use the correct guru.  Now the
>>> consolidated logic that you've done should be in a abstract base class that
>>> handles the basic cases of traffic type, network type, etc.
>>> 
>>> Darren
>>> 
>>> On Wed, Oct 30, 2013 at 6:10 AM, Hugo Trippaers <hu...@trippaers.nl> wrote:
>>>> 
>>>> On 30 okt. 2013, at 02:09, Darren Shepherd <da...@gmail.com>
>>> wrote:
>>>> 
>>>>> First, I like the idea of consolidating logic.  I could see also
>>>>> implementing this as just an abstract base class that handles this
>>>>> common logic.  I'm not sure which approach I prefer.  Exactly what is
>>>>> the problem that you are trying to solve?  Without more details, I'd
>>>>> lean towards implementing this logic in an abstract base class that
>>>>> all NetworkGurus can choose to extend.
>>>>> 
>>>> 
>>>> Not as much a problem as a design choice. It is my intention to use
>>>> capabilities to select the NetworkGuru instead of asking each network
>>>> guru in turn if it wants to design the network. The idea here is that
>>>> new network gurus only need to supply a list of capabilities to be
>>>> integrated into cloudstack. Like i can handle isolation type X in
>>>> advanced networks for traffic type Guest. The network orchestrator can
>>>> make an informed decision on who to give the task (and warn if there
>>>> is more than one capable)
>>>> 
>>>> 
>>>>> Darren
>>>>> 
>>>>> On Tue, Oct 29, 2013 at 9:42 AM, Hugo Trippaers <hu...@trippaers.nl>
>>> wrote:
>>>>>> Hey guys,
>>>>>> 
>>>>>> This is something i had on my wish list for some time. The current way
>>> network gurus are handled is that each guru is asked to design a network and
>>> will decide by itself to either do it or don't. Most gurus have sane checks on
>>> which types of networks it can handle, but we have seen issues there in the
>>> past.
>>>>>> 
>>>>>> With these changes the network orchestrator will check the capabilities
>>> of a guru and based on that ask one or more gurus to design the network.
>>> With this the power is where is should new, the network orchestrator.
>>>>>> 
>>>>>> This also means that new networking plugins with gurus will have an
>>> easier integration, just list the capabilities. It will also save some database
>>> calls (once i clean out all canHandle functions) as gurus will have to lookup
>>> less to decide if they should to their job.
>>>>>> 
>>>>>> What do you guys think?
>>>>>> 
>>>>>> Current branch is tested with devcloud in a manual test, so there is more
>>> work to do there. I wanted to get some opinions while performing more
>>> tests.
>>>>>> 
>>>>>> Cheers,
>>>>>> 
>>>>>> Hugo
>>>> 

Re: [MERGE] network-guru-orchestration into master

[Darren Shepherd <da...@gmail.com>]

I definitely don't want the setup to be done at a zone level.
Specifically, I'd like to get rid of the notion of basic and advanced
zones which would mean that you could be mixing gurus in the same
zone.  If anything, I'd say the simplest would be to just specify the
Guru in the network offering.

I like the work that Hugo has done, but the implications of that
change are 2 things.  First, we only want one guru to match.  If that
is the case, then lets change the NetworkOrchestrator.setupNetwork()
to just return one network.  I have a hard time understanding why you
would want to return two or more.  Even if the guru created two
networks internally, only one should be return to the caller.  There's
not a good way to invoke setupNetwork() and handle cases where more
than one network comes back.

Second, we are saying the only way to choose a guru is based on
networkType/trafficType/guestType/isolationmethods.  I think the core
logic should be more flexible than that.  We can first check those 4
items, and then if we find more than one guru that matches, use the
best one (before what I said about adding canHandle to the interface).
 But the check shouldn't be only those four criteria.  Hugo's change
does not strictly enforce that, but it is essentially implied.

After saying all that, if we just put the Guru in the network
offering.  Would that work for everything?  That would be a really
simple and explicit way to configure this all.

Darren


On Wed, Oct 30, 2013 at 10:31 AM, Alex Huang <Al...@citrix.com> wrote:
> I agree with Hugo that the current way of ask NetworkGuru to decide itself is not desirable.  It makes absolute sense to change that but I wonder why do it through capabilities.  Why not just ask the system admin to choose when they setup the zone?   When I originally designed NetworkGuru, I figured there's a difference in the L2-L3 technology deployed and the network services (L4-L7) provided on top of it.  So I separated out NetworkGuru and NetworkElement.  However, I didn't think it's likely that there would be two or three different type of L2-L3 technologies deployed within the same zone.  I figured we can leave it to the system admin to just pick which set of NetworkGurus should be deployed in a certain zone and that's good enough.
>
> I do think there should be more tie in between the NetworkElements and NetworkGurus.  For example, whether a certain NetworkElement can work with the L2-L3 technology deployed.
>
> --Alex
>
>> -----Original Message-----
>> From: Darren Shepherd [mailto:darren.s.shepherd@gmail.com]
>> Sent: Wednesday, October 30, 2013 10:11 AM
>> To: dev@cloudstack.apache.org; Alex Huang; Chiradeep Vittal
>> Subject: Re: [MERGE] network-guru-orchestration into master
>>
>> I don't particular like saying that picking a Guru is based solely on that criteria.
>> It should be much looser than that.  If the problem you are trying to fix is two
>> Gurus implement the same network then we need to set back a bit.  I'd like
>> Alex or Chiradeep to weigh in on this.  Currently setupNetwork returns a list
>> of networks that were created.  I've looked at the code and every single
>> invocation to setupNetwork expects (and mostly hard code) a response of
>> one network.
>>
>> So for a more proper fix I'd say that the API of setupNetwork should just
>> return Network.  Additionally NetworkGuru should have a
>> canHandle/canDesign method on the interface that returns a priority/score
>> (similar to what was just done in the storage strategies).  Then the
>> orchestrator can find the best match and then use the correct guru.  Now the
>> consolidated logic that you've done should be in a abstract base class that
>> handles the basic cases of traffic type, network type, etc.
>>
>> Darren
>>
>> On Wed, Oct 30, 2013 at 6:10 AM, Hugo Trippaers <hu...@trippaers.nl> wrote:
>> >
>> > On 30 okt. 2013, at 02:09, Darren Shepherd <da...@gmail.com>
>> wrote:
>> >
>> >> First, I like the idea of consolidating logic.  I could see also
>> >> implementing this as just an abstract base class that handles this
>> >> common logic.  I'm not sure which approach I prefer.  Exactly what is
>> >> the problem that you are trying to solve?  Without more details, I'd
>> >> lean towards implementing this logic in an abstract base class that
>> >> all NetworkGurus can choose to extend.
>> >>
>> >
>> > Not as much a problem as a design choice. It is my intention to use
>> > capabilities to select the NetworkGuru instead of asking each network
>> > guru in turn if it wants to design the network. The idea here is that
>> > new network gurus only need to supply a list of capabilities to be
>> > integrated into cloudstack. Like i can handle isolation type X in
>> > advanced networks for traffic type Guest. The network orchestrator can
>> > make an informed decision on who to give the task (and warn if there
>> > is more than one capable)
>> >
>> >
>> >> Darren
>> >>
>> >> On Tue, Oct 29, 2013 at 9:42 AM, Hugo Trippaers <hu...@trippaers.nl>
>> wrote:
>> >>> Hey guys,
>> >>>
>> >>> This is something i had on my wish list for some time. The current way
>> network gurus are handled is that each guru is asked to design a network and
>> will decide by itself to either do it or don't. Most gurus have sane checks on
>> which types of networks it can handle, but we have seen issues there in the
>> past.
>> >>>
>> >>> With these changes the network orchestrator will check the capabilities
>> of a guru and based on that ask one or more gurus to design the network.
>> With this the power is where is should new, the network orchestrator.
>> >>>
>> >>> This also means that new networking plugins with gurus will have an
>> easier integration, just list the capabilities. It will also save some database
>> calls (once i clean out all canHandle functions) as gurus will have to lookup
>> less to decide if they should to their job.
>> >>>
>> >>> What do you guys think?
>> >>>
>> >>> Current branch is tested with devcloud in a manual test, so there is more
>> work to do there. I wanted to get some opinions while performing more
>> tests.
>> >>>
>> >>> Cheers,
>> >>>
>> >>> Hugo
>> >

RE: [MERGE] network-guru-orchestration into master

[Alex Huang <Al...@citrix.com>]

I agree with Hugo that the current way of ask NetworkGuru to decide itself is not desirable.  It makes absolute sense to change that but I wonder why do it through capabilities.  Why not just ask the system admin to choose when they setup the zone?   When I originally designed NetworkGuru, I figured there's a difference in the L2-L3 technology deployed and the network services (L4-L7) provided on top of it.  So I separated out NetworkGuru and NetworkElement.  However, I didn't think it's likely that there would be two or three different type of L2-L3 technologies deployed within the same zone.  I figured we can leave it to the system admin to just pick which set of NetworkGurus should be deployed in a certain zone and that's good enough. 

I do think there should be more tie in between the NetworkElements and NetworkGurus.  For example, whether a certain NetworkElement can work with the L2-L3 technology deployed.

--Alex

> -----Original Message-----
> From: Darren Shepherd [mailto:darren.s.shepherd@gmail.com]
> Sent: Wednesday, October 30, 2013 10:11 AM
> To: dev@cloudstack.apache.org; Alex Huang; Chiradeep Vittal
> Subject: Re: [MERGE] network-guru-orchestration into master
> 
> I don't particular like saying that picking a Guru is based solely on that criteria.
> It should be much looser than that.  If the problem you are trying to fix is two
> Gurus implement the same network then we need to set back a bit.  I'd like
> Alex or Chiradeep to weigh in on this.  Currently setupNetwork returns a list
> of networks that were created.  I've looked at the code and every single
> invocation to setupNetwork expects (and mostly hard code) a response of
> one network.
> 
> So for a more proper fix I'd say that the API of setupNetwork should just
> return Network.  Additionally NetworkGuru should have a
> canHandle/canDesign method on the interface that returns a priority/score
> (similar to what was just done in the storage strategies).  Then the
> orchestrator can find the best match and then use the correct guru.  Now the
> consolidated logic that you've done should be in a abstract base class that
> handles the basic cases of traffic type, network type, etc.
> 
> Darren
> 
> On Wed, Oct 30, 2013 at 6:10 AM, Hugo Trippaers <hu...@trippaers.nl> wrote:
> >
> > On 30 okt. 2013, at 02:09, Darren Shepherd <da...@gmail.com>
> wrote:
> >
> >> First, I like the idea of consolidating logic.  I could see also
> >> implementing this as just an abstract base class that handles this
> >> common logic.  I'm not sure which approach I prefer.  Exactly what is
> >> the problem that you are trying to solve?  Without more details, I'd
> >> lean towards implementing this logic in an abstract base class that
> >> all NetworkGurus can choose to extend.
> >>
> >
> > Not as much a problem as a design choice. It is my intention to use
> > capabilities to select the NetworkGuru instead of asking each network
> > guru in turn if it wants to design the network. The idea here is that
> > new network gurus only need to supply a list of capabilities to be
> > integrated into cloudstack. Like i can handle isolation type X in
> > advanced networks for traffic type Guest. The network orchestrator can
> > make an informed decision on who to give the task (and warn if there
> > is more than one capable)
> >
> >
> >> Darren
> >>
> >> On Tue, Oct 29, 2013 at 9:42 AM, Hugo Trippaers <hu...@trippaers.nl>
> wrote:
> >>> Hey guys,
> >>>
> >>> This is something i had on my wish list for some time. The current way
> network gurus are handled is that each guru is asked to design a network and
> will decide by itself to either do it or don't. Most gurus have sane checks on
> which types of networks it can handle, but we have seen issues there in the
> past.
> >>>
> >>> With these changes the network orchestrator will check the capabilities
> of a guru and based on that ask one or more gurus to design the network.
> With this the power is where is should new, the network orchestrator.
> >>>
> >>> This also means that new networking plugins with gurus will have an
> easier integration, just list the capabilities. It will also save some database
> calls (once i clean out all canHandle functions) as gurus will have to lookup
> less to decide if they should to their job.
> >>>
> >>> What do you guys think?
> >>>
> >>> Current branch is tested with devcloud in a manual test, so there is more
> work to do there. I wanted to get some opinions while performing more
> tests.
> >>>
> >>> Cheers,
> >>>
> >>> Hugo
> >

Re: [MERGE] network-guru-orchestration into master

[Darren Shepherd <da...@gmail.com>]

I don't particular like saying that picking a Guru is based solely on
that criteria.  It should be much looser than that.  If the problem
you are trying to fix is two Gurus implement the same network then we
need to set back a bit.  I'd like Alex or Chiradeep to weigh in on
this.  Currently setupNetwork returns a list of networks that were
created.  I've looked at the code and every single invocation to
setupNetwork expects (and mostly hard code) a response of one network.

So for a more proper fix I'd say that the API of setupNetwork should
just return Network.  Additionally NetworkGuru should have a
canHandle/canDesign method on the interface that returns a
priority/score (similar to what was just done in the storage
strategies).  Then the orchestrator can find the best match and then
use the correct guru.  Now the consolidated logic that you've done
should be in a abstract base class that handles the basic cases of
traffic type, network type, etc.

Darren

On Wed, Oct 30, 2013 at 6:10 AM, Hugo Trippaers <hu...@trippaers.nl> wrote:
>
> On 30 okt. 2013, at 02:09, Darren Shepherd <da...@gmail.com> wrote:
>
>> First, I like the idea of consolidating logic.  I could see also
>> implementing this as just an abstract base class that handles this
>> common logic.  I'm not sure which approach I prefer.  Exactly what is
>> the problem that you are trying to solve?  Without more details, I'd
>> lean towards implementing this logic in an abstract base class that
>> all NetworkGurus can choose to extend.
>>
>
> Not as much a problem as a design choice. It is my intention to use capabilities to select the NetworkGuru instead of asking each network guru in turn if it wants to design the network. The idea here is that new network gurus only need to supply a list of capabilities to be integrated into cloudstack. Like i can handle isolation type X in advanced networks for traffic type Guest. The network orchestrator can make an informed decision on who to give the task (and warn if there is more than one capable)
>
>
>> Darren
>>
>> On Tue, Oct 29, 2013 at 9:42 AM, Hugo Trippaers <hu...@trippaers.nl> wrote:
>>> Hey guys,
>>>
>>> This is something i had on my wish list for some time. The current way network gurus are handled is that each guru is asked to design a network and will decide by itself to either do it or don’t. Most gurus have sane checks on which types of networks it can handle, but we have seen issues there in the past.
>>>
>>> With these changes the network orchestrator will check the capabilities of a guru and based on that ask one or more gurus to design the network. With this the power is where is should new, the network orchestrator.
>>>
>>> This also means that new networking plugins with gurus will have an easier integration, just list the capabilities. It will also save some database calls (once i clean out all canHandle functions) as gurus will have to lookup less to decide if they should to their job.
>>>
>>> What do you guys think?
>>>
>>> Current branch is tested with devcloud in a manual test, so there is more work to do there. I wanted to get some opinions while performing more tests.
>>>
>>> Cheers,
>>>
>>> Hugo
>

Re: [MERGE] network-guru-orchestration into master

[Hugo Trippaers <hu...@trippaers.nl>]

On 30 okt. 2013, at 02:09, Darren Shepherd <da...@gmail.com> wrote:

> First, I like the idea of consolidating logic.  I could see also
> implementing this as just an abstract base class that handles this
> common logic.  I'm not sure which approach I prefer.  Exactly what is
> the problem that you are trying to solve?  Without more details, I'd
> lean towards implementing this logic in an abstract base class that
> all NetworkGurus can choose to extend.
> 

Not as much a problem as a design choice. It is my intention to use capabilities to select the NetworkGuru instead of asking each network guru in turn if it wants to design the network. The idea here is that new network gurus only need to supply a list of capabilities to be integrated into cloudstack. Like i can handle isolation type X in advanced networks for traffic type Guest. The network orchestrator can make an informed decision on who to give the task (and warn if there is more than one capable)


> Darren
> 
> On Tue, Oct 29, 2013 at 9:42 AM, Hugo Trippaers <hu...@trippaers.nl> wrote:
>> Hey guys,
>> 
>> This is something i had on my wish list for some time. The current way network gurus are handled is that each guru is asked to design a network and will decide by itself to either do it or don’t. Most gurus have sane checks on which types of networks it can handle, but we have seen issues there in the past.
>> 
>> With these changes the network orchestrator will check the capabilities of a guru and based on that ask one or more gurus to design the network. With this the power is where is should new, the network orchestrator.
>> 
>> This also means that new networking plugins with gurus will have an easier integration, just list the capabilities. It will also save some database calls (once i clean out all canHandle functions) as gurus will have to lookup less to decide if they should to their job.
>> 
>> What do you guys think?
>> 
>> Current branch is tested with devcloud in a manual test, so there is more work to do there. I wanted to get some opinions while performing more tests.
>> 
>> Cheers,
>> 
>> Hugo

Re: [MERGE] network-guru-orchestration into master

[Darren Shepherd <da...@gmail.com>]

First, I like the idea of consolidating logic.  I could see also
implementing this as just an abstract base class that handles this
common logic.  I'm not sure which approach I prefer.  Exactly what is
the problem that you are trying to solve?  Without more details, I'd
lean towards implementing this logic in an abstract base class that
all NetworkGurus can choose to extend.

Darren

On Tue, Oct 29, 2013 at 9:42 AM, Hugo Trippaers <hu...@trippaers.nl> wrote:
> Hey guys,
>
> This is something i had on my wish list for some time. The current way network gurus are handled is that each guru is asked to design a network and will decide by itself to either do it or don’t. Most gurus have sane checks on which types of networks it can handle, but we have seen issues there in the past.
>
> With these changes the network orchestrator will check the capabilities of a guru and based on that ask one or more gurus to design the network. With this the power is where is should new, the network orchestrator.
>
> This also means that new networking plugins with gurus will have an easier integration, just list the capabilities. It will also save some database calls (once i clean out all canHandle functions) as gurus will have to lookup less to decide if they should to their job.
>
> What do you guys think?
>
> Current branch is tested with devcloud in a manual test, so there is more work to do there. I wanted to get some opinions while performing more tests.
>
> Cheers,
>
> Hugo

Re: [MERGE] network-guru-orchestration into master

[Murali Reddy <Mu...@citrix.com>]

On 29/10/13 10:12 PM, "Hugo Trippaers" <hu...@trippaers.nl> wrote:

>Hey guys,
>
>This is something i had on my wish list for some time. The current way
>network gurus are handled is that each guru is asked to design a network
>and will decide by itself to either do it or don¹t. Most gurus have sane
>checks on which types of networks it can handle, but we have seen issues
>there in the past.
>
>With these changes the network orchestrator will check the capabilities
>of a guru and based on that ask one or more gurus to design the network.
>With this the power is where is should new, the network orchestrator.
>
>This also means that new networking plugins with gurus will have an
>easier integration, just list the capabilities. It will also save some
>database calls (once i clean out all canHandle functions) as gurus will
>have to lookup less to decide if they should to their job.
>
>What do you guys think?

Nice. Looks like getting the supported guru's by traffic type, network
type, guest type, isolation method is already covering all the guru's with
out ambiguity.  +1 for addressing the problem and merge.


>
>Current branch is tested with devcloud in a manual test, so there is more
>work to do there. I wanted to get some opinions while performing more
>tests.
>
>Cheers,
>
>Hugo