You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stratos.apache.org by "Martin Eppel (meppel)" <me...@cisco.com> on 2015/07/30 03:51:02 UTC
DISCUSS: co-location of VMs in stratos (affinity / anti-affinity)
Hi,
As part of our original grouping design we proposed a feature to either co-locate or not co-locate individual VMs.
Quoting from the original grouping specification:
Quote begin:
Co-Location
VMs within a group will be spun up based on the model of subscriptions, deployment policies and partitions. The model needs to be extended so that:
* It must be possible to require VMs to be co-located on a single host.
* It must be possible to require VMs to be located in separate zones.
Quote end.
As we have some immediate needs we started looking into it and would like to get the input from the community for the requirement / implementation (with the idea to contribute it back to the master repository eventually).
The question would be what is the current support for it (stratos 4.1) and what are the suggestions to enhance stratos to support this enhancement ?
Any other ideas or considerations on how to support the enhancement ?
Thanks
Martin
Re: DISCUSS: co-location of VMs in stratos (affinity / anti-affinity)
Posted by Reka Thirunavukkarasu <re...@wso2.com>.
Hi Martin,
On Fri, Jul 31, 2015 at 4:37 AM, Martin Eppel (meppel) <me...@cisco.com>
wrote:
> Hi Reka,
>
>
>
> Thanks for your reply. The affinity support you mentioned below will IMHO
> work only for a small number of cartirdges / instances and only if the
> cartridges don’t scale.
>
I doubt whether scaling would be a limitation here. If you provide enough
resources in the deployment policy, then scaling can also be achieved by
this. We will have to write a deployment policy in a way that it can pick a
host for every new group instance creation. In that case, one-after-another
algorithm can be used to achieve this. In that case you can define how many
maximum of group instances reside in a particular host. Once that host is
exhausted, stratos will create new group instance in the next host. WDYT?
Is there any other limitation existing when going with group level
deployment policy to achieve this?
Instead it might be more useful to utilize affinity support of the
> underlying IAAS.
>
> For example, openstack supports affinity which, at least at the IAAS level
> meets the requirement as well. It lets you define a flag with the values
> affinity / anti-affinity which behaves as follows:
>
>
>
> If affinity is set, the scheduler will place all VMs (at boot time) on the
> same compute host
>
> If anti-affinity is set, the scheduler will place each (out of the same
> batch of VMs) booted VM on a different compute host. This will make sure
> that none the VMs (same batch of VMs referenced by same the server-group
> id) is co-located on the same compute host.
>
>
>
> I posted a link to the openstack, search for ServerGroupAntiAffinityFilter
>
>
>
>
> http://docs.openstack.org/juno/config-reference/content/section_compute-scheduler.html
>
>
>
> Other link I found while searching:
> https://raymii.org/s/articles/Openstack_Affinity_Groups-make-sure-instances-are-on-the-same-or-a-different-hypervisor-host.html
>
>
>
> I would think other IAASes will support similar concepts ?!
>
I'm also not aware of it. Will check further on the support of other IaaSes.
>
>
> One question I had while investigating the feature, in the stratos
> partition resource definition documentation (see link below) we reference
> 3 properties for openstack:
>
> region, host and zone – how do they map to openstack concepts ?
>
> I am assuming that zone refers to availability zones (correct ?), not sure
> about host (btw, is there documentation which explains the mapping of
> stratos concepts to specific IAASes ? )
>
As you mentioned zone refers to availability zone. We do support host in
openstack as well. I could see the host support implementation in the code
base. However, I'm not sure whether we have tested it.
Seems we don't have any other doc explaining the mapping of deployment
policy properties to specific IaaSes other than the one given below by you.
>
>
>
> https://cwiki.apache.org/confluence/display/STRATOS/4.1.0+Network+Partition+Resource+Definition#id-4.1.0NetworkPartitionResourceDefinition-Propertydefinitions
>
>
>
>
>
> Another question would be where the new parameter should exist, deployment
> policy seems to make the most sense, for example a user specifies the
> co-location within a certain zone.
>
>
>
> If this feature is not very generic among IAASes then it probably would
> make sense to support this feature by extending stratos configuration so it
> allows IAAS specific boot parameter ?
>
>
>
> Also, jcloud will have to be extended as well to support this enhancement.
>
>
>
> WDYT ?
>
+1 for extending the existing openstack IaaS in order to implement this
feature unless it cannot be achieved with the current capability of
stratos. Will get others input as well on this.
Thanks,
Reka
>
>
> Thanks
>
>
>
> Martin
>
>
>
> *From:* Reka Thirunavukkarasu [mailto:reka@wso2.com]
> *Sent:* Wednesday, July 29, 2015 10:09 PM
> *To:* dev
> *Subject:* Re: DISCUSS: co-location of VMs in stratos (affinity /
> anti-affinity)
>
>
>
> Hi Martin,
>
> I will provide some details according to what i have understood. Please
> correct me, if I'm wrong.
>
> A group can consist of different kind of VMs which we called as cartridges
> in stratos. In that case, co-located affinity can be provided to VMs which
> are part of same group by providing a group level deployment policy. As you
> are already aware of, all the VMs which belong to that group will reside in
> the same partition where partition can be a host or zone. Apart from that,
> if you need to achieve anti-affinity, then you can define another group
> with set of other VMs and define deployment policy at the cartridge level.
> So that individual VM will be placed by referring to its own deployment
> policy rather than looking at the group level policy.
>
> AFAIU, this is the level of Affinity aware VM colocation that we supported
> now. Also please note that prior to write such application and cartridge
> group definition, the level of affinity among different types of VM has to
> be identified and experimented well, since stratos currently doesn't have
> the capability of automatically identified the affinity aware colocation
> VMs.
>
> If this doesn't fit with what you have asked, can you elaborate more on
> your perception of the Affinity aware VM colocation?
>
> Thanks,
>
> Reka
>
>
>
>
>
> On Thu, Jul 30, 2015 at 7:21 AM, Martin Eppel (meppel) <me...@cisco.com>
> wrote:
>
> Hi,
>
>
>
> As part of our original grouping design we proposed a feature to either
> co-locate or not co-locate individual VMs.
>
>
>
> Quoting from the original grouping specification:
>
>
>
> Quote begin:
> *Co-Location*
>
> *VMs within a group will be spun up based on the model of subscriptions,
> deployment policies and partitions. The model needs to be extended so that:*
>
>
>
> · *It must be possible to require VMs to be co-located on a single
> host.*
>
> · *It must be possible to require VMs to be located in separate
> zones.*
>
>
>
> Quote end.
>
>
>
> As we have some immediate needs we started looking into it and would like
> to get the input from the community for the requirement / implementation
> (with the idea to contribute it back to the master repository eventually).
>
>
>
> The question would be what is the current support for it (stratos 4.1) and
> what are the suggestions to enhance stratos to support this enhancement ?
>
>
>
> Any other ideas or considerations on how to support the enhancement ?
>
>
>
> Thanks
>
>
>
> Martin
>
>
>
>
> --
>
> Reka Thirunavukkarasu
> Senior Software Engineer,
> WSO2, Inc.:http://wso2.com,
>
> Mobile: +94776442007
>
>
>
--
Reka Thirunavukkarasu
Senior Software Engineer,
WSO2, Inc.:http://wso2.com,
Mobile: +94776442007
RE: DISCUSS: co-location of VMs in stratos (affinity /
anti-affinity)
Posted by "Martin Eppel (meppel)" <me...@cisco.com>.
Hi Reka,
Thanks for your reply. The affinity support you mentioned below will IMHO work only for a small number of cartirdges / instances and only if the cartridges don’t scale.
Instead it might be more useful to utilize affinity support of the underlying IAAS.
For example, openstack supports affinity which, at least at the IAAS level meets the requirement as well. It lets you define a flag with the values affinity / anti-affinity which behaves as follows:
If affinity is set, the scheduler will place all VMs (at boot time) on the same compute host
If anti-affinity is set, the scheduler will place each (out of the same batch of VMs) booted VM on a different compute host. This will make sure that none the VMs (same batch of VMs referenced by same the server-group id) is co-located on the same compute host.
I posted a link to the openstack, search for ServerGroupAntiAffinityFilter
http://docs.openstack.org/juno/config-reference/content/section_compute-scheduler.html
Other link I found while searching: https://raymii.org/s/articles/Openstack_Affinity_Groups-make-sure-instances-are-on-the-same-or-a-different-hypervisor-host.html
I would think other IAASes will support similar concepts ?!
One question I had while investigating the feature, in the stratos partition resource definition documentation (see link below) we reference 3 properties for openstack:
region, host and zone – how do they map to openstack concepts ?
I am assuming that zone refers to availability zones (correct ?), not sure about host (btw, is there documentation which explains the mapping of stratos concepts to specific IAASes ? )
https://cwiki.apache.org/confluence/display/STRATOS/4.1.0+Network+Partition+Resource+Definition#id-4.1.0NetworkPartitionResourceDefinition-Propertydefinitions
Another question would be where the new parameter should exist, deployment policy seems to make the most sense, for example a user specifies the co-location within a certain zone.
If this feature is not very generic among IAASes then it probably would make sense to support this feature by extending stratos configuration so it allows IAAS specific boot parameter ?
Also, jcloud will have to be extended as well to support this enhancement.
WDYT ?
Thanks
Martin
From: Reka Thirunavukkarasu [mailto:reka@wso2.com]
Sent: Wednesday, July 29, 2015 10:09 PM
To: dev
Subject: Re: DISCUSS: co-location of VMs in stratos (affinity / anti-affinity)
Hi Martin,
I will provide some details according to what i have understood. Please correct me, if I'm wrong.
A group can consist of different kind of VMs which we called as cartridges in stratos. In that case, co-located affinity can be provided to VMs which are part of same group by providing a group level deployment policy. As you are already aware of, all the VMs which belong to that group will reside in the same partition where partition can be a host or zone. Apart from that, if you need to achieve anti-affinity, then you can define another group with set of other VMs and define deployment policy at the cartridge level. So that individual VM will be placed by referring to its own deployment policy rather than looking at the group level policy.
AFAIU, this is the level of Affinity aware VM colocation that we supported now. Also please note that prior to write such application and cartridge group definition, the level of affinity among different types of VM has to be identified and experimented well, since stratos currently doesn't have the capability of automatically identified the affinity aware colocation VMs.
If this doesn't fit with what you have asked, can you elaborate more on your perception of the Affinity aware VM colocation?
Thanks,
Reka
On Thu, Jul 30, 2015 at 7:21 AM, Martin Eppel (meppel) <me...@cisco.com>> wrote:
Hi,
As part of our original grouping design we proposed a feature to either co-locate or not co-locate individual VMs.
Quoting from the original grouping specification:
Quote begin:
Co-Location
VMs within a group will be spun up based on the model of subscriptions, deployment policies and partitions. The model needs to be extended so that:
• It must be possible to require VMs to be co-located on a single host.
• It must be possible to require VMs to be located in separate zones.
Quote end.
As we have some immediate needs we started looking into it and would like to get the input from the community for the requirement / implementation (with the idea to contribute it back to the master repository eventually).
The question would be what is the current support for it (stratos 4.1) and what are the suggestions to enhance stratos to support this enhancement ?
Any other ideas or considerations on how to support the enhancement ?
Thanks
Martin
--
Reka Thirunavukkarasu
Senior Software Engineer,
WSO2, Inc.:http://wso2.com,
Mobile: +94776442007
Re: DISCUSS: co-location of VMs in stratos (affinity / anti-affinity)
Posted by Reka Thirunavukkarasu <re...@wso2.com>.
Hi Martin,
I will provide some details according to what i have understood. Please
correct me, if I'm wrong.
A group can consist of different kind of VMs which we called as cartridges
in stratos. In that case, co-located affinity can be provided to VMs which
are part of same group by providing a group level deployment policy. As you
are already aware of, all the VMs which belong to that group will reside in
the same partition where partition can be a host or zone. Apart from that,
if you need to achieve anti-affinity, then you can define another group
with set of other VMs and define deployment policy at the cartridge level.
So that individual VM will be placed by referring to its own deployment
policy rather than looking at the group level policy.
AFAIU, this is the level of Affinity aware VM colocation that we supported
now. Also please note that prior to write such application and cartridge
group definition, the level of affinity among different types of VM has to
be identified and experimented well, since stratos currently doesn't have
the capability of automatically identified the affinity aware colocation
VMs.
If this doesn't fit with what you have asked, can you elaborate more on
your perception of the Affinity aware VM colocation?
Thanks,
Reka
On Thu, Jul 30, 2015 at 7:21 AM, Martin Eppel (meppel) <me...@cisco.com>
wrote:
> Hi,
>
>
>
> As part of our original grouping design we proposed a feature to either
> co-locate or not co-locate individual VMs.
>
>
>
> Quoting from the original grouping specification:
>
>
>
> Quote begin:
> *Co-Location*
>
> *VMs within a group will be spun up based on the model of subscriptions,
> deployment policies and partitions. The model needs to be extended so that:*
>
>
>
> · *It must be possible to require VMs to be co-located on a single
> host.*
>
> · *It must be possible to require VMs to be located in separate
> zones.*
>
>
>
> Quote end.
>
>
>
> As we have some immediate needs we started looking into it and would like
> to get the input from the community for the requirement / implementation
> (with the idea to contribute it back to the master repository eventually).
>
>
>
> The question would be what is the current support for it (stratos 4.1) and
> what are the suggestions to enhance stratos to support this enhancement ?
>
>
>
> Any other ideas or considerations on how to support the enhancement ?
>
>
>
> Thanks
>
>
>
> Martin
>
--
Reka Thirunavukkarasu
Senior Software Engineer,
WSO2, Inc.:http://wso2.com,
Mobile: +94776442007