You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stratos.apache.org by Vanson Lim <vl...@cisco.com> on 2015/09/20 07:29:42 UTC
Openstack Availability zone settings in partition definition not
working
Stratos developers,
I found an issue where defining an availability "zone" property in a partition has no affect on controlling the placement of a VM launched
under openstack.
The code which builds the template (iaas.initialize) used by jclouds is executed only once before stratos appends the availability zone
properties to the iaas datastructure.
The problem is in the buildIaas() call shown below:
https://github.com/apache/stratos/blob/92ff7e9b5800d578a37d6aa82551d60fbdd66529/components/org.apache.stratos.cloud.controller/src/main/java/org/apache/stratos/cloud/controller/iaases/openstack/OpenstackPartitionValidator.java
The issue also exists in the cloudstack, ec2, gce and docker variants of this function.
@Override
public IaasProvider validate(Partition partition, Properties properties) throws InvalidPartitionException {
try {
// validate the existence of the zone and hosts properties.
if (properties.containsKey(Scope.region.toString())) {
String region = properties.getProperty(Scope.region.toString());
if (iaasProvider.getImage() != null && !iaasProvider.getImage().contains(region)) {
String msg = "Invalid partition detected, invalid region: [partition-id] " + partition.getId() +
" [region] " + region;
log.error(msg);
throw new InvalidPartitionException(msg);
}
iaas.isValidRegion(region);
IaasProvider updatedIaasProvider = new IaasProvider(iaasProvider);
* Iaas updatedIaas = CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
updatedIaas.setIaasProvider(updatedIaasProvider);
if (properties.containsKey(Scope.zone.toString())) {
String zone = properties.getProperty(Scope.zone.toString());
iaas.isValidZone(region, zone);
updatedIaasProvider.setProperty(CloudControllerConstants.AVAILABILITY_ZONE, zone);
* updatedIaas = CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
updatedIaas.setIaasProvider(updatedIaasProvider);
}
updateOtherProperties(updatedIaasProvider, properties);
return updatedIaasProvider;
} else {
return iaasProvider;
}
} catch (Exception e) {
String msg = "Invalid partition detected: [partition-id] " + partition.getId() + e.getMessage();
log.error(msg, e);
throw new InvalidPartitionException(msg, e);
}
}
private void updateOtherProperties(IaasProvider updatedIaasProvider,
Properties properties) {
Iaas updatedIaas;
try {
* updatedIaas = CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
for (Object property : properties.keySet()) {
if (property instanceof String) {
String key = (String) property;
updatedIaasProvider.setProperty(key,
properties.getProperty(key));
if (log.isDebugEnabled()) {
log.debug("Added property " + key
+ " to the IaasProvider.");
}
}
}
* updatedIaas = CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
updatedIaas.setIaasProvider(updatedIaasProvider);
} catch (InvalidIaasProviderException ignore) {
}
}
buidIaas() calls getIaas() under the covers and only initializes the compute template the first time it is called. buildIaas() is
called a second time after the availability_zone property is set, and then a third/fourth time in updateOtherProperties().
As show below, getIaas() only calls iaas.initilize() the first time it's called.
public class CloudControllerServiceUtil {
...
public static Iaas buildIaas(IaasProvider iaasProvider) throws InvalidIaasProviderException {
return iaasProvider.getIaas();
}
....
public class IaasProvider implements Serializable {
....
public Iaas getIaas() {
if (iaas == null) {
synchronized (IaasProvider.this) {
if (iaas == null) {
try {
iaas = CloudControllerUtil.createIaasInstance(this);
iaas.initialize();
} catch (InvalidIaasProviderException e) {
throw new RuntimeException("Could not create IaaS instance", e);
}
}
}
}
return iaas;
}
....
I propose the attached diffs to define buildIaas() such that it forces the iaas datastructure to be reinitialized, if this looks
okay, I'll see about getting this pushed upstream.
I've also validated this on my openstack setup.
Regards,
-Vanson
Re: Openstack Availability zone settings in partition definition not working
Posted by Akila Ravihansa Perera <ra...@wso2.com>.
Hi,
This is now fixed in stratos-4.1.x branch. I've tested and verified in ec2,
openstack, GCE. I fixed another minor issue in GCE IaaS which caused
partition validation exception to be thrown when availabilityZone is
defined in network partition. This is becuase GCE validator checks for
region string in image ID property which is correct for ec2 and openstack,
but not applicable for GCE.
I found another issue in which updating the network partition definition
(with updated region or availability zone) has no affect in VMs launched
via CC. If you need your updated network partition to take effect then you
will have to undeploy the application and redeploy the cartridge definition
and then deploy the application again.
Thanks.
On Wed, Sep 23, 2015 at 4:21 AM, Vanson Lim <vl...@cisco.com> wrote:
> On 9/22/15, 3:25 PM, Akila Ravihansa Perera wrote:
>
> Hi Vanson,
>
> I've updated the JIRA as requested. I was under the impression that issue
> only occurs when updating a network partition definition. Further
> investigation into this revealed that zone property is not properly set
> even at the initial deployment. Defining the zone property in the cartridge
> definition does not work either. Only way to set the zone is by updating
> cloud-controller.xml which will be applied globally for that IaaS provider.
>
>
> That's correct. Only setting the zone in the cloud-controller.xml works.
>
> -Vanson
>
>
>
> Thanks.
>
> On Tue, Sep 22, 2015 at 8:01 PM, Vanson Lim <vl...@cisco.com> wrote:
>
>> On 9/22/15, 1:30 AM, Akila Ravihansa Perera wrote:
>>
>> Hi Vanson,
>>
>> Thank you for providing a fix for this. I've created a JIRA [1] and
>> attached the patch. Since we are in a middle of a planned release thought
>> of committing it in the next release (ver. 4.1.4). Hope that is okay with
>> you.
>>
>> [1] https://issues.apache.org/jira/browse/STRATOS-1572
>>
>>
>> Yes, version 4.1.4 should be fine.
>>
>> Two comments:
>>
>> 1) Can we change the JIRA title from:
>>
>> "Updating network partition with a new availability zone has no affect on
>> VMs launched by CC"
>>
>> to:
>>
>> "Network partition availability zone setting has no affect on VMs
>> launched by CC"
>>
>> 2) Affected versions from:
>>
>> Affects Version/s:
>> 4.1.2, 4.1.3
>>
>> to:
>> Affects Version/s:
>> 4.1.0, 4.1.1, 4.1.2, 4.1.3
>>
>> Regards,
>>
>> -Vanson
>>
>>
>>
>> Regards,
>>
>> On Tue, Sep 22, 2015 at 1:53 AM, Vanson Lim <vl...@cisco.com> wrote:
>>
>>> Stratos developers,
>>>
>>> Please find attached, proposed diffs with further optimizations to the
>>> iaas specific code so that there is not impact to existing performance.
>>>
>>> -Vanson
>>>
>>>
>>>
>>> On 9/20/15, 2:36 AM, Vanson Lim wrote:
>>>
>>> On 9/20/15, 2:09 AM, Akila Ravihansa Perera wrote:
>>>
>>> Hi Vanson,
>>>
>>> Thanks for reporting this. This has been a known issue causing problems
>>> when updating IaaS parameters. A work-around would be to redeploy the
>>> cartridge definition which forces it to re-build. I haven't tested it
>>> though. Your solution also works, but were you able to look into to the
>>> performance overhead of re-building every time?
>>>
>>>
>>> Akila,
>>>
>>> No I haven't had a chance to look at the performance overhead, but yes,
>>> the suggested diffs are inefficient. Alternatively, for only the partition
>>> update case, we could optimize each Iaas's partition validate routines to
>>> only call buidIaas() once after all the properties have been updated.
>>>
>>> For Example, the revised function for the openstack IAAS is attached.
>>>
>>> Currently only the partition validate routines make use of buildIaas:
>>>
>>> https://github.com/apache/stratos/search?utf8=✓&q=buildIaas
>>>
>>> I'll try updating the cartridge, but I have a hard time see how that
>>> works for partition definitions since it would execute the same codeflow
>>> below. Only the validate() function below attempts to parse the
>>> partition's zone property and propagate it into the IaasProvider definition.
>>>
>>> Besides the cartridge update workaround, do you have any suggestions
>>> for other ways we could fix this.
>>>
>>> -Vanson
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Sun, Sep 20, 2015 at 10:59 AM, Vanson Lim <vl...@cisco.com> wrote:
>>>
>>>> Stratos developers,
>>>>
>>>> I found an issue where defining an availability "zone" property in a
>>>> partition has no affect on controlling the placement of a VM launched under
>>>> openstack.
>>>>
>>>> The code which builds the template (iaas.initialize) used by jclouds is
>>>> executed only once before stratos appends the availability zone properties
>>>> to the iaas datastructure.
>>>>
>>>> The problem is in the buildIaas() call shown below:
>>>>
>>>>
>>>> https://github.com/apache/stratos/blob/92ff7e9b5800d578a37d6aa82551d60fbdd66529/components/org.apache.stratos.cloud.controller/src/main/java/org/apache/stratos/cloud/controller/iaases/openstack/OpenstackPartitionValidator.java
>>>>
>>>>
>>>> The issue also exists in the cloudstack, ec2, gce and docker variants
>>>> of this function.
>>>>
>>>> @Override
>>>> public IaasProvider validate(Partition partition, Properties
>>>> properties) throws InvalidPartitionException {
>>>> try {
>>>> // validate the existence of the zone and hosts properties.
>>>> if (properties.containsKey(Scope.region.toString())) {
>>>> String region =
>>>> properties.getProperty(Scope.region.toString());
>>>>
>>>> if (iaasProvider.getImage() != null &&
>>>> !iaasProvider.getImage().contains(region)) {
>>>>
>>>> String msg = "Invalid partition detected, invalid
>>>> region: [partition-id] " + partition.getId() +
>>>> " [region] " + region;
>>>> log.error(msg);
>>>> throw new InvalidPartitionException(msg);
>>>> }
>>>>
>>>> iaas.isValidRegion(region);
>>>>
>>>> IaasProvider updatedIaasProvider = new
>>>> IaasProvider(iaasProvider);
>>>> * Iaas updatedIaas =
>>>> CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
>>>> updatedIaas.setIaasProvider(updatedIaasProvider);
>>>>
>>>> if (properties.containsKey(Scope.zone.toString())) {
>>>> String zone =
>>>> properties.getProperty(Scope.zone.toString());
>>>> iaas.isValidZone(region, zone);
>>>>
>>>>
>>>> updatedIaasProvider.setProperty(CloudControllerConstants.AVAILABILITY_ZONE,
>>>> zone);
>>>> * updatedIaas =
>>>> CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
>>>> updatedIaas.setIaasProvider(updatedIaasProvider);
>>>> }
>>>>
>>>> updateOtherProperties(updatedIaasProvider, properties);
>>>> return updatedIaasProvider;
>>>> } else {
>>>>
>>>> return iaasProvider;
>>>> }
>>>> } catch (Exception e) {
>>>> String msg = "Invalid partition detected: [partition-id] "
>>>> + partition.getId() + e.getMessage();
>>>> log.error(msg, e);
>>>> throw new InvalidPartitionException(msg, e);
>>>> }
>>>> }
>>>>
>>>>
>>>> private void updateOtherProperties(IaasProvider updatedIaasProvider,
>>>> Properties properties) {
>>>> Iaas updatedIaas;
>>>> try {
>>>> * updatedIaas =
>>>> CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
>>>>
>>>> for (Object property : properties.keySet()) {
>>>> if (property instanceof String) {
>>>> String key = (String) property;
>>>> updatedIaasProvider.setProperty(key,
>>>> properties.getProperty(key));
>>>> if (log.isDebugEnabled()) {
>>>> log.debug("Added property " + key
>>>> + " to the IaasProvider.");
>>>> }
>>>> }
>>>> }
>>>> * updatedIaas =
>>>> CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
>>>> updatedIaas.setIaasProvider(updatedIaasProvider);
>>>> } catch (InvalidIaasProviderException ignore) {
>>>> }
>>>>
>>>> }
>>>>
>>>>
>>>>
>>>> buidIaas() calls getIaas() under the covers and only initializes the
>>>> compute template the first time it is called. buildIaas() is called a
>>>> second time after the availability_zone property is set, and then a
>>>> third/fourth time in updateOtherProperties().
>>>>
>>>> As show below, getIaas() only calls iaas.initilize() the first time
>>>> it's called.
>>>>
>>>> public class CloudControllerServiceUtil {
>>>>
>>>> ...
>>>> public static Iaas buildIaas(IaasProvider iaasProvider) throws
>>>> InvalidIaasProviderException {
>>>> return iaasProvider.getIaas();
>>>> }
>>>> ....
>>>>
>>>> public class IaasProvider implements Serializable {
>>>> ....
>>>> public Iaas getIaas() {
>>>> if (iaas == null) {
>>>> synchronized (IaasProvider.this) {
>>>> if (iaas == null) {
>>>> try {
>>>> iaas =
>>>> CloudControllerUtil.createIaasInstance(this);
>>>> iaas.initialize();
>>>> } catch (InvalidIaasProviderException e) {
>>>> throw new RuntimeException("Could not create
>>>> IaaS instance", e);
>>>> }
>>>> }
>>>> }
>>>> }
>>>> return iaas;
>>>> }
>>>> ....
>>>>
>>>> I propose the attached diffs to define buildIaas() such that it forces
>>>> the iaas datastructure to be reinitialized, if this looks okay, I'll see
>>>> about getting this pushed upstream.
>>>> I've also validated this on my openstack setup.
>>>>
>>>>
>>>> Regards,
>>>>
>>>> -Vanson
>>>>
>>>>
>>>
>>>
>>> --
>>> Akila Ravihansa Perera
>>> WSO2 Inc.; http://wso2.com/
>>>
>>> Blog: http://ravihansa3000.blogspot.com
>>>
>>>
>>>
>>>
>>
>>
>> --
>> Akila Ravihansa Perera
>> WSO2 Inc.; http://wso2.com/
>>
>> Blog: http://ravihansa3000.blogspot.com
>>
>>
>>
>
>
> --
> Akila Ravihansa Perera
> WSO2 Inc.; http://wso2.com/
>
> Blog: http://ravihansa3000.blogspot.com
>
>
>
--
Akila Ravihansa Perera
WSO2 Inc.; http://wso2.com/
Blog: http://ravihansa3000.blogspot.com
Re: Openstack Availability zone settings in partition definition
not working
Posted by Vanson Lim <vl...@cisco.com>.
On 9/22/15, 3:25 PM, Akila Ravihansa Perera wrote:
> Hi Vanson,
>
> I've updated the JIRA as requested. I was under the impression that issue only occurs when updating a network partition definition.
> Further investigation into this revealed that zone property is not properly set even at the initial deployment. Defining the zone
> property in the cartridge definition does not work either. Only way to set the zone is by updating cloud-controller.xml which will be
> applied globally for that IaaS provider.
That's correct. Only setting the zone in the cloud-controller.xml works.
-Vanson
>
> Thanks.
>
> On Tue, Sep 22, 2015 at 8:01 PM, Vanson Lim <vlim@cisco.com <ma...@cisco.com>> wrote:
>
> On 9/22/15, 1:30 AM, Akila Ravihansa Perera wrote:
>> Hi Vanson,
>>
>> Thank you for providing a fix for this. I've created a JIRA [1] and attached the patch. Since we are in a middle of a planned
>> release thought of committing it in the next release (ver. 4.1.4). Hope that is okay with you.
>>
>> [1] https://issues.apache.org/jira/browse/STRATOS-1572
>>
>
> Yes, version 4.1.4 should be fine.
>
> Two comments:
>
> 1) Can we change the JIRA title from:
>
> "Updating network partition with a new availability zone has no affect on VMs launched by CC"
>
> to:
>
> "Network partition availability zone setting has no affect on VMs launched by CC"
>
> 2) Affected versions from:
>
> Affects Version/s:
> 4.1.2, 4.1.3
>
> to:
> Affects Version/s:
> 4.1.0, 4.1.1, 4.1.2, 4.1.3
>
> Regards,
>
> -Vanson
>
>
>
>> Regards,
>>
>> On Tue, Sep 22, 2015 at 1:53 AM, Vanson Lim <vlim@cisco.com <ma...@cisco.com>> wrote:
>>
>> Stratos developers,
>>
>> Please find attached, proposed diffs with further optimizations to the iaas specific code so that there is not impact to
>> existing performance.
>>
>> -Vanson
>>
>>
>>
>> On 9/20/15, 2:36 AM, Vanson Lim wrote:
>>> On 9/20/15, 2:09 AM, Akila Ravihansa Perera wrote:
>>>> Hi Vanson,
>>>>
>>>> Thanks for reporting this. This has been a known issue causing problems when updating IaaS parameters. A work-around would be
>>>> to redeploy the cartridge definition which forces it to re-build. I haven't tested it though. Your solution also works, but
>>>> were you able to look into to the performance overhead of re-building every time?
>>>>
>>>
>>> Akila,
>>>
>>> No I haven't had a chance to look at the performance overhead, but yes, the suggested diffs are inefficient. Alternatively, for
>>> only the partition update case, we could optimize each Iaas's partition validate routines to only call buidIaas() once after
>>> all the properties have been updated.
>>>
>>> For Example, the revised function for the openstack IAAS is attached.
>>>
>>> Currently only the partition validate routines make use of buildIaas:
>>>
>>> https://github.com/apache/stratos/search?utf8=✓&q=buildIaas <https://github.com/apache/stratos/search?utf8=%E2%9C%93&q=buildIaas>
>>>
>>> I'll try updating the cartridge, but I have a hard time see how that works for partition definitions since it would execute the
>>> same codeflow below. Only the validate() function below attempts to parse the partition's zone property and propagate it into
>>> the IaasProvider definition.
>>>
>>> Besides the cartridge update workaround, do you have any suggestions for other ways we could fix this.
>>>
>>> -Vanson
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>>
>>>>
>>>> On Sun, Sep 20, 2015 at 10:59 AM, Vanson Lim <vlim@cisco.com <ma...@cisco.com>> wrote:
>>>>
>>>> Stratos developers,
>>>>
>>>> I found an issue where defining an availability "zone" property in a partition has no affect on controlling the placement
>>>> of a VM launched under openstack.
>>>>
>>>> The code which builds the template (iaas.initialize) used by jclouds is executed only once before stratos appends the
>>>> availability zone properties to the iaas datastructure.
>>>>
>>>> The problem is in the buildIaas() call shown below:
>>>>
>>>> https://github.com/apache/stratos/blob/92ff7e9b5800d578a37d6aa82551d60fbdd66529/components/org.apache.stratos.cloud.controller/src/main/java/org/apache/stratos/cloud/controller/iaases/openstack/OpenstackPartitionValidator.java
>>>>
>>>>
>>>> The issue also exists in the cloudstack, ec2, gce and docker variants of this function.
>>>>
>>>> @Override
>>>> public IaasProvider validate(Partition partition, Properties properties) throws InvalidPartitionException {
>>>> try {
>>>> // validate the existence of the zone and hosts properties.
>>>> if (properties.containsKey(Scope.region.toString())) {
>>>> String region = properties.getProperty(Scope.region.toString());
>>>>
>>>> if (iaasProvider.getImage() != null && !iaasProvider.getImage().contains(region)) {
>>>>
>>>> String msg = "Invalid partition detected, invalid region: [partition-id] " + partition.getId() +
>>>> " [region] " + region;
>>>> log.error(msg);
>>>> throw new InvalidPartitionException(msg);
>>>> }
>>>>
>>>> iaas.isValidRegion(region);
>>>>
>>>> IaasProvider updatedIaasProvider = new IaasProvider(iaasProvider);
>>>> * Iaas updatedIaas = CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
>>>> updatedIaas.setIaasProvider(updatedIaasProvider);
>>>>
>>>> if (properties.containsKey(Scope.zone.toString())) {
>>>> String zone = properties.getProperty(Scope.zone.toString());
>>>> iaas.isValidZone(region, zone);
>>>>
>>>> updatedIaasProvider.setProperty(CloudControllerConstants.AVAILABILITY_ZONE, zone);
>>>> *updatedIaas = CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
>>>> updatedIaas.setIaasProvider(updatedIaasProvider);
>>>> }
>>>>
>>>> updateOtherProperties(updatedIaasProvider, properties);
>>>> return updatedIaasProvider;
>>>> } else {
>>>>
>>>> return iaasProvider;
>>>> }
>>>> } catch (Exception e) {
>>>> String msg = "Invalid partition detected: [partition-id] " + partition.getId() + e.getMessage();
>>>> log.error(msg, e);
>>>> throw new InvalidPartitionException(msg, e);
>>>> }
>>>> }
>>>>
>>>>
>>>> private void updateOtherProperties(IaasProvider updatedIaasProvider,
>>>> Properties properties) {
>>>> Iaas updatedIaas;
>>>> try {
>>>> *updatedIaas = CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
>>>>
>>>> for (Object property : properties.keySet()) {
>>>> if (property instanceof String) {
>>>> String key = (String) property;
>>>> updatedIaasProvider.setProperty(key,
>>>> properties.getProperty(key));
>>>> if (log.isDebugEnabled()) {
>>>> log.debug("Added property " + key
>>>> + " to the IaasProvider.");
>>>> }
>>>> }
>>>> }
>>>> *updatedIaas = CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
>>>> updatedIaas.setIaasProvider(updatedIaasProvider);
>>>> } catch (InvalidIaasProviderException ignore) {
>>>> }
>>>>
>>>> }
>>>>
>>>>
>>>>
>>>> buidIaas() calls getIaas() under the covers and only initializes the compute template the first time it is called.
>>>> buildIaas() is called a second time after the availability_zone property is set, and then a third/fourth time in
>>>> updateOtherProperties().
>>>>
>>>> As show below, getIaas() only calls iaas.initilize() the first time it's called.
>>>>
>>>> public class CloudControllerServiceUtil {
>>>>
>>>> ...
>>>> public static Iaas buildIaas(IaasProvider iaasProvider) throws InvalidIaasProviderException {
>>>> return iaasProvider.getIaas();
>>>> }
>>>> ....
>>>>
>>>> public class IaasProvider implements Serializable {
>>>> ....
>>>> public Iaas getIaas() {
>>>> if (iaas == null) {
>>>> synchronized (IaasProvider.this) {
>>>> if (iaas == null) {
>>>> try {
>>>> iaas = CloudControllerUtil.createIaasInstance(this);
>>>> iaas.initialize();
>>>> } catch (InvalidIaasProviderException e) {
>>>> throw new RuntimeException("Could not create IaaS instance", e);
>>>> }
>>>> }
>>>> }
>>>> }
>>>> return iaas;
>>>> }
>>>> ....
>>>>
>>>> I propose the attached diffs to define buildIaas() such that it forces the iaas datastructure to be
>>>> reinitialized, if this looks okay, I'll see about getting this pushed upstream.
>>>> I've also validated this on my openstack setup.
>>>>
>>>>
>>>> Regards,
>>>>
>>>> -Vanson
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Akila Ravihansa Perera
>>>> WSO2 Inc.; http://wso2.com/
>>>>
>>>> Blog: http://ravihansa3000.blogspot.com
>>>
>>
>>
>>
>>
>> --
>> Akila Ravihansa Perera
>> WSO2 Inc.; http://wso2.com/
>>
>> Blog: http://ravihansa3000.blogspot.com
>
>
>
>
> --
> Akila Ravihansa Perera
> WSO2 Inc.; http://wso2.com/
>
> Blog: http://ravihansa3000.blogspot.com
Re: Openstack Availability zone settings in partition definition not working
Posted by Akila Ravihansa Perera <ra...@wso2.com>.
Hi Vanson,
I've updated the JIRA as requested. I was under the impression that issue
only occurs when updating a network partition definition. Further
investigation into this revealed that zone property is not properly set
even at the initial deployment. Defining the zone property in the cartridge
definition does not work either. Only way to set the zone is by updating
cloud-controller.xml which will be applied globally for that IaaS provider.
Thanks.
On Tue, Sep 22, 2015 at 8:01 PM, Vanson Lim <vl...@cisco.com> wrote:
> On 9/22/15, 1:30 AM, Akila Ravihansa Perera wrote:
>
> Hi Vanson,
>
> Thank you for providing a fix for this. I've created a JIRA [1] and
> attached the patch. Since we are in a middle of a planned release thought
> of committing it in the next release (ver. 4.1.4). Hope that is okay with
> you.
>
> [1] https://issues.apache.org/jira/browse/STRATOS-1572
>
>
> Yes, version 4.1.4 should be fine.
>
> Two comments:
>
> 1) Can we change the JIRA title from:
>
> "Updating network partition with a new availability zone has no affect on
> VMs launched by CC"
>
> to:
>
> "Network partition availability zone setting has no affect on VMs launched
> by CC"
>
> 2) Affected versions from:
>
> Affects Version/s:
> 4.1.2, 4.1.3
>
> to:
> Affects Version/s:
> 4.1.0, 4.1.1, 4.1.2, 4.1.3
>
> Regards,
>
> -Vanson
>
>
>
> Regards,
>
> On Tue, Sep 22, 2015 at 1:53 AM, Vanson Lim <vl...@cisco.com> wrote:
>
>> Stratos developers,
>>
>> Please find attached, proposed diffs with further optimizations to the
>> iaas specific code so that there is not impact to existing performance.
>>
>> -Vanson
>>
>>
>>
>> On 9/20/15, 2:36 AM, Vanson Lim wrote:
>>
>> On 9/20/15, 2:09 AM, Akila Ravihansa Perera wrote:
>>
>> Hi Vanson,
>>
>> Thanks for reporting this. This has been a known issue causing problems
>> when updating IaaS parameters. A work-around would be to redeploy the
>> cartridge definition which forces it to re-build. I haven't tested it
>> though. Your solution also works, but were you able to look into to the
>> performance overhead of re-building every time?
>>
>>
>> Akila,
>>
>> No I haven't had a chance to look at the performance overhead, but yes,
>> the suggested diffs are inefficient. Alternatively, for only the partition
>> update case, we could optimize each Iaas's partition validate routines to
>> only call buidIaas() once after all the properties have been updated.
>>
>> For Example, the revised function for the openstack IAAS is attached.
>>
>> Currently only the partition validate routines make use of buildIaas:
>>
>> https://github.com/apache/stratos/search?utf8=✓&q=buildIaas
>>
>> I'll try updating the cartridge, but I have a hard time see how that
>> works for partition definitions since it would execute the same codeflow
>> below. Only the validate() function below attempts to parse the
>> partition's zone property and propagate it into the IaasProvider definition.
>>
>> Besides the cartridge update workaround, do you have any suggestions for
>> other ways we could fix this.
>>
>> -Vanson
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Sun, Sep 20, 2015 at 10:59 AM, Vanson Lim <vl...@cisco.com> wrote:
>>
>>> Stratos developers,
>>>
>>> I found an issue where defining an availability "zone" property in a
>>> partition has no affect on controlling the placement of a VM launched under
>>> openstack.
>>>
>>> The code which builds the template (iaas.initialize) used by jclouds is
>>> executed only once before stratos appends the availability zone properties
>>> to the iaas datastructure.
>>>
>>> The problem is in the buildIaas() call shown below:
>>>
>>>
>>> https://github.com/apache/stratos/blob/92ff7e9b5800d578a37d6aa82551d60fbdd66529/components/org.apache.stratos.cloud.controller/src/main/java/org/apache/stratos/cloud/controller/iaases/openstack/OpenstackPartitionValidator.java
>>>
>>>
>>> The issue also exists in the cloudstack, ec2, gce and docker variants of
>>> this function.
>>>
>>> @Override
>>> public IaasProvider validate(Partition partition, Properties
>>> properties) throws InvalidPartitionException {
>>> try {
>>> // validate the existence of the zone and hosts properties.
>>> if (properties.containsKey(Scope.region.toString())) {
>>> String region =
>>> properties.getProperty(Scope.region.toString());
>>>
>>> if (iaasProvider.getImage() != null &&
>>> !iaasProvider.getImage().contains(region)) {
>>>
>>> String msg = "Invalid partition detected, invalid
>>> region: [partition-id] " + partition.getId() +
>>> " [region] " + region;
>>> log.error(msg);
>>> throw new InvalidPartitionException(msg);
>>> }
>>>
>>> iaas.isValidRegion(region);
>>>
>>> IaasProvider updatedIaasProvider = new
>>> IaasProvider(iaasProvider);
>>> * Iaas updatedIaas =
>>> CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
>>> updatedIaas.setIaasProvider(updatedIaasProvider);
>>>
>>> if (properties.containsKey(Scope.zone.toString())) {
>>> String zone =
>>> properties.getProperty(Scope.zone.toString());
>>> iaas.isValidZone(region, zone);
>>>
>>>
>>> updatedIaasProvider.setProperty(CloudControllerConstants.AVAILABILITY_ZONE,
>>> zone);
>>> * updatedIaas =
>>> CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
>>> updatedIaas.setIaasProvider(updatedIaasProvider);
>>> }
>>>
>>> updateOtherProperties(updatedIaasProvider, properties);
>>> return updatedIaasProvider;
>>> } else {
>>>
>>> return iaasProvider;
>>> }
>>> } catch (Exception e) {
>>> String msg = "Invalid partition detected: [partition-id] " +
>>> partition.getId() + e.getMessage();
>>> log.error(msg, e);
>>> throw new InvalidPartitionException(msg, e);
>>> }
>>> }
>>>
>>>
>>> private void updateOtherProperties(IaasProvider updatedIaasProvider,
>>> Properties properties) {
>>> Iaas updatedIaas;
>>> try {
>>> * updatedIaas =
>>> CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
>>>
>>> for (Object property : properties.keySet()) {
>>> if (property instanceof String) {
>>> String key = (String) property;
>>> updatedIaasProvider.setProperty(key,
>>> properties.getProperty(key));
>>> if (log.isDebugEnabled()) {
>>> log.debug("Added property " + key
>>> + " to the IaasProvider.");
>>> }
>>> }
>>> }
>>> * updatedIaas =
>>> CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
>>> updatedIaas.setIaasProvider(updatedIaasProvider);
>>> } catch (InvalidIaasProviderException ignore) {
>>> }
>>>
>>> }
>>>
>>>
>>>
>>> buidIaas() calls getIaas() under the covers and only initializes the
>>> compute template the first time it is called. buildIaas() is called a
>>> second time after the availability_zone property is set, and then a
>>> third/fourth time in updateOtherProperties().
>>>
>>> As show below, getIaas() only calls iaas.initilize() the first time
>>> it's called.
>>>
>>> public class CloudControllerServiceUtil {
>>>
>>> ...
>>> public static Iaas buildIaas(IaasProvider iaasProvider) throws
>>> InvalidIaasProviderException {
>>> return iaasProvider.getIaas();
>>> }
>>> ....
>>>
>>> public class IaasProvider implements Serializable {
>>> ....
>>> public Iaas getIaas() {
>>> if (iaas == null) {
>>> synchronized (IaasProvider.this) {
>>> if (iaas == null) {
>>> try {
>>> iaas =
>>> CloudControllerUtil.createIaasInstance(this);
>>> iaas.initialize();
>>> } catch (InvalidIaasProviderException e) {
>>> throw new RuntimeException("Could not create
>>> IaaS instance", e);
>>> }
>>> }
>>> }
>>> }
>>> return iaas;
>>> }
>>> ....
>>>
>>> I propose the attached diffs to define buildIaas() such that it forces
>>> the iaas datastructure to be reinitialized, if this looks okay, I'll see
>>> about getting this pushed upstream.
>>> I've also validated this on my openstack setup.
>>>
>>>
>>> Regards,
>>>
>>> -Vanson
>>>
>>>
>>
>>
>> --
>> Akila Ravihansa Perera
>> WSO2 Inc.; http://wso2.com/
>>
>> Blog: http://ravihansa3000.blogspot.com
>>
>>
>>
>>
>
>
> --
> Akila Ravihansa Perera
> WSO2 Inc.; http://wso2.com/
>
> Blog: http://ravihansa3000.blogspot.com
>
>
>
--
Akila Ravihansa Perera
WSO2 Inc.; http://wso2.com/
Blog: http://ravihansa3000.blogspot.com
Re: Openstack Availability zone settings in partition definition
not working
Posted by Vanson Lim <vl...@cisco.com>.
On 9/22/15, 1:30 AM, Akila Ravihansa Perera wrote:
> Hi Vanson,
>
> Thank you for providing a fix for this. I've created a JIRA [1] and attached the patch. Since we are in a middle of a planned release
> thought of committing it in the next release (ver. 4.1.4). Hope that is okay with you.
>
> [1] https://issues.apache.org/jira/browse/STRATOS-1572
>
Yes, version 4.1.4 should be fine.
Two comments:
1) Can we change the JIRA title from:
"Updating network partition with a new availability zone has no affect on VMs launched by CC"
to:
"Network partition availability zone setting has no affect on VMs launched by CC"
2) Affected versions from:
Affects Version/s:
4.1.2, 4.1.3
to:
Affects Version/s:
4.1.0, 4.1.1, 4.1.2, 4.1.3
Regards,
-Vanson
> Regards,
>
> On Tue, Sep 22, 2015 at 1:53 AM, Vanson Lim <vlim@cisco.com <ma...@cisco.com>> wrote:
>
> Stratos developers,
>
> Please find attached, proposed diffs with further optimizations to the iaas specific code so that there is not impact to existing
> performance.
>
> -Vanson
>
>
>
> On 9/20/15, 2:36 AM, Vanson Lim wrote:
>> On 9/20/15, 2:09 AM, Akila Ravihansa Perera wrote:
>>> Hi Vanson,
>>>
>>> Thanks for reporting this. This has been a known issue causing problems when updating IaaS parameters. A work-around would be to
>>> redeploy the cartridge definition which forces it to re-build. I haven't tested it though. Your solution also works, but were you
>>> able to look into to the performance overhead of re-building every time?
>>>
>>
>> Akila,
>>
>> No I haven't had a chance to look at the performance overhead, but yes, the suggested diffs are inefficient. Alternatively, for
>> only the partition update case, we could optimize each Iaas's partition validate routines to only call buidIaas() once after all the
>> properties have been updated.
>>
>> For Example, the revised function for the openstack IAAS is attached.
>>
>> Currently only the partition validate routines make use of buildIaas:
>>
>> https://github.com/apache/stratos/search?utf8=✓&q=buildIaas <https://github.com/apache/stratos/search?utf8=%E2%9C%93&q=buildIaas>
>>
>> I'll try updating the cartridge, but I have a hard time see how that works for partition definitions since it would execute the same
>> codeflow below. Only the validate() function below attempts to parse the partition's zone property and propagate it into the
>> IaasProvider definition.
>>
>> Besides the cartridge update workaround, do you have any suggestions for other ways we could fix this.
>>
>> -Vanson
>>
>>
>>
>>
>>
>>
>>
>>
>>>
>>>
>>> On Sun, Sep 20, 2015 at 10:59 AM, Vanson Lim <vlim@cisco.com <ma...@cisco.com>> wrote:
>>>
>>> Stratos developers,
>>>
>>> I found an issue where defining an availability "zone" property in a partition has no affect on controlling the placement of a
>>> VM launched under openstack.
>>>
>>> The code which builds the template (iaas.initialize) used by jclouds is executed only once before stratos appends the
>>> availability zone properties to the iaas datastructure.
>>>
>>> The problem is in the buildIaas() call shown below:
>>>
>>> https://github.com/apache/stratos/blob/92ff7e9b5800d578a37d6aa82551d60fbdd66529/components/org.apache.stratos.cloud.controller/src/main/java/org/apache/stratos/cloud/controller/iaases/openstack/OpenstackPartitionValidator.java
>>>
>>>
>>> The issue also exists in the cloudstack, ec2, gce and docker variants of this function.
>>>
>>> @Override
>>> public IaasProvider validate(Partition partition, Properties properties) throws InvalidPartitionException {
>>> try {
>>> // validate the existence of the zone and hosts properties.
>>> if (properties.containsKey(Scope.region.toString())) {
>>> String region = properties.getProperty(Scope.region.toString());
>>>
>>> if (iaasProvider.getImage() != null && !iaasProvider.getImage().contains(region)) {
>>>
>>> String msg = "Invalid partition detected, invalid region: [partition-id] " + partition.getId() +
>>> " [region] " + region;
>>> log.error(msg);
>>> throw new InvalidPartitionException(msg);
>>> }
>>>
>>> iaas.isValidRegion(region);
>>>
>>> IaasProvider updatedIaasProvider = new IaasProvider(iaasProvider);
>>> * Iaas updatedIaas = CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
>>> updatedIaas.setIaasProvider(updatedIaasProvider);
>>>
>>> if (properties.containsKey(Scope.zone.toString())) {
>>> String zone = properties.getProperty(Scope.zone.toString());
>>> iaas.isValidZone(region, zone);
>>>
>>> updatedIaasProvider.setProperty(CloudControllerConstants.AVAILABILITY_ZONE, zone);
>>> * updatedIaas = CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
>>> updatedIaas.setIaasProvider(updatedIaasProvider);
>>> }
>>>
>>> updateOtherProperties(updatedIaasProvider, properties);
>>> return updatedIaasProvider;
>>> } else {
>>>
>>> return iaasProvider;
>>> }
>>> } catch (Exception e) {
>>> String msg = "Invalid partition detected: [partition-id] " + partition.getId() + e.getMessage();
>>> log.error(msg, e);
>>> throw new InvalidPartitionException(msg, e);
>>> }
>>> }
>>>
>>>
>>> private void updateOtherProperties(IaasProvider updatedIaasProvider,
>>> Properties properties) {
>>> Iaas updatedIaas;
>>> try {
>>> * updatedIaas = CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
>>>
>>> for (Object property : properties.keySet()) {
>>> if (property instanceof String) {
>>> String key = (String) property;
>>> updatedIaasProvider.setProperty(key,
>>> properties.getProperty(key));
>>> if (log.isDebugEnabled()) {
>>> log.debug("Added property " + key
>>> + " to the IaasProvider.");
>>> }
>>> }
>>> }
>>> * updatedIaas = CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
>>> updatedIaas.setIaasProvider(updatedIaasProvider);
>>> } catch (InvalidIaasProviderException ignore) {
>>> }
>>>
>>> }
>>>
>>>
>>>
>>> buidIaas() calls getIaas() under the covers and only initializes the compute template the first time it is called.
>>> buildIaas() is called a second time after the availability_zone property is set, and then a third/fourth time in
>>> updateOtherProperties().
>>>
>>> As show below, getIaas() only calls iaas.initilize() the first time it's called.
>>>
>>> public class CloudControllerServiceUtil {
>>>
>>> ...
>>> public static Iaas buildIaas(IaasProvider iaasProvider) throws InvalidIaasProviderException {
>>> return iaasProvider.getIaas();
>>> }
>>> ....
>>>
>>> public class IaasProvider implements Serializable {
>>> ....
>>> public Iaas getIaas() {
>>> if (iaas == null) {
>>> synchronized (IaasProvider.this) {
>>> if (iaas == null) {
>>> try {
>>> iaas = CloudControllerUtil.createIaasInstance(this);
>>> iaas.initialize();
>>> } catch (InvalidIaasProviderException e) {
>>> throw new RuntimeException("Could not create IaaS instance", e);
>>> }
>>> }
>>> }
>>> }
>>> return iaas;
>>> }
>>> ....
>>>
>>> I propose the attached diffs to define buildIaas() such that it forces the iaas datastructure to be reinitialized, if
>>> this looks okay, I'll see about getting this pushed upstream.
>>> I've also validated this on my openstack setup.
>>>
>>>
>>> Regards,
>>>
>>> -Vanson
>>>
>>>
>>>
>>>
>>> --
>>> Akila Ravihansa Perera
>>> WSO2 Inc.; http://wso2.com/
>>>
>>> Blog: http://ravihansa3000.blogspot.com
>>
>
>
>
>
> --
> Akila Ravihansa Perera
> WSO2 Inc.; http://wso2.com/
>
> Blog: http://ravihansa3000.blogspot.com
Re: Openstack Availability zone settings in partition definition not working
Posted by Akila Ravihansa Perera <ra...@wso2.com>.
Hi Vanson,
Thank you for providing a fix for this. I've created a JIRA [1] and
attached the patch. Since we are in a middle of a planned release thought
of committing it in the next release (ver. 4.1.4). Hope that is okay with
you.
[1] https://issues.apache.org/jira/browse/STRATOS-1572
Regards,
On Tue, Sep 22, 2015 at 1:53 AM, Vanson Lim <vl...@cisco.com> wrote:
> Stratos developers,
>
> Please find attached, proposed diffs with further optimizations to the
> iaas specific code so that there is not impact to existing performance.
>
> -Vanson
>
>
>
> On 9/20/15, 2:36 AM, Vanson Lim wrote:
>
> On 9/20/15, 2:09 AM, Akila Ravihansa Perera wrote:
>
> Hi Vanson,
>
> Thanks for reporting this. This has been a known issue causing problems
> when updating IaaS parameters. A work-around would be to redeploy the
> cartridge definition which forces it to re-build. I haven't tested it
> though. Your solution also works, but were you able to look into to the
> performance overhead of re-building every time?
>
>
> Akila,
>
> No I haven't had a chance to look at the performance overhead, but yes,
> the suggested diffs are inefficient. Alternatively, for only the partition
> update case, we could optimize each Iaas's partition validate routines to
> only call buidIaas() once after all the properties have been updated.
>
> For Example, the revised function for the openstack IAAS is attached.
>
> Currently only the partition validate routines make use of buildIaas:
>
> https://github.com/apache/stratos/search?utf8=✓&q=buildIaas
>
> I'll try updating the cartridge, but I have a hard time see how that works
> for partition definitions since it would execute the same codeflow below.
> Only the validate() function below attempts to parse the partition's zone
> property and propagate it into the IaasProvider definition.
>
> Besides the cartridge update workaround, do you have any suggestions for
> other ways we could fix this.
>
> -Vanson
>
>
>
>
>
>
>
>
>
>
> On Sun, Sep 20, 2015 at 10:59 AM, Vanson Lim <vl...@cisco.com> wrote:
>
>> Stratos developers,
>>
>> I found an issue where defining an availability "zone" property in a
>> partition has no affect on controlling the placement of a VM launched under
>> openstack.
>>
>> The code which builds the template (iaas.initialize) used by jclouds is
>> executed only once before stratos appends the availability zone properties
>> to the iaas datastructure.
>>
>> The problem is in the buildIaas() call shown below:
>>
>>
>> https://github.com/apache/stratos/blob/92ff7e9b5800d578a37d6aa82551d60fbdd66529/components/org.apache.stratos.cloud.controller/src/main/java/org/apache/stratos/cloud/controller/iaases/openstack/OpenstackPartitionValidator.java
>>
>>
>> The issue also exists in the cloudstack, ec2, gce and docker variants of
>> this function.
>>
>> @Override
>> public IaasProvider validate(Partition partition, Properties
>> properties) throws InvalidPartitionException {
>> try {
>> // validate the existence of the zone and hosts properties.
>> if (properties.containsKey(Scope.region.toString())) {
>> String region =
>> properties.getProperty(Scope.region.toString());
>>
>> if (iaasProvider.getImage() != null &&
>> !iaasProvider.getImage().contains(region)) {
>>
>> String msg = "Invalid partition detected, invalid
>> region: [partition-id] " + partition.getId() +
>> " [region] " + region;
>> log.error(msg);
>> throw new InvalidPartitionException(msg);
>> }
>>
>> iaas.isValidRegion(region);
>>
>> IaasProvider updatedIaasProvider = new
>> IaasProvider(iaasProvider);
>> * Iaas updatedIaas =
>> CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
>> updatedIaas.setIaasProvider(updatedIaasProvider);
>>
>> if (properties.containsKey(Scope.zone.toString())) {
>> String zone =
>> properties.getProperty(Scope.zone.toString());
>> iaas.isValidZone(region, zone);
>>
>>
>> updatedIaasProvider.setProperty(CloudControllerConstants.AVAILABILITY_ZONE,
>> zone);
>> * updatedIaas =
>> CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
>> updatedIaas.setIaasProvider(updatedIaasProvider);
>> }
>>
>> updateOtherProperties(updatedIaasProvider, properties);
>> return updatedIaasProvider;
>> } else {
>>
>> return iaasProvider;
>> }
>> } catch (Exception e) {
>> String msg = "Invalid partition detected: [partition-id] " +
>> partition.getId() + e.getMessage();
>> log.error(msg, e);
>> throw new InvalidPartitionException(msg, e);
>> }
>> }
>>
>>
>> private void updateOtherProperties(IaasProvider updatedIaasProvider,
>> Properties properties) {
>> Iaas updatedIaas;
>> try {
>> * updatedIaas =
>> CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
>>
>> for (Object property : properties.keySet()) {
>> if (property instanceof String) {
>> String key = (String) property;
>> updatedIaasProvider.setProperty(key,
>> properties.getProperty(key));
>> if (log.isDebugEnabled()) {
>> log.debug("Added property " + key
>> + " to the IaasProvider.");
>> }
>> }
>> }
>> * updatedIaas =
>> CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
>> updatedIaas.setIaasProvider(updatedIaasProvider);
>> } catch (InvalidIaasProviderException ignore) {
>> }
>>
>> }
>>
>>
>>
>> buidIaas() calls getIaas() under the covers and only initializes the
>> compute template the first time it is called. buildIaas() is called a
>> second time after the availability_zone property is set, and then a
>> third/fourth time in updateOtherProperties().
>>
>> As show below, getIaas() only calls iaas.initilize() the first time
>> it's called.
>>
>> public class CloudControllerServiceUtil {
>>
>> ...
>> public static Iaas buildIaas(IaasProvider iaasProvider) throws
>> InvalidIaasProviderException {
>> return iaasProvider.getIaas();
>> }
>> ....
>>
>> public class IaasProvider implements Serializable {
>> ....
>> public Iaas getIaas() {
>> if (iaas == null) {
>> synchronized (IaasProvider.this) {
>> if (iaas == null) {
>> try {
>> iaas =
>> CloudControllerUtil.createIaasInstance(this);
>> iaas.initialize();
>> } catch (InvalidIaasProviderException e) {
>> throw new RuntimeException("Could not create IaaS
>> instance", e);
>> }
>> }
>> }
>> }
>> return iaas;
>> }
>> ....
>>
>> I propose the attached diffs to define buildIaas() such that it forces
>> the iaas datastructure to be reinitialized, if this looks okay, I'll see
>> about getting this pushed upstream.
>> I've also validated this on my openstack setup.
>>
>>
>> Regards,
>>
>> -Vanson
>>
>>
>
>
> --
> Akila Ravihansa Perera
> WSO2 Inc.; http://wso2.com/
>
> Blog: http://ravihansa3000.blogspot.com
>
>
>
>
--
Akila Ravihansa Perera
WSO2 Inc.; http://wso2.com/
Blog: http://ravihansa3000.blogspot.com
Re: Openstack Availability zone settings in partition definition
not working
Posted by Vanson Lim <vl...@cisco.com>.
Stratos developers,
Please find attached, proposed diffs with further optimizations to the iaas specific code so that there is not impact to existing performance.
-Vanson
On 9/20/15, 2:36 AM, Vanson Lim wrote:
> On 9/20/15, 2:09 AM, Akila Ravihansa Perera wrote:
>> Hi Vanson,
>>
>> Thanks for reporting this. This has been a known issue causing problems when updating IaaS parameters. A work-around would be to
>> redeploy the cartridge definition which forces it to re-build. I haven't tested it though. Your solution also works, but were you able
>> to look into to the performance overhead of re-building every time?
>>
>
> Akila,
>
> No I haven't had a chance to look at the performance overhead, but yes, the suggested diffs are inefficient. Alternatively, for only the
> partition update case, we could optimize each Iaas's partition validate routines to only call buidIaas() once after all the properties
> have been updated.
>
> For Example, the revised function for the openstack IAAS is attached.
>
> Currently only the partition validate routines make use of buildIaas:
>
> https://github.com/apache/stratos/search?utf8=✓&q=buildIaas
>
> I'll try updating the cartridge, but I have a hard time see how that works for partition definitions since it would execute the same
> codeflow below. Only the validate() function below attempts to parse the partition's zone property and propagate it into the
> IaasProvider definition.
>
> Besides the cartridge update workaround, do you have any suggestions for other ways we could fix this.
>
> -Vanson
>
>
>
>
>
>
>
>
>>
>>
>> On Sun, Sep 20, 2015 at 10:59 AM, Vanson Lim <vlim@cisco.com <ma...@cisco.com>> wrote:
>>
>> Stratos developers,
>>
>> I found an issue where defining an availability "zone" property in a partition has no affect on controlling the placement of a VM
>> launched under openstack.
>>
>> The code which builds the template (iaas.initialize) used by jclouds is executed only once before stratos appends the availability
>> zone properties to the iaas datastructure.
>>
>> The problem is in the buildIaas() call shown below:
>>
>> https://github.com/apache/stratos/blob/92ff7e9b5800d578a37d6aa82551d60fbdd66529/components/org.apache.stratos.cloud.controller/src/main/java/org/apache/stratos/cloud/controller/iaases/openstack/OpenstackPartitionValidator.java
>>
>>
>> The issue also exists in the cloudstack, ec2, gce and docker variants of this function.
>>
>> @Override
>> public IaasProvider validate(Partition partition, Properties properties) throws InvalidPartitionException {
>> try {
>> // validate the existence of the zone and hosts properties.
>> if (properties.containsKey(Scope.region.toString())) {
>> String region = properties.getProperty(Scope.region.toString());
>>
>> if (iaasProvider.getImage() != null && !iaasProvider.getImage().contains(region)) {
>>
>> String msg = "Invalid partition detected, invalid region: [partition-id] " + partition.getId() +
>> " [region] " + region;
>> log.error(msg);
>> throw new InvalidPartitionException(msg);
>> }
>>
>> iaas.isValidRegion(region);
>>
>> IaasProvider updatedIaasProvider = new IaasProvider(iaasProvider);
>> * Iaas updatedIaas = CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
>> updatedIaas.setIaasProvider(updatedIaasProvider);
>>
>> if (properties.containsKey(Scope.zone.toString())) {
>> String zone = properties.getProperty(Scope.zone.toString());
>> iaas.isValidZone(region, zone);
>>
>> updatedIaasProvider.setProperty(CloudControllerConstants.AVAILABILITY_ZONE, zone);
>> * updatedIaas = CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
>> updatedIaas.setIaasProvider(updatedIaasProvider);
>> }
>>
>> updateOtherProperties(updatedIaasProvider, properties);
>> return updatedIaasProvider;
>> } else {
>>
>> return iaasProvider;
>> }
>> } catch (Exception e) {
>> String msg = "Invalid partition detected: [partition-id] " + partition.getId() + e.getMessage();
>> log.error(msg, e);
>> throw new InvalidPartitionException(msg, e);
>> }
>> }
>>
>>
>> private void updateOtherProperties(IaasProvider updatedIaasProvider,
>> Properties properties) {
>> Iaas updatedIaas;
>> try {
>> * updatedIaas = CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
>>
>> for (Object property : properties.keySet()) {
>> if (property instanceof String) {
>> String key = (String) property;
>> updatedIaasProvider.setProperty(key,
>> properties.getProperty(key));
>> if (log.isDebugEnabled()) {
>> log.debug("Added property " + key
>> + " to the IaasProvider.");
>> }
>> }
>> }
>> * updatedIaas = CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
>> updatedIaas.setIaasProvider(updatedIaasProvider);
>> } catch (InvalidIaasProviderException ignore) {
>> }
>>
>> }
>>
>>
>>
>> buidIaas() calls getIaas() under the covers and only initializes the compute template the first time it is called.
>> buildIaas() is called a second time after the availability_zone property is set, and then a third/fourth time in
>> updateOtherProperties().
>>
>> As show below, getIaas() only calls iaas.initilize() the first time it's called.
>>
>> public class CloudControllerServiceUtil {
>>
>> ...
>> public static Iaas buildIaas(IaasProvider iaasProvider) throws InvalidIaasProviderException {
>> return iaasProvider.getIaas();
>> }
>> ....
>>
>> public class IaasProvider implements Serializable {
>> ....
>> public Iaas getIaas() {
>> if (iaas == null) {
>> synchronized (IaasProvider.this) {
>> if (iaas == null) {
>> try {
>> iaas = CloudControllerUtil.createIaasInstance(this);
>> iaas.initialize();
>> } catch (InvalidIaasProviderException e) {
>> throw new RuntimeException("Could not create IaaS instance", e);
>> }
>> }
>> }
>> }
>> return iaas;
>> }
>> ....
>>
>> I propose the attached diffs to define buildIaas() such that it forces the iaas datastructure to be reinitialized, if this
>> looks okay, I'll see about getting this pushed upstream.
>> I've also validated this on my openstack setup.
>>
>>
>> Regards,
>>
>> -Vanson
>>
>>
>>
>>
>> --
>> Akila Ravihansa Perera
>> WSO2 Inc.; http://wso2.com/
>>
>> Blog: http://ravihansa3000.blogspot.com
>
Re: Openstack Availability zone settings in partition definition
not working
Posted by Vanson Lim <vl...@cisco.com>.
On 9/20/15, 2:09 AM, Akila Ravihansa Perera wrote:
> Hi Vanson,
>
> Thanks for reporting this. This has been a known issue causing problems when updating IaaS parameters. A work-around would be to redeploy
> the cartridge definition which forces it to re-build. I haven't tested it though. Your solution also works, but were you able to look
> into to the performance overhead of re-building every time?
>
Akila,
No I haven't had a chance to look at the performance overhead, but yes, the suggested diffs are inefficient. Alternatively, for only the
partition update case, we could optimize each Iaas's partition validate routines to only call buidIaas() once after all the properties have
been updated.
For Example, the revised function for the openstack IAAS is attached.
Currently only the partition validate routines make use of buildIaas:
https://github.com/apache/stratos/search?utf8=✓&q=buildIaas
I'll try updating the cartridge, but I have a hard time see how that works for partition definitions since it would execute the same
codeflow below. Only the validate() function below attempts to parse the partition's zone property and propagate it into the IaasProvider
definition.
Besides the cartridge update workaround, do you have any suggestions for other ways we could fix this.
-Vanson
>
>
> On Sun, Sep 20, 2015 at 10:59 AM, Vanson Lim <vlim@cisco.com <ma...@cisco.com>> wrote:
>
> Stratos developers,
>
> I found an issue where defining an availability "zone" property in a partition has no affect on controlling the placement of a VM
> launched under openstack.
>
> The code which builds the template (iaas.initialize) used by jclouds is executed only once before stratos appends the availability
> zone properties to the iaas datastructure.
>
> The problem is in the buildIaas() call shown below:
>
> https://github.com/apache/stratos/blob/92ff7e9b5800d578a37d6aa82551d60fbdd66529/components/org.apache.stratos.cloud.controller/src/main/java/org/apache/stratos/cloud/controller/iaases/openstack/OpenstackPartitionValidator.java
>
>
> The issue also exists in the cloudstack, ec2, gce and docker variants of this function.
>
> @Override
> public IaasProvider validate(Partition partition, Properties properties) throws InvalidPartitionException {
> try {
> // validate the existence of the zone and hosts properties.
> if (properties.containsKey(Scope.region.toString())) {
> String region = properties.getProperty(Scope.region.toString());
>
> if (iaasProvider.getImage() != null && !iaasProvider.getImage().contains(region)) {
>
> String msg = "Invalid partition detected, invalid region: [partition-id] " + partition.getId() +
> " [region] " + region;
> log.error(msg);
> throw new InvalidPartitionException(msg);
> }
>
> iaas.isValidRegion(region);
>
> IaasProvider updatedIaasProvider = new IaasProvider(iaasProvider);
> * Iaas updatedIaas = CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
> updatedIaas.setIaasProvider(updatedIaasProvider);
>
> if (properties.containsKey(Scope.zone.toString())) {
> String zone = properties.getProperty(Scope.zone.toString());
> iaas.isValidZone(region, zone);
>
> updatedIaasProvider.setProperty(CloudControllerConstants.AVAILABILITY_ZONE, zone);
> * updatedIaas = CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
> updatedIaas.setIaasProvider(updatedIaasProvider);
> }
>
> updateOtherProperties(updatedIaasProvider, properties);
> return updatedIaasProvider;
> } else {
>
> return iaasProvider;
> }
> } catch (Exception e) {
> String msg = "Invalid partition detected: [partition-id] " + partition.getId() + e.getMessage();
> log.error(msg, e);
> throw new InvalidPartitionException(msg, e);
> }
> }
>
>
> private void updateOtherProperties(IaasProvider updatedIaasProvider,
> Properties properties) {
> Iaas updatedIaas;
> try {
> * updatedIaas = CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
>
> for (Object property : properties.keySet()) {
> if (property instanceof String) {
> String key = (String) property;
> updatedIaasProvider.setProperty(key,
> properties.getProperty(key));
> if (log.isDebugEnabled()) {
> log.debug("Added property " + key
> + " to the IaasProvider.");
> }
> }
> }
> * updatedIaas = CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
> updatedIaas.setIaasProvider(updatedIaasProvider);
> } catch (InvalidIaasProviderException ignore) {
> }
>
> }
>
>
>
> buidIaas() calls getIaas() under the covers and only initializes the compute template the first time it is called.
> buildIaas() is called a second time after the availability_zone property is set, and then a third/fourth time in
> updateOtherProperties().
>
> As show below, getIaas() only calls iaas.initilize() the first time it's called.
>
> public class CloudControllerServiceUtil {
>
> ...
> public static Iaas buildIaas(IaasProvider iaasProvider) throws InvalidIaasProviderException {
> return iaasProvider.getIaas();
> }
> ....
>
> public class IaasProvider implements Serializable {
> ....
> public Iaas getIaas() {
> if (iaas == null) {
> synchronized (IaasProvider.this) {
> if (iaas == null) {
> try {
> iaas = CloudControllerUtil.createIaasInstance(this);
> iaas.initialize();
> } catch (InvalidIaasProviderException e) {
> throw new RuntimeException("Could not create IaaS instance", e);
> }
> }
> }
> }
> return iaas;
> }
> ....
>
> I propose the attached diffs to define buildIaas() such that it forces the iaas datastructure to be reinitialized, if this
> looks okay, I'll see about getting this pushed upstream.
> I've also validated this on my openstack setup.
>
>
> Regards,
>
> -Vanson
>
>
>
>
> --
> Akila Ravihansa Perera
> WSO2 Inc.; http://wso2.com/
>
> Blog: http://ravihansa3000.blogspot.com
Re: Openstack Availability zone settings in partition definition not working
Posted by Akila Ravihansa Perera <ra...@wso2.com>.
Hi Vanson,
Thanks for reporting this. This has been a known issue causing problems
when updating IaaS parameters. A work-around would be to redeploy the
cartridge definition which forces it to re-build. I haven't tested it
though. Your solution also works, but were you able to look into to the
performance overhead of re-building every time?
On Sun, Sep 20, 2015 at 10:59 AM, Vanson Lim <vl...@cisco.com> wrote:
> Stratos developers,
>
> I found an issue where defining an availability "zone" property in a
> partition has no affect on controlling the placement of a VM launched under
> openstack.
>
> The code which builds the template (iaas.initialize) used by jclouds is
> executed only once before stratos appends the availability zone properties
> to the iaas datastructure.
>
> The problem is in the buildIaas() call shown below:
>
>
> https://github.com/apache/stratos/blob/92ff7e9b5800d578a37d6aa82551d60fbdd66529/components/org.apache.stratos.cloud.controller/src/main/java/org/apache/stratos/cloud/controller/iaases/openstack/OpenstackPartitionValidator.java
>
>
> The issue also exists in the cloudstack, ec2, gce and docker variants of
> this function.
>
> @Override
> public IaasProvider validate(Partition partition, Properties
> properties) throws InvalidPartitionException {
> try {
> // validate the existence of the zone and hosts properties.
> if (properties.containsKey(Scope.region.toString())) {
> String region =
> properties.getProperty(Scope.region.toString());
>
> if (iaasProvider.getImage() != null &&
> !iaasProvider.getImage().contains(region)) {
>
> String msg = "Invalid partition detected, invalid
> region: [partition-id] " + partition.getId() +
> " [region] " + region;
> log.error(msg);
> throw new InvalidPartitionException(msg);
> }
>
> iaas.isValidRegion(region);
>
> IaasProvider updatedIaasProvider = new
> IaasProvider(iaasProvider);
> * Iaas updatedIaas =
> CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
> updatedIaas.setIaasProvider(updatedIaasProvider);
>
> if (properties.containsKey(Scope.zone.toString())) {
> String zone =
> properties.getProperty(Scope.zone.toString());
> iaas.isValidZone(region, zone);
>
>
> updatedIaasProvider.setProperty(CloudControllerConstants.AVAILABILITY_ZONE,
> zone);
> * updatedIaas =
> CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
> updatedIaas.setIaasProvider(updatedIaasProvider);
> }
>
> updateOtherProperties(updatedIaasProvider, properties);
> return updatedIaasProvider;
> } else {
>
> return iaasProvider;
> }
> } catch (Exception e) {
> String msg = "Invalid partition detected: [partition-id] " +
> partition.getId() + e.getMessage();
> log.error(msg, e);
> throw new InvalidPartitionException(msg, e);
> }
> }
>
>
> private void updateOtherProperties(IaasProvider updatedIaasProvider,
> Properties properties) {
> Iaas updatedIaas;
> try {
> * updatedIaas =
> CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
>
> for (Object property : properties.keySet()) {
> if (property instanceof String) {
> String key = (String) property;
> updatedIaasProvider.setProperty(key,
> properties.getProperty(key));
> if (log.isDebugEnabled()) {
> log.debug("Added property " + key
> + " to the IaasProvider.");
> }
> }
> }
> * updatedIaas =
> CloudControllerServiceUtil.buildIaas(updatedIaasProvider);*
> updatedIaas.setIaasProvider(updatedIaasProvider);
> } catch (InvalidIaasProviderException ignore) {
> }
>
> }
>
>
>
> buidIaas() calls getIaas() under the covers and only initializes the
> compute template the first time it is called. buildIaas() is called a
> second time after the availability_zone property is set, and then a
> third/fourth time in updateOtherProperties().
>
> As show below, getIaas() only calls iaas.initilize() the first time it's
> called.
>
> public class CloudControllerServiceUtil {
>
> ...
> public static Iaas buildIaas(IaasProvider iaasProvider) throws
> InvalidIaasProviderException {
> return iaasProvider.getIaas();
> }
> ....
>
> public class IaasProvider implements Serializable {
> ....
> public Iaas getIaas() {
> if (iaas == null) {
> synchronized (IaasProvider.this) {
> if (iaas == null) {
> try {
> iaas =
> CloudControllerUtil.createIaasInstance(this);
> iaas.initialize();
> } catch (InvalidIaasProviderException e) {
> throw new RuntimeException("Could not create IaaS
> instance", e);
> }
> }
> }
> }
> return iaas;
> }
> ....
>
> I propose the attached diffs to define buildIaas() such that it forces the
> iaas datastructure to be reinitialized, if this looks okay, I'll see about
> getting this pushed upstream.
> I've also validated this on my openstack setup.
>
>
> Regards,
>
> -Vanson
>
>
--
Akila Ravihansa Perera
WSO2 Inc.; http://wso2.com/
Blog: http://ravihansa3000.blogspot.com