You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stratos.apache.org by Akila Ravihansa Perera <ra...@wso2.com> on 2015/11/01 07:51:19 UTC
Re: Stratos does not select the correct partition in a multi
partition deployment
Hi Reka,
Please find the complete log attached herewith. I've noticed that this
happens when there are cartridge groups defined.
While debugging the code I observed that ClusterInstanceContext ->
partitionCtxts [1] map only contains the partition-1.
[1]
https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/context/cluster/ClusterInstanceContext.java#L47
Thanks.
On Fri, Oct 30, 2015 at 11:07 AM, Reka Thirunavukkarasu <re...@wso2.com>
wrote:
> Thanks for the confirmation Gayan. I have checked it in master. I have
> verified the intergration test logs in [1] which has created 6 members in a
> round-robin manner without any issue. You can check the log as with [2] in
> the integration test. I'm not sure whether any particular issue is there
> when we have min count as 2. It is very unlikely to happen as it is
> successful with 6 members.
>
> @Akila, can you attach the complete log?
>
> [1] https://builds.apache.org/job/Stratos-41x-Nightly-Build/40/consoleFull
>
> [2] [min-check] Partition available, hence trying to spawn an instance to
> fulfil minimum count! [cluster]
> partition-round-robin-test.my-c7.c7-partition-round-robin-test.domain
>
> Thanks,
> Reka
>
> On Fri, Oct 30, 2015 at 10:51 AM, Gayan Gunarathne <ga...@wso2.com>
> wrote:
>
>> Hi Reka,
>>
>> Is it not commented out in the Stratos 4.1.x branch[1]. That test under
>> the stratos.application.deployment group. We are running that test in the
>> build.
>>
>> [1]
>> https://github.com/apache/stratos/blob/stratos-4.1.x/products/stratos/modules/integration/test-integration/src/test/java/org/apache/stratos/integration/tests/application/PartitionRoundRobinClusterTestCase.java
>>
>> Thanks,
>> Gayan
>>
>> On Fri, Oct 30, 2015 at 10:35 AM, Reka Thirunavukkarasu <re...@wso2.com>
>> wrote:
>>
>>> Hi Akila,
>>>
>>> We had a partition round-robin integration test [1]. If that is
>>> successful, then round-robin algo is working fine with partitions.
>>> Unfortunately, the particular logic[2] which verifies the round-robin is
>>> commented out. Can you attach the complete log?
>>>
>>> [1]
>>> https://github.com/apache/stratos/blob/master/products/stratos/modules/integration/test-integration/src/test/java/org/apache/stratos/integration/tests/application/PartitionRoundRobinClusterTest.java
>>>
>>> [2]
>>> https://github.com/apache/stratos/blob/master/products/stratos/modules/integration/test-integration/src/test/java/org/apache/stratos/integration/tests/application/PartitionRoundRobinClusterTest.java#L112
>>>
>>> Thanks,
>>> Reka
>>>
>>> On Fri, Oct 30, 2015 at 12:17 AM, Akila Ravihansa Perera <
>>> ravihansa@wso2.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I noticed an unexpected behavior when deploying an application with
>>>> network partition having multiple partitions on EC2 platform.
>>>>
>>>> 1. My network partition as follows;
>>>>
>>>> {
>>>> "id": "network-partition-ec2",
>>>> "provider": "ec2",
>>>> "partitions": [
>>>> {
>>>> "id": "partition-1",
>>>> "property": [
>>>> {
>>>> "name": "region",
>>>> "value": "ap-southeast-1"
>>>> },
>>>> {
>>>> "name": "zone",
>>>> "value": "ap-southeast-1a"
>>>> }
>>>> ]
>>>> },
>>>> {
>>>> "id": "partition-2",
>>>> "property": [
>>>> {
>>>> "name": "region",
>>>> "value": "ap-southeast-1"
>>>> },
>>>> {
>>>> "name": "zone",
>>>> "value": "ap-southeast-1b"
>>>> }
>>>> ]
>>>> }
>>>> ]
>>>>
>>>> 2. My deployment policy as follows;
>>>> {
>>>> "id": "deployment-policy-ec2",
>>>> "networkPartitions": [
>>>> {
>>>> "id": "network-partition-ec2",
>>>> "partitionAlgo": "round-robin",
>>>> "partitions": [
>>>> {
>>>> "id": "partition-1",
>>>> "partitionMax": 15
>>>> },
>>>> {
>>>> "id": "partition-2",
>>>> "partitionMax": 15
>>>> }
>>>> ]
>>>> }
>>>> ]
>>>> }
>>>>
>>>> 3. My application policy as follows;
>>>> {
>>>> "id": "application-policy-ec2",
>>>> "algorithm": "one-after-another",
>>>> "networkPartitions": [
>>>> "network-partition-ec2"
>>>> ],
>>>> "properties": []
>>>> }
>>>>
>>>> 4. My application as follows;
>>>>
>>>> {
>>>> "applicationId": "wso2brs-210-application",
>>>> "alias": "wso2brs-210-application",
>>>> "multiTenant": true,
>>>> "components": {
>>>> "groups": [
>>>> {
>>>> "name": "wso2brs-210-group",
>>>> "alias": "wso2brs-210-group",
>>>> "deploymentPolicy": "deployment-policy-ec2",
>>>> "groupMinInstances": 1,
>>>> "groupMaxInstances": 1,
>>>> "cartridges": [
>>>> {
>>>> "type": "wso2brs-210-manager",
>>>> "cartridgeMin": 1,
>>>> "cartridgeMax": 1,
>>>> "subscribableInfo": {
>>>> "alias": "wso2brs-210-manager",
>>>> "autoscalingPolicy":
>>>> "autoscaling-policy-economy"
>>>> }
>>>> },
>>>> {
>>>> "type": "wso2brs-210-worker",
>>>> "cartridgeMin": 2,
>>>> "cartridgeMax": 2,
>>>> "subscribableInfo": {
>>>> "alias": "wso2brs-210-worker",
>>>> "autoscalingPolicy":
>>>> "autoscaling-policy-economy"
>>>> }
>>>> }
>>>> ]
>>>> }
>>>> ]
>>>> }
>>>> }
>>>>
>>>> Now the problem is when I deploy the application, Stratos always spins
>>>> instances in ap-southeast-1a (partition-1) zone. But according to the
>>>> deployment policy it should select the partition in a round-robin fashion.
>>>> Any idea why it behaves that way?
>>>>
>>>> It should at least spin 2 wso2brs-210-worker instances in partition-1
>>>> and partition-2. Am I correct to assume that?
>>>>
>>>> Thanks.
>>>>
>>>>
>>>> --
>>>> Akila Ravihansa Perera
>>>> WSO2 Inc.; http://wso2.com/
>>>>
>>>> Blog: http://ravihansa3000.blogspot.com
>>>>
>>>
>>>
>>>
>>> --
>>> Reka Thirunavukkarasu
>>> Senior Software Engineer,
>>> WSO2, Inc.:http://wso2.com,
>>> Mobile: +94776442007
>>>
>>>
>>>
>>
>>
>> --
>>
>> Gayan Gunarathne
>> Technical Lead, WSO2 Inc. (http://wso2.com)
>> Committer & PMC Member, Apache Stratos
>> email : gayang@wso2.com | mobile : +94 775030545 <%2B94%20766819985>
>>
>>
>>
>
>
>
> --
> Reka Thirunavukkarasu
> Senior Software Engineer,
> WSO2, Inc.:http://wso2.com,
> Mobile: +94776442007
>
>
>
--
Akila Ravihansa Perera
WSO2 Inc.; http://wso2.com/
Blog: http://ravihansa3000.blogspot.com
Re: Stratos does not select the correct partition in a multi
partition deployment
Posted by Reka Thirunavukkarasu <re...@wso2.com>.
Hi Akila,
I believe, you are using group level deployment policy. In that case,
partition is chosen for a particular group instance which will have its own
cluster instances and members where all of them belongs to the same
partition of the parent group instance. This is expected behavior when you
are defined the deployment policy at the group level. According to the
deployment, we have to decide which level the deployment policy should
engaged to. If it cluster level, then individual members will be created
based on the algorithm. If it is group level, then individual group
instance will be created based on the algorithm.
Thanks,
Reka
On Sun, Nov 1, 2015 at 12:21 PM, Akila Ravihansa Perera <ra...@wso2.com>
wrote:
> Hi Reka,
>
> Please find the complete log attached herewith. I've noticed that this
> happens when there are cartridge groups defined.
> While debugging the code I observed that ClusterInstanceContext ->
> partitionCtxts [1] map only contains the partition-1.
>
>
> [1]
> https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/context/cluster/ClusterInstanceContext.java#L47
>
> Thanks.
>
> On Fri, Oct 30, 2015 at 11:07 AM, Reka Thirunavukkarasu <re...@wso2.com>
> wrote:
>
>> Thanks for the confirmation Gayan. I have checked it in master. I have
>> verified the intergration test logs in [1] which has created 6 members in a
>> round-robin manner without any issue. You can check the log as with [2] in
>> the integration test. I'm not sure whether any particular issue is there
>> when we have min count as 2. It is very unlikely to happen as it is
>> successful with 6 members.
>>
>> @Akila, can you attach the complete log?
>>
>> [1]
>> https://builds.apache.org/job/Stratos-41x-Nightly-Build/40/consoleFull
>>
>> [2] [min-check] Partition available, hence trying to spawn an instance to
>> fulfil minimum count! [cluster]
>> partition-round-robin-test.my-c7.c7-partition-round-robin-test.domain
>>
>> Thanks,
>> Reka
>>
>> On Fri, Oct 30, 2015 at 10:51 AM, Gayan Gunarathne <ga...@wso2.com>
>> wrote:
>>
>>> Hi Reka,
>>>
>>> Is it not commented out in the Stratos 4.1.x branch[1]. That test under
>>> the stratos.application.deployment group. We are running that test in the
>>> build.
>>>
>>> [1]
>>> https://github.com/apache/stratos/blob/stratos-4.1.x/products/stratos/modules/integration/test-integration/src/test/java/org/apache/stratos/integration/tests/application/PartitionRoundRobinClusterTestCase.java
>>>
>>> Thanks,
>>> Gayan
>>>
>>> On Fri, Oct 30, 2015 at 10:35 AM, Reka Thirunavukkarasu <re...@wso2.com>
>>> wrote:
>>>
>>>> Hi Akila,
>>>>
>>>> We had a partition round-robin integration test [1]. If that is
>>>> successful, then round-robin algo is working fine with partitions.
>>>> Unfortunately, the particular logic[2] which verifies the round-robin is
>>>> commented out. Can you attach the complete log?
>>>>
>>>> [1]
>>>> https://github.com/apache/stratos/blob/master/products/stratos/modules/integration/test-integration/src/test/java/org/apache/stratos/integration/tests/application/PartitionRoundRobinClusterTest.java
>>>>
>>>> [2]
>>>> https://github.com/apache/stratos/blob/master/products/stratos/modules/integration/test-integration/src/test/java/org/apache/stratos/integration/tests/application/PartitionRoundRobinClusterTest.java#L112
>>>>
>>>> Thanks,
>>>> Reka
>>>>
>>>> On Fri, Oct 30, 2015 at 12:17 AM, Akila Ravihansa Perera <
>>>> ravihansa@wso2.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I noticed an unexpected behavior when deploying an application with
>>>>> network partition having multiple partitions on EC2 platform.
>>>>>
>>>>> 1. My network partition as follows;
>>>>>
>>>>> {
>>>>> "id": "network-partition-ec2",
>>>>> "provider": "ec2",
>>>>> "partitions": [
>>>>> {
>>>>> "id": "partition-1",
>>>>> "property": [
>>>>> {
>>>>> "name": "region",
>>>>> "value": "ap-southeast-1"
>>>>> },
>>>>> {
>>>>> "name": "zone",
>>>>> "value": "ap-southeast-1a"
>>>>> }
>>>>> ]
>>>>> },
>>>>> {
>>>>> "id": "partition-2",
>>>>> "property": [
>>>>> {
>>>>> "name": "region",
>>>>> "value": "ap-southeast-1"
>>>>> },
>>>>> {
>>>>> "name": "zone",
>>>>> "value": "ap-southeast-1b"
>>>>> }
>>>>> ]
>>>>> }
>>>>> ]
>>>>>
>>>>> 2. My deployment policy as follows;
>>>>> {
>>>>> "id": "deployment-policy-ec2",
>>>>> "networkPartitions": [
>>>>> {
>>>>> "id": "network-partition-ec2",
>>>>> "partitionAlgo": "round-robin",
>>>>> "partitions": [
>>>>> {
>>>>> "id": "partition-1",
>>>>> "partitionMax": 15
>>>>> },
>>>>> {
>>>>> "id": "partition-2",
>>>>> "partitionMax": 15
>>>>> }
>>>>> ]
>>>>> }
>>>>> ]
>>>>> }
>>>>>
>>>>> 3. My application policy as follows;
>>>>> {
>>>>> "id": "application-policy-ec2",
>>>>> "algorithm": "one-after-another",
>>>>> "networkPartitions": [
>>>>> "network-partition-ec2"
>>>>> ],
>>>>> "properties": []
>>>>> }
>>>>>
>>>>> 4. My application as follows;
>>>>>
>>>>> {
>>>>> "applicationId": "wso2brs-210-application",
>>>>> "alias": "wso2brs-210-application",
>>>>> "multiTenant": true,
>>>>> "components": {
>>>>> "groups": [
>>>>> {
>>>>> "name": "wso2brs-210-group",
>>>>> "alias": "wso2brs-210-group",
>>>>> "deploymentPolicy": "deployment-policy-ec2",
>>>>> "groupMinInstances": 1,
>>>>> "groupMaxInstances": 1,
>>>>> "cartridges": [
>>>>> {
>>>>> "type": "wso2brs-210-manager",
>>>>> "cartridgeMin": 1,
>>>>> "cartridgeMax": 1,
>>>>> "subscribableInfo": {
>>>>> "alias": "wso2brs-210-manager",
>>>>> "autoscalingPolicy":
>>>>> "autoscaling-policy-economy"
>>>>> }
>>>>> },
>>>>> {
>>>>> "type": "wso2brs-210-worker",
>>>>> "cartridgeMin": 2,
>>>>> "cartridgeMax": 2,
>>>>> "subscribableInfo": {
>>>>> "alias": "wso2brs-210-worker",
>>>>> "autoscalingPolicy":
>>>>> "autoscaling-policy-economy"
>>>>> }
>>>>> }
>>>>> ]
>>>>> }
>>>>> ]
>>>>> }
>>>>> }
>>>>>
>>>>> Now the problem is when I deploy the application, Stratos always spins
>>>>> instances in ap-southeast-1a (partition-1) zone. But according to the
>>>>> deployment policy it should select the partition in a round-robin fashion.
>>>>> Any idea why it behaves that way?
>>>>>
>>>>> It should at least spin 2 wso2brs-210-worker instances in partition-1
>>>>> and partition-2. Am I correct to assume that?
>>>>>
>>>>> Thanks.
>>>>>
>>>>>
>>>>> --
>>>>> Akila Ravihansa Perera
>>>>> WSO2 Inc.; http://wso2.com/
>>>>>
>>>>> Blog: http://ravihansa3000.blogspot.com
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Reka Thirunavukkarasu
>>>> Senior Software Engineer,
>>>> WSO2, Inc.:http://wso2.com,
>>>> Mobile: +94776442007
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> Gayan Gunarathne
>>> Technical Lead, WSO2 Inc. (http://wso2.com)
>>> Committer & PMC Member, Apache Stratos
>>> email : gayang@wso2.com | mobile : +94 775030545 <%2B94%20766819985>
>>>
>>>
>>>
>>
>>
>>
>> --
>> Reka Thirunavukkarasu
>> Senior Software Engineer,
>> WSO2, Inc.:http://wso2.com,
>> Mobile: +94776442007
>>
>>
>>
>
>
> --
> Akila Ravihansa Perera
> WSO2 Inc.; http://wso2.com/
>
> Blog: http://ravihansa3000.blogspot.com
>
--
Reka Thirunavukkarasu
Senior Software Engineer,
WSO2, Inc.:http://wso2.com,
Mobile: +94776442007