You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stratos.apache.org by chris snow <ch...@gmail.com> on 2014/05/27 08:35:51 UTC
autoscaling is ignoring partiionMax
Deployment Policy partitionMax seems to be ignored. My deployment
policy looks like this:
{
"id": "economyDeployment",
"partitionGroup": [
{
"id": "G1",
"partitionAlgo": "one-after-another",
"partition": [
{
"id": "P1",
"partitionMax": "1",
"partitionMin": "1"
}
]
}
]
}
Stratos initially starts two instances:
+--------------------------------------+---------------+--------+------------+-------------+--------------------+
| ID | Name | Status | Task
State | Power State | Networks |
+--------------------------------------+---------------+--------+------------+-------------+--------------------+
| c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 | ACTIVE | -
| Running | private=10.11.12.2 |
| b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 | ACTIVE | -
| Running | private=10.11.12.3 |
+--------------------------------------+---------------+--------+------------+-------------+--------------------+
Then attempts to start more:
+--------------------------------------+---------------+--------+------------+-------------+--------------------+
| ID | Name | Status | Task
State | Power State | Networks |
+--------------------------------------+---------------+--------+------------+-------------+--------------------+
| 91029130-37b5-4df6-ad95-1d9f609a22c3 | php111php-13e | ERROR | -
| NOSTATE | |
| 71c1e361-df55-4f61-84c5-79774fef2beb | php111php-3d7 | ERROR | -
| NOSTATE | |
| b0d72602-cdb8-4621-ad12-911ade4d89b1 | php111php-412 | ERROR | -
| NOSTATE | |
| d4eac55d-bb7c-48d6-83f2-cb9a11581a18 | php111php-6f5 | ERROR | -
| NOSTATE | |
| 738a9534-2b95-4984-a51b-748bc7169b9a | php111php-73e | ERROR | -
| NOSTATE | |
| c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 | ACTIVE | -
| Running | private=10.11.12.2 |
| b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 | ACTIVE | -
| Running | private=10.11.12.3 |
| f570e654-024b-4240-bad2-8a11f04f23f0 | php111php-a30 | ERROR | -
| NOSTATE | |
| 0ab36824-9bea-4a95-aa22-7f935406d83d | php111php-bca | ERROR | -
| NOSTATE | |
| 02ed97df-02bc-4bff-b42d-9f2bcbd4f569 | php111php-f6f | ERROR | -
| NOSTATE | |
+--------------------------------------+---------------+--------+------------+-------------+--------------------+
I'm on branch: 4.0.0-incubating, using Openstack Havana + Docker.
Any ideas?
Many thanks,
Chris
Re: autoscaling is ignoring partiionMax
Posted by chris snow <ch...@gmail.com>.
Ah, I've just realised that the agent hasn't started in the cartridge.
On Thu, May 29, 2014 at 7:45 AM, chris snow <ch...@gmail.com> wrote:
> Thanks Nirmal.
>
> Non terminated member count doesn't seem to detect the two active
> instances that have been started:
>
> [2014-05-29 06:06:23,900] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} - Running minimum rule:
> [network-partition] G1 [partition] P1
> [2014-05-29 06:06:23,906] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check]
> [network-partition] G1 [partition] P1 Non terminated member count: 0
> [2014-05-29 06:06:23,908] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check]
> [network-partition] G1 [partition] P1 Minimum member count: 1
> [2014-05-29 06:06:23,925] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} - Running obsolete
> instances rule: [partition] P1 [network-partition] G1
> [2014-05-29 06:06:23,927] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} - [obsolete-check]
> [network-partition] G1 [partition] P1 Obsolete member count: 0
> [2014-05-29 06:06:23,930] INFO
> {org.apache.stratos.autoscaler.client.cloud.controller.CloudControllerClient}
> - Trying to spawn an instance via cloud controller: [cluster]
> php111.php.domain [partition] P1 [lb-cluster] null
> [network-partition-id] G1
>
>
> vagrant@puppet:~$ nova list
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> | ID | Name | Status | Task
> State | Power State | Networks |
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> | a3ce6f52-27bf-4343-a05d-d7c9da9a1cb2 | php111php-3cb | ERROR | -
> | NOSTATE | |
> | 82434b60-6c9d-40a3-8191-2d466cde725e | php111php-47e | ERROR | -
> | NOSTATE | |
> | 4aa942ff-a722-4ff4-b604-abaa2f44c1a7 | php111php-499 | ERROR | -
> | NOSTATE | |
> | 1548915b-98f6-48fc-b70a-b0acc74610a7 | php111php-5e0 | ERROR | -
> | NOSTATE | |
> | f313c93d-7de1-4ccb-8c0d-f4104e5d837f | php111php-701 | ERROR | -
> | NOSTATE | |
> | a49598bb-0d4a-4064-af16-5b4b91057fd8 | php111php-719 | ERROR | -
> | NOSTATE | |
> | f8376091-99f6-4d17-8ec6-3935291c3e40 | php111php-841 | ERROR | -
> | NOSTATE | |
> | cf3298da-eecc-448a-bcea-2296eeed23dc | php111php-ad | ACTIVE | -
> | Running | private=10.11.12.2 |
> | 1d8832d5-39e4-476c-9c2b-1b1f5ffe3b41 | php111php-d81 | ACTIVE | -
> | Running | private=10.11.12.3 |
> | 4f6d51fb-fd27-4be1-94be-e9b5200fb6ba | php111php-e9c | ERROR | -
> | NOSTATE | |
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>
> I'll investigate some more...
>
> On Thu, May 29, 2014 at 7:42 AM, Nirmal Fernando <ni...@gmail.com> wrote:
>> Ah.. probably Chris. So, in min check what happens is it always make sure
>> that you have min instance count in the system.
>>
>>
>> On Thu, May 29, 2014 at 11:42 AM, chris snow <ch...@gmail.com> wrote:
>>>
>>> Hi Nirmal,
>>>
>>> I've just see that it is the "Minimal Rule" in mincheck.drl that is
>>> trying to spawn the instance, and not scaling.drl.
>>>
>>> This is why AutoscaleAlgorithm.getNextScaleUpPartition() is not getting
>>> called?
>>>
>>> Cheers,
>>>
>>> Chris
>>>
>>>
>>> On Wed, May 28, 2014 at 11:24 PM, Nirmal Fernando
>>> <ni...@gmail.com> wrote:
>>> > No, Chris. This happens via Drools rules located in scaling.drl file of
>>> > conf
>>> > directory.
>>> >
>>> >
>>> > On Thu, May 29, 2014 at 3:39 AM, chris snow <ch...@gmail.com> wrote:
>>> >>
>>> >> Hi Nirmal,
>>> >>
>>> >> For some reason, that method is not getting called. I've pasted my
>>> >> stacktrace again with the drools methods removed:
>>> >>
>>> >> ---
>>> >> Daemon Thread [Thread-56] (Suspended (breakpoint at line 137 in
>>> >> CloudControllerClient))
>>> >> owns: CloudControllerClient (id=2433)
>>> >> owns: DefaultAgenda (id=2434)
>>> >> CloudControllerClient.spawnAnInstance(Partition, String, String,
>>> >> String) line: 137
>>> >> RuleTasksDelegator.delegateSpawn(PartitionContext, String, String)
>>> >> line: 92
>>> >> //
>>> >> // drools methods hidden
>>> >> //
>>> >> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
>>> >> FactHandle, Object) line: 86
>>> >> ClusterMonitor.monitor() line: 99
>>> >> ClusterMonitor.run() line: 72
>>> >> Thread.run() line: 744
>>> >> ---
>>> >>
>>> >> Would you expect to see AutoscaleAlgorithm.getNextScaleUpPartition
>>> >> executed somewhere in the stack of calls?
>>> >>
>>> >> Many thanks,
>>> >>
>>> >> Chris
>>> >>
>>> >> On Wed, May 28, 2014 at 11:03 PM, Nirmal Fernando
>>> >> <ni...@gmail.com> wrote:
>>> >> > Hi Chris,
>>> >> >
>>> >> > PartitionMax is considered in
>>> >> > AutoscaleAlgorithm.getNextScaleUpPartition
>>> >> > method.
>>> >> >
>>> >> >
>>> >> > On Thu, May 29, 2014 at 3:09 AM, chris snow <ch...@gmail.com>
>>> >> > wrote:
>>> >> >>
>>> >> >> I setup a breakpoint in eclipse at
>>> >> >> CloudControllerClient.spawnAnInstance() and dumped the output
>>> >> >> (below).
>>> >> >>
>>> >> >> I've scanned through the code in the stratos methods in the
>>> >> >> stacktrace, but couldn't see any code that checks the partitionMax
>>> >> >> value.
>>> >> >>
>>> >> >> Any ideas?
>>> >> >>
>>> >> >> Daemon Thread [Thread-56] (Suspended (breakpoint at line 137 in
>>> >> >> CloudControllerClient))
>>> >> >> owns: CloudControllerClient (id=2433)
>>> >> >> owns: DefaultAgenda (id=2434)
>>> >> >> CloudControllerClient.spawnAnInstance(Partition, String, String,
>>> >> >> String) line: 137
>>> >> >> RuleTasksDelegator.delegateSpawn(PartitionContext, String,
>>> >> >> String)
>>> >> >> line: 92
>>> >> >> NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line:
>>> >> >> not available [native method]
>>> >> >> NativeMethodAccessorImpl.invoke(Object, Object[]) line: 57
>>> >> >> DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
>>> >> >> Method.invoke(Object, Object...) line: 606
>>> >> >> ReflectiveAccessorOptimizer.getMethod(Object, String, Object[],
>>> >> >> Class[], ExecutableStatement[]) line: 1104
>>> >> >> ReflectiveAccessorOptimizer.getMethod(Object, String) line: 987
>>> >> >> ReflectiveAccessorOptimizer.compileGetChain() line: 377
>>> >> >> ReflectiveAccessorOptimizer.optimizeAccessor(ParserContext,
>>> >> >> char[],
>>> >> >> int, int, Object, Object, VariableResolverFactory, boolean, Class)
>>> >> >> line: 143
>>> >> >> ASTNode.optimize(Object, Object, VariableResolverFactory) line:
>>> >> >> 159
>>> >> >> ASTNode.getReducedValueAccelerated(Object, Object,
>>> >> >> VariableResolverFactory) line: 115
>>> >> >> MVELRuntime.execute(boolean, CompiledExpression, Object,
>>> >> >> VariableResolverFactory) line: 85
>>> >> >> CompiledExpression.getDirectValue(Object,
>>> >> >> VariableResolverFactory)
>>> >> >> line: 123
>>> >> >> CompiledExpression.getValue(Object, VariableResolverFactory)
>>> >> >> line:
>>> >> >> 119
>>> >> >> MVEL.executeExpression(Object, Object, VariableResolverFactory)
>>> >> >> line:
>>> >> >> 930
>>> >> >> MVELConsequence.evaluate(KnowledgeHelper, WorkingMemory) line:
>>> >> >> 104
>>> >> >> DefaultAgenda.fireActivation(Activation) line: 1287
>>> >> >> DefaultAgenda.fireNextItem(AgendaFilter) line: 1221
>>> >> >> DefaultAgenda.fireAllRules(AgendaFilter, int) line: 1456
>>> >> >>
>>> >> >>
>>> >> >> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules(AgendaFilter,
>>> >> >> int) line: 710
>>> >> >> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules() line:
>>> >> >> 674
>>> >> >> StatefulKnowledgeSessionImpl.fireAllRules() line: 230
>>> >> >>
>>> >> >> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
>>> >> >> FactHandle, Object) line: 86
>>> >> >> ClusterMonitor.monitor() line: 99
>>> >> >> ClusterMonitor.run() line: 72
>>> >> >> Thread.run() line: 744
>>> >> >>
>>> >> >> On Wed, May 28, 2014 at 3:37 AM, Lahiru Sandaruwan
>>> >> >> <la...@wso2.com>
>>> >> >> wrote:
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> > On Tue, May 27, 2014 at 11:54 PM, chris snow <ch...@gmail.com>
>>> >> >> > wrote:
>>> >> >> >>
>>> >> >> >> Hi Lahiru,
>>> >> >> >>
>>> >> >> >> I'll have a poke around - it will give me a good opportunity to
>>> >> >> >> learn
>>> >> >> >> some more Stratos code :)
>>> >> >> >>
>>> >> >> >
>>> >> >> > Cool :)
>>> >> >> >
>>> >> >> > Debug logs should be able to show us where it went wrong.
>>> >> >> >
>>> >> >> > Thanks.
>>> >> >> >>
>>> >> >> >> Many thanks,
>>> >> >> >>
>>> >> >> >> Chris
>>> >> >> >>
>>> >> >> >> On Tue, May 27, 2014 at 7:45 AM, Lahiru Sandaruwan
>>> >> >> >> <la...@wso2.com>
>>> >> >> >> wrote:
>>> >> >> >> > Hi Chris,
>>> >> >> >> >
>>> >> >> >> > Can you enable debug logs and try out?
>>> >> >> >> > If so please send them over.
>>> >> >> >> >
>>> >> >> >> > Thanks.
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> > On Tue, May 27, 2014 at 12:05 PM, chris snow
>>> >> >> >> > <ch...@gmail.com>
>>> >> >> >> > wrote:
>>> >> >> >> >>
>>> >> >> >> >> Deployment Policy partitionMax seems to be ignored. My
>>> >> >> >> >> deployment
>>> >> >> >> >> policy looks like this:
>>> >> >> >> >>
>>> >> >> >> >> {
>>> >> >> >> >> "id": "economyDeployment",
>>> >> >> >> >> "partitionGroup": [
>>> >> >> >> >> {
>>> >> >> >> >> "id": "G1",
>>> >> >> >> >> "partitionAlgo": "one-after-another",
>>> >> >> >> >> "partition": [
>>> >> >> >> >> {
>>> >> >> >> >> "id": "P1",
>>> >> >> >> >> "partitionMax": "1",
>>> >> >> >> >> "partitionMin": "1"
>>> >> >> >> >> }
>>> >> >> >> >> ]
>>> >> >> >> >> }
>>> >> >> >> >> ]
>>> >> >> >> >> }
>>> >> >> >> >>
>>> >> >> >> >> Stratos initially starts two instances:
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>>> >> >> >> >> | ID | Name |
>>> >> >> >> >> Status |
>>> >> >> >> >> Task
>>> >> >> >> >> State | Power State | Networks |
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>>> >> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 |
>>> >> >> >> >> ACTIVE |
>>> >> >> >> >> -
>>> >> >> >> >> | Running | private=10.11.12.2 |
>>> >> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 |
>>> >> >> >> >> ACTIVE |
>>> >> >> >> >> -
>>> >> >> >> >> | Running | private=10.11.12.3 |
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>>> >> >> >> >> Then attempts to start more:
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>>> >> >> >> >> | ID | Name |
>>> >> >> >> >> Status |
>>> >> >> >> >> Task
>>> >> >> >> >> State | Power State | Networks |
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>>> >> >> >> >> | 91029130-37b5-4df6-ad95-1d9f609a22c3 | php111php-13e | ERROR
>>> >> >> >> >> |
>>> >> >> >> >> -
>>> >> >> >> >> | NOSTATE | |
>>> >> >> >> >> | 71c1e361-df55-4f61-84c5-79774fef2beb | php111php-3d7 | ERROR
>>> >> >> >> >> |
>>> >> >> >> >> -
>>> >> >> >> >> | NOSTATE | |
>>> >> >> >> >> | b0d72602-cdb8-4621-ad12-911ade4d89b1 | php111php-412 | ERROR
>>> >> >> >> >> |
>>> >> >> >> >> -
>>> >> >> >> >> | NOSTATE | |
>>> >> >> >> >> | d4eac55d-bb7c-48d6-83f2-cb9a11581a18 | php111php-6f5 | ERROR
>>> >> >> >> >> |
>>> >> >> >> >> -
>>> >> >> >> >> | NOSTATE | |
>>> >> >> >> >> | 738a9534-2b95-4984-a51b-748bc7169b9a | php111php-73e | ERROR
>>> >> >> >> >> |
>>> >> >> >> >> -
>>> >> >> >> >> | NOSTATE | |
>>> >> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 |
>>> >> >> >> >> ACTIVE |
>>> >> >> >> >> -
>>> >> >> >> >> | Running | private=10.11.12.2 |
>>> >> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 |
>>> >> >> >> >> ACTIVE |
>>> >> >> >> >> -
>>> >> >> >> >> | Running | private=10.11.12.3 |
>>> >> >> >> >> | f570e654-024b-4240-bad2-8a11f04f23f0 | php111php-a30 | ERROR
>>> >> >> >> >> |
>>> >> >> >> >> -
>>> >> >> >> >> | NOSTATE | |
>>> >> >> >> >> | 0ab36824-9bea-4a95-aa22-7f935406d83d | php111php-bca | ERROR
>>> >> >> >> >> |
>>> >> >> >> >> -
>>> >> >> >> >> | NOSTATE | |
>>> >> >> >> >> | 02ed97df-02bc-4bff-b42d-9f2bcbd4f569 | php111php-f6f | ERROR
>>> >> >> >> >> |
>>> >> >> >> >> -
>>> >> >> >> >> | NOSTATE | |
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>>> >> >> >> >>
>>> >> >> >> >> I'm on branch: 4.0.0-incubating, using Openstack Havana +
>>> >> >> >> >> Docker.
>>> >> >> >> >>
>>> >> >> >> >> Any ideas?
>>> >> >> >> >>
>>> >> >> >> >> Many thanks,
>>> >> >> >> >>
>>> >> >> >> >> Chris
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> > --
>>> >> >> >> > --
>>> >> >> >> > Lahiru Sandaruwan
>>> >> >> >> > Committer and PMC member, Apache Stratos,
>>> >> >> >> > Senior Software Engineer,
>>> >> >> >> > WSO2 Inc., http://wso2.com
>>> >> >> >> > lean.enterprise.middleware
>>> >> >> >> >
>>> >> >> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
>>> >> >> >> > blog: http://lahiruwrites.blogspot.com/
>>> >> >> >> > twitter: http://twitter.com/lahirus
>>> >> >> >> > linked-in:
>>> >> >> >> > http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>> >> >> >> >
>>> >> >> >>
>>> >> >> >>
>>> >> >> >>
>>> >> >> >> --
>>> >> >> >> Check out my professional profile and connect with me on
>>> >> >> >> LinkedIn.
>>> >> >> >> http://lnkd.in/cw5k69
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> > --
>>> >> >> > --
>>> >> >> > Lahiru Sandaruwan
>>> >> >> > Committer and PMC member, Apache Stratos,
>>> >> >> > Senior Software Engineer,
>>> >> >> > WSO2 Inc., http://wso2.com
>>> >> >> > lean.enterprise.middleware
>>> >> >> >
>>> >> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
>>> >> >> > blog: http://lahiruwrites.blogspot.com/
>>> >> >> > twitter: http://twitter.com/lahirus
>>> >> >> > linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>> >> >> >
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> --
>>> >> >> Check out my professional profile and connect with me on LinkedIn.
>>> >> >> http://lnkd.in/cw5k69
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> > --
>>> >> > Best Regards,
>>> >> > Nirmal
>>> >> >
>>> >> > Nirmal Fernando.
>>> >> > PPMC Member & Committer of Apache Stratos,
>>> >> >
>>> >> > Senior Software Engineer, WSO2 Inc.
>>> >> >
>>> >> > Blog: http://nirmalfdo.blogspot.com/
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Check out my professional profile and connect with me on LinkedIn.
>>> >> http://lnkd.in/cw5k69
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > Best Regards,
>>> > Nirmal
>>> >
>>> > Nirmal Fernando.
>>> > PPMC Member & Committer of Apache Stratos,
>>> > Senior Software Engineer, WSO2 Inc.
>>> >
>>> > Blog: http://nirmalfdo.blogspot.com/
>>>
>>>
>>>
>>> --
>>> Check out my professional profile and connect with me on LinkedIn.
>>> http://lnkd.in/cw5k69
>>
>>
>>
>>
>> --
>> Best Regards,
>> Nirmal
>>
>> Nirmal Fernando.
>> PPMC Member & Committer of Apache Stratos,
>> Senior Software Engineer, WSO2 Inc.
>>
>> Blog: http://nirmalfdo.blogspot.com/
>
>
>
> --
> Check out my professional profile and connect with me on LinkedIn.
> http://lnkd.in/cw5k69
--
Check out my professional profile and connect with me on LinkedIn.
http://lnkd.in/cw5k69
Re: autoscaling is ignoring partiionMax
Posted by Lahiru Sandaruwan <la...@wso2.com>.
Can you post the complete log file. After spawning a new instance,
Autoscaler adds it to a pending instances list. If the member activated
event received, autoscaler will move that to activated instances list.
Pending instances list will be evaluated periodically and terminate after a
time limit(15 mins AFAIR).
Let's see logs about pending instances.
Thanks.
On Thu, May 29, 2014 at 11:04 PM, chris snow <ch...@gmail.com> wrote:
> Hi Lahiru,
>
> I've added a JIRA for a configurable retry limit [1].
>
> Stratos is not terminating the instances:
>
> Based on this deployment policy ...
>
> {
> "id": "economyDeployment",
> "partitionGroup": [
> {
> "id": "G1",
> "partitionAlgo": "one-after-another",
> "partition": [
> {
> "id": "P1",
> "partitionMax": "1",
> "partitionMin": "1"
> }
> ]
> }
> ]
> }
>
> ... Stratos initially starts two instances ...
>
>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> | ID | Name | Status | Task
> State | Power State | Networks |
>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 | ACTIVE | -
> | Running | private=10.11.12.2 |
> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 | ACTIVE | -
> | Running | private=10.11.12.3 |
>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>
> ... then attempts to start more:
>
>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> | ID | Name | Status | Task
> State | Power State | Networks |
>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> | 91029130-37b5-4df6-ad95-1d9f609a22c3 | php111php-13e | ERROR | -
> | NOSTATE | |
> | 71c1e361-df55-4f61-84c5-79774fef2beb | php111php-3d7 | ERROR | -
> | NOSTATE | |
> | b0d72602-cdb8-4621-ad12-911ade4d89b1 | php111php-412 | ERROR | -
> | NOSTATE | |
> | d4eac55d-bb7c-48d6-83f2-cb9a11581a18 | php111php-6f5 | ERROR | -
> | NOSTATE | |
> | 738a9534-2b95-4984-a51b-748bc7169b9a | php111php-73e | ERROR | -
> | NOSTATE | |
> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 | ACTIVE | -
> | Running | private=10.11.12.2 |
> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 | ACTIVE | -
> | Running | private=10.11.12.3 |
> | f570e654-024b-4240-bad2-8a11f04f23f0 | php111php-a30 | ERROR | -
> | NOSTATE | |
> | 0ab36824-9bea-4a95-aa22-7f935406d83d | php111php-bca | ERROR | -
> | NOSTATE | |
> | 02ed97df-02bc-4bff-b42d-9f2bcbd4f569 | php111php-f6f | ERROR | -
> | NOSTATE | |
>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>
> I'll have a dig around to see why it is not terminating the instances.
>
> ---
> [1] https://issues.apache.org/jira/browse/STRATOS-658
>
> On Thu, May 29, 2014 at 9:25 AM, Lahiru Sandaruwan <la...@wso2.com>
> wrote:
> >
> > Hi Chris,
> >
> > On Thu, May 29, 2014 at 1:50 PM, chris snow <ch...@gmail.com> wrote:
> >>
> >> Ok, thanks all. My agent definitely has not started, so there is the
> >> source of the problem. Thanks for the help getting to the bottom of
> >> this.
> >>
> >> A few thoughts on this:
> >>
> >> 1) Conceptually, I was thinking that partitionMax was equal to the
> >> number of IaaS instances "spawned", whereas it really means number of
> >> "instances" connected to Stratos. Maybe we should add that note to
> >> the wiki [1]?
> >> 2) Anyone else running into this problem on EC2 or another
> >> subscription based IaaS could end up with a big bill because Stratos
> >> will just keep spawning and spawning.
> >> 3) Should we put in a limit on the number of instances that Stratos
> >> spawns that are not connected?
> >
> >
> > Autoscaler will wait some time for the spawned instance's member
> activated
> > event and terminate if time limit is reached.
> > Didn't it happen in your case?
> >
> > So this is like retrying to get a working instance.
> > +1 for a limit of such re-tries.
> >
> > Thanks.
> >
> >
> >>
> >> ---
> >> [1]
> >>
> https://cwiki.apache.org/confluence/display/STRATOS/4.0.0+Sample+Deployment+Policy+Definition
> >>
> >> On Thu, May 29, 2014 at 9:06 AM, Lakmal Warusawithana <la...@wso2.com>
> >> wrote:
> >> > Actually its sending by CC based on "Insance activated" event at
> >> > Instance
> >> > Status topic which publish by Cartridge agent.
> >> >
> >> >
> >> > On Thu, May 29, 2014 at 1:31 PM, Nirmal Fernando
> >> > <ni...@gmail.com>
> >> > wrote:
> >> >>
> >> >> It's agent Chris.
> >> >>
> >> >>
> >> >> On Thu, May 29, 2014 at 1:25 PM, chris snow <ch...@gmail.com>
> >> >> wrote:
> >> >>>
> >> >>> Hi Lahiru, are MemberActivatedEvents sent by the Stratos agent, or
> are
> >> >>> they sent by the another component (e.g. CloudController)?
> >> >>>
> >> >>> On Thu, May 29, 2014 at 7:49 AM, Lahiru Sandaruwan <
> lahirus@wso2.com>
> >> >>> wrote:
> >> >>> > Hi Chris,
> >> >>> >
> >> >>> > Can you confirm that the Autoscaler received Member activated
> event
> >> >>> > for
> >> >>> > these particular instances? Logs should help identify that.
> >> >>> >
> >> >>> > Thanks.
> >> >>> >
> >> >>> >
> >> >>> >
> >> >>> >
> >> >>> > On Thu, May 29, 2014 at 12:15 PM, chris snow <chsnow123@gmail.com
> >
> >> >>> > wrote:
> >> >>> >>
> >> >>> >> Thanks Nirmal.
> >> >>> >>
> >> >>> >> Non terminated member count doesn't seem to detect the two active
> >> >>> >> instances that have been started:
> >> >>> >>
> >> >>> >> [2014-05-29 06:06:23,900] DEBUG
> >> >>> >> {org.apache.stratos.autoscaler.rule.RuleLog} - Running minimum
> >> >>> >> rule:
> >> >>> >> [network-partition] G1 [partition] P1
> >> >>> >> [2014-05-29 06:06:23,906] DEBUG
> >> >>> >> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check]
> >> >>> >> [network-partition] G1 [partition] P1 Non terminated member
> count:
> >> >>> >> 0
> >> >>> >> [2014-05-29 06:06:23,908] DEBUG
> >> >>> >> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check]
> >> >>> >> [network-partition] G1 [partition] P1 Minimum member count: 1
> >> >>> >> [2014-05-29 06:06:23,925] DEBUG
> >> >>> >> {org.apache.stratos.autoscaler.rule.RuleLog} - Running obsolete
> >> >>> >> instances rule: [partition] P1 [network-partition] G1
> >> >>> >> [2014-05-29 06:06:23,927] DEBUG
> >> >>> >> {org.apache.stratos.autoscaler.rule.RuleLog} - [obsolete-check]
> >> >>> >> [network-partition] G1 [partition] P1 Obsolete member count: 0
> >> >>> >> [2014-05-29 06:06:23,930] INFO
> >> >>> >>
> >> >>> >>
> >> >>> >>
> >> >>> >>
> {org.apache.stratos.autoscaler.client.cloud.controller.CloudControllerClient}
> >> >>> >> - Trying to spawn an instance via cloud controller: [cluster]
> >> >>> >> php111.php.domain [partition] P1 [lb-cluster] null
> >> >>> >> [network-partition-id] G1
> >> >>> >>
> >> >>> >>
> >> >>> >> vagrant@puppet:~$ nova list
> >> >>> >>
> >> >>> >>
> >> >>> >>
> >> >>> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >>> >> | ID | Name | Status |
> >> >>> >> Task
> >> >>> >> State | Power State | Networks |
> >> >>> >>
> >> >>> >>
> >> >>> >>
> >> >>> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >>> >> | a3ce6f52-27bf-4343-a05d-d7c9da9a1cb2 | php111php-3cb | ERROR
> | -
> >> >>> >> | NOSTATE | |
> >> >>> >> | 82434b60-6c9d-40a3-8191-2d466cde725e | php111php-47e | ERROR
> | -
> >> >>> >> | NOSTATE | |
> >> >>> >> | 4aa942ff-a722-4ff4-b604-abaa2f44c1a7 | php111php-499 | ERROR
> | -
> >> >>> >> | NOSTATE | |
> >> >>> >> | 1548915b-98f6-48fc-b70a-b0acc74610a7 | php111php-5e0 | ERROR
> | -
> >> >>> >> | NOSTATE | |
> >> >>> >> | f313c93d-7de1-4ccb-8c0d-f4104e5d837f | php111php-701 | ERROR
> | -
> >> >>> >> | NOSTATE | |
> >> >>> >> | a49598bb-0d4a-4064-af16-5b4b91057fd8 | php111php-719 | ERROR
> | -
> >> >>> >> | NOSTATE | |
> >> >>> >> | f8376091-99f6-4d17-8ec6-3935291c3e40 | php111php-841 | ERROR
> | -
> >> >>> >> | NOSTATE | |
> >> >>> >> | cf3298da-eecc-448a-bcea-2296eeed23dc | php111php-ad | ACTIVE
> | -
> >> >>> >> | Running | private=10.11.12.2 |
> >> >>> >> | 1d8832d5-39e4-476c-9c2b-1b1f5ffe3b41 | php111php-d81 | ACTIVE
> | -
> >> >>> >> | Running | private=10.11.12.3 |
> >> >>> >> | 4f6d51fb-fd27-4be1-94be-e9b5200fb6ba | php111php-e9c | ERROR
> | -
> >> >>> >> | NOSTATE | |
> >> >>> >>
> >> >>> >>
> >> >>> >>
> >> >>> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >>> >>
> >> >>> >> I'll investigate some more...
> >> >>> >>
> >> >>> >> On Thu, May 29, 2014 at 7:42 AM, Nirmal Fernando
> >> >>> >> <ni...@gmail.com>
> >> >>> >> wrote:
> >> >>> >> > Ah.. probably Chris. So, in min check what happens is it always
> >> >>> >> > make
> >> >>> >> > sure
> >> >>> >> > that you have min instance count in the system.
> >> >>> >> >
> >> >>> >> >
> >> >>> >> > On Thu, May 29, 2014 at 11:42 AM, chris snow
> >> >>> >> > <ch...@gmail.com>
> >> >>> >> > wrote:
> >> >>> >> >>
> >> >>> >> >> Hi Nirmal,
> >> >>> >> >>
> >> >>> >> >> I've just see that it is the "Minimal Rule" in mincheck.drl
> that
> >> >>> >> >> is
> >> >>> >> >> trying to spawn the instance, and not scaling.drl.
> >> >>> >> >>
> >> >>> >> >> This is why AutoscaleAlgorithm.getNextScaleUpPartition() is
> not
> >> >>> >> >> getting
> >> >>> >> >> called?
> >> >>> >> >>
> >> >>> >> >> Cheers,
> >> >>> >> >>
> >> >>> >> >> Chris
> >> >>> >> >>
> >> >>> >> >>
> >> >>> >> >> On Wed, May 28, 2014 at 11:24 PM, Nirmal Fernando
> >> >>> >> >> <ni...@gmail.com> wrote:
> >> >>> >> >> > No, Chris. This happens via Drools rules located in
> >> >>> >> >> > scaling.drl
> >> >>> >> >> > file
> >> >>> >> >> > of
> >> >>> >> >> > conf
> >> >>> >> >> > directory.
> >> >>> >> >> >
> >> >>> >> >> >
> >> >>> >> >> > On Thu, May 29, 2014 at 3:39 AM, chris snow
> >> >>> >> >> > <ch...@gmail.com>
> >> >>> >> >> > wrote:
> >> >>> >> >> >>
> >> >>> >> >> >> Hi Nirmal,
> >> >>> >> >> >>
> >> >>> >> >> >> For some reason, that method is not getting called. I've
> >> >>> >> >> >> pasted
> >> >>> >> >> >> my
> >> >>> >> >> >> stacktrace again with the drools methods removed:
> >> >>> >> >> >>
> >> >>> >> >> >> ---
> >> >>> >> >> >> Daemon Thread [Thread-56] (Suspended (breakpoint at line
> 137
> >> >>> >> >> >> in
> >> >>> >> >> >> CloudControllerClient))
> >> >>> >> >> >> owns: CloudControllerClient (id=2433)
> >> >>> >> >> >> owns: DefaultAgenda (id=2434)
> >> >>> >> >> >> CloudControllerClient.spawnAnInstance(Partition, String,
> >> >>> >> >> >> String,
> >> >>> >> >> >> String) line: 137
> >> >>> >> >> >> RuleTasksDelegator.delegateSpawn(PartitionContext,
> String,
> >> >>> >> >> >> String)
> >> >>> >> >> >> line: 92
> >> >>> >> >> >> //
> >> >>> >> >> >> // drools methods hidden
> >> >>> >> >> >> //
> >> >>> >> >> >>
> >> >>> >> >> >>
> >> >>> >> >> >>
> >> >>> >> >> >>
> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
> >> >>> >> >> >> FactHandle, Object) line: 86
> >> >>> >> >> >> ClusterMonitor.monitor() line: 99
> >> >>> >> >> >> ClusterMonitor.run() line: 72
> >> >>> >> >> >> Thread.run() line: 744
> >> >>> >> >> >> ---
> >> >>> >> >> >>
> >> >>> >> >> >> Would you expect to see
> >> >>> >> >> >> AutoscaleAlgorithm.getNextScaleUpPartition
> >> >>> >> >> >> executed somewhere in the stack of calls?
> >> >>> >> >> >>
> >> >>> >> >> >> Many thanks,
> >> >>> >> >> >>
> >> >>> >> >> >> Chris
> >> >>> >> >> >>
> >> >>> >> >> >> On Wed, May 28, 2014 at 11:03 PM, Nirmal Fernando
> >> >>> >> >> >> <ni...@gmail.com> wrote:
> >> >>> >> >> >> > Hi Chris,
> >> >>> >> >> >> >
> >> >>> >> >> >> > PartitionMax is considered in
> >> >>> >> >> >> > AutoscaleAlgorithm.getNextScaleUpPartition
> >> >>> >> >> >> > method.
> >> >>> >> >> >> >
> >> >>> >> >> >> >
> >> >>> >> >> >> > On Thu, May 29, 2014 at 3:09 AM, chris snow
> >> >>> >> >> >> > <ch...@gmail.com>
> >> >>> >> >> >> > wrote:
> >> >>> >> >> >> >>
> >> >>> >> >> >> >> I setup a breakpoint in eclipse at
> >> >>> >> >> >> >> CloudControllerClient.spawnAnInstance() and dumped the
> >> >>> >> >> >> >> output
> >> >>> >> >> >> >> (below).
> >> >>> >> >> >> >>
> >> >>> >> >> >> >> I've scanned through the code in the stratos methods in
> >> >>> >> >> >> >> the
> >> >>> >> >> >> >> stacktrace, but couldn't see any code that checks the
> >> >>> >> >> >> >> partitionMax
> >> >>> >> >> >> >> value.
> >> >>> >> >> >> >>
> >> >>> >> >> >> >> Any ideas?
> >> >>> >> >> >> >>
> >> >>> >> >> >> >> Daemon Thread [Thread-56] (Suspended (breakpoint at line
> >> >>> >> >> >> >> 137
> >> >>> >> >> >> >> in
> >> >>> >> >> >> >> CloudControllerClient))
> >> >>> >> >> >> >> owns: CloudControllerClient (id=2433)
> >> >>> >> >> >> >> owns: DefaultAgenda (id=2434)
> >> >>> >> >> >> >> CloudControllerClient.spawnAnInstance(Partition,
> >> >>> >> >> >> >> String,
> >> >>> >> >> >> >> String,
> >> >>> >> >> >> >> String) line: 137
> >> >>> >> >> >> >> RuleTasksDelegator.delegateSpawn(PartitionContext,
> >> >>> >> >> >> >> String,
> >> >>> >> >> >> >> String)
> >> >>> >> >> >> >> line: 92
> >> >>> >> >> >> >> NativeMethodAccessorImpl.invoke0(Method, Object,
> >> >>> >> >> >> >> Object[])
> >> >>> >> >> >> >> line:
> >> >>> >> >> >> >> not available [native method]
> >> >>> >> >> >> >> NativeMethodAccessorImpl.invoke(Object, Object[])
> line:
> >> >>> >> >> >> >> 57
> >> >>> >> >> >> >> DelegatingMethodAccessorImpl.invoke(Object, Object[])
> >> >>> >> >> >> >> line: 43
> >> >>> >> >> >> >> Method.invoke(Object, Object...) line: 606
> >> >>> >> >> >> >> ReflectiveAccessorOptimizer.getMethod(Object, String,
> >> >>> >> >> >> >> Object[],
> >> >>> >> >> >> >> Class[], ExecutableStatement[]) line: 1104
> >> >>> >> >> >> >> ReflectiveAccessorOptimizer.getMethod(Object, String)
> >> >>> >> >> >> >> line:
> >> >>> >> >> >> >> 987
> >> >>> >> >> >> >> ReflectiveAccessorOptimizer.compileGetChain() line:
> 377
> >> >>> >> >> >> >>
> >> >>> >> >> >> >>
> >> >>> >> >> >> >>
> ReflectiveAccessorOptimizer.optimizeAccessor(ParserContext,
> >> >>> >> >> >> >> char[],
> >> >>> >> >> >> >> int, int, Object, Object, VariableResolverFactory,
> >> >>> >> >> >> >> boolean,
> >> >>> >> >> >> >> Class)
> >> >>> >> >> >> >> line: 143
> >> >>> >> >> >> >> ASTNode.optimize(Object, Object,
> >> >>> >> >> >> >> VariableResolverFactory)
> >> >>> >> >> >> >> line:
> >> >>> >> >> >> >> 159
> >> >>> >> >> >> >> ASTNode.getReducedValueAccelerated(Object, Object,
> >> >>> >> >> >> >> VariableResolverFactory) line: 115
> >> >>> >> >> >> >> MVELRuntime.execute(boolean, CompiledExpression,
> >> >>> >> >> >> >> Object,
> >> >>> >> >> >> >> VariableResolverFactory) line: 85
> >> >>> >> >> >> >> CompiledExpression.getDirectValue(Object,
> >> >>> >> >> >> >> VariableResolverFactory)
> >> >>> >> >> >> >> line: 123
> >> >>> >> >> >> >> CompiledExpression.getValue(Object,
> >> >>> >> >> >> >> VariableResolverFactory)
> >> >>> >> >> >> >> line:
> >> >>> >> >> >> >> 119
> >> >>> >> >> >> >> MVEL.executeExpression(Object, Object,
> >> >>> >> >> >> >> VariableResolverFactory)
> >> >>> >> >> >> >> line:
> >> >>> >> >> >> >> 930
> >> >>> >> >> >> >> MVELConsequence.evaluate(KnowledgeHelper,
> >> >>> >> >> >> >> WorkingMemory)
> >> >>> >> >> >> >> line:
> >> >>> >> >> >> >> 104
> >> >>> >> >> >> >> DefaultAgenda.fireActivation(Activation) line: 1287
> >> >>> >> >> >> >> DefaultAgenda.fireNextItem(AgendaFilter) line: 1221
> >> >>> >> >> >> >> DefaultAgenda.fireAllRules(AgendaFilter, int) line:
> >> >>> >> >> >> >> 1456
> >> >>> >> >> >> >>
> >> >>> >> >> >> >>
> >> >>> >> >> >> >>
> >> >>> >> >> >> >>
> >> >>> >> >> >> >>
> >> >>> >> >> >> >>
> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules(AgendaFilter,
> >> >>> >> >> >> >> int) line: 710
> >> >>> >> >> >> >>
> >> >>> >> >> >> >>
> >> >>> >> >> >> >>
> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules()
> >> >>> >> >> >> >> line:
> >> >>> >> >> >> >> 674
> >> >>> >> >> >> >> StatefulKnowledgeSessionImpl.fireAllRules() line: 230
> >> >>> >> >> >> >>
> >> >>> >> >> >> >>
> >> >>> >> >> >> >>
> >> >>> >> >> >> >>
> >> >>> >> >> >> >>
> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
> >> >>> >> >> >> >> FactHandle, Object) line: 86
> >> >>> >> >> >> >> ClusterMonitor.monitor() line: 99
> >> >>> >> >> >> >> ClusterMonitor.run() line: 72
> >> >>> >> >> >> >> Thread.run() line: 744
> >> >>> >> >> >> >>
> >> >>> >> >> >> >> On Wed, May 28, 2014 at 3:37 AM, Lahiru Sandaruwan
> >> >>> >> >> >> >> <la...@wso2.com>
> >> >>> >> >> >> >> wrote:
> >> >>> >> >> >> >> >
> >> >>> >> >> >> >> >
> >> >>> >> >> >> >> >
> >> >>> >> >> >> >> > On Tue, May 27, 2014 at 11:54 PM, chris snow
> >> >>> >> >> >> >> > <ch...@gmail.com>
> >> >>> >> >> >> >> > wrote:
> >> >>> >> >> >> >> >>
> >> >>> >> >> >> >> >> Hi Lahiru,
> >> >>> >> >> >> >> >>
> >> >>> >> >> >> >> >> I'll have a poke around - it will give me a good
> >> >>> >> >> >> >> >> opportunity
> >> >>> >> >> >> >> >> to
> >> >>> >> >> >> >> >> learn
> >> >>> >> >> >> >> >> some more Stratos code :)
> >> >>> >> >> >> >> >>
> >> >>> >> >> >> >> >
> >> >>> >> >> >> >> > Cool :)
> >> >>> >> >> >> >> >
> >> >>> >> >> >> >> > Debug logs should be able to show us where it went
> >> >>> >> >> >> >> > wrong.
> >> >>> >> >> >> >> >
> >> >>> >> >> >> >> > Thanks.
> >> >>> >> >> >> >> >>
> >> >>> >> >> >> >> >> Many thanks,
> >> >>> >> >> >> >> >>
> >> >>> >> >> >> >> >> Chris
> >> >>> >> >> >> >> >>
> >> >>> >> >> >> >> >> On Tue, May 27, 2014 at 7:45 AM, Lahiru Sandaruwan
> >> >>> >> >> >> >> >> <la...@wso2.com>
> >> >>> >> >> >> >> >> wrote:
> >> >>> >> >> >> >> >> > Hi Chris,
> >> >>> >> >> >> >> >> >
> >> >>> >> >> >> >> >> > Can you enable debug logs and try out?
> >> >>> >> >> >> >> >> > If so please send them over.
> >> >>> >> >> >> >> >> >
> >> >>> >> >> >> >> >> > Thanks.
> >> >>> >> >> >> >> >> >
> >> >>> >> >> >> >> >> >
> >> >>> >> >> >> >> >> > On Tue, May 27, 2014 at 12:05 PM, chris snow
> >> >>> >> >> >> >> >> > <ch...@gmail.com>
> >> >>> >> >> >> >> >> > wrote:
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >> Deployment Policy partitionMax seems to be
> ignored.
> >> >>> >> >> >> >> >> >> My
> >> >>> >> >> >> >> >> >> deployment
> >> >>> >> >> >> >> >> >> policy looks like this:
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >> {
> >> >>> >> >> >> >> >> >> "id": "economyDeployment",
> >> >>> >> >> >> >> >> >> "partitionGroup": [
> >> >>> >> >> >> >> >> >> {
> >> >>> >> >> >> >> >> >> "id": "G1",
> >> >>> >> >> >> >> >> >> "partitionAlgo": "one-after-another",
> >> >>> >> >> >> >> >> >> "partition": [
> >> >>> >> >> >> >> >> >> {
> >> >>> >> >> >> >> >> >> "id": "P1",
> >> >>> >> >> >> >> >> >> "partitionMax": "1",
> >> >>> >> >> >> >> >> >> "partitionMin": "1"
> >> >>> >> >> >> >> >> >> }
> >> >>> >> >> >> >> >> >> ]
> >> >>> >> >> >> >> >> >> }
> >> >>> >> >> >> >> >> >> ]
> >> >>> >> >> >> >> >> >> }
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >> Stratos initially starts two instances:
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >>> >> >> >> >> >> >> | ID | Name
> >> >>> >> >> >> >> >> >> |
> >> >>> >> >> >> >> >> >> Status |
> >> >>> >> >> >> >> >> >> Task
> >> >>> >> >> >> >> >> >> State | Power State | Networks |
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >>> >> >> >> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d |
> >> >>> >> >> >> >> >> >> php111php-9f6
> >> >>> >> >> >> >> >> >> |
> >> >>> >> >> >> >> >> >> ACTIVE |
> >> >>> >> >> >> >> >> >> -
> >> >>> >> >> >> >> >> >> | Running | private=10.11.12.2 |
> >> >>> >> >> >> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f |
> >> >>> >> >> >> >> >> >> php111php-a05
> >> >>> >> >> >> >> >> >> |
> >> >>> >> >> >> >> >> >> ACTIVE |
> >> >>> >> >> >> >> >> >> -
> >> >>> >> >> >> >> >> >> | Running | private=10.11.12.3 |
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >>> >> >> >> >> >> >> Then attempts to start more:
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >>> >> >> >> >> >> >> | ID | Name
> >> >>> >> >> >> >> >> >> |
> >> >>> >> >> >> >> >> >> Status |
> >> >>> >> >> >> >> >> >> Task
> >> >>> >> >> >> >> >> >> State | Power State | Networks |
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >>> >> >> >> >> >> >> | 91029130-37b5-4df6-ad95-1d9f609a22c3 |
> >> >>> >> >> >> >> >> >> php111php-13e
> >> >>> >> >> >> >> >> >> |
> >> >>> >> >> >> >> >> >> ERROR
> >> >>> >> >> >> >> >> >> |
> >> >>> >> >> >> >> >> >> -
> >> >>> >> >> >> >> >> >> | NOSTATE | |
> >> >>> >> >> >> >> >> >> | 71c1e361-df55-4f61-84c5-79774fef2beb |
> >> >>> >> >> >> >> >> >> php111php-3d7
> >> >>> >> >> >> >> >> >> |
> >> >>> >> >> >> >> >> >> ERROR
> >> >>> >> >> >> >> >> >> |
> >> >>> >> >> >> >> >> >> -
> >> >>> >> >> >> >> >> >> | NOSTATE | |
> >> >>> >> >> >> >> >> >> | b0d72602-cdb8-4621-ad12-911ade4d89b1 |
> >> >>> >> >> >> >> >> >> php111php-412
> >> >>> >> >> >> >> >> >> |
> >> >>> >> >> >> >> >> >> ERROR
> >> >>> >> >> >> >> >> >> |
> >> >>> >> >> >> >> >> >> -
> >> >>> >> >> >> >> >> >> | NOSTATE | |
> >> >>> >> >> >> >> >> >> | d4eac55d-bb7c-48d6-83f2-cb9a11581a18 |
> >> >>> >> >> >> >> >> >> php111php-6f5
> >> >>> >> >> >> >> >> >> |
> >> >>> >> >> >> >> >> >> ERROR
> >> >>> >> >> >> >> >> >> |
> >> >>> >> >> >> >> >> >> -
> >> >>> >> >> >> >> >> >> | NOSTATE | |
> >> >>> >> >> >> >> >> >> | 738a9534-2b95-4984-a51b-748bc7169b9a |
> >> >>> >> >> >> >> >> >> php111php-73e
> >> >>> >> >> >> >> >> >> |
> >> >>> >> >> >> >> >> >> ERROR
> >> >>> >> >> >> >> >> >> |
> >> >>> >> >> >> >> >> >> -
> >> >>> >> >> >> >> >> >> | NOSTATE | |
> >> >>> >> >> >> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d |
> >> >>> >> >> >> >> >> >> php111php-9f6
> >> >>> >> >> >> >> >> >> |
> >> >>> >> >> >> >> >> >> ACTIVE |
> >> >>> >> >> >> >> >> >> -
> >> >>> >> >> >> >> >> >> | Running | private=10.11.12.2 |
> >> >>> >> >> >> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f |
> >> >>> >> >> >> >> >> >> php111php-a05
> >> >>> >> >> >> >> >> >> |
> >> >>> >> >> >> >> >> >> ACTIVE |
> >> >>> >> >> >> >> >> >> -
> >> >>> >> >> >> >> >> >> | Running | private=10.11.12.3 |
> >> >>> >> >> >> >> >> >> | f570e654-024b-4240-bad2-8a11f04f23f0 |
> >> >>> >> >> >> >> >> >> php111php-a30
> >> >>> >> >> >> >> >> >> |
> >> >>> >> >> >> >> >> >> ERROR
> >> >>> >> >> >> >> >> >> |
> >> >>> >> >> >> >> >> >> -
> >> >>> >> >> >> >> >> >> | NOSTATE | |
> >> >>> >> >> >> >> >> >> | 0ab36824-9bea-4a95-aa22-7f935406d83d |
> >> >>> >> >> >> >> >> >> php111php-bca
> >> >>> >> >> >> >> >> >> |
> >> >>> >> >> >> >> >> >> ERROR
> >> >>> >> >> >> >> >> >> |
> >> >>> >> >> >> >> >> >> -
> >> >>> >> >> >> >> >> >> | NOSTATE | |
> >> >>> >> >> >> >> >> >> | 02ed97df-02bc-4bff-b42d-9f2bcbd4f569 |
> >> >>> >> >> >> >> >> >> php111php-f6f
> >> >>> >> >> >> >> >> >> |
> >> >>> >> >> >> >> >> >> ERROR
> >> >>> >> >> >> >> >> >> |
> >> >>> >> >> >> >> >> >> -
> >> >>> >> >> >> >> >> >> | NOSTATE | |
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >> I'm on branch: 4.0.0-incubating, using Openstack
> >> >>> >> >> >> >> >> >> Havana
> >> >>> >> >> >> >> >> >> +
> >> >>> >> >> >> >> >> >> Docker.
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >> Any ideas?
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >> Many thanks,
> >> >>> >> >> >> >> >> >>
> >> >>> >> >> >> >> >> >> Chris
> >> >>> >> >> >> >> >> >
> >> >>> >> >> >> >> >> >
> >> >>> >> >> >> >> >> >
> >> >>> >> >> >> >> >> >
> >> >>> >> >> >> >> >> > --
> >> >>> >> >> >> >> >> > --
> >> >>> >> >> >> >> >> > Lahiru Sandaruwan
> >> >>> >> >> >> >> >> > Committer and PMC member, Apache Stratos,
> >> >>> >> >> >> >> >> > Senior Software Engineer,
> >> >>> >> >> >> >> >> > WSO2 Inc., http://wso2.com
> >> >>> >> >> >> >> >> > lean.enterprise.middleware
> >> >>> >> >> >> >> >> >
> >> >>> >> >> >> >> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
> >> >>> >> >> >> >> >> > blog: http://lahiruwrites.blogspot.com/
> >> >>> >> >> >> >> >> > twitter: http://twitter.com/lahirus
> >> >>> >> >> >> >> >> > linked-in:
> >> >>> >> >> >> >> >> >
> >> >>> >> >> >> >> >> >
> http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
> >> >>> >> >> >> >> >> >
> >> >>> >> >> >> >> >>
> >> >>> >> >> >> >> >>
> >> >>> >> >> >> >> >>
> >> >>> >> >> >> >> >> --
> >> >>> >> >> >> >> >> Check out my professional profile and connect with me
> >> >>> >> >> >> >> >> on
> >> >>> >> >> >> >> >> LinkedIn.
> >> >>> >> >> >> >> >> http://lnkd.in/cw5k69
> >> >>> >> >> >> >> >
> >> >>> >> >> >> >> >
> >> >>> >> >> >> >> >
> >> >>> >> >> >> >> >
> >> >>> >> >> >> >> > --
> >> >>> >> >> >> >> > --
> >> >>> >> >> >> >> > Lahiru Sandaruwan
> >> >>> >> >> >> >> > Committer and PMC member, Apache Stratos,
> >> >>> >> >> >> >> > Senior Software Engineer,
> >> >>> >> >> >> >> > WSO2 Inc., http://wso2.com
> >> >>> >> >> >> >> > lean.enterprise.middleware
> >> >>> >> >> >> >> >
> >> >>> >> >> >> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
> >> >>> >> >> >> >> > blog: http://lahiruwrites.blogspot.com/
> >> >>> >> >> >> >> > twitter: http://twitter.com/lahirus
> >> >>> >> >> >> >> > linked-in:
> >> >>> >> >> >> >> >
> http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
> >> >>> >> >> >> >> >
> >> >>> >> >> >> >>
> >> >>> >> >> >> >>
> >> >>> >> >> >> >>
> >> >>> >> >> >> >> --
> >> >>> >> >> >> >> Check out my professional profile and connect with me on
> >> >>> >> >> >> >> LinkedIn.
> >> >>> >> >> >> >> http://lnkd.in/cw5k69
> >> >>> >> >> >> >
> >> >>> >> >> >> >
> >> >>> >> >> >> >
> >> >>> >> >> >> >
> >> >>> >> >> >> > --
> >> >>> >> >> >> > Best Regards,
> >> >>> >> >> >> > Nirmal
> >> >>> >> >> >> >
> >> >>> >> >> >> > Nirmal Fernando.
> >> >>> >> >> >> > PPMC Member & Committer of Apache Stratos,
> >> >>> >> >> >> >
> >> >>> >> >> >> > Senior Software Engineer, WSO2 Inc.
> >> >>> >> >> >> >
> >> >>> >> >> >> > Blog: http://nirmalfdo.blogspot.com/
> >> >>> >> >> >>
> >> >>> >> >> >>
> >> >>> >> >> >>
> >> >>> >> >> >> --
> >> >>> >> >> >> Check out my professional profile and connect with me on
> >> >>> >> >> >> LinkedIn.
> >> >>> >> >> >> http://lnkd.in/cw5k69
> >> >>> >> >> >
> >> >>> >> >> >
> >> >>> >> >> >
> >> >>> >> >> >
> >> >>> >> >> > --
> >> >>> >> >> > Best Regards,
> >> >>> >> >> > Nirmal
> >> >>> >> >> >
> >> >>> >> >> > Nirmal Fernando.
> >> >>> >> >> > PPMC Member & Committer of Apache Stratos,
> >> >>> >> >> > Senior Software Engineer, WSO2 Inc.
> >> >>> >> >> >
> >> >>> >> >> > Blog: http://nirmalfdo.blogspot.com/
> >> >>> >> >>
> >> >>> >> >>
> >> >>> >> >>
> >> >>> >> >> --
> >> >>> >> >> Check out my professional profile and connect with me on
> >> >>> >> >> LinkedIn.
> >> >>> >> >> http://lnkd.in/cw5k69
> >> >>> >> >
> >> >>> >> >
> >> >>> >> >
> >> >>> >> >
> >> >>> >> > --
> >> >>> >> > Best Regards,
> >> >>> >> > Nirmal
> >> >>> >> >
> >> >>> >> > Nirmal Fernando.
> >> >>> >> > PPMC Member & Committer of Apache Stratos,
> >> >>> >> > Senior Software Engineer, WSO2 Inc.
> >> >>> >> >
> >> >>> >> > Blog: http://nirmalfdo.blogspot.com/
> >> >>> >>
> >> >>> >>
> >> >>> >>
> >> >>> >> --
> >> >>> >> Check out my professional profile and connect with me on
> LinkedIn.
> >> >>> >> http://lnkd.in/cw5k69
> >> >>> >
> >> >>> >
> >> >>> >
> >> >>> >
> >> >>> > --
> >> >>> > --
> >> >>> > Lahiru Sandaruwan
> >> >>> > Committer and PMC member, Apache Stratos,
> >> >>> > Senior Software Engineer,
> >> >>> > WSO2 Inc., http://wso2.com
> >> >>> > lean.enterprise.middleware
> >> >>> >
> >> >>> > email: lahirus@wso2.com cell: (+94) 773 325 954
> >> >>> > blog: http://lahiruwrites.blogspot.com/
> >> >>> > twitter: http://twitter.com/lahirus
> >> >>> > linked-in:
> http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
> >> >>> >
> >> >>>
> >> >>>
> >> >>>
> >> >>> --
> >> >>> Check out my professional profile and connect with me on LinkedIn.
> >> >>> http://lnkd.in/cw5k69
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Best Regards,
> >> >> Nirmal
> >> >>
> >> >> Nirmal Fernando.
> >> >> PPMC Member & Committer of Apache Stratos,
> >> >> Senior Software Engineer, WSO2 Inc.
> >> >>
> >> >> Blog: http://nirmalfdo.blogspot.com/
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > Lakmal Warusawithana
> >> > Vice President, Apache Stratos
> >> > Director - Cloud Architecture; WSO2 Inc.
> >> > Mobile : +94714289692
> >> > Blog : http://lakmalsview.blogspot.com/
> >> >
> >>
> >>
> >>
> >> --
> >> Check out my professional profile and connect with me on LinkedIn.
> >> http://lnkd.in/cw5k69
> >
> >
> >
> >
> > --
> > --
> > Lahiru Sandaruwan
> > Committer and PMC member, Apache Stratos,
> > Senior Software Engineer,
> > WSO2 Inc., http://wso2.com
> > lean.enterprise.middleware
> >
> > email: lahirus@wso2.com cell: (+94) 773 325 954
> > blog: http://lahiruwrites.blogspot.com/
> > twitter: http://twitter.com/lahirus
> > linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
> >
>
>
>
> --
> Check out my professional profile and connect with me on LinkedIn.
> http://lnkd.in/cw5k69
>
--
--
Lahiru Sandaruwan
Committer and PMC member, Apache Stratos,
Senior Software Engineer,
WSO2 Inc., http://wso2.com
lean.enterprise.middleware
email: lahirus@wso2.com cell: (+94) 773 325 954
blog: http://lahiruwrites.blogspot.com/
twitter: http://twitter.com/lahirus
linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
Re: autoscaling is ignoring partiionMax
Posted by chris snow <ch...@gmail.com>.
Hi Lahiru,
I've added a JIRA for a configurable retry limit [1].
Stratos is not terminating the instances:
Based on this deployment policy ...
{
"id": "economyDeployment",
"partitionGroup": [
{
"id": "G1",
"partitionAlgo": "one-after-another",
"partition": [
{
"id": "P1",
"partitionMax": "1",
"partitionMin": "1"
}
]
}
]
}
... Stratos initially starts two instances ...
+--------------------------------------+---------------+--------+------------+-------------+--------------------+
| ID | Name | Status | Task
State | Power State | Networks |
+--------------------------------------+---------------+--------+------------+-------------+--------------------+
| c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 | ACTIVE | -
| Running | private=10.11.12.2 |
| b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 | ACTIVE | -
| Running | private=10.11.12.3 |
+--------------------------------------+---------------+--------+------------+-------------+--------------------+
... then attempts to start more:
+--------------------------------------+---------------+--------+------------+-------------+--------------------+
| ID | Name | Status | Task
State | Power State | Networks |
+--------------------------------------+---------------+--------+------------+-------------+--------------------+
| 91029130-37b5-4df6-ad95-1d9f609a22c3 | php111php-13e | ERROR | -
| NOSTATE | |
| 71c1e361-df55-4f61-84c5-79774fef2beb | php111php-3d7 | ERROR | -
| NOSTATE | |
| b0d72602-cdb8-4621-ad12-911ade4d89b1 | php111php-412 | ERROR | -
| NOSTATE | |
| d4eac55d-bb7c-48d6-83f2-cb9a11581a18 | php111php-6f5 | ERROR | -
| NOSTATE | |
| 738a9534-2b95-4984-a51b-748bc7169b9a | php111php-73e | ERROR | -
| NOSTATE | |
| c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 | ACTIVE | -
| Running | private=10.11.12.2 |
| b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 | ACTIVE | -
| Running | private=10.11.12.3 |
| f570e654-024b-4240-bad2-8a11f04f23f0 | php111php-a30 | ERROR | -
| NOSTATE | |
| 0ab36824-9bea-4a95-aa22-7f935406d83d | php111php-bca | ERROR | -
| NOSTATE | |
| 02ed97df-02bc-4bff-b42d-9f2bcbd4f569 | php111php-f6f | ERROR | -
| NOSTATE | |
+--------------------------------------+---------------+--------+------------+-------------+--------------------+
I'll have a dig around to see why it is not terminating the instances.
---
[1] https://issues.apache.org/jira/browse/STRATOS-658
On Thu, May 29, 2014 at 9:25 AM, Lahiru Sandaruwan <la...@wso2.com> wrote:
>
> Hi Chris,
>
> On Thu, May 29, 2014 at 1:50 PM, chris snow <ch...@gmail.com> wrote:
>>
>> Ok, thanks all. My agent definitely has not started, so there is the
>> source of the problem. Thanks for the help getting to the bottom of
>> this.
>>
>> A few thoughts on this:
>>
>> 1) Conceptually, I was thinking that partitionMax was equal to the
>> number of IaaS instances "spawned", whereas it really means number of
>> "instances" connected to Stratos. Maybe we should add that note to
>> the wiki [1]?
>> 2) Anyone else running into this problem on EC2 or another
>> subscription based IaaS could end up with a big bill because Stratos
>> will just keep spawning and spawning.
>> 3) Should we put in a limit on the number of instances that Stratos
>> spawns that are not connected?
>
>
> Autoscaler will wait some time for the spawned instance's member activated
> event and terminate if time limit is reached.
> Didn't it happen in your case?
>
> So this is like retrying to get a working instance.
> +1 for a limit of such re-tries.
>
> Thanks.
>
>
>>
>> ---
>> [1]
>> https://cwiki.apache.org/confluence/display/STRATOS/4.0.0+Sample+Deployment+Policy+Definition
>>
>> On Thu, May 29, 2014 at 9:06 AM, Lakmal Warusawithana <la...@wso2.com>
>> wrote:
>> > Actually its sending by CC based on "Insance activated" event at
>> > Instance
>> > Status topic which publish by Cartridge agent.
>> >
>> >
>> > On Thu, May 29, 2014 at 1:31 PM, Nirmal Fernando
>> > <ni...@gmail.com>
>> > wrote:
>> >>
>> >> It's agent Chris.
>> >>
>> >>
>> >> On Thu, May 29, 2014 at 1:25 PM, chris snow <ch...@gmail.com>
>> >> wrote:
>> >>>
>> >>> Hi Lahiru, are MemberActivatedEvents sent by the Stratos agent, or are
>> >>> they sent by the another component (e.g. CloudController)?
>> >>>
>> >>> On Thu, May 29, 2014 at 7:49 AM, Lahiru Sandaruwan <la...@wso2.com>
>> >>> wrote:
>> >>> > Hi Chris,
>> >>> >
>> >>> > Can you confirm that the Autoscaler received Member activated event
>> >>> > for
>> >>> > these particular instances? Logs should help identify that.
>> >>> >
>> >>> > Thanks.
>> >>> >
>> >>> >
>> >>> >
>> >>> >
>> >>> > On Thu, May 29, 2014 at 12:15 PM, chris snow <ch...@gmail.com>
>> >>> > wrote:
>> >>> >>
>> >>> >> Thanks Nirmal.
>> >>> >>
>> >>> >> Non terminated member count doesn't seem to detect the two active
>> >>> >> instances that have been started:
>> >>> >>
>> >>> >> [2014-05-29 06:06:23,900] DEBUG
>> >>> >> {org.apache.stratos.autoscaler.rule.RuleLog} - Running minimum
>> >>> >> rule:
>> >>> >> [network-partition] G1 [partition] P1
>> >>> >> [2014-05-29 06:06:23,906] DEBUG
>> >>> >> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check]
>> >>> >> [network-partition] G1 [partition] P1 Non terminated member count:
>> >>> >> 0
>> >>> >> [2014-05-29 06:06:23,908] DEBUG
>> >>> >> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check]
>> >>> >> [network-partition] G1 [partition] P1 Minimum member count: 1
>> >>> >> [2014-05-29 06:06:23,925] DEBUG
>> >>> >> {org.apache.stratos.autoscaler.rule.RuleLog} - Running obsolete
>> >>> >> instances rule: [partition] P1 [network-partition] G1
>> >>> >> [2014-05-29 06:06:23,927] DEBUG
>> >>> >> {org.apache.stratos.autoscaler.rule.RuleLog} - [obsolete-check]
>> >>> >> [network-partition] G1 [partition] P1 Obsolete member count: 0
>> >>> >> [2014-05-29 06:06:23,930] INFO
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >> {org.apache.stratos.autoscaler.client.cloud.controller.CloudControllerClient}
>> >>> >> - Trying to spawn an instance via cloud controller: [cluster]
>> >>> >> php111.php.domain [partition] P1 [lb-cluster] null
>> >>> >> [network-partition-id] G1
>> >>> >>
>> >>> >>
>> >>> >> vagrant@puppet:~$ nova list
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >>> >> | ID | Name | Status |
>> >>> >> Task
>> >>> >> State | Power State | Networks |
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >>> >> | a3ce6f52-27bf-4343-a05d-d7c9da9a1cb2 | php111php-3cb | ERROR | -
>> >>> >> | NOSTATE | |
>> >>> >> | 82434b60-6c9d-40a3-8191-2d466cde725e | php111php-47e | ERROR | -
>> >>> >> | NOSTATE | |
>> >>> >> | 4aa942ff-a722-4ff4-b604-abaa2f44c1a7 | php111php-499 | ERROR | -
>> >>> >> | NOSTATE | |
>> >>> >> | 1548915b-98f6-48fc-b70a-b0acc74610a7 | php111php-5e0 | ERROR | -
>> >>> >> | NOSTATE | |
>> >>> >> | f313c93d-7de1-4ccb-8c0d-f4104e5d837f | php111php-701 | ERROR | -
>> >>> >> | NOSTATE | |
>> >>> >> | a49598bb-0d4a-4064-af16-5b4b91057fd8 | php111php-719 | ERROR | -
>> >>> >> | NOSTATE | |
>> >>> >> | f8376091-99f6-4d17-8ec6-3935291c3e40 | php111php-841 | ERROR | -
>> >>> >> | NOSTATE | |
>> >>> >> | cf3298da-eecc-448a-bcea-2296eeed23dc | php111php-ad | ACTIVE | -
>> >>> >> | Running | private=10.11.12.2 |
>> >>> >> | 1d8832d5-39e4-476c-9c2b-1b1f5ffe3b41 | php111php-d81 | ACTIVE | -
>> >>> >> | Running | private=10.11.12.3 |
>> >>> >> | 4f6d51fb-fd27-4be1-94be-e9b5200fb6ba | php111php-e9c | ERROR | -
>> >>> >> | NOSTATE | |
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >>> >>
>> >>> >> I'll investigate some more...
>> >>> >>
>> >>> >> On Thu, May 29, 2014 at 7:42 AM, Nirmal Fernando
>> >>> >> <ni...@gmail.com>
>> >>> >> wrote:
>> >>> >> > Ah.. probably Chris. So, in min check what happens is it always
>> >>> >> > make
>> >>> >> > sure
>> >>> >> > that you have min instance count in the system.
>> >>> >> >
>> >>> >> >
>> >>> >> > On Thu, May 29, 2014 at 11:42 AM, chris snow
>> >>> >> > <ch...@gmail.com>
>> >>> >> > wrote:
>> >>> >> >>
>> >>> >> >> Hi Nirmal,
>> >>> >> >>
>> >>> >> >> I've just see that it is the "Minimal Rule" in mincheck.drl that
>> >>> >> >> is
>> >>> >> >> trying to spawn the instance, and not scaling.drl.
>> >>> >> >>
>> >>> >> >> This is why AutoscaleAlgorithm.getNextScaleUpPartition() is not
>> >>> >> >> getting
>> >>> >> >> called?
>> >>> >> >>
>> >>> >> >> Cheers,
>> >>> >> >>
>> >>> >> >> Chris
>> >>> >> >>
>> >>> >> >>
>> >>> >> >> On Wed, May 28, 2014 at 11:24 PM, Nirmal Fernando
>> >>> >> >> <ni...@gmail.com> wrote:
>> >>> >> >> > No, Chris. This happens via Drools rules located in
>> >>> >> >> > scaling.drl
>> >>> >> >> > file
>> >>> >> >> > of
>> >>> >> >> > conf
>> >>> >> >> > directory.
>> >>> >> >> >
>> >>> >> >> >
>> >>> >> >> > On Thu, May 29, 2014 at 3:39 AM, chris snow
>> >>> >> >> > <ch...@gmail.com>
>> >>> >> >> > wrote:
>> >>> >> >> >>
>> >>> >> >> >> Hi Nirmal,
>> >>> >> >> >>
>> >>> >> >> >> For some reason, that method is not getting called. I've
>> >>> >> >> >> pasted
>> >>> >> >> >> my
>> >>> >> >> >> stacktrace again with the drools methods removed:
>> >>> >> >> >>
>> >>> >> >> >> ---
>> >>> >> >> >> Daemon Thread [Thread-56] (Suspended (breakpoint at line 137
>> >>> >> >> >> in
>> >>> >> >> >> CloudControllerClient))
>> >>> >> >> >> owns: CloudControllerClient (id=2433)
>> >>> >> >> >> owns: DefaultAgenda (id=2434)
>> >>> >> >> >> CloudControllerClient.spawnAnInstance(Partition, String,
>> >>> >> >> >> String,
>> >>> >> >> >> String) line: 137
>> >>> >> >> >> RuleTasksDelegator.delegateSpawn(PartitionContext, String,
>> >>> >> >> >> String)
>> >>> >> >> >> line: 92
>> >>> >> >> >> //
>> >>> >> >> >> // drools methods hidden
>> >>> >> >> >> //
>> >>> >> >> >>
>> >>> >> >> >>
>> >>> >> >> >>
>> >>> >> >> >> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
>> >>> >> >> >> FactHandle, Object) line: 86
>> >>> >> >> >> ClusterMonitor.monitor() line: 99
>> >>> >> >> >> ClusterMonitor.run() line: 72
>> >>> >> >> >> Thread.run() line: 744
>> >>> >> >> >> ---
>> >>> >> >> >>
>> >>> >> >> >> Would you expect to see
>> >>> >> >> >> AutoscaleAlgorithm.getNextScaleUpPartition
>> >>> >> >> >> executed somewhere in the stack of calls?
>> >>> >> >> >>
>> >>> >> >> >> Many thanks,
>> >>> >> >> >>
>> >>> >> >> >> Chris
>> >>> >> >> >>
>> >>> >> >> >> On Wed, May 28, 2014 at 11:03 PM, Nirmal Fernando
>> >>> >> >> >> <ni...@gmail.com> wrote:
>> >>> >> >> >> > Hi Chris,
>> >>> >> >> >> >
>> >>> >> >> >> > PartitionMax is considered in
>> >>> >> >> >> > AutoscaleAlgorithm.getNextScaleUpPartition
>> >>> >> >> >> > method.
>> >>> >> >> >> >
>> >>> >> >> >> >
>> >>> >> >> >> > On Thu, May 29, 2014 at 3:09 AM, chris snow
>> >>> >> >> >> > <ch...@gmail.com>
>> >>> >> >> >> > wrote:
>> >>> >> >> >> >>
>> >>> >> >> >> >> I setup a breakpoint in eclipse at
>> >>> >> >> >> >> CloudControllerClient.spawnAnInstance() and dumped the
>> >>> >> >> >> >> output
>> >>> >> >> >> >> (below).
>> >>> >> >> >> >>
>> >>> >> >> >> >> I've scanned through the code in the stratos methods in
>> >>> >> >> >> >> the
>> >>> >> >> >> >> stacktrace, but couldn't see any code that checks the
>> >>> >> >> >> >> partitionMax
>> >>> >> >> >> >> value.
>> >>> >> >> >> >>
>> >>> >> >> >> >> Any ideas?
>> >>> >> >> >> >>
>> >>> >> >> >> >> Daemon Thread [Thread-56] (Suspended (breakpoint at line
>> >>> >> >> >> >> 137
>> >>> >> >> >> >> in
>> >>> >> >> >> >> CloudControllerClient))
>> >>> >> >> >> >> owns: CloudControllerClient (id=2433)
>> >>> >> >> >> >> owns: DefaultAgenda (id=2434)
>> >>> >> >> >> >> CloudControllerClient.spawnAnInstance(Partition,
>> >>> >> >> >> >> String,
>> >>> >> >> >> >> String,
>> >>> >> >> >> >> String) line: 137
>> >>> >> >> >> >> RuleTasksDelegator.delegateSpawn(PartitionContext,
>> >>> >> >> >> >> String,
>> >>> >> >> >> >> String)
>> >>> >> >> >> >> line: 92
>> >>> >> >> >> >> NativeMethodAccessorImpl.invoke0(Method, Object,
>> >>> >> >> >> >> Object[])
>> >>> >> >> >> >> line:
>> >>> >> >> >> >> not available [native method]
>> >>> >> >> >> >> NativeMethodAccessorImpl.invoke(Object, Object[]) line:
>> >>> >> >> >> >> 57
>> >>> >> >> >> >> DelegatingMethodAccessorImpl.invoke(Object, Object[])
>> >>> >> >> >> >> line: 43
>> >>> >> >> >> >> Method.invoke(Object, Object...) line: 606
>> >>> >> >> >> >> ReflectiveAccessorOptimizer.getMethod(Object, String,
>> >>> >> >> >> >> Object[],
>> >>> >> >> >> >> Class[], ExecutableStatement[]) line: 1104
>> >>> >> >> >> >> ReflectiveAccessorOptimizer.getMethod(Object, String)
>> >>> >> >> >> >> line:
>> >>> >> >> >> >> 987
>> >>> >> >> >> >> ReflectiveAccessorOptimizer.compileGetChain() line: 377
>> >>> >> >> >> >>
>> >>> >> >> >> >>
>> >>> >> >> >> >> ReflectiveAccessorOptimizer.optimizeAccessor(ParserContext,
>> >>> >> >> >> >> char[],
>> >>> >> >> >> >> int, int, Object, Object, VariableResolverFactory,
>> >>> >> >> >> >> boolean,
>> >>> >> >> >> >> Class)
>> >>> >> >> >> >> line: 143
>> >>> >> >> >> >> ASTNode.optimize(Object, Object,
>> >>> >> >> >> >> VariableResolverFactory)
>> >>> >> >> >> >> line:
>> >>> >> >> >> >> 159
>> >>> >> >> >> >> ASTNode.getReducedValueAccelerated(Object, Object,
>> >>> >> >> >> >> VariableResolverFactory) line: 115
>> >>> >> >> >> >> MVELRuntime.execute(boolean, CompiledExpression,
>> >>> >> >> >> >> Object,
>> >>> >> >> >> >> VariableResolverFactory) line: 85
>> >>> >> >> >> >> CompiledExpression.getDirectValue(Object,
>> >>> >> >> >> >> VariableResolverFactory)
>> >>> >> >> >> >> line: 123
>> >>> >> >> >> >> CompiledExpression.getValue(Object,
>> >>> >> >> >> >> VariableResolverFactory)
>> >>> >> >> >> >> line:
>> >>> >> >> >> >> 119
>> >>> >> >> >> >> MVEL.executeExpression(Object, Object,
>> >>> >> >> >> >> VariableResolverFactory)
>> >>> >> >> >> >> line:
>> >>> >> >> >> >> 930
>> >>> >> >> >> >> MVELConsequence.evaluate(KnowledgeHelper,
>> >>> >> >> >> >> WorkingMemory)
>> >>> >> >> >> >> line:
>> >>> >> >> >> >> 104
>> >>> >> >> >> >> DefaultAgenda.fireActivation(Activation) line: 1287
>> >>> >> >> >> >> DefaultAgenda.fireNextItem(AgendaFilter) line: 1221
>> >>> >> >> >> >> DefaultAgenda.fireAllRules(AgendaFilter, int) line:
>> >>> >> >> >> >> 1456
>> >>> >> >> >> >>
>> >>> >> >> >> >>
>> >>> >> >> >> >>
>> >>> >> >> >> >>
>> >>> >> >> >> >>
>> >>> >> >> >> >> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules(AgendaFilter,
>> >>> >> >> >> >> int) line: 710
>> >>> >> >> >> >>
>> >>> >> >> >> >>
>> >>> >> >> >> >> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules()
>> >>> >> >> >> >> line:
>> >>> >> >> >> >> 674
>> >>> >> >> >> >> StatefulKnowledgeSessionImpl.fireAllRules() line: 230
>> >>> >> >> >> >>
>> >>> >> >> >> >>
>> >>> >> >> >> >>
>> >>> >> >> >> >>
>> >>> >> >> >> >> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
>> >>> >> >> >> >> FactHandle, Object) line: 86
>> >>> >> >> >> >> ClusterMonitor.monitor() line: 99
>> >>> >> >> >> >> ClusterMonitor.run() line: 72
>> >>> >> >> >> >> Thread.run() line: 744
>> >>> >> >> >> >>
>> >>> >> >> >> >> On Wed, May 28, 2014 at 3:37 AM, Lahiru Sandaruwan
>> >>> >> >> >> >> <la...@wso2.com>
>> >>> >> >> >> >> wrote:
>> >>> >> >> >> >> >
>> >>> >> >> >> >> >
>> >>> >> >> >> >> >
>> >>> >> >> >> >> > On Tue, May 27, 2014 at 11:54 PM, chris snow
>> >>> >> >> >> >> > <ch...@gmail.com>
>> >>> >> >> >> >> > wrote:
>> >>> >> >> >> >> >>
>> >>> >> >> >> >> >> Hi Lahiru,
>> >>> >> >> >> >> >>
>> >>> >> >> >> >> >> I'll have a poke around - it will give me a good
>> >>> >> >> >> >> >> opportunity
>> >>> >> >> >> >> >> to
>> >>> >> >> >> >> >> learn
>> >>> >> >> >> >> >> some more Stratos code :)
>> >>> >> >> >> >> >>
>> >>> >> >> >> >> >
>> >>> >> >> >> >> > Cool :)
>> >>> >> >> >> >> >
>> >>> >> >> >> >> > Debug logs should be able to show us where it went
>> >>> >> >> >> >> > wrong.
>> >>> >> >> >> >> >
>> >>> >> >> >> >> > Thanks.
>> >>> >> >> >> >> >>
>> >>> >> >> >> >> >> Many thanks,
>> >>> >> >> >> >> >>
>> >>> >> >> >> >> >> Chris
>> >>> >> >> >> >> >>
>> >>> >> >> >> >> >> On Tue, May 27, 2014 at 7:45 AM, Lahiru Sandaruwan
>> >>> >> >> >> >> >> <la...@wso2.com>
>> >>> >> >> >> >> >> wrote:
>> >>> >> >> >> >> >> > Hi Chris,
>> >>> >> >> >> >> >> >
>> >>> >> >> >> >> >> > Can you enable debug logs and try out?
>> >>> >> >> >> >> >> > If so please send them over.
>> >>> >> >> >> >> >> >
>> >>> >> >> >> >> >> > Thanks.
>> >>> >> >> >> >> >> >
>> >>> >> >> >> >> >> >
>> >>> >> >> >> >> >> > On Tue, May 27, 2014 at 12:05 PM, chris snow
>> >>> >> >> >> >> >> > <ch...@gmail.com>
>> >>> >> >> >> >> >> > wrote:
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >> Deployment Policy partitionMax seems to be ignored.
>> >>> >> >> >> >> >> >> My
>> >>> >> >> >> >> >> >> deployment
>> >>> >> >> >> >> >> >> policy looks like this:
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >> {
>> >>> >> >> >> >> >> >> "id": "economyDeployment",
>> >>> >> >> >> >> >> >> "partitionGroup": [
>> >>> >> >> >> >> >> >> {
>> >>> >> >> >> >> >> >> "id": "G1",
>> >>> >> >> >> >> >> >> "partitionAlgo": "one-after-another",
>> >>> >> >> >> >> >> >> "partition": [
>> >>> >> >> >> >> >> >> {
>> >>> >> >> >> >> >> >> "id": "P1",
>> >>> >> >> >> >> >> >> "partitionMax": "1",
>> >>> >> >> >> >> >> >> "partitionMin": "1"
>> >>> >> >> >> >> >> >> }
>> >>> >> >> >> >> >> >> ]
>> >>> >> >> >> >> >> >> }
>> >>> >> >> >> >> >> >> ]
>> >>> >> >> >> >> >> >> }
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >> Stratos initially starts two instances:
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >>> >> >> >> >> >> >> | ID | Name
>> >>> >> >> >> >> >> >> |
>> >>> >> >> >> >> >> >> Status |
>> >>> >> >> >> >> >> >> Task
>> >>> >> >> >> >> >> >> State | Power State | Networks |
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >>> >> >> >> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d |
>> >>> >> >> >> >> >> >> php111php-9f6
>> >>> >> >> >> >> >> >> |
>> >>> >> >> >> >> >> >> ACTIVE |
>> >>> >> >> >> >> >> >> -
>> >>> >> >> >> >> >> >> | Running | private=10.11.12.2 |
>> >>> >> >> >> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f |
>> >>> >> >> >> >> >> >> php111php-a05
>> >>> >> >> >> >> >> >> |
>> >>> >> >> >> >> >> >> ACTIVE |
>> >>> >> >> >> >> >> >> -
>> >>> >> >> >> >> >> >> | Running | private=10.11.12.3 |
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >>> >> >> >> >> >> >> Then attempts to start more:
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >>> >> >> >> >> >> >> | ID | Name
>> >>> >> >> >> >> >> >> |
>> >>> >> >> >> >> >> >> Status |
>> >>> >> >> >> >> >> >> Task
>> >>> >> >> >> >> >> >> State | Power State | Networks |
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >>> >> >> >> >> >> >> | 91029130-37b5-4df6-ad95-1d9f609a22c3 |
>> >>> >> >> >> >> >> >> php111php-13e
>> >>> >> >> >> >> >> >> |
>> >>> >> >> >> >> >> >> ERROR
>> >>> >> >> >> >> >> >> |
>> >>> >> >> >> >> >> >> -
>> >>> >> >> >> >> >> >> | NOSTATE | |
>> >>> >> >> >> >> >> >> | 71c1e361-df55-4f61-84c5-79774fef2beb |
>> >>> >> >> >> >> >> >> php111php-3d7
>> >>> >> >> >> >> >> >> |
>> >>> >> >> >> >> >> >> ERROR
>> >>> >> >> >> >> >> >> |
>> >>> >> >> >> >> >> >> -
>> >>> >> >> >> >> >> >> | NOSTATE | |
>> >>> >> >> >> >> >> >> | b0d72602-cdb8-4621-ad12-911ade4d89b1 |
>> >>> >> >> >> >> >> >> php111php-412
>> >>> >> >> >> >> >> >> |
>> >>> >> >> >> >> >> >> ERROR
>> >>> >> >> >> >> >> >> |
>> >>> >> >> >> >> >> >> -
>> >>> >> >> >> >> >> >> | NOSTATE | |
>> >>> >> >> >> >> >> >> | d4eac55d-bb7c-48d6-83f2-cb9a11581a18 |
>> >>> >> >> >> >> >> >> php111php-6f5
>> >>> >> >> >> >> >> >> |
>> >>> >> >> >> >> >> >> ERROR
>> >>> >> >> >> >> >> >> |
>> >>> >> >> >> >> >> >> -
>> >>> >> >> >> >> >> >> | NOSTATE | |
>> >>> >> >> >> >> >> >> | 738a9534-2b95-4984-a51b-748bc7169b9a |
>> >>> >> >> >> >> >> >> php111php-73e
>> >>> >> >> >> >> >> >> |
>> >>> >> >> >> >> >> >> ERROR
>> >>> >> >> >> >> >> >> |
>> >>> >> >> >> >> >> >> -
>> >>> >> >> >> >> >> >> | NOSTATE | |
>> >>> >> >> >> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d |
>> >>> >> >> >> >> >> >> php111php-9f6
>> >>> >> >> >> >> >> >> |
>> >>> >> >> >> >> >> >> ACTIVE |
>> >>> >> >> >> >> >> >> -
>> >>> >> >> >> >> >> >> | Running | private=10.11.12.2 |
>> >>> >> >> >> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f |
>> >>> >> >> >> >> >> >> php111php-a05
>> >>> >> >> >> >> >> >> |
>> >>> >> >> >> >> >> >> ACTIVE |
>> >>> >> >> >> >> >> >> -
>> >>> >> >> >> >> >> >> | Running | private=10.11.12.3 |
>> >>> >> >> >> >> >> >> | f570e654-024b-4240-bad2-8a11f04f23f0 |
>> >>> >> >> >> >> >> >> php111php-a30
>> >>> >> >> >> >> >> >> |
>> >>> >> >> >> >> >> >> ERROR
>> >>> >> >> >> >> >> >> |
>> >>> >> >> >> >> >> >> -
>> >>> >> >> >> >> >> >> | NOSTATE | |
>> >>> >> >> >> >> >> >> | 0ab36824-9bea-4a95-aa22-7f935406d83d |
>> >>> >> >> >> >> >> >> php111php-bca
>> >>> >> >> >> >> >> >> |
>> >>> >> >> >> >> >> >> ERROR
>> >>> >> >> >> >> >> >> |
>> >>> >> >> >> >> >> >> -
>> >>> >> >> >> >> >> >> | NOSTATE | |
>> >>> >> >> >> >> >> >> | 02ed97df-02bc-4bff-b42d-9f2bcbd4f569 |
>> >>> >> >> >> >> >> >> php111php-f6f
>> >>> >> >> >> >> >> >> |
>> >>> >> >> >> >> >> >> ERROR
>> >>> >> >> >> >> >> >> |
>> >>> >> >> >> >> >> >> -
>> >>> >> >> >> >> >> >> | NOSTATE | |
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >> I'm on branch: 4.0.0-incubating, using Openstack
>> >>> >> >> >> >> >> >> Havana
>> >>> >> >> >> >> >> >> +
>> >>> >> >> >> >> >> >> Docker.
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >> Any ideas?
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >> Many thanks,
>> >>> >> >> >> >> >> >>
>> >>> >> >> >> >> >> >> Chris
>> >>> >> >> >> >> >> >
>> >>> >> >> >> >> >> >
>> >>> >> >> >> >> >> >
>> >>> >> >> >> >> >> >
>> >>> >> >> >> >> >> > --
>> >>> >> >> >> >> >> > --
>> >>> >> >> >> >> >> > Lahiru Sandaruwan
>> >>> >> >> >> >> >> > Committer and PMC member, Apache Stratos,
>> >>> >> >> >> >> >> > Senior Software Engineer,
>> >>> >> >> >> >> >> > WSO2 Inc., http://wso2.com
>> >>> >> >> >> >> >> > lean.enterprise.middleware
>> >>> >> >> >> >> >> >
>> >>> >> >> >> >> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
>> >>> >> >> >> >> >> > blog: http://lahiruwrites.blogspot.com/
>> >>> >> >> >> >> >> > twitter: http://twitter.com/lahirus
>> >>> >> >> >> >> >> > linked-in:
>> >>> >> >> >> >> >> >
>> >>> >> >> >> >> >> > http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>> >>> >> >> >> >> >> >
>> >>> >> >> >> >> >>
>> >>> >> >> >> >> >>
>> >>> >> >> >> >> >>
>> >>> >> >> >> >> >> --
>> >>> >> >> >> >> >> Check out my professional profile and connect with me
>> >>> >> >> >> >> >> on
>> >>> >> >> >> >> >> LinkedIn.
>> >>> >> >> >> >> >> http://lnkd.in/cw5k69
>> >>> >> >> >> >> >
>> >>> >> >> >> >> >
>> >>> >> >> >> >> >
>> >>> >> >> >> >> >
>> >>> >> >> >> >> > --
>> >>> >> >> >> >> > --
>> >>> >> >> >> >> > Lahiru Sandaruwan
>> >>> >> >> >> >> > Committer and PMC member, Apache Stratos,
>> >>> >> >> >> >> > Senior Software Engineer,
>> >>> >> >> >> >> > WSO2 Inc., http://wso2.com
>> >>> >> >> >> >> > lean.enterprise.middleware
>> >>> >> >> >> >> >
>> >>> >> >> >> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
>> >>> >> >> >> >> > blog: http://lahiruwrites.blogspot.com/
>> >>> >> >> >> >> > twitter: http://twitter.com/lahirus
>> >>> >> >> >> >> > linked-in:
>> >>> >> >> >> >> > http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>> >>> >> >> >> >> >
>> >>> >> >> >> >>
>> >>> >> >> >> >>
>> >>> >> >> >> >>
>> >>> >> >> >> >> --
>> >>> >> >> >> >> Check out my professional profile and connect with me on
>> >>> >> >> >> >> LinkedIn.
>> >>> >> >> >> >> http://lnkd.in/cw5k69
>> >>> >> >> >> >
>> >>> >> >> >> >
>> >>> >> >> >> >
>> >>> >> >> >> >
>> >>> >> >> >> > --
>> >>> >> >> >> > Best Regards,
>> >>> >> >> >> > Nirmal
>> >>> >> >> >> >
>> >>> >> >> >> > Nirmal Fernando.
>> >>> >> >> >> > PPMC Member & Committer of Apache Stratos,
>> >>> >> >> >> >
>> >>> >> >> >> > Senior Software Engineer, WSO2 Inc.
>> >>> >> >> >> >
>> >>> >> >> >> > Blog: http://nirmalfdo.blogspot.com/
>> >>> >> >> >>
>> >>> >> >> >>
>> >>> >> >> >>
>> >>> >> >> >> --
>> >>> >> >> >> Check out my professional profile and connect with me on
>> >>> >> >> >> LinkedIn.
>> >>> >> >> >> http://lnkd.in/cw5k69
>> >>> >> >> >
>> >>> >> >> >
>> >>> >> >> >
>> >>> >> >> >
>> >>> >> >> > --
>> >>> >> >> > Best Regards,
>> >>> >> >> > Nirmal
>> >>> >> >> >
>> >>> >> >> > Nirmal Fernando.
>> >>> >> >> > PPMC Member & Committer of Apache Stratos,
>> >>> >> >> > Senior Software Engineer, WSO2 Inc.
>> >>> >> >> >
>> >>> >> >> > Blog: http://nirmalfdo.blogspot.com/
>> >>> >> >>
>> >>> >> >>
>> >>> >> >>
>> >>> >> >> --
>> >>> >> >> Check out my professional profile and connect with me on
>> >>> >> >> LinkedIn.
>> >>> >> >> http://lnkd.in/cw5k69
>> >>> >> >
>> >>> >> >
>> >>> >> >
>> >>> >> >
>> >>> >> > --
>> >>> >> > Best Regards,
>> >>> >> > Nirmal
>> >>> >> >
>> >>> >> > Nirmal Fernando.
>> >>> >> > PPMC Member & Committer of Apache Stratos,
>> >>> >> > Senior Software Engineer, WSO2 Inc.
>> >>> >> >
>> >>> >> > Blog: http://nirmalfdo.blogspot.com/
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >> --
>> >>> >> Check out my professional profile and connect with me on LinkedIn.
>> >>> >> http://lnkd.in/cw5k69
>> >>> >
>> >>> >
>> >>> >
>> >>> >
>> >>> > --
>> >>> > --
>> >>> > Lahiru Sandaruwan
>> >>> > Committer and PMC member, Apache Stratos,
>> >>> > Senior Software Engineer,
>> >>> > WSO2 Inc., http://wso2.com
>> >>> > lean.enterprise.middleware
>> >>> >
>> >>> > email: lahirus@wso2.com cell: (+94) 773 325 954
>> >>> > blog: http://lahiruwrites.blogspot.com/
>> >>> > twitter: http://twitter.com/lahirus
>> >>> > linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>> >>> >
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Check out my professional profile and connect with me on LinkedIn.
>> >>> http://lnkd.in/cw5k69
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Best Regards,
>> >> Nirmal
>> >>
>> >> Nirmal Fernando.
>> >> PPMC Member & Committer of Apache Stratos,
>> >> Senior Software Engineer, WSO2 Inc.
>> >>
>> >> Blog: http://nirmalfdo.blogspot.com/
>> >
>> >
>> >
>> >
>> > --
>> > Lakmal Warusawithana
>> > Vice President, Apache Stratos
>> > Director - Cloud Architecture; WSO2 Inc.
>> > Mobile : +94714289692
>> > Blog : http://lakmalsview.blogspot.com/
>> >
>>
>>
>>
>> --
>> Check out my professional profile and connect with me on LinkedIn.
>> http://lnkd.in/cw5k69
>
>
>
>
> --
> --
> Lahiru Sandaruwan
> Committer and PMC member, Apache Stratos,
> Senior Software Engineer,
> WSO2 Inc., http://wso2.com
> lean.enterprise.middleware
>
> email: lahirus@wso2.com cell: (+94) 773 325 954
> blog: http://lahiruwrites.blogspot.com/
> twitter: http://twitter.com/lahirus
> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>
--
Check out my professional profile and connect with me on LinkedIn.
http://lnkd.in/cw5k69
Re: autoscaling is ignoring partiionMax
Posted by Lahiru Sandaruwan <la...@wso2.com>.
Hi Chris,
On Thu, May 29, 2014 at 1:50 PM, chris snow <ch...@gmail.com> wrote:
> Ok, thanks all. My agent definitely has not started, so there is the
> source of the problem. Thanks for the help getting to the bottom of
> this.
>
> A few thoughts on this:
>
> 1) Conceptually, I was thinking that partitionMax was equal to the
> number of IaaS instances "spawned", whereas it really means number of
> "instances" connected to Stratos. Maybe we should add that note to
> the wiki [1]?
> 2) Anyone else running into this problem on EC2 or another
> subscription based IaaS could end up with a big bill because Stratos
> will just keep spawning and spawning.
> 3) Should we put in a limit on the number of instances that Stratos
> spawns that are not connected?
>
Autoscaler will wait some time for the spawned instance's member activated
event and terminate if time limit is reached.
Didn't it happen in your case?
So this is like retrying to get a working instance.
+1 for a limit of such re-tries.
Thanks.
> ---
> [1]
> https://cwiki.apache.org/confluence/display/STRATOS/4.0.0+Sample+Deployment+Policy+Definition
>
> On Thu, May 29, 2014 at 9:06 AM, Lakmal Warusawithana <la...@wso2.com>
> wrote:
> > Actually its sending by CC based on "Insance activated" event at Instance
> > Status topic which publish by Cartridge agent.
> >
> >
> > On Thu, May 29, 2014 at 1:31 PM, Nirmal Fernando <nirmal070125@gmail.com
> >
> > wrote:
> >>
> >> It's agent Chris.
> >>
> >>
> >> On Thu, May 29, 2014 at 1:25 PM, chris snow <ch...@gmail.com>
> wrote:
> >>>
> >>> Hi Lahiru, are MemberActivatedEvents sent by the Stratos agent, or are
> >>> they sent by the another component (e.g. CloudController)?
> >>>
> >>> On Thu, May 29, 2014 at 7:49 AM, Lahiru Sandaruwan <la...@wso2.com>
> >>> wrote:
> >>> > Hi Chris,
> >>> >
> >>> > Can you confirm that the Autoscaler received Member activated event
> for
> >>> > these particular instances? Logs should help identify that.
> >>> >
> >>> > Thanks.
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > On Thu, May 29, 2014 at 12:15 PM, chris snow <ch...@gmail.com>
> >>> > wrote:
> >>> >>
> >>> >> Thanks Nirmal.
> >>> >>
> >>> >> Non terminated member count doesn't seem to detect the two active
> >>> >> instances that have been started:
> >>> >>
> >>> >> [2014-05-29 06:06:23,900] DEBUG
> >>> >> {org.apache.stratos.autoscaler.rule.RuleLog} - Running minimum
> rule:
> >>> >> [network-partition] G1 [partition] P1
> >>> >> [2014-05-29 06:06:23,906] DEBUG
> >>> >> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check]
> >>> >> [network-partition] G1 [partition] P1 Non terminated member count: 0
> >>> >> [2014-05-29 06:06:23,908] DEBUG
> >>> >> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check]
> >>> >> [network-partition] G1 [partition] P1 Minimum member count: 1
> >>> >> [2014-05-29 06:06:23,925] DEBUG
> >>> >> {org.apache.stratos.autoscaler.rule.RuleLog} - Running obsolete
> >>> >> instances rule: [partition] P1 [network-partition] G1
> >>> >> [2014-05-29 06:06:23,927] DEBUG
> >>> >> {org.apache.stratos.autoscaler.rule.RuleLog} - [obsolete-check]
> >>> >> [network-partition] G1 [partition] P1 Obsolete member count: 0
> >>> >> [2014-05-29 06:06:23,930] INFO
> >>> >>
> >>> >>
> >>> >>
> {org.apache.stratos.autoscaler.client.cloud.controller.CloudControllerClient}
> >>> >> - Trying to spawn an instance via cloud controller: [cluster]
> >>> >> php111.php.domain [partition] P1 [lb-cluster] null
> >>> >> [network-partition-id] G1
> >>> >>
> >>> >>
> >>> >> vagrant@puppet:~$ nova list
> >>> >>
> >>> >>
> >>> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >>> >> | ID | Name | Status |
> Task
> >>> >> State | Power State | Networks |
> >>> >>
> >>> >>
> >>> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >>> >> | a3ce6f52-27bf-4343-a05d-d7c9da9a1cb2 | php111php-3cb | ERROR | -
> >>> >> | NOSTATE | |
> >>> >> | 82434b60-6c9d-40a3-8191-2d466cde725e | php111php-47e | ERROR | -
> >>> >> | NOSTATE | |
> >>> >> | 4aa942ff-a722-4ff4-b604-abaa2f44c1a7 | php111php-499 | ERROR | -
> >>> >> | NOSTATE | |
> >>> >> | 1548915b-98f6-48fc-b70a-b0acc74610a7 | php111php-5e0 | ERROR | -
> >>> >> | NOSTATE | |
> >>> >> | f313c93d-7de1-4ccb-8c0d-f4104e5d837f | php111php-701 | ERROR | -
> >>> >> | NOSTATE | |
> >>> >> | a49598bb-0d4a-4064-af16-5b4b91057fd8 | php111php-719 | ERROR | -
> >>> >> | NOSTATE | |
> >>> >> | f8376091-99f6-4d17-8ec6-3935291c3e40 | php111php-841 | ERROR | -
> >>> >> | NOSTATE | |
> >>> >> | cf3298da-eecc-448a-bcea-2296eeed23dc | php111php-ad | ACTIVE | -
> >>> >> | Running | private=10.11.12.2 |
> >>> >> | 1d8832d5-39e4-476c-9c2b-1b1f5ffe3b41 | php111php-d81 | ACTIVE | -
> >>> >> | Running | private=10.11.12.3 |
> >>> >> | 4f6d51fb-fd27-4be1-94be-e9b5200fb6ba | php111php-e9c | ERROR | -
> >>> >> | NOSTATE | |
> >>> >>
> >>> >>
> >>> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >>> >>
> >>> >> I'll investigate some more...
> >>> >>
> >>> >> On Thu, May 29, 2014 at 7:42 AM, Nirmal Fernando
> >>> >> <ni...@gmail.com>
> >>> >> wrote:
> >>> >> > Ah.. probably Chris. So, in min check what happens is it always
> make
> >>> >> > sure
> >>> >> > that you have min instance count in the system.
> >>> >> >
> >>> >> >
> >>> >> > On Thu, May 29, 2014 at 11:42 AM, chris snow <chsnow123@gmail.com
> >
> >>> >> > wrote:
> >>> >> >>
> >>> >> >> Hi Nirmal,
> >>> >> >>
> >>> >> >> I've just see that it is the "Minimal Rule" in mincheck.drl that
> is
> >>> >> >> trying to spawn the instance, and not scaling.drl.
> >>> >> >>
> >>> >> >> This is why AutoscaleAlgorithm.getNextScaleUpPartition() is not
> >>> >> >> getting
> >>> >> >> called?
> >>> >> >>
> >>> >> >> Cheers,
> >>> >> >>
> >>> >> >> Chris
> >>> >> >>
> >>> >> >>
> >>> >> >> On Wed, May 28, 2014 at 11:24 PM, Nirmal Fernando
> >>> >> >> <ni...@gmail.com> wrote:
> >>> >> >> > No, Chris. This happens via Drools rules located in scaling.drl
> >>> >> >> > file
> >>> >> >> > of
> >>> >> >> > conf
> >>> >> >> > directory.
> >>> >> >> >
> >>> >> >> >
> >>> >> >> > On Thu, May 29, 2014 at 3:39 AM, chris snow <
> chsnow123@gmail.com>
> >>> >> >> > wrote:
> >>> >> >> >>
> >>> >> >> >> Hi Nirmal,
> >>> >> >> >>
> >>> >> >> >> For some reason, that method is not getting called. I've
> pasted
> >>> >> >> >> my
> >>> >> >> >> stacktrace again with the drools methods removed:
> >>> >> >> >>
> >>> >> >> >> ---
> >>> >> >> >> Daemon Thread [Thread-56] (Suspended (breakpoint at line 137
> in
> >>> >> >> >> CloudControllerClient))
> >>> >> >> >> owns: CloudControllerClient (id=2433)
> >>> >> >> >> owns: DefaultAgenda (id=2434)
> >>> >> >> >> CloudControllerClient.spawnAnInstance(Partition, String,
> >>> >> >> >> String,
> >>> >> >> >> String) line: 137
> >>> >> >> >> RuleTasksDelegator.delegateSpawn(PartitionContext, String,
> >>> >> >> >> String)
> >>> >> >> >> line: 92
> >>> >> >> >> //
> >>> >> >> >> // drools methods hidden
> >>> >> >> >> //
> >>> >> >> >>
> >>> >> >> >>
> >>> >> >> >>
> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
> >>> >> >> >> FactHandle, Object) line: 86
> >>> >> >> >> ClusterMonitor.monitor() line: 99
> >>> >> >> >> ClusterMonitor.run() line: 72
> >>> >> >> >> Thread.run() line: 744
> >>> >> >> >> ---
> >>> >> >> >>
> >>> >> >> >> Would you expect to see
> >>> >> >> >> AutoscaleAlgorithm.getNextScaleUpPartition
> >>> >> >> >> executed somewhere in the stack of calls?
> >>> >> >> >>
> >>> >> >> >> Many thanks,
> >>> >> >> >>
> >>> >> >> >> Chris
> >>> >> >> >>
> >>> >> >> >> On Wed, May 28, 2014 at 11:03 PM, Nirmal Fernando
> >>> >> >> >> <ni...@gmail.com> wrote:
> >>> >> >> >> > Hi Chris,
> >>> >> >> >> >
> >>> >> >> >> > PartitionMax is considered in
> >>> >> >> >> > AutoscaleAlgorithm.getNextScaleUpPartition
> >>> >> >> >> > method.
> >>> >> >> >> >
> >>> >> >> >> >
> >>> >> >> >> > On Thu, May 29, 2014 at 3:09 AM, chris snow
> >>> >> >> >> > <ch...@gmail.com>
> >>> >> >> >> > wrote:
> >>> >> >> >> >>
> >>> >> >> >> >> I setup a breakpoint in eclipse at
> >>> >> >> >> >> CloudControllerClient.spawnAnInstance() and dumped the
> output
> >>> >> >> >> >> (below).
> >>> >> >> >> >>
> >>> >> >> >> >> I've scanned through the code in the stratos methods in the
> >>> >> >> >> >> stacktrace, but couldn't see any code that checks the
> >>> >> >> >> >> partitionMax
> >>> >> >> >> >> value.
> >>> >> >> >> >>
> >>> >> >> >> >> Any ideas?
> >>> >> >> >> >>
> >>> >> >> >> >> Daemon Thread [Thread-56] (Suspended (breakpoint at line
> 137
> >>> >> >> >> >> in
> >>> >> >> >> >> CloudControllerClient))
> >>> >> >> >> >> owns: CloudControllerClient (id=2433)
> >>> >> >> >> >> owns: DefaultAgenda (id=2434)
> >>> >> >> >> >> CloudControllerClient.spawnAnInstance(Partition, String,
> >>> >> >> >> >> String,
> >>> >> >> >> >> String) line: 137
> >>> >> >> >> >> RuleTasksDelegator.delegateSpawn(PartitionContext,
> String,
> >>> >> >> >> >> String)
> >>> >> >> >> >> line: 92
> >>> >> >> >> >> NativeMethodAccessorImpl.invoke0(Method, Object,
> Object[])
> >>> >> >> >> >> line:
> >>> >> >> >> >> not available [native method]
> >>> >> >> >> >> NativeMethodAccessorImpl.invoke(Object, Object[]) line:
> 57
> >>> >> >> >> >> DelegatingMethodAccessorImpl.invoke(Object, Object[])
> >>> >> >> >> >> line: 43
> >>> >> >> >> >> Method.invoke(Object, Object...) line: 606
> >>> >> >> >> >> ReflectiveAccessorOptimizer.getMethod(Object, String,
> >>> >> >> >> >> Object[],
> >>> >> >> >> >> Class[], ExecutableStatement[]) line: 1104
> >>> >> >> >> >> ReflectiveAccessorOptimizer.getMethod(Object, String)
> >>> >> >> >> >> line:
> >>> >> >> >> >> 987
> >>> >> >> >> >> ReflectiveAccessorOptimizer.compileGetChain() line: 377
> >>> >> >> >> >>
> >>> >> >> >> >> ReflectiveAccessorOptimizer.optimizeAccessor(ParserContext,
> >>> >> >> >> >> char[],
> >>> >> >> >> >> int, int, Object, Object, VariableResolverFactory, boolean,
> >>> >> >> >> >> Class)
> >>> >> >> >> >> line: 143
> >>> >> >> >> >> ASTNode.optimize(Object, Object,
> VariableResolverFactory)
> >>> >> >> >> >> line:
> >>> >> >> >> >> 159
> >>> >> >> >> >> ASTNode.getReducedValueAccelerated(Object, Object,
> >>> >> >> >> >> VariableResolverFactory) line: 115
> >>> >> >> >> >> MVELRuntime.execute(boolean, CompiledExpression, Object,
> >>> >> >> >> >> VariableResolverFactory) line: 85
> >>> >> >> >> >> CompiledExpression.getDirectValue(Object,
> >>> >> >> >> >> VariableResolverFactory)
> >>> >> >> >> >> line: 123
> >>> >> >> >> >> CompiledExpression.getValue(Object,
> >>> >> >> >> >> VariableResolverFactory)
> >>> >> >> >> >> line:
> >>> >> >> >> >> 119
> >>> >> >> >> >> MVEL.executeExpression(Object, Object,
> >>> >> >> >> >> VariableResolverFactory)
> >>> >> >> >> >> line:
> >>> >> >> >> >> 930
> >>> >> >> >> >> MVELConsequence.evaluate(KnowledgeHelper, WorkingMemory)
> >>> >> >> >> >> line:
> >>> >> >> >> >> 104
> >>> >> >> >> >> DefaultAgenda.fireActivation(Activation) line: 1287
> >>> >> >> >> >> DefaultAgenda.fireNextItem(AgendaFilter) line: 1221
> >>> >> >> >> >> DefaultAgenda.fireAllRules(AgendaFilter, int) line: 1456
> >>> >> >> >> >>
> >>> >> >> >> >>
> >>> >> >> >> >>
> >>> >> >> >> >>
> >>> >> >> >> >>
> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules(AgendaFilter,
> >>> >> >> >> >> int) line: 710
> >>> >> >> >> >>
> >>> >> >> >> >> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules()
> >>> >> >> >> >> line:
> >>> >> >> >> >> 674
> >>> >> >> >> >> StatefulKnowledgeSessionImpl.fireAllRules() line: 230
> >>> >> >> >> >>
> >>> >> >> >> >>
> >>> >> >> >> >>
> >>> >> >> >> >>
> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
> >>> >> >> >> >> FactHandle, Object) line: 86
> >>> >> >> >> >> ClusterMonitor.monitor() line: 99
> >>> >> >> >> >> ClusterMonitor.run() line: 72
> >>> >> >> >> >> Thread.run() line: 744
> >>> >> >> >> >>
> >>> >> >> >> >> On Wed, May 28, 2014 at 3:37 AM, Lahiru Sandaruwan
> >>> >> >> >> >> <la...@wso2.com>
> >>> >> >> >> >> wrote:
> >>> >> >> >> >> >
> >>> >> >> >> >> >
> >>> >> >> >> >> >
> >>> >> >> >> >> > On Tue, May 27, 2014 at 11:54 PM, chris snow
> >>> >> >> >> >> > <ch...@gmail.com>
> >>> >> >> >> >> > wrote:
> >>> >> >> >> >> >>
> >>> >> >> >> >> >> Hi Lahiru,
> >>> >> >> >> >> >>
> >>> >> >> >> >> >> I'll have a poke around - it will give me a good
> >>> >> >> >> >> >> opportunity
> >>> >> >> >> >> >> to
> >>> >> >> >> >> >> learn
> >>> >> >> >> >> >> some more Stratos code :)
> >>> >> >> >> >> >>
> >>> >> >> >> >> >
> >>> >> >> >> >> > Cool :)
> >>> >> >> >> >> >
> >>> >> >> >> >> > Debug logs should be able to show us where it went wrong.
> >>> >> >> >> >> >
> >>> >> >> >> >> > Thanks.
> >>> >> >> >> >> >>
> >>> >> >> >> >> >> Many thanks,
> >>> >> >> >> >> >>
> >>> >> >> >> >> >> Chris
> >>> >> >> >> >> >>
> >>> >> >> >> >> >> On Tue, May 27, 2014 at 7:45 AM, Lahiru Sandaruwan
> >>> >> >> >> >> >> <la...@wso2.com>
> >>> >> >> >> >> >> wrote:
> >>> >> >> >> >> >> > Hi Chris,
> >>> >> >> >> >> >> >
> >>> >> >> >> >> >> > Can you enable debug logs and try out?
> >>> >> >> >> >> >> > If so please send them over.
> >>> >> >> >> >> >> >
> >>> >> >> >> >> >> > Thanks.
> >>> >> >> >> >> >> >
> >>> >> >> >> >> >> >
> >>> >> >> >> >> >> > On Tue, May 27, 2014 at 12:05 PM, chris snow
> >>> >> >> >> >> >> > <ch...@gmail.com>
> >>> >> >> >> >> >> > wrote:
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >> Deployment Policy partitionMax seems to be ignored.
> My
> >>> >> >> >> >> >> >> deployment
> >>> >> >> >> >> >> >> policy looks like this:
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >> {
> >>> >> >> >> >> >> >> "id": "economyDeployment",
> >>> >> >> >> >> >> >> "partitionGroup": [
> >>> >> >> >> >> >> >> {
> >>> >> >> >> >> >> >> "id": "G1",
> >>> >> >> >> >> >> >> "partitionAlgo": "one-after-another",
> >>> >> >> >> >> >> >> "partition": [
> >>> >> >> >> >> >> >> {
> >>> >> >> >> >> >> >> "id": "P1",
> >>> >> >> >> >> >> >> "partitionMax": "1",
> >>> >> >> >> >> >> >> "partitionMin": "1"
> >>> >> >> >> >> >> >> }
> >>> >> >> >> >> >> >> ]
> >>> >> >> >> >> >> >> }
> >>> >> >> >> >> >> >> ]
> >>> >> >> >> >> >> >> }
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >> Stratos initially starts two instances:
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >>> >> >> >> >> >> >> | ID | Name
> >>> >> >> >> >> >> >> |
> >>> >> >> >> >> >> >> Status |
> >>> >> >> >> >> >> >> Task
> >>> >> >> >> >> >> >> State | Power State | Networks |
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >>> >> >> >> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d |
> php111php-9f6
> >>> >> >> >> >> >> >> |
> >>> >> >> >> >> >> >> ACTIVE |
> >>> >> >> >> >> >> >> -
> >>> >> >> >> >> >> >> | Running | private=10.11.12.2 |
> >>> >> >> >> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f |
> php111php-a05
> >>> >> >> >> >> >> >> |
> >>> >> >> >> >> >> >> ACTIVE |
> >>> >> >> >> >> >> >> -
> >>> >> >> >> >> >> >> | Running | private=10.11.12.3 |
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >>> >> >> >> >> >> >> Then attempts to start more:
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >>> >> >> >> >> >> >> | ID | Name
> >>> >> >> >> >> >> >> |
> >>> >> >> >> >> >> >> Status |
> >>> >> >> >> >> >> >> Task
> >>> >> >> >> >> >> >> State | Power State | Networks |
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >>> >> >> >> >> >> >> | 91029130-37b5-4df6-ad95-1d9f609a22c3 |
> php111php-13e
> >>> >> >> >> >> >> >> |
> >>> >> >> >> >> >> >> ERROR
> >>> >> >> >> >> >> >> |
> >>> >> >> >> >> >> >> -
> >>> >> >> >> >> >> >> | NOSTATE | |
> >>> >> >> >> >> >> >> | 71c1e361-df55-4f61-84c5-79774fef2beb |
> php111php-3d7
> >>> >> >> >> >> >> >> |
> >>> >> >> >> >> >> >> ERROR
> >>> >> >> >> >> >> >> |
> >>> >> >> >> >> >> >> -
> >>> >> >> >> >> >> >> | NOSTATE | |
> >>> >> >> >> >> >> >> | b0d72602-cdb8-4621-ad12-911ade4d89b1 |
> php111php-412
> >>> >> >> >> >> >> >> |
> >>> >> >> >> >> >> >> ERROR
> >>> >> >> >> >> >> >> |
> >>> >> >> >> >> >> >> -
> >>> >> >> >> >> >> >> | NOSTATE | |
> >>> >> >> >> >> >> >> | d4eac55d-bb7c-48d6-83f2-cb9a11581a18 |
> php111php-6f5
> >>> >> >> >> >> >> >> |
> >>> >> >> >> >> >> >> ERROR
> >>> >> >> >> >> >> >> |
> >>> >> >> >> >> >> >> -
> >>> >> >> >> >> >> >> | NOSTATE | |
> >>> >> >> >> >> >> >> | 738a9534-2b95-4984-a51b-748bc7169b9a |
> php111php-73e
> >>> >> >> >> >> >> >> |
> >>> >> >> >> >> >> >> ERROR
> >>> >> >> >> >> >> >> |
> >>> >> >> >> >> >> >> -
> >>> >> >> >> >> >> >> | NOSTATE | |
> >>> >> >> >> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d |
> php111php-9f6
> >>> >> >> >> >> >> >> |
> >>> >> >> >> >> >> >> ACTIVE |
> >>> >> >> >> >> >> >> -
> >>> >> >> >> >> >> >> | Running | private=10.11.12.2 |
> >>> >> >> >> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f |
> php111php-a05
> >>> >> >> >> >> >> >> |
> >>> >> >> >> >> >> >> ACTIVE |
> >>> >> >> >> >> >> >> -
> >>> >> >> >> >> >> >> | Running | private=10.11.12.3 |
> >>> >> >> >> >> >> >> | f570e654-024b-4240-bad2-8a11f04f23f0 |
> php111php-a30
> >>> >> >> >> >> >> >> |
> >>> >> >> >> >> >> >> ERROR
> >>> >> >> >> >> >> >> |
> >>> >> >> >> >> >> >> -
> >>> >> >> >> >> >> >> | NOSTATE | |
> >>> >> >> >> >> >> >> | 0ab36824-9bea-4a95-aa22-7f935406d83d |
> php111php-bca
> >>> >> >> >> >> >> >> |
> >>> >> >> >> >> >> >> ERROR
> >>> >> >> >> >> >> >> |
> >>> >> >> >> >> >> >> -
> >>> >> >> >> >> >> >> | NOSTATE | |
> >>> >> >> >> >> >> >> | 02ed97df-02bc-4bff-b42d-9f2bcbd4f569 |
> php111php-f6f
> >>> >> >> >> >> >> >> |
> >>> >> >> >> >> >> >> ERROR
> >>> >> >> >> >> >> >> |
> >>> >> >> >> >> >> >> -
> >>> >> >> >> >> >> >> | NOSTATE | |
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >> I'm on branch: 4.0.0-incubating, using Openstack
> Havana
> >>> >> >> >> >> >> >> +
> >>> >> >> >> >> >> >> Docker.
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >> Any ideas?
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >> Many thanks,
> >>> >> >> >> >> >> >>
> >>> >> >> >> >> >> >> Chris
> >>> >> >> >> >> >> >
> >>> >> >> >> >> >> >
> >>> >> >> >> >> >> >
> >>> >> >> >> >> >> >
> >>> >> >> >> >> >> > --
> >>> >> >> >> >> >> > --
> >>> >> >> >> >> >> > Lahiru Sandaruwan
> >>> >> >> >> >> >> > Committer and PMC member, Apache Stratos,
> >>> >> >> >> >> >> > Senior Software Engineer,
> >>> >> >> >> >> >> > WSO2 Inc., http://wso2.com
> >>> >> >> >> >> >> > lean.enterprise.middleware
> >>> >> >> >> >> >> >
> >>> >> >> >> >> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
> >>> >> >> >> >> >> > blog: http://lahiruwrites.blogspot.com/
> >>> >> >> >> >> >> > twitter: http://twitter.com/lahirus
> >>> >> >> >> >> >> > linked-in:
> >>> >> >> >> >> >> >
> http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
> >>> >> >> >> >> >> >
> >>> >> >> >> >> >>
> >>> >> >> >> >> >>
> >>> >> >> >> >> >>
> >>> >> >> >> >> >> --
> >>> >> >> >> >> >> Check out my professional profile and connect with me on
> >>> >> >> >> >> >> LinkedIn.
> >>> >> >> >> >> >> http://lnkd.in/cw5k69
> >>> >> >> >> >> >
> >>> >> >> >> >> >
> >>> >> >> >> >> >
> >>> >> >> >> >> >
> >>> >> >> >> >> > --
> >>> >> >> >> >> > --
> >>> >> >> >> >> > Lahiru Sandaruwan
> >>> >> >> >> >> > Committer and PMC member, Apache Stratos,
> >>> >> >> >> >> > Senior Software Engineer,
> >>> >> >> >> >> > WSO2 Inc., http://wso2.com
> >>> >> >> >> >> > lean.enterprise.middleware
> >>> >> >> >> >> >
> >>> >> >> >> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
> >>> >> >> >> >> > blog: http://lahiruwrites.blogspot.com/
> >>> >> >> >> >> > twitter: http://twitter.com/lahirus
> >>> >> >> >> >> > linked-in:
> >>> >> >> >> >> > http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
> >>> >> >> >> >> >
> >>> >> >> >> >>
> >>> >> >> >> >>
> >>> >> >> >> >>
> >>> >> >> >> >> --
> >>> >> >> >> >> Check out my professional profile and connect with me on
> >>> >> >> >> >> LinkedIn.
> >>> >> >> >> >> http://lnkd.in/cw5k69
> >>> >> >> >> >
> >>> >> >> >> >
> >>> >> >> >> >
> >>> >> >> >> >
> >>> >> >> >> > --
> >>> >> >> >> > Best Regards,
> >>> >> >> >> > Nirmal
> >>> >> >> >> >
> >>> >> >> >> > Nirmal Fernando.
> >>> >> >> >> > PPMC Member & Committer of Apache Stratos,
> >>> >> >> >> >
> >>> >> >> >> > Senior Software Engineer, WSO2 Inc.
> >>> >> >> >> >
> >>> >> >> >> > Blog: http://nirmalfdo.blogspot.com/
> >>> >> >> >>
> >>> >> >> >>
> >>> >> >> >>
> >>> >> >> >> --
> >>> >> >> >> Check out my professional profile and connect with me on
> >>> >> >> >> LinkedIn.
> >>> >> >> >> http://lnkd.in/cw5k69
> >>> >> >> >
> >>> >> >> >
> >>> >> >> >
> >>> >> >> >
> >>> >> >> > --
> >>> >> >> > Best Regards,
> >>> >> >> > Nirmal
> >>> >> >> >
> >>> >> >> > Nirmal Fernando.
> >>> >> >> > PPMC Member & Committer of Apache Stratos,
> >>> >> >> > Senior Software Engineer, WSO2 Inc.
> >>> >> >> >
> >>> >> >> > Blog: http://nirmalfdo.blogspot.com/
> >>> >> >>
> >>> >> >>
> >>> >> >>
> >>> >> >> --
> >>> >> >> Check out my professional profile and connect with me on
> LinkedIn.
> >>> >> >> http://lnkd.in/cw5k69
> >>> >> >
> >>> >> >
> >>> >> >
> >>> >> >
> >>> >> > --
> >>> >> > Best Regards,
> >>> >> > Nirmal
> >>> >> >
> >>> >> > Nirmal Fernando.
> >>> >> > PPMC Member & Committer of Apache Stratos,
> >>> >> > Senior Software Engineer, WSO2 Inc.
> >>> >> >
> >>> >> > Blog: http://nirmalfdo.blogspot.com/
> >>> >>
> >>> >>
> >>> >>
> >>> >> --
> >>> >> Check out my professional profile and connect with me on LinkedIn.
> >>> >> http://lnkd.in/cw5k69
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > --
> >>> > Lahiru Sandaruwan
> >>> > Committer and PMC member, Apache Stratos,
> >>> > Senior Software Engineer,
> >>> > WSO2 Inc., http://wso2.com
> >>> > lean.enterprise.middleware
> >>> >
> >>> > email: lahirus@wso2.com cell: (+94) 773 325 954
> >>> > blog: http://lahiruwrites.blogspot.com/
> >>> > twitter: http://twitter.com/lahirus
> >>> > linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> Check out my professional profile and connect with me on LinkedIn.
> >>> http://lnkd.in/cw5k69
> >>
> >>
> >>
> >>
> >> --
> >> Best Regards,
> >> Nirmal
> >>
> >> Nirmal Fernando.
> >> PPMC Member & Committer of Apache Stratos,
> >> Senior Software Engineer, WSO2 Inc.
> >>
> >> Blog: http://nirmalfdo.blogspot.com/
> >
> >
> >
> >
> > --
> > Lakmal Warusawithana
> > Vice President, Apache Stratos
> > Director - Cloud Architecture; WSO2 Inc.
> > Mobile : +94714289692
> > Blog : http://lakmalsview.blogspot.com/
> >
>
>
>
> --
> Check out my professional profile and connect with me on LinkedIn.
> http://lnkd.in/cw5k69
>
--
--
Lahiru Sandaruwan
Committer and PMC member, Apache Stratos,
Senior Software Engineer,
WSO2 Inc., http://wso2.com
lean.enterprise.middleware
email: lahirus@wso2.com cell: (+94) 773 325 954
blog: http://lahiruwrites.blogspot.com/
twitter: http://twitter.com/lahirus
linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
Re: autoscaling is ignoring partiionMax
Posted by chris snow <ch...@gmail.com>.
Ok, thanks all. My agent definitely has not started, so there is the
source of the problem. Thanks for the help getting to the bottom of
this.
A few thoughts on this:
1) Conceptually, I was thinking that partitionMax was equal to the
number of IaaS instances "spawned", whereas it really means number of
"instances" connected to Stratos. Maybe we should add that note to
the wiki [1]?
2) Anyone else running into this problem on EC2 or another
subscription based IaaS could end up with a big bill because Stratos
will just keep spawning and spawning.
3) Should we put in a limit on the number of instances that Stratos
spawns that are not connected?
---
[1] https://cwiki.apache.org/confluence/display/STRATOS/4.0.0+Sample+Deployment+Policy+Definition
On Thu, May 29, 2014 at 9:06 AM, Lakmal Warusawithana <la...@wso2.com> wrote:
> Actually its sending by CC based on "Insance activated" event at Instance
> Status topic which publish by Cartridge agent.
>
>
> On Thu, May 29, 2014 at 1:31 PM, Nirmal Fernando <ni...@gmail.com>
> wrote:
>>
>> It's agent Chris.
>>
>>
>> On Thu, May 29, 2014 at 1:25 PM, chris snow <ch...@gmail.com> wrote:
>>>
>>> Hi Lahiru, are MemberActivatedEvents sent by the Stratos agent, or are
>>> they sent by the another component (e.g. CloudController)?
>>>
>>> On Thu, May 29, 2014 at 7:49 AM, Lahiru Sandaruwan <la...@wso2.com>
>>> wrote:
>>> > Hi Chris,
>>> >
>>> > Can you confirm that the Autoscaler received Member activated event for
>>> > these particular instances? Logs should help identify that.
>>> >
>>> > Thanks.
>>> >
>>> >
>>> >
>>> >
>>> > On Thu, May 29, 2014 at 12:15 PM, chris snow <ch...@gmail.com>
>>> > wrote:
>>> >>
>>> >> Thanks Nirmal.
>>> >>
>>> >> Non terminated member count doesn't seem to detect the two active
>>> >> instances that have been started:
>>> >>
>>> >> [2014-05-29 06:06:23,900] DEBUG
>>> >> {org.apache.stratos.autoscaler.rule.RuleLog} - Running minimum rule:
>>> >> [network-partition] G1 [partition] P1
>>> >> [2014-05-29 06:06:23,906] DEBUG
>>> >> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check]
>>> >> [network-partition] G1 [partition] P1 Non terminated member count: 0
>>> >> [2014-05-29 06:06:23,908] DEBUG
>>> >> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check]
>>> >> [network-partition] G1 [partition] P1 Minimum member count: 1
>>> >> [2014-05-29 06:06:23,925] DEBUG
>>> >> {org.apache.stratos.autoscaler.rule.RuleLog} - Running obsolete
>>> >> instances rule: [partition] P1 [network-partition] G1
>>> >> [2014-05-29 06:06:23,927] DEBUG
>>> >> {org.apache.stratos.autoscaler.rule.RuleLog} - [obsolete-check]
>>> >> [network-partition] G1 [partition] P1 Obsolete member count: 0
>>> >> [2014-05-29 06:06:23,930] INFO
>>> >>
>>> >>
>>> >> {org.apache.stratos.autoscaler.client.cloud.controller.CloudControllerClient}
>>> >> - Trying to spawn an instance via cloud controller: [cluster]
>>> >> php111.php.domain [partition] P1 [lb-cluster] null
>>> >> [network-partition-id] G1
>>> >>
>>> >>
>>> >> vagrant@puppet:~$ nova list
>>> >>
>>> >>
>>> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>>> >> | ID | Name | Status | Task
>>> >> State | Power State | Networks |
>>> >>
>>> >>
>>> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>>> >> | a3ce6f52-27bf-4343-a05d-d7c9da9a1cb2 | php111php-3cb | ERROR | -
>>> >> | NOSTATE | |
>>> >> | 82434b60-6c9d-40a3-8191-2d466cde725e | php111php-47e | ERROR | -
>>> >> | NOSTATE | |
>>> >> | 4aa942ff-a722-4ff4-b604-abaa2f44c1a7 | php111php-499 | ERROR | -
>>> >> | NOSTATE | |
>>> >> | 1548915b-98f6-48fc-b70a-b0acc74610a7 | php111php-5e0 | ERROR | -
>>> >> | NOSTATE | |
>>> >> | f313c93d-7de1-4ccb-8c0d-f4104e5d837f | php111php-701 | ERROR | -
>>> >> | NOSTATE | |
>>> >> | a49598bb-0d4a-4064-af16-5b4b91057fd8 | php111php-719 | ERROR | -
>>> >> | NOSTATE | |
>>> >> | f8376091-99f6-4d17-8ec6-3935291c3e40 | php111php-841 | ERROR | -
>>> >> | NOSTATE | |
>>> >> | cf3298da-eecc-448a-bcea-2296eeed23dc | php111php-ad | ACTIVE | -
>>> >> | Running | private=10.11.12.2 |
>>> >> | 1d8832d5-39e4-476c-9c2b-1b1f5ffe3b41 | php111php-d81 | ACTIVE | -
>>> >> | Running | private=10.11.12.3 |
>>> >> | 4f6d51fb-fd27-4be1-94be-e9b5200fb6ba | php111php-e9c | ERROR | -
>>> >> | NOSTATE | |
>>> >>
>>> >>
>>> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>>> >>
>>> >> I'll investigate some more...
>>> >>
>>> >> On Thu, May 29, 2014 at 7:42 AM, Nirmal Fernando
>>> >> <ni...@gmail.com>
>>> >> wrote:
>>> >> > Ah.. probably Chris. So, in min check what happens is it always make
>>> >> > sure
>>> >> > that you have min instance count in the system.
>>> >> >
>>> >> >
>>> >> > On Thu, May 29, 2014 at 11:42 AM, chris snow <ch...@gmail.com>
>>> >> > wrote:
>>> >> >>
>>> >> >> Hi Nirmal,
>>> >> >>
>>> >> >> I've just see that it is the "Minimal Rule" in mincheck.drl that is
>>> >> >> trying to spawn the instance, and not scaling.drl.
>>> >> >>
>>> >> >> This is why AutoscaleAlgorithm.getNextScaleUpPartition() is not
>>> >> >> getting
>>> >> >> called?
>>> >> >>
>>> >> >> Cheers,
>>> >> >>
>>> >> >> Chris
>>> >> >>
>>> >> >>
>>> >> >> On Wed, May 28, 2014 at 11:24 PM, Nirmal Fernando
>>> >> >> <ni...@gmail.com> wrote:
>>> >> >> > No, Chris. This happens via Drools rules located in scaling.drl
>>> >> >> > file
>>> >> >> > of
>>> >> >> > conf
>>> >> >> > directory.
>>> >> >> >
>>> >> >> >
>>> >> >> > On Thu, May 29, 2014 at 3:39 AM, chris snow <ch...@gmail.com>
>>> >> >> > wrote:
>>> >> >> >>
>>> >> >> >> Hi Nirmal,
>>> >> >> >>
>>> >> >> >> For some reason, that method is not getting called. I've pasted
>>> >> >> >> my
>>> >> >> >> stacktrace again with the drools methods removed:
>>> >> >> >>
>>> >> >> >> ---
>>> >> >> >> Daemon Thread [Thread-56] (Suspended (breakpoint at line 137 in
>>> >> >> >> CloudControllerClient))
>>> >> >> >> owns: CloudControllerClient (id=2433)
>>> >> >> >> owns: DefaultAgenda (id=2434)
>>> >> >> >> CloudControllerClient.spawnAnInstance(Partition, String,
>>> >> >> >> String,
>>> >> >> >> String) line: 137
>>> >> >> >> RuleTasksDelegator.delegateSpawn(PartitionContext, String,
>>> >> >> >> String)
>>> >> >> >> line: 92
>>> >> >> >> //
>>> >> >> >> // drools methods hidden
>>> >> >> >> //
>>> >> >> >>
>>> >> >> >>
>>> >> >> >> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
>>> >> >> >> FactHandle, Object) line: 86
>>> >> >> >> ClusterMonitor.monitor() line: 99
>>> >> >> >> ClusterMonitor.run() line: 72
>>> >> >> >> Thread.run() line: 744
>>> >> >> >> ---
>>> >> >> >>
>>> >> >> >> Would you expect to see
>>> >> >> >> AutoscaleAlgorithm.getNextScaleUpPartition
>>> >> >> >> executed somewhere in the stack of calls?
>>> >> >> >>
>>> >> >> >> Many thanks,
>>> >> >> >>
>>> >> >> >> Chris
>>> >> >> >>
>>> >> >> >> On Wed, May 28, 2014 at 11:03 PM, Nirmal Fernando
>>> >> >> >> <ni...@gmail.com> wrote:
>>> >> >> >> > Hi Chris,
>>> >> >> >> >
>>> >> >> >> > PartitionMax is considered in
>>> >> >> >> > AutoscaleAlgorithm.getNextScaleUpPartition
>>> >> >> >> > method.
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> > On Thu, May 29, 2014 at 3:09 AM, chris snow
>>> >> >> >> > <ch...@gmail.com>
>>> >> >> >> > wrote:
>>> >> >> >> >>
>>> >> >> >> >> I setup a breakpoint in eclipse at
>>> >> >> >> >> CloudControllerClient.spawnAnInstance() and dumped the output
>>> >> >> >> >> (below).
>>> >> >> >> >>
>>> >> >> >> >> I've scanned through the code in the stratos methods in the
>>> >> >> >> >> stacktrace, but couldn't see any code that checks the
>>> >> >> >> >> partitionMax
>>> >> >> >> >> value.
>>> >> >> >> >>
>>> >> >> >> >> Any ideas?
>>> >> >> >> >>
>>> >> >> >> >> Daemon Thread [Thread-56] (Suspended (breakpoint at line 137
>>> >> >> >> >> in
>>> >> >> >> >> CloudControllerClient))
>>> >> >> >> >> owns: CloudControllerClient (id=2433)
>>> >> >> >> >> owns: DefaultAgenda (id=2434)
>>> >> >> >> >> CloudControllerClient.spawnAnInstance(Partition, String,
>>> >> >> >> >> String,
>>> >> >> >> >> String) line: 137
>>> >> >> >> >> RuleTasksDelegator.delegateSpawn(PartitionContext, String,
>>> >> >> >> >> String)
>>> >> >> >> >> line: 92
>>> >> >> >> >> NativeMethodAccessorImpl.invoke0(Method, Object, Object[])
>>> >> >> >> >> line:
>>> >> >> >> >> not available [native method]
>>> >> >> >> >> NativeMethodAccessorImpl.invoke(Object, Object[]) line: 57
>>> >> >> >> >> DelegatingMethodAccessorImpl.invoke(Object, Object[])
>>> >> >> >> >> line: 43
>>> >> >> >> >> Method.invoke(Object, Object...) line: 606
>>> >> >> >> >> ReflectiveAccessorOptimizer.getMethod(Object, String,
>>> >> >> >> >> Object[],
>>> >> >> >> >> Class[], ExecutableStatement[]) line: 1104
>>> >> >> >> >> ReflectiveAccessorOptimizer.getMethod(Object, String)
>>> >> >> >> >> line:
>>> >> >> >> >> 987
>>> >> >> >> >> ReflectiveAccessorOptimizer.compileGetChain() line: 377
>>> >> >> >> >>
>>> >> >> >> >> ReflectiveAccessorOptimizer.optimizeAccessor(ParserContext,
>>> >> >> >> >> char[],
>>> >> >> >> >> int, int, Object, Object, VariableResolverFactory, boolean,
>>> >> >> >> >> Class)
>>> >> >> >> >> line: 143
>>> >> >> >> >> ASTNode.optimize(Object, Object, VariableResolverFactory)
>>> >> >> >> >> line:
>>> >> >> >> >> 159
>>> >> >> >> >> ASTNode.getReducedValueAccelerated(Object, Object,
>>> >> >> >> >> VariableResolverFactory) line: 115
>>> >> >> >> >> MVELRuntime.execute(boolean, CompiledExpression, Object,
>>> >> >> >> >> VariableResolverFactory) line: 85
>>> >> >> >> >> CompiledExpression.getDirectValue(Object,
>>> >> >> >> >> VariableResolverFactory)
>>> >> >> >> >> line: 123
>>> >> >> >> >> CompiledExpression.getValue(Object,
>>> >> >> >> >> VariableResolverFactory)
>>> >> >> >> >> line:
>>> >> >> >> >> 119
>>> >> >> >> >> MVEL.executeExpression(Object, Object,
>>> >> >> >> >> VariableResolverFactory)
>>> >> >> >> >> line:
>>> >> >> >> >> 930
>>> >> >> >> >> MVELConsequence.evaluate(KnowledgeHelper, WorkingMemory)
>>> >> >> >> >> line:
>>> >> >> >> >> 104
>>> >> >> >> >> DefaultAgenda.fireActivation(Activation) line: 1287
>>> >> >> >> >> DefaultAgenda.fireNextItem(AgendaFilter) line: 1221
>>> >> >> >> >> DefaultAgenda.fireAllRules(AgendaFilter, int) line: 1456
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules(AgendaFilter,
>>> >> >> >> >> int) line: 710
>>> >> >> >> >>
>>> >> >> >> >> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules()
>>> >> >> >> >> line:
>>> >> >> >> >> 674
>>> >> >> >> >> StatefulKnowledgeSessionImpl.fireAllRules() line: 230
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
>>> >> >> >> >> FactHandle, Object) line: 86
>>> >> >> >> >> ClusterMonitor.monitor() line: 99
>>> >> >> >> >> ClusterMonitor.run() line: 72
>>> >> >> >> >> Thread.run() line: 744
>>> >> >> >> >>
>>> >> >> >> >> On Wed, May 28, 2014 at 3:37 AM, Lahiru Sandaruwan
>>> >> >> >> >> <la...@wso2.com>
>>> >> >> >> >> wrote:
>>> >> >> >> >> >
>>> >> >> >> >> >
>>> >> >> >> >> >
>>> >> >> >> >> > On Tue, May 27, 2014 at 11:54 PM, chris snow
>>> >> >> >> >> > <ch...@gmail.com>
>>> >> >> >> >> > wrote:
>>> >> >> >> >> >>
>>> >> >> >> >> >> Hi Lahiru,
>>> >> >> >> >> >>
>>> >> >> >> >> >> I'll have a poke around - it will give me a good
>>> >> >> >> >> >> opportunity
>>> >> >> >> >> >> to
>>> >> >> >> >> >> learn
>>> >> >> >> >> >> some more Stratos code :)
>>> >> >> >> >> >>
>>> >> >> >> >> >
>>> >> >> >> >> > Cool :)
>>> >> >> >> >> >
>>> >> >> >> >> > Debug logs should be able to show us where it went wrong.
>>> >> >> >> >> >
>>> >> >> >> >> > Thanks.
>>> >> >> >> >> >>
>>> >> >> >> >> >> Many thanks,
>>> >> >> >> >> >>
>>> >> >> >> >> >> Chris
>>> >> >> >> >> >>
>>> >> >> >> >> >> On Tue, May 27, 2014 at 7:45 AM, Lahiru Sandaruwan
>>> >> >> >> >> >> <la...@wso2.com>
>>> >> >> >> >> >> wrote:
>>> >> >> >> >> >> > Hi Chris,
>>> >> >> >> >> >> >
>>> >> >> >> >> >> > Can you enable debug logs and try out?
>>> >> >> >> >> >> > If so please send them over.
>>> >> >> >> >> >> >
>>> >> >> >> >> >> > Thanks.
>>> >> >> >> >> >> >
>>> >> >> >> >> >> >
>>> >> >> >> >> >> > On Tue, May 27, 2014 at 12:05 PM, chris snow
>>> >> >> >> >> >> > <ch...@gmail.com>
>>> >> >> >> >> >> > wrote:
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >> Deployment Policy partitionMax seems to be ignored. My
>>> >> >> >> >> >> >> deployment
>>> >> >> >> >> >> >> policy looks like this:
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >> {
>>> >> >> >> >> >> >> "id": "economyDeployment",
>>> >> >> >> >> >> >> "partitionGroup": [
>>> >> >> >> >> >> >> {
>>> >> >> >> >> >> >> "id": "G1",
>>> >> >> >> >> >> >> "partitionAlgo": "one-after-another",
>>> >> >> >> >> >> >> "partition": [
>>> >> >> >> >> >> >> {
>>> >> >> >> >> >> >> "id": "P1",
>>> >> >> >> >> >> >> "partitionMax": "1",
>>> >> >> >> >> >> >> "partitionMin": "1"
>>> >> >> >> >> >> >> }
>>> >> >> >> >> >> >> ]
>>> >> >> >> >> >> >> }
>>> >> >> >> >> >> >> ]
>>> >> >> >> >> >> >> }
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >> Stratos initially starts two instances:
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>>> >> >> >> >> >> >> | ID | Name
>>> >> >> >> >> >> >> |
>>> >> >> >> >> >> >> Status |
>>> >> >> >> >> >> >> Task
>>> >> >> >> >> >> >> State | Power State | Networks |
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>>> >> >> >> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6
>>> >> >> >> >> >> >> |
>>> >> >> >> >> >> >> ACTIVE |
>>> >> >> >> >> >> >> -
>>> >> >> >> >> >> >> | Running | private=10.11.12.2 |
>>> >> >> >> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05
>>> >> >> >> >> >> >> |
>>> >> >> >> >> >> >> ACTIVE |
>>> >> >> >> >> >> >> -
>>> >> >> >> >> >> >> | Running | private=10.11.12.3 |
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>>> >> >> >> >> >> >> Then attempts to start more:
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>>> >> >> >> >> >> >> | ID | Name
>>> >> >> >> >> >> >> |
>>> >> >> >> >> >> >> Status |
>>> >> >> >> >> >> >> Task
>>> >> >> >> >> >> >> State | Power State | Networks |
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>>> >> >> >> >> >> >> | 91029130-37b5-4df6-ad95-1d9f609a22c3 | php111php-13e
>>> >> >> >> >> >> >> |
>>> >> >> >> >> >> >> ERROR
>>> >> >> >> >> >> >> |
>>> >> >> >> >> >> >> -
>>> >> >> >> >> >> >> | NOSTATE | |
>>> >> >> >> >> >> >> | 71c1e361-df55-4f61-84c5-79774fef2beb | php111php-3d7
>>> >> >> >> >> >> >> |
>>> >> >> >> >> >> >> ERROR
>>> >> >> >> >> >> >> |
>>> >> >> >> >> >> >> -
>>> >> >> >> >> >> >> | NOSTATE | |
>>> >> >> >> >> >> >> | b0d72602-cdb8-4621-ad12-911ade4d89b1 | php111php-412
>>> >> >> >> >> >> >> |
>>> >> >> >> >> >> >> ERROR
>>> >> >> >> >> >> >> |
>>> >> >> >> >> >> >> -
>>> >> >> >> >> >> >> | NOSTATE | |
>>> >> >> >> >> >> >> | d4eac55d-bb7c-48d6-83f2-cb9a11581a18 | php111php-6f5
>>> >> >> >> >> >> >> |
>>> >> >> >> >> >> >> ERROR
>>> >> >> >> >> >> >> |
>>> >> >> >> >> >> >> -
>>> >> >> >> >> >> >> | NOSTATE | |
>>> >> >> >> >> >> >> | 738a9534-2b95-4984-a51b-748bc7169b9a | php111php-73e
>>> >> >> >> >> >> >> |
>>> >> >> >> >> >> >> ERROR
>>> >> >> >> >> >> >> |
>>> >> >> >> >> >> >> -
>>> >> >> >> >> >> >> | NOSTATE | |
>>> >> >> >> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6
>>> >> >> >> >> >> >> |
>>> >> >> >> >> >> >> ACTIVE |
>>> >> >> >> >> >> >> -
>>> >> >> >> >> >> >> | Running | private=10.11.12.2 |
>>> >> >> >> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05
>>> >> >> >> >> >> >> |
>>> >> >> >> >> >> >> ACTIVE |
>>> >> >> >> >> >> >> -
>>> >> >> >> >> >> >> | Running | private=10.11.12.3 |
>>> >> >> >> >> >> >> | f570e654-024b-4240-bad2-8a11f04f23f0 | php111php-a30
>>> >> >> >> >> >> >> |
>>> >> >> >> >> >> >> ERROR
>>> >> >> >> >> >> >> |
>>> >> >> >> >> >> >> -
>>> >> >> >> >> >> >> | NOSTATE | |
>>> >> >> >> >> >> >> | 0ab36824-9bea-4a95-aa22-7f935406d83d | php111php-bca
>>> >> >> >> >> >> >> |
>>> >> >> >> >> >> >> ERROR
>>> >> >> >> >> >> >> |
>>> >> >> >> >> >> >> -
>>> >> >> >> >> >> >> | NOSTATE | |
>>> >> >> >> >> >> >> | 02ed97df-02bc-4bff-b42d-9f2bcbd4f569 | php111php-f6f
>>> >> >> >> >> >> >> |
>>> >> >> >> >> >> >> ERROR
>>> >> >> >> >> >> >> |
>>> >> >> >> >> >> >> -
>>> >> >> >> >> >> >> | NOSTATE | |
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >> I'm on branch: 4.0.0-incubating, using Openstack Havana
>>> >> >> >> >> >> >> +
>>> >> >> >> >> >> >> Docker.
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >> Any ideas?
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >> Many thanks,
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >> Chris
>>> >> >> >> >> >> >
>>> >> >> >> >> >> >
>>> >> >> >> >> >> >
>>> >> >> >> >> >> >
>>> >> >> >> >> >> > --
>>> >> >> >> >> >> > --
>>> >> >> >> >> >> > Lahiru Sandaruwan
>>> >> >> >> >> >> > Committer and PMC member, Apache Stratos,
>>> >> >> >> >> >> > Senior Software Engineer,
>>> >> >> >> >> >> > WSO2 Inc., http://wso2.com
>>> >> >> >> >> >> > lean.enterprise.middleware
>>> >> >> >> >> >> >
>>> >> >> >> >> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
>>> >> >> >> >> >> > blog: http://lahiruwrites.blogspot.com/
>>> >> >> >> >> >> > twitter: http://twitter.com/lahirus
>>> >> >> >> >> >> > linked-in:
>>> >> >> >> >> >> > http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>> >> >> >> >> >> >
>>> >> >> >> >> >>
>>> >> >> >> >> >>
>>> >> >> >> >> >>
>>> >> >> >> >> >> --
>>> >> >> >> >> >> Check out my professional profile and connect with me on
>>> >> >> >> >> >> LinkedIn.
>>> >> >> >> >> >> http://lnkd.in/cw5k69
>>> >> >> >> >> >
>>> >> >> >> >> >
>>> >> >> >> >> >
>>> >> >> >> >> >
>>> >> >> >> >> > --
>>> >> >> >> >> > --
>>> >> >> >> >> > Lahiru Sandaruwan
>>> >> >> >> >> > Committer and PMC member, Apache Stratos,
>>> >> >> >> >> > Senior Software Engineer,
>>> >> >> >> >> > WSO2 Inc., http://wso2.com
>>> >> >> >> >> > lean.enterprise.middleware
>>> >> >> >> >> >
>>> >> >> >> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
>>> >> >> >> >> > blog: http://lahiruwrites.blogspot.com/
>>> >> >> >> >> > twitter: http://twitter.com/lahirus
>>> >> >> >> >> > linked-in:
>>> >> >> >> >> > http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>> >> >> >> >> >
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >> --
>>> >> >> >> >> Check out my professional profile and connect with me on
>>> >> >> >> >> LinkedIn.
>>> >> >> >> >> http://lnkd.in/cw5k69
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> > --
>>> >> >> >> > Best Regards,
>>> >> >> >> > Nirmal
>>> >> >> >> >
>>> >> >> >> > Nirmal Fernando.
>>> >> >> >> > PPMC Member & Committer of Apache Stratos,
>>> >> >> >> >
>>> >> >> >> > Senior Software Engineer, WSO2 Inc.
>>> >> >> >> >
>>> >> >> >> > Blog: http://nirmalfdo.blogspot.com/
>>> >> >> >>
>>> >> >> >>
>>> >> >> >>
>>> >> >> >> --
>>> >> >> >> Check out my professional profile and connect with me on
>>> >> >> >> LinkedIn.
>>> >> >> >> http://lnkd.in/cw5k69
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> > --
>>> >> >> > Best Regards,
>>> >> >> > Nirmal
>>> >> >> >
>>> >> >> > Nirmal Fernando.
>>> >> >> > PPMC Member & Committer of Apache Stratos,
>>> >> >> > Senior Software Engineer, WSO2 Inc.
>>> >> >> >
>>> >> >> > Blog: http://nirmalfdo.blogspot.com/
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> --
>>> >> >> Check out my professional profile and connect with me on LinkedIn.
>>> >> >> http://lnkd.in/cw5k69
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> > --
>>> >> > Best Regards,
>>> >> > Nirmal
>>> >> >
>>> >> > Nirmal Fernando.
>>> >> > PPMC Member & Committer of Apache Stratos,
>>> >> > Senior Software Engineer, WSO2 Inc.
>>> >> >
>>> >> > Blog: http://nirmalfdo.blogspot.com/
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Check out my professional profile and connect with me on LinkedIn.
>>> >> http://lnkd.in/cw5k69
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > --
>>> > Lahiru Sandaruwan
>>> > Committer and PMC member, Apache Stratos,
>>> > Senior Software Engineer,
>>> > WSO2 Inc., http://wso2.com
>>> > lean.enterprise.middleware
>>> >
>>> > email: lahirus@wso2.com cell: (+94) 773 325 954
>>> > blog: http://lahiruwrites.blogspot.com/
>>> > twitter: http://twitter.com/lahirus
>>> > linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>> >
>>>
>>>
>>>
>>> --
>>> Check out my professional profile and connect with me on LinkedIn.
>>> http://lnkd.in/cw5k69
>>
>>
>>
>>
>> --
>> Best Regards,
>> Nirmal
>>
>> Nirmal Fernando.
>> PPMC Member & Committer of Apache Stratos,
>> Senior Software Engineer, WSO2 Inc.
>>
>> Blog: http://nirmalfdo.blogspot.com/
>
>
>
>
> --
> Lakmal Warusawithana
> Vice President, Apache Stratos
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
--
Check out my professional profile and connect with me on LinkedIn.
http://lnkd.in/cw5k69
Re: autoscaling is ignoring partiionMax
Posted by Lakmal Warusawithana <la...@wso2.com>.
Actually its sending by CC based on "Insance activated" event at Instance
Status topic which publish by Cartridge agent.
On Thu, May 29, 2014 at 1:31 PM, Nirmal Fernando <ni...@gmail.com>
wrote:
> It's agent Chris.
>
>
> On Thu, May 29, 2014 at 1:25 PM, chris snow <ch...@gmail.com> wrote:
>
>> Hi Lahiru, are MemberActivatedEvents sent by the Stratos agent, or are
>> they sent by the another component (e.g. CloudController)?
>>
>> On Thu, May 29, 2014 at 7:49 AM, Lahiru Sandaruwan <la...@wso2.com>
>> wrote:
>> > Hi Chris,
>> >
>> > Can you confirm that the Autoscaler received Member activated event for
>> > these particular instances? Logs should help identify that.
>> >
>> > Thanks.
>> >
>> >
>> >
>> >
>> > On Thu, May 29, 2014 at 12:15 PM, chris snow <ch...@gmail.com>
>> wrote:
>> >>
>> >> Thanks Nirmal.
>> >>
>> >> Non terminated member count doesn't seem to detect the two active
>> >> instances that have been started:
>> >>
>> >> [2014-05-29 06:06:23,900] DEBUG
>> >> {org.apache.stratos.autoscaler.rule.RuleLog} - Running minimum rule:
>> >> [network-partition] G1 [partition] P1
>> >> [2014-05-29 06:06:23,906] DEBUG
>> >> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check]
>> >> [network-partition] G1 [partition] P1 Non terminated member count: 0
>> >> [2014-05-29 06:06:23,908] DEBUG
>> >> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check]
>> >> [network-partition] G1 [partition] P1 Minimum member count: 1
>> >> [2014-05-29 06:06:23,925] DEBUG
>> >> {org.apache.stratos.autoscaler.rule.RuleLog} - Running obsolete
>> >> instances rule: [partition] P1 [network-partition] G1
>> >> [2014-05-29 06:06:23,927] DEBUG
>> >> {org.apache.stratos.autoscaler.rule.RuleLog} - [obsolete-check]
>> >> [network-partition] G1 [partition] P1 Obsolete member count: 0
>> >> [2014-05-29 06:06:23,930] INFO
>> >>
>> >>
>> {org.apache.stratos.autoscaler.client.cloud.controller.CloudControllerClient}
>> >> - Trying to spawn an instance via cloud controller: [cluster]
>> >> php111.php.domain [partition] P1 [lb-cluster] null
>> >> [network-partition-id] G1
>> >>
>> >>
>> >> vagrant@puppet:~$ nova list
>> >>
>> >>
>> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> | ID | Name | Status | Task
>> >> State | Power State | Networks |
>> >>
>> >>
>> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> | a3ce6f52-27bf-4343-a05d-d7c9da9a1cb2 | php111php-3cb | ERROR | -
>> >> | NOSTATE | |
>> >> | 82434b60-6c9d-40a3-8191-2d466cde725e | php111php-47e | ERROR | -
>> >> | NOSTATE | |
>> >> | 4aa942ff-a722-4ff4-b604-abaa2f44c1a7 | php111php-499 | ERROR | -
>> >> | NOSTATE | |
>> >> | 1548915b-98f6-48fc-b70a-b0acc74610a7 | php111php-5e0 | ERROR | -
>> >> | NOSTATE | |
>> >> | f313c93d-7de1-4ccb-8c0d-f4104e5d837f | php111php-701 | ERROR | -
>> >> | NOSTATE | |
>> >> | a49598bb-0d4a-4064-af16-5b4b91057fd8 | php111php-719 | ERROR | -
>> >> | NOSTATE | |
>> >> | f8376091-99f6-4d17-8ec6-3935291c3e40 | php111php-841 | ERROR | -
>> >> | NOSTATE | |
>> >> | cf3298da-eecc-448a-bcea-2296eeed23dc | php111php-ad | ACTIVE | -
>> >> | Running | private=10.11.12.2 |
>> >> | 1d8832d5-39e4-476c-9c2b-1b1f5ffe3b41 | php111php-d81 | ACTIVE | -
>> >> | Running | private=10.11.12.3 |
>> >> | 4f6d51fb-fd27-4be1-94be-e9b5200fb6ba | php111php-e9c | ERROR | -
>> >> | NOSTATE | |
>> >>
>> >>
>> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >>
>> >> I'll investigate some more...
>> >>
>> >> On Thu, May 29, 2014 at 7:42 AM, Nirmal Fernando <
>> nirmal070125@gmail.com>
>> >> wrote:
>> >> > Ah.. probably Chris. So, in min check what happens is it always make
>> >> > sure
>> >> > that you have min instance count in the system.
>> >> >
>> >> >
>> >> > On Thu, May 29, 2014 at 11:42 AM, chris snow <ch...@gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> Hi Nirmal,
>> >> >>
>> >> >> I've just see that it is the "Minimal Rule" in mincheck.drl that is
>> >> >> trying to spawn the instance, and not scaling.drl.
>> >> >>
>> >> >> This is why AutoscaleAlgorithm.getNextScaleUpPartition() is not
>> >> >> getting
>> >> >> called?
>> >> >>
>> >> >> Cheers,
>> >> >>
>> >> >> Chris
>> >> >>
>> >> >>
>> >> >> On Wed, May 28, 2014 at 11:24 PM, Nirmal Fernando
>> >> >> <ni...@gmail.com> wrote:
>> >> >> > No, Chris. This happens via Drools rules located in scaling.drl
>> file
>> >> >> > of
>> >> >> > conf
>> >> >> > directory.
>> >> >> >
>> >> >> >
>> >> >> > On Thu, May 29, 2014 at 3:39 AM, chris snow <ch...@gmail.com>
>> >> >> > wrote:
>> >> >> >>
>> >> >> >> Hi Nirmal,
>> >> >> >>
>> >> >> >> For some reason, that method is not getting called. I've pasted
>> my
>> >> >> >> stacktrace again with the drools methods removed:
>> >> >> >>
>> >> >> >> ---
>> >> >> >> Daemon Thread [Thread-56] (Suspended (breakpoint at line 137 in
>> >> >> >> CloudControllerClient))
>> >> >> >> owns: CloudControllerClient (id=2433)
>> >> >> >> owns: DefaultAgenda (id=2434)
>> >> >> >> CloudControllerClient.spawnAnInstance(Partition, String,
>> String,
>> >> >> >> String) line: 137
>> >> >> >> RuleTasksDelegator.delegateSpawn(PartitionContext, String,
>> >> >> >> String)
>> >> >> >> line: 92
>> >> >> >> //
>> >> >> >> // drools methods hidden
>> >> >> >> //
>> >> >> >>
>> >> >> >>
>> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
>> >> >> >> FactHandle, Object) line: 86
>> >> >> >> ClusterMonitor.monitor() line: 99
>> >> >> >> ClusterMonitor.run() line: 72
>> >> >> >> Thread.run() line: 744
>> >> >> >> ---
>> >> >> >>
>> >> >> >> Would you expect to see
>> AutoscaleAlgorithm.getNextScaleUpPartition
>> >> >> >> executed somewhere in the stack of calls?
>> >> >> >>
>> >> >> >> Many thanks,
>> >> >> >>
>> >> >> >> Chris
>> >> >> >>
>> >> >> >> On Wed, May 28, 2014 at 11:03 PM, Nirmal Fernando
>> >> >> >> <ni...@gmail.com> wrote:
>> >> >> >> > Hi Chris,
>> >> >> >> >
>> >> >> >> > PartitionMax is considered in
>> >> >> >> > AutoscaleAlgorithm.getNextScaleUpPartition
>> >> >> >> > method.
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > On Thu, May 29, 2014 at 3:09 AM, chris snow <
>> chsnow123@gmail.com>
>> >> >> >> > wrote:
>> >> >> >> >>
>> >> >> >> >> I setup a breakpoint in eclipse at
>> >> >> >> >> CloudControllerClient.spawnAnInstance() and dumped the output
>> >> >> >> >> (below).
>> >> >> >> >>
>> >> >> >> >> I've scanned through the code in the stratos methods in the
>> >> >> >> >> stacktrace, but couldn't see any code that checks the
>> >> >> >> >> partitionMax
>> >> >> >> >> value.
>> >> >> >> >>
>> >> >> >> >> Any ideas?
>> >> >> >> >>
>> >> >> >> >> Daemon Thread [Thread-56] (Suspended (breakpoint at line 137
>> in
>> >> >> >> >> CloudControllerClient))
>> >> >> >> >> owns: CloudControllerClient (id=2433)
>> >> >> >> >> owns: DefaultAgenda (id=2434)
>> >> >> >> >> CloudControllerClient.spawnAnInstance(Partition, String,
>> >> >> >> >> String,
>> >> >> >> >> String) line: 137
>> >> >> >> >> RuleTasksDelegator.delegateSpawn(PartitionContext, String,
>> >> >> >> >> String)
>> >> >> >> >> line: 92
>> >> >> >> >> NativeMethodAccessorImpl.invoke0(Method, Object, Object[])
>> >> >> >> >> line:
>> >> >> >> >> not available [native method]
>> >> >> >> >> NativeMethodAccessorImpl.invoke(Object, Object[]) line: 57
>> >> >> >> >> DelegatingMethodAccessorImpl.invoke(Object, Object[])
>> line: 43
>> >> >> >> >> Method.invoke(Object, Object...) line: 606
>> >> >> >> >> ReflectiveAccessorOptimizer.getMethod(Object, String,
>> >> >> >> >> Object[],
>> >> >> >> >> Class[], ExecutableStatement[]) line: 1104
>> >> >> >> >> ReflectiveAccessorOptimizer.getMethod(Object, String) line:
>> >> >> >> >> 987
>> >> >> >> >> ReflectiveAccessorOptimizer.compileGetChain() line: 377
>> >> >> >> >> ReflectiveAccessorOptimizer.optimizeAccessor(ParserContext,
>> >> >> >> >> char[],
>> >> >> >> >> int, int, Object, Object, VariableResolverFactory, boolean,
>> >> >> >> >> Class)
>> >> >> >> >> line: 143
>> >> >> >> >> ASTNode.optimize(Object, Object, VariableResolverFactory)
>> >> >> >> >> line:
>> >> >> >> >> 159
>> >> >> >> >> ASTNode.getReducedValueAccelerated(Object, Object,
>> >> >> >> >> VariableResolverFactory) line: 115
>> >> >> >> >> MVELRuntime.execute(boolean, CompiledExpression, Object,
>> >> >> >> >> VariableResolverFactory) line: 85
>> >> >> >> >> CompiledExpression.getDirectValue(Object,
>> >> >> >> >> VariableResolverFactory)
>> >> >> >> >> line: 123
>> >> >> >> >> CompiledExpression.getValue(Object,
>> VariableResolverFactory)
>> >> >> >> >> line:
>> >> >> >> >> 119
>> >> >> >> >> MVEL.executeExpression(Object, Object,
>> >> >> >> >> VariableResolverFactory)
>> >> >> >> >> line:
>> >> >> >> >> 930
>> >> >> >> >> MVELConsequence.evaluate(KnowledgeHelper, WorkingMemory)
>> line:
>> >> >> >> >> 104
>> >> >> >> >> DefaultAgenda.fireActivation(Activation) line: 1287
>> >> >> >> >> DefaultAgenda.fireNextItem(AgendaFilter) line: 1221
>> >> >> >> >> DefaultAgenda.fireAllRules(AgendaFilter, int) line: 1456
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules(AgendaFilter,
>> >> >> >> >> int) line: 710
>> >> >> >> >> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules()
>> >> >> >> >> line:
>> >> >> >> >> 674
>> >> >> >> >> StatefulKnowledgeSessionImpl.fireAllRules() line: 230
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
>> >> >> >> >> FactHandle, Object) line: 86
>> >> >> >> >> ClusterMonitor.monitor() line: 99
>> >> >> >> >> ClusterMonitor.run() line: 72
>> >> >> >> >> Thread.run() line: 744
>> >> >> >> >>
>> >> >> >> >> On Wed, May 28, 2014 at 3:37 AM, Lahiru Sandaruwan
>> >> >> >> >> <la...@wso2.com>
>> >> >> >> >> wrote:
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > On Tue, May 27, 2014 at 11:54 PM, chris snow
>> >> >> >> >> > <ch...@gmail.com>
>> >> >> >> >> > wrote:
>> >> >> >> >> >>
>> >> >> >> >> >> Hi Lahiru,
>> >> >> >> >> >>
>> >> >> >> >> >> I'll have a poke around - it will give me a good
>> opportunity
>> >> >> >> >> >> to
>> >> >> >> >> >> learn
>> >> >> >> >> >> some more Stratos code :)
>> >> >> >> >> >>
>> >> >> >> >> >
>> >> >> >> >> > Cool :)
>> >> >> >> >> >
>> >> >> >> >> > Debug logs should be able to show us where it went wrong.
>> >> >> >> >> >
>> >> >> >> >> > Thanks.
>> >> >> >> >> >>
>> >> >> >> >> >> Many thanks,
>> >> >> >> >> >>
>> >> >> >> >> >> Chris
>> >> >> >> >> >>
>> >> >> >> >> >> On Tue, May 27, 2014 at 7:45 AM, Lahiru Sandaruwan
>> >> >> >> >> >> <la...@wso2.com>
>> >> >> >> >> >> wrote:
>> >> >> >> >> >> > Hi Chris,
>> >> >> >> >> >> >
>> >> >> >> >> >> > Can you enable debug logs and try out?
>> >> >> >> >> >> > If so please send them over.
>> >> >> >> >> >> >
>> >> >> >> >> >> > Thanks.
>> >> >> >> >> >> >
>> >> >> >> >> >> >
>> >> >> >> >> >> > On Tue, May 27, 2014 at 12:05 PM, chris snow
>> >> >> >> >> >> > <ch...@gmail.com>
>> >> >> >> >> >> > wrote:
>> >> >> >> >> >> >>
>> >> >> >> >> >> >> Deployment Policy partitionMax seems to be ignored. My
>> >> >> >> >> >> >> deployment
>> >> >> >> >> >> >> policy looks like this:
>> >> >> >> >> >> >>
>> >> >> >> >> >> >> {
>> >> >> >> >> >> >> "id": "economyDeployment",
>> >> >> >> >> >> >> "partitionGroup": [
>> >> >> >> >> >> >> {
>> >> >> >> >> >> >> "id": "G1",
>> >> >> >> >> >> >> "partitionAlgo": "one-after-another",
>> >> >> >> >> >> >> "partition": [
>> >> >> >> >> >> >> {
>> >> >> >> >> >> >> "id": "P1",
>> >> >> >> >> >> >> "partitionMax": "1",
>> >> >> >> >> >> >> "partitionMin": "1"
>> >> >> >> >> >> >> }
>> >> >> >> >> >> >> ]
>> >> >> >> >> >> >> }
>> >> >> >> >> >> >> ]
>> >> >> >> >> >> >> }
>> >> >> >> >> >> >>
>> >> >> >> >> >> >> Stratos initially starts two instances:
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> >> >> >> >> | ID | Name |
>> >> >> >> >> >> >> Status |
>> >> >> >> >> >> >> Task
>> >> >> >> >> >> >> State | Power State | Networks |
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> >> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 |
>> >> >> >> >> >> >> ACTIVE |
>> >> >> >> >> >> >> -
>> >> >> >> >> >> >> | Running | private=10.11.12.2 |
>> >> >> >> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 |
>> >> >> >> >> >> >> ACTIVE |
>> >> >> >> >> >> >> -
>> >> >> >> >> >> >> | Running | private=10.11.12.3 |
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> >> >> >> >> Then attempts to start more:
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> >> >> >> >> | ID | Name |
>> >> >> >> >> >> >> Status |
>> >> >> >> >> >> >> Task
>> >> >> >> >> >> >> State | Power State | Networks |
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> >> >> >> >> | 91029130-37b5-4df6-ad95-1d9f609a22c3 | php111php-13e |
>> >> >> >> >> >> >> ERROR
>> >> >> >> >> >> >> |
>> >> >> >> >> >> >> -
>> >> >> >> >> >> >> | NOSTATE | |
>> >> >> >> >> >> >> | 71c1e361-df55-4f61-84c5-79774fef2beb | php111php-3d7 |
>> >> >> >> >> >> >> ERROR
>> >> >> >> >> >> >> |
>> >> >> >> >> >> >> -
>> >> >> >> >> >> >> | NOSTATE | |
>> >> >> >> >> >> >> | b0d72602-cdb8-4621-ad12-911ade4d89b1 | php111php-412 |
>> >> >> >> >> >> >> ERROR
>> >> >> >> >> >> >> |
>> >> >> >> >> >> >> -
>> >> >> >> >> >> >> | NOSTATE | |
>> >> >> >> >> >> >> | d4eac55d-bb7c-48d6-83f2-cb9a11581a18 | php111php-6f5 |
>> >> >> >> >> >> >> ERROR
>> >> >> >> >> >> >> |
>> >> >> >> >> >> >> -
>> >> >> >> >> >> >> | NOSTATE | |
>> >> >> >> >> >> >> | 738a9534-2b95-4984-a51b-748bc7169b9a | php111php-73e |
>> >> >> >> >> >> >> ERROR
>> >> >> >> >> >> >> |
>> >> >> >> >> >> >> -
>> >> >> >> >> >> >> | NOSTATE | |
>> >> >> >> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 |
>> >> >> >> >> >> >> ACTIVE |
>> >> >> >> >> >> >> -
>> >> >> >> >> >> >> | Running | private=10.11.12.2 |
>> >> >> >> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 |
>> >> >> >> >> >> >> ACTIVE |
>> >> >> >> >> >> >> -
>> >> >> >> >> >> >> | Running | private=10.11.12.3 |
>> >> >> >> >> >> >> | f570e654-024b-4240-bad2-8a11f04f23f0 | php111php-a30 |
>> >> >> >> >> >> >> ERROR
>> >> >> >> >> >> >> |
>> >> >> >> >> >> >> -
>> >> >> >> >> >> >> | NOSTATE | |
>> >> >> >> >> >> >> | 0ab36824-9bea-4a95-aa22-7f935406d83d | php111php-bca |
>> >> >> >> >> >> >> ERROR
>> >> >> >> >> >> >> |
>> >> >> >> >> >> >> -
>> >> >> >> >> >> >> | NOSTATE | |
>> >> >> >> >> >> >> | 02ed97df-02bc-4bff-b42d-9f2bcbd4f569 | php111php-f6f |
>> >> >> >> >> >> >> ERROR
>> >> >> >> >> >> >> |
>> >> >> >> >> >> >> -
>> >> >> >> >> >> >> | NOSTATE | |
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> >> >> >> >>
>> >> >> >> >> >> >> I'm on branch: 4.0.0-incubating, using Openstack Havana
>> +
>> >> >> >> >> >> >> Docker.
>> >> >> >> >> >> >>
>> >> >> >> >> >> >> Any ideas?
>> >> >> >> >> >> >>
>> >> >> >> >> >> >> Many thanks,
>> >> >> >> >> >> >>
>> >> >> >> >> >> >> Chris
>> >> >> >> >> >> >
>> >> >> >> >> >> >
>> >> >> >> >> >> >
>> >> >> >> >> >> >
>> >> >> >> >> >> > --
>> >> >> >> >> >> > --
>> >> >> >> >> >> > Lahiru Sandaruwan
>> >> >> >> >> >> > Committer and PMC member, Apache Stratos,
>> >> >> >> >> >> > Senior Software Engineer,
>> >> >> >> >> >> > WSO2 Inc., http://wso2.com
>> >> >> >> >> >> > lean.enterprise.middleware
>> >> >> >> >> >> >
>> >> >> >> >> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
>> >> >> >> >> >> > blog: http://lahiruwrites.blogspot.com/
>> >> >> >> >> >> > twitter: http://twitter.com/lahirus
>> >> >> >> >> >> > linked-in:
>> >> >> >> >> >> > http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>> >> >> >> >> >> >
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >> --
>> >> >> >> >> >> Check out my professional profile and connect with me on
>> >> >> >> >> >> LinkedIn.
>> >> >> >> >> >> http://lnkd.in/cw5k69
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > --
>> >> >> >> >> > --
>> >> >> >> >> > Lahiru Sandaruwan
>> >> >> >> >> > Committer and PMC member, Apache Stratos,
>> >> >> >> >> > Senior Software Engineer,
>> >> >> >> >> > WSO2 Inc., http://wso2.com
>> >> >> >> >> > lean.enterprise.middleware
>> >> >> >> >> >
>> >> >> >> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
>> >> >> >> >> > blog: http://lahiruwrites.blogspot.com/
>> >> >> >> >> > twitter: http://twitter.com/lahirus
>> >> >> >> >> > linked-in:
>> >> >> >> >> > http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>> >> >> >> >> >
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> --
>> >> >> >> >> Check out my professional profile and connect with me on
>> >> >> >> >> LinkedIn.
>> >> >> >> >> http://lnkd.in/cw5k69
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > --
>> >> >> >> > Best Regards,
>> >> >> >> > Nirmal
>> >> >> >> >
>> >> >> >> > Nirmal Fernando.
>> >> >> >> > PPMC Member & Committer of Apache Stratos,
>> >> >> >> >
>> >> >> >> > Senior Software Engineer, WSO2 Inc.
>> >> >> >> >
>> >> >> >> > Blog: http://nirmalfdo.blogspot.com/
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> --
>> >> >> >> Check out my professional profile and connect with me on
>> LinkedIn.
>> >> >> >> http://lnkd.in/cw5k69
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > --
>> >> >> > Best Regards,
>> >> >> > Nirmal
>> >> >> >
>> >> >> > Nirmal Fernando.
>> >> >> > PPMC Member & Committer of Apache Stratos,
>> >> >> > Senior Software Engineer, WSO2 Inc.
>> >> >> >
>> >> >> > Blog: http://nirmalfdo.blogspot.com/
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Check out my professional profile and connect with me on LinkedIn.
>> >> >> http://lnkd.in/cw5k69
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Best Regards,
>> >> > Nirmal
>> >> >
>> >> > Nirmal Fernando.
>> >> > PPMC Member & Committer of Apache Stratos,
>> >> > Senior Software Engineer, WSO2 Inc.
>> >> >
>> >> > Blog: http://nirmalfdo.blogspot.com/
>> >>
>> >>
>> >>
>> >> --
>> >> Check out my professional profile and connect with me on LinkedIn.
>> >> http://lnkd.in/cw5k69
>> >
>> >
>> >
>> >
>> > --
>> > --
>> > Lahiru Sandaruwan
>> > Committer and PMC member, Apache Stratos,
>> > Senior Software Engineer,
>> > WSO2 Inc., http://wso2.com
>> > lean.enterprise.middleware
>> >
>> > email: lahirus@wso2.com cell: (+94) 773 325 954
>> > blog: http://lahiruwrites.blogspot.com/
>> > twitter: http://twitter.com/lahirus
>> > linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>> >
>>
>>
>>
>> --
>> Check out my professional profile and connect with me on LinkedIn.
>> http://lnkd.in/cw5k69
>>
>
>
>
> --
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
> Blog: http://nirmalfdo.blogspot.com/
>
--
Lakmal Warusawithana
Vice President, Apache Stratos
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692
Blog : http://lakmalsview.blogspot.com/
Re: autoscaling is ignoring partiionMax
Posted by Nirmal Fernando <ni...@gmail.com>.
It's agent Chris.
On Thu, May 29, 2014 at 1:25 PM, chris snow <ch...@gmail.com> wrote:
> Hi Lahiru, are MemberActivatedEvents sent by the Stratos agent, or are
> they sent by the another component (e.g. CloudController)?
>
> On Thu, May 29, 2014 at 7:49 AM, Lahiru Sandaruwan <la...@wso2.com>
> wrote:
> > Hi Chris,
> >
> > Can you confirm that the Autoscaler received Member activated event for
> > these particular instances? Logs should help identify that.
> >
> > Thanks.
> >
> >
> >
> >
> > On Thu, May 29, 2014 at 12:15 PM, chris snow <ch...@gmail.com>
> wrote:
> >>
> >> Thanks Nirmal.
> >>
> >> Non terminated member count doesn't seem to detect the two active
> >> instances that have been started:
> >>
> >> [2014-05-29 06:06:23,900] DEBUG
> >> {org.apache.stratos.autoscaler.rule.RuleLog} - Running minimum rule:
> >> [network-partition] G1 [partition] P1
> >> [2014-05-29 06:06:23,906] DEBUG
> >> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check]
> >> [network-partition] G1 [partition] P1 Non terminated member count: 0
> >> [2014-05-29 06:06:23,908] DEBUG
> >> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check]
> >> [network-partition] G1 [partition] P1 Minimum member count: 1
> >> [2014-05-29 06:06:23,925] DEBUG
> >> {org.apache.stratos.autoscaler.rule.RuleLog} - Running obsolete
> >> instances rule: [partition] P1 [network-partition] G1
> >> [2014-05-29 06:06:23,927] DEBUG
> >> {org.apache.stratos.autoscaler.rule.RuleLog} - [obsolete-check]
> >> [network-partition] G1 [partition] P1 Obsolete member count: 0
> >> [2014-05-29 06:06:23,930] INFO
> >>
> >>
> {org.apache.stratos.autoscaler.client.cloud.controller.CloudControllerClient}
> >> - Trying to spawn an instance via cloud controller: [cluster]
> >> php111.php.domain [partition] P1 [lb-cluster] null
> >> [network-partition-id] G1
> >>
> >>
> >> vagrant@puppet:~$ nova list
> >>
> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> | ID | Name | Status | Task
> >> State | Power State | Networks |
> >>
> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> | a3ce6f52-27bf-4343-a05d-d7c9da9a1cb2 | php111php-3cb | ERROR | -
> >> | NOSTATE | |
> >> | 82434b60-6c9d-40a3-8191-2d466cde725e | php111php-47e | ERROR | -
> >> | NOSTATE | |
> >> | 4aa942ff-a722-4ff4-b604-abaa2f44c1a7 | php111php-499 | ERROR | -
> >> | NOSTATE | |
> >> | 1548915b-98f6-48fc-b70a-b0acc74610a7 | php111php-5e0 | ERROR | -
> >> | NOSTATE | |
> >> | f313c93d-7de1-4ccb-8c0d-f4104e5d837f | php111php-701 | ERROR | -
> >> | NOSTATE | |
> >> | a49598bb-0d4a-4064-af16-5b4b91057fd8 | php111php-719 | ERROR | -
> >> | NOSTATE | |
> >> | f8376091-99f6-4d17-8ec6-3935291c3e40 | php111php-841 | ERROR | -
> >> | NOSTATE | |
> >> | cf3298da-eecc-448a-bcea-2296eeed23dc | php111php-ad | ACTIVE | -
> >> | Running | private=10.11.12.2 |
> >> | 1d8832d5-39e4-476c-9c2b-1b1f5ffe3b41 | php111php-d81 | ACTIVE | -
> >> | Running | private=10.11.12.3 |
> >> | 4f6d51fb-fd27-4be1-94be-e9b5200fb6ba | php111php-e9c | ERROR | -
> >> | NOSTATE | |
> >>
> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >>
> >> I'll investigate some more...
> >>
> >> On Thu, May 29, 2014 at 7:42 AM, Nirmal Fernando <
> nirmal070125@gmail.com>
> >> wrote:
> >> > Ah.. probably Chris. So, in min check what happens is it always make
> >> > sure
> >> > that you have min instance count in the system.
> >> >
> >> >
> >> > On Thu, May 29, 2014 at 11:42 AM, chris snow <ch...@gmail.com>
> >> > wrote:
> >> >>
> >> >> Hi Nirmal,
> >> >>
> >> >> I've just see that it is the "Minimal Rule" in mincheck.drl that is
> >> >> trying to spawn the instance, and not scaling.drl.
> >> >>
> >> >> This is why AutoscaleAlgorithm.getNextScaleUpPartition() is not
> >> >> getting
> >> >> called?
> >> >>
> >> >> Cheers,
> >> >>
> >> >> Chris
> >> >>
> >> >>
> >> >> On Wed, May 28, 2014 at 11:24 PM, Nirmal Fernando
> >> >> <ni...@gmail.com> wrote:
> >> >> > No, Chris. This happens via Drools rules located in scaling.drl
> file
> >> >> > of
> >> >> > conf
> >> >> > directory.
> >> >> >
> >> >> >
> >> >> > On Thu, May 29, 2014 at 3:39 AM, chris snow <ch...@gmail.com>
> >> >> > wrote:
> >> >> >>
> >> >> >> Hi Nirmal,
> >> >> >>
> >> >> >> For some reason, that method is not getting called. I've pasted
> my
> >> >> >> stacktrace again with the drools methods removed:
> >> >> >>
> >> >> >> ---
> >> >> >> Daemon Thread [Thread-56] (Suspended (breakpoint at line 137 in
> >> >> >> CloudControllerClient))
> >> >> >> owns: CloudControllerClient (id=2433)
> >> >> >> owns: DefaultAgenda (id=2434)
> >> >> >> CloudControllerClient.spawnAnInstance(Partition, String,
> String,
> >> >> >> String) line: 137
> >> >> >> RuleTasksDelegator.delegateSpawn(PartitionContext, String,
> >> >> >> String)
> >> >> >> line: 92
> >> >> >> //
> >> >> >> // drools methods hidden
> >> >> >> //
> >> >> >>
> >> >> >> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
> >> >> >> FactHandle, Object) line: 86
> >> >> >> ClusterMonitor.monitor() line: 99
> >> >> >> ClusterMonitor.run() line: 72
> >> >> >> Thread.run() line: 744
> >> >> >> ---
> >> >> >>
> >> >> >> Would you expect to see AutoscaleAlgorithm.getNextScaleUpPartition
> >> >> >> executed somewhere in the stack of calls?
> >> >> >>
> >> >> >> Many thanks,
> >> >> >>
> >> >> >> Chris
> >> >> >>
> >> >> >> On Wed, May 28, 2014 at 11:03 PM, Nirmal Fernando
> >> >> >> <ni...@gmail.com> wrote:
> >> >> >> > Hi Chris,
> >> >> >> >
> >> >> >> > PartitionMax is considered in
> >> >> >> > AutoscaleAlgorithm.getNextScaleUpPartition
> >> >> >> > method.
> >> >> >> >
> >> >> >> >
> >> >> >> > On Thu, May 29, 2014 at 3:09 AM, chris snow <
> chsnow123@gmail.com>
> >> >> >> > wrote:
> >> >> >> >>
> >> >> >> >> I setup a breakpoint in eclipse at
> >> >> >> >> CloudControllerClient.spawnAnInstance() and dumped the output
> >> >> >> >> (below).
> >> >> >> >>
> >> >> >> >> I've scanned through the code in the stratos methods in the
> >> >> >> >> stacktrace, but couldn't see any code that checks the
> >> >> >> >> partitionMax
> >> >> >> >> value.
> >> >> >> >>
> >> >> >> >> Any ideas?
> >> >> >> >>
> >> >> >> >> Daemon Thread [Thread-56] (Suspended (breakpoint at line 137 in
> >> >> >> >> CloudControllerClient))
> >> >> >> >> owns: CloudControllerClient (id=2433)
> >> >> >> >> owns: DefaultAgenda (id=2434)
> >> >> >> >> CloudControllerClient.spawnAnInstance(Partition, String,
> >> >> >> >> String,
> >> >> >> >> String) line: 137
> >> >> >> >> RuleTasksDelegator.delegateSpawn(PartitionContext, String,
> >> >> >> >> String)
> >> >> >> >> line: 92
> >> >> >> >> NativeMethodAccessorImpl.invoke0(Method, Object, Object[])
> >> >> >> >> line:
> >> >> >> >> not available [native method]
> >> >> >> >> NativeMethodAccessorImpl.invoke(Object, Object[]) line: 57
> >> >> >> >> DelegatingMethodAccessorImpl.invoke(Object, Object[]) line:
> 43
> >> >> >> >> Method.invoke(Object, Object...) line: 606
> >> >> >> >> ReflectiveAccessorOptimizer.getMethod(Object, String,
> >> >> >> >> Object[],
> >> >> >> >> Class[], ExecutableStatement[]) line: 1104
> >> >> >> >> ReflectiveAccessorOptimizer.getMethod(Object, String) line:
> >> >> >> >> 987
> >> >> >> >> ReflectiveAccessorOptimizer.compileGetChain() line: 377
> >> >> >> >> ReflectiveAccessorOptimizer.optimizeAccessor(ParserContext,
> >> >> >> >> char[],
> >> >> >> >> int, int, Object, Object, VariableResolverFactory, boolean,
> >> >> >> >> Class)
> >> >> >> >> line: 143
> >> >> >> >> ASTNode.optimize(Object, Object, VariableResolverFactory)
> >> >> >> >> line:
> >> >> >> >> 159
> >> >> >> >> ASTNode.getReducedValueAccelerated(Object, Object,
> >> >> >> >> VariableResolverFactory) line: 115
> >> >> >> >> MVELRuntime.execute(boolean, CompiledExpression, Object,
> >> >> >> >> VariableResolverFactory) line: 85
> >> >> >> >> CompiledExpression.getDirectValue(Object,
> >> >> >> >> VariableResolverFactory)
> >> >> >> >> line: 123
> >> >> >> >> CompiledExpression.getValue(Object, VariableResolverFactory)
> >> >> >> >> line:
> >> >> >> >> 119
> >> >> >> >> MVEL.executeExpression(Object, Object,
> >> >> >> >> VariableResolverFactory)
> >> >> >> >> line:
> >> >> >> >> 930
> >> >> >> >> MVELConsequence.evaluate(KnowledgeHelper, WorkingMemory)
> line:
> >> >> >> >> 104
> >> >> >> >> DefaultAgenda.fireActivation(Activation) line: 1287
> >> >> >> >> DefaultAgenda.fireNextItem(AgendaFilter) line: 1221
> >> >> >> >> DefaultAgenda.fireAllRules(AgendaFilter, int) line: 1456
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules(AgendaFilter,
> >> >> >> >> int) line: 710
> >> >> >> >> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules()
> >> >> >> >> line:
> >> >> >> >> 674
> >> >> >> >> StatefulKnowledgeSessionImpl.fireAllRules() line: 230
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
> >> >> >> >> FactHandle, Object) line: 86
> >> >> >> >> ClusterMonitor.monitor() line: 99
> >> >> >> >> ClusterMonitor.run() line: 72
> >> >> >> >> Thread.run() line: 744
> >> >> >> >>
> >> >> >> >> On Wed, May 28, 2014 at 3:37 AM, Lahiru Sandaruwan
> >> >> >> >> <la...@wso2.com>
> >> >> >> >> wrote:
> >> >> >> >> >
> >> >> >> >> >
> >> >> >> >> >
> >> >> >> >> > On Tue, May 27, 2014 at 11:54 PM, chris snow
> >> >> >> >> > <ch...@gmail.com>
> >> >> >> >> > wrote:
> >> >> >> >> >>
> >> >> >> >> >> Hi Lahiru,
> >> >> >> >> >>
> >> >> >> >> >> I'll have a poke around - it will give me a good opportunity
> >> >> >> >> >> to
> >> >> >> >> >> learn
> >> >> >> >> >> some more Stratos code :)
> >> >> >> >> >>
> >> >> >> >> >
> >> >> >> >> > Cool :)
> >> >> >> >> >
> >> >> >> >> > Debug logs should be able to show us where it went wrong.
> >> >> >> >> >
> >> >> >> >> > Thanks.
> >> >> >> >> >>
> >> >> >> >> >> Many thanks,
> >> >> >> >> >>
> >> >> >> >> >> Chris
> >> >> >> >> >>
> >> >> >> >> >> On Tue, May 27, 2014 at 7:45 AM, Lahiru Sandaruwan
> >> >> >> >> >> <la...@wso2.com>
> >> >> >> >> >> wrote:
> >> >> >> >> >> > Hi Chris,
> >> >> >> >> >> >
> >> >> >> >> >> > Can you enable debug logs and try out?
> >> >> >> >> >> > If so please send them over.
> >> >> >> >> >> >
> >> >> >> >> >> > Thanks.
> >> >> >> >> >> >
> >> >> >> >> >> >
> >> >> >> >> >> > On Tue, May 27, 2014 at 12:05 PM, chris snow
> >> >> >> >> >> > <ch...@gmail.com>
> >> >> >> >> >> > wrote:
> >> >> >> >> >> >>
> >> >> >> >> >> >> Deployment Policy partitionMax seems to be ignored. My
> >> >> >> >> >> >> deployment
> >> >> >> >> >> >> policy looks like this:
> >> >> >> >> >> >>
> >> >> >> >> >> >> {
> >> >> >> >> >> >> "id": "economyDeployment",
> >> >> >> >> >> >> "partitionGroup": [
> >> >> >> >> >> >> {
> >> >> >> >> >> >> "id": "G1",
> >> >> >> >> >> >> "partitionAlgo": "one-after-another",
> >> >> >> >> >> >> "partition": [
> >> >> >> >> >> >> {
> >> >> >> >> >> >> "id": "P1",
> >> >> >> >> >> >> "partitionMax": "1",
> >> >> >> >> >> >> "partitionMin": "1"
> >> >> >> >> >> >> }
> >> >> >> >> >> >> ]
> >> >> >> >> >> >> }
> >> >> >> >> >> >> ]
> >> >> >> >> >> >> }
> >> >> >> >> >> >>
> >> >> >> >> >> >> Stratos initially starts two instances:
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> >> >> >> >> | ID | Name |
> >> >> >> >> >> >> Status |
> >> >> >> >> >> >> Task
> >> >> >> >> >> >> State | Power State | Networks |
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> >> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 |
> >> >> >> >> >> >> ACTIVE |
> >> >> >> >> >> >> -
> >> >> >> >> >> >> | Running | private=10.11.12.2 |
> >> >> >> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 |
> >> >> >> >> >> >> ACTIVE |
> >> >> >> >> >> >> -
> >> >> >> >> >> >> | Running | private=10.11.12.3 |
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> >> >> >> >> Then attempts to start more:
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> >> >> >> >> | ID | Name |
> >> >> >> >> >> >> Status |
> >> >> >> >> >> >> Task
> >> >> >> >> >> >> State | Power State | Networks |
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> >> >> >> >> | 91029130-37b5-4df6-ad95-1d9f609a22c3 | php111php-13e |
> >> >> >> >> >> >> ERROR
> >> >> >> >> >> >> |
> >> >> >> >> >> >> -
> >> >> >> >> >> >> | NOSTATE | |
> >> >> >> >> >> >> | 71c1e361-df55-4f61-84c5-79774fef2beb | php111php-3d7 |
> >> >> >> >> >> >> ERROR
> >> >> >> >> >> >> |
> >> >> >> >> >> >> -
> >> >> >> >> >> >> | NOSTATE | |
> >> >> >> >> >> >> | b0d72602-cdb8-4621-ad12-911ade4d89b1 | php111php-412 |
> >> >> >> >> >> >> ERROR
> >> >> >> >> >> >> |
> >> >> >> >> >> >> -
> >> >> >> >> >> >> | NOSTATE | |
> >> >> >> >> >> >> | d4eac55d-bb7c-48d6-83f2-cb9a11581a18 | php111php-6f5 |
> >> >> >> >> >> >> ERROR
> >> >> >> >> >> >> |
> >> >> >> >> >> >> -
> >> >> >> >> >> >> | NOSTATE | |
> >> >> >> >> >> >> | 738a9534-2b95-4984-a51b-748bc7169b9a | php111php-73e |
> >> >> >> >> >> >> ERROR
> >> >> >> >> >> >> |
> >> >> >> >> >> >> -
> >> >> >> >> >> >> | NOSTATE | |
> >> >> >> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 |
> >> >> >> >> >> >> ACTIVE |
> >> >> >> >> >> >> -
> >> >> >> >> >> >> | Running | private=10.11.12.2 |
> >> >> >> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 |
> >> >> >> >> >> >> ACTIVE |
> >> >> >> >> >> >> -
> >> >> >> >> >> >> | Running | private=10.11.12.3 |
> >> >> >> >> >> >> | f570e654-024b-4240-bad2-8a11f04f23f0 | php111php-a30 |
> >> >> >> >> >> >> ERROR
> >> >> >> >> >> >> |
> >> >> >> >> >> >> -
> >> >> >> >> >> >> | NOSTATE | |
> >> >> >> >> >> >> | 0ab36824-9bea-4a95-aa22-7f935406d83d | php111php-bca |
> >> >> >> >> >> >> ERROR
> >> >> >> >> >> >> |
> >> >> >> >> >> >> -
> >> >> >> >> >> >> | NOSTATE | |
> >> >> >> >> >> >> | 02ed97df-02bc-4bff-b42d-9f2bcbd4f569 | php111php-f6f |
> >> >> >> >> >> >> ERROR
> >> >> >> >> >> >> |
> >> >> >> >> >> >> -
> >> >> >> >> >> >> | NOSTATE | |
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> >> >> >> >>
> >> >> >> >> >> >> I'm on branch: 4.0.0-incubating, using Openstack Havana +
> >> >> >> >> >> >> Docker.
> >> >> >> >> >> >>
> >> >> >> >> >> >> Any ideas?
> >> >> >> >> >> >>
> >> >> >> >> >> >> Many thanks,
> >> >> >> >> >> >>
> >> >> >> >> >> >> Chris
> >> >> >> >> >> >
> >> >> >> >> >> >
> >> >> >> >> >> >
> >> >> >> >> >> >
> >> >> >> >> >> > --
> >> >> >> >> >> > --
> >> >> >> >> >> > Lahiru Sandaruwan
> >> >> >> >> >> > Committer and PMC member, Apache Stratos,
> >> >> >> >> >> > Senior Software Engineer,
> >> >> >> >> >> > WSO2 Inc., http://wso2.com
> >> >> >> >> >> > lean.enterprise.middleware
> >> >> >> >> >> >
> >> >> >> >> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
> >> >> >> >> >> > blog: http://lahiruwrites.blogspot.com/
> >> >> >> >> >> > twitter: http://twitter.com/lahirus
> >> >> >> >> >> > linked-in:
> >> >> >> >> >> > http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
> >> >> >> >> >> >
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >> --
> >> >> >> >> >> Check out my professional profile and connect with me on
> >> >> >> >> >> LinkedIn.
> >> >> >> >> >> http://lnkd.in/cw5k69
> >> >> >> >> >
> >> >> >> >> >
> >> >> >> >> >
> >> >> >> >> >
> >> >> >> >> > --
> >> >> >> >> > --
> >> >> >> >> > Lahiru Sandaruwan
> >> >> >> >> > Committer and PMC member, Apache Stratos,
> >> >> >> >> > Senior Software Engineer,
> >> >> >> >> > WSO2 Inc., http://wso2.com
> >> >> >> >> > lean.enterprise.middleware
> >> >> >> >> >
> >> >> >> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
> >> >> >> >> > blog: http://lahiruwrites.blogspot.com/
> >> >> >> >> > twitter: http://twitter.com/lahirus
> >> >> >> >> > linked-in:
> >> >> >> >> > http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
> >> >> >> >> >
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >> --
> >> >> >> >> Check out my professional profile and connect with me on
> >> >> >> >> LinkedIn.
> >> >> >> >> http://lnkd.in/cw5k69
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> > --
> >> >> >> > Best Regards,
> >> >> >> > Nirmal
> >> >> >> >
> >> >> >> > Nirmal Fernando.
> >> >> >> > PPMC Member & Committer of Apache Stratos,
> >> >> >> >
> >> >> >> > Senior Software Engineer, WSO2 Inc.
> >> >> >> >
> >> >> >> > Blog: http://nirmalfdo.blogspot.com/
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> --
> >> >> >> Check out my professional profile and connect with me on LinkedIn.
> >> >> >> http://lnkd.in/cw5k69
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Best Regards,
> >> >> > Nirmal
> >> >> >
> >> >> > Nirmal Fernando.
> >> >> > PPMC Member & Committer of Apache Stratos,
> >> >> > Senior Software Engineer, WSO2 Inc.
> >> >> >
> >> >> > Blog: http://nirmalfdo.blogspot.com/
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Check out my professional profile and connect with me on LinkedIn.
> >> >> http://lnkd.in/cw5k69
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > Best Regards,
> >> > Nirmal
> >> >
> >> > Nirmal Fernando.
> >> > PPMC Member & Committer of Apache Stratos,
> >> > Senior Software Engineer, WSO2 Inc.
> >> >
> >> > Blog: http://nirmalfdo.blogspot.com/
> >>
> >>
> >>
> >> --
> >> Check out my professional profile and connect with me on LinkedIn.
> >> http://lnkd.in/cw5k69
> >
> >
> >
> >
> > --
> > --
> > Lahiru Sandaruwan
> > Committer and PMC member, Apache Stratos,
> > Senior Software Engineer,
> > WSO2 Inc., http://wso2.com
> > lean.enterprise.middleware
> >
> > email: lahirus@wso2.com cell: (+94) 773 325 954
> > blog: http://lahiruwrites.blogspot.com/
> > twitter: http://twitter.com/lahirus
> > linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
> >
>
>
>
> --
> Check out my professional profile and connect with me on LinkedIn.
> http://lnkd.in/cw5k69
>
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
Re: autoscaling is ignoring partiionMax
Posted by chris snow <ch...@gmail.com>.
Hi Lahiru, are MemberActivatedEvents sent by the Stratos agent, or are
they sent by the another component (e.g. CloudController)?
On Thu, May 29, 2014 at 7:49 AM, Lahiru Sandaruwan <la...@wso2.com> wrote:
> Hi Chris,
>
> Can you confirm that the Autoscaler received Member activated event for
> these particular instances? Logs should help identify that.
>
> Thanks.
>
>
>
>
> On Thu, May 29, 2014 at 12:15 PM, chris snow <ch...@gmail.com> wrote:
>>
>> Thanks Nirmal.
>>
>> Non terminated member count doesn't seem to detect the two active
>> instances that have been started:
>>
>> [2014-05-29 06:06:23,900] DEBUG
>> {org.apache.stratos.autoscaler.rule.RuleLog} - Running minimum rule:
>> [network-partition] G1 [partition] P1
>> [2014-05-29 06:06:23,906] DEBUG
>> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check]
>> [network-partition] G1 [partition] P1 Non terminated member count: 0
>> [2014-05-29 06:06:23,908] DEBUG
>> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check]
>> [network-partition] G1 [partition] P1 Minimum member count: 1
>> [2014-05-29 06:06:23,925] DEBUG
>> {org.apache.stratos.autoscaler.rule.RuleLog} - Running obsolete
>> instances rule: [partition] P1 [network-partition] G1
>> [2014-05-29 06:06:23,927] DEBUG
>> {org.apache.stratos.autoscaler.rule.RuleLog} - [obsolete-check]
>> [network-partition] G1 [partition] P1 Obsolete member count: 0
>> [2014-05-29 06:06:23,930] INFO
>>
>> {org.apache.stratos.autoscaler.client.cloud.controller.CloudControllerClient}
>> - Trying to spawn an instance via cloud controller: [cluster]
>> php111.php.domain [partition] P1 [lb-cluster] null
>> [network-partition-id] G1
>>
>>
>> vagrant@puppet:~$ nova list
>>
>> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> | ID | Name | Status | Task
>> State | Power State | Networks |
>>
>> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> | a3ce6f52-27bf-4343-a05d-d7c9da9a1cb2 | php111php-3cb | ERROR | -
>> | NOSTATE | |
>> | 82434b60-6c9d-40a3-8191-2d466cde725e | php111php-47e | ERROR | -
>> | NOSTATE | |
>> | 4aa942ff-a722-4ff4-b604-abaa2f44c1a7 | php111php-499 | ERROR | -
>> | NOSTATE | |
>> | 1548915b-98f6-48fc-b70a-b0acc74610a7 | php111php-5e0 | ERROR | -
>> | NOSTATE | |
>> | f313c93d-7de1-4ccb-8c0d-f4104e5d837f | php111php-701 | ERROR | -
>> | NOSTATE | |
>> | a49598bb-0d4a-4064-af16-5b4b91057fd8 | php111php-719 | ERROR | -
>> | NOSTATE | |
>> | f8376091-99f6-4d17-8ec6-3935291c3e40 | php111php-841 | ERROR | -
>> | NOSTATE | |
>> | cf3298da-eecc-448a-bcea-2296eeed23dc | php111php-ad | ACTIVE | -
>> | Running | private=10.11.12.2 |
>> | 1d8832d5-39e4-476c-9c2b-1b1f5ffe3b41 | php111php-d81 | ACTIVE | -
>> | Running | private=10.11.12.3 |
>> | 4f6d51fb-fd27-4be1-94be-e9b5200fb6ba | php111php-e9c | ERROR | -
>> | NOSTATE | |
>>
>> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>>
>> I'll investigate some more...
>>
>> On Thu, May 29, 2014 at 7:42 AM, Nirmal Fernando <ni...@gmail.com>
>> wrote:
>> > Ah.. probably Chris. So, in min check what happens is it always make
>> > sure
>> > that you have min instance count in the system.
>> >
>> >
>> > On Thu, May 29, 2014 at 11:42 AM, chris snow <ch...@gmail.com>
>> > wrote:
>> >>
>> >> Hi Nirmal,
>> >>
>> >> I've just see that it is the "Minimal Rule" in mincheck.drl that is
>> >> trying to spawn the instance, and not scaling.drl.
>> >>
>> >> This is why AutoscaleAlgorithm.getNextScaleUpPartition() is not
>> >> getting
>> >> called?
>> >>
>> >> Cheers,
>> >>
>> >> Chris
>> >>
>> >>
>> >> On Wed, May 28, 2014 at 11:24 PM, Nirmal Fernando
>> >> <ni...@gmail.com> wrote:
>> >> > No, Chris. This happens via Drools rules located in scaling.drl file
>> >> > of
>> >> > conf
>> >> > directory.
>> >> >
>> >> >
>> >> > On Thu, May 29, 2014 at 3:39 AM, chris snow <ch...@gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> Hi Nirmal,
>> >> >>
>> >> >> For some reason, that method is not getting called. I've pasted my
>> >> >> stacktrace again with the drools methods removed:
>> >> >>
>> >> >> ---
>> >> >> Daemon Thread [Thread-56] (Suspended (breakpoint at line 137 in
>> >> >> CloudControllerClient))
>> >> >> owns: CloudControllerClient (id=2433)
>> >> >> owns: DefaultAgenda (id=2434)
>> >> >> CloudControllerClient.spawnAnInstance(Partition, String, String,
>> >> >> String) line: 137
>> >> >> RuleTasksDelegator.delegateSpawn(PartitionContext, String,
>> >> >> String)
>> >> >> line: 92
>> >> >> //
>> >> >> // drools methods hidden
>> >> >> //
>> >> >>
>> >> >> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
>> >> >> FactHandle, Object) line: 86
>> >> >> ClusterMonitor.monitor() line: 99
>> >> >> ClusterMonitor.run() line: 72
>> >> >> Thread.run() line: 744
>> >> >> ---
>> >> >>
>> >> >> Would you expect to see AutoscaleAlgorithm.getNextScaleUpPartition
>> >> >> executed somewhere in the stack of calls?
>> >> >>
>> >> >> Many thanks,
>> >> >>
>> >> >> Chris
>> >> >>
>> >> >> On Wed, May 28, 2014 at 11:03 PM, Nirmal Fernando
>> >> >> <ni...@gmail.com> wrote:
>> >> >> > Hi Chris,
>> >> >> >
>> >> >> > PartitionMax is considered in
>> >> >> > AutoscaleAlgorithm.getNextScaleUpPartition
>> >> >> > method.
>> >> >> >
>> >> >> >
>> >> >> > On Thu, May 29, 2014 at 3:09 AM, chris snow <ch...@gmail.com>
>> >> >> > wrote:
>> >> >> >>
>> >> >> >> I setup a breakpoint in eclipse at
>> >> >> >> CloudControllerClient.spawnAnInstance() and dumped the output
>> >> >> >> (below).
>> >> >> >>
>> >> >> >> I've scanned through the code in the stratos methods in the
>> >> >> >> stacktrace, but couldn't see any code that checks the
>> >> >> >> partitionMax
>> >> >> >> value.
>> >> >> >>
>> >> >> >> Any ideas?
>> >> >> >>
>> >> >> >> Daemon Thread [Thread-56] (Suspended (breakpoint at line 137 in
>> >> >> >> CloudControllerClient))
>> >> >> >> owns: CloudControllerClient (id=2433)
>> >> >> >> owns: DefaultAgenda (id=2434)
>> >> >> >> CloudControllerClient.spawnAnInstance(Partition, String,
>> >> >> >> String,
>> >> >> >> String) line: 137
>> >> >> >> RuleTasksDelegator.delegateSpawn(PartitionContext, String,
>> >> >> >> String)
>> >> >> >> line: 92
>> >> >> >> NativeMethodAccessorImpl.invoke0(Method, Object, Object[])
>> >> >> >> line:
>> >> >> >> not available [native method]
>> >> >> >> NativeMethodAccessorImpl.invoke(Object, Object[]) line: 57
>> >> >> >> DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
>> >> >> >> Method.invoke(Object, Object...) line: 606
>> >> >> >> ReflectiveAccessorOptimizer.getMethod(Object, String,
>> >> >> >> Object[],
>> >> >> >> Class[], ExecutableStatement[]) line: 1104
>> >> >> >> ReflectiveAccessorOptimizer.getMethod(Object, String) line:
>> >> >> >> 987
>> >> >> >> ReflectiveAccessorOptimizer.compileGetChain() line: 377
>> >> >> >> ReflectiveAccessorOptimizer.optimizeAccessor(ParserContext,
>> >> >> >> char[],
>> >> >> >> int, int, Object, Object, VariableResolverFactory, boolean,
>> >> >> >> Class)
>> >> >> >> line: 143
>> >> >> >> ASTNode.optimize(Object, Object, VariableResolverFactory)
>> >> >> >> line:
>> >> >> >> 159
>> >> >> >> ASTNode.getReducedValueAccelerated(Object, Object,
>> >> >> >> VariableResolverFactory) line: 115
>> >> >> >> MVELRuntime.execute(boolean, CompiledExpression, Object,
>> >> >> >> VariableResolverFactory) line: 85
>> >> >> >> CompiledExpression.getDirectValue(Object,
>> >> >> >> VariableResolverFactory)
>> >> >> >> line: 123
>> >> >> >> CompiledExpression.getValue(Object, VariableResolverFactory)
>> >> >> >> line:
>> >> >> >> 119
>> >> >> >> MVEL.executeExpression(Object, Object,
>> >> >> >> VariableResolverFactory)
>> >> >> >> line:
>> >> >> >> 930
>> >> >> >> MVELConsequence.evaluate(KnowledgeHelper, WorkingMemory) line:
>> >> >> >> 104
>> >> >> >> DefaultAgenda.fireActivation(Activation) line: 1287
>> >> >> >> DefaultAgenda.fireNextItem(AgendaFilter) line: 1221
>> >> >> >> DefaultAgenda.fireAllRules(AgendaFilter, int) line: 1456
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules(AgendaFilter,
>> >> >> >> int) line: 710
>> >> >> >> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules()
>> >> >> >> line:
>> >> >> >> 674
>> >> >> >> StatefulKnowledgeSessionImpl.fireAllRules() line: 230
>> >> >> >>
>> >> >> >>
>> >> >> >> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
>> >> >> >> FactHandle, Object) line: 86
>> >> >> >> ClusterMonitor.monitor() line: 99
>> >> >> >> ClusterMonitor.run() line: 72
>> >> >> >> Thread.run() line: 744
>> >> >> >>
>> >> >> >> On Wed, May 28, 2014 at 3:37 AM, Lahiru Sandaruwan
>> >> >> >> <la...@wso2.com>
>> >> >> >> wrote:
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > On Tue, May 27, 2014 at 11:54 PM, chris snow
>> >> >> >> > <ch...@gmail.com>
>> >> >> >> > wrote:
>> >> >> >> >>
>> >> >> >> >> Hi Lahiru,
>> >> >> >> >>
>> >> >> >> >> I'll have a poke around - it will give me a good opportunity
>> >> >> >> >> to
>> >> >> >> >> learn
>> >> >> >> >> some more Stratos code :)
>> >> >> >> >>
>> >> >> >> >
>> >> >> >> > Cool :)
>> >> >> >> >
>> >> >> >> > Debug logs should be able to show us where it went wrong.
>> >> >> >> >
>> >> >> >> > Thanks.
>> >> >> >> >>
>> >> >> >> >> Many thanks,
>> >> >> >> >>
>> >> >> >> >> Chris
>> >> >> >> >>
>> >> >> >> >> On Tue, May 27, 2014 at 7:45 AM, Lahiru Sandaruwan
>> >> >> >> >> <la...@wso2.com>
>> >> >> >> >> wrote:
>> >> >> >> >> > Hi Chris,
>> >> >> >> >> >
>> >> >> >> >> > Can you enable debug logs and try out?
>> >> >> >> >> > If so please send them over.
>> >> >> >> >> >
>> >> >> >> >> > Thanks.
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > On Tue, May 27, 2014 at 12:05 PM, chris snow
>> >> >> >> >> > <ch...@gmail.com>
>> >> >> >> >> > wrote:
>> >> >> >> >> >>
>> >> >> >> >> >> Deployment Policy partitionMax seems to be ignored. My
>> >> >> >> >> >> deployment
>> >> >> >> >> >> policy looks like this:
>> >> >> >> >> >>
>> >> >> >> >> >> {
>> >> >> >> >> >> "id": "economyDeployment",
>> >> >> >> >> >> "partitionGroup": [
>> >> >> >> >> >> {
>> >> >> >> >> >> "id": "G1",
>> >> >> >> >> >> "partitionAlgo": "one-after-another",
>> >> >> >> >> >> "partition": [
>> >> >> >> >> >> {
>> >> >> >> >> >> "id": "P1",
>> >> >> >> >> >> "partitionMax": "1",
>> >> >> >> >> >> "partitionMin": "1"
>> >> >> >> >> >> }
>> >> >> >> >> >> ]
>> >> >> >> >> >> }
>> >> >> >> >> >> ]
>> >> >> >> >> >> }
>> >> >> >> >> >>
>> >> >> >> >> >> Stratos initially starts two instances:
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> >> >> >> | ID | Name |
>> >> >> >> >> >> Status |
>> >> >> >> >> >> Task
>> >> >> >> >> >> State | Power State | Networks |
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 |
>> >> >> >> >> >> ACTIVE |
>> >> >> >> >> >> -
>> >> >> >> >> >> | Running | private=10.11.12.2 |
>> >> >> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 |
>> >> >> >> >> >> ACTIVE |
>> >> >> >> >> >> -
>> >> >> >> >> >> | Running | private=10.11.12.3 |
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> >> >> >> Then attempts to start more:
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> >> >> >> | ID | Name |
>> >> >> >> >> >> Status |
>> >> >> >> >> >> Task
>> >> >> >> >> >> State | Power State | Networks |
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> >> >> >> | 91029130-37b5-4df6-ad95-1d9f609a22c3 | php111php-13e |
>> >> >> >> >> >> ERROR
>> >> >> >> >> >> |
>> >> >> >> >> >> -
>> >> >> >> >> >> | NOSTATE | |
>> >> >> >> >> >> | 71c1e361-df55-4f61-84c5-79774fef2beb | php111php-3d7 |
>> >> >> >> >> >> ERROR
>> >> >> >> >> >> |
>> >> >> >> >> >> -
>> >> >> >> >> >> | NOSTATE | |
>> >> >> >> >> >> | b0d72602-cdb8-4621-ad12-911ade4d89b1 | php111php-412 |
>> >> >> >> >> >> ERROR
>> >> >> >> >> >> |
>> >> >> >> >> >> -
>> >> >> >> >> >> | NOSTATE | |
>> >> >> >> >> >> | d4eac55d-bb7c-48d6-83f2-cb9a11581a18 | php111php-6f5 |
>> >> >> >> >> >> ERROR
>> >> >> >> >> >> |
>> >> >> >> >> >> -
>> >> >> >> >> >> | NOSTATE | |
>> >> >> >> >> >> | 738a9534-2b95-4984-a51b-748bc7169b9a | php111php-73e |
>> >> >> >> >> >> ERROR
>> >> >> >> >> >> |
>> >> >> >> >> >> -
>> >> >> >> >> >> | NOSTATE | |
>> >> >> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 |
>> >> >> >> >> >> ACTIVE |
>> >> >> >> >> >> -
>> >> >> >> >> >> | Running | private=10.11.12.2 |
>> >> >> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 |
>> >> >> >> >> >> ACTIVE |
>> >> >> >> >> >> -
>> >> >> >> >> >> | Running | private=10.11.12.3 |
>> >> >> >> >> >> | f570e654-024b-4240-bad2-8a11f04f23f0 | php111php-a30 |
>> >> >> >> >> >> ERROR
>> >> >> >> >> >> |
>> >> >> >> >> >> -
>> >> >> >> >> >> | NOSTATE | |
>> >> >> >> >> >> | 0ab36824-9bea-4a95-aa22-7f935406d83d | php111php-bca |
>> >> >> >> >> >> ERROR
>> >> >> >> >> >> |
>> >> >> >> >> >> -
>> >> >> >> >> >> | NOSTATE | |
>> >> >> >> >> >> | 02ed97df-02bc-4bff-b42d-9f2bcbd4f569 | php111php-f6f |
>> >> >> >> >> >> ERROR
>> >> >> >> >> >> |
>> >> >> >> >> >> -
>> >> >> >> >> >> | NOSTATE | |
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> >> >> >>
>> >> >> >> >> >> I'm on branch: 4.0.0-incubating, using Openstack Havana +
>> >> >> >> >> >> Docker.
>> >> >> >> >> >>
>> >> >> >> >> >> Any ideas?
>> >> >> >> >> >>
>> >> >> >> >> >> Many thanks,
>> >> >> >> >> >>
>> >> >> >> >> >> Chris
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > --
>> >> >> >> >> > --
>> >> >> >> >> > Lahiru Sandaruwan
>> >> >> >> >> > Committer and PMC member, Apache Stratos,
>> >> >> >> >> > Senior Software Engineer,
>> >> >> >> >> > WSO2 Inc., http://wso2.com
>> >> >> >> >> > lean.enterprise.middleware
>> >> >> >> >> >
>> >> >> >> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
>> >> >> >> >> > blog: http://lahiruwrites.blogspot.com/
>> >> >> >> >> > twitter: http://twitter.com/lahirus
>> >> >> >> >> > linked-in:
>> >> >> >> >> > http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>> >> >> >> >> >
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> --
>> >> >> >> >> Check out my professional profile and connect with me on
>> >> >> >> >> LinkedIn.
>> >> >> >> >> http://lnkd.in/cw5k69
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > --
>> >> >> >> > --
>> >> >> >> > Lahiru Sandaruwan
>> >> >> >> > Committer and PMC member, Apache Stratos,
>> >> >> >> > Senior Software Engineer,
>> >> >> >> > WSO2 Inc., http://wso2.com
>> >> >> >> > lean.enterprise.middleware
>> >> >> >> >
>> >> >> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
>> >> >> >> > blog: http://lahiruwrites.blogspot.com/
>> >> >> >> > twitter: http://twitter.com/lahirus
>> >> >> >> > linked-in:
>> >> >> >> > http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>> >> >> >> >
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> --
>> >> >> >> Check out my professional profile and connect with me on
>> >> >> >> LinkedIn.
>> >> >> >> http://lnkd.in/cw5k69
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > --
>> >> >> > Best Regards,
>> >> >> > Nirmal
>> >> >> >
>> >> >> > Nirmal Fernando.
>> >> >> > PPMC Member & Committer of Apache Stratos,
>> >> >> >
>> >> >> > Senior Software Engineer, WSO2 Inc.
>> >> >> >
>> >> >> > Blog: http://nirmalfdo.blogspot.com/
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Check out my professional profile and connect with me on LinkedIn.
>> >> >> http://lnkd.in/cw5k69
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Best Regards,
>> >> > Nirmal
>> >> >
>> >> > Nirmal Fernando.
>> >> > PPMC Member & Committer of Apache Stratos,
>> >> > Senior Software Engineer, WSO2 Inc.
>> >> >
>> >> > Blog: http://nirmalfdo.blogspot.com/
>> >>
>> >>
>> >>
>> >> --
>> >> Check out my professional profile and connect with me on LinkedIn.
>> >> http://lnkd.in/cw5k69
>> >
>> >
>> >
>> >
>> > --
>> > Best Regards,
>> > Nirmal
>> >
>> > Nirmal Fernando.
>> > PPMC Member & Committer of Apache Stratos,
>> > Senior Software Engineer, WSO2 Inc.
>> >
>> > Blog: http://nirmalfdo.blogspot.com/
>>
>>
>>
>> --
>> Check out my professional profile and connect with me on LinkedIn.
>> http://lnkd.in/cw5k69
>
>
>
>
> --
> --
> Lahiru Sandaruwan
> Committer and PMC member, Apache Stratos,
> Senior Software Engineer,
> WSO2 Inc., http://wso2.com
> lean.enterprise.middleware
>
> email: lahirus@wso2.com cell: (+94) 773 325 954
> blog: http://lahiruwrites.blogspot.com/
> twitter: http://twitter.com/lahirus
> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>
--
Check out my professional profile and connect with me on LinkedIn.
http://lnkd.in/cw5k69
Re: autoscaling is ignoring partiionMax
Posted by Lahiru Sandaruwan <la...@wso2.com>.
Hi Chris,
Can you confirm that the Autoscaler received Member activated event for
these particular instances? Logs should help identify that.
Thanks.
On Thu, May 29, 2014 at 12:15 PM, chris snow <ch...@gmail.com> wrote:
> Thanks Nirmal.
>
> Non terminated member count doesn't seem to detect the two active
> instances that have been started:
>
> [2014-05-29 06:06:23,900] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} - Running minimum rule:
> [network-partition] G1 [partition] P1
> [2014-05-29 06:06:23,906] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check]
> [network-partition] G1 [partition] P1 Non terminated member count: 0
> [2014-05-29 06:06:23,908] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} - [min-check]
> [network-partition] G1 [partition] P1 Minimum member count: 1
> [2014-05-29 06:06:23,925] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} - Running obsolete
> instances rule: [partition] P1 [network-partition] G1
> [2014-05-29 06:06:23,927] DEBUG
> {org.apache.stratos.autoscaler.rule.RuleLog} - [obsolete-check]
> [network-partition] G1 [partition] P1 Obsolete member count: 0
> [2014-05-29 06:06:23,930] INFO
>
> {org.apache.stratos.autoscaler.client.cloud.controller.CloudControllerClient}
> - Trying to spawn an instance via cloud controller: [cluster]
> php111.php.domain [partition] P1 [lb-cluster] null
> [network-partition-id] G1
>
>
> vagrant@puppet:~$ nova list
>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> | ID | Name | Status | Task
> State | Power State | Networks |
>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> | a3ce6f52-27bf-4343-a05d-d7c9da9a1cb2 | php111php-3cb | ERROR | -
> | NOSTATE | |
> | 82434b60-6c9d-40a3-8191-2d466cde725e | php111php-47e | ERROR | -
> | NOSTATE | |
> | 4aa942ff-a722-4ff4-b604-abaa2f44c1a7 | php111php-499 | ERROR | -
> | NOSTATE | |
> | 1548915b-98f6-48fc-b70a-b0acc74610a7 | php111php-5e0 | ERROR | -
> | NOSTATE | |
> | f313c93d-7de1-4ccb-8c0d-f4104e5d837f | php111php-701 | ERROR | -
> | NOSTATE | |
> | a49598bb-0d4a-4064-af16-5b4b91057fd8 | php111php-719 | ERROR | -
> | NOSTATE | |
> | f8376091-99f6-4d17-8ec6-3935291c3e40 | php111php-841 | ERROR | -
> | NOSTATE | |
> | cf3298da-eecc-448a-bcea-2296eeed23dc | php111php-ad | ACTIVE | -
> | Running | private=10.11.12.2 |
> | 1d8832d5-39e4-476c-9c2b-1b1f5ffe3b41 | php111php-d81 | ACTIVE | -
> | Running | private=10.11.12.3 |
> | 4f6d51fb-fd27-4be1-94be-e9b5200fb6ba | php111php-e9c | ERROR | -
> | NOSTATE | |
>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>
> I'll investigate some more...
>
> On Thu, May 29, 2014 at 7:42 AM, Nirmal Fernando <ni...@gmail.com>
> wrote:
> > Ah.. probably Chris. So, in min check what happens is it always make sure
> > that you have min instance count in the system.
> >
> >
> > On Thu, May 29, 2014 at 11:42 AM, chris snow <ch...@gmail.com>
> wrote:
> >>
> >> Hi Nirmal,
> >>
> >> I've just see that it is the "Minimal Rule" in mincheck.drl that is
> >> trying to spawn the instance, and not scaling.drl.
> >>
> >> This is why AutoscaleAlgorithm.getNextScaleUpPartition() is not getting
> >> called?
> >>
> >> Cheers,
> >>
> >> Chris
> >>
> >>
> >> On Wed, May 28, 2014 at 11:24 PM, Nirmal Fernando
> >> <ni...@gmail.com> wrote:
> >> > No, Chris. This happens via Drools rules located in scaling.drl file
> of
> >> > conf
> >> > directory.
> >> >
> >> >
> >> > On Thu, May 29, 2014 at 3:39 AM, chris snow <ch...@gmail.com>
> wrote:
> >> >>
> >> >> Hi Nirmal,
> >> >>
> >> >> For some reason, that method is not getting called. I've pasted my
> >> >> stacktrace again with the drools methods removed:
> >> >>
> >> >> ---
> >> >> Daemon Thread [Thread-56] (Suspended (breakpoint at line 137 in
> >> >> CloudControllerClient))
> >> >> owns: CloudControllerClient (id=2433)
> >> >> owns: DefaultAgenda (id=2434)
> >> >> CloudControllerClient.spawnAnInstance(Partition, String, String,
> >> >> String) line: 137
> >> >> RuleTasksDelegator.delegateSpawn(PartitionContext, String, String)
> >> >> line: 92
> >> >> //
> >> >> // drools methods hidden
> >> >> //
> >> >> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
> >> >> FactHandle, Object) line: 86
> >> >> ClusterMonitor.monitor() line: 99
> >> >> ClusterMonitor.run() line: 72
> >> >> Thread.run() line: 744
> >> >> ---
> >> >>
> >> >> Would you expect to see AutoscaleAlgorithm.getNextScaleUpPartition
> >> >> executed somewhere in the stack of calls?
> >> >>
> >> >> Many thanks,
> >> >>
> >> >> Chris
> >> >>
> >> >> On Wed, May 28, 2014 at 11:03 PM, Nirmal Fernando
> >> >> <ni...@gmail.com> wrote:
> >> >> > Hi Chris,
> >> >> >
> >> >> > PartitionMax is considered in
> >> >> > AutoscaleAlgorithm.getNextScaleUpPartition
> >> >> > method.
> >> >> >
> >> >> >
> >> >> > On Thu, May 29, 2014 at 3:09 AM, chris snow <ch...@gmail.com>
> >> >> > wrote:
> >> >> >>
> >> >> >> I setup a breakpoint in eclipse at
> >> >> >> CloudControllerClient.spawnAnInstance() and dumped the output
> >> >> >> (below).
> >> >> >>
> >> >> >> I've scanned through the code in the stratos methods in the
> >> >> >> stacktrace, but couldn't see any code that checks the partitionMax
> >> >> >> value.
> >> >> >>
> >> >> >> Any ideas?
> >> >> >>
> >> >> >> Daemon Thread [Thread-56] (Suspended (breakpoint at line 137 in
> >> >> >> CloudControllerClient))
> >> >> >> owns: CloudControllerClient (id=2433)
> >> >> >> owns: DefaultAgenda (id=2434)
> >> >> >> CloudControllerClient.spawnAnInstance(Partition, String,
> String,
> >> >> >> String) line: 137
> >> >> >> RuleTasksDelegator.delegateSpawn(PartitionContext, String,
> >> >> >> String)
> >> >> >> line: 92
> >> >> >> NativeMethodAccessorImpl.invoke0(Method, Object, Object[])
> line:
> >> >> >> not available [native method]
> >> >> >> NativeMethodAccessorImpl.invoke(Object, Object[]) line: 57
> >> >> >> DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
> >> >> >> Method.invoke(Object, Object...) line: 606
> >> >> >> ReflectiveAccessorOptimizer.getMethod(Object, String, Object[],
> >> >> >> Class[], ExecutableStatement[]) line: 1104
> >> >> >> ReflectiveAccessorOptimizer.getMethod(Object, String) line: 987
> >> >> >> ReflectiveAccessorOptimizer.compileGetChain() line: 377
> >> >> >> ReflectiveAccessorOptimizer.optimizeAccessor(ParserContext,
> >> >> >> char[],
> >> >> >> int, int, Object, Object, VariableResolverFactory, boolean, Class)
> >> >> >> line: 143
> >> >> >> ASTNode.optimize(Object, Object, VariableResolverFactory) line:
> >> >> >> 159
> >> >> >> ASTNode.getReducedValueAccelerated(Object, Object,
> >> >> >> VariableResolverFactory) line: 115
> >> >> >> MVELRuntime.execute(boolean, CompiledExpression, Object,
> >> >> >> VariableResolverFactory) line: 85
> >> >> >> CompiledExpression.getDirectValue(Object,
> >> >> >> VariableResolverFactory)
> >> >> >> line: 123
> >> >> >> CompiledExpression.getValue(Object, VariableResolverFactory)
> >> >> >> line:
> >> >> >> 119
> >> >> >> MVEL.executeExpression(Object, Object, VariableResolverFactory)
> >> >> >> line:
> >> >> >> 930
> >> >> >> MVELConsequence.evaluate(KnowledgeHelper, WorkingMemory) line:
> >> >> >> 104
> >> >> >> DefaultAgenda.fireActivation(Activation) line: 1287
> >> >> >> DefaultAgenda.fireNextItem(AgendaFilter) line: 1221
> >> >> >> DefaultAgenda.fireAllRules(AgendaFilter, int) line: 1456
> >> >> >>
> >> >> >>
> >> >> >>
> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules(AgendaFilter,
> >> >> >> int) line: 710
> >> >> >> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules()
> line:
> >> >> >> 674
> >> >> >> StatefulKnowledgeSessionImpl.fireAllRules() line: 230
> >> >> >>
> >> >> >> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
> >> >> >> FactHandle, Object) line: 86
> >> >> >> ClusterMonitor.monitor() line: 99
> >> >> >> ClusterMonitor.run() line: 72
> >> >> >> Thread.run() line: 744
> >> >> >>
> >> >> >> On Wed, May 28, 2014 at 3:37 AM, Lahiru Sandaruwan
> >> >> >> <la...@wso2.com>
> >> >> >> wrote:
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> > On Tue, May 27, 2014 at 11:54 PM, chris snow <
> chsnow123@gmail.com>
> >> >> >> > wrote:
> >> >> >> >>
> >> >> >> >> Hi Lahiru,
> >> >> >> >>
> >> >> >> >> I'll have a poke around - it will give me a good opportunity to
> >> >> >> >> learn
> >> >> >> >> some more Stratos code :)
> >> >> >> >>
> >> >> >> >
> >> >> >> > Cool :)
> >> >> >> >
> >> >> >> > Debug logs should be able to show us where it went wrong.
> >> >> >> >
> >> >> >> > Thanks.
> >> >> >> >>
> >> >> >> >> Many thanks,
> >> >> >> >>
> >> >> >> >> Chris
> >> >> >> >>
> >> >> >> >> On Tue, May 27, 2014 at 7:45 AM, Lahiru Sandaruwan
> >> >> >> >> <la...@wso2.com>
> >> >> >> >> wrote:
> >> >> >> >> > Hi Chris,
> >> >> >> >> >
> >> >> >> >> > Can you enable debug logs and try out?
> >> >> >> >> > If so please send them over.
> >> >> >> >> >
> >> >> >> >> > Thanks.
> >> >> >> >> >
> >> >> >> >> >
> >> >> >> >> > On Tue, May 27, 2014 at 12:05 PM, chris snow
> >> >> >> >> > <ch...@gmail.com>
> >> >> >> >> > wrote:
> >> >> >> >> >>
> >> >> >> >> >> Deployment Policy partitionMax seems to be ignored. My
> >> >> >> >> >> deployment
> >> >> >> >> >> policy looks like this:
> >> >> >> >> >>
> >> >> >> >> >> {
> >> >> >> >> >> "id": "economyDeployment",
> >> >> >> >> >> "partitionGroup": [
> >> >> >> >> >> {
> >> >> >> >> >> "id": "G1",
> >> >> >> >> >> "partitionAlgo": "one-after-another",
> >> >> >> >> >> "partition": [
> >> >> >> >> >> {
> >> >> >> >> >> "id": "P1",
> >> >> >> >> >> "partitionMax": "1",
> >> >> >> >> >> "partitionMin": "1"
> >> >> >> >> >> }
> >> >> >> >> >> ]
> >> >> >> >> >> }
> >> >> >> >> >> ]
> >> >> >> >> >> }
> >> >> >> >> >>
> >> >> >> >> >> Stratos initially starts two instances:
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> >> >> >> | ID | Name |
> >> >> >> >> >> Status |
> >> >> >> >> >> Task
> >> >> >> >> >> State | Power State | Networks |
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 |
> >> >> >> >> >> ACTIVE |
> >> >> >> >> >> -
> >> >> >> >> >> | Running | private=10.11.12.2 |
> >> >> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 |
> >> >> >> >> >> ACTIVE |
> >> >> >> >> >> -
> >> >> >> >> >> | Running | private=10.11.12.3 |
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> >> >> >> Then attempts to start more:
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> >> >> >> | ID | Name |
> >> >> >> >> >> Status |
> >> >> >> >> >> Task
> >> >> >> >> >> State | Power State | Networks |
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> >> >> >> | 91029130-37b5-4df6-ad95-1d9f609a22c3 | php111php-13e |
> ERROR
> >> >> >> >> >> |
> >> >> >> >> >> -
> >> >> >> >> >> | NOSTATE | |
> >> >> >> >> >> | 71c1e361-df55-4f61-84c5-79774fef2beb | php111php-3d7 |
> ERROR
> >> >> >> >> >> |
> >> >> >> >> >> -
> >> >> >> >> >> | NOSTATE | |
> >> >> >> >> >> | b0d72602-cdb8-4621-ad12-911ade4d89b1 | php111php-412 |
> ERROR
> >> >> >> >> >> |
> >> >> >> >> >> -
> >> >> >> >> >> | NOSTATE | |
> >> >> >> >> >> | d4eac55d-bb7c-48d6-83f2-cb9a11581a18 | php111php-6f5 |
> ERROR
> >> >> >> >> >> |
> >> >> >> >> >> -
> >> >> >> >> >> | NOSTATE | |
> >> >> >> >> >> | 738a9534-2b95-4984-a51b-748bc7169b9a | php111php-73e |
> ERROR
> >> >> >> >> >> |
> >> >> >> >> >> -
> >> >> >> >> >> | NOSTATE | |
> >> >> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 |
> >> >> >> >> >> ACTIVE |
> >> >> >> >> >> -
> >> >> >> >> >> | Running | private=10.11.12.2 |
> >> >> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 |
> >> >> >> >> >> ACTIVE |
> >> >> >> >> >> -
> >> >> >> >> >> | Running | private=10.11.12.3 |
> >> >> >> >> >> | f570e654-024b-4240-bad2-8a11f04f23f0 | php111php-a30 |
> ERROR
> >> >> >> >> >> |
> >> >> >> >> >> -
> >> >> >> >> >> | NOSTATE | |
> >> >> >> >> >> | 0ab36824-9bea-4a95-aa22-7f935406d83d | php111php-bca |
> ERROR
> >> >> >> >> >> |
> >> >> >> >> >> -
> >> >> >> >> >> | NOSTATE | |
> >> >> >> >> >> | 02ed97df-02bc-4bff-b42d-9f2bcbd4f569 | php111php-f6f |
> ERROR
> >> >> >> >> >> |
> >> >> >> >> >> -
> >> >> >> >> >> | NOSTATE | |
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> >> >> >>
> >> >> >> >> >> I'm on branch: 4.0.0-incubating, using Openstack Havana +
> >> >> >> >> >> Docker.
> >> >> >> >> >>
> >> >> >> >> >> Any ideas?
> >> >> >> >> >>
> >> >> >> >> >> Many thanks,
> >> >> >> >> >>
> >> >> >> >> >> Chris
> >> >> >> >> >
> >> >> >> >> >
> >> >> >> >> >
> >> >> >> >> >
> >> >> >> >> > --
> >> >> >> >> > --
> >> >> >> >> > Lahiru Sandaruwan
> >> >> >> >> > Committer and PMC member, Apache Stratos,
> >> >> >> >> > Senior Software Engineer,
> >> >> >> >> > WSO2 Inc., http://wso2.com
> >> >> >> >> > lean.enterprise.middleware
> >> >> >> >> >
> >> >> >> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
> >> >> >> >> > blog: http://lahiruwrites.blogspot.com/
> >> >> >> >> > twitter: http://twitter.com/lahirus
> >> >> >> >> > linked-in:
> >> >> >> >> > http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
> >> >> >> >> >
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >> --
> >> >> >> >> Check out my professional profile and connect with me on
> >> >> >> >> LinkedIn.
> >> >> >> >> http://lnkd.in/cw5k69
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> > --
> >> >> >> > --
> >> >> >> > Lahiru Sandaruwan
> >> >> >> > Committer and PMC member, Apache Stratos,
> >> >> >> > Senior Software Engineer,
> >> >> >> > WSO2 Inc., http://wso2.com
> >> >> >> > lean.enterprise.middleware
> >> >> >> >
> >> >> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
> >> >> >> > blog: http://lahiruwrites.blogspot.com/
> >> >> >> > twitter: http://twitter.com/lahirus
> >> >> >> > linked-in:
> http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
> >> >> >> >
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> --
> >> >> >> Check out my professional profile and connect with me on LinkedIn.
> >> >> >> http://lnkd.in/cw5k69
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Best Regards,
> >> >> > Nirmal
> >> >> >
> >> >> > Nirmal Fernando.
> >> >> > PPMC Member & Committer of Apache Stratos,
> >> >> >
> >> >> > Senior Software Engineer, WSO2 Inc.
> >> >> >
> >> >> > Blog: http://nirmalfdo.blogspot.com/
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Check out my professional profile and connect with me on LinkedIn.
> >> >> http://lnkd.in/cw5k69
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > Best Regards,
> >> > Nirmal
> >> >
> >> > Nirmal Fernando.
> >> > PPMC Member & Committer of Apache Stratos,
> >> > Senior Software Engineer, WSO2 Inc.
> >> >
> >> > Blog: http://nirmalfdo.blogspot.com/
> >>
> >>
> >>
> >> --
> >> Check out my professional profile and connect with me on LinkedIn.
> >> http://lnkd.in/cw5k69
> >
> >
> >
> >
> > --
> > Best Regards,
> > Nirmal
> >
> > Nirmal Fernando.
> > PPMC Member & Committer of Apache Stratos,
> > Senior Software Engineer, WSO2 Inc.
> >
> > Blog: http://nirmalfdo.blogspot.com/
>
>
>
> --
> Check out my professional profile and connect with me on LinkedIn.
> http://lnkd.in/cw5k69
>
--
--
Lahiru Sandaruwan
Committer and PMC member, Apache Stratos,
Senior Software Engineer,
WSO2 Inc., http://wso2.com
lean.enterprise.middleware
email: lahirus@wso2.com cell: (+94) 773 325 954
blog: http://lahiruwrites.blogspot.com/
twitter: http://twitter.com/lahirus
linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
Re: autoscaling is ignoring partiionMax
Posted by chris snow <ch...@gmail.com>.
Thanks Nirmal.
Non terminated member count doesn't seem to detect the two active
instances that have been started:
[2014-05-29 06:06:23,900] DEBUG
{org.apache.stratos.autoscaler.rule.RuleLog} - Running minimum rule:
[network-partition] G1 [partition] P1
[2014-05-29 06:06:23,906] DEBUG
{org.apache.stratos.autoscaler.rule.RuleLog} - [min-check]
[network-partition] G1 [partition] P1 Non terminated member count: 0
[2014-05-29 06:06:23,908] DEBUG
{org.apache.stratos.autoscaler.rule.RuleLog} - [min-check]
[network-partition] G1 [partition] P1 Minimum member count: 1
[2014-05-29 06:06:23,925] DEBUG
{org.apache.stratos.autoscaler.rule.RuleLog} - Running obsolete
instances rule: [partition] P1 [network-partition] G1
[2014-05-29 06:06:23,927] DEBUG
{org.apache.stratos.autoscaler.rule.RuleLog} - [obsolete-check]
[network-partition] G1 [partition] P1 Obsolete member count: 0
[2014-05-29 06:06:23,930] INFO
{org.apache.stratos.autoscaler.client.cloud.controller.CloudControllerClient}
- Trying to spawn an instance via cloud controller: [cluster]
php111.php.domain [partition] P1 [lb-cluster] null
[network-partition-id] G1
vagrant@puppet:~$ nova list
+--------------------------------------+---------------+--------+------------+-------------+--------------------+
| ID | Name | Status | Task
State | Power State | Networks |
+--------------------------------------+---------------+--------+------------+-------------+--------------------+
| a3ce6f52-27bf-4343-a05d-d7c9da9a1cb2 | php111php-3cb | ERROR | -
| NOSTATE | |
| 82434b60-6c9d-40a3-8191-2d466cde725e | php111php-47e | ERROR | -
| NOSTATE | |
| 4aa942ff-a722-4ff4-b604-abaa2f44c1a7 | php111php-499 | ERROR | -
| NOSTATE | |
| 1548915b-98f6-48fc-b70a-b0acc74610a7 | php111php-5e0 | ERROR | -
| NOSTATE | |
| f313c93d-7de1-4ccb-8c0d-f4104e5d837f | php111php-701 | ERROR | -
| NOSTATE | |
| a49598bb-0d4a-4064-af16-5b4b91057fd8 | php111php-719 | ERROR | -
| NOSTATE | |
| f8376091-99f6-4d17-8ec6-3935291c3e40 | php111php-841 | ERROR | -
| NOSTATE | |
| cf3298da-eecc-448a-bcea-2296eeed23dc | php111php-ad | ACTIVE | -
| Running | private=10.11.12.2 |
| 1d8832d5-39e4-476c-9c2b-1b1f5ffe3b41 | php111php-d81 | ACTIVE | -
| Running | private=10.11.12.3 |
| 4f6d51fb-fd27-4be1-94be-e9b5200fb6ba | php111php-e9c | ERROR | -
| NOSTATE | |
+--------------------------------------+---------------+--------+------------+-------------+--------------------+
I'll investigate some more...
On Thu, May 29, 2014 at 7:42 AM, Nirmal Fernando <ni...@gmail.com> wrote:
> Ah.. probably Chris. So, in min check what happens is it always make sure
> that you have min instance count in the system.
>
>
> On Thu, May 29, 2014 at 11:42 AM, chris snow <ch...@gmail.com> wrote:
>>
>> Hi Nirmal,
>>
>> I've just see that it is the "Minimal Rule" in mincheck.drl that is
>> trying to spawn the instance, and not scaling.drl.
>>
>> This is why AutoscaleAlgorithm.getNextScaleUpPartition() is not getting
>> called?
>>
>> Cheers,
>>
>> Chris
>>
>>
>> On Wed, May 28, 2014 at 11:24 PM, Nirmal Fernando
>> <ni...@gmail.com> wrote:
>> > No, Chris. This happens via Drools rules located in scaling.drl file of
>> > conf
>> > directory.
>> >
>> >
>> > On Thu, May 29, 2014 at 3:39 AM, chris snow <ch...@gmail.com> wrote:
>> >>
>> >> Hi Nirmal,
>> >>
>> >> For some reason, that method is not getting called. I've pasted my
>> >> stacktrace again with the drools methods removed:
>> >>
>> >> ---
>> >> Daemon Thread [Thread-56] (Suspended (breakpoint at line 137 in
>> >> CloudControllerClient))
>> >> owns: CloudControllerClient (id=2433)
>> >> owns: DefaultAgenda (id=2434)
>> >> CloudControllerClient.spawnAnInstance(Partition, String, String,
>> >> String) line: 137
>> >> RuleTasksDelegator.delegateSpawn(PartitionContext, String, String)
>> >> line: 92
>> >> //
>> >> // drools methods hidden
>> >> //
>> >> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
>> >> FactHandle, Object) line: 86
>> >> ClusterMonitor.monitor() line: 99
>> >> ClusterMonitor.run() line: 72
>> >> Thread.run() line: 744
>> >> ---
>> >>
>> >> Would you expect to see AutoscaleAlgorithm.getNextScaleUpPartition
>> >> executed somewhere in the stack of calls?
>> >>
>> >> Many thanks,
>> >>
>> >> Chris
>> >>
>> >> On Wed, May 28, 2014 at 11:03 PM, Nirmal Fernando
>> >> <ni...@gmail.com> wrote:
>> >> > Hi Chris,
>> >> >
>> >> > PartitionMax is considered in
>> >> > AutoscaleAlgorithm.getNextScaleUpPartition
>> >> > method.
>> >> >
>> >> >
>> >> > On Thu, May 29, 2014 at 3:09 AM, chris snow <ch...@gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> I setup a breakpoint in eclipse at
>> >> >> CloudControllerClient.spawnAnInstance() and dumped the output
>> >> >> (below).
>> >> >>
>> >> >> I've scanned through the code in the stratos methods in the
>> >> >> stacktrace, but couldn't see any code that checks the partitionMax
>> >> >> value.
>> >> >>
>> >> >> Any ideas?
>> >> >>
>> >> >> Daemon Thread [Thread-56] (Suspended (breakpoint at line 137 in
>> >> >> CloudControllerClient))
>> >> >> owns: CloudControllerClient (id=2433)
>> >> >> owns: DefaultAgenda (id=2434)
>> >> >> CloudControllerClient.spawnAnInstance(Partition, String, String,
>> >> >> String) line: 137
>> >> >> RuleTasksDelegator.delegateSpawn(PartitionContext, String,
>> >> >> String)
>> >> >> line: 92
>> >> >> NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line:
>> >> >> not available [native method]
>> >> >> NativeMethodAccessorImpl.invoke(Object, Object[]) line: 57
>> >> >> DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
>> >> >> Method.invoke(Object, Object...) line: 606
>> >> >> ReflectiveAccessorOptimizer.getMethod(Object, String, Object[],
>> >> >> Class[], ExecutableStatement[]) line: 1104
>> >> >> ReflectiveAccessorOptimizer.getMethod(Object, String) line: 987
>> >> >> ReflectiveAccessorOptimizer.compileGetChain() line: 377
>> >> >> ReflectiveAccessorOptimizer.optimizeAccessor(ParserContext,
>> >> >> char[],
>> >> >> int, int, Object, Object, VariableResolverFactory, boolean, Class)
>> >> >> line: 143
>> >> >> ASTNode.optimize(Object, Object, VariableResolverFactory) line:
>> >> >> 159
>> >> >> ASTNode.getReducedValueAccelerated(Object, Object,
>> >> >> VariableResolverFactory) line: 115
>> >> >> MVELRuntime.execute(boolean, CompiledExpression, Object,
>> >> >> VariableResolverFactory) line: 85
>> >> >> CompiledExpression.getDirectValue(Object,
>> >> >> VariableResolverFactory)
>> >> >> line: 123
>> >> >> CompiledExpression.getValue(Object, VariableResolverFactory)
>> >> >> line:
>> >> >> 119
>> >> >> MVEL.executeExpression(Object, Object, VariableResolverFactory)
>> >> >> line:
>> >> >> 930
>> >> >> MVELConsequence.evaluate(KnowledgeHelper, WorkingMemory) line:
>> >> >> 104
>> >> >> DefaultAgenda.fireActivation(Activation) line: 1287
>> >> >> DefaultAgenda.fireNextItem(AgendaFilter) line: 1221
>> >> >> DefaultAgenda.fireAllRules(AgendaFilter, int) line: 1456
>> >> >>
>> >> >>
>> >> >> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules(AgendaFilter,
>> >> >> int) line: 710
>> >> >> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules() line:
>> >> >> 674
>> >> >> StatefulKnowledgeSessionImpl.fireAllRules() line: 230
>> >> >>
>> >> >> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
>> >> >> FactHandle, Object) line: 86
>> >> >> ClusterMonitor.monitor() line: 99
>> >> >> ClusterMonitor.run() line: 72
>> >> >> Thread.run() line: 744
>> >> >>
>> >> >> On Wed, May 28, 2014 at 3:37 AM, Lahiru Sandaruwan
>> >> >> <la...@wso2.com>
>> >> >> wrote:
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > On Tue, May 27, 2014 at 11:54 PM, chris snow <ch...@gmail.com>
>> >> >> > wrote:
>> >> >> >>
>> >> >> >> Hi Lahiru,
>> >> >> >>
>> >> >> >> I'll have a poke around - it will give me a good opportunity to
>> >> >> >> learn
>> >> >> >> some more Stratos code :)
>> >> >> >>
>> >> >> >
>> >> >> > Cool :)
>> >> >> >
>> >> >> > Debug logs should be able to show us where it went wrong.
>> >> >> >
>> >> >> > Thanks.
>> >> >> >>
>> >> >> >> Many thanks,
>> >> >> >>
>> >> >> >> Chris
>> >> >> >>
>> >> >> >> On Tue, May 27, 2014 at 7:45 AM, Lahiru Sandaruwan
>> >> >> >> <la...@wso2.com>
>> >> >> >> wrote:
>> >> >> >> > Hi Chris,
>> >> >> >> >
>> >> >> >> > Can you enable debug logs and try out?
>> >> >> >> > If so please send them over.
>> >> >> >> >
>> >> >> >> > Thanks.
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > On Tue, May 27, 2014 at 12:05 PM, chris snow
>> >> >> >> > <ch...@gmail.com>
>> >> >> >> > wrote:
>> >> >> >> >>
>> >> >> >> >> Deployment Policy partitionMax seems to be ignored. My
>> >> >> >> >> deployment
>> >> >> >> >> policy looks like this:
>> >> >> >> >>
>> >> >> >> >> {
>> >> >> >> >> "id": "economyDeployment",
>> >> >> >> >> "partitionGroup": [
>> >> >> >> >> {
>> >> >> >> >> "id": "G1",
>> >> >> >> >> "partitionAlgo": "one-after-another",
>> >> >> >> >> "partition": [
>> >> >> >> >> {
>> >> >> >> >> "id": "P1",
>> >> >> >> >> "partitionMax": "1",
>> >> >> >> >> "partitionMin": "1"
>> >> >> >> >> }
>> >> >> >> >> ]
>> >> >> >> >> }
>> >> >> >> >> ]
>> >> >> >> >> }
>> >> >> >> >>
>> >> >> >> >> Stratos initially starts two instances:
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> >> >> | ID | Name |
>> >> >> >> >> Status |
>> >> >> >> >> Task
>> >> >> >> >> State | Power State | Networks |
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 |
>> >> >> >> >> ACTIVE |
>> >> >> >> >> -
>> >> >> >> >> | Running | private=10.11.12.2 |
>> >> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 |
>> >> >> >> >> ACTIVE |
>> >> >> >> >> -
>> >> >> >> >> | Running | private=10.11.12.3 |
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> >> >> Then attempts to start more:
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> >> >> | ID | Name |
>> >> >> >> >> Status |
>> >> >> >> >> Task
>> >> >> >> >> State | Power State | Networks |
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> >> >> | 91029130-37b5-4df6-ad95-1d9f609a22c3 | php111php-13e | ERROR
>> >> >> >> >> |
>> >> >> >> >> -
>> >> >> >> >> | NOSTATE | |
>> >> >> >> >> | 71c1e361-df55-4f61-84c5-79774fef2beb | php111php-3d7 | ERROR
>> >> >> >> >> |
>> >> >> >> >> -
>> >> >> >> >> | NOSTATE | |
>> >> >> >> >> | b0d72602-cdb8-4621-ad12-911ade4d89b1 | php111php-412 | ERROR
>> >> >> >> >> |
>> >> >> >> >> -
>> >> >> >> >> | NOSTATE | |
>> >> >> >> >> | d4eac55d-bb7c-48d6-83f2-cb9a11581a18 | php111php-6f5 | ERROR
>> >> >> >> >> |
>> >> >> >> >> -
>> >> >> >> >> | NOSTATE | |
>> >> >> >> >> | 738a9534-2b95-4984-a51b-748bc7169b9a | php111php-73e | ERROR
>> >> >> >> >> |
>> >> >> >> >> -
>> >> >> >> >> | NOSTATE | |
>> >> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 |
>> >> >> >> >> ACTIVE |
>> >> >> >> >> -
>> >> >> >> >> | Running | private=10.11.12.2 |
>> >> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 |
>> >> >> >> >> ACTIVE |
>> >> >> >> >> -
>> >> >> >> >> | Running | private=10.11.12.3 |
>> >> >> >> >> | f570e654-024b-4240-bad2-8a11f04f23f0 | php111php-a30 | ERROR
>> >> >> >> >> |
>> >> >> >> >> -
>> >> >> >> >> | NOSTATE | |
>> >> >> >> >> | 0ab36824-9bea-4a95-aa22-7f935406d83d | php111php-bca | ERROR
>> >> >> >> >> |
>> >> >> >> >> -
>> >> >> >> >> | NOSTATE | |
>> >> >> >> >> | 02ed97df-02bc-4bff-b42d-9f2bcbd4f569 | php111php-f6f | ERROR
>> >> >> >> >> |
>> >> >> >> >> -
>> >> >> >> >> | NOSTATE | |
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> >> >>
>> >> >> >> >> I'm on branch: 4.0.0-incubating, using Openstack Havana +
>> >> >> >> >> Docker.
>> >> >> >> >>
>> >> >> >> >> Any ideas?
>> >> >> >> >>
>> >> >> >> >> Many thanks,
>> >> >> >> >>
>> >> >> >> >> Chris
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > --
>> >> >> >> > --
>> >> >> >> > Lahiru Sandaruwan
>> >> >> >> > Committer and PMC member, Apache Stratos,
>> >> >> >> > Senior Software Engineer,
>> >> >> >> > WSO2 Inc., http://wso2.com
>> >> >> >> > lean.enterprise.middleware
>> >> >> >> >
>> >> >> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
>> >> >> >> > blog: http://lahiruwrites.blogspot.com/
>> >> >> >> > twitter: http://twitter.com/lahirus
>> >> >> >> > linked-in:
>> >> >> >> > http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>> >> >> >> >
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> --
>> >> >> >> Check out my professional profile and connect with me on
>> >> >> >> LinkedIn.
>> >> >> >> http://lnkd.in/cw5k69
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > --
>> >> >> > --
>> >> >> > Lahiru Sandaruwan
>> >> >> > Committer and PMC member, Apache Stratos,
>> >> >> > Senior Software Engineer,
>> >> >> > WSO2 Inc., http://wso2.com
>> >> >> > lean.enterprise.middleware
>> >> >> >
>> >> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
>> >> >> > blog: http://lahiruwrites.blogspot.com/
>> >> >> > twitter: http://twitter.com/lahirus
>> >> >> > linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>> >> >> >
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Check out my professional profile and connect with me on LinkedIn.
>> >> >> http://lnkd.in/cw5k69
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Best Regards,
>> >> > Nirmal
>> >> >
>> >> > Nirmal Fernando.
>> >> > PPMC Member & Committer of Apache Stratos,
>> >> >
>> >> > Senior Software Engineer, WSO2 Inc.
>> >> >
>> >> > Blog: http://nirmalfdo.blogspot.com/
>> >>
>> >>
>> >>
>> >> --
>> >> Check out my professional profile and connect with me on LinkedIn.
>> >> http://lnkd.in/cw5k69
>> >
>> >
>> >
>> >
>> > --
>> > Best Regards,
>> > Nirmal
>> >
>> > Nirmal Fernando.
>> > PPMC Member & Committer of Apache Stratos,
>> > Senior Software Engineer, WSO2 Inc.
>> >
>> > Blog: http://nirmalfdo.blogspot.com/
>>
>>
>>
>> --
>> Check out my professional profile and connect with me on LinkedIn.
>> http://lnkd.in/cw5k69
>
>
>
>
> --
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
> Blog: http://nirmalfdo.blogspot.com/
--
Check out my professional profile and connect with me on LinkedIn.
http://lnkd.in/cw5k69
Re: autoscaling is ignoring partiionMax
Posted by Nirmal Fernando <ni...@gmail.com>.
Ah.. probably Chris. So, in min check what happens is it always make sure
that you have min instance count in the system.
On Thu, May 29, 2014 at 11:42 AM, chris snow <ch...@gmail.com> wrote:
> Hi Nirmal,
>
> I've just see that it is the "Minimal Rule" in mincheck.drl that is
> trying to spawn the instance, and not scaling.drl.
>
> This is why AutoscaleAlgorithm.getNextScaleUpPartition() is not getting
> called?
>
> Cheers,
>
> Chris
>
>
> On Wed, May 28, 2014 at 11:24 PM, Nirmal Fernando
> <ni...@gmail.com> wrote:
> > No, Chris. This happens via Drools rules located in scaling.drl file of
> conf
> > directory.
> >
> >
> > On Thu, May 29, 2014 at 3:39 AM, chris snow <ch...@gmail.com> wrote:
> >>
> >> Hi Nirmal,
> >>
> >> For some reason, that method is not getting called. I've pasted my
> >> stacktrace again with the drools methods removed:
> >>
> >> ---
> >> Daemon Thread [Thread-56] (Suspended (breakpoint at line 137 in
> >> CloudControllerClient))
> >> owns: CloudControllerClient (id=2433)
> >> owns: DefaultAgenda (id=2434)
> >> CloudControllerClient.spawnAnInstance(Partition, String, String,
> >> String) line: 137
> >> RuleTasksDelegator.delegateSpawn(PartitionContext, String, String)
> >> line: 92
> >> //
> >> // drools methods hidden
> >> //
> >> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
> >> FactHandle, Object) line: 86
> >> ClusterMonitor.monitor() line: 99
> >> ClusterMonitor.run() line: 72
> >> Thread.run() line: 744
> >> ---
> >>
> >> Would you expect to see AutoscaleAlgorithm.getNextScaleUpPartition
> >> executed somewhere in the stack of calls?
> >>
> >> Many thanks,
> >>
> >> Chris
> >>
> >> On Wed, May 28, 2014 at 11:03 PM, Nirmal Fernando
> >> <ni...@gmail.com> wrote:
> >> > Hi Chris,
> >> >
> >> > PartitionMax is considered in
> AutoscaleAlgorithm.getNextScaleUpPartition
> >> > method.
> >> >
> >> >
> >> > On Thu, May 29, 2014 at 3:09 AM, chris snow <ch...@gmail.com>
> wrote:
> >> >>
> >> >> I setup a breakpoint in eclipse at
> >> >> CloudControllerClient.spawnAnInstance() and dumped the output
> (below).
> >> >>
> >> >> I've scanned through the code in the stratos methods in the
> >> >> stacktrace, but couldn't see any code that checks the partitionMax
> >> >> value.
> >> >>
> >> >> Any ideas?
> >> >>
> >> >> Daemon Thread [Thread-56] (Suspended (breakpoint at line 137 in
> >> >> CloudControllerClient))
> >> >> owns: CloudControllerClient (id=2433)
> >> >> owns: DefaultAgenda (id=2434)
> >> >> CloudControllerClient.spawnAnInstance(Partition, String, String,
> >> >> String) line: 137
> >> >> RuleTasksDelegator.delegateSpawn(PartitionContext, String, String)
> >> >> line: 92
> >> >> NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line:
> >> >> not available [native method]
> >> >> NativeMethodAccessorImpl.invoke(Object, Object[]) line: 57
> >> >> DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
> >> >> Method.invoke(Object, Object...) line: 606
> >> >> ReflectiveAccessorOptimizer.getMethod(Object, String, Object[],
> >> >> Class[], ExecutableStatement[]) line: 1104
> >> >> ReflectiveAccessorOptimizer.getMethod(Object, String) line: 987
> >> >> ReflectiveAccessorOptimizer.compileGetChain() line: 377
> >> >> ReflectiveAccessorOptimizer.optimizeAccessor(ParserContext,
> char[],
> >> >> int, int, Object, Object, VariableResolverFactory, boolean, Class)
> >> >> line: 143
> >> >> ASTNode.optimize(Object, Object, VariableResolverFactory) line:
> 159
> >> >> ASTNode.getReducedValueAccelerated(Object, Object,
> >> >> VariableResolverFactory) line: 115
> >> >> MVELRuntime.execute(boolean, CompiledExpression, Object,
> >> >> VariableResolverFactory) line: 85
> >> >> CompiledExpression.getDirectValue(Object, VariableResolverFactory)
> >> >> line: 123
> >> >> CompiledExpression.getValue(Object, VariableResolverFactory) line:
> >> >> 119
> >> >> MVEL.executeExpression(Object, Object, VariableResolverFactory)
> >> >> line:
> >> >> 930
> >> >> MVELConsequence.evaluate(KnowledgeHelper, WorkingMemory) line: 104
> >> >> DefaultAgenda.fireActivation(Activation) line: 1287
> >> >> DefaultAgenda.fireNextItem(AgendaFilter) line: 1221
> >> >> DefaultAgenda.fireAllRules(AgendaFilter, int) line: 1456
> >> >>
> >> >>
> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules(AgendaFilter,
> >> >> int) line: 710
> >> >> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules() line:
> >> >> 674
> >> >> StatefulKnowledgeSessionImpl.fireAllRules() line: 230
> >> >> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
> >> >> FactHandle, Object) line: 86
> >> >> ClusterMonitor.monitor() line: 99
> >> >> ClusterMonitor.run() line: 72
> >> >> Thread.run() line: 744
> >> >>
> >> >> On Wed, May 28, 2014 at 3:37 AM, Lahiru Sandaruwan <lahirus@wso2.com
> >
> >> >> wrote:
> >> >> >
> >> >> >
> >> >> >
> >> >> > On Tue, May 27, 2014 at 11:54 PM, chris snow <ch...@gmail.com>
> >> >> > wrote:
> >> >> >>
> >> >> >> Hi Lahiru,
> >> >> >>
> >> >> >> I'll have a poke around - it will give me a good opportunity to
> >> >> >> learn
> >> >> >> some more Stratos code :)
> >> >> >>
> >> >> >
> >> >> > Cool :)
> >> >> >
> >> >> > Debug logs should be able to show us where it went wrong.
> >> >> >
> >> >> > Thanks.
> >> >> >>
> >> >> >> Many thanks,
> >> >> >>
> >> >> >> Chris
> >> >> >>
> >> >> >> On Tue, May 27, 2014 at 7:45 AM, Lahiru Sandaruwan
> >> >> >> <la...@wso2.com>
> >> >> >> wrote:
> >> >> >> > Hi Chris,
> >> >> >> >
> >> >> >> > Can you enable debug logs and try out?
> >> >> >> > If so please send them over.
> >> >> >> >
> >> >> >> > Thanks.
> >> >> >> >
> >> >> >> >
> >> >> >> > On Tue, May 27, 2014 at 12:05 PM, chris snow <
> chsnow123@gmail.com>
> >> >> >> > wrote:
> >> >> >> >>
> >> >> >> >> Deployment Policy partitionMax seems to be ignored. My
> >> >> >> >> deployment
> >> >> >> >> policy looks like this:
> >> >> >> >>
> >> >> >> >> {
> >> >> >> >> "id": "economyDeployment",
> >> >> >> >> "partitionGroup": [
> >> >> >> >> {
> >> >> >> >> "id": "G1",
> >> >> >> >> "partitionAlgo": "one-after-another",
> >> >> >> >> "partition": [
> >> >> >> >> {
> >> >> >> >> "id": "P1",
> >> >> >> >> "partitionMax": "1",
> >> >> >> >> "partitionMin": "1"
> >> >> >> >> }
> >> >> >> >> ]
> >> >> >> >> }
> >> >> >> >> ]
> >> >> >> >> }
> >> >> >> >>
> >> >> >> >> Stratos initially starts two instances:
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> >> >> | ID | Name |
> Status |
> >> >> >> >> Task
> >> >> >> >> State | Power State | Networks |
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 |
> ACTIVE |
> >> >> >> >> -
> >> >> >> >> | Running | private=10.11.12.2 |
> >> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 |
> ACTIVE |
> >> >> >> >> -
> >> >> >> >> | Running | private=10.11.12.3 |
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> >> >> Then attempts to start more:
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> >> >> | ID | Name |
> Status |
> >> >> >> >> Task
> >> >> >> >> State | Power State | Networks |
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> >> >> | 91029130-37b5-4df6-ad95-1d9f609a22c3 | php111php-13e | ERROR
> |
> >> >> >> >> -
> >> >> >> >> | NOSTATE | |
> >> >> >> >> | 71c1e361-df55-4f61-84c5-79774fef2beb | php111php-3d7 | ERROR
> |
> >> >> >> >> -
> >> >> >> >> | NOSTATE | |
> >> >> >> >> | b0d72602-cdb8-4621-ad12-911ade4d89b1 | php111php-412 | ERROR
> |
> >> >> >> >> -
> >> >> >> >> | NOSTATE | |
> >> >> >> >> | d4eac55d-bb7c-48d6-83f2-cb9a11581a18 | php111php-6f5 | ERROR
> |
> >> >> >> >> -
> >> >> >> >> | NOSTATE | |
> >> >> >> >> | 738a9534-2b95-4984-a51b-748bc7169b9a | php111php-73e | ERROR
> |
> >> >> >> >> -
> >> >> >> >> | NOSTATE | |
> >> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 |
> ACTIVE |
> >> >> >> >> -
> >> >> >> >> | Running | private=10.11.12.2 |
> >> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 |
> ACTIVE |
> >> >> >> >> -
> >> >> >> >> | Running | private=10.11.12.3 |
> >> >> >> >> | f570e654-024b-4240-bad2-8a11f04f23f0 | php111php-a30 | ERROR
> |
> >> >> >> >> -
> >> >> >> >> | NOSTATE | |
> >> >> >> >> | 0ab36824-9bea-4a95-aa22-7f935406d83d | php111php-bca | ERROR
> |
> >> >> >> >> -
> >> >> >> >> | NOSTATE | |
> >> >> >> >> | 02ed97df-02bc-4bff-b42d-9f2bcbd4f569 | php111php-f6f | ERROR
> |
> >> >> >> >> -
> >> >> >> >> | NOSTATE | |
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> >> >>
> >> >> >> >> I'm on branch: 4.0.0-incubating, using Openstack Havana +
> Docker.
> >> >> >> >>
> >> >> >> >> Any ideas?
> >> >> >> >>
> >> >> >> >> Many thanks,
> >> >> >> >>
> >> >> >> >> Chris
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> > --
> >> >> >> > --
> >> >> >> > Lahiru Sandaruwan
> >> >> >> > Committer and PMC member, Apache Stratos,
> >> >> >> > Senior Software Engineer,
> >> >> >> > WSO2 Inc., http://wso2.com
> >> >> >> > lean.enterprise.middleware
> >> >> >> >
> >> >> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
> >> >> >> > blog: http://lahiruwrites.blogspot.com/
> >> >> >> > twitter: http://twitter.com/lahirus
> >> >> >> > linked-in:
> http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
> >> >> >> >
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> --
> >> >> >> Check out my professional profile and connect with me on LinkedIn.
> >> >> >> http://lnkd.in/cw5k69
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > --
> >> >> > Lahiru Sandaruwan
> >> >> > Committer and PMC member, Apache Stratos,
> >> >> > Senior Software Engineer,
> >> >> > WSO2 Inc., http://wso2.com
> >> >> > lean.enterprise.middleware
> >> >> >
> >> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
> >> >> > blog: http://lahiruwrites.blogspot.com/
> >> >> > twitter: http://twitter.com/lahirus
> >> >> > linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Check out my professional profile and connect with me on LinkedIn.
> >> >> http://lnkd.in/cw5k69
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > Best Regards,
> >> > Nirmal
> >> >
> >> > Nirmal Fernando.
> >> > PPMC Member & Committer of Apache Stratos,
> >> >
> >> > Senior Software Engineer, WSO2 Inc.
> >> >
> >> > Blog: http://nirmalfdo.blogspot.com/
> >>
> >>
> >>
> >> --
> >> Check out my professional profile and connect with me on LinkedIn.
> >> http://lnkd.in/cw5k69
> >
> >
> >
> >
> > --
> > Best Regards,
> > Nirmal
> >
> > Nirmal Fernando.
> > PPMC Member & Committer of Apache Stratos,
> > Senior Software Engineer, WSO2 Inc.
> >
> > Blog: http://nirmalfdo.blogspot.com/
>
>
>
> --
> Check out my professional profile and connect with me on LinkedIn.
> http://lnkd.in/cw5k69
>
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
Re: autoscaling is ignoring partiionMax
Posted by chris snow <ch...@gmail.com>.
Hi Nirmal,
I've just see that it is the "Minimal Rule" in mincheck.drl that is
trying to spawn the instance, and not scaling.drl.
This is why AutoscaleAlgorithm.getNextScaleUpPartition() is not getting called?
Cheers,
Chris
On Wed, May 28, 2014 at 11:24 PM, Nirmal Fernando
<ni...@gmail.com> wrote:
> No, Chris. This happens via Drools rules located in scaling.drl file of conf
> directory.
>
>
> On Thu, May 29, 2014 at 3:39 AM, chris snow <ch...@gmail.com> wrote:
>>
>> Hi Nirmal,
>>
>> For some reason, that method is not getting called. I've pasted my
>> stacktrace again with the drools methods removed:
>>
>> ---
>> Daemon Thread [Thread-56] (Suspended (breakpoint at line 137 in
>> CloudControllerClient))
>> owns: CloudControllerClient (id=2433)
>> owns: DefaultAgenda (id=2434)
>> CloudControllerClient.spawnAnInstance(Partition, String, String,
>> String) line: 137
>> RuleTasksDelegator.delegateSpawn(PartitionContext, String, String)
>> line: 92
>> //
>> // drools methods hidden
>> //
>> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
>> FactHandle, Object) line: 86
>> ClusterMonitor.monitor() line: 99
>> ClusterMonitor.run() line: 72
>> Thread.run() line: 744
>> ---
>>
>> Would you expect to see AutoscaleAlgorithm.getNextScaleUpPartition
>> executed somewhere in the stack of calls?
>>
>> Many thanks,
>>
>> Chris
>>
>> On Wed, May 28, 2014 at 11:03 PM, Nirmal Fernando
>> <ni...@gmail.com> wrote:
>> > Hi Chris,
>> >
>> > PartitionMax is considered in AutoscaleAlgorithm.getNextScaleUpPartition
>> > method.
>> >
>> >
>> > On Thu, May 29, 2014 at 3:09 AM, chris snow <ch...@gmail.com> wrote:
>> >>
>> >> I setup a breakpoint in eclipse at
>> >> CloudControllerClient.spawnAnInstance() and dumped the output (below).
>> >>
>> >> I've scanned through the code in the stratos methods in the
>> >> stacktrace, but couldn't see any code that checks the partitionMax
>> >> value.
>> >>
>> >> Any ideas?
>> >>
>> >> Daemon Thread [Thread-56] (Suspended (breakpoint at line 137 in
>> >> CloudControllerClient))
>> >> owns: CloudControllerClient (id=2433)
>> >> owns: DefaultAgenda (id=2434)
>> >> CloudControllerClient.spawnAnInstance(Partition, String, String,
>> >> String) line: 137
>> >> RuleTasksDelegator.delegateSpawn(PartitionContext, String, String)
>> >> line: 92
>> >> NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line:
>> >> not available [native method]
>> >> NativeMethodAccessorImpl.invoke(Object, Object[]) line: 57
>> >> DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
>> >> Method.invoke(Object, Object...) line: 606
>> >> ReflectiveAccessorOptimizer.getMethod(Object, String, Object[],
>> >> Class[], ExecutableStatement[]) line: 1104
>> >> ReflectiveAccessorOptimizer.getMethod(Object, String) line: 987
>> >> ReflectiveAccessorOptimizer.compileGetChain() line: 377
>> >> ReflectiveAccessorOptimizer.optimizeAccessor(ParserContext, char[],
>> >> int, int, Object, Object, VariableResolverFactory, boolean, Class)
>> >> line: 143
>> >> ASTNode.optimize(Object, Object, VariableResolverFactory) line: 159
>> >> ASTNode.getReducedValueAccelerated(Object, Object,
>> >> VariableResolverFactory) line: 115
>> >> MVELRuntime.execute(boolean, CompiledExpression, Object,
>> >> VariableResolverFactory) line: 85
>> >> CompiledExpression.getDirectValue(Object, VariableResolverFactory)
>> >> line: 123
>> >> CompiledExpression.getValue(Object, VariableResolverFactory) line:
>> >> 119
>> >> MVEL.executeExpression(Object, Object, VariableResolverFactory)
>> >> line:
>> >> 930
>> >> MVELConsequence.evaluate(KnowledgeHelper, WorkingMemory) line: 104
>> >> DefaultAgenda.fireActivation(Activation) line: 1287
>> >> DefaultAgenda.fireNextItem(AgendaFilter) line: 1221
>> >> DefaultAgenda.fireAllRules(AgendaFilter, int) line: 1456
>> >>
>> >> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules(AgendaFilter,
>> >> int) line: 710
>> >> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules() line:
>> >> 674
>> >> StatefulKnowledgeSessionImpl.fireAllRules() line: 230
>> >> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
>> >> FactHandle, Object) line: 86
>> >> ClusterMonitor.monitor() line: 99
>> >> ClusterMonitor.run() line: 72
>> >> Thread.run() line: 744
>> >>
>> >> On Wed, May 28, 2014 at 3:37 AM, Lahiru Sandaruwan <la...@wso2.com>
>> >> wrote:
>> >> >
>> >> >
>> >> >
>> >> > On Tue, May 27, 2014 at 11:54 PM, chris snow <ch...@gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> Hi Lahiru,
>> >> >>
>> >> >> I'll have a poke around - it will give me a good opportunity to
>> >> >> learn
>> >> >> some more Stratos code :)
>> >> >>
>> >> >
>> >> > Cool :)
>> >> >
>> >> > Debug logs should be able to show us where it went wrong.
>> >> >
>> >> > Thanks.
>> >> >>
>> >> >> Many thanks,
>> >> >>
>> >> >> Chris
>> >> >>
>> >> >> On Tue, May 27, 2014 at 7:45 AM, Lahiru Sandaruwan
>> >> >> <la...@wso2.com>
>> >> >> wrote:
>> >> >> > Hi Chris,
>> >> >> >
>> >> >> > Can you enable debug logs and try out?
>> >> >> > If so please send them over.
>> >> >> >
>> >> >> > Thanks.
>> >> >> >
>> >> >> >
>> >> >> > On Tue, May 27, 2014 at 12:05 PM, chris snow <ch...@gmail.com>
>> >> >> > wrote:
>> >> >> >>
>> >> >> >> Deployment Policy partitionMax seems to be ignored. My
>> >> >> >> deployment
>> >> >> >> policy looks like this:
>> >> >> >>
>> >> >> >> {
>> >> >> >> "id": "economyDeployment",
>> >> >> >> "partitionGroup": [
>> >> >> >> {
>> >> >> >> "id": "G1",
>> >> >> >> "partitionAlgo": "one-after-another",
>> >> >> >> "partition": [
>> >> >> >> {
>> >> >> >> "id": "P1",
>> >> >> >> "partitionMax": "1",
>> >> >> >> "partitionMin": "1"
>> >> >> >> }
>> >> >> >> ]
>> >> >> >> }
>> >> >> >> ]
>> >> >> >> }
>> >> >> >>
>> >> >> >> Stratos initially starts two instances:
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> >> | ID | Name | Status |
>> >> >> >> Task
>> >> >> >> State | Power State | Networks |
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 | ACTIVE |
>> >> >> >> -
>> >> >> >> | Running | private=10.11.12.2 |
>> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 | ACTIVE |
>> >> >> >> -
>> >> >> >> | Running | private=10.11.12.3 |
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> >> Then attempts to start more:
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> >> | ID | Name | Status |
>> >> >> >> Task
>> >> >> >> State | Power State | Networks |
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> >> | 91029130-37b5-4df6-ad95-1d9f609a22c3 | php111php-13e | ERROR |
>> >> >> >> -
>> >> >> >> | NOSTATE | |
>> >> >> >> | 71c1e361-df55-4f61-84c5-79774fef2beb | php111php-3d7 | ERROR |
>> >> >> >> -
>> >> >> >> | NOSTATE | |
>> >> >> >> | b0d72602-cdb8-4621-ad12-911ade4d89b1 | php111php-412 | ERROR |
>> >> >> >> -
>> >> >> >> | NOSTATE | |
>> >> >> >> | d4eac55d-bb7c-48d6-83f2-cb9a11581a18 | php111php-6f5 | ERROR |
>> >> >> >> -
>> >> >> >> | NOSTATE | |
>> >> >> >> | 738a9534-2b95-4984-a51b-748bc7169b9a | php111php-73e | ERROR |
>> >> >> >> -
>> >> >> >> | NOSTATE | |
>> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 | ACTIVE |
>> >> >> >> -
>> >> >> >> | Running | private=10.11.12.2 |
>> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 | ACTIVE |
>> >> >> >> -
>> >> >> >> | Running | private=10.11.12.3 |
>> >> >> >> | f570e654-024b-4240-bad2-8a11f04f23f0 | php111php-a30 | ERROR |
>> >> >> >> -
>> >> >> >> | NOSTATE | |
>> >> >> >> | 0ab36824-9bea-4a95-aa22-7f935406d83d | php111php-bca | ERROR |
>> >> >> >> -
>> >> >> >> | NOSTATE | |
>> >> >> >> | 02ed97df-02bc-4bff-b42d-9f2bcbd4f569 | php111php-f6f | ERROR |
>> >> >> >> -
>> >> >> >> | NOSTATE | |
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> >>
>> >> >> >> I'm on branch: 4.0.0-incubating, using Openstack Havana + Docker.
>> >> >> >>
>> >> >> >> Any ideas?
>> >> >> >>
>> >> >> >> Many thanks,
>> >> >> >>
>> >> >> >> Chris
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > --
>> >> >> > --
>> >> >> > Lahiru Sandaruwan
>> >> >> > Committer and PMC member, Apache Stratos,
>> >> >> > Senior Software Engineer,
>> >> >> > WSO2 Inc., http://wso2.com
>> >> >> > lean.enterprise.middleware
>> >> >> >
>> >> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
>> >> >> > blog: http://lahiruwrites.blogspot.com/
>> >> >> > twitter: http://twitter.com/lahirus
>> >> >> > linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>> >> >> >
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Check out my professional profile and connect with me on LinkedIn.
>> >> >> http://lnkd.in/cw5k69
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > --
>> >> > Lahiru Sandaruwan
>> >> > Committer and PMC member, Apache Stratos,
>> >> > Senior Software Engineer,
>> >> > WSO2 Inc., http://wso2.com
>> >> > lean.enterprise.middleware
>> >> >
>> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
>> >> > blog: http://lahiruwrites.blogspot.com/
>> >> > twitter: http://twitter.com/lahirus
>> >> > linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Check out my professional profile and connect with me on LinkedIn.
>> >> http://lnkd.in/cw5k69
>> >
>> >
>> >
>> >
>> > --
>> > Best Regards,
>> > Nirmal
>> >
>> > Nirmal Fernando.
>> > PPMC Member & Committer of Apache Stratos,
>> >
>> > Senior Software Engineer, WSO2 Inc.
>> >
>> > Blog: http://nirmalfdo.blogspot.com/
>>
>>
>>
>> --
>> Check out my professional profile and connect with me on LinkedIn.
>> http://lnkd.in/cw5k69
>
>
>
>
> --
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
> Blog: http://nirmalfdo.blogspot.com/
--
Check out my professional profile and connect with me on LinkedIn.
http://lnkd.in/cw5k69
Re: autoscaling is ignoring partiionMax
Posted by Nirmal Fernando <ni...@gmail.com>.
No, Chris. This happens via Drools rules located in scaling.drl file of
conf directory.
On Thu, May 29, 2014 at 3:39 AM, chris snow <ch...@gmail.com> wrote:
> Hi Nirmal,
>
> For some reason, that method is not getting called. I've pasted my
> stacktrace again with the drools methods removed:
>
> ---
> Daemon Thread [Thread-56] (Suspended (breakpoint at line 137 in
> CloudControllerClient))
> owns: CloudControllerClient (id=2433)
> owns: DefaultAgenda (id=2434)
> CloudControllerClient.spawnAnInstance(Partition, String, String,
> String) line: 137
> RuleTasksDelegator.delegateSpawn(PartitionContext, String, String)
> line: 92
> //
> // drools methods hidden
> //
> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
> FactHandle, Object) line: 86
> ClusterMonitor.monitor() line: 99
> ClusterMonitor.run() line: 72
> Thread.run() line: 744
> ---
>
> Would you expect to see AutoscaleAlgorithm.getNextScaleUpPartition
> executed somewhere in the stack of calls?
>
> Many thanks,
>
> Chris
>
> On Wed, May 28, 2014 at 11:03 PM, Nirmal Fernando
> <ni...@gmail.com> wrote:
> > Hi Chris,
> >
> > PartitionMax is considered in AutoscaleAlgorithm.getNextScaleUpPartition
> > method.
> >
> >
> > On Thu, May 29, 2014 at 3:09 AM, chris snow <ch...@gmail.com> wrote:
> >>
> >> I setup a breakpoint in eclipse at
> >> CloudControllerClient.spawnAnInstance() and dumped the output (below).
> >>
> >> I've scanned through the code in the stratos methods in the
> >> stacktrace, but couldn't see any code that checks the partitionMax
> >> value.
> >>
> >> Any ideas?
> >>
> >> Daemon Thread [Thread-56] (Suspended (breakpoint at line 137 in
> >> CloudControllerClient))
> >> owns: CloudControllerClient (id=2433)
> >> owns: DefaultAgenda (id=2434)
> >> CloudControllerClient.spawnAnInstance(Partition, String, String,
> >> String) line: 137
> >> RuleTasksDelegator.delegateSpawn(PartitionContext, String, String)
> >> line: 92
> >> NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line:
> >> not available [native method]
> >> NativeMethodAccessorImpl.invoke(Object, Object[]) line: 57
> >> DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
> >> Method.invoke(Object, Object...) line: 606
> >> ReflectiveAccessorOptimizer.getMethod(Object, String, Object[],
> >> Class[], ExecutableStatement[]) line: 1104
> >> ReflectiveAccessorOptimizer.getMethod(Object, String) line: 987
> >> ReflectiveAccessorOptimizer.compileGetChain() line: 377
> >> ReflectiveAccessorOptimizer.optimizeAccessor(ParserContext, char[],
> >> int, int, Object, Object, VariableResolverFactory, boolean, Class)
> >> line: 143
> >> ASTNode.optimize(Object, Object, VariableResolverFactory) line: 159
> >> ASTNode.getReducedValueAccelerated(Object, Object,
> >> VariableResolverFactory) line: 115
> >> MVELRuntime.execute(boolean, CompiledExpression, Object,
> >> VariableResolverFactory) line: 85
> >> CompiledExpression.getDirectValue(Object, VariableResolverFactory)
> >> line: 123
> >> CompiledExpression.getValue(Object, VariableResolverFactory) line:
> 119
> >> MVEL.executeExpression(Object, Object, VariableResolverFactory) line:
> >> 930
> >> MVELConsequence.evaluate(KnowledgeHelper, WorkingMemory) line: 104
> >> DefaultAgenda.fireActivation(Activation) line: 1287
> >> DefaultAgenda.fireNextItem(AgendaFilter) line: 1221
> >> DefaultAgenda.fireAllRules(AgendaFilter, int) line: 1456
> >>
> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules(AgendaFilter,
> >> int) line: 710
> >> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules() line: 674
> >> StatefulKnowledgeSessionImpl.fireAllRules() line: 230
> >> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
> >> FactHandle, Object) line: 86
> >> ClusterMonitor.monitor() line: 99
> >> ClusterMonitor.run() line: 72
> >> Thread.run() line: 744
> >>
> >> On Wed, May 28, 2014 at 3:37 AM, Lahiru Sandaruwan <la...@wso2.com>
> >> wrote:
> >> >
> >> >
> >> >
> >> > On Tue, May 27, 2014 at 11:54 PM, chris snow <ch...@gmail.com>
> >> > wrote:
> >> >>
> >> >> Hi Lahiru,
> >> >>
> >> >> I'll have a poke around - it will give me a good opportunity to learn
> >> >> some more Stratos code :)
> >> >>
> >> >
> >> > Cool :)
> >> >
> >> > Debug logs should be able to show us where it went wrong.
> >> >
> >> > Thanks.
> >> >>
> >> >> Many thanks,
> >> >>
> >> >> Chris
> >> >>
> >> >> On Tue, May 27, 2014 at 7:45 AM, Lahiru Sandaruwan <lahirus@wso2.com
> >
> >> >> wrote:
> >> >> > Hi Chris,
> >> >> >
> >> >> > Can you enable debug logs and try out?
> >> >> > If so please send them over.
> >> >> >
> >> >> > Thanks.
> >> >> >
> >> >> >
> >> >> > On Tue, May 27, 2014 at 12:05 PM, chris snow <ch...@gmail.com>
> >> >> > wrote:
> >> >> >>
> >> >> >> Deployment Policy partitionMax seems to be ignored. My deployment
> >> >> >> policy looks like this:
> >> >> >>
> >> >> >> {
> >> >> >> "id": "economyDeployment",
> >> >> >> "partitionGroup": [
> >> >> >> {
> >> >> >> "id": "G1",
> >> >> >> "partitionAlgo": "one-after-another",
> >> >> >> "partition": [
> >> >> >> {
> >> >> >> "id": "P1",
> >> >> >> "partitionMax": "1",
> >> >> >> "partitionMin": "1"
> >> >> >> }
> >> >> >> ]
> >> >> >> }
> >> >> >> ]
> >> >> >> }
> >> >> >>
> >> >> >> Stratos initially starts two instances:
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> >> | ID | Name | Status |
> >> >> >> Task
> >> >> >> State | Power State | Networks |
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 | ACTIVE |
> -
> >> >> >> | Running | private=10.11.12.2 |
> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 | ACTIVE |
> -
> >> >> >> | Running | private=10.11.12.3 |
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> >> Then attempts to start more:
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> >> | ID | Name | Status |
> >> >> >> Task
> >> >> >> State | Power State | Networks |
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> >> | 91029130-37b5-4df6-ad95-1d9f609a22c3 | php111php-13e | ERROR |
> -
> >> >> >> | NOSTATE | |
> >> >> >> | 71c1e361-df55-4f61-84c5-79774fef2beb | php111php-3d7 | ERROR |
> -
> >> >> >> | NOSTATE | |
> >> >> >> | b0d72602-cdb8-4621-ad12-911ade4d89b1 | php111php-412 | ERROR |
> -
> >> >> >> | NOSTATE | |
> >> >> >> | d4eac55d-bb7c-48d6-83f2-cb9a11581a18 | php111php-6f5 | ERROR |
> -
> >> >> >> | NOSTATE | |
> >> >> >> | 738a9534-2b95-4984-a51b-748bc7169b9a | php111php-73e | ERROR |
> -
> >> >> >> | NOSTATE | |
> >> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 | ACTIVE |
> -
> >> >> >> | Running | private=10.11.12.2 |
> >> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 | ACTIVE |
> -
> >> >> >> | Running | private=10.11.12.3 |
> >> >> >> | f570e654-024b-4240-bad2-8a11f04f23f0 | php111php-a30 | ERROR |
> -
> >> >> >> | NOSTATE | |
> >> >> >> | 0ab36824-9bea-4a95-aa22-7f935406d83d | php111php-bca | ERROR |
> -
> >> >> >> | NOSTATE | |
> >> >> >> | 02ed97df-02bc-4bff-b42d-9f2bcbd4f569 | php111php-f6f | ERROR |
> -
> >> >> >> | NOSTATE | |
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> >>
> >> >> >> I'm on branch: 4.0.0-incubating, using Openstack Havana + Docker.
> >> >> >>
> >> >> >> Any ideas?
> >> >> >>
> >> >> >> Many thanks,
> >> >> >>
> >> >> >> Chris
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > --
> >> >> > Lahiru Sandaruwan
> >> >> > Committer and PMC member, Apache Stratos,
> >> >> > Senior Software Engineer,
> >> >> > WSO2 Inc., http://wso2.com
> >> >> > lean.enterprise.middleware
> >> >> >
> >> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
> >> >> > blog: http://lahiruwrites.blogspot.com/
> >> >> > twitter: http://twitter.com/lahirus
> >> >> > linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Check out my professional profile and connect with me on LinkedIn.
> >> >> http://lnkd.in/cw5k69
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > --
> >> > Lahiru Sandaruwan
> >> > Committer and PMC member, Apache Stratos,
> >> > Senior Software Engineer,
> >> > WSO2 Inc., http://wso2.com
> >> > lean.enterprise.middleware
> >> >
> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
> >> > blog: http://lahiruwrites.blogspot.com/
> >> > twitter: http://twitter.com/lahirus
> >> > linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
> >> >
> >>
> >>
> >>
> >> --
> >> Check out my professional profile and connect with me on LinkedIn.
> >> http://lnkd.in/cw5k69
> >
> >
> >
> >
> > --
> > Best Regards,
> > Nirmal
> >
> > Nirmal Fernando.
> > PPMC Member & Committer of Apache Stratos,
> >
> > Senior Software Engineer, WSO2 Inc.
> >
> > Blog: http://nirmalfdo.blogspot.com/
>
>
>
> --
> Check out my professional profile and connect with me on LinkedIn.
> http://lnkd.in/cw5k69
>
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
Re: autoscaling is ignoring partiionMax
Posted by chris snow <ch...@gmail.com>.
Hi Nirmal,
For some reason, that method is not getting called. I've pasted my
stacktrace again with the drools methods removed:
---
Daemon Thread [Thread-56] (Suspended (breakpoint at line 137 in
CloudControllerClient))
owns: CloudControllerClient (id=2433)
owns: DefaultAgenda (id=2434)
CloudControllerClient.spawnAnInstance(Partition, String, String,
String) line: 137
RuleTasksDelegator.delegateSpawn(PartitionContext, String, String) line: 92
//
// drools methods hidden
//
AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
FactHandle, Object) line: 86
ClusterMonitor.monitor() line: 99
ClusterMonitor.run() line: 72
Thread.run() line: 744
---
Would you expect to see AutoscaleAlgorithm.getNextScaleUpPartition
executed somewhere in the stack of calls?
Many thanks,
Chris
On Wed, May 28, 2014 at 11:03 PM, Nirmal Fernando
<ni...@gmail.com> wrote:
> Hi Chris,
>
> PartitionMax is considered in AutoscaleAlgorithm.getNextScaleUpPartition
> method.
>
>
> On Thu, May 29, 2014 at 3:09 AM, chris snow <ch...@gmail.com> wrote:
>>
>> I setup a breakpoint in eclipse at
>> CloudControllerClient.spawnAnInstance() and dumped the output (below).
>>
>> I've scanned through the code in the stratos methods in the
>> stacktrace, but couldn't see any code that checks the partitionMax
>> value.
>>
>> Any ideas?
>>
>> Daemon Thread [Thread-56] (Suspended (breakpoint at line 137 in
>> CloudControllerClient))
>> owns: CloudControllerClient (id=2433)
>> owns: DefaultAgenda (id=2434)
>> CloudControllerClient.spawnAnInstance(Partition, String, String,
>> String) line: 137
>> RuleTasksDelegator.delegateSpawn(PartitionContext, String, String)
>> line: 92
>> NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line:
>> not available [native method]
>> NativeMethodAccessorImpl.invoke(Object, Object[]) line: 57
>> DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
>> Method.invoke(Object, Object...) line: 606
>> ReflectiveAccessorOptimizer.getMethod(Object, String, Object[],
>> Class[], ExecutableStatement[]) line: 1104
>> ReflectiveAccessorOptimizer.getMethod(Object, String) line: 987
>> ReflectiveAccessorOptimizer.compileGetChain() line: 377
>> ReflectiveAccessorOptimizer.optimizeAccessor(ParserContext, char[],
>> int, int, Object, Object, VariableResolverFactory, boolean, Class)
>> line: 143
>> ASTNode.optimize(Object, Object, VariableResolverFactory) line: 159
>> ASTNode.getReducedValueAccelerated(Object, Object,
>> VariableResolverFactory) line: 115
>> MVELRuntime.execute(boolean, CompiledExpression, Object,
>> VariableResolverFactory) line: 85
>> CompiledExpression.getDirectValue(Object, VariableResolverFactory)
>> line: 123
>> CompiledExpression.getValue(Object, VariableResolverFactory) line: 119
>> MVEL.executeExpression(Object, Object, VariableResolverFactory) line:
>> 930
>> MVELConsequence.evaluate(KnowledgeHelper, WorkingMemory) line: 104
>> DefaultAgenda.fireActivation(Activation) line: 1287
>> DefaultAgenda.fireNextItem(AgendaFilter) line: 1221
>> DefaultAgenda.fireAllRules(AgendaFilter, int) line: 1456
>> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules(AgendaFilter,
>> int) line: 710
>> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules() line: 674
>> StatefulKnowledgeSessionImpl.fireAllRules() line: 230
>> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
>> FactHandle, Object) line: 86
>> ClusterMonitor.monitor() line: 99
>> ClusterMonitor.run() line: 72
>> Thread.run() line: 744
>>
>> On Wed, May 28, 2014 at 3:37 AM, Lahiru Sandaruwan <la...@wso2.com>
>> wrote:
>> >
>> >
>> >
>> > On Tue, May 27, 2014 at 11:54 PM, chris snow <ch...@gmail.com>
>> > wrote:
>> >>
>> >> Hi Lahiru,
>> >>
>> >> I'll have a poke around - it will give me a good opportunity to learn
>> >> some more Stratos code :)
>> >>
>> >
>> > Cool :)
>> >
>> > Debug logs should be able to show us where it went wrong.
>> >
>> > Thanks.
>> >>
>> >> Many thanks,
>> >>
>> >> Chris
>> >>
>> >> On Tue, May 27, 2014 at 7:45 AM, Lahiru Sandaruwan <la...@wso2.com>
>> >> wrote:
>> >> > Hi Chris,
>> >> >
>> >> > Can you enable debug logs and try out?
>> >> > If so please send them over.
>> >> >
>> >> > Thanks.
>> >> >
>> >> >
>> >> > On Tue, May 27, 2014 at 12:05 PM, chris snow <ch...@gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> Deployment Policy partitionMax seems to be ignored. My deployment
>> >> >> policy looks like this:
>> >> >>
>> >> >> {
>> >> >> "id": "economyDeployment",
>> >> >> "partitionGroup": [
>> >> >> {
>> >> >> "id": "G1",
>> >> >> "partitionAlgo": "one-after-another",
>> >> >> "partition": [
>> >> >> {
>> >> >> "id": "P1",
>> >> >> "partitionMax": "1",
>> >> >> "partitionMin": "1"
>> >> >> }
>> >> >> ]
>> >> >> }
>> >> >> ]
>> >> >> }
>> >> >>
>> >> >> Stratos initially starts two instances:
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> | ID | Name | Status |
>> >> >> Task
>> >> >> State | Power State | Networks |
>> >> >>
>> >> >>
>> >> >>
>> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 | ACTIVE | -
>> >> >> | Running | private=10.11.12.2 |
>> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 | ACTIVE | -
>> >> >> | Running | private=10.11.12.3 |
>> >> >>
>> >> >>
>> >> >>
>> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> Then attempts to start more:
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> | ID | Name | Status |
>> >> >> Task
>> >> >> State | Power State | Networks |
>> >> >>
>> >> >>
>> >> >>
>> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >> | 91029130-37b5-4df6-ad95-1d9f609a22c3 | php111php-13e | ERROR | -
>> >> >> | NOSTATE | |
>> >> >> | 71c1e361-df55-4f61-84c5-79774fef2beb | php111php-3d7 | ERROR | -
>> >> >> | NOSTATE | |
>> >> >> | b0d72602-cdb8-4621-ad12-911ade4d89b1 | php111php-412 | ERROR | -
>> >> >> | NOSTATE | |
>> >> >> | d4eac55d-bb7c-48d6-83f2-cb9a11581a18 | php111php-6f5 | ERROR | -
>> >> >> | NOSTATE | |
>> >> >> | 738a9534-2b95-4984-a51b-748bc7169b9a | php111php-73e | ERROR | -
>> >> >> | NOSTATE | |
>> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 | ACTIVE | -
>> >> >> | Running | private=10.11.12.2 |
>> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 | ACTIVE | -
>> >> >> | Running | private=10.11.12.3 |
>> >> >> | f570e654-024b-4240-bad2-8a11f04f23f0 | php111php-a30 | ERROR | -
>> >> >> | NOSTATE | |
>> >> >> | 0ab36824-9bea-4a95-aa22-7f935406d83d | php111php-bca | ERROR | -
>> >> >> | NOSTATE | |
>> >> >> | 02ed97df-02bc-4bff-b42d-9f2bcbd4f569 | php111php-f6f | ERROR | -
>> >> >> | NOSTATE | |
>> >> >>
>> >> >>
>> >> >>
>> >> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> >>
>> >> >> I'm on branch: 4.0.0-incubating, using Openstack Havana + Docker.
>> >> >>
>> >> >> Any ideas?
>> >> >>
>> >> >> Many thanks,
>> >> >>
>> >> >> Chris
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > --
>> >> > Lahiru Sandaruwan
>> >> > Committer and PMC member, Apache Stratos,
>> >> > Senior Software Engineer,
>> >> > WSO2 Inc., http://wso2.com
>> >> > lean.enterprise.middleware
>> >> >
>> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
>> >> > blog: http://lahiruwrites.blogspot.com/
>> >> > twitter: http://twitter.com/lahirus
>> >> > linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Check out my professional profile and connect with me on LinkedIn.
>> >> http://lnkd.in/cw5k69
>> >
>> >
>> >
>> >
>> > --
>> > --
>> > Lahiru Sandaruwan
>> > Committer and PMC member, Apache Stratos,
>> > Senior Software Engineer,
>> > WSO2 Inc., http://wso2.com
>> > lean.enterprise.middleware
>> >
>> > email: lahirus@wso2.com cell: (+94) 773 325 954
>> > blog: http://lahiruwrites.blogspot.com/
>> > twitter: http://twitter.com/lahirus
>> > linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>> >
>>
>>
>>
>> --
>> Check out my professional profile and connect with me on LinkedIn.
>> http://lnkd.in/cw5k69
>
>
>
>
> --
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
>
> Senior Software Engineer, WSO2 Inc.
>
> Blog: http://nirmalfdo.blogspot.com/
--
Check out my professional profile and connect with me on LinkedIn.
http://lnkd.in/cw5k69
Re: autoscaling is ignoring partiionMax
Posted by Nirmal Fernando <ni...@gmail.com>.
Hi Chris,
PartitionMax is considered in AutoscaleAlgorithm.getNextScaleUpPartition
method.
On Thu, May 29, 2014 at 3:09 AM, chris snow <ch...@gmail.com> wrote:
> I setup a breakpoint in eclipse at
> CloudControllerClient.spawnAnInstance() and dumped the output (below).
>
> I've scanned through the code in the stratos methods in the
> stacktrace, but couldn't see any code that checks the partitionMax
> value.
>
> Any ideas?
>
> Daemon Thread [Thread-56] (Suspended (breakpoint at line 137 in
> CloudControllerClient))
> owns: CloudControllerClient (id=2433)
> owns: DefaultAgenda (id=2434)
> CloudControllerClient.spawnAnInstance(Partition, String, String,
> String) line: 137
> RuleTasksDelegator.delegateSpawn(PartitionContext, String, String)
> line: 92
> NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line:
> not available [native method]
> NativeMethodAccessorImpl.invoke(Object, Object[]) line: 57
> DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
> Method.invoke(Object, Object...) line: 606
> ReflectiveAccessorOptimizer.getMethod(Object, String, Object[],
> Class[], ExecutableStatement[]) line: 1104
> ReflectiveAccessorOptimizer.getMethod(Object, String) line: 987
> ReflectiveAccessorOptimizer.compileGetChain() line: 377
> ReflectiveAccessorOptimizer.optimizeAccessor(ParserContext, char[],
> int, int, Object, Object, VariableResolverFactory, boolean, Class)
> line: 143
> ASTNode.optimize(Object, Object, VariableResolverFactory) line: 159
> ASTNode.getReducedValueAccelerated(Object, Object,
> VariableResolverFactory) line: 115
> MVELRuntime.execute(boolean, CompiledExpression, Object,
> VariableResolverFactory) line: 85
> CompiledExpression.getDirectValue(Object, VariableResolverFactory)
> line: 123
> CompiledExpression.getValue(Object, VariableResolverFactory) line: 119
> MVEL.executeExpression(Object, Object, VariableResolverFactory) line:
> 930
> MVELConsequence.evaluate(KnowledgeHelper, WorkingMemory) line: 104
> DefaultAgenda.fireActivation(Activation) line: 1287
> DefaultAgenda.fireNextItem(AgendaFilter) line: 1221
> DefaultAgenda.fireAllRules(AgendaFilter, int) line: 1456
> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules(AgendaFilter,
> int) line: 710
> ReteooStatefulSession(AbstractWorkingMemory).fireAllRules() line: 674
> StatefulKnowledgeSessionImpl.fireAllRules() line: 230
> AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
> FactHandle, Object) line: 86
> ClusterMonitor.monitor() line: 99
> ClusterMonitor.run() line: 72
> Thread.run() line: 744
>
> On Wed, May 28, 2014 at 3:37 AM, Lahiru Sandaruwan <la...@wso2.com>
> wrote:
> >
> >
> >
> > On Tue, May 27, 2014 at 11:54 PM, chris snow <ch...@gmail.com>
> wrote:
> >>
> >> Hi Lahiru,
> >>
> >> I'll have a poke around - it will give me a good opportunity to learn
> >> some more Stratos code :)
> >>
> >
> > Cool :)
> >
> > Debug logs should be able to show us where it went wrong.
> >
> > Thanks.
> >>
> >> Many thanks,
> >>
> >> Chris
> >>
> >> On Tue, May 27, 2014 at 7:45 AM, Lahiru Sandaruwan <la...@wso2.com>
> >> wrote:
> >> > Hi Chris,
> >> >
> >> > Can you enable debug logs and try out?
> >> > If so please send them over.
> >> >
> >> > Thanks.
> >> >
> >> >
> >> > On Tue, May 27, 2014 at 12:05 PM, chris snow <ch...@gmail.com>
> >> > wrote:
> >> >>
> >> >> Deployment Policy partitionMax seems to be ignored. My deployment
> >> >> policy looks like this:
> >> >>
> >> >> {
> >> >> "id": "economyDeployment",
> >> >> "partitionGroup": [
> >> >> {
> >> >> "id": "G1",
> >> >> "partitionAlgo": "one-after-another",
> >> >> "partition": [
> >> >> {
> >> >> "id": "P1",
> >> >> "partitionMax": "1",
> >> >> "partitionMin": "1"
> >> >> }
> >> >> ]
> >> >> }
> >> >> ]
> >> >> }
> >> >>
> >> >> Stratos initially starts two instances:
> >> >>
> >> >>
> >> >>
> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> | ID | Name | Status |
> Task
> >> >> State | Power State | Networks |
> >> >>
> >> >>
> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 | ACTIVE | -
> >> >> | Running | private=10.11.12.2 |
> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 | ACTIVE | -
> >> >> | Running | private=10.11.12.3 |
> >> >>
> >> >>
> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> Then attempts to start more:
> >> >>
> >> >>
> >> >>
> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> | ID | Name | Status |
> Task
> >> >> State | Power State | Networks |
> >> >>
> >> >>
> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >> | 91029130-37b5-4df6-ad95-1d9f609a22c3 | php111php-13e | ERROR | -
> >> >> | NOSTATE | |
> >> >> | 71c1e361-df55-4f61-84c5-79774fef2beb | php111php-3d7 | ERROR | -
> >> >> | NOSTATE | |
> >> >> | b0d72602-cdb8-4621-ad12-911ade4d89b1 | php111php-412 | ERROR | -
> >> >> | NOSTATE | |
> >> >> | d4eac55d-bb7c-48d6-83f2-cb9a11581a18 | php111php-6f5 | ERROR | -
> >> >> | NOSTATE | |
> >> >> | 738a9534-2b95-4984-a51b-748bc7169b9a | php111php-73e | ERROR | -
> >> >> | NOSTATE | |
> >> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 | ACTIVE | -
> >> >> | Running | private=10.11.12.2 |
> >> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 | ACTIVE | -
> >> >> | Running | private=10.11.12.3 |
> >> >> | f570e654-024b-4240-bad2-8a11f04f23f0 | php111php-a30 | ERROR | -
> >> >> | NOSTATE | |
> >> >> | 0ab36824-9bea-4a95-aa22-7f935406d83d | php111php-bca | ERROR | -
> >> >> | NOSTATE | |
> >> >> | 02ed97df-02bc-4bff-b42d-9f2bcbd4f569 | php111php-f6f | ERROR | -
> >> >> | NOSTATE | |
> >> >>
> >> >>
> >> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> >>
> >> >> I'm on branch: 4.0.0-incubating, using Openstack Havana + Docker.
> >> >>
> >> >> Any ideas?
> >> >>
> >> >> Many thanks,
> >> >>
> >> >> Chris
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > --
> >> > Lahiru Sandaruwan
> >> > Committer and PMC member, Apache Stratos,
> >> > Senior Software Engineer,
> >> > WSO2 Inc., http://wso2.com
> >> > lean.enterprise.middleware
> >> >
> >> > email: lahirus@wso2.com cell: (+94) 773 325 954
> >> > blog: http://lahiruwrites.blogspot.com/
> >> > twitter: http://twitter.com/lahirus
> >> > linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
> >> >
> >>
> >>
> >>
> >> --
> >> Check out my professional profile and connect with me on LinkedIn.
> >> http://lnkd.in/cw5k69
> >
> >
> >
> >
> > --
> > --
> > Lahiru Sandaruwan
> > Committer and PMC member, Apache Stratos,
> > Senior Software Engineer,
> > WSO2 Inc., http://wso2.com
> > lean.enterprise.middleware
> >
> > email: lahirus@wso2.com cell: (+94) 773 325 954
> > blog: http://lahiruwrites.blogspot.com/
> > twitter: http://twitter.com/lahirus
> > linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
> >
>
>
>
> --
> Check out my professional profile and connect with me on LinkedIn.
> http://lnkd.in/cw5k69
>
--
Best Regards,
Nirmal
Nirmal Fernando.
PPMC Member & Committer of Apache Stratos,
Senior Software Engineer, WSO2 Inc.
Blog: http://nirmalfdo.blogspot.com/
Re: autoscaling is ignoring partiionMax
Posted by chris snow <ch...@gmail.com>.
I setup a breakpoint in eclipse at
CloudControllerClient.spawnAnInstance() and dumped the output (below).
I've scanned through the code in the stratos methods in the
stacktrace, but couldn't see any code that checks the partitionMax
value.
Any ideas?
Daemon Thread [Thread-56] (Suspended (breakpoint at line 137 in
CloudControllerClient))
owns: CloudControllerClient (id=2433)
owns: DefaultAgenda (id=2434)
CloudControllerClient.spawnAnInstance(Partition, String, String,
String) line: 137
RuleTasksDelegator.delegateSpawn(PartitionContext, String, String) line: 92
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line:
not available [native method]
NativeMethodAccessorImpl.invoke(Object, Object[]) line: 57
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
Method.invoke(Object, Object...) line: 606
ReflectiveAccessorOptimizer.getMethod(Object, String, Object[],
Class[], ExecutableStatement[]) line: 1104
ReflectiveAccessorOptimizer.getMethod(Object, String) line: 987
ReflectiveAccessorOptimizer.compileGetChain() line: 377
ReflectiveAccessorOptimizer.optimizeAccessor(ParserContext, char[],
int, int, Object, Object, VariableResolverFactory, boolean, Class)
line: 143
ASTNode.optimize(Object, Object, VariableResolverFactory) line: 159
ASTNode.getReducedValueAccelerated(Object, Object,
VariableResolverFactory) line: 115
MVELRuntime.execute(boolean, CompiledExpression, Object,
VariableResolverFactory) line: 85
CompiledExpression.getDirectValue(Object, VariableResolverFactory) line: 123
CompiledExpression.getValue(Object, VariableResolverFactory) line: 119
MVEL.executeExpression(Object, Object, VariableResolverFactory) line: 930
MVELConsequence.evaluate(KnowledgeHelper, WorkingMemory) line: 104
DefaultAgenda.fireActivation(Activation) line: 1287
DefaultAgenda.fireNextItem(AgendaFilter) line: 1221
DefaultAgenda.fireAllRules(AgendaFilter, int) line: 1456
ReteooStatefulSession(AbstractWorkingMemory).fireAllRules(AgendaFilter,
int) line: 710
ReteooStatefulSession(AbstractWorkingMemory).fireAllRules() line: 674
StatefulKnowledgeSessionImpl.fireAllRules() line: 230
AutoscalerRuleEvaluator.evaluateMinCheck(StatefulKnowledgeSession,
FactHandle, Object) line: 86
ClusterMonitor.monitor() line: 99
ClusterMonitor.run() line: 72
Thread.run() line: 744
On Wed, May 28, 2014 at 3:37 AM, Lahiru Sandaruwan <la...@wso2.com> wrote:
>
>
>
> On Tue, May 27, 2014 at 11:54 PM, chris snow <ch...@gmail.com> wrote:
>>
>> Hi Lahiru,
>>
>> I'll have a poke around - it will give me a good opportunity to learn
>> some more Stratos code :)
>>
>
> Cool :)
>
> Debug logs should be able to show us where it went wrong.
>
> Thanks.
>>
>> Many thanks,
>>
>> Chris
>>
>> On Tue, May 27, 2014 at 7:45 AM, Lahiru Sandaruwan <la...@wso2.com>
>> wrote:
>> > Hi Chris,
>> >
>> > Can you enable debug logs and try out?
>> > If so please send them over.
>> >
>> > Thanks.
>> >
>> >
>> > On Tue, May 27, 2014 at 12:05 PM, chris snow <ch...@gmail.com>
>> > wrote:
>> >>
>> >> Deployment Policy partitionMax seems to be ignored. My deployment
>> >> policy looks like this:
>> >>
>> >> {
>> >> "id": "economyDeployment",
>> >> "partitionGroup": [
>> >> {
>> >> "id": "G1",
>> >> "partitionAlgo": "one-after-another",
>> >> "partition": [
>> >> {
>> >> "id": "P1",
>> >> "partitionMax": "1",
>> >> "partitionMin": "1"
>> >> }
>> >> ]
>> >> }
>> >> ]
>> >> }
>> >>
>> >> Stratos initially starts two instances:
>> >>
>> >>
>> >>
>> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> | ID | Name | Status | Task
>> >> State | Power State | Networks |
>> >>
>> >>
>> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 | ACTIVE | -
>> >> | Running | private=10.11.12.2 |
>> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 | ACTIVE | -
>> >> | Running | private=10.11.12.3 |
>> >>
>> >>
>> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> Then attempts to start more:
>> >>
>> >>
>> >>
>> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> | ID | Name | Status | Task
>> >> State | Power State | Networks |
>> >>
>> >>
>> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >> | 91029130-37b5-4df6-ad95-1d9f609a22c3 | php111php-13e | ERROR | -
>> >> | NOSTATE | |
>> >> | 71c1e361-df55-4f61-84c5-79774fef2beb | php111php-3d7 | ERROR | -
>> >> | NOSTATE | |
>> >> | b0d72602-cdb8-4621-ad12-911ade4d89b1 | php111php-412 | ERROR | -
>> >> | NOSTATE | |
>> >> | d4eac55d-bb7c-48d6-83f2-cb9a11581a18 | php111php-6f5 | ERROR | -
>> >> | NOSTATE | |
>> >> | 738a9534-2b95-4984-a51b-748bc7169b9a | php111php-73e | ERROR | -
>> >> | NOSTATE | |
>> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 | ACTIVE | -
>> >> | Running | private=10.11.12.2 |
>> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 | ACTIVE | -
>> >> | Running | private=10.11.12.3 |
>> >> | f570e654-024b-4240-bad2-8a11f04f23f0 | php111php-a30 | ERROR | -
>> >> | NOSTATE | |
>> >> | 0ab36824-9bea-4a95-aa22-7f935406d83d | php111php-bca | ERROR | -
>> >> | NOSTATE | |
>> >> | 02ed97df-02bc-4bff-b42d-9f2bcbd4f569 | php111php-f6f | ERROR | -
>> >> | NOSTATE | |
>> >>
>> >>
>> >> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> >>
>> >> I'm on branch: 4.0.0-incubating, using Openstack Havana + Docker.
>> >>
>> >> Any ideas?
>> >>
>> >> Many thanks,
>> >>
>> >> Chris
>> >
>> >
>> >
>> >
>> > --
>> > --
>> > Lahiru Sandaruwan
>> > Committer and PMC member, Apache Stratos,
>> > Senior Software Engineer,
>> > WSO2 Inc., http://wso2.com
>> > lean.enterprise.middleware
>> >
>> > email: lahirus@wso2.com cell: (+94) 773 325 954
>> > blog: http://lahiruwrites.blogspot.com/
>> > twitter: http://twitter.com/lahirus
>> > linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>> >
>>
>>
>>
>> --
>> Check out my professional profile and connect with me on LinkedIn.
>> http://lnkd.in/cw5k69
>
>
>
>
> --
> --
> Lahiru Sandaruwan
> Committer and PMC member, Apache Stratos,
> Senior Software Engineer,
> WSO2 Inc., http://wso2.com
> lean.enterprise.middleware
>
> email: lahirus@wso2.com cell: (+94) 773 325 954
> blog: http://lahiruwrites.blogspot.com/
> twitter: http://twitter.com/lahirus
> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>
--
Check out my professional profile and connect with me on LinkedIn.
http://lnkd.in/cw5k69
Re: autoscaling is ignoring partiionMax
Posted by Lahiru Sandaruwan <la...@wso2.com>.
On Tue, May 27, 2014 at 11:54 PM, chris snow <ch...@gmail.com> wrote:
> Hi Lahiru,
>
> I'll have a poke around - it will give me a good opportunity to learn
> some more Stratos code :)
>
>
Cool :)
Debug logs should be able to show us where it went wrong.
Thanks.
> Many thanks,
>
> Chris
>
> On Tue, May 27, 2014 at 7:45 AM, Lahiru Sandaruwan <la...@wso2.com>
> wrote:
> > Hi Chris,
> >
> > Can you enable debug logs and try out?
> > If so please send them over.
> >
> > Thanks.
> >
> >
> > On Tue, May 27, 2014 at 12:05 PM, chris snow <ch...@gmail.com>
> wrote:
> >>
> >> Deployment Policy partitionMax seems to be ignored. My deployment
> >> policy looks like this:
> >>
> >> {
> >> "id": "economyDeployment",
> >> "partitionGroup": [
> >> {
> >> "id": "G1",
> >> "partitionAlgo": "one-after-another",
> >> "partition": [
> >> {
> >> "id": "P1",
> >> "partitionMax": "1",
> >> "partitionMin": "1"
> >> }
> >> ]
> >> }
> >> ]
> >> }
> >>
> >> Stratos initially starts two instances:
> >>
> >>
> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> | ID | Name | Status | Task
> >> State | Power State | Networks |
> >>
> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 | ACTIVE | -
> >> | Running | private=10.11.12.2 |
> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 | ACTIVE | -
> >> | Running | private=10.11.12.3 |
> >>
> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> Then attempts to start more:
> >>
> >>
> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> | ID | Name | Status | Task
> >> State | Power State | Networks |
> >>
> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >> | 91029130-37b5-4df6-ad95-1d9f609a22c3 | php111php-13e | ERROR | -
> >> | NOSTATE | |
> >> | 71c1e361-df55-4f61-84c5-79774fef2beb | php111php-3d7 | ERROR | -
> >> | NOSTATE | |
> >> | b0d72602-cdb8-4621-ad12-911ade4d89b1 | php111php-412 | ERROR | -
> >> | NOSTATE | |
> >> | d4eac55d-bb7c-48d6-83f2-cb9a11581a18 | php111php-6f5 | ERROR | -
> >> | NOSTATE | |
> >> | 738a9534-2b95-4984-a51b-748bc7169b9a | php111php-73e | ERROR | -
> >> | NOSTATE | |
> >> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 | ACTIVE | -
> >> | Running | private=10.11.12.2 |
> >> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 | ACTIVE | -
> >> | Running | private=10.11.12.3 |
> >> | f570e654-024b-4240-bad2-8a11f04f23f0 | php111php-a30 | ERROR | -
> >> | NOSTATE | |
> >> | 0ab36824-9bea-4a95-aa22-7f935406d83d | php111php-bca | ERROR | -
> >> | NOSTATE | |
> >> | 02ed97df-02bc-4bff-b42d-9f2bcbd4f569 | php111php-f6f | ERROR | -
> >> | NOSTATE | |
> >>
> >>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> >>
> >> I'm on branch: 4.0.0-incubating, using Openstack Havana + Docker.
> >>
> >> Any ideas?
> >>
> >> Many thanks,
> >>
> >> Chris
> >
> >
> >
> >
> > --
> > --
> > Lahiru Sandaruwan
> > Committer and PMC member, Apache Stratos,
> > Senior Software Engineer,
> > WSO2 Inc., http://wso2.com
> > lean.enterprise.middleware
> >
> > email: lahirus@wso2.com cell: (+94) 773 325 954
> > blog: http://lahiruwrites.blogspot.com/
> > twitter: http://twitter.com/lahirus
> > linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
> >
>
>
>
> --
> Check out my professional profile and connect with me on LinkedIn.
> http://lnkd.in/cw5k69
>
--
--
Lahiru Sandaruwan
Committer and PMC member, Apache Stratos,
Senior Software Engineer,
WSO2 Inc., http://wso2.com
lean.enterprise.middleware
email: lahirus@wso2.com cell: (+94) 773 325 954
blog: http://lahiruwrites.blogspot.com/
twitter: http://twitter.com/lahirus
linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
Re: autoscaling is ignoring partiionMax
Posted by chris snow <ch...@gmail.com>.
Hi Lahiru,
I'll have a poke around - it will give me a good opportunity to learn
some more Stratos code :)
Many thanks,
Chris
On Tue, May 27, 2014 at 7:45 AM, Lahiru Sandaruwan <la...@wso2.com> wrote:
> Hi Chris,
>
> Can you enable debug logs and try out?
> If so please send them over.
>
> Thanks.
>
>
> On Tue, May 27, 2014 at 12:05 PM, chris snow <ch...@gmail.com> wrote:
>>
>> Deployment Policy partitionMax seems to be ignored. My deployment
>> policy looks like this:
>>
>> {
>> "id": "economyDeployment",
>> "partitionGroup": [
>> {
>> "id": "G1",
>> "partitionAlgo": "one-after-another",
>> "partition": [
>> {
>> "id": "P1",
>> "partitionMax": "1",
>> "partitionMin": "1"
>> }
>> ]
>> }
>> ]
>> }
>>
>> Stratos initially starts two instances:
>>
>>
>> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> | ID | Name | Status | Task
>> State | Power State | Networks |
>>
>> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 | ACTIVE | -
>> | Running | private=10.11.12.2 |
>> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 | ACTIVE | -
>> | Running | private=10.11.12.3 |
>>
>> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> Then attempts to start more:
>>
>>
>> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> | ID | Name | Status | Task
>> State | Power State | Networks |
>>
>> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>> | 91029130-37b5-4df6-ad95-1d9f609a22c3 | php111php-13e | ERROR | -
>> | NOSTATE | |
>> | 71c1e361-df55-4f61-84c5-79774fef2beb | php111php-3d7 | ERROR | -
>> | NOSTATE | |
>> | b0d72602-cdb8-4621-ad12-911ade4d89b1 | php111php-412 | ERROR | -
>> | NOSTATE | |
>> | d4eac55d-bb7c-48d6-83f2-cb9a11581a18 | php111php-6f5 | ERROR | -
>> | NOSTATE | |
>> | 738a9534-2b95-4984-a51b-748bc7169b9a | php111php-73e | ERROR | -
>> | NOSTATE | |
>> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 | ACTIVE | -
>> | Running | private=10.11.12.2 |
>> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 | ACTIVE | -
>> | Running | private=10.11.12.3 |
>> | f570e654-024b-4240-bad2-8a11f04f23f0 | php111php-a30 | ERROR | -
>> | NOSTATE | |
>> | 0ab36824-9bea-4a95-aa22-7f935406d83d | php111php-bca | ERROR | -
>> | NOSTATE | |
>> | 02ed97df-02bc-4bff-b42d-9f2bcbd4f569 | php111php-f6f | ERROR | -
>> | NOSTATE | |
>>
>> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>>
>> I'm on branch: 4.0.0-incubating, using Openstack Havana + Docker.
>>
>> Any ideas?
>>
>> Many thanks,
>>
>> Chris
>
>
>
>
> --
> --
> Lahiru Sandaruwan
> Committer and PMC member, Apache Stratos,
> Senior Software Engineer,
> WSO2 Inc., http://wso2.com
> lean.enterprise.middleware
>
> email: lahirus@wso2.com cell: (+94) 773 325 954
> blog: http://lahiruwrites.blogspot.com/
> twitter: http://twitter.com/lahirus
> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>
--
Check out my professional profile and connect with me on LinkedIn.
http://lnkd.in/cw5k69
Re: autoscaling is ignoring partiionMax
Posted by Lahiru Sandaruwan <la...@wso2.com>.
Hi Chris,
Can you enable debug logs and try out?
If so please send them over.
Thanks.
On Tue, May 27, 2014 at 12:05 PM, chris snow <ch...@gmail.com> wrote:
> Deployment Policy partitionMax seems to be ignored. My deployment
> policy looks like this:
>
> {
> "id": "economyDeployment",
> "partitionGroup": [
> {
> "id": "G1",
> "partitionAlgo": "one-after-another",
> "partition": [
> {
> "id": "P1",
> "partitionMax": "1",
> "partitionMin": "1"
> }
> ]
> }
> ]
> }
>
> Stratos initially starts two instances:
>
>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> | ID | Name | Status | Task
> State | Power State | Networks |
>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 | ACTIVE | -
> | Running | private=10.11.12.2 |
> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 | ACTIVE | -
> | Running | private=10.11.12.3 |
>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> Then attempts to start more:
>
>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> | ID | Name | Status | Task
> State | Power State | Networks |
>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
> | 91029130-37b5-4df6-ad95-1d9f609a22c3 | php111php-13e | ERROR | -
> | NOSTATE | |
> | 71c1e361-df55-4f61-84c5-79774fef2beb | php111php-3d7 | ERROR | -
> | NOSTATE | |
> | b0d72602-cdb8-4621-ad12-911ade4d89b1 | php111php-412 | ERROR | -
> | NOSTATE | |
> | d4eac55d-bb7c-48d6-83f2-cb9a11581a18 | php111php-6f5 | ERROR | -
> | NOSTATE | |
> | 738a9534-2b95-4984-a51b-748bc7169b9a | php111php-73e | ERROR | -
> | NOSTATE | |
> | c91eddd7-f2d5-49e5-b602-fde5df38e16d | php111php-9f6 | ACTIVE | -
> | Running | private=10.11.12.2 |
> | b38536d2-bc9e-4ab2-984c-bc3f2091947f | php111php-a05 | ACTIVE | -
> | Running | private=10.11.12.3 |
> | f570e654-024b-4240-bad2-8a11f04f23f0 | php111php-a30 | ERROR | -
> | NOSTATE | |
> | 0ab36824-9bea-4a95-aa22-7f935406d83d | php111php-bca | ERROR | -
> | NOSTATE | |
> | 02ed97df-02bc-4bff-b42d-9f2bcbd4f569 | php111php-f6f | ERROR | -
> | NOSTATE | |
>
> +--------------------------------------+---------------+--------+------------+-------------+--------------------+
>
> I'm on branch: 4.0.0-incubating, using Openstack Havana + Docker.
>
> Any ideas?
>
> Many thanks,
>
> Chris
>
--
--
Lahiru Sandaruwan
Committer and PMC member, Apache Stratos,
Senior Software Engineer,
WSO2 Inc., http://wso2.com
lean.enterprise.middleware
email: lahirus@wso2.com cell: (+94) 773 325 954
blog: http://lahiruwrites.blogspot.com/
twitter: http://twitter.com/lahirus
linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146