You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stratos.apache.org by Udara Liyanage <ud...@wso2.com> on 2014/12/22 07:42:33 UTC
Why agent wait to be intialized even after complete topology is received
Hi Chamila,
I get the ":Waiting for Cartridge Agent to be initialized..." logs
continuously even though complete topology and complete tenant event is
already received.
2014-12-22 12:04:47,595:DEBUG:Waiting for Cartridge Agent to be
initialized...
2014-12-22 12:04:48,596:DEBUG:Waiting for Cartridge Agent to be
initialized...
2014-12-22 12:04:49,597:DEBUG:Waiting for Cartridge Agent to be
initialized...
2014-12-22 12:04:49,781:DEBUG:Message received:
'topology/CompleteTopologyEvent':
'{"topology":{"serviceMap":{"tomcat":{"serviceName":"tomcat","serviceType":"SingleTenant","clusterIdClusterMap":{"mytomcat.tomcat.domain":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","autoscalePolicyName":"autoscale_policy_1","hostNames":["
mytomcat.isuruh.lk
"],"tenantRange":"*","isLbCluster":false,"isKubernetesCluster":false,"memberMap":{"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","memberId":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101","instanceId":"ap-southeast-1/i-df7dde12","clusterInstanceId":"app_cartridge_v1-1","networkPartitionId":"p1","partitionId":"P1","initTime":1419229605140,"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"memberPublicIPs":["54.151.190.32"],"defaultPublicIP":"54.151.190.32","memberPrivateIPs":["10.144.68.9"],"defaultPrivateIP":"10.144.68.9","properties":{"PRIMARY":"false","MIN_COUNT":"1"},"memberStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101"}}},"appId":"app_cartridge_v1","properties":{},"instanceIdToInstanceContextMap":{"app_cartridge_v1-1":{"instanceId":"app_cartridge_v1-1","instanceProperties":{},"lifeCycleStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domain_app_cartridge_v1-1"},"networkPartitionId":"p1"}}}},"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"properties":{}}},"initialized":false}}'
2014-12-22 12:04:49,781:DEBUG:Executing handler for event
'CompleteTopologyEvent'
2014-12-22 12:04:49,781:DEBUG:Complete topology event received
2014-12-22 12:04:49,782:ERROR:Error processing 'CompleteTopologyEvent' event
Traceback (most recent call last):
File
"/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
line 81, in on_message
handler(msg)
File "agent.py", line 255, in on_complete_topology
event_obj = CompleteTopologyEvent.create_from_json(msg.payload)
File
"/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/event/topology/events.py",
line 190, in create_from_json
member_obj.member_ip = member_str["memberIp"]
KeyError: 'memberIp'
2014-12-22 12:04:49,820:DEBUG:Message received:
'tenant/CompleteTenantEvent':
'{"tenants":[]}'
2014-12-22 12:04:49,820:DEBUG:Executing handler for event
'CompleteTenantEvent'
2014-12-22 12:04:49,820:INFO:Complete tenant event updating task disabled
*Carbon logs:*
TID: [0] [STRATOS] [2014-12-22 06:27:21,240] INFO
{org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
- Publishing instance spawned event: [service-name] tomcat [cluster-id]
mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 [instance-id]
ap-southeast-1/i-df7dde12 [network-partition-id] p1 [partition-id] P1
[lb-cluster-id] null
TID: [0] [STRATOS] [2014-12-22 06:27:21,261] INFO
{org.apache.stratos.messaging.domain.topology.lifecycle.LifeCycleStateManager}
- Life Cycle State Manager started for Element [
mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 ], initial
state: Created
TID: [0] [STRATOS] [2014-12-22 06:27:21,264] INFO
{org.apache.stratos.messaging.message.processor.topology.InstanceSpawnedMessageProcessor}
- Member created: [service] tomcat [cluster] mytomcat.tomcat.domain
[member] mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101
[instance] ap-southeast-1/i-df7dde12 [hierarchy-instance] app_cartridge_v1-1
TID: [0] [STRATOS] [2014-12-22 06:27:21,268] INFO
{org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
- [InstanceSpawnedEventListener] Received: class
org.apache.stratos.messaging.event.topology.InstanceSpawnedEvent
--
Udara Liyanage
Software Engineer
WSO2, Inc.: http://wso2.com
lean. enterprise. middleware
web: http://udaraliyanage.wordpress.com
phone: +94 71 443 6897
Re: Why agent wait to be intialized even after complete topology is received
Posted by Udara Liyanage <ud...@wso2.com>.
Hi Isuru, Raj,
Current codebase have below line on 190th line of events.py. May be agent
in my puppet master is outdated. I will update it and try again.
cluster_obj.hostnames = cluster_str["hostNames"]
On Mon, Dec 22, 2014 at 12:22 PM, Rajkumar Rajaratnam <ra...@wso2.com>
wrote:
>
> Just noticed that there is an error in parsing complete topology event. So
> initialization didn't happen as Isuru mentioned.
>
> Thanks.
>
> On Mon, Dec 22, 2014 at 12:19 PM, Rajkumar Rajaratnam <ra...@wso2.com>
> wrote:
>
>> Hi,
>>
>> AFAIR, agent will wait for two conditions to happen. It will proceed if
>> either of two conditions becomes true.
>>
>> 1. Member should be in the topology
>> 2. Agent should get instance spawned event from CC
>>
>> So either of them didn't happen for you.
>>
>> Thanks.
>>
>> On Mon, Dec 22, 2014 at 12:17 PM, Isuru Haththotuwa <is...@apache.org>
>> wrote:
>>
>>> HI Udara,
>>>
>>> There is an error log about parsing the complete Topology event. Maybe
>>> the event did not get parsed correctly, and the required initialization did
>>> not happen.
>>>
>>> On Mon, Dec 22, 2014 at 12:12 PM, Udara Liyanage <ud...@wso2.com> wrote:
>>>
>>>> Hi Chamila,
>>>>
>>>> I get the ":Waiting for Cartridge Agent to be initialized..." logs
>>>> continuously even though complete topology and complete tenant event is
>>>> already received.
>>>>
>>>>
>>>> 2014-12-22 12:04:47,595:DEBUG:Waiting for Cartridge Agent to be
>>>> initialized...
>>>> 2014-12-22 12:04:48,596:DEBUG:Waiting for Cartridge Agent to be
>>>> initialized...
>>>> 2014-12-22 12:04:49,597:DEBUG:Waiting for Cartridge Agent to be
>>>> initialized...
>>>> 2014-12-22 12:04:49,781:DEBUG:Message received:
>>>> 'topology/CompleteTopologyEvent':
>>>>
>>>> '{"topology":{"serviceMap":{"tomcat":{"serviceName":"tomcat","serviceType":"SingleTenant","clusterIdClusterMap":{"mytomcat.tomcat.domain":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","autoscalePolicyName":"autoscale_policy_1","hostNames":["
>>>> mytomcat.isuruh.lk
>>>> "],"tenantRange":"*","isLbCluster":false,"isKubernetesCluster":false,"memberMap":{"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","memberId":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101","instanceId":"ap-southeast-1/i-df7dde12","clusterInstanceId":"app_cartridge_v1-1","networkPartitionId":"p1","partitionId":"P1","initTime":1419229605140,"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"memberPublicIPs":["54.151.190.32"],"defaultPublicIP":"54.151.190.32","memberPrivateIPs":["10.144.68.9"],"defaultPrivateIP":"10.144.68.9","properties":{"PRIMARY":"false","MIN_COUNT":"1"},"memberStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101"}}},"appId":"app_cartridge_v1","properties":{},"instanceIdToInstanceContextMap":{"app_cartridge_v1-1":{"instanceId":"app_cartridge_v1-1","instanceProperties":{},"lifeCycleStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domain_app_cartridge_v1-1"},"networkPartitionId":"p1"}}}},"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"properties":{}}},"initialized":false}}'
>>>> 2014-12-22 12:04:49,781:DEBUG:Executing handler for event
>>>> 'CompleteTopologyEvent'
>>>> 2014-12-22 12:04:49,781:DEBUG:Complete topology event received
>>>> 2014-12-22 12:04:49,782:ERROR:Error processing 'CompleteTopologyEvent'
>>>> event
>>>> Traceback (most recent call last):
>>>> File
>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>>> line 81, in on_message
>>>> handler(msg)
>>>> File "agent.py", line 255, in on_complete_topology
>>>> event_obj = CompleteTopologyEvent.create_from_json(msg.payload)
>>>> File
>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/event/topology/events.py",
>>>> line 190, in create_from_json
>>>> member_obj.member_ip = member_str["memberIp"]
>>>> KeyError: 'memberIp'
>>>> 2014-12-22 12:04:49,820:DEBUG:Message received:
>>>> 'tenant/CompleteTenantEvent':
>>>> '{"tenants":[]}'
>>>> 2014-12-22 12:04:49,820:DEBUG:Executing handler for event
>>>> 'CompleteTenantEvent'
>>>> 2014-12-22 12:04:49,820:INFO:Complete tenant event updating task
>>>> disabled
>>>>
>>>>
>>>> *Carbon logs:*
>>>>
>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,240] INFO
>>>> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
>>>> - Publishing instance spawned event: [service-name] tomcat [cluster-id]
>>>> mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
>>>> mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 [instance-id]
>>>> ap-southeast-1/i-df7dde12 [network-partition-id] p1 [partition-id] P1
>>>> [lb-cluster-id] null
>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,261] INFO
>>>> {org.apache.stratos.messaging.domain.topology.lifecycle.LifeCycleStateManager}
>>>> - Life Cycle State Manager started for Element [
>>>> mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 ], initial
>>>> state: Created
>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,264] INFO
>>>> {org.apache.stratos.messaging.message.processor.topology.InstanceSpawnedMessageProcessor}
>>>> - Member created: [service] tomcat [cluster] mytomcat.tomcat.domain
>>>> [member] mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101
>>>> [instance] ap-southeast-1/i-df7dde12 [hierarchy-instance] app_cartridge_v1-1
>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,268] INFO
>>>> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
>>>> - [InstanceSpawnedEventListener] Received: class
>>>> org.apache.stratos.messaging.event.topology.InstanceSpawnedEvent
>>>>
>>>> --
>>>>
>>>> Udara Liyanage
>>>> Software Engineer
>>>> WSO2, Inc.: http://wso2.com
>>>> lean. enterprise. middleware
>>>>
>>>> web: http://udaraliyanage.wordpress.com
>>>> phone:
>>>> +94 71 443 6897
>>>>
>>>> --
>>>> Thanks and Regards,
>>>>
>>>> Isuru H.
>>>> +94 716 358 048* <http://wso2.com/>*
>>>>
>>>>
>>>> * <http://wso2.com/>*
>>>>
>>>>
>>>>
>>
>>
>> --
>> Rajkumar Rajaratnam
>> Committer & PMC Member, Apache Stratos
>> Software Engineer, WSO2
>>
>> Mobile : +94777568639
>> Blog : rajkumarr.com
>>
>
>
>
> --
> Rajkumar Rajaratnam
> Committer & PMC Member, Apache Stratos
> Software Engineer, WSO2
>
> Mobile : +94777568639
> Blog : rajkumarr.com
>
--
Udara Liyanage
Software Engineer
WSO2, Inc.: http://wso2.com
lean. enterprise. middleware
web: http://udaraliyanage.wordpress.com
phone: +94 71 443 6897
Re: Why agent wait to be intialized even after complete topology is received
Posted by Rajkumar Rajaratnam <ra...@wso2.com>.
And make sure that the ports are open. Instance activated event will be
sent by the agent only if ports are open.
Thanks.
On Mon, Dec 22, 2014 at 1:15 PM, Udara Liyanage <ud...@wso2.com> wrote:
> Hi Raj,
>
> Logs are missed since instance is killed as an obsolete member. I will
> provide the logs next time.
>
> On Mon, Dec 22, 2014 at 1:13 PM, Rajkumar Rajaratnam <ra...@wso2.com>
> wrote:
>>
>> Hi Udara,
>>
>> Can you attach the agent log?
>>
>> Thanks.
>>
>> On Mon, Dec 22, 2014 at 1:04 PM, Udara Liyanage <ud...@wso2.com> wrote:
>>
>>> Hi raj,
>>>
>>> I tested with lates python code base. This time member is in topology
>>> and instance spawned event is also received. But yet instance activated is
>>> not sent.
>>>
>>> *1) Instance started event received*
>>>
>>> 2014-12-22 12:53:49,904:DEBUG:Property added: [key] STRATOS_
>>> MEMBER_PROPERTY_PRIMARY[value] false
>>> 2014-12-22 12:53:49,911:DEBUG:Message received:
>>> 'instance/status/InstanceStartedEvent':
>>> '{\n "clusterId": "mytomcat.tomcat.domain", \n
>>> "clusterInstanceId": "app_cartridge_v1-1", \n "instanceId": "", \n
>>> "memberId": "mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130",
>>> \n "networkPartitionId": "p1", \n "partitionId": "P1", \n
>>> "serviceName": "tomcat"\n}'
>>>
>>> 2) Member in topology
>>>
>>> *Complete Topology:*
>>>
>>>
>>> '{"topology":{"serviceMap":{"tomcat":{"serviceName":"tomcat","serviceType":"SingleTenant","clusterIdClusterMap":{"mytomcat.tomcat.domain":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","autoscalePolicyName":"autoscale_policy_1","hostNames":["
>>> mytomcat.isuruh.lk
>>> "],"tenantRange":"*","isLbCluster":false,"isKubernetesCluster":false,"memberMap":{"mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","memberId":"
>>> *mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130*
>>> ","instanceId":"ap-southeast-1/i-0147e4cc","clusterInstanceId":"app_cartridge_v1-1","networkPartitionId":"p1","partitionId":"P1","initTime":1419232682397,"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"memberPublicIPs":["54.254.118.165"],"defaultPublicIP":"54.254.118.165","memberPrivateIPs":["10.144.150.199"],"defaultPrivateIP":"10.144.150.199","properties":{"PRIMARY":"false","MIN_COUNT":"1"},"memberStateManager":{"stateStack":["Created","Starting"],"identifier":"mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130"}}},"appId":"app_cartridge_v1","properties":{},"instanceIdToInstanceContextMap":{"app_cartridge_v1-1":{"instanceId":"app_cartridge_v1-1","instanceProperties":{},"lifeCycleStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domain_app_cartridge_v1-1"},"networkPartitionId":"p1"}}}},"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"properties":{}}},"initialized":false}}'
>>>
>>> *Payload:*
>>> APP_ID=app_cartridge_v1,GROUP_NAME=null,SERVICE_NAME=tomcat,HOST_NAME=
>>> mytomcat.isuruh.lk
>>> ,MULTITENANT=false,TENANT_ID=-1234,TENANT_RANGE=*,CARTRIDGE_ALIAS=mytomcat,CLUSTER_ID=mytomcat.tomcat.domain,CARTRIDGE_KEY=kYcZ4B4DPAeA1RGz,REPO_URL=
>>> https://github.com/udaraliyanage/php,PORTS=80|443,PROVIDER=apache,PUPPET_IP=10.148.159.109,PUPPET_HOSTNAME=ip-10-148-159-109.ap-southeast-1.compute.internal,PUPPET_DNS_AVAILABLE=null,PUPPET_ENV=null,,,DEPLOYMENT=default,TOKEN=eyJhbGciOiJSUzI1NiJ9.eyJleHAiOjE4OTY4OTAwNDgsInN1YiI6ImFkbWluIiwiYXpwIjoiMjU1UUZET1UwbFlONmY5NE44aUVhQm5LdlBrYSIsImFwcElkIjoiYXBwX2NhcnRyaWRnZV92MSIsImF1ZCI6WyIyNTVRRkRPVTBsWU42Zjk0TjhpRWFCbkt2UGthIl0sImlzcyI6Imh0dHBzOlwvXC9sb2NhbGhvc3Q6OTQ0M1wvb2F1dGgyZW5kcG9pbnRzXC90b2tlbiIsImlhdCI6MTg5MzI5MDA0OH0.XB4wg8gQxzyH0jv5EIiWFjOG0VDZuFBswYG0G6zX3sjOEgXauywoga4pHmb9DWkKZU9nD5qRFLRyyiHi2onWr-TGtLNBn4qgRq94VDwjVgx5in4ND3f7q-xNSpLipMr_g_gtkbp9st-gZVeuB8y3u7jnjE8fuDEs63pQ75Drrhs,MEMBER_ID=
>>> *mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130*
>>> ,INSTANCE_ID=null,CLUSTER_INSTANCE_ID=app_cartridge_v1-1,LB_CLUSTER_ID=null,NETWORK_PARTITION_ID=p1,PARTITION_ID=P1,PRIMARY=false,MIN_COUNT=1
>>>
>>> Carbon logs:
>>>
>>> [2014-12-22 07:18:41,444] INFO
>>> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
>>> - Publishing instance spawned event: [service-name] tomcat [cluster-id]
>>> mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
>>> mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130 [instance-id]
>>> ap-southeast-1/i-0147e4cc [network-partition-id] p1 [partition-id] P1
>>> [lb-cluster-id] null
>>> [2014-12-22 07:18:41,464] INFO
>>> {org.apache.stratos.messaging.domain.topology.lifecycle.LifeCycleStateManager}
>>> - Life Cycle State Manager started for Element [
>>> mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130 ], initial
>>> state: Created
>>> [2014-12-22 07:18:41,469] INFO
>>> {org.apache.stratos.messaging.message.processor.topology.InstanceSpawnedMessageProcessor}
>>> - Member created: [service] tomcat [cluster] mytomcat.tomcat.domain
>>> [member] mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130
>>> [instance] ap-southeast-1/i-0147e4cc [hierarchy-instance] app_cartridge_v1-1
>>> [2014-12-22 07:18:41,480] INFO
>>> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
>>> - [InstanceSpawnedEventListener] Received: class
>>> org.apache.stratos.messaging.event.topology.InstanceSpawnedEvent
>>> [2014-12-22 07:24:01,218] INFO
>>> {org.apache.stratos.manager.listener.InstanceStatusListener} - Instance
>>> status message received
>>> [2014-12-22 07:24:01,218] INFO
>>> {org.apache.stratos.manager.listener.InstanceStatusListener} - Event class
>>> name:
>>> org.apache.stratos.messaging.event.instance.status.InstanceStartedEvent
>>> [2014-12-22 07:24:01,219] INFO
>>> {org.apache.stratos.manager.listener.InstanceStatusListener} - Cluster id:
>>> mytomcat.tomcat.domain
>>> [2014-12-22 07:24:01,220] INFO
>>> {org.apache.stratos.cloud.controller.messaging.topology.TopologyBuilder} -
>>> member started event adding status started
>>> [2014-12-22 07:24:01,235] INFO
>>> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
>>> - Publishing member started event: [service-name] tomcat [cluster-id]
>>> mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
>>> mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130 [instance-id]
>>> [network-partition-id] p1 [partition-id] P1
>>> [2014-12-22 07:24:01,250] INFO
>>> {org.apache.stratos.messaging.message.processor.topology.MemberStartedMessageProcessor}
>>> - Member started: [service] tomcat [cluster] mytomcat.tomcat.domain
>>> [member] mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130
>>> [2014-12-22 07:24:01,250] INFO
>>> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
>>> - [MemberStartedEventListener] Received: class
>>> org.apache.stratos.messaging.event.topology.MemberStartedEvent
>>>
>>>
>>>
>>> On Mon, Dec 22, 2014 at 12:26 PM, Udara Liyanage <ud...@wso2.com> wrote:
>>>>
>>>> Hi Raj,
>>>>
>>>> Thanks for the info. I will check with updated agent.
>>>>
>>>> On Mon, Dec 22, 2014 at 12:24 PM, Rajkumar Rajaratnam <
>>>> rajkumarr@wso2.com> wrote:
>>>>>
>>>>> Hi Udara,
>>>>>
>>>>> Are you using the latest python agent?
>>>>>
>>>>> Chamila changed the agent yesterday. Member object doesn't have an
>>>>> attribute called "memberIp anymore. It has been changed to defaultPrivateIP.
>>>>>
>>>>> Latest python agent has this changes.
>>>>>
>>>>> Thanks.
>>>>>
>>>>> On Mon, Dec 22, 2014 at 12:22 PM, Rajkumar Rajaratnam <
>>>>> rajkumarr@wso2.com> wrote:
>>>>>
>>>>>> Just noticed that there is an error in parsing complete topology
>>>>>> event. So initialization didn't happen as Isuru mentioned.
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> On Mon, Dec 22, 2014 at 12:19 PM, Rajkumar Rajaratnam <
>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> AFAIR, agent will wait for two conditions to happen. It will proceed
>>>>>>> if either of two conditions becomes true.
>>>>>>>
>>>>>>> 1. Member should be in the topology
>>>>>>> 2. Agent should get instance spawned event from CC
>>>>>>>
>>>>>>> So either of them didn't happen for you.
>>>>>>>
>>>>>>> Thanks.
>>>>>>>
>>>>>>> On Mon, Dec 22, 2014 at 12:17 PM, Isuru Haththotuwa <
>>>>>>> isuruh@apache.org> wrote:
>>>>>>>
>>>>>>>> HI Udara,
>>>>>>>>
>>>>>>>> There is an error log about parsing the complete Topology event.
>>>>>>>> Maybe the event did not get parsed correctly, and the required
>>>>>>>> initialization did not happen.
>>>>>>>>
>>>>>>>> On Mon, Dec 22, 2014 at 12:12 PM, Udara Liyanage <ud...@wso2.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Chamila,
>>>>>>>>>
>>>>>>>>> I get the ":Waiting for Cartridge Agent to be initialized..." logs
>>>>>>>>> continuously even though complete topology and complete tenant event is
>>>>>>>>> already received.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2014-12-22 12:04:47,595:DEBUG:Waiting for Cartridge Agent to be
>>>>>>>>> initialized...
>>>>>>>>> 2014-12-22 12:04:48,596:DEBUG:Waiting for Cartridge Agent to be
>>>>>>>>> initialized...
>>>>>>>>> 2014-12-22 12:04:49,597:DEBUG:Waiting for Cartridge Agent to be
>>>>>>>>> initialized...
>>>>>>>>> 2014-12-22 12:04:49,781:DEBUG:Message received:
>>>>>>>>> 'topology/CompleteTopologyEvent':
>>>>>>>>>
>>>>>>>>> '{"topology":{"serviceMap":{"tomcat":{"serviceName":"tomcat","serviceType":"SingleTenant","clusterIdClusterMap":{"mytomcat.tomcat.domain":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","autoscalePolicyName":"autoscale_policy_1","hostNames":["
>>>>>>>>> mytomcat.isuruh.lk
>>>>>>>>> "],"tenantRange":"*","isLbCluster":false,"isKubernetesCluster":false,"memberMap":{"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","memberId":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101","instanceId":"ap-southeast-1/i-df7dde12","clusterInstanceId":"app_cartridge_v1-1","networkPartitionId":"p1","partitionId":"P1","initTime":1419229605140,"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"memberPublicIPs":["54.151.190.32"],"defaultPublicIP":"54.151.190.32","memberPrivateIPs":["10.144.68.9"],"defaultPrivateIP":"10.144.68.9","properties":{"PRIMARY":"false","MIN_COUNT":"1"},"memberStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101"}}},"appId":"app_cartridge_v1","properties":{},"instanceIdToInstanceContextMap":{"app_cartridge_v1-1":{"instanceId":"app_cartridge_v1-1","instanceProperties":{},"lifeCycleStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domain_app_cartridge_v1-1"},"networkPartitionId":"p1"}}}},"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"properties":{}}},"initialized":false}}'
>>>>>>>>> 2014-12-22 12:04:49,781:DEBUG:Executing handler for event
>>>>>>>>> 'CompleteTopologyEvent'
>>>>>>>>> 2014-12-22 12:04:49,781:DEBUG:Complete topology event received
>>>>>>>>> 2014-12-22 12:04:49,782:ERROR:Error processing
>>>>>>>>> 'CompleteTopologyEvent' event
>>>>>>>>> Traceback (most recent call last):
>>>>>>>>> File
>>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>>>>>>>> line 81, in on_message
>>>>>>>>> handler(msg)
>>>>>>>>> File "agent.py", line 255, in on_complete_topology
>>>>>>>>> event_obj = CompleteTopologyEvent.create_from_json(msg.payload)
>>>>>>>>> File
>>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/event/topology/events.py",
>>>>>>>>> line 190, in create_from_json
>>>>>>>>> member_obj.member_ip = member_str["memberIp"]
>>>>>>>>> KeyError: 'memberIp'
>>>>>>>>> 2014-12-22 12:04:49,820:DEBUG:Message received:
>>>>>>>>> 'tenant/CompleteTenantEvent':
>>>>>>>>> '{"tenants":[]}'
>>>>>>>>> 2014-12-22 12:04:49,820:DEBUG:Executing handler for event
>>>>>>>>> 'CompleteTenantEvent'
>>>>>>>>> 2014-12-22 12:04:49,820:INFO:Complete tenant event updating task
>>>>>>>>> disabled
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> *Carbon logs:*
>>>>>>>>>
>>>>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,240] INFO
>>>>>>>>> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
>>>>>>>>> - Publishing instance spawned event: [service-name] tomcat [cluster-id]
>>>>>>>>> mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
>>>>>>>>> mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 [instance-id]
>>>>>>>>> ap-southeast-1/i-df7dde12 [network-partition-id] p1 [partition-id] P1
>>>>>>>>> [lb-cluster-id] null
>>>>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,261] INFO
>>>>>>>>> {org.apache.stratos.messaging.domain.topology.lifecycle.LifeCycleStateManager}
>>>>>>>>> - Life Cycle State Manager started for Element [
>>>>>>>>> mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 ], initial
>>>>>>>>> state: Created
>>>>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,264] INFO
>>>>>>>>> {org.apache.stratos.messaging.message.processor.topology.InstanceSpawnedMessageProcessor}
>>>>>>>>> - Member created: [service] tomcat [cluster] mytomcat.tomcat.domain
>>>>>>>>> [member] mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101
>>>>>>>>> [instance] ap-southeast-1/i-df7dde12 [hierarchy-instance] app_cartridge_v1-1
>>>>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,268] INFO
>>>>>>>>> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
>>>>>>>>> - [InstanceSpawnedEventListener] Received: class
>>>>>>>>> org.apache.stratos.messaging.event.topology.InstanceSpawnedEvent
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>> Udara Liyanage
>>>>>>>>> Software Engineer
>>>>>>>>> WSO2, Inc.: http://wso2.com
>>>>>>>>> lean. enterprise. middleware
>>>>>>>>>
>>>>>>>>> web: http://udaraliyanage.wordpress.com
>>>>>>>>> phone:
>>>>>>>>> +94 71 443 6897
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Thanks and Regards,
>>>>>>>>>
>>>>>>>>> Isuru H.
>>>>>>>>> +94 716 358 048* <http://wso2.com/>*
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> * <http://wso2.com/>*
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Rajkumar Rajaratnam
>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>> Software Engineer, WSO2
>>>>>>>
>>>>>>> Mobile : +94777568639
>>>>>>> Blog : rajkumarr.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Rajkumar Rajaratnam
>>>>>> Committer & PMC Member, Apache Stratos
>>>>>> Software Engineer, WSO2
>>>>>>
>>>>>> Mobile : +94777568639
>>>>>> Blog : rajkumarr.com
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Rajkumar Rajaratnam
>>>>> Committer & PMC Member, Apache Stratos
>>>>> Software Engineer, WSO2
>>>>>
>>>>> Mobile : +94777568639
>>>>> Blog : rajkumarr.com
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Udara Liyanage
>>>> Software Engineer
>>>> WSO2, Inc.: http://wso2.com
>>>> lean. enterprise. middleware
>>>>
>>>> web: http://udaraliyanage.wordpress.com
>>>> phone: +94 71 443 6897
>>>>
>>>
>>>
>>> --
>>>
>>> Udara Liyanage
>>> Software Engineer
>>> WSO2, Inc.: http://wso2.com
>>> lean. enterprise. middleware
>>>
>>> web: http://udaraliyanage.wordpress.com
>>> phone: +94 71 443 6897
>>>
>>
>>
>>
>> --
>> Rajkumar Rajaratnam
>> Committer & PMC Member, Apache Stratos
>> Software Engineer, WSO2
>>
>> Mobile : +94777568639
>> Blog : rajkumarr.com
>>
>
>
> --
>
> Udara Liyanage
> Software Engineer
> WSO2, Inc.: http://wso2.com
> lean. enterprise. middleware
>
> web: http://udaraliyanage.wordpress.com
> phone: +94 71 443 6897
>
--
Rajkumar Rajaratnam
Committer & PMC Member, Apache Stratos
Software Engineer, WSO2
Mobile : +94777568639
Blog : rajkumarr.com
Re: Why agent wait to be intialized even after complete topology is received
Posted by Udara Liyanage <ud...@wso2.com>.
Hi Raj,
Logs are missed since instance is killed as an obsolete member. I will
provide the logs next time.
On Mon, Dec 22, 2014 at 1:13 PM, Rajkumar Rajaratnam <ra...@wso2.com>
wrote:
>
> Hi Udara,
>
> Can you attach the agent log?
>
> Thanks.
>
> On Mon, Dec 22, 2014 at 1:04 PM, Udara Liyanage <ud...@wso2.com> wrote:
>
>> Hi raj,
>>
>> I tested with lates python code base. This time member is in topology and
>> instance spawned event is also received. But yet instance activated is not
>> sent.
>>
>> *1) Instance started event received*
>>
>> 2014-12-22 12:53:49,904:DEBUG:Property added: [key] STRATOS_
>> MEMBER_PROPERTY_PRIMARY[value] false
>> 2014-12-22 12:53:49,911:DEBUG:Message received:
>> 'instance/status/InstanceStartedEvent':
>> '{\n "clusterId": "mytomcat.tomcat.domain", \n "clusterInstanceId":
>> "app_cartridge_v1-1", \n "instanceId": "", \n "memberId":
>> "mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130", \n
>> "networkPartitionId": "p1", \n "partitionId": "P1", \n
>> "serviceName": "tomcat"\n}'
>>
>> 2) Member in topology
>>
>> *Complete Topology:*
>>
>>
>> '{"topology":{"serviceMap":{"tomcat":{"serviceName":"tomcat","serviceType":"SingleTenant","clusterIdClusterMap":{"mytomcat.tomcat.domain":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","autoscalePolicyName":"autoscale_policy_1","hostNames":["
>> mytomcat.isuruh.lk
>> "],"tenantRange":"*","isLbCluster":false,"isKubernetesCluster":false,"memberMap":{"mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","memberId":"
>> *mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130*
>> ","instanceId":"ap-southeast-1/i-0147e4cc","clusterInstanceId":"app_cartridge_v1-1","networkPartitionId":"p1","partitionId":"P1","initTime":1419232682397,"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"memberPublicIPs":["54.254.118.165"],"defaultPublicIP":"54.254.118.165","memberPrivateIPs":["10.144.150.199"],"defaultPrivateIP":"10.144.150.199","properties":{"PRIMARY":"false","MIN_COUNT":"1"},"memberStateManager":{"stateStack":["Created","Starting"],"identifier":"mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130"}}},"appId":"app_cartridge_v1","properties":{},"instanceIdToInstanceContextMap":{"app_cartridge_v1-1":{"instanceId":"app_cartridge_v1-1","instanceProperties":{},"lifeCycleStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domain_app_cartridge_v1-1"},"networkPartitionId":"p1"}}}},"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"properties":{}}},"initialized":false}}'
>>
>> *Payload:*
>> APP_ID=app_cartridge_v1,GROUP_NAME=null,SERVICE_NAME=tomcat,HOST_NAME=
>> mytomcat.isuruh.lk
>> ,MULTITENANT=false,TENANT_ID=-1234,TENANT_RANGE=*,CARTRIDGE_ALIAS=mytomcat,CLUSTER_ID=mytomcat.tomcat.domain,CARTRIDGE_KEY=kYcZ4B4DPAeA1RGz,REPO_URL=
>> https://github.com/udaraliyanage/php,PORTS=80|443,PROVIDER=apache,PUPPET_IP=10.148.159.109,PUPPET_HOSTNAME=ip-10-148-159-109.ap-southeast-1.compute.internal,PUPPET_DNS_AVAILABLE=null,PUPPET_ENV=null,,,DEPLOYMENT=default,TOKEN=eyJhbGciOiJSUzI1NiJ9.eyJleHAiOjE4OTY4OTAwNDgsInN1YiI6ImFkbWluIiwiYXpwIjoiMjU1UUZET1UwbFlONmY5NE44aUVhQm5LdlBrYSIsImFwcElkIjoiYXBwX2NhcnRyaWRnZV92MSIsImF1ZCI6WyIyNTVRRkRPVTBsWU42Zjk0TjhpRWFCbkt2UGthIl0sImlzcyI6Imh0dHBzOlwvXC9sb2NhbGhvc3Q6OTQ0M1wvb2F1dGgyZW5kcG9pbnRzXC90b2tlbiIsImlhdCI6MTg5MzI5MDA0OH0.XB4wg8gQxzyH0jv5EIiWFjOG0VDZuFBswYG0G6zX3sjOEgXauywoga4pHmb9DWkKZU9nD5qRFLRyyiHi2onWr-TGtLNBn4qgRq94VDwjVgx5in4ND3f7q-xNSpLipMr_g_gtkbp9st-gZVeuB8y3u7jnjE8fuDEs63pQ75Drrhs,MEMBER_ID=
>> *mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130*
>> ,INSTANCE_ID=null,CLUSTER_INSTANCE_ID=app_cartridge_v1-1,LB_CLUSTER_ID=null,NETWORK_PARTITION_ID=p1,PARTITION_ID=P1,PRIMARY=false,MIN_COUNT=1
>>
>> Carbon logs:
>>
>> [2014-12-22 07:18:41,444] INFO
>> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
>> - Publishing instance spawned event: [service-name] tomcat [cluster-id]
>> mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
>> mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130 [instance-id]
>> ap-southeast-1/i-0147e4cc [network-partition-id] p1 [partition-id] P1
>> [lb-cluster-id] null
>> [2014-12-22 07:18:41,464] INFO
>> {org.apache.stratos.messaging.domain.topology.lifecycle.LifeCycleStateManager}
>> - Life Cycle State Manager started for Element [
>> mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130 ], initial
>> state: Created
>> [2014-12-22 07:18:41,469] INFO
>> {org.apache.stratos.messaging.message.processor.topology.InstanceSpawnedMessageProcessor}
>> - Member created: [service] tomcat [cluster] mytomcat.tomcat.domain
>> [member] mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130
>> [instance] ap-southeast-1/i-0147e4cc [hierarchy-instance] app_cartridge_v1-1
>> [2014-12-22 07:18:41,480] INFO
>> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
>> - [InstanceSpawnedEventListener] Received: class
>> org.apache.stratos.messaging.event.topology.InstanceSpawnedEvent
>> [2014-12-22 07:24:01,218] INFO
>> {org.apache.stratos.manager.listener.InstanceStatusListener} - Instance
>> status message received
>> [2014-12-22 07:24:01,218] INFO
>> {org.apache.stratos.manager.listener.InstanceStatusListener} - Event class
>> name:
>> org.apache.stratos.messaging.event.instance.status.InstanceStartedEvent
>> [2014-12-22 07:24:01,219] INFO
>> {org.apache.stratos.manager.listener.InstanceStatusListener} - Cluster id:
>> mytomcat.tomcat.domain
>> [2014-12-22 07:24:01,220] INFO
>> {org.apache.stratos.cloud.controller.messaging.topology.TopologyBuilder} -
>> member started event adding status started
>> [2014-12-22 07:24:01,235] INFO
>> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
>> - Publishing member started event: [service-name] tomcat [cluster-id]
>> mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
>> mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130 [instance-id]
>> [network-partition-id] p1 [partition-id] P1
>> [2014-12-22 07:24:01,250] INFO
>> {org.apache.stratos.messaging.message.processor.topology.MemberStartedMessageProcessor}
>> - Member started: [service] tomcat [cluster] mytomcat.tomcat.domain
>> [member] mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130
>> [2014-12-22 07:24:01,250] INFO
>> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
>> - [MemberStartedEventListener] Received: class
>> org.apache.stratos.messaging.event.topology.MemberStartedEvent
>>
>>
>>
>> On Mon, Dec 22, 2014 at 12:26 PM, Udara Liyanage <ud...@wso2.com> wrote:
>>>
>>> Hi Raj,
>>>
>>> Thanks for the info. I will check with updated agent.
>>>
>>> On Mon, Dec 22, 2014 at 12:24 PM, Rajkumar Rajaratnam <
>>> rajkumarr@wso2.com> wrote:
>>>>
>>>> Hi Udara,
>>>>
>>>> Are you using the latest python agent?
>>>>
>>>> Chamila changed the agent yesterday. Member object doesn't have an
>>>> attribute called "memberIp anymore. It has been changed to defaultPrivateIP.
>>>>
>>>> Latest python agent has this changes.
>>>>
>>>> Thanks.
>>>>
>>>> On Mon, Dec 22, 2014 at 12:22 PM, Rajkumar Rajaratnam <
>>>> rajkumarr@wso2.com> wrote:
>>>>
>>>>> Just noticed that there is an error in parsing complete topology
>>>>> event. So initialization didn't happen as Isuru mentioned.
>>>>>
>>>>> Thanks.
>>>>>
>>>>> On Mon, Dec 22, 2014 at 12:19 PM, Rajkumar Rajaratnam <
>>>>> rajkumarr@wso2.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> AFAIR, agent will wait for two conditions to happen. It will proceed
>>>>>> if either of two conditions becomes true.
>>>>>>
>>>>>> 1. Member should be in the topology
>>>>>> 2. Agent should get instance spawned event from CC
>>>>>>
>>>>>> So either of them didn't happen for you.
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> On Mon, Dec 22, 2014 at 12:17 PM, Isuru Haththotuwa <
>>>>>> isuruh@apache.org> wrote:
>>>>>>
>>>>>>> HI Udara,
>>>>>>>
>>>>>>> There is an error log about parsing the complete Topology event.
>>>>>>> Maybe the event did not get parsed correctly, and the required
>>>>>>> initialization did not happen.
>>>>>>>
>>>>>>> On Mon, Dec 22, 2014 at 12:12 PM, Udara Liyanage <ud...@wso2.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Chamila,
>>>>>>>>
>>>>>>>> I get the ":Waiting for Cartridge Agent to be initialized..." logs
>>>>>>>> continuously even though complete topology and complete tenant event is
>>>>>>>> already received.
>>>>>>>>
>>>>>>>>
>>>>>>>> 2014-12-22 12:04:47,595:DEBUG:Waiting for Cartridge Agent to be
>>>>>>>> initialized...
>>>>>>>> 2014-12-22 12:04:48,596:DEBUG:Waiting for Cartridge Agent to be
>>>>>>>> initialized...
>>>>>>>> 2014-12-22 12:04:49,597:DEBUG:Waiting for Cartridge Agent to be
>>>>>>>> initialized...
>>>>>>>> 2014-12-22 12:04:49,781:DEBUG:Message received:
>>>>>>>> 'topology/CompleteTopologyEvent':
>>>>>>>>
>>>>>>>> '{"topology":{"serviceMap":{"tomcat":{"serviceName":"tomcat","serviceType":"SingleTenant","clusterIdClusterMap":{"mytomcat.tomcat.domain":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","autoscalePolicyName":"autoscale_policy_1","hostNames":["
>>>>>>>> mytomcat.isuruh.lk
>>>>>>>> "],"tenantRange":"*","isLbCluster":false,"isKubernetesCluster":false,"memberMap":{"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","memberId":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101","instanceId":"ap-southeast-1/i-df7dde12","clusterInstanceId":"app_cartridge_v1-1","networkPartitionId":"p1","partitionId":"P1","initTime":1419229605140,"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"memberPublicIPs":["54.151.190.32"],"defaultPublicIP":"54.151.190.32","memberPrivateIPs":["10.144.68.9"],"defaultPrivateIP":"10.144.68.9","properties":{"PRIMARY":"false","MIN_COUNT":"1"},"memberStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101"}}},"appId":"app_cartridge_v1","properties":{},"instanceIdToInstanceContextMap":{"app_cartridge_v1-1":{"instanceId":"app_cartridge_v1-1","instanceProperties":{},"lifeCycleStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domain_app_cartridge_v1-1"},"networkPartitionId":"p1"}}}},"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"properties":{}}},"initialized":false}}'
>>>>>>>> 2014-12-22 12:04:49,781:DEBUG:Executing handler for event
>>>>>>>> 'CompleteTopologyEvent'
>>>>>>>> 2014-12-22 12:04:49,781:DEBUG:Complete topology event received
>>>>>>>> 2014-12-22 12:04:49,782:ERROR:Error processing
>>>>>>>> 'CompleteTopologyEvent' event
>>>>>>>> Traceback (most recent call last):
>>>>>>>> File
>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>>>>>>> line 81, in on_message
>>>>>>>> handler(msg)
>>>>>>>> File "agent.py", line 255, in on_complete_topology
>>>>>>>> event_obj = CompleteTopologyEvent.create_from_json(msg.payload)
>>>>>>>> File
>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/event/topology/events.py",
>>>>>>>> line 190, in create_from_json
>>>>>>>> member_obj.member_ip = member_str["memberIp"]
>>>>>>>> KeyError: 'memberIp'
>>>>>>>> 2014-12-22 12:04:49,820:DEBUG:Message received:
>>>>>>>> 'tenant/CompleteTenantEvent':
>>>>>>>> '{"tenants":[]}'
>>>>>>>> 2014-12-22 12:04:49,820:DEBUG:Executing handler for event
>>>>>>>> 'CompleteTenantEvent'
>>>>>>>> 2014-12-22 12:04:49,820:INFO:Complete tenant event updating task
>>>>>>>> disabled
>>>>>>>>
>>>>>>>>
>>>>>>>> *Carbon logs:*
>>>>>>>>
>>>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,240] INFO
>>>>>>>> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
>>>>>>>> - Publishing instance spawned event: [service-name] tomcat [cluster-id]
>>>>>>>> mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
>>>>>>>> mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 [instance-id]
>>>>>>>> ap-southeast-1/i-df7dde12 [network-partition-id] p1 [partition-id] P1
>>>>>>>> [lb-cluster-id] null
>>>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,261] INFO
>>>>>>>> {org.apache.stratos.messaging.domain.topology.lifecycle.LifeCycleStateManager}
>>>>>>>> - Life Cycle State Manager started for Element [
>>>>>>>> mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 ], initial
>>>>>>>> state: Created
>>>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,264] INFO
>>>>>>>> {org.apache.stratos.messaging.message.processor.topology.InstanceSpawnedMessageProcessor}
>>>>>>>> - Member created: [service] tomcat [cluster] mytomcat.tomcat.domain
>>>>>>>> [member] mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101
>>>>>>>> [instance] ap-southeast-1/i-df7dde12 [hierarchy-instance] app_cartridge_v1-1
>>>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,268] INFO
>>>>>>>> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
>>>>>>>> - [InstanceSpawnedEventListener] Received: class
>>>>>>>> org.apache.stratos.messaging.event.topology.InstanceSpawnedEvent
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> Udara Liyanage
>>>>>>>> Software Engineer
>>>>>>>> WSO2, Inc.: http://wso2.com
>>>>>>>> lean. enterprise. middleware
>>>>>>>>
>>>>>>>> web: http://udaraliyanage.wordpress.com
>>>>>>>> phone:
>>>>>>>> +94 71 443 6897
>>>>>>>>
>>>>>>>> --
>>>>>>>> Thanks and Regards,
>>>>>>>>
>>>>>>>> Isuru H.
>>>>>>>> +94 716 358 048* <http://wso2.com/>*
>>>>>>>>
>>>>>>>>
>>>>>>>> * <http://wso2.com/>*
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Rajkumar Rajaratnam
>>>>>> Committer & PMC Member, Apache Stratos
>>>>>> Software Engineer, WSO2
>>>>>>
>>>>>> Mobile : +94777568639
>>>>>> Blog : rajkumarr.com
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Rajkumar Rajaratnam
>>>>> Committer & PMC Member, Apache Stratos
>>>>> Software Engineer, WSO2
>>>>>
>>>>> Mobile : +94777568639
>>>>> Blog : rajkumarr.com
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Rajkumar Rajaratnam
>>>> Committer & PMC Member, Apache Stratos
>>>> Software Engineer, WSO2
>>>>
>>>> Mobile : +94777568639
>>>> Blog : rajkumarr.com
>>>>
>>>
>>>
>>> --
>>>
>>> Udara Liyanage
>>> Software Engineer
>>> WSO2, Inc.: http://wso2.com
>>> lean. enterprise. middleware
>>>
>>> web: http://udaraliyanage.wordpress.com
>>> phone: +94 71 443 6897
>>>
>>
>>
>> --
>>
>> Udara Liyanage
>> Software Engineer
>> WSO2, Inc.: http://wso2.com
>> lean. enterprise. middleware
>>
>> web: http://udaraliyanage.wordpress.com
>> phone: +94 71 443 6897
>>
>
>
>
> --
> Rajkumar Rajaratnam
> Committer & PMC Member, Apache Stratos
> Software Engineer, WSO2
>
> Mobile : +94777568639
> Blog : rajkumarr.com
>
--
Udara Liyanage
Software Engineer
WSO2, Inc.: http://wso2.com
lean. enterprise. middleware
web: http://udaraliyanage.wordpress.com
phone: +94 71 443 6897
Re: Why agent wait to be intialized even after complete topology is received
Posted by Chamila De Alwis <ch...@wso2.com>.
Hi Udara,
The instance activated event is sent,
1) after instance started if the repo_url is null. so in this case this
doesn't happen.
2) when the first ArtifactUpdatedEvent is processed.
Could you check if the artifact updated event is published?
Regards,
Chamila de Alwis
Software Engineer | WSO2 | +94772207163
Blog: code.chamiladealwis.com
On Mon, Dec 22, 2014 at 2:16 PM, Udara Liyanage <ud...@wso2.com> wrote:
> Hi raj,
>
> Could not fetch agent's log, instance got killed since it did not send
> activate :-(
>
> On Mon, Dec 22, 2014 at 1:13 PM, Rajkumar Rajaratnam <ra...@wso2.com>
> wrote:
>>
>> Hi Udara,
>>
>> Can you attach the agent log?
>>
>> Thanks.
>>
>> On Mon, Dec 22, 2014 at 1:04 PM, Udara Liyanage <ud...@wso2.com> wrote:
>>
>>> Hi raj,
>>>
>>> I tested with lates python code base. This time member is in topology
>>> and instance spawned event is also received. But yet instance activated is
>>> not sent.
>>>
>>> *1) Instance started event received*
>>>
>>> 2014-12-22 12:53:49,904:DEBUG:Property added: [key] STRATOS_
>>> MEMBER_PROPERTY_PRIMARY[value] false
>>> 2014-12-22 12:53:49,911:DEBUG:Message received:
>>> 'instance/status/InstanceStartedEvent':
>>> '{\n "clusterId": "mytomcat.tomcat.domain", \n
>>> "clusterInstanceId": "app_cartridge_v1-1", \n "instanceId": "", \n
>>> "memberId": "mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130",
>>> \n "networkPartitionId": "p1", \n "partitionId": "P1", \n
>>> "serviceName": "tomcat"\n}'
>>>
>>> 2) Member in topology
>>>
>>> *Complete Topology:*
>>>
>>>
>>> '{"topology":{"serviceMap":{"tomcat":{"serviceName":"tomcat","serviceType":"SingleTenant","clusterIdClusterMap":{"mytomcat.tomcat.domain":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","autoscalePolicyName":"autoscale_policy_1","hostNames":["
>>> mytomcat.isuruh.lk
>>> "],"tenantRange":"*","isLbCluster":false,"isKubernetesCluster":false,"memberMap":{"mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","memberId":"
>>> *mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130*
>>> ","instanceId":"ap-southeast-1/i-0147e4cc","clusterInstanceId":"app_cartridge_v1-1","networkPartitionId":"p1","partitionId":"P1","initTime":1419232682397,"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"memberPublicIPs":["54.254.118.165"],"defaultPublicIP":"54.254.118.165","memberPrivateIPs":["10.144.150.199"],"defaultPrivateIP":"10.144.150.199","properties":{"PRIMARY":"false","MIN_COUNT":"1"},"memberStateManager":{"stateStack":["Created","Starting"],"identifier":"mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130"}}},"appId":"app_cartridge_v1","properties":{},"instanceIdToInstanceContextMap":{"app_cartridge_v1-1":{"instanceId":"app_cartridge_v1-1","instanceProperties":{},"lifeCycleStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domain_app_cartridge_v1-1"},"networkPartitionId":"p1"}}}},"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"properties":{}}},"initialized":false}}'
>>>
>>> *Payload:*
>>> APP_ID=app_cartridge_v1,GROUP_NAME=null,SERVICE_NAME=tomcat,HOST_NAME=
>>> mytomcat.isuruh.lk
>>> ,MULTITENANT=false,TENANT_ID=-1234,TENANT_RANGE=*,CARTRIDGE_ALIAS=mytomcat,CLUSTER_ID=mytomcat.tomcat.domain,CARTRIDGE_KEY=kYcZ4B4DPAeA1RGz,REPO_URL=
>>> https://github.com/udaraliyanage/php,PORTS=80|443,PROVIDER=apache,PUPPET_IP=10.148.159.109,PUPPET_HOSTNAME=ip-10-148-159-109.ap-southeast-1.compute.internal,PUPPET_DNS_AVAILABLE=null,PUPPET_ENV=null,,,DEPLOYMENT=default,TOKEN=eyJhbGciOiJSUzI1NiJ9.eyJleHAiOjE4OTY4OTAwNDgsInN1YiI6ImFkbWluIiwiYXpwIjoiMjU1UUZET1UwbFlONmY5NE44aUVhQm5LdlBrYSIsImFwcElkIjoiYXBwX2NhcnRyaWRnZV92MSIsImF1ZCI6WyIyNTVRRkRPVTBsWU42Zjk0TjhpRWFCbkt2UGthIl0sImlzcyI6Imh0dHBzOlwvXC9sb2NhbGhvc3Q6OTQ0M1wvb2F1dGgyZW5kcG9pbnRzXC90b2tlbiIsImlhdCI6MTg5MzI5MDA0OH0.XB4wg8gQxzyH0jv5EIiWFjOG0VDZuFBswYG0G6zX3sjOEgXauywoga4pHmb9DWkKZU9nD5qRFLRyyiHi2onWr-TGtLNBn4qgRq94VDwjVgx5in4ND3f7q-xNSpLipMr_g_gtkbp9st-gZVeuB8y3u7jnjE8fuDEs63pQ75Drrhs,MEMBER_ID=
>>> *mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130*
>>> ,INSTANCE_ID=null,CLUSTER_INSTANCE_ID=app_cartridge_v1-1,LB_CLUSTER_ID=null,NETWORK_PARTITION_ID=p1,PARTITION_ID=P1,PRIMARY=false,MIN_COUNT=1
>>>
>>> Carbon logs:
>>>
>>> [2014-12-22 07:18:41,444] INFO
>>> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
>>> - Publishing instance spawned event: [service-name] tomcat [cluster-id]
>>> mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
>>> mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130 [instance-id]
>>> ap-southeast-1/i-0147e4cc [network-partition-id] p1 [partition-id] P1
>>> [lb-cluster-id] null
>>> [2014-12-22 07:18:41,464] INFO
>>> {org.apache.stratos.messaging.domain.topology.lifecycle.LifeCycleStateManager}
>>> - Life Cycle State Manager started for Element [
>>> mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130 ], initial
>>> state: Created
>>> [2014-12-22 07:18:41,469] INFO
>>> {org.apache.stratos.messaging.message.processor.topology.InstanceSpawnedMessageProcessor}
>>> - Member created: [service] tomcat [cluster] mytomcat.tomcat.domain
>>> [member] mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130
>>> [instance] ap-southeast-1/i-0147e4cc [hierarchy-instance] app_cartridge_v1-1
>>> [2014-12-22 07:18:41,480] INFO
>>> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
>>> - [InstanceSpawnedEventListener] Received: class
>>> org.apache.stratos.messaging.event.topology.InstanceSpawnedEvent
>>> [2014-12-22 07:24:01,218] INFO
>>> {org.apache.stratos.manager.listener.InstanceStatusListener} - Instance
>>> status message received
>>> [2014-12-22 07:24:01,218] INFO
>>> {org.apache.stratos.manager.listener.InstanceStatusListener} - Event class
>>> name:
>>> org.apache.stratos.messaging.event.instance.status.InstanceStartedEvent
>>> [2014-12-22 07:24:01,219] INFO
>>> {org.apache.stratos.manager.listener.InstanceStatusListener} - Cluster id:
>>> mytomcat.tomcat.domain
>>> [2014-12-22 07:24:01,220] INFO
>>> {org.apache.stratos.cloud.controller.messaging.topology.TopologyBuilder} -
>>> member started event adding status started
>>> [2014-12-22 07:24:01,235] INFO
>>> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
>>> - Publishing member started event: [service-name] tomcat [cluster-id]
>>> mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
>>> mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130 [instance-id]
>>> [network-partition-id] p1 [partition-id] P1
>>> [2014-12-22 07:24:01,250] INFO
>>> {org.apache.stratos.messaging.message.processor.topology.MemberStartedMessageProcessor}
>>> - Member started: [service] tomcat [cluster] mytomcat.tomcat.domain
>>> [member] mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130
>>> [2014-12-22 07:24:01,250] INFO
>>> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
>>> - [MemberStartedEventListener] Received: class
>>> org.apache.stratos.messaging.event.topology.MemberStartedEvent
>>>
>>>
>>>
>>> On Mon, Dec 22, 2014 at 12:26 PM, Udara Liyanage <ud...@wso2.com> wrote:
>>>>
>>>> Hi Raj,
>>>>
>>>> Thanks for the info. I will check with updated agent.
>>>>
>>>> On Mon, Dec 22, 2014 at 12:24 PM, Rajkumar Rajaratnam <
>>>> rajkumarr@wso2.com> wrote:
>>>>>
>>>>> Hi Udara,
>>>>>
>>>>> Are you using the latest python agent?
>>>>>
>>>>> Chamila changed the agent yesterday. Member object doesn't have an
>>>>> attribute called "memberIp anymore. It has been changed to defaultPrivateIP.
>>>>>
>>>>> Latest python agent has this changes.
>>>>>
>>>>> Thanks.
>>>>>
>>>>> On Mon, Dec 22, 2014 at 12:22 PM, Rajkumar Rajaratnam <
>>>>> rajkumarr@wso2.com> wrote:
>>>>>
>>>>>> Just noticed that there is an error in parsing complete topology
>>>>>> event. So initialization didn't happen as Isuru mentioned.
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> On Mon, Dec 22, 2014 at 12:19 PM, Rajkumar Rajaratnam <
>>>>>> rajkumarr@wso2.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> AFAIR, agent will wait for two conditions to happen. It will proceed
>>>>>>> if either of two conditions becomes true.
>>>>>>>
>>>>>>> 1. Member should be in the topology
>>>>>>> 2. Agent should get instance spawned event from CC
>>>>>>>
>>>>>>> So either of them didn't happen for you.
>>>>>>>
>>>>>>> Thanks.
>>>>>>>
>>>>>>> On Mon, Dec 22, 2014 at 12:17 PM, Isuru Haththotuwa <
>>>>>>> isuruh@apache.org> wrote:
>>>>>>>
>>>>>>>> HI Udara,
>>>>>>>>
>>>>>>>> There is an error log about parsing the complete Topology event.
>>>>>>>> Maybe the event did not get parsed correctly, and the required
>>>>>>>> initialization did not happen.
>>>>>>>>
>>>>>>>> On Mon, Dec 22, 2014 at 12:12 PM, Udara Liyanage <ud...@wso2.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Chamila,
>>>>>>>>>
>>>>>>>>> I get the ":Waiting for Cartridge Agent to be initialized..." logs
>>>>>>>>> continuously even though complete topology and complete tenant event is
>>>>>>>>> already received.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2014-12-22 12:04:47,595:DEBUG:Waiting for Cartridge Agent to be
>>>>>>>>> initialized...
>>>>>>>>> 2014-12-22 12:04:48,596:DEBUG:Waiting for Cartridge Agent to be
>>>>>>>>> initialized...
>>>>>>>>> 2014-12-22 12:04:49,597:DEBUG:Waiting for Cartridge Agent to be
>>>>>>>>> initialized...
>>>>>>>>> 2014-12-22 12:04:49,781:DEBUG:Message received:
>>>>>>>>> 'topology/CompleteTopologyEvent':
>>>>>>>>>
>>>>>>>>> '{"topology":{"serviceMap":{"tomcat":{"serviceName":"tomcat","serviceType":"SingleTenant","clusterIdClusterMap":{"mytomcat.tomcat.domain":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","autoscalePolicyName":"autoscale_policy_1","hostNames":["
>>>>>>>>> mytomcat.isuruh.lk
>>>>>>>>> "],"tenantRange":"*","isLbCluster":false,"isKubernetesCluster":false,"memberMap":{"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","memberId":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101","instanceId":"ap-southeast-1/i-df7dde12","clusterInstanceId":"app_cartridge_v1-1","networkPartitionId":"p1","partitionId":"P1","initTime":1419229605140,"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"memberPublicIPs":["54.151.190.32"],"defaultPublicIP":"54.151.190.32","memberPrivateIPs":["10.144.68.9"],"defaultPrivateIP":"10.144.68.9","properties":{"PRIMARY":"false","MIN_COUNT":"1"},"memberStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101"}}},"appId":"app_cartridge_v1","properties":{},"instanceIdToInstanceContextMap":{"app_cartridge_v1-1":{"instanceId":"app_cartridge_v1-1","instanceProperties":{},"lifeCycleStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domain_app_cartridge_v1-1"},"networkPartitionId":"p1"}}}},"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"properties":{}}},"initialized":false}}'
>>>>>>>>> 2014-12-22 12:04:49,781:DEBUG:Executing handler for event
>>>>>>>>> 'CompleteTopologyEvent'
>>>>>>>>> 2014-12-22 12:04:49,781:DEBUG:Complete topology event received
>>>>>>>>> 2014-12-22 12:04:49,782:ERROR:Error processing
>>>>>>>>> 'CompleteTopologyEvent' event
>>>>>>>>> Traceback (most recent call last):
>>>>>>>>> File
>>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>>>>>>>> line 81, in on_message
>>>>>>>>> handler(msg)
>>>>>>>>> File "agent.py", line 255, in on_complete_topology
>>>>>>>>> event_obj = CompleteTopologyEvent.create_from_json(msg.payload)
>>>>>>>>> File
>>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/event/topology/events.py",
>>>>>>>>> line 190, in create_from_json
>>>>>>>>> member_obj.member_ip = member_str["memberIp"]
>>>>>>>>> KeyError: 'memberIp'
>>>>>>>>> 2014-12-22 12:04:49,820:DEBUG:Message received:
>>>>>>>>> 'tenant/CompleteTenantEvent':
>>>>>>>>> '{"tenants":[]}'
>>>>>>>>> 2014-12-22 12:04:49,820:DEBUG:Executing handler for event
>>>>>>>>> 'CompleteTenantEvent'
>>>>>>>>> 2014-12-22 12:04:49,820:INFO:Complete tenant event updating task
>>>>>>>>> disabled
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> *Carbon logs:*
>>>>>>>>>
>>>>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,240] INFO
>>>>>>>>> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
>>>>>>>>> - Publishing instance spawned event: [service-name] tomcat [cluster-id]
>>>>>>>>> mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
>>>>>>>>> mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 [instance-id]
>>>>>>>>> ap-southeast-1/i-df7dde12 [network-partition-id] p1 [partition-id] P1
>>>>>>>>> [lb-cluster-id] null
>>>>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,261] INFO
>>>>>>>>> {org.apache.stratos.messaging.domain.topology.lifecycle.LifeCycleStateManager}
>>>>>>>>> - Life Cycle State Manager started for Element [
>>>>>>>>> mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 ], initial
>>>>>>>>> state: Created
>>>>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,264] INFO
>>>>>>>>> {org.apache.stratos.messaging.message.processor.topology.InstanceSpawnedMessageProcessor}
>>>>>>>>> - Member created: [service] tomcat [cluster] mytomcat.tomcat.domain
>>>>>>>>> [member] mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101
>>>>>>>>> [instance] ap-southeast-1/i-df7dde12 [hierarchy-instance] app_cartridge_v1-1
>>>>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,268] INFO
>>>>>>>>> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
>>>>>>>>> - [InstanceSpawnedEventListener] Received: class
>>>>>>>>> org.apache.stratos.messaging.event.topology.InstanceSpawnedEvent
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>> Udara Liyanage
>>>>>>>>> Software Engineer
>>>>>>>>> WSO2, Inc.: http://wso2.com
>>>>>>>>> lean. enterprise. middleware
>>>>>>>>>
>>>>>>>>> web: http://udaraliyanage.wordpress.com
>>>>>>>>> phone:
>>>>>>>>> +94 71 443 6897
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Thanks and Regards,
>>>>>>>>>
>>>>>>>>> Isuru H.
>>>>>>>>> +94 716 358 048* <http://wso2.com/>*
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> * <http://wso2.com/>*
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Rajkumar Rajaratnam
>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>> Software Engineer, WSO2
>>>>>>>
>>>>>>> Mobile : +94777568639
>>>>>>> Blog : rajkumarr.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Rajkumar Rajaratnam
>>>>>> Committer & PMC Member, Apache Stratos
>>>>>> Software Engineer, WSO2
>>>>>>
>>>>>> Mobile : +94777568639
>>>>>> Blog : rajkumarr.com
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Rajkumar Rajaratnam
>>>>> Committer & PMC Member, Apache Stratos
>>>>> Software Engineer, WSO2
>>>>>
>>>>> Mobile : +94777568639
>>>>> Blog : rajkumarr.com
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Udara Liyanage
>>>> Software Engineer
>>>> WSO2, Inc.: http://wso2.com
>>>> lean. enterprise. middleware
>>>>
>>>> web: http://udaraliyanage.wordpress.com
>>>> phone: +94 71 443 6897
>>>>
>>>
>>>
>>> --
>>>
>>> Udara Liyanage
>>> Software Engineer
>>> WSO2, Inc.: http://wso2.com
>>> lean. enterprise. middleware
>>>
>>> web: http://udaraliyanage.wordpress.com
>>> phone: +94 71 443 6897
>>>
>>
>>
>>
>> --
>> Rajkumar Rajaratnam
>> Committer & PMC Member, Apache Stratos
>> Software Engineer, WSO2
>>
>> Mobile : +94777568639
>> Blog : rajkumarr.com
>>
>
>
> --
>
> Udara Liyanage
> Software Engineer
> WSO2, Inc.: http://wso2.com
> lean. enterprise. middleware
>
> web: http://udaraliyanage.wordpress.com
> phone: +94 71 443 6897
>
Re: Why agent wait to be intialized even after complete topology is received
Posted by Udara Liyanage <ud...@wso2.com>.
Hi raj,
Could not fetch agent's log, instance got killed since it did not send
activate :-(
On Mon, Dec 22, 2014 at 1:13 PM, Rajkumar Rajaratnam <ra...@wso2.com>
wrote:
>
> Hi Udara,
>
> Can you attach the agent log?
>
> Thanks.
>
> On Mon, Dec 22, 2014 at 1:04 PM, Udara Liyanage <ud...@wso2.com> wrote:
>
>> Hi raj,
>>
>> I tested with lates python code base. This time member is in topology and
>> instance spawned event is also received. But yet instance activated is not
>> sent.
>>
>> *1) Instance started event received*
>>
>> 2014-12-22 12:53:49,904:DEBUG:Property added: [key] STRATOS_
>> MEMBER_PROPERTY_PRIMARY[value] false
>> 2014-12-22 12:53:49,911:DEBUG:Message received:
>> 'instance/status/InstanceStartedEvent':
>> '{\n "clusterId": "mytomcat.tomcat.domain", \n "clusterInstanceId":
>> "app_cartridge_v1-1", \n "instanceId": "", \n "memberId":
>> "mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130", \n
>> "networkPartitionId": "p1", \n "partitionId": "P1", \n
>> "serviceName": "tomcat"\n}'
>>
>> 2) Member in topology
>>
>> *Complete Topology:*
>>
>>
>> '{"topology":{"serviceMap":{"tomcat":{"serviceName":"tomcat","serviceType":"SingleTenant","clusterIdClusterMap":{"mytomcat.tomcat.domain":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","autoscalePolicyName":"autoscale_policy_1","hostNames":["
>> mytomcat.isuruh.lk
>> "],"tenantRange":"*","isLbCluster":false,"isKubernetesCluster":false,"memberMap":{"mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","memberId":"
>> *mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130*
>> ","instanceId":"ap-southeast-1/i-0147e4cc","clusterInstanceId":"app_cartridge_v1-1","networkPartitionId":"p1","partitionId":"P1","initTime":1419232682397,"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"memberPublicIPs":["54.254.118.165"],"defaultPublicIP":"54.254.118.165","memberPrivateIPs":["10.144.150.199"],"defaultPrivateIP":"10.144.150.199","properties":{"PRIMARY":"false","MIN_COUNT":"1"},"memberStateManager":{"stateStack":["Created","Starting"],"identifier":"mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130"}}},"appId":"app_cartridge_v1","properties":{},"instanceIdToInstanceContextMap":{"app_cartridge_v1-1":{"instanceId":"app_cartridge_v1-1","instanceProperties":{},"lifeCycleStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domain_app_cartridge_v1-1"},"networkPartitionId":"p1"}}}},"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"properties":{}}},"initialized":false}}'
>>
>> *Payload:*
>> APP_ID=app_cartridge_v1,GROUP_NAME=null,SERVICE_NAME=tomcat,HOST_NAME=
>> mytomcat.isuruh.lk
>> ,MULTITENANT=false,TENANT_ID=-1234,TENANT_RANGE=*,CARTRIDGE_ALIAS=mytomcat,CLUSTER_ID=mytomcat.tomcat.domain,CARTRIDGE_KEY=kYcZ4B4DPAeA1RGz,REPO_URL=
>> https://github.com/udaraliyanage/php,PORTS=80|443,PROVIDER=apache,PUPPET_IP=10.148.159.109,PUPPET_HOSTNAME=ip-10-148-159-109.ap-southeast-1.compute.internal,PUPPET_DNS_AVAILABLE=null,PUPPET_ENV=null,,,DEPLOYMENT=default,TOKEN=eyJhbGciOiJSUzI1NiJ9.eyJleHAiOjE4OTY4OTAwNDgsInN1YiI6ImFkbWluIiwiYXpwIjoiMjU1UUZET1UwbFlONmY5NE44aUVhQm5LdlBrYSIsImFwcElkIjoiYXBwX2NhcnRyaWRnZV92MSIsImF1ZCI6WyIyNTVRRkRPVTBsWU42Zjk0TjhpRWFCbkt2UGthIl0sImlzcyI6Imh0dHBzOlwvXC9sb2NhbGhvc3Q6OTQ0M1wvb2F1dGgyZW5kcG9pbnRzXC90b2tlbiIsImlhdCI6MTg5MzI5MDA0OH0.XB4wg8gQxzyH0jv5EIiWFjOG0VDZuFBswYG0G6zX3sjOEgXauywoga4pHmb9DWkKZU9nD5qRFLRyyiHi2onWr-TGtLNBn4qgRq94VDwjVgx5in4ND3f7q-xNSpLipMr_g_gtkbp9st-gZVeuB8y3u7jnjE8fuDEs63pQ75Drrhs,MEMBER_ID=
>> *mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130*
>> ,INSTANCE_ID=null,CLUSTER_INSTANCE_ID=app_cartridge_v1-1,LB_CLUSTER_ID=null,NETWORK_PARTITION_ID=p1,PARTITION_ID=P1,PRIMARY=false,MIN_COUNT=1
>>
>> Carbon logs:
>>
>> [2014-12-22 07:18:41,444] INFO
>> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
>> - Publishing instance spawned event: [service-name] tomcat [cluster-id]
>> mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
>> mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130 [instance-id]
>> ap-southeast-1/i-0147e4cc [network-partition-id] p1 [partition-id] P1
>> [lb-cluster-id] null
>> [2014-12-22 07:18:41,464] INFO
>> {org.apache.stratos.messaging.domain.topology.lifecycle.LifeCycleStateManager}
>> - Life Cycle State Manager started for Element [
>> mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130 ], initial
>> state: Created
>> [2014-12-22 07:18:41,469] INFO
>> {org.apache.stratos.messaging.message.processor.topology.InstanceSpawnedMessageProcessor}
>> - Member created: [service] tomcat [cluster] mytomcat.tomcat.domain
>> [member] mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130
>> [instance] ap-southeast-1/i-0147e4cc [hierarchy-instance] app_cartridge_v1-1
>> [2014-12-22 07:18:41,480] INFO
>> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
>> - [InstanceSpawnedEventListener] Received: class
>> org.apache.stratos.messaging.event.topology.InstanceSpawnedEvent
>> [2014-12-22 07:24:01,218] INFO
>> {org.apache.stratos.manager.listener.InstanceStatusListener} - Instance
>> status message received
>> [2014-12-22 07:24:01,218] INFO
>> {org.apache.stratos.manager.listener.InstanceStatusListener} - Event class
>> name:
>> org.apache.stratos.messaging.event.instance.status.InstanceStartedEvent
>> [2014-12-22 07:24:01,219] INFO
>> {org.apache.stratos.manager.listener.InstanceStatusListener} - Cluster id:
>> mytomcat.tomcat.domain
>> [2014-12-22 07:24:01,220] INFO
>> {org.apache.stratos.cloud.controller.messaging.topology.TopologyBuilder} -
>> member started event adding status started
>> [2014-12-22 07:24:01,235] INFO
>> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
>> - Publishing member started event: [service-name] tomcat [cluster-id]
>> mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
>> mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130 [instance-id]
>> [network-partition-id] p1 [partition-id] P1
>> [2014-12-22 07:24:01,250] INFO
>> {org.apache.stratos.messaging.message.processor.topology.MemberStartedMessageProcessor}
>> - Member started: [service] tomcat [cluster] mytomcat.tomcat.domain
>> [member] mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130
>> [2014-12-22 07:24:01,250] INFO
>> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
>> - [MemberStartedEventListener] Received: class
>> org.apache.stratos.messaging.event.topology.MemberStartedEvent
>>
>>
>>
>> On Mon, Dec 22, 2014 at 12:26 PM, Udara Liyanage <ud...@wso2.com> wrote:
>>>
>>> Hi Raj,
>>>
>>> Thanks for the info. I will check with updated agent.
>>>
>>> On Mon, Dec 22, 2014 at 12:24 PM, Rajkumar Rajaratnam <
>>> rajkumarr@wso2.com> wrote:
>>>>
>>>> Hi Udara,
>>>>
>>>> Are you using the latest python agent?
>>>>
>>>> Chamila changed the agent yesterday. Member object doesn't have an
>>>> attribute called "memberIp anymore. It has been changed to defaultPrivateIP.
>>>>
>>>> Latest python agent has this changes.
>>>>
>>>> Thanks.
>>>>
>>>> On Mon, Dec 22, 2014 at 12:22 PM, Rajkumar Rajaratnam <
>>>> rajkumarr@wso2.com> wrote:
>>>>
>>>>> Just noticed that there is an error in parsing complete topology
>>>>> event. So initialization didn't happen as Isuru mentioned.
>>>>>
>>>>> Thanks.
>>>>>
>>>>> On Mon, Dec 22, 2014 at 12:19 PM, Rajkumar Rajaratnam <
>>>>> rajkumarr@wso2.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> AFAIR, agent will wait for two conditions to happen. It will proceed
>>>>>> if either of two conditions becomes true.
>>>>>>
>>>>>> 1. Member should be in the topology
>>>>>> 2. Agent should get instance spawned event from CC
>>>>>>
>>>>>> So either of them didn't happen for you.
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> On Mon, Dec 22, 2014 at 12:17 PM, Isuru Haththotuwa <
>>>>>> isuruh@apache.org> wrote:
>>>>>>
>>>>>>> HI Udara,
>>>>>>>
>>>>>>> There is an error log about parsing the complete Topology event.
>>>>>>> Maybe the event did not get parsed correctly, and the required
>>>>>>> initialization did not happen.
>>>>>>>
>>>>>>> On Mon, Dec 22, 2014 at 12:12 PM, Udara Liyanage <ud...@wso2.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Chamila,
>>>>>>>>
>>>>>>>> I get the ":Waiting for Cartridge Agent to be initialized..." logs
>>>>>>>> continuously even though complete topology and complete tenant event is
>>>>>>>> already received.
>>>>>>>>
>>>>>>>>
>>>>>>>> 2014-12-22 12:04:47,595:DEBUG:Waiting for Cartridge Agent to be
>>>>>>>> initialized...
>>>>>>>> 2014-12-22 12:04:48,596:DEBUG:Waiting for Cartridge Agent to be
>>>>>>>> initialized...
>>>>>>>> 2014-12-22 12:04:49,597:DEBUG:Waiting for Cartridge Agent to be
>>>>>>>> initialized...
>>>>>>>> 2014-12-22 12:04:49,781:DEBUG:Message received:
>>>>>>>> 'topology/CompleteTopologyEvent':
>>>>>>>>
>>>>>>>> '{"topology":{"serviceMap":{"tomcat":{"serviceName":"tomcat","serviceType":"SingleTenant","clusterIdClusterMap":{"mytomcat.tomcat.domain":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","autoscalePolicyName":"autoscale_policy_1","hostNames":["
>>>>>>>> mytomcat.isuruh.lk
>>>>>>>> "],"tenantRange":"*","isLbCluster":false,"isKubernetesCluster":false,"memberMap":{"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","memberId":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101","instanceId":"ap-southeast-1/i-df7dde12","clusterInstanceId":"app_cartridge_v1-1","networkPartitionId":"p1","partitionId":"P1","initTime":1419229605140,"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"memberPublicIPs":["54.151.190.32"],"defaultPublicIP":"54.151.190.32","memberPrivateIPs":["10.144.68.9"],"defaultPrivateIP":"10.144.68.9","properties":{"PRIMARY":"false","MIN_COUNT":"1"},"memberStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101"}}},"appId":"app_cartridge_v1","properties":{},"instanceIdToInstanceContextMap":{"app_cartridge_v1-1":{"instanceId":"app_cartridge_v1-1","instanceProperties":{},"lifeCycleStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domain_app_cartridge_v1-1"},"networkPartitionId":"p1"}}}},"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"properties":{}}},"initialized":false}}'
>>>>>>>> 2014-12-22 12:04:49,781:DEBUG:Executing handler for event
>>>>>>>> 'CompleteTopologyEvent'
>>>>>>>> 2014-12-22 12:04:49,781:DEBUG:Complete topology event received
>>>>>>>> 2014-12-22 12:04:49,782:ERROR:Error processing
>>>>>>>> 'CompleteTopologyEvent' event
>>>>>>>> Traceback (most recent call last):
>>>>>>>> File
>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>>>>>>> line 81, in on_message
>>>>>>>> handler(msg)
>>>>>>>> File "agent.py", line 255, in on_complete_topology
>>>>>>>> event_obj = CompleteTopologyEvent.create_from_json(msg.payload)
>>>>>>>> File
>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/event/topology/events.py",
>>>>>>>> line 190, in create_from_json
>>>>>>>> member_obj.member_ip = member_str["memberIp"]
>>>>>>>> KeyError: 'memberIp'
>>>>>>>> 2014-12-22 12:04:49,820:DEBUG:Message received:
>>>>>>>> 'tenant/CompleteTenantEvent':
>>>>>>>> '{"tenants":[]}'
>>>>>>>> 2014-12-22 12:04:49,820:DEBUG:Executing handler for event
>>>>>>>> 'CompleteTenantEvent'
>>>>>>>> 2014-12-22 12:04:49,820:INFO:Complete tenant event updating task
>>>>>>>> disabled
>>>>>>>>
>>>>>>>>
>>>>>>>> *Carbon logs:*
>>>>>>>>
>>>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,240] INFO
>>>>>>>> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
>>>>>>>> - Publishing instance spawned event: [service-name] tomcat [cluster-id]
>>>>>>>> mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
>>>>>>>> mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 [instance-id]
>>>>>>>> ap-southeast-1/i-df7dde12 [network-partition-id] p1 [partition-id] P1
>>>>>>>> [lb-cluster-id] null
>>>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,261] INFO
>>>>>>>> {org.apache.stratos.messaging.domain.topology.lifecycle.LifeCycleStateManager}
>>>>>>>> - Life Cycle State Manager started for Element [
>>>>>>>> mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 ], initial
>>>>>>>> state: Created
>>>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,264] INFO
>>>>>>>> {org.apache.stratos.messaging.message.processor.topology.InstanceSpawnedMessageProcessor}
>>>>>>>> - Member created: [service] tomcat [cluster] mytomcat.tomcat.domain
>>>>>>>> [member] mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101
>>>>>>>> [instance] ap-southeast-1/i-df7dde12 [hierarchy-instance] app_cartridge_v1-1
>>>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,268] INFO
>>>>>>>> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
>>>>>>>> - [InstanceSpawnedEventListener] Received: class
>>>>>>>> org.apache.stratos.messaging.event.topology.InstanceSpawnedEvent
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> Udara Liyanage
>>>>>>>> Software Engineer
>>>>>>>> WSO2, Inc.: http://wso2.com
>>>>>>>> lean. enterprise. middleware
>>>>>>>>
>>>>>>>> web: http://udaraliyanage.wordpress.com
>>>>>>>> phone:
>>>>>>>> +94 71 443 6897
>>>>>>>>
>>>>>>>> --
>>>>>>>> Thanks and Regards,
>>>>>>>>
>>>>>>>> Isuru H.
>>>>>>>> +94 716 358 048* <http://wso2.com/>*
>>>>>>>>
>>>>>>>>
>>>>>>>> * <http://wso2.com/>*
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Rajkumar Rajaratnam
>>>>>> Committer & PMC Member, Apache Stratos
>>>>>> Software Engineer, WSO2
>>>>>>
>>>>>> Mobile : +94777568639
>>>>>> Blog : rajkumarr.com
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Rajkumar Rajaratnam
>>>>> Committer & PMC Member, Apache Stratos
>>>>> Software Engineer, WSO2
>>>>>
>>>>> Mobile : +94777568639
>>>>> Blog : rajkumarr.com
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Rajkumar Rajaratnam
>>>> Committer & PMC Member, Apache Stratos
>>>> Software Engineer, WSO2
>>>>
>>>> Mobile : +94777568639
>>>> Blog : rajkumarr.com
>>>>
>>>
>>>
>>> --
>>>
>>> Udara Liyanage
>>> Software Engineer
>>> WSO2, Inc.: http://wso2.com
>>> lean. enterprise. middleware
>>>
>>> web: http://udaraliyanage.wordpress.com
>>> phone: +94 71 443 6897
>>>
>>
>>
>> --
>>
>> Udara Liyanage
>> Software Engineer
>> WSO2, Inc.: http://wso2.com
>> lean. enterprise. middleware
>>
>> web: http://udaraliyanage.wordpress.com
>> phone: +94 71 443 6897
>>
>
>
>
> --
> Rajkumar Rajaratnam
> Committer & PMC Member, Apache Stratos
> Software Engineer, WSO2
>
> Mobile : +94777568639
> Blog : rajkumarr.com
>
--
Udara Liyanage
Software Engineer
WSO2, Inc.: http://wso2.com
lean. enterprise. middleware
web: http://udaraliyanage.wordpress.com
phone: +94 71 443 6897
Re: Why agent wait to be intialized even after complete topology is received
Posted by Rajkumar Rajaratnam <ra...@wso2.com>.
Hi Udara,
Can you attach the agent log?
Thanks.
On Mon, Dec 22, 2014 at 1:04 PM, Udara Liyanage <ud...@wso2.com> wrote:
> Hi raj,
>
> I tested with lates python code base. This time member is in topology and
> instance spawned event is also received. But yet instance activated is not
> sent.
>
> *1) Instance started event received*
>
> 2014-12-22 12:53:49,904:DEBUG:Property added: [key] STRATOS_
> MEMBER_PROPERTY_PRIMARY[value] false
> 2014-12-22 12:53:49,911:DEBUG:Message received:
> 'instance/status/InstanceStartedEvent':
> '{\n "clusterId": "mytomcat.tomcat.domain", \n "clusterInstanceId":
> "app_cartridge_v1-1", \n "instanceId": "", \n "memberId":
> "mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130", \n
> "networkPartitionId": "p1", \n "partitionId": "P1", \n
> "serviceName": "tomcat"\n}'
>
> 2) Member in topology
>
> *Complete Topology:*
>
>
> '{"topology":{"serviceMap":{"tomcat":{"serviceName":"tomcat","serviceType":"SingleTenant","clusterIdClusterMap":{"mytomcat.tomcat.domain":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","autoscalePolicyName":"autoscale_policy_1","hostNames":["
> mytomcat.isuruh.lk
> "],"tenantRange":"*","isLbCluster":false,"isKubernetesCluster":false,"memberMap":{"mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","memberId":"
> *mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130*
> ","instanceId":"ap-southeast-1/i-0147e4cc","clusterInstanceId":"app_cartridge_v1-1","networkPartitionId":"p1","partitionId":"P1","initTime":1419232682397,"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"memberPublicIPs":["54.254.118.165"],"defaultPublicIP":"54.254.118.165","memberPrivateIPs":["10.144.150.199"],"defaultPrivateIP":"10.144.150.199","properties":{"PRIMARY":"false","MIN_COUNT":"1"},"memberStateManager":{"stateStack":["Created","Starting"],"identifier":"mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130"}}},"appId":"app_cartridge_v1","properties":{},"instanceIdToInstanceContextMap":{"app_cartridge_v1-1":{"instanceId":"app_cartridge_v1-1","instanceProperties":{},"lifeCycleStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domain_app_cartridge_v1-1"},"networkPartitionId":"p1"}}}},"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"properties":{}}},"initialized":false}}'
>
> *Payload:*
> APP_ID=app_cartridge_v1,GROUP_NAME=null,SERVICE_NAME=tomcat,HOST_NAME=
> mytomcat.isuruh.lk
> ,MULTITENANT=false,TENANT_ID=-1234,TENANT_RANGE=*,CARTRIDGE_ALIAS=mytomcat,CLUSTER_ID=mytomcat.tomcat.domain,CARTRIDGE_KEY=kYcZ4B4DPAeA1RGz,REPO_URL=
> https://github.com/udaraliyanage/php,PORTS=80|443,PROVIDER=apache,PUPPET_IP=10.148.159.109,PUPPET_HOSTNAME=ip-10-148-159-109.ap-southeast-1.compute.internal,PUPPET_DNS_AVAILABLE=null,PUPPET_ENV=null,,,DEPLOYMENT=default,TOKEN=eyJhbGciOiJSUzI1NiJ9.eyJleHAiOjE4OTY4OTAwNDgsInN1YiI6ImFkbWluIiwiYXpwIjoiMjU1UUZET1UwbFlONmY5NE44aUVhQm5LdlBrYSIsImFwcElkIjoiYXBwX2NhcnRyaWRnZV92MSIsImF1ZCI6WyIyNTVRRkRPVTBsWU42Zjk0TjhpRWFCbkt2UGthIl0sImlzcyI6Imh0dHBzOlwvXC9sb2NhbGhvc3Q6OTQ0M1wvb2F1dGgyZW5kcG9pbnRzXC90b2tlbiIsImlhdCI6MTg5MzI5MDA0OH0.XB4wg8gQxzyH0jv5EIiWFjOG0VDZuFBswYG0G6zX3sjOEgXauywoga4pHmb9DWkKZU9nD5qRFLRyyiHi2onWr-TGtLNBn4qgRq94VDwjVgx5in4ND3f7q-xNSpLipMr_g_gtkbp9st-gZVeuB8y3u7jnjE8fuDEs63pQ75Drrhs,MEMBER_ID=
> *mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130*
> ,INSTANCE_ID=null,CLUSTER_INSTANCE_ID=app_cartridge_v1-1,LB_CLUSTER_ID=null,NETWORK_PARTITION_ID=p1,PARTITION_ID=P1,PRIMARY=false,MIN_COUNT=1
>
> Carbon logs:
>
> [2014-12-22 07:18:41,444] INFO
> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
> - Publishing instance spawned event: [service-name] tomcat [cluster-id]
> mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
> mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130 [instance-id]
> ap-southeast-1/i-0147e4cc [network-partition-id] p1 [partition-id] P1
> [lb-cluster-id] null
> [2014-12-22 07:18:41,464] INFO
> {org.apache.stratos.messaging.domain.topology.lifecycle.LifeCycleStateManager}
> - Life Cycle State Manager started for Element [
> mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130 ], initial
> state: Created
> [2014-12-22 07:18:41,469] INFO
> {org.apache.stratos.messaging.message.processor.topology.InstanceSpawnedMessageProcessor}
> - Member created: [service] tomcat [cluster] mytomcat.tomcat.domain
> [member] mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130
> [instance] ap-southeast-1/i-0147e4cc [hierarchy-instance] app_cartridge_v1-1
> [2014-12-22 07:18:41,480] INFO
> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
> - [InstanceSpawnedEventListener] Received: class
> org.apache.stratos.messaging.event.topology.InstanceSpawnedEvent
> [2014-12-22 07:24:01,218] INFO
> {org.apache.stratos.manager.listener.InstanceStatusListener} - Instance
> status message received
> [2014-12-22 07:24:01,218] INFO
> {org.apache.stratos.manager.listener.InstanceStatusListener} - Event class
> name:
> org.apache.stratos.messaging.event.instance.status.InstanceStartedEvent
> [2014-12-22 07:24:01,219] INFO
> {org.apache.stratos.manager.listener.InstanceStatusListener} - Cluster id:
> mytomcat.tomcat.domain
> [2014-12-22 07:24:01,220] INFO
> {org.apache.stratos.cloud.controller.messaging.topology.TopologyBuilder} -
> member started event adding status started
> [2014-12-22 07:24:01,235] INFO
> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
> - Publishing member started event: [service-name] tomcat [cluster-id]
> mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
> mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130 [instance-id]
> [network-partition-id] p1 [partition-id] P1
> [2014-12-22 07:24:01,250] INFO
> {org.apache.stratos.messaging.message.processor.topology.MemberStartedMessageProcessor}
> - Member started: [service] tomcat [cluster] mytomcat.tomcat.domain
> [member] mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130
> [2014-12-22 07:24:01,250] INFO
> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
> - [MemberStartedEventListener] Received: class
> org.apache.stratos.messaging.event.topology.MemberStartedEvent
>
>
>
> On Mon, Dec 22, 2014 at 12:26 PM, Udara Liyanage <ud...@wso2.com> wrote:
>>
>> Hi Raj,
>>
>> Thanks for the info. I will check with updated agent.
>>
>> On Mon, Dec 22, 2014 at 12:24 PM, Rajkumar Rajaratnam <rajkumarr@wso2.com
>> > wrote:
>>>
>>> Hi Udara,
>>>
>>> Are you using the latest python agent?
>>>
>>> Chamila changed the agent yesterday. Member object doesn't have an
>>> attribute called "memberIp anymore. It has been changed to defaultPrivateIP.
>>>
>>> Latest python agent has this changes.
>>>
>>> Thanks.
>>>
>>> On Mon, Dec 22, 2014 at 12:22 PM, Rajkumar Rajaratnam <
>>> rajkumarr@wso2.com> wrote:
>>>
>>>> Just noticed that there is an error in parsing complete topology event.
>>>> So initialization didn't happen as Isuru mentioned.
>>>>
>>>> Thanks.
>>>>
>>>> On Mon, Dec 22, 2014 at 12:19 PM, Rajkumar Rajaratnam <
>>>> rajkumarr@wso2.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> AFAIR, agent will wait for two conditions to happen. It will proceed
>>>>> if either of two conditions becomes true.
>>>>>
>>>>> 1. Member should be in the topology
>>>>> 2. Agent should get instance spawned event from CC
>>>>>
>>>>> So either of them didn't happen for you.
>>>>>
>>>>> Thanks.
>>>>>
>>>>> On Mon, Dec 22, 2014 at 12:17 PM, Isuru Haththotuwa <isuruh@apache.org
>>>>> > wrote:
>>>>>
>>>>>> HI Udara,
>>>>>>
>>>>>> There is an error log about parsing the complete Topology event.
>>>>>> Maybe the event did not get parsed correctly, and the required
>>>>>> initialization did not happen.
>>>>>>
>>>>>> On Mon, Dec 22, 2014 at 12:12 PM, Udara Liyanage <ud...@wso2.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Chamila,
>>>>>>>
>>>>>>> I get the ":Waiting for Cartridge Agent to be initialized..." logs
>>>>>>> continuously even though complete topology and complete tenant event is
>>>>>>> already received.
>>>>>>>
>>>>>>>
>>>>>>> 2014-12-22 12:04:47,595:DEBUG:Waiting for Cartridge Agent to be
>>>>>>> initialized...
>>>>>>> 2014-12-22 12:04:48,596:DEBUG:Waiting for Cartridge Agent to be
>>>>>>> initialized...
>>>>>>> 2014-12-22 12:04:49,597:DEBUG:Waiting for Cartridge Agent to be
>>>>>>> initialized...
>>>>>>> 2014-12-22 12:04:49,781:DEBUG:Message received:
>>>>>>> 'topology/CompleteTopologyEvent':
>>>>>>>
>>>>>>> '{"topology":{"serviceMap":{"tomcat":{"serviceName":"tomcat","serviceType":"SingleTenant","clusterIdClusterMap":{"mytomcat.tomcat.domain":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","autoscalePolicyName":"autoscale_policy_1","hostNames":["
>>>>>>> mytomcat.isuruh.lk
>>>>>>> "],"tenantRange":"*","isLbCluster":false,"isKubernetesCluster":false,"memberMap":{"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","memberId":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101","instanceId":"ap-southeast-1/i-df7dde12","clusterInstanceId":"app_cartridge_v1-1","networkPartitionId":"p1","partitionId":"P1","initTime":1419229605140,"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"memberPublicIPs":["54.151.190.32"],"defaultPublicIP":"54.151.190.32","memberPrivateIPs":["10.144.68.9"],"defaultPrivateIP":"10.144.68.9","properties":{"PRIMARY":"false","MIN_COUNT":"1"},"memberStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101"}}},"appId":"app_cartridge_v1","properties":{},"instanceIdToInstanceContextMap":{"app_cartridge_v1-1":{"instanceId":"app_cartridge_v1-1","instanceProperties":{},"lifeCycleStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domain_app_cartridge_v1-1"},"networkPartitionId":"p1"}}}},"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"properties":{}}},"initialized":false}}'
>>>>>>> 2014-12-22 12:04:49,781:DEBUG:Executing handler for event
>>>>>>> 'CompleteTopologyEvent'
>>>>>>> 2014-12-22 12:04:49,781:DEBUG:Complete topology event received
>>>>>>> 2014-12-22 12:04:49,782:ERROR:Error processing
>>>>>>> 'CompleteTopologyEvent' event
>>>>>>> Traceback (most recent call last):
>>>>>>> File
>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>>>>>> line 81, in on_message
>>>>>>> handler(msg)
>>>>>>> File "agent.py", line 255, in on_complete_topology
>>>>>>> event_obj = CompleteTopologyEvent.create_from_json(msg.payload)
>>>>>>> File
>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/event/topology/events.py",
>>>>>>> line 190, in create_from_json
>>>>>>> member_obj.member_ip = member_str["memberIp"]
>>>>>>> KeyError: 'memberIp'
>>>>>>> 2014-12-22 12:04:49,820:DEBUG:Message received:
>>>>>>> 'tenant/CompleteTenantEvent':
>>>>>>> '{"tenants":[]}'
>>>>>>> 2014-12-22 12:04:49,820:DEBUG:Executing handler for event
>>>>>>> 'CompleteTenantEvent'
>>>>>>> 2014-12-22 12:04:49,820:INFO:Complete tenant event updating task
>>>>>>> disabled
>>>>>>>
>>>>>>>
>>>>>>> *Carbon logs:*
>>>>>>>
>>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,240] INFO
>>>>>>> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
>>>>>>> - Publishing instance spawned event: [service-name] tomcat [cluster-id]
>>>>>>> mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
>>>>>>> mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 [instance-id]
>>>>>>> ap-southeast-1/i-df7dde12 [network-partition-id] p1 [partition-id] P1
>>>>>>> [lb-cluster-id] null
>>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,261] INFO
>>>>>>> {org.apache.stratos.messaging.domain.topology.lifecycle.LifeCycleStateManager}
>>>>>>> - Life Cycle State Manager started for Element [
>>>>>>> mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 ], initial
>>>>>>> state: Created
>>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,264] INFO
>>>>>>> {org.apache.stratos.messaging.message.processor.topology.InstanceSpawnedMessageProcessor}
>>>>>>> - Member created: [service] tomcat [cluster] mytomcat.tomcat.domain
>>>>>>> [member] mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101
>>>>>>> [instance] ap-southeast-1/i-df7dde12 [hierarchy-instance] app_cartridge_v1-1
>>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,268] INFO
>>>>>>> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
>>>>>>> - [InstanceSpawnedEventListener] Received: class
>>>>>>> org.apache.stratos.messaging.event.topology.InstanceSpawnedEvent
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Udara Liyanage
>>>>>>> Software Engineer
>>>>>>> WSO2, Inc.: http://wso2.com
>>>>>>> lean. enterprise. middleware
>>>>>>>
>>>>>>> web: http://udaraliyanage.wordpress.com
>>>>>>> phone:
>>>>>>> +94 71 443 6897
>>>>>>>
>>>>>>> --
>>>>>>> Thanks and Regards,
>>>>>>>
>>>>>>> Isuru H.
>>>>>>> +94 716 358 048* <http://wso2.com/>*
>>>>>>>
>>>>>>>
>>>>>>> * <http://wso2.com/>*
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Rajkumar Rajaratnam
>>>>> Committer & PMC Member, Apache Stratos
>>>>> Software Engineer, WSO2
>>>>>
>>>>> Mobile : +94777568639
>>>>> Blog : rajkumarr.com
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Rajkumar Rajaratnam
>>>> Committer & PMC Member, Apache Stratos
>>>> Software Engineer, WSO2
>>>>
>>>> Mobile : +94777568639
>>>> Blog : rajkumarr.com
>>>>
>>>
>>>
>>>
>>> --
>>> Rajkumar Rajaratnam
>>> Committer & PMC Member, Apache Stratos
>>> Software Engineer, WSO2
>>>
>>> Mobile : +94777568639
>>> Blog : rajkumarr.com
>>>
>>
>>
>> --
>>
>> Udara Liyanage
>> Software Engineer
>> WSO2, Inc.: http://wso2.com
>> lean. enterprise. middleware
>>
>> web: http://udaraliyanage.wordpress.com
>> phone: +94 71 443 6897
>>
>
>
> --
>
> Udara Liyanage
> Software Engineer
> WSO2, Inc.: http://wso2.com
> lean. enterprise. middleware
>
> web: http://udaraliyanage.wordpress.com
> phone: +94 71 443 6897
>
--
Rajkumar Rajaratnam
Committer & PMC Member, Apache Stratos
Software Engineer, WSO2
Mobile : +94777568639
Blog : rajkumarr.com
Re: Why agent wait to be intialized even after complete topology is received
Posted by Udara Liyanage <ud...@wso2.com>.
Hi raj,
I tested with lates python code base. This time member is in topology and
instance spawned event is also received. But yet instance activated is not
sent.
*1) Instance started event received*
2014-12-22 12:53:49,904:DEBUG:Property added: [key] STRATOS_
MEMBER_PROPERTY_PRIMARY[value] false
2014-12-22 12:53:49,911:DEBUG:Message received:
'instance/status/InstanceStartedEvent':
'{\n "clusterId": "mytomcat.tomcat.domain", \n "clusterInstanceId":
"app_cartridge_v1-1", \n "instanceId": "", \n "memberId":
"mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130", \n
"networkPartitionId": "p1", \n "partitionId": "P1", \n
"serviceName": "tomcat"\n}'
2) Member in topology
*Complete Topology:*
'{"topology":{"serviceMap":{"tomcat":{"serviceName":"tomcat","serviceType":"SingleTenant","clusterIdClusterMap":{"mytomcat.tomcat.domain":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","autoscalePolicyName":"autoscale_policy_1","hostNames":["
mytomcat.isuruh.lk
"],"tenantRange":"*","isLbCluster":false,"isKubernetesCluster":false,"memberMap":{"mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","memberId":"
*mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130*
","instanceId":"ap-southeast-1/i-0147e4cc","clusterInstanceId":"app_cartridge_v1-1","networkPartitionId":"p1","partitionId":"P1","initTime":1419232682397,"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"memberPublicIPs":["54.254.118.165"],"defaultPublicIP":"54.254.118.165","memberPrivateIPs":["10.144.150.199"],"defaultPrivateIP":"10.144.150.199","properties":{"PRIMARY":"false","MIN_COUNT":"1"},"memberStateManager":{"stateStack":["Created","Starting"],"identifier":"mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130"}}},"appId":"app_cartridge_v1","properties":{},"instanceIdToInstanceContextMap":{"app_cartridge_v1-1":{"instanceId":"app_cartridge_v1-1","instanceProperties":{},"lifeCycleStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domain_app_cartridge_v1-1"},"networkPartitionId":"p1"}}}},"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"properties":{}}},"initialized":false}}'
*Payload:*
APP_ID=app_cartridge_v1,GROUP_NAME=null,SERVICE_NAME=tomcat,HOST_NAME=
mytomcat.isuruh.lk
,MULTITENANT=false,TENANT_ID=-1234,TENANT_RANGE=*,CARTRIDGE_ALIAS=mytomcat,CLUSTER_ID=mytomcat.tomcat.domain,CARTRIDGE_KEY=kYcZ4B4DPAeA1RGz,REPO_URL=
https://github.com/udaraliyanage/php,PORTS=80|443,PROVIDER=apache,PUPPET_IP=10.148.159.109,PUPPET_HOSTNAME=ip-10-148-159-109.ap-southeast-1.compute.internal,PUPPET_DNS_AVAILABLE=null,PUPPET_ENV=null,,,DEPLOYMENT=default,TOKEN=eyJhbGciOiJSUzI1NiJ9.eyJleHAiOjE4OTY4OTAwNDgsInN1YiI6ImFkbWluIiwiYXpwIjoiMjU1UUZET1UwbFlONmY5NE44aUVhQm5LdlBrYSIsImFwcElkIjoiYXBwX2NhcnRyaWRnZV92MSIsImF1ZCI6WyIyNTVRRkRPVTBsWU42Zjk0TjhpRWFCbkt2UGthIl0sImlzcyI6Imh0dHBzOlwvXC9sb2NhbGhvc3Q6OTQ0M1wvb2F1dGgyZW5kcG9pbnRzXC90b2tlbiIsImlhdCI6MTg5MzI5MDA0OH0.XB4wg8gQxzyH0jv5EIiWFjOG0VDZuFBswYG0G6zX3sjOEgXauywoga4pHmb9DWkKZU9nD5qRFLRyyiHi2onWr-TGtLNBn4qgRq94VDwjVgx5in4ND3f7q-xNSpLipMr_g_gtkbp9st-gZVeuB8y3u7jnjE8fuDEs63pQ75Drrhs,MEMBER_ID=
*mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130*
,INSTANCE_ID=null,CLUSTER_INSTANCE_ID=app_cartridge_v1-1,LB_CLUSTER_ID=null,NETWORK_PARTITION_ID=p1,PARTITION_ID=P1,PRIMARY=false,MIN_COUNT=1
Carbon logs:
[2014-12-22 07:18:41,444] INFO
{org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
- Publishing instance spawned event: [service-name] tomcat [cluster-id]
mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130 [instance-id]
ap-southeast-1/i-0147e4cc [network-partition-id] p1 [partition-id] P1
[lb-cluster-id] null
[2014-12-22 07:18:41,464] INFO
{org.apache.stratos.messaging.domain.topology.lifecycle.LifeCycleStateManager}
- Life Cycle State Manager started for Element [
mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130 ], initial
state: Created
[2014-12-22 07:18:41,469] INFO
{org.apache.stratos.messaging.message.processor.topology.InstanceSpawnedMessageProcessor}
- Member created: [service] tomcat [cluster] mytomcat.tomcat.domain
[member] mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130
[instance] ap-southeast-1/i-0147e4cc [hierarchy-instance] app_cartridge_v1-1
[2014-12-22 07:18:41,480] INFO
{org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
- [InstanceSpawnedEventListener] Received: class
org.apache.stratos.messaging.event.topology.InstanceSpawnedEvent
[2014-12-22 07:24:01,218] INFO
{org.apache.stratos.manager.listener.InstanceStatusListener} - Instance
status message received
[2014-12-22 07:24:01,218] INFO
{org.apache.stratos.manager.listener.InstanceStatusListener} - Event class
name:
org.apache.stratos.messaging.event.instance.status.InstanceStartedEvent
[2014-12-22 07:24:01,219] INFO
{org.apache.stratos.manager.listener.InstanceStatusListener} - Cluster id:
mytomcat.tomcat.domain
[2014-12-22 07:24:01,220] INFO
{org.apache.stratos.cloud.controller.messaging.topology.TopologyBuilder} -
member started event adding status started
[2014-12-22 07:24:01,235] INFO
{org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
- Publishing member started event: [service-name] tomcat [cluster-id]
mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130 [instance-id]
[network-partition-id] p1 [partition-id] P1
[2014-12-22 07:24:01,250] INFO
{org.apache.stratos.messaging.message.processor.topology.MemberStartedMessageProcessor}
- Member started: [service] tomcat [cluster] mytomcat.tomcat.domain
[member] mytomcat.tomcat.domain5dd544b1-d2d6-4ace-8e31-ecb07e116130
[2014-12-22 07:24:01,250] INFO
{org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
- [MemberStartedEventListener] Received: class
org.apache.stratos.messaging.event.topology.MemberStartedEvent
On Mon, Dec 22, 2014 at 12:26 PM, Udara Liyanage <ud...@wso2.com> wrote:
>
> Hi Raj,
>
> Thanks for the info. I will check with updated agent.
>
> On Mon, Dec 22, 2014 at 12:24 PM, Rajkumar Rajaratnam <ra...@wso2.com>
> wrote:
>>
>> Hi Udara,
>>
>> Are you using the latest python agent?
>>
>> Chamila changed the agent yesterday. Member object doesn't have an
>> attribute called "memberIp anymore. It has been changed to defaultPrivateIP.
>>
>> Latest python agent has this changes.
>>
>> Thanks.
>>
>> On Mon, Dec 22, 2014 at 12:22 PM, Rajkumar Rajaratnam <rajkumarr@wso2.com
>> > wrote:
>>
>>> Just noticed that there is an error in parsing complete topology event.
>>> So initialization didn't happen as Isuru mentioned.
>>>
>>> Thanks.
>>>
>>> On Mon, Dec 22, 2014 at 12:19 PM, Rajkumar Rajaratnam <
>>> rajkumarr@wso2.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> AFAIR, agent will wait for two conditions to happen. It will proceed if
>>>> either of two conditions becomes true.
>>>>
>>>> 1. Member should be in the topology
>>>> 2. Agent should get instance spawned event from CC
>>>>
>>>> So either of them didn't happen for you.
>>>>
>>>> Thanks.
>>>>
>>>> On Mon, Dec 22, 2014 at 12:17 PM, Isuru Haththotuwa <is...@apache.org>
>>>> wrote:
>>>>
>>>>> HI Udara,
>>>>>
>>>>> There is an error log about parsing the complete Topology event. Maybe
>>>>> the event did not get parsed correctly, and the required initialization did
>>>>> not happen.
>>>>>
>>>>> On Mon, Dec 22, 2014 at 12:12 PM, Udara Liyanage <ud...@wso2.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Chamila,
>>>>>>
>>>>>> I get the ":Waiting for Cartridge Agent to be initialized..." logs
>>>>>> continuously even though complete topology and complete tenant event is
>>>>>> already received.
>>>>>>
>>>>>>
>>>>>> 2014-12-22 12:04:47,595:DEBUG:Waiting for Cartridge Agent to be
>>>>>> initialized...
>>>>>> 2014-12-22 12:04:48,596:DEBUG:Waiting for Cartridge Agent to be
>>>>>> initialized...
>>>>>> 2014-12-22 12:04:49,597:DEBUG:Waiting for Cartridge Agent to be
>>>>>> initialized...
>>>>>> 2014-12-22 12:04:49,781:DEBUG:Message received:
>>>>>> 'topology/CompleteTopologyEvent':
>>>>>>
>>>>>> '{"topology":{"serviceMap":{"tomcat":{"serviceName":"tomcat","serviceType":"SingleTenant","clusterIdClusterMap":{"mytomcat.tomcat.domain":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","autoscalePolicyName":"autoscale_policy_1","hostNames":["
>>>>>> mytomcat.isuruh.lk
>>>>>> "],"tenantRange":"*","isLbCluster":false,"isKubernetesCluster":false,"memberMap":{"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","memberId":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101","instanceId":"ap-southeast-1/i-df7dde12","clusterInstanceId":"app_cartridge_v1-1","networkPartitionId":"p1","partitionId":"P1","initTime":1419229605140,"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"memberPublicIPs":["54.151.190.32"],"defaultPublicIP":"54.151.190.32","memberPrivateIPs":["10.144.68.9"],"defaultPrivateIP":"10.144.68.9","properties":{"PRIMARY":"false","MIN_COUNT":"1"},"memberStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101"}}},"appId":"app_cartridge_v1","properties":{},"instanceIdToInstanceContextMap":{"app_cartridge_v1-1":{"instanceId":"app_cartridge_v1-1","instanceProperties":{},"lifeCycleStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domain_app_cartridge_v1-1"},"networkPartitionId":"p1"}}}},"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"properties":{}}},"initialized":false}}'
>>>>>> 2014-12-22 12:04:49,781:DEBUG:Executing handler for event
>>>>>> 'CompleteTopologyEvent'
>>>>>> 2014-12-22 12:04:49,781:DEBUG:Complete topology event received
>>>>>> 2014-12-22 12:04:49,782:ERROR:Error processing
>>>>>> 'CompleteTopologyEvent' event
>>>>>> Traceback (most recent call last):
>>>>>> File
>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>>>>> line 81, in on_message
>>>>>> handler(msg)
>>>>>> File "agent.py", line 255, in on_complete_topology
>>>>>> event_obj = CompleteTopologyEvent.create_from_json(msg.payload)
>>>>>> File
>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/event/topology/events.py",
>>>>>> line 190, in create_from_json
>>>>>> member_obj.member_ip = member_str["memberIp"]
>>>>>> KeyError: 'memberIp'
>>>>>> 2014-12-22 12:04:49,820:DEBUG:Message received:
>>>>>> 'tenant/CompleteTenantEvent':
>>>>>> '{"tenants":[]}'
>>>>>> 2014-12-22 12:04:49,820:DEBUG:Executing handler for event
>>>>>> 'CompleteTenantEvent'
>>>>>> 2014-12-22 12:04:49,820:INFO:Complete tenant event updating task
>>>>>> disabled
>>>>>>
>>>>>>
>>>>>> *Carbon logs:*
>>>>>>
>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,240] INFO
>>>>>> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
>>>>>> - Publishing instance spawned event: [service-name] tomcat [cluster-id]
>>>>>> mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
>>>>>> mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 [instance-id]
>>>>>> ap-southeast-1/i-df7dde12 [network-partition-id] p1 [partition-id] P1
>>>>>> [lb-cluster-id] null
>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,261] INFO
>>>>>> {org.apache.stratos.messaging.domain.topology.lifecycle.LifeCycleStateManager}
>>>>>> - Life Cycle State Manager started for Element [
>>>>>> mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 ], initial
>>>>>> state: Created
>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,264] INFO
>>>>>> {org.apache.stratos.messaging.message.processor.topology.InstanceSpawnedMessageProcessor}
>>>>>> - Member created: [service] tomcat [cluster] mytomcat.tomcat.domain
>>>>>> [member] mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101
>>>>>> [instance] ap-southeast-1/i-df7dde12 [hierarchy-instance] app_cartridge_v1-1
>>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,268] INFO
>>>>>> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
>>>>>> - [InstanceSpawnedEventListener] Received: class
>>>>>> org.apache.stratos.messaging.event.topology.InstanceSpawnedEvent
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Udara Liyanage
>>>>>> Software Engineer
>>>>>> WSO2, Inc.: http://wso2.com
>>>>>> lean. enterprise. middleware
>>>>>>
>>>>>> web: http://udaraliyanage.wordpress.com
>>>>>> phone:
>>>>>> +94 71 443 6897
>>>>>>
>>>>>> --
>>>>>> Thanks and Regards,
>>>>>>
>>>>>> Isuru H.
>>>>>> +94 716 358 048* <http://wso2.com/>*
>>>>>>
>>>>>>
>>>>>> * <http://wso2.com/>*
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>>> --
>>>> Rajkumar Rajaratnam
>>>> Committer & PMC Member, Apache Stratos
>>>> Software Engineer, WSO2
>>>>
>>>> Mobile : +94777568639
>>>> Blog : rajkumarr.com
>>>>
>>>
>>>
>>>
>>> --
>>> Rajkumar Rajaratnam
>>> Committer & PMC Member, Apache Stratos
>>> Software Engineer, WSO2
>>>
>>> Mobile : +94777568639
>>> Blog : rajkumarr.com
>>>
>>
>>
>>
>> --
>> Rajkumar Rajaratnam
>> Committer & PMC Member, Apache Stratos
>> Software Engineer, WSO2
>>
>> Mobile : +94777568639
>> Blog : rajkumarr.com
>>
>
>
> --
>
> Udara Liyanage
> Software Engineer
> WSO2, Inc.: http://wso2.com
> lean. enterprise. middleware
>
> web: http://udaraliyanage.wordpress.com
> phone: +94 71 443 6897
>
--
Udara Liyanage
Software Engineer
WSO2, Inc.: http://wso2.com
lean. enterprise. middleware
web: http://udaraliyanage.wordpress.com
phone: +94 71 443 6897
Re: Why agent wait to be intialized even after complete topology is received
Posted by Udara Liyanage <ud...@wso2.com>.
Hi Raj,
Thanks for the info. I will check with updated agent.
On Mon, Dec 22, 2014 at 12:24 PM, Rajkumar Rajaratnam <ra...@wso2.com>
wrote:
>
> Hi Udara,
>
> Are you using the latest python agent?
>
> Chamila changed the agent yesterday. Member object doesn't have an
> attribute called "memberIp anymore. It has been changed to defaultPrivateIP.
>
> Latest python agent has this changes.
>
> Thanks.
>
> On Mon, Dec 22, 2014 at 12:22 PM, Rajkumar Rajaratnam <ra...@wso2.com>
> wrote:
>
>> Just noticed that there is an error in parsing complete topology event.
>> So initialization didn't happen as Isuru mentioned.
>>
>> Thanks.
>>
>> On Mon, Dec 22, 2014 at 12:19 PM, Rajkumar Rajaratnam <rajkumarr@wso2.com
>> > wrote:
>>
>>> Hi,
>>>
>>> AFAIR, agent will wait for two conditions to happen. It will proceed if
>>> either of two conditions becomes true.
>>>
>>> 1. Member should be in the topology
>>> 2. Agent should get instance spawned event from CC
>>>
>>> So either of them didn't happen for you.
>>>
>>> Thanks.
>>>
>>> On Mon, Dec 22, 2014 at 12:17 PM, Isuru Haththotuwa <is...@apache.org>
>>> wrote:
>>>
>>>> HI Udara,
>>>>
>>>> There is an error log about parsing the complete Topology event. Maybe
>>>> the event did not get parsed correctly, and the required initialization did
>>>> not happen.
>>>>
>>>> On Mon, Dec 22, 2014 at 12:12 PM, Udara Liyanage <ud...@wso2.com>
>>>> wrote:
>>>>
>>>>> Hi Chamila,
>>>>>
>>>>> I get the ":Waiting for Cartridge Agent to be initialized..." logs
>>>>> continuously even though complete topology and complete tenant event is
>>>>> already received.
>>>>>
>>>>>
>>>>> 2014-12-22 12:04:47,595:DEBUG:Waiting for Cartridge Agent to be
>>>>> initialized...
>>>>> 2014-12-22 12:04:48,596:DEBUG:Waiting for Cartridge Agent to be
>>>>> initialized...
>>>>> 2014-12-22 12:04:49,597:DEBUG:Waiting for Cartridge Agent to be
>>>>> initialized...
>>>>> 2014-12-22 12:04:49,781:DEBUG:Message received:
>>>>> 'topology/CompleteTopologyEvent':
>>>>>
>>>>> '{"topology":{"serviceMap":{"tomcat":{"serviceName":"tomcat","serviceType":"SingleTenant","clusterIdClusterMap":{"mytomcat.tomcat.domain":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","autoscalePolicyName":"autoscale_policy_1","hostNames":["
>>>>> mytomcat.isuruh.lk
>>>>> "],"tenantRange":"*","isLbCluster":false,"isKubernetesCluster":false,"memberMap":{"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","memberId":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101","instanceId":"ap-southeast-1/i-df7dde12","clusterInstanceId":"app_cartridge_v1-1","networkPartitionId":"p1","partitionId":"P1","initTime":1419229605140,"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"memberPublicIPs":["54.151.190.32"],"defaultPublicIP":"54.151.190.32","memberPrivateIPs":["10.144.68.9"],"defaultPrivateIP":"10.144.68.9","properties":{"PRIMARY":"false","MIN_COUNT":"1"},"memberStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101"}}},"appId":"app_cartridge_v1","properties":{},"instanceIdToInstanceContextMap":{"app_cartridge_v1-1":{"instanceId":"app_cartridge_v1-1","instanceProperties":{},"lifeCycleStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domain_app_cartridge_v1-1"},"networkPartitionId":"p1"}}}},"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"properties":{}}},"initialized":false}}'
>>>>> 2014-12-22 12:04:49,781:DEBUG:Executing handler for event
>>>>> 'CompleteTopologyEvent'
>>>>> 2014-12-22 12:04:49,781:DEBUG:Complete topology event received
>>>>> 2014-12-22 12:04:49,782:ERROR:Error processing 'CompleteTopologyEvent'
>>>>> event
>>>>> Traceback (most recent call last):
>>>>> File
>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>>>> line 81, in on_message
>>>>> handler(msg)
>>>>> File "agent.py", line 255, in on_complete_topology
>>>>> event_obj = CompleteTopologyEvent.create_from_json(msg.payload)
>>>>> File
>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/event/topology/events.py",
>>>>> line 190, in create_from_json
>>>>> member_obj.member_ip = member_str["memberIp"]
>>>>> KeyError: 'memberIp'
>>>>> 2014-12-22 12:04:49,820:DEBUG:Message received:
>>>>> 'tenant/CompleteTenantEvent':
>>>>> '{"tenants":[]}'
>>>>> 2014-12-22 12:04:49,820:DEBUG:Executing handler for event
>>>>> 'CompleteTenantEvent'
>>>>> 2014-12-22 12:04:49,820:INFO:Complete tenant event updating task
>>>>> disabled
>>>>>
>>>>>
>>>>> *Carbon logs:*
>>>>>
>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,240] INFO
>>>>> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
>>>>> - Publishing instance spawned event: [service-name] tomcat [cluster-id]
>>>>> mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
>>>>> mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 [instance-id]
>>>>> ap-southeast-1/i-df7dde12 [network-partition-id] p1 [partition-id] P1
>>>>> [lb-cluster-id] null
>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,261] INFO
>>>>> {org.apache.stratos.messaging.domain.topology.lifecycle.LifeCycleStateManager}
>>>>> - Life Cycle State Manager started for Element [
>>>>> mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 ], initial
>>>>> state: Created
>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,264] INFO
>>>>> {org.apache.stratos.messaging.message.processor.topology.InstanceSpawnedMessageProcessor}
>>>>> - Member created: [service] tomcat [cluster] mytomcat.tomcat.domain
>>>>> [member] mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101
>>>>> [instance] ap-southeast-1/i-df7dde12 [hierarchy-instance] app_cartridge_v1-1
>>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,268] INFO
>>>>> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
>>>>> - [InstanceSpawnedEventListener] Received: class
>>>>> org.apache.stratos.messaging.event.topology.InstanceSpawnedEvent
>>>>>
>>>>> --
>>>>>
>>>>> Udara Liyanage
>>>>> Software Engineer
>>>>> WSO2, Inc.: http://wso2.com
>>>>> lean. enterprise. middleware
>>>>>
>>>>> web: http://udaraliyanage.wordpress.com
>>>>> phone:
>>>>> +94 71 443 6897
>>>>>
>>>>> --
>>>>> Thanks and Regards,
>>>>>
>>>>> Isuru H.
>>>>> +94 716 358 048* <http://wso2.com/>*
>>>>>
>>>>>
>>>>> * <http://wso2.com/>*
>>>>>
>>>>>
>>>>>
>>>
>>>
>>> --
>>> Rajkumar Rajaratnam
>>> Committer & PMC Member, Apache Stratos
>>> Software Engineer, WSO2
>>>
>>> Mobile : +94777568639
>>> Blog : rajkumarr.com
>>>
>>
>>
>>
>> --
>> Rajkumar Rajaratnam
>> Committer & PMC Member, Apache Stratos
>> Software Engineer, WSO2
>>
>> Mobile : +94777568639
>> Blog : rajkumarr.com
>>
>
>
>
> --
> Rajkumar Rajaratnam
> Committer & PMC Member, Apache Stratos
> Software Engineer, WSO2
>
> Mobile : +94777568639
> Blog : rajkumarr.com
>
--
Udara Liyanage
Software Engineer
WSO2, Inc.: http://wso2.com
lean. enterprise. middleware
web: http://udaraliyanage.wordpress.com
phone: +94 71 443 6897
Re: Why agent wait to be intialized even after complete topology is received
Posted by Rajkumar Rajaratnam <ra...@wso2.com>.
Hi Udara,
Are you using the latest python agent?
Chamila changed the agent yesterday. Member object doesn't have an
attribute called "memberIp anymore. It has been changed to defaultPrivateIP.
Latest python agent has this changes.
Thanks.
On Mon, Dec 22, 2014 at 12:22 PM, Rajkumar Rajaratnam <ra...@wso2.com>
wrote:
> Just noticed that there is an error in parsing complete topology event. So
> initialization didn't happen as Isuru mentioned.
>
> Thanks.
>
> On Mon, Dec 22, 2014 at 12:19 PM, Rajkumar Rajaratnam <ra...@wso2.com>
> wrote:
>
>> Hi,
>>
>> AFAIR, agent will wait for two conditions to happen. It will proceed if
>> either of two conditions becomes true.
>>
>> 1. Member should be in the topology
>> 2. Agent should get instance spawned event from CC
>>
>> So either of them didn't happen for you.
>>
>> Thanks.
>>
>> On Mon, Dec 22, 2014 at 12:17 PM, Isuru Haththotuwa <is...@apache.org>
>> wrote:
>>
>>> HI Udara,
>>>
>>> There is an error log about parsing the complete Topology event. Maybe
>>> the event did not get parsed correctly, and the required initialization did
>>> not happen.
>>>
>>> On Mon, Dec 22, 2014 at 12:12 PM, Udara Liyanage <ud...@wso2.com> wrote:
>>>
>>>> Hi Chamila,
>>>>
>>>> I get the ":Waiting for Cartridge Agent to be initialized..." logs
>>>> continuously even though complete topology and complete tenant event is
>>>> already received.
>>>>
>>>>
>>>> 2014-12-22 12:04:47,595:DEBUG:Waiting for Cartridge Agent to be
>>>> initialized...
>>>> 2014-12-22 12:04:48,596:DEBUG:Waiting for Cartridge Agent to be
>>>> initialized...
>>>> 2014-12-22 12:04:49,597:DEBUG:Waiting for Cartridge Agent to be
>>>> initialized...
>>>> 2014-12-22 12:04:49,781:DEBUG:Message received:
>>>> 'topology/CompleteTopologyEvent':
>>>>
>>>> '{"topology":{"serviceMap":{"tomcat":{"serviceName":"tomcat","serviceType":"SingleTenant","clusterIdClusterMap":{"mytomcat.tomcat.domain":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","autoscalePolicyName":"autoscale_policy_1","hostNames":["
>>>> mytomcat.isuruh.lk
>>>> "],"tenantRange":"*","isLbCluster":false,"isKubernetesCluster":false,"memberMap":{"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","memberId":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101","instanceId":"ap-southeast-1/i-df7dde12","clusterInstanceId":"app_cartridge_v1-1","networkPartitionId":"p1","partitionId":"P1","initTime":1419229605140,"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"memberPublicIPs":["54.151.190.32"],"defaultPublicIP":"54.151.190.32","memberPrivateIPs":["10.144.68.9"],"defaultPrivateIP":"10.144.68.9","properties":{"PRIMARY":"false","MIN_COUNT":"1"},"memberStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101"}}},"appId":"app_cartridge_v1","properties":{},"instanceIdToInstanceContextMap":{"app_cartridge_v1-1":{"instanceId":"app_cartridge_v1-1","instanceProperties":{},"lifeCycleStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domain_app_cartridge_v1-1"},"networkPartitionId":"p1"}}}},"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"properties":{}}},"initialized":false}}'
>>>> 2014-12-22 12:04:49,781:DEBUG:Executing handler for event
>>>> 'CompleteTopologyEvent'
>>>> 2014-12-22 12:04:49,781:DEBUG:Complete topology event received
>>>> 2014-12-22 12:04:49,782:ERROR:Error processing 'CompleteTopologyEvent'
>>>> event
>>>> Traceback (most recent call last):
>>>> File
>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>>> line 81, in on_message
>>>> handler(msg)
>>>> File "agent.py", line 255, in on_complete_topology
>>>> event_obj = CompleteTopologyEvent.create_from_json(msg.payload)
>>>> File
>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/event/topology/events.py",
>>>> line 190, in create_from_json
>>>> member_obj.member_ip = member_str["memberIp"]
>>>> KeyError: 'memberIp'
>>>> 2014-12-22 12:04:49,820:DEBUG:Message received:
>>>> 'tenant/CompleteTenantEvent':
>>>> '{"tenants":[]}'
>>>> 2014-12-22 12:04:49,820:DEBUG:Executing handler for event
>>>> 'CompleteTenantEvent'
>>>> 2014-12-22 12:04:49,820:INFO:Complete tenant event updating task
>>>> disabled
>>>>
>>>>
>>>> *Carbon logs:*
>>>>
>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,240] INFO
>>>> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
>>>> - Publishing instance spawned event: [service-name] tomcat [cluster-id]
>>>> mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
>>>> mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 [instance-id]
>>>> ap-southeast-1/i-df7dde12 [network-partition-id] p1 [partition-id] P1
>>>> [lb-cluster-id] null
>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,261] INFO
>>>> {org.apache.stratos.messaging.domain.topology.lifecycle.LifeCycleStateManager}
>>>> - Life Cycle State Manager started for Element [
>>>> mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 ], initial
>>>> state: Created
>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,264] INFO
>>>> {org.apache.stratos.messaging.message.processor.topology.InstanceSpawnedMessageProcessor}
>>>> - Member created: [service] tomcat [cluster] mytomcat.tomcat.domain
>>>> [member] mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101
>>>> [instance] ap-southeast-1/i-df7dde12 [hierarchy-instance] app_cartridge_v1-1
>>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,268] INFO
>>>> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
>>>> - [InstanceSpawnedEventListener] Received: class
>>>> org.apache.stratos.messaging.event.topology.InstanceSpawnedEvent
>>>>
>>>> --
>>>>
>>>> Udara Liyanage
>>>> Software Engineer
>>>> WSO2, Inc.: http://wso2.com
>>>> lean. enterprise. middleware
>>>>
>>>> web: http://udaraliyanage.wordpress.com
>>>> phone:
>>>> +94 71 443 6897
>>>>
>>>> --
>>>> Thanks and Regards,
>>>>
>>>> Isuru H.
>>>> +94 716 358 048* <http://wso2.com/>*
>>>>
>>>>
>>>> * <http://wso2.com/>*
>>>>
>>>>
>>>>
>>
>>
>> --
>> Rajkumar Rajaratnam
>> Committer & PMC Member, Apache Stratos
>> Software Engineer, WSO2
>>
>> Mobile : +94777568639
>> Blog : rajkumarr.com
>>
>
>
>
> --
> Rajkumar Rajaratnam
> Committer & PMC Member, Apache Stratos
> Software Engineer, WSO2
>
> Mobile : +94777568639
> Blog : rajkumarr.com
>
--
Rajkumar Rajaratnam
Committer & PMC Member, Apache Stratos
Software Engineer, WSO2
Mobile : +94777568639
Blog : rajkumarr.com
Re: Why agent wait to be intialized even after complete topology is received
Posted by Rajkumar Rajaratnam <ra...@wso2.com>.
Just noticed that there is an error in parsing complete topology event. So
initialization didn't happen as Isuru mentioned.
Thanks.
On Mon, Dec 22, 2014 at 12:19 PM, Rajkumar Rajaratnam <ra...@wso2.com>
wrote:
> Hi,
>
> AFAIR, agent will wait for two conditions to happen. It will proceed if
> either of two conditions becomes true.
>
> 1. Member should be in the topology
> 2. Agent should get instance spawned event from CC
>
> So either of them didn't happen for you.
>
> Thanks.
>
> On Mon, Dec 22, 2014 at 12:17 PM, Isuru Haththotuwa <is...@apache.org>
> wrote:
>
>> HI Udara,
>>
>> There is an error log about parsing the complete Topology event. Maybe
>> the event did not get parsed correctly, and the required initialization did
>> not happen.
>>
>> On Mon, Dec 22, 2014 at 12:12 PM, Udara Liyanage <ud...@wso2.com> wrote:
>>
>>> Hi Chamila,
>>>
>>> I get the ":Waiting for Cartridge Agent to be initialized..." logs
>>> continuously even though complete topology and complete tenant event is
>>> already received.
>>>
>>>
>>> 2014-12-22 12:04:47,595:DEBUG:Waiting for Cartridge Agent to be
>>> initialized...
>>> 2014-12-22 12:04:48,596:DEBUG:Waiting for Cartridge Agent to be
>>> initialized...
>>> 2014-12-22 12:04:49,597:DEBUG:Waiting for Cartridge Agent to be
>>> initialized...
>>> 2014-12-22 12:04:49,781:DEBUG:Message received:
>>> 'topology/CompleteTopologyEvent':
>>>
>>> '{"topology":{"serviceMap":{"tomcat":{"serviceName":"tomcat","serviceType":"SingleTenant","clusterIdClusterMap":{"mytomcat.tomcat.domain":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","autoscalePolicyName":"autoscale_policy_1","hostNames":["
>>> mytomcat.isuruh.lk
>>> "],"tenantRange":"*","isLbCluster":false,"isKubernetesCluster":false,"memberMap":{"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","memberId":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101","instanceId":"ap-southeast-1/i-df7dde12","clusterInstanceId":"app_cartridge_v1-1","networkPartitionId":"p1","partitionId":"P1","initTime":1419229605140,"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"memberPublicIPs":["54.151.190.32"],"defaultPublicIP":"54.151.190.32","memberPrivateIPs":["10.144.68.9"],"defaultPrivateIP":"10.144.68.9","properties":{"PRIMARY":"false","MIN_COUNT":"1"},"memberStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101"}}},"appId":"app_cartridge_v1","properties":{},"instanceIdToInstanceContextMap":{"app_cartridge_v1-1":{"instanceId":"app_cartridge_v1-1","instanceProperties":{},"lifeCycleStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domain_app_cartridge_v1-1"},"networkPartitionId":"p1"}}}},"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"properties":{}}},"initialized":false}}'
>>> 2014-12-22 12:04:49,781:DEBUG:Executing handler for event
>>> 'CompleteTopologyEvent'
>>> 2014-12-22 12:04:49,781:DEBUG:Complete topology event received
>>> 2014-12-22 12:04:49,782:ERROR:Error processing 'CompleteTopologyEvent'
>>> event
>>> Traceback (most recent call last):
>>> File
>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>> line 81, in on_message
>>> handler(msg)
>>> File "agent.py", line 255, in on_complete_topology
>>> event_obj = CompleteTopologyEvent.create_from_json(msg.payload)
>>> File
>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/event/topology/events.py",
>>> line 190, in create_from_json
>>> member_obj.member_ip = member_str["memberIp"]
>>> KeyError: 'memberIp'
>>> 2014-12-22 12:04:49,820:DEBUG:Message received:
>>> 'tenant/CompleteTenantEvent':
>>> '{"tenants":[]}'
>>> 2014-12-22 12:04:49,820:DEBUG:Executing handler for event
>>> 'CompleteTenantEvent'
>>> 2014-12-22 12:04:49,820:INFO:Complete tenant event updating task disabled
>>>
>>>
>>> *Carbon logs:*
>>>
>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,240] INFO
>>> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
>>> - Publishing instance spawned event: [service-name] tomcat [cluster-id]
>>> mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
>>> mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 [instance-id]
>>> ap-southeast-1/i-df7dde12 [network-partition-id] p1 [partition-id] P1
>>> [lb-cluster-id] null
>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,261] INFO
>>> {org.apache.stratos.messaging.domain.topology.lifecycle.LifeCycleStateManager}
>>> - Life Cycle State Manager started for Element [
>>> mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 ], initial
>>> state: Created
>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,264] INFO
>>> {org.apache.stratos.messaging.message.processor.topology.InstanceSpawnedMessageProcessor}
>>> - Member created: [service] tomcat [cluster] mytomcat.tomcat.domain
>>> [member] mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101
>>> [instance] ap-southeast-1/i-df7dde12 [hierarchy-instance] app_cartridge_v1-1
>>> TID: [0] [STRATOS] [2014-12-22 06:27:21,268] INFO
>>> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
>>> - [InstanceSpawnedEventListener] Received: class
>>> org.apache.stratos.messaging.event.topology.InstanceSpawnedEvent
>>>
>>> --
>>>
>>> Udara Liyanage
>>> Software Engineer
>>> WSO2, Inc.: http://wso2.com
>>> lean. enterprise. middleware
>>>
>>> web: http://udaraliyanage.wordpress.com
>>> phone:
>>> +94 71 443 6897
>>>
>>> --
>>> Thanks and Regards,
>>>
>>> Isuru H.
>>> +94 716 358 048* <http://wso2.com/>*
>>>
>>>
>>> * <http://wso2.com/>*
>>>
>>>
>>>
>
>
> --
> Rajkumar Rajaratnam
> Committer & PMC Member, Apache Stratos
> Software Engineer, WSO2
>
> Mobile : +94777568639
> Blog : rajkumarr.com
>
--
Rajkumar Rajaratnam
Committer & PMC Member, Apache Stratos
Software Engineer, WSO2
Mobile : +94777568639
Blog : rajkumarr.com
Re: Why agent wait to be intialized even after complete topology is received
Posted by Rajkumar Rajaratnam <ra...@wso2.com>.
Hi,
AFAIR, agent will wait for two conditions to happen. It will proceed if
either of two conditions becomes true.
1. Member should be in the topology
2. Agent should get instance spawned event from CC
So either of them didn't happen for you.
Thanks.
On Mon, Dec 22, 2014 at 12:17 PM, Isuru Haththotuwa <is...@apache.org>
wrote:
> HI Udara,
>
> There is an error log about parsing the complete Topology event. Maybe the
> event did not get parsed correctly, and the required initialization did not
> happen.
>
> On Mon, Dec 22, 2014 at 12:12 PM, Udara Liyanage <ud...@wso2.com> wrote:
>
>> Hi Chamila,
>>
>> I get the ":Waiting for Cartridge Agent to be initialized..." logs
>> continuously even though complete topology and complete tenant event is
>> already received.
>>
>>
>> 2014-12-22 12:04:47,595:DEBUG:Waiting for Cartridge Agent to be
>> initialized...
>> 2014-12-22 12:04:48,596:DEBUG:Waiting for Cartridge Agent to be
>> initialized...
>> 2014-12-22 12:04:49,597:DEBUG:Waiting for Cartridge Agent to be
>> initialized...
>> 2014-12-22 12:04:49,781:DEBUG:Message received:
>> 'topology/CompleteTopologyEvent':
>>
>> '{"topology":{"serviceMap":{"tomcat":{"serviceName":"tomcat","serviceType":"SingleTenant","clusterIdClusterMap":{"mytomcat.tomcat.domain":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","autoscalePolicyName":"autoscale_policy_1","hostNames":["
>> mytomcat.isuruh.lk
>> "],"tenantRange":"*","isLbCluster":false,"isKubernetesCluster":false,"memberMap":{"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","memberId":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101","instanceId":"ap-southeast-1/i-df7dde12","clusterInstanceId":"app_cartridge_v1-1","networkPartitionId":"p1","partitionId":"P1","initTime":1419229605140,"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"memberPublicIPs":["54.151.190.32"],"defaultPublicIP":"54.151.190.32","memberPrivateIPs":["10.144.68.9"],"defaultPrivateIP":"10.144.68.9","properties":{"PRIMARY":"false","MIN_COUNT":"1"},"memberStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101"}}},"appId":"app_cartridge_v1","properties":{},"instanceIdToInstanceContextMap":{"app_cartridge_v1-1":{"instanceId":"app_cartridge_v1-1","instanceProperties":{},"lifeCycleStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domain_app_cartridge_v1-1"},"networkPartitionId":"p1"}}}},"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"properties":{}}},"initialized":false}}'
>> 2014-12-22 12:04:49,781:DEBUG:Executing handler for event
>> 'CompleteTopologyEvent'
>> 2014-12-22 12:04:49,781:DEBUG:Complete topology event received
>> 2014-12-22 12:04:49,782:ERROR:Error processing 'CompleteTopologyEvent'
>> event
>> Traceback (most recent call last):
>> File
>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>> line 81, in on_message
>> handler(msg)
>> File "agent.py", line 255, in on_complete_topology
>> event_obj = CompleteTopologyEvent.create_from_json(msg.payload)
>> File
>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/event/topology/events.py",
>> line 190, in create_from_json
>> member_obj.member_ip = member_str["memberIp"]
>> KeyError: 'memberIp'
>> 2014-12-22 12:04:49,820:DEBUG:Message received:
>> 'tenant/CompleteTenantEvent':
>> '{"tenants":[]}'
>> 2014-12-22 12:04:49,820:DEBUG:Executing handler for event
>> 'CompleteTenantEvent'
>> 2014-12-22 12:04:49,820:INFO:Complete tenant event updating task disabled
>>
>>
>> *Carbon logs:*
>>
>> TID: [0] [STRATOS] [2014-12-22 06:27:21,240] INFO
>> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
>> - Publishing instance spawned event: [service-name] tomcat [cluster-id]
>> mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
>> mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 [instance-id]
>> ap-southeast-1/i-df7dde12 [network-partition-id] p1 [partition-id] P1
>> [lb-cluster-id] null
>> TID: [0] [STRATOS] [2014-12-22 06:27:21,261] INFO
>> {org.apache.stratos.messaging.domain.topology.lifecycle.LifeCycleStateManager}
>> - Life Cycle State Manager started for Element [
>> mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 ], initial
>> state: Created
>> TID: [0] [STRATOS] [2014-12-22 06:27:21,264] INFO
>> {org.apache.stratos.messaging.message.processor.topology.InstanceSpawnedMessageProcessor}
>> - Member created: [service] tomcat [cluster] mytomcat.tomcat.domain
>> [member] mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101
>> [instance] ap-southeast-1/i-df7dde12 [hierarchy-instance] app_cartridge_v1-1
>> TID: [0] [STRATOS] [2014-12-22 06:27:21,268] INFO
>> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
>> - [InstanceSpawnedEventListener] Received: class
>> org.apache.stratos.messaging.event.topology.InstanceSpawnedEvent
>>
>> --
>>
>> Udara Liyanage
>> Software Engineer
>> WSO2, Inc.: http://wso2.com
>> lean. enterprise. middleware
>>
>> web: http://udaraliyanage.wordpress.com
>> phone:
>> +94 71 443 6897
>>
>> --
>> Thanks and Regards,
>>
>> Isuru H.
>> +94 716 358 048* <http://wso2.com/>*
>>
>>
>> * <http://wso2.com/>*
>>
>>
>>
--
Rajkumar Rajaratnam
Committer & PMC Member, Apache Stratos
Software Engineer, WSO2
Mobile : +94777568639
Blog : rajkumarr.com
Re: Why agent wait to be intialized even after complete topology is received
Posted by Isuru Haththotuwa <is...@apache.org>.
HI Udara,
There is an error log about parsing the complete Topology event. Maybe the
event did not get parsed correctly, and the required initialization did not
happen.
On Mon, Dec 22, 2014 at 12:12 PM, Udara Liyanage <ud...@wso2.com> wrote:
> Hi Chamila,
>
> I get the ":Waiting for Cartridge Agent to be initialized..." logs
> continuously even though complete topology and complete tenant event is
> already received.
>
>
> 2014-12-22 12:04:47,595:DEBUG:Waiting for Cartridge Agent to be
> initialized...
> 2014-12-22 12:04:48,596:DEBUG:Waiting for Cartridge Agent to be
> initialized...
> 2014-12-22 12:04:49,597:DEBUG:Waiting for Cartridge Agent to be
> initialized...
> 2014-12-22 12:04:49,781:DEBUG:Message received:
> 'topology/CompleteTopologyEvent':
>
> '{"topology":{"serviceMap":{"tomcat":{"serviceName":"tomcat","serviceType":"SingleTenant","clusterIdClusterMap":{"mytomcat.tomcat.domain":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","autoscalePolicyName":"autoscale_policy_1","hostNames":["
> mytomcat.isuruh.lk
> "],"tenantRange":"*","isLbCluster":false,"isKubernetesCluster":false,"memberMap":{"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101":{"serviceName":"tomcat","clusterId":"mytomcat.tomcat.domain","memberId":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101","instanceId":"ap-southeast-1/i-df7dde12","clusterInstanceId":"app_cartridge_v1-1","networkPartitionId":"p1","partitionId":"P1","initTime":1419229605140,"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"memberPublicIPs":["54.151.190.32"],"defaultPublicIP":"54.151.190.32","memberPrivateIPs":["10.144.68.9"],"defaultPrivateIP":"10.144.68.9","properties":{"PRIMARY":"false","MIN_COUNT":"1"},"memberStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101"}}},"appId":"app_cartridge_v1","properties":{},"instanceIdToInstanceContextMap":{"app_cartridge_v1-1":{"instanceId":"app_cartridge_v1-1","instanceProperties":{},"lifeCycleStateManager":{"stateStack":["Created"],"identifier":"mytomcat.tomcat.domain_app_cartridge_v1-1"},"networkPartitionId":"p1"}}}},"portMap":{"443":{"protocol":"https","value":443,"proxy":443},"80":{"protocol":"http","value":80,"proxy":80}},"properties":{}}},"initialized":false}}'
> 2014-12-22 12:04:49,781:DEBUG:Executing handler for event
> 'CompleteTopologyEvent'
> 2014-12-22 12:04:49,781:DEBUG:Complete topology event received
> 2014-12-22 12:04:49,782:ERROR:Error processing 'CompleteTopologyEvent'
> event
> Traceback (most recent call last):
> File
> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
> line 81, in on_message
> handler(msg)
> File "agent.py", line 255, in on_complete_topology
> event_obj = CompleteTopologyEvent.create_from_json(msg.payload)
> File
> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/event/topology/events.py",
> line 190, in create_from_json
> member_obj.member_ip = member_str["memberIp"]
> KeyError: 'memberIp'
> 2014-12-22 12:04:49,820:DEBUG:Message received:
> 'tenant/CompleteTenantEvent':
> '{"tenants":[]}'
> 2014-12-22 12:04:49,820:DEBUG:Executing handler for event
> 'CompleteTenantEvent'
> 2014-12-22 12:04:49,820:INFO:Complete tenant event updating task disabled
>
>
> *Carbon logs:*
>
> TID: [0] [STRATOS] [2014-12-22 06:27:21,240] INFO
> {org.apache.stratos.cloud.controller.messaging.publisher.TopologyEventPublisher}
> - Publishing instance spawned event: [service-name] tomcat [cluster-id]
> mytomcat.tomcat.domain [cluster-instance-id] app_cartridge_v1-1 [member-id]
> mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 [instance-id]
> ap-southeast-1/i-df7dde12 [network-partition-id] p1 [partition-id] P1
> [lb-cluster-id] null
> TID: [0] [STRATOS] [2014-12-22 06:27:21,261] INFO
> {org.apache.stratos.messaging.domain.topology.lifecycle.LifeCycleStateManager}
> - Life Cycle State Manager started for Element [
> mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101 ], initial
> state: Created
> TID: [0] [STRATOS] [2014-12-22 06:27:21,264] INFO
> {org.apache.stratos.messaging.message.processor.topology.InstanceSpawnedMessageProcessor}
> - Member created: [service] tomcat [cluster] mytomcat.tomcat.domain
> [member] mytomcat.tomcat.domainf1ee1c36-db95-452d-a1e7-1daf936e9101
> [instance] ap-southeast-1/i-df7dde12 [hierarchy-instance] app_cartridge_v1-1
> TID: [0] [STRATOS] [2014-12-22 06:27:21,268] INFO
> {org.apache.stratos.manager.topology.receiver.StratosManagerTopologyEventReceiver}
> - [InstanceSpawnedEventListener] Received: class
> org.apache.stratos.messaging.event.topology.InstanceSpawnedEvent
>
> --
>
> Udara Liyanage
> Software Engineer
> WSO2, Inc.: http://wso2.com
> lean. enterprise. middleware
>
> web: http://udaraliyanage.wordpress.com
> phone: +94 71 443 6897
>
> --
> Thanks and Regards,
>
> Isuru H.
> +94 716 358 048* <http://wso2.com/>*
>
>
> * <http://wso2.com/>*
>
>
>