You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stratos.apache.org by Chamila De Alwis <ch...@wso2.com> on 2015/05/18 20:36:25 UTC
[Discuss] PCA - Log Event Publishing to BAM
Hi,
I've fixed and tested the $subject in the Python Cartridge Agent. Now when
the following parameters are set in the agent.conf configuration file, the
LogPublisher job will be activated, which will scan the specified logs
every 1 second and publish each entry as a Thrift event to the specified
thrift receiver.
enable.data.publisher =true
monitoring.server.ip =localhost
monitoring.server.port =7611
monitoring.server.secure.port =7711
monitoring.server.admin.username =admin
monitoring.server.admin.password =admin
log.file.paths
=/home/chamilad/dev/wso2esb-4.8.1/repository/logs/wso2carbon.log
I have verified the functionality using the Integration test for the PCA.
When verifying the VM and the Kubernetes end-to-end scenario, this
functionality can be again tested for any unseen edge cases. I used a
standalone WSO2 BAM server as the monitoring server. However, any Thrift
receiver can be used as the publishing endpoint for Log publishing.
Regards,
Chamila de Alwis
Software Engineer | WSO2 | +94772207163
Blog: code.chamiladealwis.com
Re: [Discuss] PCA - Log Event Publishing to BAM
Posted by Mariangela Hills <ma...@wso2.com>.
Thanks, for the clarification!
Regards,
Mariangela
*--*
*Mariangela Hills*
PMC Member & Committer of Apache Stratos
Senior Technical Writer
WSO2, Inc.
lean.enterprise.middleware.
m: +94 773 500185
w: http://wso2.com
<http://wso2.com/events/>
On Thu, May 21, 2015 at 9:39 AM, Chamila De Alwis <ch...@wso2.com> wrote:
> Hi Mary,
>
> We have not yet dropped Java Cartridge Agent. It will be available for
> 4.1.0.
> On May 21, 2015 5:13 AM, "Mariangela Hills" <ma...@wso2.com> wrote:
>
>> Chamila, as we have the Python CA, I was under the impression that we are
>> not going to support the Java Cartridge Agent from Stratos 4.1.0 onwards.
>> Have I got the wrong picture?
>>
>> Regards,
>> Mariangela
>>
>>
>>
>>
>> *--*
>> *Mariangela Hills*
>> PMC Member & Committer of Apache Stratos
>> Senior Technical Writer
>> WSO2, Inc.
>> lean.enterprise.middleware.
>> m: +94 773 500185
>> w: http://wso2.com
>> <http://wso2.com/events/>
>>
>> On Thu, May 21, 2015 at 5:09 AM, Mariangela Hills <ma...@wso2.com>
>> wrote:
>>
>>> Thanks, Chamila! I will update this wiki page.
>>>
>>> Regards,
>>> Mariangela
>>>
>>>
>>>
>>>
>>> *--*
>>> *Mariangela Hills*
>>> PMC Member & Committer of Apache Stratos
>>> Senior Technical Writer
>>> WSO2, Inc.
>>> lean.enterprise.middleware.
>>> m: +94 773 500185
>>> w: http://wso2.com
>>> <http://wso2.com/events/>
>>>
>>> On Thu, May 21, 2015 at 2:06 AM, Rajkumar Rajaratnam <rajkumarr@wso2.com
>>> > wrote:
>>>
>>>> Thanks for the info Chamila!
>>>>
>>>> On Wed, May 20, 2015 at 7:19 PM, Chamila De Alwis <ch...@wso2.com>
>>>> wrote:
>>>>
>>>>> Hi Raj,
>>>>>
>>>>> Please find my comments inline.
>>>>>
>>>>> On Wed, May 20, 2015 at 6:57 PM, Rajkumar Rajaratnam <
>>>>> rajkumarr@wso2.com> wrote:
>>>>>
>>>>>> Sorry I am not up-to-date with this feature.
>>>>>>
>>>>>> 1) Is this publishing PCA's logs to BAM or Cartridge's logs to BAM?
>>>>>>
>>>>> The intention is to allow to publish service logs (eg: httpd.log),
>>>>> however any specified log file location will be scanned.
>>>>>
>>>>>
>>>>>> 2) Can it publish logs to more than one thrift receivers?
>>>>>> 3) Can it publish logs to more than one thrift receivers in a
>>>>>> load-balancing manner (1st event to 1st server and 2nd event to 2nd server)?
>>>>>> 4) Can it publish logs to more then one thrift receivers in a
>>>>>> fail-over manner (if sending to 1st server fails, then try to send it to
>>>>>> 2nd server and so on)?
>>>>>> 5) Is there any receiver groups concept implemented? For example,
>>>>>> publish events to CEP cluster in a load-balancing manner, while publish
>>>>>> same events to BAM cluster in a fail-over manner? Here CEP cluster is one
>>>>>> receiver group and BAM cluster is another one.
>>>>>> 6) Is there any event queueing mechanism implemented? What happens if
>>>>>> thrift receivers are not available?
>>>>>>
>>>>> Unfortunately, the Python Thrift publisher does not have these
>>>>> features. Currently no multiple Thrift receiver concept is implemented, and
>>>>> there are no WSO2 Carbon DataBridge like event queuing in the Thrift
>>>>> publisher. The Java Cartridge Agent makes use of the WSO2 Carbon's
>>>>> DataBridge, therefore it can handle multiple Thrift receivers. I'm not sure
>>>>> of the load balancing functionality of the Carbon DataBridge. I agree that
>>>>> these are some interesting and important features that need to be included,
>>>>> so
>>>>>
>>>>>
>>>>>>
>>>>>> If these are not implemented yet, we will create a JIRA and track
>>>>>> this for future release.
>>>>>>
>>>>> .. sounds good. :)
>>>>>
>>>>>
>>>>>
>>>>> Regards,
>>>>> Chamila de Alwis
>>>>> Software Engineer | WSO2 | +94772207163
>>>>> Blog: code.chamiladealwis.com
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Rajkumar Rajaratnam
>>>> Committer & PMC Member, Apache Stratos
>>>> Software Engineer, WSO2
>>>>
>>>> Mobile : +94777568639
>>>> Blog : rajkumarr.com
>>>>
>>>
>>>
>>
Re: [Discuss] PCA - Log Event Publishing to BAM
Posted by Chamila De Alwis <ch...@wso2.com>.
Hi Mary,
We have not yet dropped Java Cartridge Agent. It will be available for
4.1.0.
On May 21, 2015 5:13 AM, "Mariangela Hills" <ma...@wso2.com> wrote:
> Chamila, as we have the Python CA, I was under the impression that we are
> not going to support the Java Cartridge Agent from Stratos 4.1.0 onwards.
> Have I got the wrong picture?
>
> Regards,
> Mariangela
>
>
>
>
> *--*
> *Mariangela Hills*
> PMC Member & Committer of Apache Stratos
> Senior Technical Writer
> WSO2, Inc.
> lean.enterprise.middleware.
> m: +94 773 500185
> w: http://wso2.com
> <http://wso2.com/events/>
>
> On Thu, May 21, 2015 at 5:09 AM, Mariangela Hills <ma...@wso2.com>
> wrote:
>
>> Thanks, Chamila! I will update this wiki page.
>>
>> Regards,
>> Mariangela
>>
>>
>>
>>
>> *--*
>> *Mariangela Hills*
>> PMC Member & Committer of Apache Stratos
>> Senior Technical Writer
>> WSO2, Inc.
>> lean.enterprise.middleware.
>> m: +94 773 500185
>> w: http://wso2.com
>> <http://wso2.com/events/>
>>
>> On Thu, May 21, 2015 at 2:06 AM, Rajkumar Rajaratnam <ra...@wso2.com>
>> wrote:
>>
>>> Thanks for the info Chamila!
>>>
>>> On Wed, May 20, 2015 at 7:19 PM, Chamila De Alwis <ch...@wso2.com>
>>> wrote:
>>>
>>>> Hi Raj,
>>>>
>>>> Please find my comments inline.
>>>>
>>>> On Wed, May 20, 2015 at 6:57 PM, Rajkumar Rajaratnam <
>>>> rajkumarr@wso2.com> wrote:
>>>>
>>>>> Sorry I am not up-to-date with this feature.
>>>>>
>>>>> 1) Is this publishing PCA's logs to BAM or Cartridge's logs to BAM?
>>>>>
>>>> The intention is to allow to publish service logs (eg: httpd.log),
>>>> however any specified log file location will be scanned.
>>>>
>>>>
>>>>> 2) Can it publish logs to more than one thrift receivers?
>>>>> 3) Can it publish logs to more than one thrift receivers in a
>>>>> load-balancing manner (1st event to 1st server and 2nd event to 2nd server)?
>>>>> 4) Can it publish logs to more then one thrift receivers in a
>>>>> fail-over manner (if sending to 1st server fails, then try to send it to
>>>>> 2nd server and so on)?
>>>>> 5) Is there any receiver groups concept implemented? For example,
>>>>> publish events to CEP cluster in a load-balancing manner, while publish
>>>>> same events to BAM cluster in a fail-over manner? Here CEP cluster is one
>>>>> receiver group and BAM cluster is another one.
>>>>> 6) Is there any event queueing mechanism implemented? What happens if
>>>>> thrift receivers are not available?
>>>>>
>>>> Unfortunately, the Python Thrift publisher does not have these
>>>> features. Currently no multiple Thrift receiver concept is implemented, and
>>>> there are no WSO2 Carbon DataBridge like event queuing in the Thrift
>>>> publisher. The Java Cartridge Agent makes use of the WSO2 Carbon's
>>>> DataBridge, therefore it can handle multiple Thrift receivers. I'm not sure
>>>> of the load balancing functionality of the Carbon DataBridge. I agree that
>>>> these are some interesting and important features that need to be included,
>>>> so
>>>>
>>>>
>>>>>
>>>>> If these are not implemented yet, we will create a JIRA and track this
>>>>> for future release.
>>>>>
>>>> .. sounds good. :)
>>>>
>>>>
>>>>
>>>> Regards,
>>>> Chamila de Alwis
>>>> Software Engineer | WSO2 | +94772207163
>>>> Blog: code.chamiladealwis.com
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Rajkumar Rajaratnam
>>> Committer & PMC Member, Apache Stratos
>>> Software Engineer, WSO2
>>>
>>> Mobile : +94777568639
>>> Blog : rajkumarr.com
>>>
>>
>>
>
Re: [Discuss] PCA - Log Event Publishing to BAM
Posted by Mariangela Hills <ma...@wso2.com>.
Chamila, as we have the Python CA, I was under the impression that we are
not going to support the Java Cartridge Agent from Stratos 4.1.0 onwards.
Have I got the wrong picture?
Regards,
Mariangela
*--*
*Mariangela Hills*
PMC Member & Committer of Apache Stratos
Senior Technical Writer
WSO2, Inc.
lean.enterprise.middleware.
m: +94 773 500185
w: http://wso2.com
<http://wso2.com/events/>
On Thu, May 21, 2015 at 5:09 AM, Mariangela Hills <ma...@wso2.com>
wrote:
> Thanks, Chamila! I will update this wiki page.
>
> Regards,
> Mariangela
>
>
>
>
> *--*
> *Mariangela Hills*
> PMC Member & Committer of Apache Stratos
> Senior Technical Writer
> WSO2, Inc.
> lean.enterprise.middleware.
> m: +94 773 500185
> w: http://wso2.com
> <http://wso2.com/events/>
>
> On Thu, May 21, 2015 at 2:06 AM, Rajkumar Rajaratnam <ra...@wso2.com>
> wrote:
>
>> Thanks for the info Chamila!
>>
>> On Wed, May 20, 2015 at 7:19 PM, Chamila De Alwis <ch...@wso2.com>
>> wrote:
>>
>>> Hi Raj,
>>>
>>> Please find my comments inline.
>>>
>>> On Wed, May 20, 2015 at 6:57 PM, Rajkumar Rajaratnam <rajkumarr@wso2.com
>>> > wrote:
>>>
>>>> Sorry I am not up-to-date with this feature.
>>>>
>>>> 1) Is this publishing PCA's logs to BAM or Cartridge's logs to BAM?
>>>>
>>> The intention is to allow to publish service logs (eg: httpd.log),
>>> however any specified log file location will be scanned.
>>>
>>>
>>>> 2) Can it publish logs to more than one thrift receivers?
>>>> 3) Can it publish logs to more than one thrift receivers in a
>>>> load-balancing manner (1st event to 1st server and 2nd event to 2nd server)?
>>>> 4) Can it publish logs to more then one thrift receivers in a fail-over
>>>> manner (if sending to 1st server fails, then try to send it to 2nd server
>>>> and so on)?
>>>> 5) Is there any receiver groups concept implemented? For example,
>>>> publish events to CEP cluster in a load-balancing manner, while publish
>>>> same events to BAM cluster in a fail-over manner? Here CEP cluster is one
>>>> receiver group and BAM cluster is another one.
>>>> 6) Is there any event queueing mechanism implemented? What happens if
>>>> thrift receivers are not available?
>>>>
>>> Unfortunately, the Python Thrift publisher does not have these features.
>>> Currently no multiple Thrift receiver concept is implemented, and there are
>>> no WSO2 Carbon DataBridge like event queuing in the Thrift publisher. The
>>> Java Cartridge Agent makes use of the WSO2 Carbon's DataBridge, therefore
>>> it can handle multiple Thrift receivers. I'm not sure of the load balancing
>>> functionality of the Carbon DataBridge. I agree that these are some
>>> interesting and important features that need to be included, so
>>>
>>>
>>>>
>>>> If these are not implemented yet, we will create a JIRA and track this
>>>> for future release.
>>>>
>>> .. sounds good. :)
>>>
>>>
>>>
>>> Regards,
>>> Chamila de Alwis
>>> Software Engineer | WSO2 | +94772207163
>>> Blog: code.chamiladealwis.com
>>>
>>>
>>>
>>
>>
>> --
>> Rajkumar Rajaratnam
>> Committer & PMC Member, Apache Stratos
>> Software Engineer, WSO2
>>
>> Mobile : +94777568639
>> Blog : rajkumarr.com
>>
>
>
Re: [Discuss] PCA - Log Event Publishing to BAM
Posted by Mariangela Hills <ma...@wso2.com>.
Thanks, Chamila! I will update this wiki page.
Regards,
Mariangela
*--*
*Mariangela Hills*
PMC Member & Committer of Apache Stratos
Senior Technical Writer
WSO2, Inc.
lean.enterprise.middleware.
m: +94 773 500185
w: http://wso2.com
<http://wso2.com/events/>
On Thu, May 21, 2015 at 2:06 AM, Rajkumar Rajaratnam <ra...@wso2.com>
wrote:
> Thanks for the info Chamila!
>
> On Wed, May 20, 2015 at 7:19 PM, Chamila De Alwis <ch...@wso2.com>
> wrote:
>
>> Hi Raj,
>>
>> Please find my comments inline.
>>
>> On Wed, May 20, 2015 at 6:57 PM, Rajkumar Rajaratnam <ra...@wso2.com>
>> wrote:
>>
>>> Sorry I am not up-to-date with this feature.
>>>
>>> 1) Is this publishing PCA's logs to BAM or Cartridge's logs to BAM?
>>>
>> The intention is to allow to publish service logs (eg: httpd.log),
>> however any specified log file location will be scanned.
>>
>>
>>> 2) Can it publish logs to more than one thrift receivers?
>>> 3) Can it publish logs to more than one thrift receivers in a
>>> load-balancing manner (1st event to 1st server and 2nd event to 2nd server)?
>>> 4) Can it publish logs to more then one thrift receivers in a fail-over
>>> manner (if sending to 1st server fails, then try to send it to 2nd server
>>> and so on)?
>>> 5) Is there any receiver groups concept implemented? For example,
>>> publish events to CEP cluster in a load-balancing manner, while publish
>>> same events to BAM cluster in a fail-over manner? Here CEP cluster is one
>>> receiver group and BAM cluster is another one.
>>> 6) Is there any event queueing mechanism implemented? What happens if
>>> thrift receivers are not available?
>>>
>> Unfortunately, the Python Thrift publisher does not have these features.
>> Currently no multiple Thrift receiver concept is implemented, and there are
>> no WSO2 Carbon DataBridge like event queuing in the Thrift publisher. The
>> Java Cartridge Agent makes use of the WSO2 Carbon's DataBridge, therefore
>> it can handle multiple Thrift receivers. I'm not sure of the load balancing
>> functionality of the Carbon DataBridge. I agree that these are some
>> interesting and important features that need to be included, so
>>
>>
>>>
>>> If these are not implemented yet, we will create a JIRA and track this
>>> for future release.
>>>
>> .. sounds good. :)
>>
>>
>>
>> Regards,
>> Chamila de Alwis
>> Software Engineer | WSO2 | +94772207163
>> Blog: code.chamiladealwis.com
>>
>>
>>
>
>
> --
> Rajkumar Rajaratnam
> Committer & PMC Member, Apache Stratos
> Software Engineer, WSO2
>
> Mobile : +94777568639
> Blog : rajkumarr.com
>
Re: [Discuss] PCA - Log Event Publishing to BAM
Posted by Rajkumar Rajaratnam <ra...@wso2.com>.
Thanks for the info Chamila!
On Wed, May 20, 2015 at 7:19 PM, Chamila De Alwis <ch...@wso2.com> wrote:
> Hi Raj,
>
> Please find my comments inline.
>
> On Wed, May 20, 2015 at 6:57 PM, Rajkumar Rajaratnam <ra...@wso2.com>
> wrote:
>
>> Sorry I am not up-to-date with this feature.
>>
>> 1) Is this publishing PCA's logs to BAM or Cartridge's logs to BAM?
>>
> The intention is to allow to publish service logs (eg: httpd.log), however
> any specified log file location will be scanned.
>
>
>> 2) Can it publish logs to more than one thrift receivers?
>> 3) Can it publish logs to more than one thrift receivers in a
>> load-balancing manner (1st event to 1st server and 2nd event to 2nd server)?
>> 4) Can it publish logs to more then one thrift receivers in a fail-over
>> manner (if sending to 1st server fails, then try to send it to 2nd server
>> and so on)?
>> 5) Is there any receiver groups concept implemented? For example, publish
>> events to CEP cluster in a load-balancing manner, while publish same events
>> to BAM cluster in a fail-over manner? Here CEP cluster is one receiver
>> group and BAM cluster is another one.
>> 6) Is there any event queueing mechanism implemented? What happens if
>> thrift receivers are not available?
>>
> Unfortunately, the Python Thrift publisher does not have these features.
> Currently no multiple Thrift receiver concept is implemented, and there are
> no WSO2 Carbon DataBridge like event queuing in the Thrift publisher. The
> Java Cartridge Agent makes use of the WSO2 Carbon's DataBridge, therefore
> it can handle multiple Thrift receivers. I'm not sure of the load balancing
> functionality of the Carbon DataBridge. I agree that these are some
> interesting and important features that need to be included, so
>
>
>>
>> If these are not implemented yet, we will create a JIRA and track this
>> for future release.
>>
> .. sounds good. :)
>
>
>
> Regards,
> Chamila de Alwis
> Software Engineer | WSO2 | +94772207163
> Blog: code.chamiladealwis.com
>
>
>
--
Rajkumar Rajaratnam
Committer & PMC Member, Apache Stratos
Software Engineer, WSO2
Mobile : +94777568639
Blog : rajkumarr.com
Re: [Discuss] PCA - Log Event Publishing to BAM
Posted by Chamila De Alwis <ch...@wso2.com>.
Hi Raj,
Please find my comments inline.
On Wed, May 20, 2015 at 6:57 PM, Rajkumar Rajaratnam <ra...@wso2.com>
wrote:
> Sorry I am not up-to-date with this feature.
>
> 1) Is this publishing PCA's logs to BAM or Cartridge's logs to BAM?
>
The intention is to allow to publish service logs (eg: httpd.log), however
any specified log file location will be scanned.
> 2) Can it publish logs to more than one thrift receivers?
> 3) Can it publish logs to more than one thrift receivers in a
> load-balancing manner (1st event to 1st server and 2nd event to 2nd server)?
> 4) Can it publish logs to more then one thrift receivers in a fail-over
> manner (if sending to 1st server fails, then try to send it to 2nd server
> and so on)?
> 5) Is there any receiver groups concept implemented? For example, publish
> events to CEP cluster in a load-balancing manner, while publish same events
> to BAM cluster in a fail-over manner? Here CEP cluster is one receiver
> group and BAM cluster is another one.
> 6) Is there any event queueing mechanism implemented? What happens if
> thrift receivers are not available?
>
Unfortunately, the Python Thrift publisher does not have these features.
Currently no multiple Thrift receiver concept is implemented, and there are
no WSO2 Carbon DataBridge like event queuing in the Thrift publisher. The
Java Cartridge Agent makes use of the WSO2 Carbon's DataBridge, therefore
it can handle multiple Thrift receivers. I'm not sure of the load balancing
functionality of the Carbon DataBridge. I agree that these are some
interesting and important features that need to be included, so
>
> If these are not implemented yet, we will create a JIRA and track this for
> future release.
>
.. sounds good. :)
Regards,
Chamila de Alwis
Software Engineer | WSO2 | +94772207163
Blog: code.chamiladealwis.com
Re: [Discuss] PCA - Log Event Publishing to BAM
Posted by Chamila De Alwis <ch...@wso2.com>.
Hi Mary,
A few notes on the documentation page on Log publishing.
- Let's put the location of the agent.conf file as
<PYTHON_CARTRIDGE_AGENT_HOME>/agent.conf. The location mentioned is the
location in the source code.
- Minor typo - "This actives" to "This activates"
- Let's also mention that multiple log files can be specified as "|"
separated values.
- The Java Cartridge Agent also has this functionality. The parameters
to specify are in <JAVA_CARTRIDGE_AGENT_HOME>/bin/stratos.sh (same
parameter names).
- Specifying the parameters is done in Puppet's base.pp file (for the VM
scenario) and in the Cartridge definition (for the Kubernetes workflow).
Regards,
Chamila de Alwis
Software Engineer | WSO2 | +94772207163
Blog: code.chamiladealwis.com
On Wed, May 20, 2015 at 6:57 PM, Rajkumar Rajaratnam <ra...@wso2.com>
wrote:
> Great work Chamila!
>
> Sorry I am not up-to-date with this feature.
>
> 1) Is this publishing PCA's logs to BAM or Cartridge's logs to BAM?
> 2) Can it publish logs to more than one thrift receivers?
> 3) Can it publish logs to more than one thrift receivers in a
> load-balancing manner (1st event to 1st server and 2nd event to 2nd server)?
> 4) Can it publish logs to more then one thrift receivers in a fail-over
> manner (if sending to 1st server fails, then try to send it to 2nd server
> and so on)?
> 5) Is there any receiver groups concept implemented? For example, publish
> events to CEP cluster in a load-balancing manner, while publish same events
> to BAM cluster in a fail-over manner? Here CEP cluster is one receiver
> group and BAM cluster is another one.
> 6) Is there any event queueing mechanism implemented? What happens if
> thrift receivers are not available?
>
> If these are not implemented yet, we will create a JIRA and track this for
> future release.
>
> WDYT?
>
> Thanks.
>
> On Wed, May 20, 2015 at 6:20 PM, Mariangela Hills <ma...@wso2.com>
> wrote:
>
>> I have added this content in [1]. Please let me know whether any changes
>> are needed.
>>
>> [1] https://cwiki.apache.org/confluence/display/STRATOS/4.1.0+Logging
>>
>> Regards,
>> Mariangela
>>
>>
>>
>>
>> *--*
>> *Mariangela Hills*
>> PMC Member & Committer of Apache Stratos
>> Senior Technical Writer
>> WSO2, Inc.
>> lean.enterprise.middleware.
>> m: +94 773 500185
>> w: http://wso2.com
>> <http://wso2.com/events/>
>>
>> On Tue, May 19, 2015 at 11:53 AM, Mariangela Hills <ma...@wso2.com>
>> wrote:
>>
>>> Will do!
>>>
>>> Regards,
>>> Mariangela
>>>
>>>
>>>
>>>
>>> *--*
>>> *Mariangela Hills*
>>> PMC Member & Committer of Apache Stratos
>>> Senior Technical Writer
>>> WSO2, Inc.
>>> lean.enterprise.middleware.
>>> m: +94 773 500185
>>> w: http://wso2.com
>>> <http://wso2.com/events/>
>>>
>>> On Tue, May 19, 2015 at 11:47 AM, Imesh Gunaratne <im...@apache.org>
>>> wrote:
>>>
>>>> Great work Chamila! @Mari: It would be great if we can document this.
>>>>
>>>> Thanks
>>>>
>>>> On Tue, May 19, 2015 at 12:06 AM, Chamila De Alwis <ch...@wso2.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I've fixed and tested the $subject in the Python Cartridge Agent. Now
>>>>> when the following parameters are set in the agent.conf configuration file,
>>>>> the LogPublisher job will be activated, which will scan the specified logs
>>>>> every 1 second and publish each entry as a Thrift event to the specified
>>>>> thrift receiver.
>>>>>
>>>>> enable.data.publisher =true
>>>>> monitoring.server.ip =localhost
>>>>> monitoring.server.port =7611
>>>>> monitoring.server.secure.port =7711
>>>>> monitoring.server.admin.username =admin
>>>>> monitoring.server.admin.password =admin
>>>>> log.file.paths =/home/chamilad/dev/wso2esb-4.8.1/repository/logs/wso2carbon.log
>>>>>
>>>>>
>>>>> I have verified the functionality using the Integration test for the
>>>>> PCA. When verifying the VM and the Kubernetes end-to-end scenario, this
>>>>> functionality can be again tested for any unseen edge cases. I used a
>>>>> standalone WSO2 BAM server as the monitoring server. However, any Thrift
>>>>> receiver can be used as the publishing endpoint for Log publishing.
>>>>>
>>>>>
>>>>> Regards,
>>>>> Chamila de Alwis
>>>>> Software Engineer | WSO2 | +94772207163
>>>>> Blog: code.chamiladealwis.com
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Imesh Gunaratne
>>>>
>>>> Senior Technical Lead, WSO2
>>>> Committer & PMC Member, Apache Stratos
>>>>
>>>
>>>
>>
>
>
> --
> Rajkumar Rajaratnam
> Committer & PMC Member, Apache Stratos
> Software Engineer, WSO2
>
> Mobile : +94777568639
> Blog : rajkumarr.com
>
Re: [Discuss] PCA - Log Event Publishing to BAM
Posted by Rajkumar Rajaratnam <ra...@wso2.com>.
Great work Chamila!
Sorry I am not up-to-date with this feature.
1) Is this publishing PCA's logs to BAM or Cartridge's logs to BAM?
2) Can it publish logs to more than one thrift receivers?
3) Can it publish logs to more than one thrift receivers in a
load-balancing manner (1st event to 1st server and 2nd event to 2nd server)?
4) Can it publish logs to more then one thrift receivers in a fail-over
manner (if sending to 1st server fails, then try to send it to 2nd server
and so on)?
5) Is there any receiver groups concept implemented? For example, publish
events to CEP cluster in a load-balancing manner, while publish same events
to BAM cluster in a fail-over manner? Here CEP cluster is one receiver
group and BAM cluster is another one.
6) Is there any event queueing mechanism implemented? What happens if
thrift receivers are not available?
If these are not implemented yet, we will create a JIRA and track this for
future release.
WDYT?
Thanks.
On Wed, May 20, 2015 at 6:20 PM, Mariangela Hills <ma...@wso2.com>
wrote:
> I have added this content in [1]. Please let me know whether any changes
> are needed.
>
> [1] https://cwiki.apache.org/confluence/display/STRATOS/4.1.0+Logging
>
> Regards,
> Mariangela
>
>
>
>
> *--*
> *Mariangela Hills*
> PMC Member & Committer of Apache Stratos
> Senior Technical Writer
> WSO2, Inc.
> lean.enterprise.middleware.
> m: +94 773 500185
> w: http://wso2.com
> <http://wso2.com/events/>
>
> On Tue, May 19, 2015 at 11:53 AM, Mariangela Hills <ma...@wso2.com>
> wrote:
>
>> Will do!
>>
>> Regards,
>> Mariangela
>>
>>
>>
>>
>> *--*
>> *Mariangela Hills*
>> PMC Member & Committer of Apache Stratos
>> Senior Technical Writer
>> WSO2, Inc.
>> lean.enterprise.middleware.
>> m: +94 773 500185
>> w: http://wso2.com
>> <http://wso2.com/events/>
>>
>> On Tue, May 19, 2015 at 11:47 AM, Imesh Gunaratne <im...@apache.org>
>> wrote:
>>
>>> Great work Chamila! @Mari: It would be great if we can document this.
>>>
>>> Thanks
>>>
>>> On Tue, May 19, 2015 at 12:06 AM, Chamila De Alwis <ch...@wso2.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I've fixed and tested the $subject in the Python Cartridge Agent. Now
>>>> when the following parameters are set in the agent.conf configuration file,
>>>> the LogPublisher job will be activated, which will scan the specified logs
>>>> every 1 second and publish each entry as a Thrift event to the specified
>>>> thrift receiver.
>>>>
>>>> enable.data.publisher =true
>>>> monitoring.server.ip =localhost
>>>> monitoring.server.port =7611
>>>> monitoring.server.secure.port =7711
>>>> monitoring.server.admin.username =admin
>>>> monitoring.server.admin.password =admin
>>>> log.file.paths =/home/chamilad/dev/wso2esb-4.8.1/repository/logs/wso2carbon.log
>>>>
>>>>
>>>> I have verified the functionality using the Integration test for the
>>>> PCA. When verifying the VM and the Kubernetes end-to-end scenario, this
>>>> functionality can be again tested for any unseen edge cases. I used a
>>>> standalone WSO2 BAM server as the monitoring server. However, any Thrift
>>>> receiver can be used as the publishing endpoint for Log publishing.
>>>>
>>>>
>>>> Regards,
>>>> Chamila de Alwis
>>>> Software Engineer | WSO2 | +94772207163
>>>> Blog: code.chamiladealwis.com
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Imesh Gunaratne
>>>
>>> Senior Technical Lead, WSO2
>>> Committer & PMC Member, Apache Stratos
>>>
>>
>>
>
--
Rajkumar Rajaratnam
Committer & PMC Member, Apache Stratos
Software Engineer, WSO2
Mobile : +94777568639
Blog : rajkumarr.com
Re: [Discuss] PCA - Log Event Publishing to BAM
Posted by Mariangela Hills <ma...@wso2.com>.
I have added this content in [1]. Please let me know whether any changes
are needed.
[1] https://cwiki.apache.org/confluence/display/STRATOS/4.1.0+Logging
Regards,
Mariangela
*--*
*Mariangela Hills*
PMC Member & Committer of Apache Stratos
Senior Technical Writer
WSO2, Inc.
lean.enterprise.middleware.
m: +94 773 500185
w: http://wso2.com
<http://wso2.com/events/>
On Tue, May 19, 2015 at 11:53 AM, Mariangela Hills <ma...@wso2.com>
wrote:
> Will do!
>
> Regards,
> Mariangela
>
>
>
>
> *--*
> *Mariangela Hills*
> PMC Member & Committer of Apache Stratos
> Senior Technical Writer
> WSO2, Inc.
> lean.enterprise.middleware.
> m: +94 773 500185
> w: http://wso2.com
> <http://wso2.com/events/>
>
> On Tue, May 19, 2015 at 11:47 AM, Imesh Gunaratne <im...@apache.org>
> wrote:
>
>> Great work Chamila! @Mari: It would be great if we can document this.
>>
>> Thanks
>>
>> On Tue, May 19, 2015 at 12:06 AM, Chamila De Alwis <ch...@wso2.com>
>> wrote:
>>
>>> Hi,
>>>
>>> I've fixed and tested the $subject in the Python Cartridge Agent. Now
>>> when the following parameters are set in the agent.conf configuration file,
>>> the LogPublisher job will be activated, which will scan the specified logs
>>> every 1 second and publish each entry as a Thrift event to the specified
>>> thrift receiver.
>>>
>>> enable.data.publisher =true
>>> monitoring.server.ip =localhost
>>> monitoring.server.port =7611
>>> monitoring.server.secure.port =7711
>>> monitoring.server.admin.username =admin
>>> monitoring.server.admin.password =admin
>>> log.file.paths =/home/chamilad/dev/wso2esb-4.8.1/repository/logs/wso2carbon.log
>>>
>>>
>>> I have verified the functionality using the Integration test for the
>>> PCA. When verifying the VM and the Kubernetes end-to-end scenario, this
>>> functionality can be again tested for any unseen edge cases. I used a
>>> standalone WSO2 BAM server as the monitoring server. However, any Thrift
>>> receiver can be used as the publishing endpoint for Log publishing.
>>>
>>>
>>> Regards,
>>> Chamila de Alwis
>>> Software Engineer | WSO2 | +94772207163
>>> Blog: code.chamiladealwis.com
>>>
>>>
>>>
>>
>>
>> --
>> Imesh Gunaratne
>>
>> Senior Technical Lead, WSO2
>> Committer & PMC Member, Apache Stratos
>>
>
>
Re: [Discuss] PCA - Log Event Publishing to BAM
Posted by Mariangela Hills <ma...@wso2.com>.
Will do!
Regards,
Mariangela
*--*
*Mariangela Hills*
PMC Member & Committer of Apache Stratos
Senior Technical Writer
WSO2, Inc.
lean.enterprise.middleware.
m: +94 773 500185
w: http://wso2.com
<http://wso2.com/events/>
On Tue, May 19, 2015 at 11:47 AM, Imesh Gunaratne <im...@apache.org> wrote:
> Great work Chamila! @Mari: It would be great if we can document this.
>
> Thanks
>
> On Tue, May 19, 2015 at 12:06 AM, Chamila De Alwis <ch...@wso2.com>
> wrote:
>
>> Hi,
>>
>> I've fixed and tested the $subject in the Python Cartridge Agent. Now
>> when the following parameters are set in the agent.conf configuration file,
>> the LogPublisher job will be activated, which will scan the specified logs
>> every 1 second and publish each entry as a Thrift event to the specified
>> thrift receiver.
>>
>> enable.data.publisher =true
>> monitoring.server.ip =localhost
>> monitoring.server.port =7611
>> monitoring.server.secure.port =7711
>> monitoring.server.admin.username =admin
>> monitoring.server.admin.password =admin
>> log.file.paths =/home/chamilad/dev/wso2esb-4.8.1/repository/logs/wso2carbon.log
>>
>>
>> I have verified the functionality using the Integration test for the PCA.
>> When verifying the VM and the Kubernetes end-to-end scenario, this
>> functionality can be again tested for any unseen edge cases. I used a
>> standalone WSO2 BAM server as the monitoring server. However, any Thrift
>> receiver can be used as the publishing endpoint for Log publishing.
>>
>>
>> Regards,
>> Chamila de Alwis
>> Software Engineer | WSO2 | +94772207163
>> Blog: code.chamiladealwis.com
>>
>>
>>
>
>
> --
> Imesh Gunaratne
>
> Senior Technical Lead, WSO2
> Committer & PMC Member, Apache Stratos
>
Re: [Discuss] PCA - Log Event Publishing to BAM
Posted by Imesh Gunaratne <im...@apache.org>.
Great work Chamila! @Mari: It would be great if we can document this.
Thanks
On Tue, May 19, 2015 at 12:06 AM, Chamila De Alwis <ch...@wso2.com>
wrote:
> Hi,
>
> I've fixed and tested the $subject in the Python Cartridge Agent. Now when
> the following parameters are set in the agent.conf configuration file, the
> LogPublisher job will be activated, which will scan the specified logs
> every 1 second and publish each entry as a Thrift event to the specified
> thrift receiver.
>
> enable.data.publisher =true
> monitoring.server.ip =localhost
> monitoring.server.port =7611
> monitoring.server.secure.port =7711
> monitoring.server.admin.username =admin
> monitoring.server.admin.password =admin
> log.file.paths =/home/chamilad/dev/wso2esb-4.8.1/repository/logs/wso2carbon.log
>
>
> I have verified the functionality using the Integration test for the PCA.
> When verifying the VM and the Kubernetes end-to-end scenario, this
> functionality can be again tested for any unseen edge cases. I used a
> standalone WSO2 BAM server as the monitoring server. However, any Thrift
> receiver can be used as the publishing endpoint for Log publishing.
>
>
> Regards,
> Chamila de Alwis
> Software Engineer | WSO2 | +94772207163
> Blog: code.chamiladealwis.com
>
>
>
--
Imesh Gunaratne
Senior Technical Lead, WSO2
Committer & PMC Member, Apache Stratos