You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stratos.apache.org by Imesh Gunaratne <im...@apache.org> on 2015/06/15 20:45:50 UTC

[Discuss] Support for Integrating Kibana with Stratos

Hi Devs,

I did some analysis on integrating Kibana [2] with Stratos. This is for
tracking and visualizing Health Statistics and Metering information.



*What is Kibana?*

Just to give a brief introduction to Kibana, it is a data visualizing tool
developed by Elastic [1] and released under Apache 2 license [4]. Kibana
runs on top of Elasticsearch [3]. Elasticsearch provides a highly efficient
way of storing and searching data in a distributed fashion.

Data can be published to Elasticsearch using a simple HTTP POST request in
JSON format. Once data is stored in Elasticsearch an attractive Dashboard
can be created in Kibana in few minutes. The above is a sample Dashboard I
created in a Kibana instance installed on my local machine.

In addition these two products Elastic provides another tool called
LogStash [5]. LogStash can be used for tracking logs of multiple servers in
a central location. Elastic call this product stack ELK.

*Proposal*

IMO we could add two extension points in Stratos to publish Health
Statistics and Metering information to Elasticsearch:

1. In CEP we can implement an Output Data Adapter to publish Health
Statistics to Elasticsearch.
2. In AS and CC we could introduce an interface for Metering Statistics
Publisher and implement it for both BAM and Elasticsearch.

I'm +1 for this approach, please share your thoughts on this. If everyone
agrees may be we could introduce this in a coming milestone release
(4.1.x). WDYT?

[1] https://www.elastic.co/
[2] https://www.elastic.co/products/kibana
[3] https://www.elastic.co/products/elasticsearch
[4] https://github.com/elastic/kibana/blob/master/LICENSE.md
[5] https://www.elastic.co/products/logstash

Thanks

-- 
Imesh Gunaratne

Senior Technical Lead, WSO2
Committer & PMC Member, Apache Stratos

Re: [Discuss] Support for Integrating Kibana with Stratos

Posted by Gayan Gunarathne <ga...@wso2.com>.
+1 . Great thoughts Imesh to integrate this for Stratos.

Seems integrating the Elasticsearch will be great addition to Stratos. It
also exposed some useful APIs which we can look at later.

I assume Kibana we need to provide the index pattern for visualize data. Is
it a one time configuration or can we add/modify that pattern anytime?

Thanks,
Gayan

On Tue, Jun 16, 2015 at 12:15 AM, Imesh Gunaratne <im...@apache.org> wrote:

> Hi Devs,
>
> I did some analysis on integrating Kibana [2] with Stratos. This is for
> tracking and visualizing Health Statistics and Metering information.
>
>
>
> *What is Kibana?*
>
> Just to give a brief introduction to Kibana, it is a data visualizing tool
> developed by Elastic [1] and released under Apache 2 license [4]. Kibana
> runs on top of Elasticsearch [3]. Elasticsearch provides a highly efficient
> way of storing and searching data in a distributed fashion.
>
> Data can be published to Elasticsearch using a simple HTTP POST request in
> JSON format. Once data is stored in Elasticsearch an attractive Dashboard
> can be created in Kibana in few minutes. The above is a sample Dashboard I
> created in a Kibana instance installed on my local machine.
>
> In addition these two products Elastic provides another tool called
> LogStash [5]. LogStash can be used for tracking logs of multiple servers in
> a central location. Elastic call this product stack ELK.
>
> *Proposal*
>
> IMO we could add two extension points in Stratos to publish Health
> Statistics and Metering information to Elasticsearch:
>
> 1. In CEP we can implement an Output Data Adapter to publish Health
> Statistics to Elasticsearch.
> 2. In AS and CC we could introduce an interface for Metering Statistics
> Publisher and implement it for both BAM and Elasticsearch.
>
> I'm +1 for this approach, please share your thoughts on this. If everyone
> agrees may be we could introduce this in a coming milestone release
> (4.1.x). WDYT?
>
> [1] https://www.elastic.co/
> [2] https://www.elastic.co/products/kibana
> [3] https://www.elastic.co/products/elasticsearch
> [4] https://github.com/elastic/kibana/blob/master/LICENSE.md
> [5] https://www.elastic.co/products/logstash
>
> Thanks
>
> --
> Imesh Gunaratne
>
> Senior Technical Lead, WSO2
> Committer & PMC Member, Apache Stratos
>



-- 

Gayan Gunarathne
Technical Lead, WSO2 Inc. (http://wso2.com)
Committer & PMC Member, Apache Stratos
email : gayang@wso2.com  | mobile : +94 766819985

Re: [Discuss] Support for Integrating Kibana with Stratos

Posted by Anuruddha Liyanarachchi <an...@wso2.com>.
+1 For this approach. It will be easier to visualize what is going on in
side stratos.

On Tue, Jun 16, 2015 at 12:58 AM, Chamila De Alwis <ch...@wso2.com>
wrote:

> +1 ELK seems to be getting really popular, and supporting it would be a
> plus point.
>
>
> Regards,
> Chamila de Alwis
> Committer and PMC Member - Apache Stratos
> Software Engineer | WSO2 | +94772207163
> Blog: code.chamiladealwis.com
>
>
>
> On Mon, Jun 15, 2015 at 2:56 PM, Lahiru Sandaruwan <la...@wso2.com>
> wrote:
>
>> Great tool. +1 for enabling this support.
>>
>> Thanks.
>>
>> On Tue, Jun 16, 2015 at 12:22 AM, Lakmal Warusawithana <la...@wso2.com>
>> wrote:
>>
>>> +1 imesh
>>>
>>>
>>> On Monday, June 15, 2015, Imesh Gunaratne <im...@apache.org> wrote:
>>>
>>>> Hi Devs,
>>>>
>>>> I did some analysis on integrating Kibana [2] with Stratos. This is for
>>>> tracking and visualizing Health Statistics and Metering information.
>>>>
>>>>
>>>>
>>>> *What is Kibana?*
>>>>
>>>> Just to give a brief introduction to Kibana, it is a data visualizing
>>>> tool developed by Elastic [1] and released under Apache 2 license [4].
>>>> Kibana runs on top of Elasticsearch [3]. Elasticsearch provides a highly
>>>> efficient way of storing and searching data in a distributed fashion.
>>>>
>>>> Data can be published to Elasticsearch using a simple HTTP POST request
>>>> in JSON format. Once data is stored in Elasticsearch an attractive
>>>> Dashboard can be created in Kibana in few minutes. The above is a sample
>>>> Dashboard I created in a Kibana instance installed on my local machine.
>>>>
>>>> In addition these two products Elastic provides another tool called
>>>> LogStash [5]. LogStash can be used for tracking logs of multiple servers in
>>>> a central location. Elastic call this product stack ELK.
>>>>
>>>> *Proposal*
>>>>
>>>> IMO we could add two extension points in Stratos to publish Health
>>>> Statistics and Metering information to Elasticsearch:
>>>>
>>>> 1. In CEP we can implement an Output Data Adapter to publish Health
>>>> Statistics to Elasticsearch.
>>>> 2. In AS and CC we could introduce an interface for Metering Statistics
>>>> Publisher and implement it for both BAM and Elasticsearch.
>>>>
>>>> I'm +1 for this approach, please share your thoughts on this. If
>>>> everyone agrees may be we could introduce this in a coming milestone
>>>> release (4.1.x). WDYT?
>>>>
>>>> [1] https://www.elastic.co/
>>>> [2] https://www.elastic.co/products/kibana
>>>> [3] https://www.elastic.co/products/elasticsearch
>>>> [4] https://github.com/elastic/kibana/blob/master/LICENSE.md
>>>> [5] https://www.elastic.co/products/logstash
>>>>
>>>> Thanks
>>>>
>>>> --
>>>> Imesh Gunaratne
>>>>
>>>> Senior Technical Lead, WSO2
>>>> Committer & PMC Member, Apache Stratos
>>>>
>>>
>>>
>>> --
>>> Sent from Gmail Mobile
>>>
>>
>>
>>
>> --
>> --
>> Lahiru Sandaruwan
>> Committer and PMC member, Apache Stratos,
>> Senior Software Engineer,
>> WSO2 Inc., http://wso2.com
>> lean.enterprise.middleware
>>
>> phone: +94773325954
>> email: lahirus@wso2.com blog: http://lahiruwrites.blogspot.com/
>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>
>>
>


-- 
*Thanks and Regards,*
Anuruddha Lanka Liyanarachchi
Software Engineer - WSO2
Mobile : +94 (0) 712762611
Tel      : +94 112 145 345
a <th...@wso2.com>nuruddhal@wso2.com

Re: [Discuss] Support for Integrating Kibana with Stratos

Posted by Chamila De Alwis <ch...@wso2.com>.
+1 ELK seems to be getting really popular, and supporting it would be a
plus point.


Regards,
Chamila de Alwis
Committer and PMC Member - Apache Stratos
Software Engineer | WSO2 | +94772207163
Blog: code.chamiladealwis.com



On Mon, Jun 15, 2015 at 2:56 PM, Lahiru Sandaruwan <la...@wso2.com> wrote:

> Great tool. +1 for enabling this support.
>
> Thanks.
>
> On Tue, Jun 16, 2015 at 12:22 AM, Lakmal Warusawithana <la...@wso2.com>
> wrote:
>
>> +1 imesh
>>
>>
>> On Monday, June 15, 2015, Imesh Gunaratne <im...@apache.org> wrote:
>>
>>> Hi Devs,
>>>
>>> I did some analysis on integrating Kibana [2] with Stratos. This is for
>>> tracking and visualizing Health Statistics and Metering information.
>>>
>>>
>>>
>>> *What is Kibana?*
>>>
>>> Just to give a brief introduction to Kibana, it is a data visualizing
>>> tool developed by Elastic [1] and released under Apache 2 license [4].
>>> Kibana runs on top of Elasticsearch [3]. Elasticsearch provides a highly
>>> efficient way of storing and searching data in a distributed fashion.
>>>
>>> Data can be published to Elasticsearch using a simple HTTP POST request
>>> in JSON format. Once data is stored in Elasticsearch an attractive
>>> Dashboard can be created in Kibana in few minutes. The above is a sample
>>> Dashboard I created in a Kibana instance installed on my local machine.
>>>
>>> In addition these two products Elastic provides another tool called
>>> LogStash [5]. LogStash can be used for tracking logs of multiple servers in
>>> a central location. Elastic call this product stack ELK.
>>>
>>> *Proposal*
>>>
>>> IMO we could add two extension points in Stratos to publish Health
>>> Statistics and Metering information to Elasticsearch:
>>>
>>> 1. In CEP we can implement an Output Data Adapter to publish Health
>>> Statistics to Elasticsearch.
>>> 2. In AS and CC we could introduce an interface for Metering Statistics
>>> Publisher and implement it for both BAM and Elasticsearch.
>>>
>>> I'm +1 for this approach, please share your thoughts on this. If
>>> everyone agrees may be we could introduce this in a coming milestone
>>> release (4.1.x). WDYT?
>>>
>>> [1] https://www.elastic.co/
>>> [2] https://www.elastic.co/products/kibana
>>> [3] https://www.elastic.co/products/elasticsearch
>>> [4] https://github.com/elastic/kibana/blob/master/LICENSE.md
>>> [5] https://www.elastic.co/products/logstash
>>>
>>> Thanks
>>>
>>> --
>>> Imesh Gunaratne
>>>
>>> Senior Technical Lead, WSO2
>>> Committer & PMC Member, Apache Stratos
>>>
>>
>>
>> --
>> Sent from Gmail Mobile
>>
>
>
>
> --
> --
> Lahiru Sandaruwan
> Committer and PMC member, Apache Stratos,
> Senior Software Engineer,
> WSO2 Inc., http://wso2.com
> lean.enterprise.middleware
>
> phone: +94773325954
> email: lahirus@wso2.com blog: http://lahiruwrites.blogspot.com/
> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>
>

Re: [Discuss] Support for Integrating Kibana with Stratos

Posted by Lahiru Sandaruwan <la...@wso2.com>.
Great tool. +1 for enabling this support.

Thanks.

On Tue, Jun 16, 2015 at 12:22 AM, Lakmal Warusawithana <la...@wso2.com>
wrote:

> +1 imesh
>
>
> On Monday, June 15, 2015, Imesh Gunaratne <im...@apache.org> wrote:
>
>> Hi Devs,
>>
>> I did some analysis on integrating Kibana [2] with Stratos. This is for
>> tracking and visualizing Health Statistics and Metering information.
>>
>>
>>
>> *What is Kibana?*
>>
>> Just to give a brief introduction to Kibana, it is a data visualizing
>> tool developed by Elastic [1] and released under Apache 2 license [4].
>> Kibana runs on top of Elasticsearch [3]. Elasticsearch provides a highly
>> efficient way of storing and searching data in a distributed fashion.
>>
>> Data can be published to Elasticsearch using a simple HTTP POST request
>> in JSON format. Once data is stored in Elasticsearch an attractive
>> Dashboard can be created in Kibana in few minutes. The above is a sample
>> Dashboard I created in a Kibana instance installed on my local machine.
>>
>> In addition these two products Elastic provides another tool called
>> LogStash [5]. LogStash can be used for tracking logs of multiple servers in
>> a central location. Elastic call this product stack ELK.
>>
>> *Proposal*
>>
>> IMO we could add two extension points in Stratos to publish Health
>> Statistics and Metering information to Elasticsearch:
>>
>> 1. In CEP we can implement an Output Data Adapter to publish Health
>> Statistics to Elasticsearch.
>> 2. In AS and CC we could introduce an interface for Metering Statistics
>> Publisher and implement it for both BAM and Elasticsearch.
>>
>> I'm +1 for this approach, please share your thoughts on this. If everyone
>> agrees may be we could introduce this in a coming milestone release
>> (4.1.x). WDYT?
>>
>> [1] https://www.elastic.co/
>> [2] https://www.elastic.co/products/kibana
>> [3] https://www.elastic.co/products/elasticsearch
>> [4] https://github.com/elastic/kibana/blob/master/LICENSE.md
>> [5] https://www.elastic.co/products/logstash
>>
>> Thanks
>>
>> --
>> Imesh Gunaratne
>>
>> Senior Technical Lead, WSO2
>> Committer & PMC Member, Apache Stratos
>>
>
>
> --
> Sent from Gmail Mobile
>



-- 
--
Lahiru Sandaruwan
Committer and PMC member, Apache Stratos,
Senior Software Engineer,
WSO2 Inc., http://wso2.com
lean.enterprise.middleware

phone: +94773325954
email: lahirus@wso2.com blog: http://lahiruwrites.blogspot.com/
linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146

Re: [Discuss] Support for Integrating Kibana with Stratos

Posted by Lakmal Warusawithana <la...@wso2.com>.
+1 imesh

On Monday, June 15, 2015, Imesh Gunaratne <im...@apache.org> wrote:

> Hi Devs,
>
> I did some analysis on integrating Kibana [2] with Stratos. This is for
> tracking and visualizing Health Statistics and Metering information.
>
>
>
> *What is Kibana?*
>
> Just to give a brief introduction to Kibana, it is a data visualizing tool
> developed by Elastic [1] and released under Apache 2 license [4]. Kibana
> runs on top of Elasticsearch [3]. Elasticsearch provides a highly efficient
> way of storing and searching data in a distributed fashion.
>
> Data can be published to Elasticsearch using a simple HTTP POST request in
> JSON format. Once data is stored in Elasticsearch an attractive Dashboard
> can be created in Kibana in few minutes. The above is a sample Dashboard I
> created in a Kibana instance installed on my local machine.
>
> In addition these two products Elastic provides another tool called
> LogStash [5]. LogStash can be used for tracking logs of multiple servers in
> a central location. Elastic call this product stack ELK.
>
> *Proposal*
>
> IMO we could add two extension points in Stratos to publish Health
> Statistics and Metering information to Elasticsearch:
>
> 1. In CEP we can implement an Output Data Adapter to publish Health
> Statistics to Elasticsearch.
> 2. In AS and CC we could introduce an interface for Metering Statistics
> Publisher and implement it for both BAM and Elasticsearch.
>
> I'm +1 for this approach, please share your thoughts on this. If everyone
> agrees may be we could introduce this in a coming milestone release
> (4.1.x). WDYT?
>
> [1] https://www.elastic.co/
> [2] https://www.elastic.co/products/kibana
> [3] https://www.elastic.co/products/elasticsearch
> [4] https://github.com/elastic/kibana/blob/master/LICENSE.md
> [5] https://www.elastic.co/products/logstash
>
> Thanks
>
> --
> Imesh Gunaratne
>
> Senior Technical Lead, WSO2
> Committer & PMC Member, Apache Stratos
>


-- 
Sent from Gmail Mobile