You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Lucas Ponce <po...@gmail.com> on 2017/08/17 14:18:39 UTC

Federated Alerting for Prometheus, Elasticsearch and Apache Kafka systems

Hello,


From Hawkular team we are working to create a lightweight alerting
component that can combine alerts from several systems like Prometheus,
Elasticsearch, Apache Kafka and others.


We would like to get feedback about these features and how we can improve
them to be useful for the Apache Kafka community.


The aim is to complement a gap where today an integration between systems
is mandatory.


We have prepared a blog post about this feature


http://www.hawkular.org/blog/2017/08/alerts-multiple-sources.html


And a demo


https://www.youtube.com/watch?v=mM1mwJneKO4


Please, let us know what do you think and if this could be interesting for
your projects.


Thanks !

Lucas Ponce

Re: Federated Alerting for Prometheus, Elasticsearch and Apache Kafka systems

Posted by Lucas Ponce <po...@gmail.com>.
On Fri, Aug 18, 2017 at 1:15 AM, Martin Gainty <mg...@hotmail.com> wrote:

>
>
> ________________________________
> From: Lucas Ponce <po...@gmail.com>
> Sent: Thursday, August 17, 2017 10:18 AM
> To: users@kafka.apache.org
> Subject: Federated Alerting for Prometheus, Elasticsearch and Apache Kafka
> systems
>
> Hello,
>
>
> From Hawkular team we are working to create a lightweight alerting
> component that can combine alerts from several systems like Prometheus,
> Elasticsearch, Apache Kafka and others.
>
>
> We would like to get feedback about these features and how we can improve
> them to be useful for the Apache Kafka community.
>
>
> The aim is to complement a gap where today an integration between systems
> is mandatory.
>
>
> We have prepared a blog post about this feature
>
>
> http://www.hawkular.org/blog/2017/08/alerts-multiple-sources.html
>
>
> And a demo
>
>
> https://www.youtube.com/watch?v=mM1mwJneKO4
> [https://www.bing.com/th?id=OVP.fhWnIutMqryd7JFypCvDMAHgFo&pid=Api]<
> https://www.youtube.com/watch?v=mM1mwJneKO4>
>
> Hawkular Alerts Integration with Prometheus, ElasticSearch, and Kafka<
> https://www.youtube.com/watch?v=mM1mwJneKO4>
> www.youtube.com
> This is a quick demo illustrating how Hawkular Alerts can fire alerts and
> events and send notifications based on Prometheus metrics, ElasticSearch
> logs, and ...
>
>
>
>
>
> Please, let us know what do you think and if this could be interesting for
> your projects.
> MG>are there alerting systems either existing or being developed which are
> designed for federated environments?
>

We haven't found similar systems in the open source area that are focus on
the "federated" concept (we haven't considered comercial solutions yet),
meaning that include data from several sources is a primary feature.
At least, on the systems we are working, alerting systems are close to
their data storage, meaning that you would need to consolidate and move
your data into one to detect.

That is the gap we want to work on, being able to define conditions that
can trigger events/alerts on different sources without attachment of any
specific scheme/model/storage for your data.


> MG>assume you will need to reserve a port or range of ports?
>

Yes, basically Hawkular Alerting can be used through a REST query or a UI,
which defines a 8080 port by default (configurable) and an administrative
port.


> MG>are you considering publishing your schema so other messaging systems
> can post to your alerting system or are you using SMS?
>

Indeed, the API is public, so anyone can send data for evaluation, also the
architecture of alerters/actions is plugable.

A small intro is published here

https://www.slideshare.net/ponceballesteros/hawkular-alerting


> MG>any security requirements (if any)?
>

The model is multitenant, having a strong separation of models.

We are collecting requeriments about security use cases to decide if a
simple security design is enough or if it is necessary to include a third
party component (i.e. keycloak).

Some requeriment in mind ?

Thanks for your questions !

Re: Federated Alerting for Prometheus, Elasticsearch and Apache Kafka systems

Posted by Martin Gainty <mg...@hotmail.com>.

________________________________
From: Lucas Ponce <po...@gmail.com>
Sent: Thursday, August 17, 2017 10:18 AM
To: users@kafka.apache.org
Subject: Federated Alerting for Prometheus, Elasticsearch and Apache Kafka systems

Hello,


From Hawkular team we are working to create a lightweight alerting
component that can combine alerts from several systems like Prometheus,
Elasticsearch, Apache Kafka and others.


We would like to get feedback about these features and how we can improve
them to be useful for the Apache Kafka community.


The aim is to complement a gap where today an integration between systems
is mandatory.


We have prepared a blog post about this feature


http://www.hawkular.org/blog/2017/08/alerts-multiple-sources.html


And a demo


https://www.youtube.com/watch?v=mM1mwJneKO4
[https://www.bing.com/th?id=OVP.fhWnIutMqryd7JFypCvDMAHgFo&pid=Api]<https://www.youtube.com/watch?v=mM1mwJneKO4>

Hawkular Alerts Integration with Prometheus, ElasticSearch, and Kafka<https://www.youtube.com/watch?v=mM1mwJneKO4>
www.youtube.com
This is a quick demo illustrating how Hawkular Alerts can fire alerts and events and send notifications based on Prometheus metrics, ElasticSearch logs, and ...





Please, let us know what do you think and if this could be interesting for
your projects.
MG>are there alerting systems either existing or being developed which are designed for federated environments?
MG>assume you will need to reserve a port or range of ports?
MG>are you considering publishing your schema so other messaging systems can post to your alerting system or are you using SMS?
MG>any security requirements (if any)?

Thanks !

Lucas Ponce