You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@streampipes.apache.org by Grainier Perera <gr...@apache.org> on 2020/05/26 13:25:08 UTC
Telegram publisher to send event notifications
Hi all,
I've implemented a publisher to send notifications to a Telegram channel.
Since Telegram[1] is a free and cross-platform application, I think this
will be a good addition to notification sinks. What do you think?
Issue: https://issues.apache.org/jira/browse/STREAMPIPES-144
PR: https://github.com/apache/incubator-streampipes-extensions/pull/16
[1] https://telegram.org/
Thanks,
Grainier.
Re: Telegram publisher to send event notifications
Posted by Philipp Zehnder <ze...@apache.org>.
Hi,
I really like the ideas that came up for the rate limiter and thanks for opening the issue.
Philipp
> On 28. May 2020, at 06:42, Grainier Perera <gr...@apache.org> wrote:
>
> We'll start with a basic implementation and improve further. I've also
> created issues to track the progress.
> I'll start working on the [1] and update [2] as time permits.
>
> [1] RateLimitter: https://issues.apache.org/jira/browse/STREAMPIPES-147
> [2] Siddhi Processor improvements:
> https://issues.apache.org/jira/browse/STREAMPIPES-148
>
> Grainier Perera.
>
>
> On Wed, 27 May 2020 at 23:25, Patrick Wiener <wi...@apache.org> wrote:
>
>> also like the idea to check how others are tackling such scenarios. maybe
>> we can
>> adapt and improve from potential suitable designs.
>>
>> other than that I’m with Dominik to start with a basic functionality for a
>> rate limiting processor
>> and further extend it.
>>
>> @Grainier: true, in Siddhi this would also include having multiple „steps“
>> or queries.
>> I still think it’s worth extending our current toolbox with new Siddhi
>> processors to also
>> support more lightweight deployments, e.g. the aggregation processor [1]
>> is currently only
>> available in flink - but that’s another topic :)
>>
>> Patrick
>>
>> [1]
>> https://github.com/apache/incubator-streampipes-extensions/tree/dev/streampipes-processors-aggregation-flink/src/main/java/org/apache/streampipes/processors/aggregation/flink/processor/aggregation
>> <
>> https://github.com/apache/incubator-streampipes-extensions/tree/dev/streampipes-processors-aggregation-flink/src/main/java/org/apache/streampipes/processors/aggregation/flink/processor/aggregation
>>>
>>
>>> Am 27.05.2020 um 19:08 schrieb Dominik Riemer <ri...@apache.org>:
>>>
>>> Hi,
>>>
>>> +1 for a new pipeline element/standalone feature to rate limit events.
>>>
>>> Just as an idea, in a version 2 or 3 we could also extend this further:
>>> - allow users to configure to not only emit the first or last event, but
>> also to perform some aggregation function before emitting, e.g., output the
>> average value from a 15-minute time window.
>>> - output the event at fixed intervals (e.g., every full hour)
>>>
>>> @Grainier, great I think it's a good idea to analyze how other tools are
>> doing this!
>>>
>>> Dominik
>>>
>>> -----Original Message-----
>>> From: Grainier Perera <gr...@apache.org>
>>> Sent: Wednesday, May 27, 2020 2:01 PM
>>> To: dev@streampipes.apache.org
>>> Subject: Re: Telegram publisher to send event notifications
>>>
>>> Hi all,
>>>
>>> I also think it should be a standalone feature. And the RateLimiter
>> should allow users to configure;
>>>
>>> - a window (time or count)
>>> - partitioning field (because there can be many sensors sending data to
>>> the same stream)
>>> - event to emit (first or last event of the window)
>>> - retry logic. (something like; if there are no new events on the
>>> current window, emit the last event of the previous window)
>>> - etc...
>>>
>>> So we can use that with existing filters, sequences (which we can
>> improve for event change capture), and notification sinks to tackle most of
>> the above scenarios.
>>> Even with Siddhi, we have to use several queries to create a pipeline
>> and achieve the same. i.e;
>>>
>>> define stream TemperatureStream (sensorId string, temperature float,
>>>> timestamp int);
>>>
>>>
>>>> -- Filter[1] to capture tempStatus
>>>> from TemperatureStream
>>>> select sensorId, ifThenElse((temperature < -10 || 50 < temperature),
>>>> "abnormal", "normal") as as tempStatus insert into
>>>> ProcessedTemperatureStream;
>>>
>>>
>>>> -- Sequence[2] to tempStatus change (realtime) from every
>>>> e1=ProcessedTemperatureStream,
>>>> e2=ProcessedTemperatureStream[e1.tempStatus != tempStatus] group by
>>>> sensorId select e1.sensorId, e2.tempStatus insert into
>>>> NotificationStream;
>>>
>>>
>>>> -- Ratelimit[3] to send only the last notification per 15 min
>>>> (somewhat
>>>> batching)
>>>> from NotificationStream
>>>> select *
>>>> group by sensorId
>>>> output last every 15 min
>>>> insert into RateLimitedNotificationStream;
>>>
>>>
>>> Further, I'll do some research on how similar products have done this,
>> and update the thread with how we can implement the same with StreamPipes.
>>> What do you think?
>>>
>>> [1] https://siddhi.io/en/v5.1/docs/examples/if-then-else/
>>> [2] https://siddhi.io/en/v5.1/docs/examples/simple-sequence/
>>> [3] https://siddhi.io/en/v5.1/docs/examples/time-rate-limit/
>>>
>>> Grainier Perera.
>>>
>>>
>>> On Wed, 27 May 2020 at 14:02, Patrick Wiener <wi...@apache.org> wrote:
>>>
>>>> I think this should not be a feature included in the notification sinks.
>>>> Rather it should be part of the dedicated PE either as a „standalone“
>>>> feature such as a rate limiter or as part of the application logic
>>>> itself.
>>>>
>>>> For the latter, lets consider a CEP like use case where we want to
>>>> monitor sensor temperature in a given range, i.e. where there exists a
>>>> „normal“ range (e.g. 50+-10°C). Everything below or above would be
>>>> abnormal and should throw a warning.
>>>>
>>>> Now the question is, when is that warning thrown and how often? And
>>>> can the warning be reset e.g. by getting back in the normal range
>>>> within a given time/count window?
>>>>
>>>> These involve questions in how to design such a pattern or pattern
>>>> sequences, how to handle consecutive occurrences of events satisfying
>>>> the pattern.
>>>>
>>>> Now we could imagine various output options:
>>>>
>>>> 1) output warning once and never again
>>>> 2) output warning and wait for „reset“ of warning state, e.g.
>>>> temperature back in normal range for a given time
>>>> 2) output warning and set a timeout, after timeout if pattern still
>>>> satisfied output another reminder warning event
>>>> 3) output warning every time pattern is satisfied
>>>>
>>>> IMO, these choices greatly depend on the individual users. However, we
>>>> should somehow provide her/him with possible options to suit their use
>>>> case and prevent spamming notifications.
>>>>
>>>> @Grainier: would such a scenario be possible to implement out of the
>>>> box using Siddhi?
>>>>
>>>> For all other simple scenarios, I guess an output rate-limiter giving
>>>> the user a simple way of „reducing“ the messages send (e.g. send 1 of
>>>> 10 events) out to the notification sink (Slack, Telegram etc) should
>>>> be feasible.
>>>>
>>>> Any other thoughts?
>>>>
>>>> Patrick
>>>>
>>>>
>>>>
>>>>> Am 27.05.2020 um 06:54 schrieb Philipp Zehnder <ze...@apache.org>:
>>>>>
>>>>> I agree, maybe it is better to keep the notification-sinks as simple
>>>>> as
>>>> possible.
>>>>> Then users could also use multiple channels (e.g. Email, Slack, and
>>>> Telegram) for their notification and have a single output rate-limiter
>>>> in the pipeline.
>>>>> I was just wondering, if that’s a feature a user would expect in
>>>>> each
>>>> data sink or if this is an additional functionality for a separate
>>>> pipeline element?
>>>>>
>>>>> Which functionalities would such a rate-limiter need?
>>>>> I think it should not ‘just’ remove events, e.g. when a user is not
>>>> reacting to the situation we might have to send the notification again
>>>> after some time.
>>>>> Any thoughts on that?
>>>>>
>>>>> Philipp
>>>>>
>>>>>
>>>>>> On 26. May 2020, at 19:11, Grainier Perera
>>>>>> <gr...@gmail.com>
>>>> wrote:
>>>>>>
>>>>>> IMO, the sink shouldn't bother which events to publish. Instead, we
>>>>>> can have event change capture & output rate-limiting as separate
>>>>>> pipeline elements which can be used with notification sinks.
>>>> So it
>>>>>> gives users the flexibility to limit events or not, depending on
>>>>>> their use-case. What do you think?
>>>>>>
>>>>>> pipeline: [event-source -> output rate-limiter ->
>>>>>> notification-sink]
>>>>>>
>>>>>> Grainier.
>>>>>>
>>>>>> On Tue, 26 May 2020 at 21:47, Philipp Zehnder <ze...@apache.org>
>>>> wrote:
>>>>>>
>>>>>>> Hi Grainier,
>>>>>>>
>>>>>>> this sink is very cool and useful. I directly merged it.
>>>>>>>
>>>>>>> @all I think we still have a little conceptual problem with our
>>>>>>> notification sinks (e.g. Notification, Email, …) Currently we emit
>>>>>>> a notification for each event, even if the event did
>>>> not
>>>>>>> change. This could annoy users and also be counter productive as
>>>>>>> you
>>>> don't
>>>>>>> want to receive hundreds of messages.
>>>>>>> We need a solution to set the maximum number of notification or
>>>>>>> just notify the user when the situation changed.
>>>>>>> Any ideas on that?
>>>>>>>
>>>>>>> Philipp
>>>>>>>
>>>>>>>> On 26. May 2020, at 16:28, Patrick Wiener <wi...@apache.org>
>> wrote:
>>>>>>>>
>>>>>>>> Hi Grainier,
>>>>>>>>
>>>>>>>> this is really nice. We were at some point actually talking about
>>>> such a
>>>>>>> sink so super cool
>>>>>>>> that you implemented it. I used telegram once in another project
>>>>>>>> where
>>>>>>> one was informed
>>>>>>>> about alerts/warnings and quite liked it.
>>>>>>>>
>>>>>>>> Thanks for the contribution and valuable work.
>>>>>>>>
>>>>>>>> Patrick
>>>>>>>>
>>>>>>>>
>>>>>>>>> Am 26.05.2020 um 15:25 schrieb Grainier Perera
>>>>>>>>> <grainier@apache.org
>>>>> :
>>>>>>>>>
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> I've implemented a publisher to send notifications to a Telegram
>>>>>>> channel.
>>>>>>>>> Since Telegram[1] is a free and cross-platform application, I
>>>>>>>>> think
>>>> this
>>>>>>>>> will be a good addition to notification sinks. What do you think?
>>>>>>>>>
>>>>>>>>> Issue: https://issues.apache.org/jira/browse/STREAMPIPES-144
>>>>>>>>> PR:
>>>> https://github.com/apache/incubator-streampipes-extensions/pull/16
>>>>>>>>>
>>>>>>>>> [1] https://telegram.org/
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Grainier.
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
Re: Telegram publisher to send event notifications
Posted by Grainier Perera <gr...@apache.org>.
We'll start with a basic implementation and improve further. I've also
created issues to track the progress.
I'll start working on the [1] and update [2] as time permits.
[1] RateLimitter: https://issues.apache.org/jira/browse/STREAMPIPES-147
[2] Siddhi Processor improvements:
https://issues.apache.org/jira/browse/STREAMPIPES-148
Grainier Perera.
On Wed, 27 May 2020 at 23:25, Patrick Wiener <wi...@apache.org> wrote:
> also like the idea to check how others are tackling such scenarios. maybe
> we can
> adapt and improve from potential suitable designs.
>
> other than that I’m with Dominik to start with a basic functionality for a
> rate limiting processor
> and further extend it.
>
> @Grainier: true, in Siddhi this would also include having multiple „steps“
> or queries.
> I still think it’s worth extending our current toolbox with new Siddhi
> processors to also
> support more lightweight deployments, e.g. the aggregation processor [1]
> is currently only
> available in flink - but that’s another topic :)
>
> Patrick
>
> [1]
> https://github.com/apache/incubator-streampipes-extensions/tree/dev/streampipes-processors-aggregation-flink/src/main/java/org/apache/streampipes/processors/aggregation/flink/processor/aggregation
> <
> https://github.com/apache/incubator-streampipes-extensions/tree/dev/streampipes-processors-aggregation-flink/src/main/java/org/apache/streampipes/processors/aggregation/flink/processor/aggregation
> >
>
> > Am 27.05.2020 um 19:08 schrieb Dominik Riemer <ri...@apache.org>:
> >
> > Hi,
> >
> > +1 for a new pipeline element/standalone feature to rate limit events.
> >
> > Just as an idea, in a version 2 or 3 we could also extend this further:
> > - allow users to configure to not only emit the first or last event, but
> also to perform some aggregation function before emitting, e.g., output the
> average value from a 15-minute time window.
> > - output the event at fixed intervals (e.g., every full hour)
> >
> > @Grainier, great I think it's a good idea to analyze how other tools are
> doing this!
> >
> > Dominik
> >
> > -----Original Message-----
> > From: Grainier Perera <gr...@apache.org>
> > Sent: Wednesday, May 27, 2020 2:01 PM
> > To: dev@streampipes.apache.org
> > Subject: Re: Telegram publisher to send event notifications
> >
> > Hi all,
> >
> > I also think it should be a standalone feature. And the RateLimiter
> should allow users to configure;
> >
> > - a window (time or count)
> > - partitioning field (because there can be many sensors sending data to
> > the same stream)
> > - event to emit (first or last event of the window)
> > - retry logic. (something like; if there are no new events on the
> > current window, emit the last event of the previous window)
> > - etc...
> >
> > So we can use that with existing filters, sequences (which we can
> improve for event change capture), and notification sinks to tackle most of
> the above scenarios.
> > Even with Siddhi, we have to use several queries to create a pipeline
> and achieve the same. i.e;
> >
> > define stream TemperatureStream (sensorId string, temperature float,
> >> timestamp int);
> >
> >
> >> -- Filter[1] to capture tempStatus
> >> from TemperatureStream
> >> select sensorId, ifThenElse((temperature < -10 || 50 < temperature),
> >> "abnormal", "normal") as as tempStatus insert into
> >> ProcessedTemperatureStream;
> >
> >
> >> -- Sequence[2] to tempStatus change (realtime) from every
> >> e1=ProcessedTemperatureStream,
> >> e2=ProcessedTemperatureStream[e1.tempStatus != tempStatus] group by
> >> sensorId select e1.sensorId, e2.tempStatus insert into
> >> NotificationStream;
> >
> >
> >> -- Ratelimit[3] to send only the last notification per 15 min
> >> (somewhat
> >> batching)
> >> from NotificationStream
> >> select *
> >> group by sensorId
> >> output last every 15 min
> >> insert into RateLimitedNotificationStream;
> >
> >
> > Further, I'll do some research on how similar products have done this,
> and update the thread with how we can implement the same with StreamPipes.
> > What do you think?
> >
> > [1] https://siddhi.io/en/v5.1/docs/examples/if-then-else/
> > [2] https://siddhi.io/en/v5.1/docs/examples/simple-sequence/
> > [3] https://siddhi.io/en/v5.1/docs/examples/time-rate-limit/
> >
> > Grainier Perera.
> >
> >
> > On Wed, 27 May 2020 at 14:02, Patrick Wiener <wi...@apache.org> wrote:
> >
> >> I think this should not be a feature included in the notification sinks.
> >> Rather it should be part of the dedicated PE either as a „standalone“
> >> feature such as a rate limiter or as part of the application logic
> >> itself.
> >>
> >> For the latter, lets consider a CEP like use case where we want to
> >> monitor sensor temperature in a given range, i.e. where there exists a
> >> „normal“ range (e.g. 50+-10°C). Everything below or above would be
> >> abnormal and should throw a warning.
> >>
> >> Now the question is, when is that warning thrown and how often? And
> >> can the warning be reset e.g. by getting back in the normal range
> >> within a given time/count window?
> >>
> >> These involve questions in how to design such a pattern or pattern
> >> sequences, how to handle consecutive occurrences of events satisfying
> >> the pattern.
> >>
> >> Now we could imagine various output options:
> >>
> >> 1) output warning once and never again
> >> 2) output warning and wait for „reset“ of warning state, e.g.
> >> temperature back in normal range for a given time
> >> 2) output warning and set a timeout, after timeout if pattern still
> >> satisfied output another reminder warning event
> >> 3) output warning every time pattern is satisfied
> >>
> >> IMO, these choices greatly depend on the individual users. However, we
> >> should somehow provide her/him with possible options to suit their use
> >> case and prevent spamming notifications.
> >>
> >> @Grainier: would such a scenario be possible to implement out of the
> >> box using Siddhi?
> >>
> >> For all other simple scenarios, I guess an output rate-limiter giving
> >> the user a simple way of „reducing“ the messages send (e.g. send 1 of
> >> 10 events) out to the notification sink (Slack, Telegram etc) should
> >> be feasible.
> >>
> >> Any other thoughts?
> >>
> >> Patrick
> >>
> >>
> >>
> >>> Am 27.05.2020 um 06:54 schrieb Philipp Zehnder <ze...@apache.org>:
> >>>
> >>> I agree, maybe it is better to keep the notification-sinks as simple
> >>> as
> >> possible.
> >>> Then users could also use multiple channels (e.g. Email, Slack, and
> >> Telegram) for their notification and have a single output rate-limiter
> >> in the pipeline.
> >>> I was just wondering, if that’s a feature a user would expect in
> >>> each
> >> data sink or if this is an additional functionality for a separate
> >> pipeline element?
> >>>
> >>> Which functionalities would such a rate-limiter need?
> >>> I think it should not ‘just’ remove events, e.g. when a user is not
> >> reacting to the situation we might have to send the notification again
> >> after some time.
> >>> Any thoughts on that?
> >>>
> >>> Philipp
> >>>
> >>>
> >>>> On 26. May 2020, at 19:11, Grainier Perera
> >>>> <gr...@gmail.com>
> >> wrote:
> >>>>
> >>>> IMO, the sink shouldn't bother which events to publish. Instead, we
> >>>> can have event change capture & output rate-limiting as separate
> >>>> pipeline elements which can be used with notification sinks.
> >> So it
> >>>> gives users the flexibility to limit events or not, depending on
> >>>> their use-case. What do you think?
> >>>>
> >>>> pipeline: [event-source -> output rate-limiter ->
> >>>> notification-sink]
> >>>>
> >>>> Grainier.
> >>>>
> >>>> On Tue, 26 May 2020 at 21:47, Philipp Zehnder <ze...@apache.org>
> >> wrote:
> >>>>
> >>>>> Hi Grainier,
> >>>>>
> >>>>> this sink is very cool and useful. I directly merged it.
> >>>>>
> >>>>> @all I think we still have a little conceptual problem with our
> >>>>> notification sinks (e.g. Notification, Email, …) Currently we emit
> >>>>> a notification for each event, even if the event did
> >> not
> >>>>> change. This could annoy users and also be counter productive as
> >>>>> you
> >> don't
> >>>>> want to receive hundreds of messages.
> >>>>> We need a solution to set the maximum number of notification or
> >>>>> just notify the user when the situation changed.
> >>>>> Any ideas on that?
> >>>>>
> >>>>> Philipp
> >>>>>
> >>>>>> On 26. May 2020, at 16:28, Patrick Wiener <wi...@apache.org>
> wrote:
> >>>>>>
> >>>>>> Hi Grainier,
> >>>>>>
> >>>>>> this is really nice. We were at some point actually talking about
> >> such a
> >>>>> sink so super cool
> >>>>>> that you implemented it. I used telegram once in another project
> >>>>>> where
> >>>>> one was informed
> >>>>>> about alerts/warnings and quite liked it.
> >>>>>>
> >>>>>> Thanks for the contribution and valuable work.
> >>>>>>
> >>>>>> Patrick
> >>>>>>
> >>>>>>
> >>>>>>> Am 26.05.2020 um 15:25 schrieb Grainier Perera
> >>>>>>> <grainier@apache.org
> >>> :
> >>>>>>>
> >>>>>>> Hi all,
> >>>>>>>
> >>>>>>> I've implemented a publisher to send notifications to a Telegram
> >>>>> channel.
> >>>>>>> Since Telegram[1] is a free and cross-platform application, I
> >>>>>>> think
> >> this
> >>>>>>> will be a good addition to notification sinks. What do you think?
> >>>>>>>
> >>>>>>> Issue: https://issues.apache.org/jira/browse/STREAMPIPES-144
> >>>>>>> PR:
> >> https://github.com/apache/incubator-streampipes-extensions/pull/16
> >>>>>>>
> >>>>>>> [1] https://telegram.org/
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Grainier.
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>
> >>>
> >>
> >>
> >
>
>
Re: Telegram publisher to send event notifications
Posted by Patrick Wiener <wi...@apache.org>.
also like the idea to check how others are tackling such scenarios. maybe we can
adapt and improve from potential suitable designs.
other than that I’m with Dominik to start with a basic functionality for a rate limiting processor
and further extend it.
@Grainier: true, in Siddhi this would also include having multiple „steps“ or queries.
I still think it’s worth extending our current toolbox with new Siddhi processors to also
support more lightweight deployments, e.g. the aggregation processor [1] is currently only
available in flink - but that’s another topic :)
Patrick
[1] https://github.com/apache/incubator-streampipes-extensions/tree/dev/streampipes-processors-aggregation-flink/src/main/java/org/apache/streampipes/processors/aggregation/flink/processor/aggregation <https://github.com/apache/incubator-streampipes-extensions/tree/dev/streampipes-processors-aggregation-flink/src/main/java/org/apache/streampipes/processors/aggregation/flink/processor/aggregation>
> Am 27.05.2020 um 19:08 schrieb Dominik Riemer <ri...@apache.org>:
>
> Hi,
>
> +1 for a new pipeline element/standalone feature to rate limit events.
>
> Just as an idea, in a version 2 or 3 we could also extend this further:
> - allow users to configure to not only emit the first or last event, but also to perform some aggregation function before emitting, e.g., output the average value from a 15-minute time window.
> - output the event at fixed intervals (e.g., every full hour)
>
> @Grainier, great I think it's a good idea to analyze how other tools are doing this!
>
> Dominik
>
> -----Original Message-----
> From: Grainier Perera <gr...@apache.org>
> Sent: Wednesday, May 27, 2020 2:01 PM
> To: dev@streampipes.apache.org
> Subject: Re: Telegram publisher to send event notifications
>
> Hi all,
>
> I also think it should be a standalone feature. And the RateLimiter should allow users to configure;
>
> - a window (time or count)
> - partitioning field (because there can be many sensors sending data to
> the same stream)
> - event to emit (first or last event of the window)
> - retry logic. (something like; if there are no new events on the
> current window, emit the last event of the previous window)
> - etc...
>
> So we can use that with existing filters, sequences (which we can improve for event change capture), and notification sinks to tackle most of the above scenarios.
> Even with Siddhi, we have to use several queries to create a pipeline and achieve the same. i.e;
>
> define stream TemperatureStream (sensorId string, temperature float,
>> timestamp int);
>
>
>> -- Filter[1] to capture tempStatus
>> from TemperatureStream
>> select sensorId, ifThenElse((temperature < -10 || 50 < temperature),
>> "abnormal", "normal") as as tempStatus insert into
>> ProcessedTemperatureStream;
>
>
>> -- Sequence[2] to tempStatus change (realtime) from every
>> e1=ProcessedTemperatureStream,
>> e2=ProcessedTemperatureStream[e1.tempStatus != tempStatus] group by
>> sensorId select e1.sensorId, e2.tempStatus insert into
>> NotificationStream;
>
>
>> -- Ratelimit[3] to send only the last notification per 15 min
>> (somewhat
>> batching)
>> from NotificationStream
>> select *
>> group by sensorId
>> output last every 15 min
>> insert into RateLimitedNotificationStream;
>
>
> Further, I'll do some research on how similar products have done this, and update the thread with how we can implement the same with StreamPipes.
> What do you think?
>
> [1] https://siddhi.io/en/v5.1/docs/examples/if-then-else/
> [2] https://siddhi.io/en/v5.1/docs/examples/simple-sequence/
> [3] https://siddhi.io/en/v5.1/docs/examples/time-rate-limit/
>
> Grainier Perera.
>
>
> On Wed, 27 May 2020 at 14:02, Patrick Wiener <wi...@apache.org> wrote:
>
>> I think this should not be a feature included in the notification sinks.
>> Rather it should be part of the dedicated PE either as a „standalone“
>> feature such as a rate limiter or as part of the application logic
>> itself.
>>
>> For the latter, lets consider a CEP like use case where we want to
>> monitor sensor temperature in a given range, i.e. where there exists a
>> „normal“ range (e.g. 50+-10°C). Everything below or above would be
>> abnormal and should throw a warning.
>>
>> Now the question is, when is that warning thrown and how often? And
>> can the warning be reset e.g. by getting back in the normal range
>> within a given time/count window?
>>
>> These involve questions in how to design such a pattern or pattern
>> sequences, how to handle consecutive occurrences of events satisfying
>> the pattern.
>>
>> Now we could imagine various output options:
>>
>> 1) output warning once and never again
>> 2) output warning and wait for „reset“ of warning state, e.g.
>> temperature back in normal range for a given time
>> 2) output warning and set a timeout, after timeout if pattern still
>> satisfied output another reminder warning event
>> 3) output warning every time pattern is satisfied
>>
>> IMO, these choices greatly depend on the individual users. However, we
>> should somehow provide her/him with possible options to suit their use
>> case and prevent spamming notifications.
>>
>> @Grainier: would such a scenario be possible to implement out of the
>> box using Siddhi?
>>
>> For all other simple scenarios, I guess an output rate-limiter giving
>> the user a simple way of „reducing“ the messages send (e.g. send 1 of
>> 10 events) out to the notification sink (Slack, Telegram etc) should
>> be feasible.
>>
>> Any other thoughts?
>>
>> Patrick
>>
>>
>>
>>> Am 27.05.2020 um 06:54 schrieb Philipp Zehnder <ze...@apache.org>:
>>>
>>> I agree, maybe it is better to keep the notification-sinks as simple
>>> as
>> possible.
>>> Then users could also use multiple channels (e.g. Email, Slack, and
>> Telegram) for their notification and have a single output rate-limiter
>> in the pipeline.
>>> I was just wondering, if that’s a feature a user would expect in
>>> each
>> data sink or if this is an additional functionality for a separate
>> pipeline element?
>>>
>>> Which functionalities would such a rate-limiter need?
>>> I think it should not ‘just’ remove events, e.g. when a user is not
>> reacting to the situation we might have to send the notification again
>> after some time.
>>> Any thoughts on that?
>>>
>>> Philipp
>>>
>>>
>>>> On 26. May 2020, at 19:11, Grainier Perera
>>>> <gr...@gmail.com>
>> wrote:
>>>>
>>>> IMO, the sink shouldn't bother which events to publish. Instead, we
>>>> can have event change capture & output rate-limiting as separate
>>>> pipeline elements which can be used with notification sinks.
>> So it
>>>> gives users the flexibility to limit events or not, depending on
>>>> their use-case. What do you think?
>>>>
>>>> pipeline: [event-source -> output rate-limiter ->
>>>> notification-sink]
>>>>
>>>> Grainier.
>>>>
>>>> On Tue, 26 May 2020 at 21:47, Philipp Zehnder <ze...@apache.org>
>> wrote:
>>>>
>>>>> Hi Grainier,
>>>>>
>>>>> this sink is very cool and useful. I directly merged it.
>>>>>
>>>>> @all I think we still have a little conceptual problem with our
>>>>> notification sinks (e.g. Notification, Email, …) Currently we emit
>>>>> a notification for each event, even if the event did
>> not
>>>>> change. This could annoy users and also be counter productive as
>>>>> you
>> don't
>>>>> want to receive hundreds of messages.
>>>>> We need a solution to set the maximum number of notification or
>>>>> just notify the user when the situation changed.
>>>>> Any ideas on that?
>>>>>
>>>>> Philipp
>>>>>
>>>>>> On 26. May 2020, at 16:28, Patrick Wiener <wi...@apache.org> wrote:
>>>>>>
>>>>>> Hi Grainier,
>>>>>>
>>>>>> this is really nice. We were at some point actually talking about
>> such a
>>>>> sink so super cool
>>>>>> that you implemented it. I used telegram once in another project
>>>>>> where
>>>>> one was informed
>>>>>> about alerts/warnings and quite liked it.
>>>>>>
>>>>>> Thanks for the contribution and valuable work.
>>>>>>
>>>>>> Patrick
>>>>>>
>>>>>>
>>>>>>> Am 26.05.2020 um 15:25 schrieb Grainier Perera
>>>>>>> <grainier@apache.org
>>> :
>>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> I've implemented a publisher to send notifications to a Telegram
>>>>> channel.
>>>>>>> Since Telegram[1] is a free and cross-platform application, I
>>>>>>> think
>> this
>>>>>>> will be a good addition to notification sinks. What do you think?
>>>>>>>
>>>>>>> Issue: https://issues.apache.org/jira/browse/STREAMPIPES-144
>>>>>>> PR:
>> https://github.com/apache/incubator-streampipes-extensions/pull/16
>>>>>>>
>>>>>>> [1] https://telegram.org/
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Grainier.
>>>>>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>
>>
>
RE: Telegram publisher to send event notifications
Posted by Dominik Riemer <ri...@apache.org>.
Hi,
+1 for a new pipeline element/standalone feature to rate limit events.
Just as an idea, in a version 2 or 3 we could also extend this further:
- allow users to configure to not only emit the first or last event, but also to perform some aggregation function before emitting, e.g., output the average value from a 15-minute time window.
- output the event at fixed intervals (e.g., every full hour)
@Grainier, great I think it's a good idea to analyze how other tools are doing this!
Dominik
-----Original Message-----
From: Grainier Perera <gr...@apache.org>
Sent: Wednesday, May 27, 2020 2:01 PM
To: dev@streampipes.apache.org
Subject: Re: Telegram publisher to send event notifications
Hi all,
I also think it should be a standalone feature. And the RateLimiter should allow users to configure;
- a window (time or count)
- partitioning field (because there can be many sensors sending data to
the same stream)
- event to emit (first or last event of the window)
- retry logic. (something like; if there are no new events on the
current window, emit the last event of the previous window)
- etc...
So we can use that with existing filters, sequences (which we can improve for event change capture), and notification sinks to tackle most of the above scenarios.
Even with Siddhi, we have to use several queries to create a pipeline and achieve the same. i.e;
define stream TemperatureStream (sensorId string, temperature float,
> timestamp int);
> -- Filter[1] to capture tempStatus
> from TemperatureStream
> select sensorId, ifThenElse((temperature < -10 || 50 < temperature),
> "abnormal", "normal") as as tempStatus insert into
> ProcessedTemperatureStream;
> -- Sequence[2] to tempStatus change (realtime) from every
> e1=ProcessedTemperatureStream,
> e2=ProcessedTemperatureStream[e1.tempStatus != tempStatus] group by
> sensorId select e1.sensorId, e2.tempStatus insert into
> NotificationStream;
> -- Ratelimit[3] to send only the last notification per 15 min
> (somewhat
> batching)
> from NotificationStream
> select *
> group by sensorId
> output last every 15 min
> insert into RateLimitedNotificationStream;
Further, I'll do some research on how similar products have done this, and update the thread with how we can implement the same with StreamPipes.
What do you think?
[1] https://siddhi.io/en/v5.1/docs/examples/if-then-else/
[2] https://siddhi.io/en/v5.1/docs/examples/simple-sequence/
[3] https://siddhi.io/en/v5.1/docs/examples/time-rate-limit/
Grainier Perera.
On Wed, 27 May 2020 at 14:02, Patrick Wiener <wi...@apache.org> wrote:
> I think this should not be a feature included in the notification sinks.
> Rather it should be part of the dedicated PE either as a „standalone“
> feature such as a rate limiter or as part of the application logic
> itself.
>
> For the latter, lets consider a CEP like use case where we want to
> monitor sensor temperature in a given range, i.e. where there exists a
> „normal“ range (e.g. 50+-10°C). Everything below or above would be
> abnormal and should throw a warning.
>
> Now the question is, when is that warning thrown and how often? And
> can the warning be reset e.g. by getting back in the normal range
> within a given time/count window?
>
> These involve questions in how to design such a pattern or pattern
> sequences, how to handle consecutive occurrences of events satisfying
> the pattern.
>
> Now we could imagine various output options:
>
> 1) output warning once and never again
> 2) output warning and wait for „reset“ of warning state, e.g.
> temperature back in normal range for a given time
> 2) output warning and set a timeout, after timeout if pattern still
> satisfied output another reminder warning event
> 3) output warning every time pattern is satisfied
>
> IMO, these choices greatly depend on the individual users. However, we
> should somehow provide her/him with possible options to suit their use
> case and prevent spamming notifications.
>
> @Grainier: would such a scenario be possible to implement out of the
> box using Siddhi?
>
> For all other simple scenarios, I guess an output rate-limiter giving
> the user a simple way of „reducing“ the messages send (e.g. send 1 of
> 10 events) out to the notification sink (Slack, Telegram etc) should
> be feasible.
>
> Any other thoughts?
>
> Patrick
>
>
>
> > Am 27.05.2020 um 06:54 schrieb Philipp Zehnder <ze...@apache.org>:
> >
> > I agree, maybe it is better to keep the notification-sinks as simple
> > as
> possible.
> > Then users could also use multiple channels (e.g. Email, Slack, and
> Telegram) for their notification and have a single output rate-limiter
> in the pipeline.
> > I was just wondering, if that’s a feature a user would expect in
> > each
> data sink or if this is an additional functionality for a separate
> pipeline element?
> >
> > Which functionalities would such a rate-limiter need?
> > I think it should not ‘just’ remove events, e.g. when a user is not
> reacting to the situation we might have to send the notification again
> after some time.
> > Any thoughts on that?
> >
> > Philipp
> >
> >
> >> On 26. May 2020, at 19:11, Grainier Perera
> >> <gr...@gmail.com>
> wrote:
> >>
> >> IMO, the sink shouldn't bother which events to publish. Instead, we
> >> can have event change capture & output rate-limiting as separate
> >> pipeline elements which can be used with notification sinks.
> So it
> >> gives users the flexibility to limit events or not, depending on
> >> their use-case. What do you think?
> >>
> >> pipeline: [event-source -> output rate-limiter ->
> >> notification-sink]
> >>
> >> Grainier.
> >>
> >> On Tue, 26 May 2020 at 21:47, Philipp Zehnder <ze...@apache.org>
> wrote:
> >>
> >>> Hi Grainier,
> >>>
> >>> this sink is very cool and useful. I directly merged it.
> >>>
> >>> @all I think we still have a little conceptual problem with our
> >>> notification sinks (e.g. Notification, Email, …) Currently we emit
> >>> a notification for each event, even if the event did
> not
> >>> change. This could annoy users and also be counter productive as
> >>> you
> don't
> >>> want to receive hundreds of messages.
> >>> We need a solution to set the maximum number of notification or
> >>> just notify the user when the situation changed.
> >>> Any ideas on that?
> >>>
> >>> Philipp
> >>>
> >>>> On 26. May 2020, at 16:28, Patrick Wiener <wi...@apache.org> wrote:
> >>>>
> >>>> Hi Grainier,
> >>>>
> >>>> this is really nice. We were at some point actually talking about
> such a
> >>> sink so super cool
> >>>> that you implemented it. I used telegram once in another project
> >>>> where
> >>> one was informed
> >>>> about alerts/warnings and quite liked it.
> >>>>
> >>>> Thanks for the contribution and valuable work.
> >>>>
> >>>> Patrick
> >>>>
> >>>>
> >>>>> Am 26.05.2020 um 15:25 schrieb Grainier Perera
> >>>>> <grainier@apache.org
> >:
> >>>>>
> >>>>> Hi all,
> >>>>>
> >>>>> I've implemented a publisher to send notifications to a Telegram
> >>> channel.
> >>>>> Since Telegram[1] is a free and cross-platform application, I
> >>>>> think
> this
> >>>>> will be a good addition to notification sinks. What do you think?
> >>>>>
> >>>>> Issue: https://issues.apache.org/jira/browse/STREAMPIPES-144
> >>>>> PR:
> https://github.com/apache/incubator-streampipes-extensions/pull/16
> >>>>>
> >>>>> [1] https://telegram.org/
> >>>>>
> >>>>> Thanks,
> >>>>> Grainier.
> >>>>
> >>>
> >>>
> >>>
> >
> >
>
>
Re: Telegram publisher to send event notifications
Posted by Grainier Perera <gr...@apache.org>.
Hi all,
I also think it should be a standalone feature. And the RateLimiter should
allow users to configure;
- a window (time or count)
- partitioning field (because there can be many sensors sending data to
the same stream)
- event to emit (first or last event of the window)
- retry logic. (something like; if there are no new events on the
current window, emit the last event of the previous window)
- etc...
So we can use that with existing filters, sequences (which we can improve
for event change capture), and notification sinks to tackle most of the
above scenarios.
Even with Siddhi, we have to use several queries to create a pipeline and
achieve the same. i.e;
define stream TemperatureStream (sensorId string, temperature float,
> timestamp int);
> -- Filter[1] to capture tempStatus
> from TemperatureStream
> select sensorId, ifThenElse((temperature < -10 || 50 < temperature),
> "abnormal", "normal") as as tempStatus
> insert into ProcessedTemperatureStream;
> -- Sequence[2] to tempStatus change (realtime)
> from every e1=ProcessedTemperatureStream,
> e2=ProcessedTemperatureStream[e1.tempStatus != tempStatus]
> group by sensorId
> select e1.sensorId, e2.tempStatus
> insert into NotificationStream;
> -- Ratelimit[3] to send only the last notification per 15 min (somewhat
> batching)
> from NotificationStream
> select *
> group by sensorId
> output last every 15 min
> insert into RateLimitedNotificationStream;
Further, I'll do some research on how similar products have done this, and
update the thread with how we can implement the same with StreamPipes.
What do you think?
[1] https://siddhi.io/en/v5.1/docs/examples/if-then-else/
[2] https://siddhi.io/en/v5.1/docs/examples/simple-sequence/
[3] https://siddhi.io/en/v5.1/docs/examples/time-rate-limit/
Grainier Perera.
On Wed, 27 May 2020 at 14:02, Patrick Wiener <wi...@apache.org> wrote:
> I think this should not be a feature included in the notification sinks.
> Rather it should be part of the dedicated PE either
> as a „standalone“ feature such as a rate limiter or as part of the
> application logic itself.
>
> For the latter, lets consider a CEP like use case where we want to monitor
> sensor temperature in a given range, i.e. where
> there exists a „normal“ range (e.g. 50+-10°C). Everything below or above
> would be abnormal and should throw a warning.
>
> Now the question is, when is that warning thrown and how often? And can
> the warning be reset e.g. by getting back in the
> normal range within a given time/count window?
>
> These involve questions in how to design such a pattern or pattern
> sequences, how to handle consecutive occurrences of
> events satisfying the pattern.
>
> Now we could imagine various output options:
>
> 1) output warning once and never again
> 2) output warning and wait for „reset“ of warning state, e.g. temperature
> back in normal range for a given time
> 2) output warning and set a timeout, after timeout if pattern still
> satisfied output another reminder warning event
> 3) output warning every time pattern is satisfied
>
> IMO, these choices greatly depend on the individual users. However, we
> should somehow provide her/him with possible
> options to suit their use case and prevent spamming notifications.
>
> @Grainier: would such a scenario be possible to implement out of the box
> using Siddhi?
>
> For all other simple scenarios, I guess an output rate-limiter giving the
> user a simple way of „reducing“ the messages
> send (e.g. send 1 of 10 events) out to the notification sink (Slack,
> Telegram etc) should be feasible.
>
> Any other thoughts?
>
> Patrick
>
>
>
> > Am 27.05.2020 um 06:54 schrieb Philipp Zehnder <ze...@apache.org>:
> >
> > I agree, maybe it is better to keep the notification-sinks as simple as
> possible.
> > Then users could also use multiple channels (e.g. Email, Slack, and
> Telegram) for their notification and have a single output rate-limiter in
> the pipeline.
> > I was just wondering, if that’s a feature a user would expect in each
> data sink or if this is an additional functionality for a separate pipeline
> element?
> >
> > Which functionalities would such a rate-limiter need?
> > I think it should not ‘just’ remove events, e.g. when a user is not
> reacting to the situation we might have to send the notification again
> after some time.
> > Any thoughts on that?
> >
> > Philipp
> >
> >
> >> On 26. May 2020, at 19:11, Grainier Perera <gr...@gmail.com>
> wrote:
> >>
> >> IMO, the sink shouldn't bother which events to publish. Instead, we can
> >> have event change capture & output rate-limiting as
> >> separate pipeline elements which can be used with notification sinks.
> So it
> >> gives users the flexibility to limit events or not, depending on their
> >> use-case. What do you think?
> >>
> >> pipeline: [event-source -> output rate-limiter -> notification-sink]
> >>
> >> Grainier.
> >>
> >> On Tue, 26 May 2020 at 21:47, Philipp Zehnder <ze...@apache.org>
> wrote:
> >>
> >>> Hi Grainier,
> >>>
> >>> this sink is very cool and useful. I directly merged it.
> >>>
> >>> @all I think we still have a little conceptual problem with our
> >>> notification sinks (e.g. Notification, Email, …)
> >>> Currently we emit a notification for each event, even if the event did
> not
> >>> change. This could annoy users and also be counter productive as you
> don't
> >>> want to receive hundreds of messages.
> >>> We need a solution to set the maximum number of notification or just
> >>> notify the user when the situation changed.
> >>> Any ideas on that?
> >>>
> >>> Philipp
> >>>
> >>>> On 26. May 2020, at 16:28, Patrick Wiener <wi...@apache.org> wrote:
> >>>>
> >>>> Hi Grainier,
> >>>>
> >>>> this is really nice. We were at some point actually talking about
> such a
> >>> sink so super cool
> >>>> that you implemented it. I used telegram once in another project where
> >>> one was informed
> >>>> about alerts/warnings and quite liked it.
> >>>>
> >>>> Thanks for the contribution and valuable work.
> >>>>
> >>>> Patrick
> >>>>
> >>>>
> >>>>> Am 26.05.2020 um 15:25 schrieb Grainier Perera <grainier@apache.org
> >:
> >>>>>
> >>>>> Hi all,
> >>>>>
> >>>>> I've implemented a publisher to send notifications to a Telegram
> >>> channel.
> >>>>> Since Telegram[1] is a free and cross-platform application, I think
> this
> >>>>> will be a good addition to notification sinks. What do you think?
> >>>>>
> >>>>> Issue: https://issues.apache.org/jira/browse/STREAMPIPES-144
> >>>>> PR:
> https://github.com/apache/incubator-streampipes-extensions/pull/16
> >>>>>
> >>>>> [1] https://telegram.org/
> >>>>>
> >>>>> Thanks,
> >>>>> Grainier.
> >>>>
> >>>
> >>>
> >>>
> >
> >
>
>
Re: Telegram publisher to send event notifications
Posted by Patrick Wiener <wi...@apache.org>.
I think this should not be a feature included in the notification sinks. Rather it should be part of the dedicated PE either
as a „standalone“ feature such as a rate limiter or as part of the application logic itself.
For the latter, lets consider a CEP like use case where we want to monitor sensor temperature in a given range, i.e. where
there exists a „normal“ range (e.g. 50+-10°C). Everything below or above would be abnormal and should throw a warning.
Now the question is, when is that warning thrown and how often? And can the warning be reset e.g. by getting back in the
normal range within a given time/count window?
These involve questions in how to design such a pattern or pattern sequences, how to handle consecutive occurrences of
events satisfying the pattern.
Now we could imagine various output options:
1) output warning once and never again
2) output warning and wait for „reset“ of warning state, e.g. temperature back in normal range for a given time
2) output warning and set a timeout, after timeout if pattern still satisfied output another reminder warning event
3) output warning every time pattern is satisfied
IMO, these choices greatly depend on the individual users. However, we should somehow provide her/him with possible
options to suit their use case and prevent spamming notifications.
@Grainier: would such a scenario be possible to implement out of the box using Siddhi?
For all other simple scenarios, I guess an output rate-limiter giving the user a simple way of „reducing“ the messages
send (e.g. send 1 of 10 events) out to the notification sink (Slack, Telegram etc) should be feasible.
Any other thoughts?
Patrick
> Am 27.05.2020 um 06:54 schrieb Philipp Zehnder <ze...@apache.org>:
>
> I agree, maybe it is better to keep the notification-sinks as simple as possible.
> Then users could also use multiple channels (e.g. Email, Slack, and Telegram) for their notification and have a single output rate-limiter in the pipeline.
> I was just wondering, if that’s a feature a user would expect in each data sink or if this is an additional functionality for a separate pipeline element?
>
> Which functionalities would such a rate-limiter need?
> I think it should not ‘just’ remove events, e.g. when a user is not reacting to the situation we might have to send the notification again after some time.
> Any thoughts on that?
>
> Philipp
>
>
>> On 26. May 2020, at 19:11, Grainier Perera <gr...@gmail.com> wrote:
>>
>> IMO, the sink shouldn't bother which events to publish. Instead, we can
>> have event change capture & output rate-limiting as
>> separate pipeline elements which can be used with notification sinks. So it
>> gives users the flexibility to limit events or not, depending on their
>> use-case. What do you think?
>>
>> pipeline: [event-source -> output rate-limiter -> notification-sink]
>>
>> Grainier.
>>
>> On Tue, 26 May 2020 at 21:47, Philipp Zehnder <ze...@apache.org> wrote:
>>
>>> Hi Grainier,
>>>
>>> this sink is very cool and useful. I directly merged it.
>>>
>>> @all I think we still have a little conceptual problem with our
>>> notification sinks (e.g. Notification, Email, …)
>>> Currently we emit a notification for each event, even if the event did not
>>> change. This could annoy users and also be counter productive as you don't
>>> want to receive hundreds of messages.
>>> We need a solution to set the maximum number of notification or just
>>> notify the user when the situation changed.
>>> Any ideas on that?
>>>
>>> Philipp
>>>
>>>> On 26. May 2020, at 16:28, Patrick Wiener <wi...@apache.org> wrote:
>>>>
>>>> Hi Grainier,
>>>>
>>>> this is really nice. We were at some point actually talking about such a
>>> sink so super cool
>>>> that you implemented it. I used telegram once in another project where
>>> one was informed
>>>> about alerts/warnings and quite liked it.
>>>>
>>>> Thanks for the contribution and valuable work.
>>>>
>>>> Patrick
>>>>
>>>>
>>>>> Am 26.05.2020 um 15:25 schrieb Grainier Perera <gr...@apache.org>:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> I've implemented a publisher to send notifications to a Telegram
>>> channel.
>>>>> Since Telegram[1] is a free and cross-platform application, I think this
>>>>> will be a good addition to notification sinks. What do you think?
>>>>>
>>>>> Issue: https://issues.apache.org/jira/browse/STREAMPIPES-144
>>>>> PR: https://github.com/apache/incubator-streampipes-extensions/pull/16
>>>>>
>>>>> [1] https://telegram.org/
>>>>>
>>>>> Thanks,
>>>>> Grainier.
>>>>
>>>
>>>
>>>
>
>
Re: Telegram publisher to send event notifications
Posted by Philipp Zehnder <ze...@apache.org>.
I agree, maybe it is better to keep the notification-sinks as simple as possible.
Then users could also use multiple channels (e.g. Email, Slack, and Telegram) for their notification and have a single output rate-limiter in the pipeline.
I was just wondering, if that’s a feature a user would expect in each data sink or if this is an additional functionality for a separate pipeline element?
Which functionalities would such a rate-limiter need?
I think it should not ‘just’ remove events, e.g. when a user is not reacting to the situation we might have to send the notification again after some time.
Any thoughts on that?
Philipp
> On 26. May 2020, at 19:11, Grainier Perera <gr...@gmail.com> wrote:
>
> IMO, the sink shouldn't bother which events to publish. Instead, we can
> have event change capture & output rate-limiting as
> separate pipeline elements which can be used with notification sinks. So it
> gives users the flexibility to limit events or not, depending on their
> use-case. What do you think?
>
> pipeline: [event-source -> output rate-limiter -> notification-sink]
>
> Grainier.
>
> On Tue, 26 May 2020 at 21:47, Philipp Zehnder <ze...@apache.org> wrote:
>
>> Hi Grainier,
>>
>> this sink is very cool and useful. I directly merged it.
>>
>> @all I think we still have a little conceptual problem with our
>> notification sinks (e.g. Notification, Email, …)
>> Currently we emit a notification for each event, even if the event did not
>> change. This could annoy users and also be counter productive as you don't
>> want to receive hundreds of messages.
>> We need a solution to set the maximum number of notification or just
>> notify the user when the situation changed.
>> Any ideas on that?
>>
>> Philipp
>>
>>> On 26. May 2020, at 16:28, Patrick Wiener <wi...@apache.org> wrote:
>>>
>>> Hi Grainier,
>>>
>>> this is really nice. We were at some point actually talking about such a
>> sink so super cool
>>> that you implemented it. I used telegram once in another project where
>> one was informed
>>> about alerts/warnings and quite liked it.
>>>
>>> Thanks for the contribution and valuable work.
>>>
>>> Patrick
>>>
>>>
>>>> Am 26.05.2020 um 15:25 schrieb Grainier Perera <gr...@apache.org>:
>>>>
>>>> Hi all,
>>>>
>>>> I've implemented a publisher to send notifications to a Telegram
>> channel.
>>>> Since Telegram[1] is a free and cross-platform application, I think this
>>>> will be a good addition to notification sinks. What do you think?
>>>>
>>>> Issue: https://issues.apache.org/jira/browse/STREAMPIPES-144
>>>> PR: https://github.com/apache/incubator-streampipes-extensions/pull/16
>>>>
>>>> [1] https://telegram.org/
>>>>
>>>> Thanks,
>>>> Grainier.
>>>
>>
>>
>>
Re: Telegram publisher to send event notifications
Posted by Grainier Perera <gr...@gmail.com>.
IMO, the sink shouldn't bother which events to publish. Instead, we can
have event change capture & output rate-limiting as
separate pipeline elements which can be used with notification sinks. So it
gives users the flexibility to limit events or not, depending on their
use-case. What do you think?
pipeline: [event-source -> output rate-limiter -> notification-sink]
Grainier.
On Tue, 26 May 2020 at 21:47, Philipp Zehnder <ze...@apache.org> wrote:
> Hi Grainier,
>
> this sink is very cool and useful. I directly merged it.
>
> @all I think we still have a little conceptual problem with our
> notification sinks (e.g. Notification, Email, …)
> Currently we emit a notification for each event, even if the event did not
> change. This could annoy users and also be counter productive as you don't
> want to receive hundreds of messages.
> We need a solution to set the maximum number of notification or just
> notify the user when the situation changed.
> Any ideas on that?
>
> Philipp
>
> > On 26. May 2020, at 16:28, Patrick Wiener <wi...@apache.org> wrote:
> >
> > Hi Grainier,
> >
> > this is really nice. We were at some point actually talking about such a
> sink so super cool
> > that you implemented it. I used telegram once in another project where
> one was informed
> > about alerts/warnings and quite liked it.
> >
> > Thanks for the contribution and valuable work.
> >
> > Patrick
> >
> >
> >> Am 26.05.2020 um 15:25 schrieb Grainier Perera <gr...@apache.org>:
> >>
> >> Hi all,
> >>
> >> I've implemented a publisher to send notifications to a Telegram
> channel.
> >> Since Telegram[1] is a free and cross-platform application, I think this
> >> will be a good addition to notification sinks. What do you think?
> >>
> >> Issue: https://issues.apache.org/jira/browse/STREAMPIPES-144
> >> PR: https://github.com/apache/incubator-streampipes-extensions/pull/16
> >>
> >> [1] https://telegram.org/
> >>
> >> Thanks,
> >> Grainier.
> >
>
>
>
Re: Telegram publisher to send event notifications
Posted by Philipp Zehnder <ze...@apache.org>.
Hi Grainier,
this sink is very cool and useful. I directly merged it.
@all I think we still have a little conceptual problem with our notification sinks (e.g. Notification, Email, …)
Currently we emit a notification for each event, even if the event did not change. This could annoy users and also be counter productive as you don't want to receive hundreds of messages.
We need a solution to set the maximum number of notification or just notify the user when the situation changed.
Any ideas on that?
Philipp
> On 26. May 2020, at 16:28, Patrick Wiener <wi...@apache.org> wrote:
>
> Hi Grainier,
>
> this is really nice. We were at some point actually talking about such a sink so super cool
> that you implemented it. I used telegram once in another project where one was informed
> about alerts/warnings and quite liked it.
>
> Thanks for the contribution and valuable work.
>
> Patrick
>
>
>> Am 26.05.2020 um 15:25 schrieb Grainier Perera <gr...@apache.org>:
>>
>> Hi all,
>>
>> I've implemented a publisher to send notifications to a Telegram channel.
>> Since Telegram[1] is a free and cross-platform application, I think this
>> will be a good addition to notification sinks. What do you think?
>>
>> Issue: https://issues.apache.org/jira/browse/STREAMPIPES-144
>> PR: https://github.com/apache/incubator-streampipes-extensions/pull/16
>>
>> [1] https://telegram.org/
>>
>> Thanks,
>> Grainier.
>
Re: Telegram publisher to send event notifications
Posted by Patrick Wiener <wi...@apache.org>.
Hi Grainier,
this is really nice. We were at some point actually talking about such a sink so super cool
that you implemented it. I used telegram once in another project where one was informed
about alerts/warnings and quite liked it.
Thanks for the contribution and valuable work.
Patrick
> Am 26.05.2020 um 15:25 schrieb Grainier Perera <gr...@apache.org>:
>
> Hi all,
>
> I've implemented a publisher to send notifications to a Telegram channel.
> Since Telegram[1] is a free and cross-platform application, I think this
> will be a good addition to notification sinks. What do you think?
>
> Issue: https://issues.apache.org/jira/browse/STREAMPIPES-144
> PR: https://github.com/apache/incubator-streampipes-extensions/pull/16
>
> [1] https://telegram.org/
>
> Thanks,
> Grainier.