You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Spico Florin <sp...@gmail.com> on 2018/11/23 15:04:52 UTC

[flink-cep] Flick CEP support for the group By operator

Hello!

I'm using Flink 1.4.2 and I would like to use a group by operator based on
value of my event stream. The functionality that I would like to achieve is
similar to the following Esper EPL
(excerpt
http://esper.espertech.com/release-5.2.0/esper-reference/html/epl_clauses.html#epl-grouping-aggregating
)


select symbol, tickDataFeed, median(volume)
from StockTickEvent.win:time(30 sec)
group by symbol, tickDataFeed


So, does the Flick CEP  support such a group by functionality?

If yes what is syntax?


I look forward for your answers.


Best regards,

 Florin



(excerpt
http://esper.espertech.com/release-5.2.0/esper-reference/html/epl_clauses.html#epl-grouping-aggregating
)
"You can list more then one expression in the group by clause to nest
groups. Once the sets are established with group by the aggregation
functions are applied. This statement posts the median volume for all stock
tick events in the last 30 seconds per symbol and tick data feed. Esper
posts one event for each group to statement listeners:

In the statement above the event properties in the select list (symbol,
tickDataFeed) are also listed in the group by clause. The statement thus
follows the SQL standard which prescribes that non-aggregated event
properties in the select list must match the group by columns."

Re: [flink-cep] Flick CEP support for the group By operator

Posted by Piotr Nowojski <pi...@data-artisans.com>.
Hey,

As a matter of fact, you do not need a Flink's CEP library to run the same query. The same functionality can be achieved by simple tumbling window with a “median” aggregate (“median" you would have to implement by your self).

https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/operators/windows.html

Piotrek

> On 23 Nov 2018, at 16:32, Piotr Nowojski <pi...@data-artisans.com> wrote:
> 
> Hi,
> 
> Yes, sure. Just use CEP on top of KeyedStream. Take a look at `keyBy`:
> 
> https://ci.apache.org/projects/flink/flink-docs-release-1.6/dev/stream/operators/ <https://ci.apache.org/projects/flink/flink-docs-release-1.6/dev/stream/operators/>
> 
> Piotrek 
> 
>> On 23 Nov 2018, at 16:04, Spico Florin <spicoflorin@gmail.com <ma...@gmail.com>> wrote:
>> 
>> Hello!
>> 
>> I'm using Flink 1.4.2 and I would like to use a group by operator based on value of my event stream. The functionality that I would like to achieve is similar to the following Esper EPL
>> (excerpt http://esper.espertech.com/release-5.2.0/esper-reference/html/epl_clauses.html#epl-grouping-aggregating <http://esper.espertech.com/release-5.2.0/esper-reference/html/epl_clauses.html#epl-grouping-aggregating>)
>> 
>> 
>> select symbol, tickDataFeed, median(volume) 
>> from StockTickEvent.win <http://stocktickevent.win/>:time(30 sec) 
>> group by symbol, tickDataFeed
>>  <>
>> 
>> So, does the Flick CEP  support such a group by functionality? 
>> 
>> If yes what is syntax?
>> 
>> 
>> 
>> I look forward for your answers.
>> 
>> 
>> 
>> Best regards,
>> 
>>  Florin 
>> 
>> 
>> 
>> 
>> 
>> (excerpt http://esper.espertech.com/release-5.2.0/esper-reference/html/epl_clauses.html#epl-grouping-aggregating <http://esper.espertech.com/release-5.2.0/esper-reference/html/epl_clauses.html#epl-grouping-aggregating>)
>> "You can list more then one expression in the group by clause to nest groups. Once the sets are established with group by the aggregation functions are applied. This statement posts the median volume for all stock tick events in the last 30 seconds per symbol and tick data feed. Esper posts one event for each group to statement listeners:
>> In the statement above the event properties in the select list (symbol, tickDataFeed) are also listed in the group by clause. The statement thus follows the SQL standard which prescribes that non-aggregated event properties in the select list must match the group by columns."
>> 
>

Re: [flink-cep] Flick CEP support for the group By operator

Posted by Piotr Nowojski <pi...@data-artisans.com>.
Hi,

Yes, sure. Just use CEP on top of KeyedStream. Take a look at `keyBy`:

https://ci.apache.org/projects/flink/flink-docs-release-1.6/dev/stream/operators/

Piotrek 

> On 23 Nov 2018, at 16:04, Spico Florin <sp...@gmail.com> wrote:
> 
> Hello!
> 
> I'm using Flink 1.4.2 and I would like to use a group by operator based on value of my event stream. The functionality that I would like to achieve is similar to the following Esper EPL
> (excerpt http://esper.espertech.com/release-5.2.0/esper-reference/html/epl_clauses.html#epl-grouping-aggregating <http://esper.espertech.com/release-5.2.0/esper-reference/html/epl_clauses.html#epl-grouping-aggregating>)
> 
> 
> select symbol, tickDataFeed, median(volume) 
> from StockTickEvent.win:time(30 sec) 
> group by symbol, tickDataFeed
>  <>
> 
> So, does the Flick CEP  support such a group by functionality? 
> 
> If yes what is syntax?
> 
> 
> 
> I look forward for your answers.
> 
> 
> 
> Best regards,
> 
>  Florin 
> 
> 
> 
> 
> 
> (excerpt http://esper.espertech.com/release-5.2.0/esper-reference/html/epl_clauses.html#epl-grouping-aggregating <http://esper.espertech.com/release-5.2.0/esper-reference/html/epl_clauses.html#epl-grouping-aggregating>)
> "You can list more then one expression in the group by clause to nest groups. Once the sets are established with group by the aggregation functions are applied. This statement posts the median volume for all stock tick events in the last 30 seconds per symbol and tick data feed. Esper posts one event for each group to statement listeners:
> In the statement above the event properties in the select list (symbol, tickDataFeed) are also listed in the group by clause. The statement thus follows the SQL standard which prescribes that non-aggregated event properties in the select list must match the group by columns."
>