You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by Austin Cawley-Edwards <au...@gmail.com> on 2018/10/25 21:07:54 UTC

Accumulating a batch

Hi there,

Is it possible to use an AggregationFunction to accumulate n values in a
buffer until a threshold is met, then stream down all records in the batch?

Thank you!
Austin Cawley-Edwards

Re: Accumulating a batch

Posted by Austin Cawley-Edwards <au...@gmail.com>.

Hi Hequn,

Thank you for the info! Much appreciated! Do you know if a GroupBy Window
could be used to buffer until an arbitrary record came through?

Best,
Austin

On Thu, Oct 25, 2018, 9:17 PM Hequn Cheng <ch...@gmail.com> wrote:

> Hi Austin,
>
> You can use GroupBy Window[1], such as TUMBLE Window. The size of the
> window either as time or row-count interval. You can also define your
> own User-Defined Aggregate Functions[2] to be used in window.
>
> Best, Hequn
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tableApi.html#tumble-tumbling-windows
> [2]
> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/udfs.html#aggregation-functions
>
> On Fri, Oct 26, 2018 at 5:08 AM Austin Cawley-Edwards <
> austin.cawley@gmail.com> wrote:
>
>> Hi there,
>>
>> Is it possible to use an AggregationFunction to accumulate n values in a
>> buffer until a threshold is met, then stream down all records in the batch?
>>
>> Thank you!
>> Austin Cawley-Edwards
>>
>

Re: Accumulating a batch

Posted by Hequn Cheng <ch...@gmail.com>.

Hi Austin,

You can use GroupBy Window[1], such as TUMBLE Window. The size of the
window either as time or row-count interval. You can also define your
own User-Defined Aggregate Functions[2] to be used in window.

Best, Hequn

[1]
https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tableApi.html#tumble-tumbling-windows
[2]
https://ci.apache.org/projects/flink/flink-docs-master/dev/table/udfs.html#aggregation-functions

On Fri, Oct 26, 2018 at 5:08 AM Austin Cawley-Edwards <
austin.cawley@gmail.com> wrote:

> Hi there,
>
> Is it possible to use an AggregationFunction to accumulate n values in a
> buffer until a threshold is met, then stream down all records in the batch?
>
> Thank you!
> Austin Cawley-Edwards
>