You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Pariksheet Barapatre <pa...@gmail.com> on 2016/06/13 07:31:13 UTC
Kafka Streaming - Window expiration
Hello Experts,
As per documentation in kafka docs -
*Windowing* is a common prerequisite for stateful transformations which
group records in a stream, for example, by their timestamps. A local state
store is usually needed for a windowing operation to store recently
received records based on the window interval, while old records in the
store are purged after the specified window retention period
<http://docs.confluent.io/3.0.0/streams/concepts.html#streams-concepts-windowing>
.
But I am not finding any example to set window retention period example.
Could somebody help me with example?
long windowSizeMs = 60 *
1000L;TimeWindows.of("tumbling-window-example", windowSizeMs);
How to set expiration/purge time for above window example.
Many Thanks
Pari
Re: Kafka Streaming - Window expiration
Posted by Pariksheet Barapatre <pa...@gmail.com>.
Many Thanks Eno. I will try .until method.
Cheers
Pari
On 13 June 2016 at 14:10, Eno Thereska <en...@gmail.com> wrote:
> Hi Pari,
>
> Try the .until method like this:
>
> > (TimeWindows) TimeWindows.of("tumbling-window-example",
> windowSizeMs).until(60 * 1000L)
>
> Thanks
> Eno
>
>
> > On 13 Jun 2016, at 08:31, Pariksheet Barapatre <pa...@gmail.com>
> wrote:
> >
> > Hello Experts,
> >
> > As per documentation in kafka docs -
> > *Windowing* is a common prerequisite for stateful transformations which
> > group records in a stream, for example, by their timestamps. A local
> state
> > store is usually needed for a windowing operation to store recently
> > received records based on the window interval, while old records in the
> > store are purged after the specified window retention period
> > <
> http://docs.confluent.io/3.0.0/streams/concepts.html#streams-concepts-windowing
> >
> > .
> >
> > But I am not finding any example to set window retention period example.
> >
> > Could somebody help me with example?
> >
> > long windowSizeMs = 60 *
> > 1000L;TimeWindows.of("tumbling-window-example", windowSizeMs);
> >
> > How to set expiration/purge time for above window example.
> >
> > Many Thanks
> >
> > Pari
>
>
Re: Kafka Streaming - Window expiration
Posted by Pariksheet Barapatre <pa...@gmail.com>.
Many Thanks Eno. I will try .until method.
Cheers
Pari
On 13 June 2016 at 14:10, Eno Thereska <en...@gmail.com> wrote:
> Hi Pari,
>
> Try the .until method like this:
>
> > (TimeWindows) TimeWindows.of("tumbling-window-example",
> windowSizeMs).until(60 * 1000L)
>
> Thanks
> Eno
>
>
> > On 13 Jun 2016, at 08:31, Pariksheet Barapatre <pa...@gmail.com>
> wrote:
> >
> > Hello Experts,
> >
> > As per documentation in kafka docs -
> > *Windowing* is a common prerequisite for stateful transformations which
> > group records in a stream, for example, by their timestamps. A local
> state
> > store is usually needed for a windowing operation to store recently
> > received records based on the window interval, while old records in the
> > store are purged after the specified window retention period
> > <
> http://docs.confluent.io/3.0.0/streams/concepts.html#streams-concepts-windowing
> >
> > .
> >
> > But I am not finding any example to set window retention period example.
> >
> > Could somebody help me with example?
> >
> > long windowSizeMs = 60 *
> > 1000L;TimeWindows.of("tumbling-window-example", windowSizeMs);
> >
> > How to set expiration/purge time for above window example.
> >
> > Many Thanks
> >
> > Pari
>
>
Re: Kafka Streaming - Window expiration
Posted by Eno Thereska <en...@gmail.com>.
Hi Pari,
Try the .until method like this:
> (TimeWindows) TimeWindows.of("tumbling-window-example", windowSizeMs).until(60 * 1000L)
Thanks
Eno
> On 13 Jun 2016, at 08:31, Pariksheet Barapatre <pa...@gmail.com> wrote:
>
> Hello Experts,
>
> As per documentation in kafka docs -
> *Windowing* is a common prerequisite for stateful transformations which
> group records in a stream, for example, by their timestamps. A local state
> store is usually needed for a windowing operation to store recently
> received records based on the window interval, while old records in the
> store are purged after the specified window retention period
> <http://docs.confluent.io/3.0.0/streams/concepts.html#streams-concepts-windowing>
> .
>
> But I am not finding any example to set window retention period example.
>
> Could somebody help me with example?
>
> long windowSizeMs = 60 *
> 1000L;TimeWindows.of("tumbling-window-example", windowSizeMs);
>
> How to set expiration/purge time for above window example.
>
> Many Thanks
>
> Pari
Re: Kafka Streaming - Window expiration
Posted by Eno Thereska <en...@gmail.com>.
Hi Pari,
Try the .until method like this:
> (TimeWindows) TimeWindows.of("tumbling-window-example", windowSizeMs).until(60 * 1000L)
Thanks
Eno
> On 13 Jun 2016, at 08:31, Pariksheet Barapatre <pa...@gmail.com> wrote:
>
> Hello Experts,
>
> As per documentation in kafka docs -
> *Windowing* is a common prerequisite for stateful transformations which
> group records in a stream, for example, by their timestamps. A local state
> store is usually needed for a windowing operation to store recently
> received records based on the window interval, while old records in the
> store are purged after the specified window retention period
> <http://docs.confluent.io/3.0.0/streams/concepts.html#streams-concepts-windowing>
> .
>
> But I am not finding any example to set window retention period example.
>
> Could somebody help me with example?
>
> long windowSizeMs = 60 *
> 1000L;TimeWindows.of("tumbling-window-example", windowSizeMs);
>
> How to set expiration/purge time for above window example.
>
> Many Thanks
>
> Pari