You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Dhruv Kumar <ga...@gmail.com> on 2018/05/27 19:43:11 UTC
Clarification in TumblingProcessing TimeWindow Documentation
Hi
I was looking at TumblingProcessingTimeWindows.java <https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/windowing/assigners/TumblingProcessingTimeWindows.java> and was a bit confused with the documentation at the start of this class. It says the following:
/**
* A {@link WindowAssigner} that windows elements into windows based on the current
* system time of the machine the operation is running on. Windows cannot overlap.
*
* <p>For example, in order to window into windows of 1 minute, every 10 seconds:
* <pre> {@code
* DataStream<Tuple2<String, Integer>> in = ...;
* KeyedStream<String, Tuple2<String, Integer>> keyed = in.keyBy(...);
* WindowedStream<Tuple2<String, Integer>, String, TimeWindows> windowed =
* keyed.window(TumblingProcessingTimeWindows.of(Time.of(1, MINUTES), Time.of(10, SECONDS));
* } </pre>
*/
It says one can have tumbling windows of 1 minute, every 10 seconds. Doesn’t this become a sliding window then? The SlidingProcessTimeWindows.java <https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/windowing/assigners/SlidingProcessingTimeWindows.java> has the exact same documentation with just one tiny change (“Windows can possibly overlap”). It seems to me that in the above documentation, the second Time argument of 10 seconds is for providing the window offset (as confirmed here <https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/windowing/assigners/TumblingProcessingTimeWindows.java#L92>) and not for starting the tumbling window every 10 seconds.
Thanks
--------------------------------------------------
Dhruv Kumar
PhD Candidate
Department of Computer Science and Engineering
University of Minnesota
www.dhruvkumar.me
Re: Clarification in TumblingProcessing TimeWindow Documentation
Posted by Dhruv Kumar <ga...@gmail.com>.
Sure thanks a lot! Bowen and Fabian. I will create the JIRA and also submit a PR with a better comment.
--------------------------------------------------
Dhruv Kumar
PhD Candidate
Department of Computer Science and Engineering
University of Minnesota
www.dhruvkumar.me
> On May 28, 2018, at 03:50, Fabian Hueske <fh...@gmail.com> wrote:
>
> I agree, this should be fixed.
>
> Thanks for noticing, Dhruv.
> Would you mind creating a JIRA for this?
>
> Thank you,
> Fabian
>
> 2018-05-28 8:39 GMT+02:00 Bowen Li <bowenli86@gmail.com <ma...@gmail.com>>:
> Hi Dhruv,
>
> I can see it's confusing, and it does seem the comment should be improved. You can find concrete explanation of tumbling window and relative arguments at https://ci.apache.org/projects/flink/flink-docs-master/dev/stream/operators/windows.html#tumbling-windows <https://ci.apache.org/projects/flink/flink-docs-master/dev/stream/operators/windows.html#tumbling-windows>
>
> Feel free to open a PR with better comment.
>
> Thanks,
> Bowen
>
>
>
> On Sun, May 27, 2018 at 12:43 PM, Dhruv Kumar <gargdhruv36@gmail.com <ma...@gmail.com>> wrote:
> Hi
>
> I was looking at TumblingProcessingTimeWindows.java <https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/windowing/assigners/TumblingProcessingTimeWindows.java> and was a bit confused with the documentation at the start of this class. It says the following:
>
> /**
> * A {@link WindowAssigner} that windows elements into windows based on the current
> * system time of the machine the operation is running on. Windows cannot overlap.
> *
> * <p>For example, in order to window into windows of 1 minute, every 10 seconds:
> * <pre> {@code
> * DataStream<Tuple2<String, Integer>> in = ...;
> * KeyedStream<String, Tuple2<String, Integer>> keyed = in.keyBy(...);
> * WindowedStream<Tuple2<String, Integer>, String, TimeWindows> windowed =
> * keyed.window(TumblingProcessingTimeWindows.of(Time.of(1, MINUTES), Time.of(10, SECONDS));
> * } </pre>
> */
>
>
> It says one can have tumbling windows of 1 minute, every 10 seconds. Doesn’t this become a sliding window then? The SlidingProcessTimeWindows.java <https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/windowing/assigners/SlidingProcessingTimeWindows.java> has the exact same documentation with just one tiny change (“Windows can possibly overlap”). It seems to me that in the above documentation, the second Time argument of 10 seconds is for providing the window offset (as confirmed here <https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/windowing/assigners/TumblingProcessingTimeWindows.java#L92>) and not for starting the tumbling window every 10 seconds.
>
> Thanks
>
>
> --------------------------------------------------
> Dhruv Kumar
> PhD Candidate
> Department of Computer Science and Engineering
> University of Minnesota
> www.dhruvkumar.me <http://www.dhruvkumar.me/>
>
>
Re: Clarification in TumblingProcessing TimeWindow Documentation
Posted by Fabian Hueske <fh...@gmail.com>.
I agree, this should be fixed.
Thanks for noticing, Dhruv.
Would you mind creating a JIRA for this?
Thank you,
Fabian
2018-05-28 8:39 GMT+02:00 Bowen Li <bo...@gmail.com>:
> Hi Dhruv,
>
> I can see it's confusing, and it does seem the comment should be improved.
> You can find concrete explanation of tumbling window and relative arguments
> at https://ci.apache.org/projects/flink/flink-docs-
> master/dev/stream/operators/windows.html#tumbling-windows
>
> Feel free to open a PR with better comment.
>
> Thanks,
> Bowen
>
>
>
> On Sun, May 27, 2018 at 12:43 PM, Dhruv Kumar <ga...@gmail.com>
> wrote:
>
>> Hi
>>
>> I was looking at TumblingProcessingTimeWindows.java
>> <https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/windowing/assigners/TumblingProcessingTimeWindows.java> and
>> was a bit confused with the documentation at the start of this class. It
>> says the following:
>>
>> /**
>> * A {@link WindowAssigner} that windows elements into windows based on
>> the current
>> * system time of the machine the operation is running on. Windows cannot
>> overlap.
>> *
>> * <p>For example, in order to window into windows of 1 minute, every 10
>> seconds:
>> * <pre> {@code
>> * DataStream<Tuple2<String, Integer>> in = ...;
>> * KeyedStream<String, Tuple2<String, Integer>> keyed = in.keyBy(...);
>> * WindowedStream<Tuple2<String, Integer>, String, TimeWindows> windowed =
>> * keyed.window(TumblingProcessingTimeWindows.of(Time.of(1, MINUTES),
>> Time.of(10, SECONDS));
>> * } </pre>
>> */
>> It says one can have tumbling windows of 1 minute, every 10 seconds.
>> Doesn’t this become a sliding window then? The SlidingProcessTimeWindows.
>> java
>> <https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/windowing/assigners/SlidingProcessingTimeWindows.java> has
>> the exact same documentation with just one tiny change (“Windows can
>> possibly overlap”). It seems to me that in the above documentation, the
>> second Time argument of 10 seconds is for providing the window offset (as
>> confirmed here
>> <https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/windowing/assigners/TumblingProcessingTimeWindows.java#L92>)
>> and not for starting the tumbling window every 10 seconds.
>>
>> Thanks
>>
>>
>> --------------------------------------------------
>> *Dhruv Kumar*
>> PhD Candidate
>> Department of Computer Science and Engineering
>> University of Minnesota
>> www.dhruvkumar.me
>>
>>
>
Re: Clarification in TumblingProcessing TimeWindow Documentation
Posted by Bowen Li <bo...@gmail.com>.
Hi Dhruv,
I can see it's confusing, and it does seem the comment should be improved.
You can find concrete explanation of tumbling window and relative arguments
at
https://ci.apache.org/projects/flink/flink-docs-master/dev/stream/operators/windows.html#tumbling-windows
Feel free to open a PR with better comment.
Thanks,
Bowen
On Sun, May 27, 2018 at 12:43 PM, Dhruv Kumar <ga...@gmail.com> wrote:
> Hi
>
> I was looking at TumblingProcessingTimeWindows.java
> <https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/windowing/assigners/TumblingProcessingTimeWindows.java> and
> was a bit confused with the documentation at the start of this class. It
> says the following:
>
> /**
> * A {@link WindowAssigner} that windows elements into windows based on
> the current
> * system time of the machine the operation is running on. Windows cannot
> overlap.
> *
> * <p>For example, in order to window into windows of 1 minute, every 10
> seconds:
> * <pre> {@code
> * DataStream<Tuple2<String, Integer>> in = ...;
> * KeyedStream<String, Tuple2<String, Integer>> keyed = in.keyBy(...);
> * WindowedStream<Tuple2<String, Integer>, String, TimeWindows> windowed =
> * keyed.window(TumblingProcessingTimeWindows.of(Time.of(1, MINUTES),
> Time.of(10, SECONDS));
> * } </pre>
> */
> It says one can have tumbling windows of 1 minute, every 10 seconds.
> Doesn’t this become a sliding window then? The SlidingProcessTimeWindows.
> java
> <https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/windowing/assigners/SlidingProcessingTimeWindows.java> has
> the exact same documentation with just one tiny change (“Windows can
> possibly overlap”). It seems to me that in the above documentation, the
> second Time argument of 10 seconds is for providing the window offset (as
> confirmed here
> <https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/windowing/assigners/TumblingProcessingTimeWindows.java#L92>)
> and not for starting the tumbling window every 10 seconds.
>
> Thanks
>
>
> --------------------------------------------------
> *Dhruv Kumar*
> PhD Candidate
> Department of Computer Science and Engineering
> University of Minnesota
> www.dhruvkumar.me
>
>