You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Jeff Zhang <zj...@gmail.com> on 2018/11/14 02:31:23 UTC
Confused window operation
Hi all,
I am a little confused with the following windows operation. Here's the
code,
val senv = StreamExecutionEnvironment.getExecutionEnvironment
senv.setParallelism(1)
val data = senv.fromElements("hello world", "hello flink", "hello hadoop")
data.flatMap(line => line.split("\\s"))
.map(w => (w, 1))
.keyBy(0)
.countWindow(2, 1)
.sum(1)
.print("******************")
senv.execute()
And this is the output:
******************> (hello,1)
******************> (world,1)
******************> (hello,2)
******************> (flink,1)
******************> (hello,2)
******************> (hadoop,1)
As my understanding, here we have 3 windows.
window 1
(hello, world)
window 2
(hello, world)
(hello, flink)
window 3
(hello flink)
(hello hadoop)
So for the first window, we have output (hello, 1) (world, 1)
for the second window we should output (hello, 2), (world,1 ), (flink, 1)
for the third window we should have output (hello, 2), (flink, 1), (hadoop, 1)
But as you can see, in the above I get different result, do I
misunderstand the window ? Could anyone help me to understand that ?
Thanks
Re: Confused window operation
Posted by Jeff Zhang <zj...@gmail.com>.
Thanks hequn & acqua.csq <ac...@alibaba-inc.com>
On Wed, Nov 14, 2018 at 2:17 PM Hequn Cheng <ch...@gmail.com> wrote:
> Hi Jeff,
>
> The window is not a global window. It is related to a specified key. You
> would have 6 windows after flatMap() and keyBy().
> key: hello with 3 windows
> key: world with 1 window
> key: flink with 1 window
> key: hadoop with 1 window
>
> Best, Hequn
>
>
> On Wed, Nov 14, 2018 at 10:31 AM Jeff Zhang <zj...@gmail.com> wrote:
>
>> Hi all,
>>
>> I am a little confused with the following windows operation. Here's the
>> code,
>>
>> val senv = StreamExecutionEnvironment.getExecutionEnvironment
>> senv.setParallelism(1)
>> val data = senv.fromElements("hello world", "hello flink", "hello hadoop")
>>
>> data.flatMap(line => line.split("\\s"))
>> .map(w => (w, 1))
>> .keyBy(0)
>> .countWindow(2, 1)
>> .sum(1)
>> .print("******************")
>>
>> senv.execute()
>>
>>
>> And this is the output:
>>
>> ******************> (hello,1)
>> ******************> (world,1)
>> ******************> (hello,2)
>> ******************> (flink,1)
>> ******************> (hello,2)
>> ******************> (hadoop,1)
>>
>>
>> As my understanding, here we have 3 windows.
>>
>> window 1
>>
>> (hello, world)
>>
>> window 2
>>
>> (hello, world)
>>
>> (hello, flink)
>>
>> window 3
>>
>> (hello flink)
>>
>> (hello hadoop)
>>
>> So for the first window, we have output (hello, 1) (world, 1)
>>
>> for the second window we should output (hello, 2), (world,1 ), (flink, 1)
>>
>> for the third window we should have output (hello, 2), (flink, 1), (hadoop, 1)
>>
>>
>> But as you can see, in the above I get different result, do I misunderstand the window ? Could anyone help me to understand that ? Thanks
>>
>>
--
Best Regards
Jeff Zhang
Re: Confused window operation
Posted by Hequn Cheng <ch...@gmail.com>.
Hi Jeff,
The window is not a global window. It is related to a specified key. You
would have 6 windows after flatMap() and keyBy().
key: hello with 3 windows
key: world with 1 window
key: flink with 1 window
key: hadoop with 1 window
Best, Hequn
On Wed, Nov 14, 2018 at 10:31 AM Jeff Zhang <zj...@gmail.com> wrote:
> Hi all,
>
> I am a little confused with the following windows operation. Here's the
> code,
>
> val senv = StreamExecutionEnvironment.getExecutionEnvironment
> senv.setParallelism(1)
> val data = senv.fromElements("hello world", "hello flink", "hello hadoop")
>
> data.flatMap(line => line.split("\\s"))
> .map(w => (w, 1))
> .keyBy(0)
> .countWindow(2, 1)
> .sum(1)
> .print("******************")
>
> senv.execute()
>
>
> And this is the output:
>
> ******************> (hello,1)
> ******************> (world,1)
> ******************> (hello,2)
> ******************> (flink,1)
> ******************> (hello,2)
> ******************> (hadoop,1)
>
>
> As my understanding, here we have 3 windows.
>
> window 1
>
> (hello, world)
>
> window 2
>
> (hello, world)
>
> (hello, flink)
>
> window 3
>
> (hello flink)
>
> (hello hadoop)
>
> So for the first window, we have output (hello, 1) (world, 1)
>
> for the second window we should output (hello, 2), (world,1 ), (flink, 1)
>
> for the third window we should have output (hello, 2), (flink, 1), (hadoop, 1)
>
>
> But as you can see, in the above I get different result, do I misunderstand the window ? Could anyone help me to understand that ? Thanks
>
>