You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Jeff Zhang <zj...@gmail.com> on 2018/11/14 02:31:23 UTC

Confused window operation

Hi all,

I am a little confused with the following windows operation. Here's the
code,

val senv = StreamExecutionEnvironment.getExecutionEnvironment
senv.setParallelism(1)
val data = senv.fromElements("hello world", "hello flink", "hello hadoop")

data.flatMap(line => line.split("\\s"))
  .map(w => (w, 1))
  .keyBy(0)
  .countWindow(2, 1)
  .sum(1)
  .print("******************")

senv.execute()


And this is the output:

******************> (hello,1)
******************> (world,1)
******************> (hello,2)
******************> (flink,1)
******************> (hello,2)
******************> (hadoop,1)


As my understanding, here we have 3 windows.

window 1

(hello, world)

window 2

(hello, world)

(hello, flink)

window 3

(hello flink)

(hello hadoop)

So for the first window, we have output (hello, 1) (world, 1)

for the second window we should output (hello, 2), (world,1 ), (flink, 1)

for the third window we should have output (hello, 2), (flink, 1), (hadoop, 1)


But as you can see, in the above I get different result, do I
misunderstand the window ? Could anyone help me to understand that ?
Thanks

Re: Confused window operation

Posted by Jeff Zhang <zj...@gmail.com>.
Thanks hequn & acqua.csq <ac...@alibaba-inc.com>

On Wed, Nov 14, 2018 at 2:17 PM Hequn Cheng <ch...@gmail.com> wrote:

> Hi Jeff,
>
> The window is not a global window. It is related to a specified key. You
> would have 6 windows after flatMap() and keyBy().
> key: hello with 3 windows
> key: world with 1 window
> key: flink with 1 window
> key: hadoop with 1 window
>
> Best, Hequn
>
>
> On Wed, Nov 14, 2018 at 10:31 AM Jeff Zhang <zj...@gmail.com> wrote:
>
>> Hi all,
>>
>> I am a little confused with the following windows operation. Here's the
>> code,
>>
>> val senv = StreamExecutionEnvironment.getExecutionEnvironment
>> senv.setParallelism(1)
>> val data = senv.fromElements("hello world", "hello flink", "hello hadoop")
>>
>> data.flatMap(line => line.split("\\s"))
>>   .map(w => (w, 1))
>>   .keyBy(0)
>>   .countWindow(2, 1)
>>   .sum(1)
>>   .print("******************")
>>
>> senv.execute()
>>
>>
>> And this is the output:
>>
>> ******************> (hello,1)
>> ******************> (world,1)
>> ******************> (hello,2)
>> ******************> (flink,1)
>> ******************> (hello,2)
>> ******************> (hadoop,1)
>>
>>
>> As my understanding, here we have 3 windows.
>>
>> window 1
>>
>> (hello, world)
>>
>> window 2
>>
>> (hello, world)
>>
>> (hello, flink)
>>
>> window 3
>>
>> (hello flink)
>>
>> (hello hadoop)
>>
>> So for the first window, we have output (hello, 1) (world, 1)
>>
>> for the second window we should output (hello, 2), (world,1 ), (flink, 1)
>>
>> for the third window we should have output (hello, 2), (flink, 1), (hadoop, 1)
>>
>>
>> But as you can see, in the above I get different result, do I misunderstand the window ? Could anyone help me to understand that ? Thanks
>>
>>

-- 
Best Regards

Jeff Zhang

Re: Confused window operation

Posted by Hequn Cheng <ch...@gmail.com>.
Hi Jeff,

The window is not a global window. It is related to a specified key. You
would have 6 windows after flatMap() and keyBy().
key: hello with 3 windows
key: world with 1 window
key: flink with 1 window
key: hadoop with 1 window

Best, Hequn


On Wed, Nov 14, 2018 at 10:31 AM Jeff Zhang <zj...@gmail.com> wrote:

> Hi all,
>
> I am a little confused with the following windows operation. Here's the
> code,
>
> val senv = StreamExecutionEnvironment.getExecutionEnvironment
> senv.setParallelism(1)
> val data = senv.fromElements("hello world", "hello flink", "hello hadoop")
>
> data.flatMap(line => line.split("\\s"))
>   .map(w => (w, 1))
>   .keyBy(0)
>   .countWindow(2, 1)
>   .sum(1)
>   .print("******************")
>
> senv.execute()
>
>
> And this is the output:
>
> ******************> (hello,1)
> ******************> (world,1)
> ******************> (hello,2)
> ******************> (flink,1)
> ******************> (hello,2)
> ******************> (hadoop,1)
>
>
> As my understanding, here we have 3 windows.
>
> window 1
>
> (hello, world)
>
> window 2
>
> (hello, world)
>
> (hello, flink)
>
> window 3
>
> (hello flink)
>
> (hello hadoop)
>
> So for the first window, we have output (hello, 1) (world, 1)
>
> for the second window we should output (hello, 2), (world,1 ), (flink, 1)
>
> for the third window we should have output (hello, 2), (flink, 1), (hadoop, 1)
>
>
> But as you can see, in the above I get different result, do I misunderstand the window ? Could anyone help me to understand that ? Thanks
>
>