You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by M Singh <ma...@yahoo.com> on 2022/01/22 20:24:28 UTC

Apache Flink - Can AllWindowedStream be parallel ?

Hi Folks:
The documentation for AllWindowedStream (https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/operators/overview/#datastream-rarr-allwindowedstream) has a note:

This is in many cases a non-parallel transformation. All records will be gathered in one task for the windowAll operator.


Does this mean that in some cases it might be parallel ?  If so, is there an example of such a scenario ?
Thanks

Re: Apache Flink - Can AllWindowedStream be parallel ?

Posted by M Singh <ma...@yahoo.com>.

 
Thanks Yun for your response, but I a not sure why the doc states that in many cases it is non-parallel.  

    On Monday, January 24, 2022, 01:30:27 AM EST, Yun Tang <my...@live.com> wrote:  
 
 #yiv5427582454 P {margin-top:0;margin-bottom:0;}Hi Singh,
All the output operator transformed by AllWindowedStream would be SingleOutputStreamOperator, which cannot be parallel.

[1] https://github.com/apache/flink/blob/master/flink-streaming-scala/src/main/scala/org/apache/flink/streaming/api/scala/AllWindowedStream.scala
BestYun TangFrom: M Singh <ma...@yahoo.com>
Sent: Sunday, January 23, 2022 4:24
To: User-Flink <us...@flink.apache.org>
Subject: Apache Flink - Can AllWindowedStream be parallel ? Hi Folks:
The documentation for AllWindowedStream (https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/operators/overview/#datastream-rarr-allwindowedstream) has a note:

This is inmany cases a non-parallel transformation. All records will be gathered in one task for the windowAll operator.


Does this mean that in some cases it might be parallel ?  If so, is there an example of such a scenario ?
Thanks

Re: Apache Flink - Can AllWindowedStream be parallel ?

Posted by Yun Tang <my...@live.com>.

Hi Singh,

All the output operator transformed by AllWindowedStream would be SingleOutputStreamOperator, which cannot be parallel.


[1] https://github.com/apache/flink/blob/master/flink-streaming-scala/src/main/scala/org/apache/flink/streaming/api/scala/AllWindowedStream.scala

Best
Yun Tang
________________________________
From: M Singh <ma...@yahoo.com>
Sent: Sunday, January 23, 2022 4:24
To: User-Flink <us...@flink.apache.org>
Subject: Apache Flink - Can AllWindowedStream be parallel ?

Hi Folks:

The documentation for AllWindowedStream (https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/operators/overview/#datastream-rarr-allwindowedstream) has a note:


This is in many cases a non-parallel transformation. All records will be gathered in one task for the windowAll operator.


Does this mean that in some cases it might be parallel ?  If so, is there an example of such a scenario ?

Thanks