You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by calvin beloy <cc...@yahoo.com> on 2022/06/04 14:12:01 UTC

Re: Re:How to achieve processing model as Storm

Hi Xuyang,
What will happen if there's a lot of queries (thousands or extreme millions) ? Looping thru lots of queries won't be performant. How to scale in this case? Is there's something like partition the broadcasted copies and somehow send the data event to all the partition broadcast copies. 
Thanks, Calvin 
Sent from Yahoo Mail on Android 
 
  On Sun, May 29, 2022 at 9:58 PM, Xuyang<xy...@163.com> wrote:   Hi, it seems like you want a broadcast join, and the query stream is the broadcast side. You can ref this[1] for more details.

[1] https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/fault-tolerance/broadcast_state/


At 2022-05-27 22:10:04, "calvin beloy" <cc...@yahoo.com> wrote:

Hi,
Is it possible to have the same Storm processing model as described below image?

The doc and query stream don't have a common key to keyby on. The queries is like a distributed shard with replicas to handle high volume of docs and high volume of queries. What could be the closest flink processing model that includes state.
Appreciate your help.
Thanks, Calvin

Sent from Yahoo Mail on Android