Can balance by attribute guarantee the order of the FlowFile？

["wanglei2@geekplus.com.cn" <wa...@geekplus.com.cn>]

Since data will be balanced to different node, the order after balanced will be the same as before?

Thanks,
Lei



wanglei2@geekplus.com.cn

Re: Re: Can balance by attribute guarantee the order of the FlowFile？

["wanglei2@geekplus.com.cn" <wa...@geekplus.com.cn>]

Hi Joe, 

In high data volume circumstance, seems it is impossible to keep order accross the whole cluster. 
Even if EnforceOrder  is used,  it can be only scheduled  on primary node and becomes  a performance bettleneck.

I tried to set the concurrency to 1 from the begining and not load balance. But if the execution set to all nodes, there will be one thread on every node. There will be also  order issues in the final. Is there any method that can let the processor run only on one specified node?

Thanks,
Lei





wanglei2@geekplus.com.cn
 
Sender: Joe Witt
Send Time: 2019-10-16 20:27
Receiver: users
Subject: Re: Can balance by attribute guarantee the order of the FlowFile？
Lei

The order won't necessarily be the same. You'd want EnforceOrder to follow the load balanced connection most likely.

It is important to keep in mind the queues are basically insertion order and the system is inherently multi-threaded so the data can be shuffled in such cases.  You can carefully design your flow and use mechanisms like EnforceOrder to help.

Thanks

On Wed, Oct 16, 2019 at 8:25 AM wanglei2@geekplus.com.cn <wa...@geekplus.com.cn> wrote:

Since data will be balanced to different node, the order after balanced will be the same as before?

Thanks,
Lei



wanglei2@geekplus.com.cn

Re: Can balance by attribute guarantee the order of the FlowFile？

[Joe Witt <jo...@gmail.com>]

Lei

The order won't necessarily be the same. You'd want EnforceOrder to follow
the load balanced connection most likely.

It is important to keep in mind the queues are basically insertion order
and the system is inherently multi-threaded so the data can be shuffled in
such cases.  You can carefully design your flow and use mechanisms like
EnforceOrder to help.

Thanks

On Wed, Oct 16, 2019 at 8:25 AM wanglei2@geekplus.com.cn <
wanglei2@geekplus.com.cn> wrote:

>
> Since data will be balanced to different node, the order after balanced
> will be the same as before?
>
> Thanks,
> Lei
>
> ------------------------------
> wanglei2@geekplus.com.cn
>