Re: Share state across operators

["m@xi" <ma...@gmail.com>]

Hey Timo!

I am using Java for my implementation and I have found this article [1] in
stackoverflow for simulating the Either<A,B> in Java. 

Now, for my case, I have a coordinator instance (parallelism = 1) that needs
to both distribute incoming tuples in a specific way, but also needs to
redistribute some state (previously distributed tuples) that maintains to
other downstream operators.

I am wondering whether I should use SideOutputs or Either<A,B> to succeed
the aforementioned goal. Nevertheless, I am wondering whether there are any
differences regarding efficiency/effectiveness of the above 2 proposed
workarounds to share state.

Also, in the case of Either<A,B>, I am first doing all my distribution of Bs
(parts of the state) and afterwards in the same .processElement() function I
am distributing the newly arrived tuples to the downstream operators. So,
given that my coordinator's parallelism is 1, I assume that the order of my
collect()-s is preserved, meaning the first the state redistribution will
get settled in the downstream operators, and afterwards the processing of
new tuples will happen. Is that correct? (I do not no if inside the network
buffers things get twisted somehow).

Thanks a lot in advance!

Best,
Max

[1] --
https://stackoverflow.com/questions/9975836/how-can-i-simulate-haskells-either-a-b-in-java




--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Share state across operators

["m@xi" <ma...@gmail.com>]

Thank a lot Timo!

Best,
Max



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Share state across operators

[Timo Walther <tw...@apache.org>]

Hi Max,

I would go with the Either<A, B> approach if you want to ensure that the 
initital state and the first element arrive in the right order. 
Performance-wise there should not be a big different between both 
approaches. The side outputs are more meant for have a side channel 
beside the main stream that is not keyed anymore (e.g. for late elements 
that need a special treatment next to the main processing).

Regards,
Timo

Am 12.03.18 um 10:17 schrieb m@xi:
> Hey Flinker!
>
> Anyone? Anybody? Someone with experience or any idea on the question above?
>
> Best,
> Max
>
>
>
> --
> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Share state across operators

["m@xi" <ma...@gmail.com>]

Hey Flinker!

Anyone? Anybody? Someone with experience or any idea on the question above?

Best,
Max



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/