Thread Locals in Flink Streaming Application

[Vinay Patil <vi...@gmail.com>]

Hi,

I am updating the DTO in one operator , so is it safe to use Thread Local
for it, I am taking care to clear it,

My confusion here is : is it required to create a Thread Local variable in
a streaming application, if the same piece of code is executed by different
threads, does flink handles this ?

Can we make use of open method to avoid ThreadLocal by initializing the
variable there, since the open will be called once per thread (correct me
here)

Can you please explain this to make it clear

Regards,
Vinay Patil

Re: Thread Locals in Flink Streaming Application

[Till Rohrmann <tr...@apache.org>]

Hi Vinay,

I think the answer depends on what you want to achieve.

Subtasks of an operator are all executed by separate threads. Some of them
run in the same JVM others not. So if you want to share data across
subtasks being executed in the same process, you can use static variables
bound to the function of this operator.

In case of chaining, multiple subtasks (all belonging to different
operators, though) are executed by the same thread. Here you can use thread
local variables to share data between these operators in a thread safe
manner.

If you create variables/initialize fields in the open method of an
operator, then this variable is bound to the specific subtask instance
which runs the code.

I hope this helps you a little bit. But I'm not sure whether I got your
question completely right.

Cheers,
Till

On Fri, Sep 16, 2016 at 5:50 PM, Vinay Patil <vi...@gmail.com>
wrote:

> Hi,
>
> I am updating the DTO in one operator , so is it safe to use Thread Local
> for it, I am taking care to clear it,
>
> My confusion here is : is it required to create a Thread Local variable in
> a streaming application, if the same piece of code is executed by different
> threads, does flink handles this ?
>
> Can we make use of open method to avoid ThreadLocal by initializing the
> variable there, since the open will be called once per thread (correct me
> here)
>
> Can you please explain this to make it clear
>
> Regards,
> Vinay Patil
>