Queryable State race condition or serialization errors?

[burgesschen <tc...@bloomberg.net>]

Hi Guys,

I observed some strange behaviors while using Queryable state with Flink
1.6.2. Here is the story:

My state is of type MapState[String, Map[String, String]]. the inner map is
frequently updated. Upon querying, sometimes the returned inner map can miss
some fields. What's more, sometimes the returned inner map has the values
assigned to other keys! 

Changing the type to MapState[String, String] seem to solve the problem.

The code is a little too deep to dig into. But my guess is that when the
state is being updated and queried at the same time, there can be a race
condition and cause data corruption. Please let me know if you have a better
idea what could be happening. Much appreciated!

Best,
Burgess Chen  



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Queryable State race condition or serialization errors?

["Tzu-Li (Gordon) Tai" <tz...@apache.org>]

Hi Burgess,

Would you be able to provide a minimal project that can reproduce your
error?
That would help a lot with figuring out the issue.
If you prefer to share that only privately, please feel free to send me a
private email with the details.
Another thing you can do is set logging level to "DEBUG". We have some
checks enabled at that level to see if serializers are being concurrently
used across threads.

Cheers,
Gordon

On Tue, May 21, 2019 at 9:59 PM burgesschen <tc...@bloomberg.net> wrote:

> Hi Gary.
>
> Thanks for the reply. I am using RocksDBStateBackend though.
>
> Best,
> Chen
>
>
>
> --
> Sent from:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>

Re: Queryable State race condition or serialization errors?

[burgesschen <tc...@bloomberg.net>]

Hi Gary.

Thanks for the reply. I am using RocksDBStateBackend though. 

Best,
Chen



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Queryable State race condition or serialization errors?

[Gary Yao <ga...@ververica.com>]

Hi Burgess Chen,

If you are using MemoryStateBackend or FsStateBackend, you can observe race
conditions on the state objects. However, the RocksDBStateBackend should be
safe from these issues [1].

Best,
Gary

[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.6/dev/stream/state/queryable_state.html

On Tue, May 21, 2019 at 5:06 AM burgesschen <tc...@bloomberg.net> wrote:

> Hi Guys,
>
> I observed some strange behaviors while using Queryable state with Flink
> 1.6.2. Here is the story:
>
> My state is of type MapState[String, Map[String, String]]. the inner map is
> frequently updated. Upon querying, sometimes the returned inner map can
> miss
> some fields. What's more, sometimes the returned inner map has the values
> assigned to other keys!
>
> Changing the type to MapState[String, String] seem to solve the problem.
>
> The code is a little too deep to dig into. But my guess is that when the
> state is being updated and queried at the same time, there can be a race
> condition and cause data corruption. Please let me know if you have a
> better
> idea what could be happening. Much appreciated!
>
> Best,
> Burgess Chen
>
>
>
> --
> Sent from:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>