[jira] [Commented] (FLINK-5642) queryable state: race condition with HeadListState

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-5642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15851295#comment-15851295 ] 

ASF GitHub Bot commented on FLINK-5642:
---------------------------------------

Github user NicoK commented on the issue:

    https://github.com/apache/flink/pull/3218
  
    1. Actually, RocksDB state's get() method has the idiom of returning a (deserialized) **copy** with which the user can do whatever he likes to, knowing that changes are not reflected in the state back-end. This is also what the window evictors rely on, e.g. `TimeEvictor#evict()`: it uses list state and iterates over the elements removing the ones it wants to remove. Later on, `EvictingWindowOperator#emitWindowContents()` clears the list state and (re-)adds all remaining elements of that iterator back to the list state. Forbidding `remove()` for all cases would require a lot of refactoring there with potential affects in user-defined evictors, too. Since window operators are not queryable though, that was the next-best and least-invasive solution since returning an actual copy in the heap list state implementation was also not desired.
    
    2. Sounds nice, I'll do some changes to create a version without the code branch using a specialised class for queryable list state. The branching then moves to `HeapKeyedStateBackend#createListState` where it is only called once per creation.
    
    2. The specialised `ArrayList` implementation sounds like a nice idea but unfortunately, the window evictors' use mentioned above does not follow the assumption of an ever-growing list and makes things more complicated again.


> queryable state: race condition with HeadListState
> --------------------------------------------------
>
>                 Key: FLINK-5642
>                 URL: https://issues.apache.org/jira/browse/FLINK-5642
>             Project: Flink
>          Issue Type: Bug
>          Components: Queryable State
>    Affects Versions: 1.2.0
>            Reporter: Nico Kruber
>            Assignee: Nico Kruber
>
> If queryable state accesses a HeapListState instance that is being modified during the value's serialisation, it may crash, e.g. with a NullPointerException during the serialisation or with an EOFException during de-serialisation.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)