[GitHub] flink issue #3218: [FLINK-5642][query] fix a race condition with HeadListSta...

[NicoK <gi...@git.apache.org>]

Github user NicoK commented on the issue:

    https://github.com/apache/flink/pull/3218
  
    1. Actually, RocksDB state's get() method has the idiom of returning a (deserialized) **copy** with which the user can do whatever he likes to, knowing that changes are not reflected in the state back-end. This is also what the window evictors rely on, e.g. `TimeEvictor#evict()`: it uses list state and iterates over the elements removing the ones it wants to remove. Later on, `EvictingWindowOperator#emitWindowContents()` clears the list state and (re-)adds all remaining elements of that iterator back to the list state. Forbidding `remove()` for all cases would require a lot of refactoring there with potential affects in user-defined evictors, too. Since window operators are not queryable though, that was the next-best and least-invasive solution since returning an actual copy in the heap list state implementation was also not desired.
    
    2. Sounds nice, I'll do some changes to create a version without the code branch using a specialised class for queryable list state. The branching then moves to `HeapKeyedStateBackend#createListState` where it is only called once per creation.
    
    2. The specialised `ArrayList` implementation sounds like a nice idea but unfortunately, the window evictors' use mentioned above does not follow the assumption of an ever-growing list and makes things more complicated again.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---