[GitHub] flink issue #5908: [FLINK-9182]async checkpoints for timer service

[StefanRRichter <gi...@git.apache.org>]

Github user StefanRRichter commented on the issue:

    https://github.com/apache/flink/pull/5908
  
    Maybe let me add some more. First, about introducing a separate new state handle. Our long term plan is actually to integrate timers more closely with the backends, so that we can also have a timer service in RocksDB. In general, I would target for reducing the handles(e.g. timers could then be considered also as managed keyed stated) instead of adding more. Second, I would probably suggest a simpler model for the async snapshots. You dropped the idea of making flat copies, but I wonder if this was premature. I can see that maybe calling `set.toArray(...)` per keygroup could (maybe) turn out a bit slow because it has to potentially iterate and flatten linked entries. However, with async snapshots, we could get rid of the key-group partitioning of sets, and instead do a flat copy of the internal array of the priority queue. This would translate to just a single memcopy call internally, which is very efficient. In the async part, we can still partition the timers by key-group in a 
 similar way as the copy-on-write state table does. This would avoid slowing down the event processing path (in fact improving it be unifying the sets) and also keep the approach very straight forward and less invasive.


---