[jira] [Updated] (KAFKA-3778) Change the RocksDB implementation for KStream windowed aggregations, to not use “range queries” but multiple gets

["Guozhang Wang (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-3778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Guozhang Wang updated KAFKA-3778:
---------------------------------
    Description: 
RocksDbWindowStore currently does not use caches, but its window segments implemented as RocksDbStore does. However, its range query {{fetch(key, fromTime, toTime)}} will cause all its touched segments' cache to be flushed. 

After KAFKA-3777, we should change its implementation for KStreamWindowAggregation / KStreamWindowReduce to not use {{fetch}}, but just as multiple {{get}} calls on the underlying segments, one for each affected window range.

  was:RocksDbWindowStore currently does not use caches (while RocksDbStore does). Enable caching for RocksDbWindowStore. This might require changing the API from using range queries to using multiple gets(), one for each sub-range.


> Change the RocksDB implementation for KStream windowed aggregations, to not use “range queries” but multiple gets
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-3778
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3778
>             Project: Kafka
>          Issue Type: Sub-task
>          Components: streams
>    Affects Versions: 0.10.1.0
>            Reporter: Eno Thereska
>             Fix For: 0.10.1.0
>
>
> RocksDbWindowStore currently does not use caches, but its window segments implemented as RocksDbStore does. However, its range query {{fetch(key, fromTime, toTime)}} will cause all its touched segments' cache to be flushed. 
> After KAFKA-3777, we should change its implementation for KStreamWindowAggregation / KStreamWindowReduce to not use {{fetch}}, but just as multiple {{get}} calls on the underlying segments, one for each affected window range.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)