[jira] [Resolved] (KAFKA-6560) Use single-point queries than range queries for windowed aggregation operators

["Guozhang Wang (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-6560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Guozhang Wang resolved KAFKA-6560.
----------------------------------
       Resolution: Fixed
    Fix Version/s: 1.2.0

> Use single-point queries than range queries for windowed aggregation operators
> ------------------------------------------------------------------------------
>
>                 Key: KAFKA-6560
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6560
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: Guozhang Wang
>            Assignee: Guozhang Wang
>            Priority: Critical
>              Labels: needs-kip
>             Fix For: 1.2.0
>
>
> Today for windowed aggregations in Streams DSL, the underlying implementation is leveraging the fetch(key, from, to) API to get all the related windows for a single record to update. However, this is a very inefficient operation with significant amount of CPU time iterating over window stores. On the other hand, since the operator implementation itself have full knowledge of the window specs it can actually translate this operation into multiple single-point queries with the accurate window start timestamp, which would largely reduce the overhead.
> The proposed approach is to add a single fetch API to the WindowedStore and use that in the KStreamWindowedAggregate / KStreamWindowedReduce operators.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)