You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kafka.apache.org by "Guozhang Wang (JIRA)" <ji...@apache.org> on 2016/05/04 22:59:12 UTC

[jira] [Created] (KAFKA-3658) RocksDBWindowStore should guarantee a single window locates completely in one segment

Guozhang Wang created KAFKA-3658:
------------------------------------

             Summary: RocksDBWindowStore should guarantee a single window locates completely in one segment
                 Key: KAFKA-3658
                 URL: https://issues.apache.org/jira/browse/KAFKA-3658
             Project: Kafka
          Issue Type: Bug
          Components: streams
            Reporter: Guozhang Wang
            Assignee: Guozhang Wang
             Fix For: 0.10.0.0


As [~hcai@pinterest.com] found out, the current implementation of {{RocksDBWindowStore}} does not guarantee a single window locates completely in one segment, and hence when we expiring a segment, that would result in partial window expiration (i.e. some records of the window are dropped, while some others are still available for queries). We need to fix this issue in setting the segment size to consider the window size.

Another minor issue is that retention size should be validated correctly to be no less than the window size.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)