[jira] [Updated] (KAFKA-3595) Add capability to specify replication compact option for stream store

["Guozhang Wang (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-3595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Guozhang Wang updated KAFKA-3595:
---------------------------------
    Resolution: Fixed
        Status: Resolved  (was: Patch Available)

Issue resolved by pull request 1792
[https://github.com/apache/kafka/pull/1792]

> Add capability to specify replication compact option for stream store
> ---------------------------------------------------------------------
>
>                 Key: KAFKA-3595
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3595
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>    Affects Versions: 0.10.1.0
>            Reporter: Henry Cai
>            Assignee: Damian Guy
>            Priority: Minor
>              Labels: user-experience
>             Fix For: 0.10.1.0
>
>
> Currently state store replication always go through a compact kafka topic. For some state stores, e.g. JoinWindow, there are no duplicates in the store, there is not much benefit using a compacted topic.
> The problem of using compacted topic is the records can stay in kafka broker forever. In my use case, my key is ad_id, it's incrementing all the time, not bounded, I am worried the disk space on broker for that topic will go forever.
> I think we either need the capability to purge the compacted records on broker, or allow us to specify different compact option for state store replication.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)