[jira] [Created] (KAFKA-4587) Rethink Unification of Caching with Dedupping

["Guozhang Wang (JIRA)" <ji...@apache.org>]

Guozhang Wang created KAFKA-4587:
------------------------------------

             Summary: Rethink Unification of Caching with Dedupping
                 Key: KAFKA-4587
                 URL: https://issues.apache.org/jira/browse/KAFKA-4587
             Project: Kafka
          Issue Type: Bug
          Components: streams
            Reporter: Guozhang Wang


This is discussed in PR https://github.com/apache/kafka/pull/1588

In order to support user-customizable state store suppliers in the DSL we did the following:

1) Introduce a {{TupleForwarder}} to forward tuples from cached stores that is wrapping user customized stores.
2) Narrow the scope to only dedup on forwarding if it is the default CachingXXStore with wrapper RocksDB. 

With this, the unification of dedupping and caching is less useful now, and we are complicating the inner implementations for forwarding a lot. We need to re-consider this feature with finer granularity of turning on / off caching per store, potentially with explicit triggers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)