[jira] [Created] (KAFKA-10892) Add Topology#connectSourceStoreAndTopic as a public method

["Tomohiro Hashidate (Jira)" <ji...@apache.org>]

Tomohiro Hashidate created KAFKA-10892:
------------------------------------------

             Summary: Add Topology#connectSourceStoreAndTopic as a public method
                 Key: KAFKA-10892
                 URL: https://issues.apache.org/jira/browse/KAFKA-10892
             Project: Kafka
          Issue Type: Improvement
          Components: streams
    Affects Versions: 2.7.0
            Reporter: Tomohiro Hashidate


I want Topology#connectSourceStoreAndTopic.

Because I want to use a topic as a source topic directly without a redundant changelog topic for not only KeyValueStore but also WindowStore.

This issue is similar to [KAFKA-6840], but is a suggestion for a more general approach
{code:java}
    public synchronized Topology connectSourceStoreAndTopic(final String sourceStoreName,
                                                            final String topic) {
        internalTopologyBuilder.connectSourceStoreAndTopic(sourceStoreName, topic);
        return this;
    }
{code}
h3. Background

I want to use a topic as a source topic for WindowStore because using WindowStore is suitable for the feature that I'm implementing.
 The records stored in the topic are aggregated with a time window by another application. The size of the topic is over 10TB.
 I want to use the topic as a source topic for WindowStore directly.
 But, I cannot do so on the current interface. 
 I need a redundant topic only for storing the records into WindowStore.

If this API is public, I can use topics incoming from other applications (not only Kafka Streams applications) as source topics for any StateStore implementation without redundant changelog topics.

Of course, I need to implement a processor for storing incoming records in such a case.
 But I think it's not difficult.

Please consider this.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)