[jira] [Commented] (KAFKA-6182) Automatic co-partitioning of topics via automatic intermediate topic with matching partitions

["Matthias J. Sax (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-6182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16956539#comment-16956539 ] 

Matthias J. Sax commented on KAFKA-6182:
----------------------------------------

Does KIP-221 ([https://cwiki.apache.org/confluence/display/KAFKA/KIP-221%3A+Enhance+DSL+with+Connecting+Topic+Creation+and+Repartition+Hint]) meet the request of this ticket? Repartitioning is not fully automated, but a full automation is rather hard, because at DSL->Topology translation time, the number of topic partitions are not known and hence, a repartition topic cannot be inserted easily.

> Automatic co-partitioning of topics via automatic intermediate topic with matching partitions
> ---------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-6182
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6182
>             Project: Kafka
>          Issue Type: New Feature
>          Components: streams
>    Affects Versions: 1.0.0
>            Reporter: Antony Stubbs
>            Priority: Major
>
> Currently it is up to the user to ensure that two input topics for a join have the same number of partitions, and if they don't, manually create an intermediate topic, and send the stream #through that topic first, and then performing the join.
> It would be great to have Kafka streams detect this and at least give the user the option to create an intermediate topic automatically with the same number of partitions as the topic being joined with.
> See https://docs.confluent.io/current/streams/developer-guide.html#joins-require-co-partitioning-of-the-input-data



--
This message was sent by Atlassian Jira
(v8.3.4#803005)