You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "John Roesler (JIRA)" <ji...@apache.org> on 2018/12/20 17:02:00 UTC
[jira] [Commented] (KAFKA-7758) When Naming a Repartition Topic
with Aggregations Reuse Repartition Graph Node for Multiple Operations
[ https://issues.apache.org/jira/browse/KAFKA-7758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16726039#comment-16726039 ]
John Roesler commented on KAFKA-7758:
-------------------------------------
+1 from me. I guess there's no compatibility concern here, since the above code is currently impossible.
> When Naming a Repartition Topic with Aggregations Reuse Repartition Graph Node for Multiple Operations
> ------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-7758
> URL: https://issues.apache.org/jira/browse/KAFKA-7758
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Affects Versions: 2.1.0
> Reporter: Bill Bejeck
> Assignee: Bill Bejeck
> Priority: Major
> Fix For: 2.2.0
>
>
> When performing aggregations that require repartitioning and the repartition topic name is specified, and using the resulting {{KGroupedStream}} for multiple operations i.e.
>
> {code:java}
> final KGroupedStream<String, String> kGroupedStream = builder.<String, String>stream("topic").selectKey((k, v) -> k).groupByKey(Grouped.as("grouping"));
> kGroupedStream.windowedBy(TimeWindows.of(Duration.ofMillis(10L))).count();
> kGroupedStream.windowedBy(TimeWindows.of(Duration.ofMillis(30L))).count();
> {code}
> If optimizations aren't enabled, Streams will attempt to build two repartition topics of the same name resulting in a failure creating the topology.
>
> However, we have enough information to re-use the existing repartition node via graph nodes used for building the intermediate representation of the topology. This ticket will make the
> behavior of reusing a {{KGroupedStream}} consistent regardless if optimizations are turned on or not.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)