[jira] [Commented] (KAFKA-6717) TopicPartition Assined twice to a consumer group for 2 consumer instances

["Vahid Hashemian (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16425964#comment-16425964 ] 

Vahid Hashemian commented on KAFKA-6717:
----------------------------------------

[~Yuancheng] Would it be possible to share a bit more about the use case?

I see consumers are subscribing to a pattern. Is this pattern frequently result in a different set of topics? What were the topics that matched the regex when the issue occurred? Anything extra you can share to better understand the issue would be a great help. Thanks!

> TopicPartition Assined twice to a consumer group for 2 consumer instances 
> --------------------------------------------------------------------------
>
>                 Key: KAFKA-6717
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6717
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 0.11.0.1
>            Reporter: Yuancheng PENG
>            Priority: Major
>
> I'm using \{{StickyAssignor}} for consuming more than 100 topics with certain pattern.
> There are 10 consumers with the same group id.
> I expected that topic-partition to be assigned to only one consumer instance. However some topic partitions are assigned twice in 2 different difference instance, hence the consumer group process duplicate messages.
> {code:java}
> props.put(ConsumerConfig.PARTITION_ASSIGNMENT_STRATEGY_CONFIG, Collections.singletonList(StickyAssignor.class));
> KafkaConsumer<Record, Record> c = new KafkaConsumer<>(props);
> c.subscribe(Pattern.compile(TOPIC_PATTERN), new NoOpConsumerRebalanceListener());
> {code}
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)