You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@storm.apache.org by "Hugo Louro (JIRA)" <ji...@apache.org> on 2017/05/30 16:15:04 UTC

[jira] [Assigned] (STORM-2538) New kafka spout emits duplicate tuples

     [ https://issues.apache.org/jira/browse/STORM-2538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hugo Louro reassigned STORM-2538:
---------------------------------

    Assignee: Hugo Louro

> New kafka spout emits duplicate tuples
> --------------------------------------
>
>                 Key: STORM-2538
>                 URL: https://issues.apache.org/jira/browse/STORM-2538
>             Project: Apache Storm
>          Issue Type: Bug
>          Components: storm-kafka-client
>    Affects Versions: 2.0.0, 1.x
>            Reporter: Priyank Shah
>            Assignee: Hugo Louro
>            Priority: Critical
>
> Currently, KafkaSpout in storm-kafka-client can cause duplicate tuples to be emitted. Reason is the implementation of ConsumerRebalanceListener interface is called by kafka everytime a new executor comes up. However, on PartitionsRevoked we already have some in flight tuples and are emitting the same ones from the new executor on which the onPartitionsAssigned was called. We need to make sure that we emit only one tuple per kafka message.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)