You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@storm.apache.org by "Priyank Shah (JIRA)" <ji...@apache.org> on 2017/05/30 16:13:04 UTC

[jira] [Created] (STORM-2538) New kafka spout emits duplicate tuples

Priyank Shah created STORM-2538:
-----------------------------------

             Summary: New kafka spout emits duplicate tuples
                 Key: STORM-2538
                 URL: https://issues.apache.org/jira/browse/STORM-2538
             Project: Apache Storm
          Issue Type: Bug
          Components: storm-kafka-client
    Affects Versions: 2.0.0, 1.x
            Reporter: Priyank Shah
            Priority: Critical


Currently, KafkaSpout in storm-kafka-client can cause duplicate tuples to be emitted. Reason is the implementation of ConsumerRebalanceListener interface is called by kafka everytime a new executor comes up. However, on PartitionsRevoked we already have some in flight tuples and are emitting the same ones from the new executor on which the onPartitionsAssigned was called. We need to make sure that we emit only one tuple per kafka message.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)