You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@storm.apache.org by "Sriharsha Chintalapani (JIRA)" <ji...@apache.org> on 2015/10/27 02:56:27 UTC

[jira] [Created] (STORM-1131) Kafka Spout can get into a loop of refreshing offset when there are failed messages

Sriharsha Chintalapani created STORM-1131:
---------------------------------------------

             Summary: Kafka Spout can get into a loop of refreshing offset when there are failed messages
                 Key: STORM-1131
                 URL: https://issues.apache.org/jira/browse/STORM-1131
             Project: Apache Storm
          Issue Type: Bug
          Components: storm-kafka
            Reporter: Sriharsha Chintalapani
            Assignee: Priyank Shah
            Priority: Critical
             Fix For: 0.11.0


When Kafka spout offset is out of range we update _emittedToOffset
https://github.com/hortonworks/storm/blob/2.3-maint/external/storm-kafka/src/jvm/storm/kafka/PartitionManager.java#L172
and return immediately.
If there are failed messages https://github.com/hortonworks/storm/blob/2.3-maint/external/storm-kafka/src/jvm/storm/kafka/PartitionManager.java#L162
we never update the local offset to _emittedToOffset. 

We should discard failedMessages if there is offsetOutOfexception as there is no point in keeping these messages around. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)