[jira] [Commented] (STORM-1131) Kafka Spout can get into a loop of refreshing offset when there are failed messages

["Aaron Dossett (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/STORM-1131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14976330#comment-14976330 ] 

Aaron Dossett commented on STORM-1131:
--------------------------------------

Those links are to private repos.

> Kafka Spout can get into a loop of refreshing offset when there are failed messages
> -----------------------------------------------------------------------------------
>
>                 Key: STORM-1131
>                 URL: https://issues.apache.org/jira/browse/STORM-1131
>             Project: Apache Storm
>          Issue Type: Bug
>          Components: storm-kafka
>            Reporter: Sriharsha Chintalapani
>            Assignee: Priyank Shah
>            Priority: Critical
>             Fix For: 0.11.0
>
>
> When Kafka spout offset is out of range we update _emittedToOffset
> https://github.com/hortonworks/storm/blob/2.3-maint/external/storm-kafka/src/jvm/storm/kafka/PartitionManager.java#L172
> and return immediately.
> If there are failed messages https://github.com/hortonworks/storm/blob/2.3-maint/external/storm-kafka/src/jvm/storm/kafka/PartitionManager.java#L162
> we never update the local offset to _emittedToOffset. 
> We should discard failedMessages if there is offsetOutOfexception as there is no point in keeping these messages around. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)