[jira] [Resolved] (STORM-2546) Kafka spout can stall / get stuck due to edge case with failing tuples

["Stig Rohde Døssing (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/STORM-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Stig Rohde Døssing resolved STORM-2546.
---------------------------------------
       Resolution: Fixed
    Fix Version/s: 1.2.0

> Kafka spout can stall / get stuck due to edge case with failing tuples
> ----------------------------------------------------------------------
>
>                 Key: STORM-2546
>                 URL: https://issues.apache.org/jira/browse/STORM-2546
>             Project: Apache Storm
>          Issue Type: Bug
>          Components: storm-kafka-client
>    Affects Versions: 2.0.0, 1.1.0
>            Reporter: Prasanna Ranganathan
>            Assignee: Stig Rohde Døssing
>              Labels: pull-request-available
>             Fix For: 2.0.0, 1.2.0
>
>          Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> The mechanism for replaying a failed tuple involves seeking the kafka consumer to the failing offset and then re-emitting it into the topology. A tuple, when emitted the first time, will have an entry created in OffsetManager. This entry will be removed only after the tuple is successfully acknowledged and its offset successfully committed. Till then, commits for offsets beyond the failing offset for that TopicPartition will be blocked.
> It is possible that when the spout seeks the consumer to the failing offset, the corresponding kafka message is not returned in the poll response. This can happen due to that offset being deleted or compacted away. In this scenario that partition will be blocked from committing and progressing.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)