[jira] [Resolved] (STORM-2608) Out Of Range Offsets Should Be Removed From Pending Queue

["Jungtaek Lim (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/STORM-2608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jungtaek Lim resolved STORM-2608.
---------------------------------
       Resolution: Fixed
         Assignee: Kevin Conaway
    Fix Version/s: 1.2.0
                   1.1.1
                   2.0.0

Thanks [~kevinconaway], I merged into master, 1.x, 1.1.x branches.

> Out Of Range Offsets Should Be Removed From Pending Queue
> ---------------------------------------------------------
>
>                 Key: STORM-2608
>                 URL: https://issues.apache.org/jira/browse/STORM-2608
>             Project: Apache Storm
>          Issue Type: Bug
>          Components: storm-kafka
>    Affects Versions: 0.10.0, 1.0.4, 1.1.1
>            Reporter: Kevin Conaway
>            Assignee: Kevin Conaway
>             Fix For: 2.0.0, 1.1.1, 1.2.0
>
>          Time Spent: 4h
>  Remaining Estimate: 0h
>
> There is a bug that occurs when failed tuples are invalidated due to Kafka throwing a {{TopicOffsetOutOfRangeException}}. 
> Below is what happens:
> - Spout emits tuples
> - Offsets are added to the _pending_ tree
> - Some tuples fail and are added to the _failedMsgRetryManager_
> - On the next fetch request, a {{TopicOffsetOutOfRangeException}} is thrown and the new offset is _after_ the offset that are currently sitting in both the _pending_ tree and the _failedMsgRetryManager_
> - All offsets smaller than the the new offset are removed from the _failedMsgRetryManager_ but *not* the _pending_ tree.
> - Since those offsets were removed from the _failedMsgRetryManager_ they will never be retried and thus never get removed from _pending_
> - {{lastCommittedOffset()}} will always return the same value which means that offset in zookeeper for that partition will never get updated.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)