[jira] [Commented] (STORM-3393) OffsetManager doesn't recover after missing offsets

["Stig Rohde Døssing (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/STORM-3393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16844996#comment-16844996 ] 

Stig Rohde Døssing commented on STORM-3393:
-------------------------------------------

Please upgrade to the latest Storm version. 1.1.x has known stability issues in storm-kafka-client.

> OffsetManager doesn't recover after missing offsets
> ---------------------------------------------------
>
>                 Key: STORM-3393
>                 URL: https://issues.apache.org/jira/browse/STORM-3393
>             Project: Apache Storm
>          Issue Type: Bug
>          Components: storm-client
>    Affects Versions: 1.1.0, 1.1.3
>            Reporter: Greg Chatham
>            Priority: Major
>
> When missing offsets are encountered, but a committable offset exists after the missing offset, the condition is detected and logged but not properly processed.  You will see three log messages in this case:
> {code:java}
> Processed non-sequential offset.  The earliest uncommitted offset is no longer part of the topic.  Missing offset: [{}], Processed: [{}]
> ...
> Found committable offset: [{}] after missing offset: [{}], skipping to the committable offset
> ...
> Topic-partition [{}] has no offsets ready to be committed{code}
> However, this is not the proper handling.  While a committable offset has been found, the found flag is not set to true (resulting in the 3rd log message).
> The fix is to add a found=true within this logic:
> In OffsetManager.java
> {code:java}
> if (nextEmittedOffset != null && currOffset == nextEmittedOffset) {
>                         LOG.debug("Found committable offset: [{}] after missing offset: [{}], skipping to the committable offset",
>                             currOffset, nextCommitOffset);
>                         nextCommitOffset = currOffset + 1;
>                         found = true;       //  ADD THIS LINE TO FIX THIS BUG
>                     }{code}
> Because of this bug, offsets are not committed properly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)