[jira] [Created] (STORM-3393) OffsetManager doesn't recover after missing offsets

["Greg Chatham (JIRA)" <ji...@apache.org>]

Greg Chatham created STORM-3393:
-----------------------------------

             Summary: OffsetManager doesn't recover after missing offsets
                 Key: STORM-3393
                 URL: https://issues.apache.org/jira/browse/STORM-3393
             Project: Apache Storm
          Issue Type: Bug
          Components: storm-client
    Affects Versions: 1.1.3, 1.1.0
            Reporter: Greg Chatham


When missing offsets are encountered, but a committable offset exists after the missing offset, the condition is detected and logged but not properly processed.  You will see three log messages in this case:
{code:java}
Processed non-sequential offset.  The earliest uncommitted offset is no longer part of the topic.  Missing offset: [{}], Processed: [{}]
Found committable offset: [{}] after missing offset: [{}], skipping to the committable offset
Topic-partition [{}] has no offsets ready to be committed{code}
However, this is not the proper handling.  While a committable offset has been found, the found flag is not set to true (resulting in the 3rd log message).

The fix is to add a found=true within this logic:

In OffsetManager.java
{code:java}
if (nextEmittedOffset != null && currOffset == nextEmittedOffset) {
                        LOG.debug("Found committable offset: [{}] after missing offset: [{}], skipping to the committable offset",
                            currOffset, nextCommitOffset);
                        nextCommitOffset = currOffset + 1;
                        found = true;       //  ADD THIS LINE TO FIX THIS BUG
                    }{code}
This results in offsets not being committed, although processing will continue



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)