[jira] [Resolved] (STORM-2869) KafkaSpout discards all pending records when adjusting the consumer position after a commit

["Jungtaek Lim (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/STORM-2869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jungtaek Lim resolved STORM-2869.
---------------------------------
       Resolution: Fixed
    Fix Version/s: 1.2.0
                   2.0.0

Thanks [~Srdo], I merged into master and 1.x-branch.

> KafkaSpout discards all pending records when adjusting the consumer position after a commit
> -------------------------------------------------------------------------------------------
>
>                 Key: STORM-2869
>                 URL: https://issues.apache.org/jira/browse/STORM-2869
>             Project: Apache Storm
>          Issue Type: Bug
>          Components: storm-kafka-client
>    Affects Versions: 2.0.0, 1.2.0
>            Reporter: Stig Rohde Døssing
>            Assignee: Stig Rohde Døssing
>              Labels: pull-request-available
>             Fix For: 2.0.0, 1.2.0
>
>          Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> As part of the STORM-2666 fix the spout clears out waitingToEmit when the consumer position falls behind the committed offset during a commit. We only need to do it for the affected partition, and then only for the records that are behind the committed offset.
> Also the validation check in emitOrRetryTuple is slightly too permissive, it should check whether the current record is behind the committed offset, not whether the consumer position is behind the committed offset.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)