[jira] [Commented] (KAFKA-6367) Fix StateRestoreListener To Use Correct Ending Offset

["Matthias J. Sax (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-6367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16292054#comment-16292054 ] 

Matthias J. Sax commented on KAFKA-6367:
----------------------------------------

{quote}
We can't automatically use {{nextPosition - 1}} as this could be a commit marker.
{quote}

It a valid point, but I am not sure if returning the offset of a commit marker would be an issue or not. The provided offset is called {{batchEndOffset}} -- this does not imply "offset of last restored record"; or does it?

If we don't return commit marker offsets, we get "gaps" -- thus, if someone tracks the offsets in the callback and assume "completeness" this would be violated. On the other hand, if there are aborted messages, we might have gaps anyway...

Not sure what other think. To me it seems, we don't get much advantage if we don't "return" commit markers but get lot of additional complexity. Thus, I tend to think that using {{nextPosition - 1}} might be ok.

> Fix StateRestoreListener To Use Correct Ending Offset
> -----------------------------------------------------
>
>                 Key: KAFKA-6367
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6367
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions: 0.11.0.0, 1.0.0
>            Reporter: Bill Bejeck
>            Assignee: Bill Bejeck
>             Fix For: 1.0.1
>
>
> {{StateRestoreListener#restoreBatchCompleted}} takes the {{nextPosition}} long  but the {{nextPosition}} is not correct, it should be the offset of the latest restored offset, but {{nextPosition}} is the offset of the first not restored offset.
> We can't automatically use {{nextPosition}} - 1 as this could be a commit marker.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)