[jira] [Commented] (SAMZA-1914) out of range starting offset for EH consumer

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SAMZA-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629157#comment-16629157 ] 

ASF GitHub Bot commented on SAMZA-1914:
---------------------------------------

GitHub user lhaiesp opened a pull request:

    https://github.com/apache/samza/pull/664

    SAMZA-1914: fix out of range starting offset in EH consumer

    

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/lhaiesp/samza master

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/samza/pull/664.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #664
    
----
commit 563423c02a212cf96391c8adbc98d5d37f8848d8
Author: Hai Lu <ha...@...>
Date:   2018-09-26T17:14:07Z

    SAMZA-1914: fix out of range starting offset in EH consumer

----


> out of range starting offset for EH consumer
> --------------------------------------------
>
>                 Key: SAMZA-1914
>                 URL: https://issues.apache.org/jira/browse/SAMZA-1914
>             Project: Samza
>          Issue Type: Bug
>            Reporter: Hai
>            Assignee: Hai
>            Priority: Major
>
> In EventHubs today we use offset + 1 as the next offset. This would cause problem if the consumer restarts and there is no event produced into the server. Because then offset + 1 would be an out of range offset and eventhubs client would fail to initialize.
> The proposed fix here is to always use offset (+0) as the "next" offset. And when we initialize the EH client, we specify to exclude the offset that we provide. This way we won't be reprocessing the last event and we avoid out of range offset error.
> We need to be careful that this doesn't cause Brooklin to miss event; so should also double check the EH connector in Brooklin as well.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)