[jira] [Updated] (SAMZA-1481) KafkaSystemConsumer does not properly track highwatermark in some cases.

["Xinyu Liu (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SAMZA-1481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Xinyu Liu updated SAMZA-1481:
-----------------------------
    Fix Version/s:     (was: 0.14.0)
                   0.15.0

> KafkaSystemConsumer does not properly track highwatermark in some cases. 
> -------------------------------------------------------------------------
>
>                 Key: SAMZA-1481
>                 URL: https://issues.apache.org/jira/browse/SAMZA-1481
>             Project: Samza
>          Issue Type: Bug
>            Reporter: Jake Maes
>            Assignee: Jake Maes
>            Priority: Trivial
>             Fix For: 0.15.0
>
>
> The KafkaSystemConsumer currently only updates the isAtHighWatermark when it receives a payload from Kafka. 
> This does not work when:
> 1. The topic is empty
> 2. The oldest or newest offset changes on the broker
> 3. An invalid initial offset is used for the consumer and the Kafka auto reset policy kicks in. 
> The isAtHighWatermark flag is crucial for topics that we bootstrap (checkpoint, changelog, coordinator) because we poll infinitely until we're at head of stream. And if this flag is wrong, we poll indefinitely.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)