You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@beam.apache.org by "Jean-Baptiste Onofré (JIRA)" <ji...@apache.org> on 2017/07/03 14:57:00 UTC

[jira] [Commented] (BEAM-2534) KafkaIO should allow gaps in message offsets

    [ https://issues.apache.org/jira/browse/BEAM-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16072571#comment-16072571 ] 

Jean-Baptiste Onofré commented on BEAM-2534:
--------------------------------------------

Any update on this Jira for 2.1.0 release ?

> KafkaIO should allow gaps in message offsets
> --------------------------------------------
>
>                 Key: BEAM-2534
>                 URL: https://issues.apache.org/jira/browse/BEAM-2534
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-java-extensions
>    Affects Versions: 2.0.0
>            Reporter: Raghu Angadi
>            Assignee: Raghu Angadi
>            Priority: Minor
>             Fix For: 2.1.0
>
>
> KafkaIO reader logs a warning when it notices gaps in offsets for messages. While such gaps are not expected for normal Kafka topics, there could be gaps when log compaction is enabled (which deletes older messages for a key). 
> This warning log is not very useful. Also we should take such gaps while estimating backlog.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)