[jira] [Resolved] (KAFKA-1977) Make logEndOffset available in the Zookeeper consumer

["Manikumar (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Manikumar resolved KAFKA-1977.
------------------------------
    Resolution: Auto Closed

Closing this as the Scala consumers have been deprecated and no further work is planned. This requirement will be tracked in KAFKA-2500 for java consumer.

> Make logEndOffset available in the Zookeeper consumer
> -----------------------------------------------------
>
>                 Key: KAFKA-1977
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1977
>             Project: Kafka
>          Issue Type: Improvement
>          Components: core
>            Reporter: Will Funnell
>            Priority: Minor
>         Attachments: Make_logEndOffset_available_in_the_Zookeeper_consumer.patch
>
>
> The requirement is to create a snapshot from the Kafka topic but NOT do continual reads after that point. For example you might be creating a backup of the data to a file.
> In order to achieve that, a recommended solution by Joel Koshy and Jay Kreps was to expose the high watermark, as maxEndOffset, from the FetchResponse object through to each MessageAndMetadata object in order to be aware when the consumer has reached the end of each partition.
> The submitted patch achieves this by adding the maxEndOffset to the PartitionTopicInfo, which is updated when a new message arrives in the ConsumerFetcherThread and then exposed in MessageAndMetadata.
> See here for discussion:
> http://search-hadoop.com/m/4TaT4TpJy71



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)