You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@samza.apache.org by "Cameron Lee (JIRA)" <ji...@apache.org> on 2019/03/05 20:20:00 UTC

[jira] [Created] (SAMZA-2120) Enable custom handling of ConsumerRecords consumed by Kafka

Cameron Lee created SAMZA-2120:
----------------------------------

             Summary: Enable custom handling of ConsumerRecords consumed by Kafka
                 Key: SAMZA-2120
                 URL: https://issues.apache.org/jira/browse/SAMZA-2120
             Project: Samza
          Issue Type: Improvement
    Affects Versions: 1.0
            Reporter: Cameron Lee
            Assignee: Cameron Lee


Add a way to be able to customize the handling of ConsumerRecords when converting them to IncomingMessageEnvelope when extending KakfaSystemConsumer functionality.

An example of this extension would be to specify a checkpoint offset in IncomingMessageEnvelope which is different than the plain offset that is returned by Kafka. If large messages in Kafka are implemented by segmenting into multiple individual messages, then the plain offset might be in the middle of a large message, and we would not want to checkpoint in the middle of a large message.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)