[jira] [Resolved] (SAMZA-2121) Add checkpoint offset field to IncomingMessageEnvelope

["Cameron Lee (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SAMZA-2121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Cameron Lee resolved SAMZA-2121.
--------------------------------
    Resolution: Won't Fix

Instead of adding another field, we can leverage the existing offset field. It was unnecessary to expand the API for this Kafka large message use case.

By leveraging the existing offset field, we do need to clarify the semantics for the offset field in IncomingMessageEnvelope, in that it is provided by the system that consumed the message ([https://github.com/apache/samza/pull/945). |https://github.com/apache/samza/pull/945)]Samza does not guarantee any characteristics (such as uniqueness) about this field.

> Add checkpoint offset field to IncomingMessageEnvelope
> ------------------------------------------------------
>
>                 Key: SAMZA-2121
>                 URL: https://issues.apache.org/jira/browse/SAMZA-2121
>             Project: Samza
>          Issue Type: New Feature
>            Reporter: Cameron Lee
>            Assignee: Cameron Lee
>            Priority: Major
>          Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> In some cases, the offset to checkpoint may not match the plain offset associated with an individual message.
> If large messages are supported in Kafka by segmenting them into smaller individual messages, then the plain offset of a message might fall in the middle of the segments of a large message. We do not want to checkpoint in the middle of a large message.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)