[jira] [Created] (SAMZA-2088) Make TaskInstance work with Startpoints

["Shanthoosh Venkataraman (JIRA)" <ji...@apache.org>]

Shanthoosh Venkataraman created SAMZA-2088:
----------------------------------------------

             Summary:  Make TaskInstance work with Startpoints
                 Key: SAMZA-2088
                 URL: https://issues.apache.org/jira/browse/SAMZA-2088
             Project: Samza
          Issue Type: New Feature
            Reporter: Shanthoosh Venkataraman
            Assignee: Shanthoosh Venkataraman


TaskInstance receives input messages from the AsyncRunLoop(i.e, event loop to coordinate message dispatch to the tasks within a SamzaContainer) and routes them to the user-defined task implementation. 

Starting offset for a input SystemStreamPartition is calculated in samza as the one offset ahead of the previous checkpointed offset of the SystemStreamPartition.

Currently, TaskInstance is instantiated with the starting offsets during the runLoop initialization. TaskInstance does not route the message for a SSP to the user-defined task implementation when the offset of received message is lower than the starting offset of SSP.

SAMZA-1985 introducted the startpoint abstraction, where during RunLoop instantiation, the idea is to move the input consumer to an offset specified by user-defined startpoint.

However, moving the consumer to an offset back alone does not suffice. For instance, if the checkpointed offset is at 100 for a SSP and the startpoint intends to move the consumer back to a offset 0. Even if we successfully moved the consumer instance to offset 0, the messages from offset 0 to 99 will not be routed by TaskInstance to the user defined task implementation. 





--
This message was sent by Atlassian JIRA
(v7.6.3#76005)