You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@samza.apache.org by "Shanthoosh Venkataraman (JIRA)" <ji...@apache.org> on 2017/03/15 21:54:41 UTC

[jira] [Created] (SAMZA-1146) TaskCallbackManager commit fix.

Shanthoosh Venkataraman created SAMZA-1146:
----------------------------------------------

             Summary: TaskCallbackManager commit fix.
                 Key: SAMZA-1146
                 URL: https://issues.apache.org/jira/browse/SAMZA-1146
             Project: Samza
          Issue Type: Bug
            Reporter: Shanthoosh Venkataraman
            Assignee: Shanthoosh Venkataraman


Each task callback in samza belongs to different SystemStreamPartition. When multiple callbacks in contagious order are available for commit, callback with highest sequence number is chosen for commit. This will prevent checkpointing of completed callbacks that has commit request and doesn't have highest sequence number. Upon task restart this will lead to duplicate reprocessing of already processed messages (since completed callbacks for some SystemStreamPartition's aren't committed earlier). 

Expected behavior is to commit all completed callbacks that has commit request associated with it.





--
This message was sent by Atlassian JIRA
(v6.3.15#6346)