You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@samza.apache.org by "Chris Riccomini (JIRA)" <ji...@apache.org> on 2013/08/20 00:35:47 UTC

[jira] [Created] (SAMZA-23) TaskInstance commits for all TaskCoordinator.commit calls

Chris Riccomini created SAMZA-23:
------------------------------------

             Summary: TaskInstance commits for all TaskCoordinator.commit calls
                 Key: SAMZA-23
                 URL: https://issues.apache.org/jira/browse/SAMZA-23
             Project: Samza
          Issue Type: Bug
            Reporter: Chris Riccomini


If a StreamTask calls TaskCoordinator.commit, all TaskInstances will commit their SystemProducers, TaskStorageManager, and CheckpointManager. The problem with this is that if you have 400 partitions in a SamzaContainer (for example), and each calls TaskCoordinator.commit once per second, you actually get 400 commits per TaskInstance per second. This is incorrect behavior. We should make TaskInstance commit itself only when its task.commit.ms window has expired, or when TaskCoordinator.commit was called ONLY by its StreamTask partition.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira