[jira] [Created] (SAMZA-2590) Async State Backup

["Daniel Chen (Jira)" <ji...@apache.org>]

Daniel Chen  created SAMZA-2590:
-----------------------------------

             Summary: Async State Backup
                 Key: SAMZA-2590
                 URL: https://issues.apache.org/jira/browse/SAMZA-2590
             Project: Samza
          Issue Type: New Feature
            Reporter: Daniel Chen 
            Assignee: Daniel Chen 


# We must create a way of committing to the state store that allows for the remote stores to be uploaded independently from processing.
 # The commit phase must be transactional for Kafka changelog commits (preserve existing behaviour) and remote store commits.
 # For remote store commits, we require exclusive, non-overlapping commits so that the commits that appear in the remote store are in order.
 # We need to timeout on the asynchronous commit such that we will stop further processing until the state in the remote store is caught up to the current state of the job.
 # This feature will be required to be enabled by default of all Samza jobs regardless of the remote state store backup implementation (Kafka, Azure blob store, HDFS) and the processing parallelism (async or sync processing) to improve performance for all present and future remote state commits.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)