[jira] [Updated] (SAMZA-2590) Async State Backup

["Daniel Chen (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SAMZA-2590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Daniel Chen updated SAMZA-2590:
-------------------------------
    Fix Version/s: 1.7

> Async State Backup
> ------------------
>
>                 Key: SAMZA-2590
>                 URL: https://issues.apache.org/jira/browse/SAMZA-2590
>             Project: Samza
>          Issue Type: New Feature
>            Reporter: Daniel Chen
>            Assignee: Daniel Chen
>            Priority: Major
>             Fix For: 1.7
>
>         Attachments: Samza State Backup and Async Checkpointing (1).pdf
>
>
> # We must create a way of committing to the state store that allows for the remote stores to be uploaded independently from processing.
>  # The commit phase must be transactional for Kafka changelog commits (preserve existing behaviour) and remote store commits.
>  # For remote store commits, we require exclusive, non-overlapping commits so that the commits that appear in the remote store are in order.
>  # We need to timeout on the asynchronous commit such that we will stop further processing until the state in the remote store is caught up to the current state of the job.
>  # This feature will be required to be enabled by default of all Samza jobs regardless of the remote state store backup implementation (Kafka, Azure blob store, HDFS) and the processing parallelism (async or sync processing) to improve performance for all present and future remote state commits.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)