[jira] [Resolved] (SAMZA-2741) Pipeline Drain: Support the ability to drain pipelines to allow incompatible intermediate schema changes

["Ajo Thomas (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SAMZA-2741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ajo Thomas resolved SAMZA-2741.
-------------------------------
    Resolution: Fixed

> Pipeline Drain: Support the ability to drain pipelines to allow incompatible intermediate schema changes
> --------------------------------------------------------------------------------------------------------
>
>                 Key: SAMZA-2741
>                 URL: https://issues.apache.org/jira/browse/SAMZA-2741
>             Project: Samza
>          Issue Type: New Feature
>            Reporter: Ajo Thomas
>            Assignee: Ajo Thomas
>            Priority: Major
>          Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> Samza users running long running pipelines need a way to gracefully drain them in preparation for an upgrade. The upgrade could be due to changes to transforms or backward incompatible changes in data schema. The only option currently available to the user is to abruptly stop the pipeline and reset intermediate data which could lead to loss of buffered data. User logic is expected to constantly evolve and schema changes are often backward incompatible. It is therefore imperative to provide the ability to seamlessly stop and upgrade their pipelines without losing any buffered data.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)