[jira] [Assigned] (SAMZA-1482) Restart or fail Samza jobs in YARN when input topic partition changes

["Yi Pan (Data Infrastructure) (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SAMZA-1482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yi Pan (Data Infrastructure) reassigned SAMZA-1482:
---------------------------------------------------

    Assignee: Yi Pan (Data Infrastructure)

> Restart or fail Samza jobs in YARN when input topic partition changes
> ---------------------------------------------------------------------
>
>                 Key: SAMZA-1482
>                 URL: https://issues.apache.org/jira/browse/SAMZA-1482
>             Project: Samza
>          Issue Type: Bug
>            Reporter: Yi Pan (Data Infrastructure)
>            Assignee: Yi Pan (Data Infrastructure)
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Currently, after a Samza job is started, it works only on a set of fixed input topic partitions at the start-up time. When input topic partitions are expanded, we often lose the messages sent in the new partitions, until we restart the job.
> SAMZA-882 added a input stream partition count monitor inside the JobCoordinator. This ticket is targeted to use this monitor metrics and trigger the following actions in YARN:
> # for stateless jobs, shutdown the JobCoordinator w/ UNDEFINED status code s.t. YARN will restart the whole job
> # for stateful jobs, shutdown the JobCoordinator w/ FAILED status code s.t. YARN will stop the whole job



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)