You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "Bharath Kumarasubramanian (Jira)" <ji...@apache.org> on 2021/07/12 21:04:00 UTC
[jira] [Assigned] (SAMZA-2663) Update active job model to proposed
job model on job model expiration
[ https://issues.apache.org/jira/browse/SAMZA-2663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bharath Kumarasubramanian reassigned SAMZA-2663:
------------------------------------------------
Assignee: Bharath Kumarasubramanian
> Update active job model to proposed job model on job model expiration
> ---------------------------------------------------------------------
>
> Key: SAMZA-2663
> URL: https://issues.apache.org/jira/browse/SAMZA-2663
> Project: Samza
> Issue Type: Bug
> Reporter: Bharath Kumarasubramanian
> Assignee: Bharath Kumarasubramanian
> Priority: Major
>
> *Problem*:
> As part of SAMZA-2638, we introduced skipping container restart and stops on no changes to work assignment for processors across rebalances. However, we only update the active job model with the proposed job model on starting the container as part of `onNewJobModel`. This leads to a scenario where the processor is stopped but the future rebalances assume the container is still running. More information on scenario below.
> *Scenario*:
> Imagine the quorum is in steady state with job model version v1. A new rebalance occurs and the leader generates v2. Processor P1 has changes in work assignment and as a result stops the container as part of job model expiration. However, in the event of the rebalance being unsuccessful (barrier times out), a new rebalance occurs which generates a job model version v3. In the scenario where work assignment for P1 in v3 is same as v1, then the state transition assumes the processor hasn't stopped the container and proceeds to do an no-op.
> *Changes*:
> Update the active job model regardless of whether we render the current job model obsolete or not for the current processor during checkAndExpireJobModel
--
This message was sent by Atlassian Jira
(v8.3.4#803005)