[jira] [Created] (SAMZA-2305) Stream processor should ensure previous container is stopped during a rebalance

["Bharath Kumarasubramanian (Jira)" <ji...@apache.org>]

Bharath Kumarasubramanian created SAMZA-2305:
------------------------------------------------

             Summary: Stream processor should ensure previous container is stopped during a rebalance
                 Key: SAMZA-2305
                 URL: https://issues.apache.org/jira/browse/SAMZA-2305
             Project: Samza
          Issue Type: Bug
            Reporter: Bharath Kumarasubramanian
            Assignee: Bharath Kumarasubramanian


In standalone, when we have a rebalance before the container has started successfully with the previous job model, OnJobModelExpired does a no-op. It happens since the stream processor doesn’t transition to RUNNING until the container callback gets invoked. OnJobModelExpired, assumes the processor is still in the REBALANCE phase and does a no-op.

 

Often times, containers take longer time to start. Especially stateful containers that have large state to restore. In those scenarios, if a rebalance happens before the container has started successfully, processors in the quorum will run with inconsistent job model.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)