You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/04/27 08:14:00 UTC
[jira] [Commented] (SAMZA-1689) Add validations before state
transitions in ZkBarrierForVersionUpgrade.
[ https://issues.apache.org/jira/browse/SAMZA-1689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16456101#comment-16456101 ]
ASF GitHub Bot commented on SAMZA-1689:
---------------------------------------
GitHub user shanthoosh opened a pull request:
https://github.com/apache/samza/pull/490
SAMZA-1689: Add validations before state transitions in ZkBarrierForVersionUpgrade.
Prevent invalid state updations on barrier.
* Introduced a additional barrier state NEW.
* Add state validations before updating the barrier.
* Fix existing TestZkBarrier tests that are disabled and add new tests
to verify the intended behavior.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/shanthoosh/samza fix_barrier_state_transitions
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/samza/pull/490.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #490
----
commit 856c29d14d579c36e7d41dc7d574d4ea2752fd8f
Author: Shanthoosh Venkataraman <sv...@...>
Date: 2018-04-27T01:41:45Z
SAMZA-1689: Add state validations in ZkBarrierForVersionUpgrade.
Prevent invalid state updations on barrier.
* Introduced a additional barrier state NEW.
* Add state validations before updating the barrier.
* Fix existing TestZkBarrier tests that are disabled and add new tests
to verify the intended behavior.
----
> Add validations before state transitions in ZkBarrierForVersionUpgrade.
> ------------------------------------------------------------------------
>
> Key: SAMZA-1689
> URL: https://issues.apache.org/jira/browse/SAMZA-1689
> Project: Samza
> Issue Type: Bug
> Reporter: Shanthoosh Venkataraman
> Assignee: Shanthoosh Venkataraman
> Priority: Major
>
> *Problem:*
> A barrier in zookeeper can be in either DONE or TimedOut state. Existing ZkBarrier implementation doesn't have any state validations before acting upon the barrier events received from zookeeper. This can corrupt barrier states when any of the following occur:
> - When a stream processor joins a barrier marked as timed out by the leader of the group.
> - When the leader of a group expires a barrier marked as Done. This can happen when a leader plays out a stale events from its in-memory buffer.
> This will cause the processors group to go through unnecessary rebalancing phase(worse, corrupt the barrier state).
> *Changes:*
> - Introduce a barrier state: NEW which will be the beginning state of a barrier(In existing implementation, barrier is created with empty/null value). Barrier will be marked as Done or TimedOut only if the current barrier state is NEW.
> - Fix the disabled barrier test cases. Add new test cases to validate the changed functionality.
> - Improve logging to aid debugging.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)