You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "Shanthoosh Venkataraman (JIRA)" <ji...@apache.org> on 2018/05/22 22:37:00 UTC
[jira] [Resolved] (SAMZA-1621) Delete the ephemeral processor node
in StreamProcessor shutdown phase.
[ https://issues.apache.org/jira/browse/SAMZA-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shanthoosh Venkataraman resolved SAMZA-1621.
--------------------------------------------
Resolution: Fixed
> Delete the ephemeral processor node in StreamProcessor shutdown phase.
> ----------------------------------------------------------------------
>
> Key: SAMZA-1621
> URL: https://issues.apache.org/jira/browse/SAMZA-1621
> Project: Samza
> Issue Type: Bug
> Reporter: Shanthoosh Venkataraman
> Assignee: Shanthoosh Venkataraman
> Priority: Major
>
> Currently when a StreamProcessor dies, we rely upon the session expiration phase in zookeeper to delete the ephemeral processor node in zookeeper. This behavior might not be acceptable in the following scenarios.
> A. In standalone, we disallow multiple processors in a group to have the same processorId. If the zookeeper session timeout is greater than the processor restart time, then upon restart, the processor will be unable to join the group with the same processorId(since there will be a ephemeral node in zookeeper with processorId corresponding to the restarted processor).
> B. In zookeeper the writes are serialized by a single writer thread performing actions out of a request queue. In cases, where the request queue is flooded by zookeeper write/delete requests from other processes, action for session expiration will be delayed more than the configured zookeeper session timeout.
> Moreover, the general notion behind the zookeeper session timeout is to safeguard the StreamProcessors from network partitioning(as well as transient failures in the zookeeper server ensemble). Leaving the zookeeper ephemeral processor nodes to remain after the StreamProcessor shutdown doesn't add much value.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)