[jira] [Commented] (SPARK-28442) Potentially persist data without leadership

["Hyukjin Kwon (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-28442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16888478#comment-16888478 ] 

Hyukjin Kwon commented on SPARK-28442:
--------------------------------------

[~Tison], please provide reproducer and/or console print out if you faced this problem. It's difficult to see what's an issue without reading the current JIRA.

> Potentially persist data without leadership
> -------------------------------------------
>
>                 Key: SPARK-28442
>                 URL: https://issues.apache.org/jira/browse/SPARK-28442
>             Project: Spark
>          Issue Type: Bug
>          Components: Deploy
>    Affects Versions: 2.4.3
>            Reporter: TisonKun
>            Priority: Major
>
> Spark Master could potentially persist data via {{ZooKeeperPersistenceEngine}} even if it is not the leader. See the execution order below.
> 1. master-1 became the leader.
> 2. master-1 received message and wanted to addApplication(or addWorker)
> 3. master-1 stuck because of a full gc
> 4. master-1 lost leadership on zk. master-2 became the leader.
> 5. master-1 received {{RevokedLeadership}} message but the message was pending.
> 6. master-1 finished persisting data.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org