You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "TisonKun (JIRA)" <ji...@apache.org> on 2019/07/19 01:24:00 UTC

[jira] [Created] (SPARK-28442) Potentially persist data without leadership

TisonKun created SPARK-28442:
--------------------------------

             Summary: Potentially persist data without leadership
                 Key: SPARK-28442
                 URL: https://issues.apache.org/jira/browse/SPARK-28442
             Project: Spark
          Issue Type: Bug
          Components: Documentation
    Affects Versions: 2.4.3
            Reporter: TisonKun


Spark Master could potentially persist data via {{ZooKeeperPersistenceEngine}} even if it is not the leader. See the execution order below.

1. master-1 became the leader.
2. master-1 received message and wanted to addApplication(or addWorker)
3. master-1 stuck because of a full gc
4. master-1 lost leadership on zk, received {{RevokedLeadership}} message but it was pending.
5. master-1 finished persisting data.




--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org