You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@bigtop.apache.org by "Konstantinos Tsakalozos (JIRA)" <ji...@apache.org> on 2016/07/01 10:18:11 UTC

[jira] [Created] (BIGTOP-2490) Spark in HA when Zookeeper is available

Konstantinos Tsakalozos created BIGTOP-2490:
-----------------------------------------------

             Summary: Spark in HA when Zookeeper is available 
                 Key: BIGTOP-2490
                 URL: https://issues.apache.org/jira/browse/BIGTOP-2490
             Project: Bigtop
          Issue Type: Improvement
          Components: spark
    Affects Versions: 1.1.0
            Reporter: Konstantinos Tsakalozos
            Assignee: Konstantinos Tsakalozos
            Priority: Minor


Spark could work in HA mode as long as a Zookeeper deployment is available. In this improvement we add an option to the Spark configuration through which  we can set the Zookeeper ensemble string. This string is pushed down to Spark configs causing the Spark to function in HA mode.

In HA mode Zookeeper is used to select the Spark leader and the workers need to ping all the potential leaders to pinpoint the current one. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)