How Spark HA works

[Akmal Abbasov <ak...@icloud.com>]

Hi, 
I'm trying to understand how Spark HA works. I'm using Spark 1.6.1 and Zookeeper 3.4.6.
I've add the following line to $SPARK_HOME/conf/spark-env.sh
export SPARK_DAEMON_JAVA_OPTS="-Dspark.deploy.recoveryMode=ZOOKEEPER -Dspark.deploy.zookeeper.url=zk1:2181,zk2:2181,zk3:2181 -Dspark.deploy.zookeeper.dir=/spark
It's working so far.
I'd like to setup a link which will always go to active master UI(I'm using Spark in Standalone).  
I've checked the znode /spark, and it contains 
[leader_election, master_status]
I'm assuming that master_status znode will contain ip address of the current active master, is it true? Because in my case this znode isn't updated after failover.
And how /spark/leader_election works, because it doesn't contain any data.
Thank you.

Regards,
Akmal

Re: How Spark HA works

[Mohit Jaggi <mo...@gmail.com>]

what did you mean by “link” ? an HTTP URL to the spark monitoring UI? AFAIK, it is not directly supported. i typically go to both masters and check which one is active :-)

did you check if the failover actually happened in other ways (i don’t know what the znode should say)? you can try sending a spark job and if you used the right master config in your code, it should go to the new master. that will confirm that failover worked.



Mohit Jaggi
Founder,
Data Orchard LLC
www.dataorchardllc.com




> On Aug 19, 2016, at 8:56 PM, Charles Nnamdi Akalugwu <cp...@gmail.com> wrote:
> 
> I am experiencing this exact issue. Does anyone know what's going on with the zookeeper setup?
> 
> On Jul 5, 2016 10:34 AM, "Akmal Abbasov" <akmal.abbasov@icloud.com <ma...@icloud.com>> wrote:
> >
> > Hi, 
> > I'm trying to understand how Spark HA works. I'm using Spark 1.6.1 and Zookeeper 3.4.6.
> > I've add the following line to $SPARK_HOME/conf/spark-env.sh
> > export SPARK_DAEMON_JAVA_OPTS="-Dspark.deploy.recoveryMode=ZOOKEEPER -Dspark.deploy.zookeeper.url=zk1:2181,zk2:2181,zk3:2181 -Dspark.deploy.zookeeper.dir=/spark
> > It's working so far.
> > I'd like to setup a link which will always go to active master UI(I'm using Spark in Standalone).  
> > I've checked the znode /spark, and it contains 
> > [leader_election, master_status]
> > I'm assuming that master_status znode will contain ip address of the current active master, is it true? Because in my case this znode isn't updated after failover.
> > And how /spark/leader_election works, because it doesn't contain any data.
> > Thank you.
> >
> > Regards,
> > Akmal
> >
> >
> 

Re: How Spark HA works

[Charles Nnamdi Akalugwu <cp...@gmail.com>]

I am experiencing this exact issue. Does anyone know what's going on with
the zookeeper setup?

On Jul 5, 2016 10:34 AM, "Akmal Abbasov" <ak...@icloud.com> wrote:
>
> Hi,
> I'm trying to understand how Spark HA works. I'm using Spark 1.6.1 and
Zookeeper 3.4.6.
> I've add the following line to $SPARK_HOME/conf/spark-env.sh
> export SPARK_DAEMON_JAVA_OPTS="-Dspark.deploy.recoveryMode=ZOOKEEPER
-Dspark.deploy.zookeeper.url=zk1:2181,zk2:2181,zk3:2181
-Dspark.deploy.zookeeper.dir=/spark
> It's working so far.
> I'd like to setup a link which will always go to active master UI(I'm
using Spark in Standalone).
> I've checked the znode /spark, and it contains
> [leader_election, master_status]
> I'm assuming that master_status znode will contain ip address of the
current active master, is it true? Because in my case this znode isn't
updated after failover.
> And how /spark/leader_election works, because it doesn't contain any data.
> Thank you.
>
> Regards,
> Akmal
>
>