You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "partha bishnu (JIRA)" <ji...@apache.org> on 2015/08/03 16:02:05 UTC
[jira] [Commented] (SPARK-9559) Worker redundancy/failover in spark
stand-alone mode
[ https://issues.apache.org/jira/browse/SPARK-9559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14651882#comment-14651882 ]
partha bishnu commented on SPARK-9559:
--------------------------------------
Hi
I am running some tests on spark in stand-alone mode with 3 nodes cluster. spark master is running on n-1, and slaves are on n-2 and n-3. Each machine is with 8G RAM and 4 core cpu. I am trying to test worker redundancy.
I wanted to set up the cluster such a way so that there will be two worker JVM, one on each slave (n-2 and n-3) after I start up the cluster.
Then one of the slave's worker JVM will launch the executor jvm to process the tasks when I submit the job with the following flags:
---total-executor-cores 1 and --executor-memory 1G
(1) Job submitted successfully in client mode. n-2 had worker jvm launched a executor jvm. So now n-2 had one worker jvm and one executor jvm running and n-3 just had the worker jvm running as before.
(2) I killed the worker jvm and the executor jvm on n-2
(3) I expected spark master on n-1 will then ask the worker jvm on n-3 to launch a new executor to start processing jobs but that did not happen. driver just got hung on the screen. n-2 disappeared from spark cluster as expected. n-3 just had the worker jvm running as before and no new executor was launched as expected after n-2 disappeared.
> Worker redundancy/failover in spark stand-alone mode
> ----------------------------------------------------
>
> Key: SPARK-9559
> URL: https://issues.apache.org/jira/browse/SPARK-9559
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.3.0
> Reporter: partha bishnu
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org