Would driver shutdown cause app dead?

[ZhuGe <tc...@outlook.com>]

Hi all:I am a bit confuse about the work of driver.In our productin enviroment, we have a spark streaming app running in standone mode. what we concern is that if  the driver shutdown accidently(host shutdown or whatever). would the app  running normally?
Any explanation would be appreciated!!
Cheers 		 	   		  

RE: Would driver shutdown cause app dead?

["Young, Matthew T" <ma...@intel.com>]

ZhuGe,

If you run your program in the "cluster" deploy-mode you get resiliency against driver failure, though there are some steps you have to take in how you write your streaming job to allow for transparent resume. Netflix did a nice writeup of this resiliency here<http://techblog.netflix.com/2015/03/can-spark-streaming-survive-chaos-monkey.html>.

If you tie in ZooKeeper you can also get resiliency against Master failure, which has some documentation here<http://spark.apache.org/docs/1.4.0/spark-standalone.html>.

Regards,

-- Matthew

________________________________
From: ZhuGe [tczg@outlook.com]
Sent: Tuesday, July 21, 2015 3:07 AM
To: user@spark.apache.org
Subject: Would driver shutdown cause app dead?

Hi all:
I am a bit confuse about the work of driver.
In our productin enviroment, we have a spark streaming app running in standone mode. what we concern is that if  the driver shutdown accidently(host shutdown or whatever). would the app  running normally?

Any explanation would be appreciated!!

Cheers

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org