Re: Catching executor exception from executor in driver

[Imran Rashid <ir...@cloudera.com>]

(+dev)

Hi Justin,

short answer: no, there is no way to do that.

I'm just guessing here, but I imagine this was done to eliminate
serialization problems (eg., what if we got an error trying to serialize
the user exception to send from the executors back to the driver?).
Though, actually that isn't a great explanation either, since even when the
info gets back to the driver, its broken into a few string fields (eg., we
have the class name of the root exception), but eventually it just gets
converted to one big string.

I've cc'ed dev b/c I think this is an oversight in Spark.  It makes it
really hard to write an app to deal gracefully with various exceptions --
all you can do is look at the string in SparkException (which could change
arbitrarily between versions, in addition to just being a pain to work
with).  We should probably add much more fine-grained subclasses of
SparkException, at the very least distinguishing errors in user code vs.
errors in spark.  I could imagine there might be a few other cases we'd
like to distinguish more carefully as well.

Any thoughts from other devs?

thanks
Imran





On Tue, Apr 14, 2015 at 4:46 PM, Justin Yip <yi...@prediction.io> wrote:

> Hello,
>
> I would like to know if there is a way of catching exception throw from
> executor exception from the driver program. Here is an example:
>
> try {
>   val x = sc.parallelize(Seq(1,2,3)).map(e => e / 0).collect
> } catch {
>   case e: SparkException => {
>     println(s"ERROR: $e")
>     println(s"CAUSE: ${e.getCause}")
>   }
> }
>
> Output:
> ERROR: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 1 in stage 1.0 failed 4 times, most recent failure: Lost task 1.3 in
> stage 1.0 (TID 15, pio1.c.ace-lotus-714.internal):
> java.lang.ArithmeticException: / by zero
> at
> $line71.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$anonfun$1.apply$mcII$sp(<console>:51)
> ...
> CAUSE: null
>
> The exception cause is a null value. Is there any way that I can catch the
> ArithmeticException?
>
> Thanks
>
> Justin
>
> ------------------------------
> View this message in context: Catching executor exception from executor
> in driver
> <http://apache-spark-user-list.1001560.n3.nabble.com/Catching-executor-exception-from-executor-in-driver-tp22495.html>
> Sent from the Apache Spark User List mailing list archive
> <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com.
>