You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Piotr Kołaczkowski (JIRA)" <ji...@apache.org> on 2015/10/13 15:57:05 UTC
[jira] [Commented] (SPARK-6187) Report full executor exceptions to
the driver
[ https://issues.apache.org/jira/browse/SPARK-6187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14954954#comment-14954954 ]
Piotr Kołaczkowski commented on SPARK-6187:
-------------------------------------------
The problem is not that the exception is not reported at all, but that the full stacktrace of the exception is not reported. Spark seems to report just the top-level exception and ignores the nested exceptions stacktraces. The full stacktrace is reported only in the executor logs.
So your example is too simple to reproduce it, because you're throwing just one, flat exception with null cause.
> Report full executor exceptions to the driver
> ---------------------------------------------
>
> Key: SPARK-6187
> URL: https://issues.apache.org/jira/browse/SPARK-6187
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 1.2.1
> Reporter: Piotr Kołaczkowski
> Priority: Minor
>
> If the task fails for some reason, the driver seems to report only the top-level exception, without the cause(s). While it is possible to recover the full stacktrace from executor's logs, it is quite annoying and would be better to just report the full stacktrace, with all the causes to the driver application.
> Example stacktrace I just got reported by the application:
> {noformat}
> org.apache.spark.SparkException: Job aborted due to stage failure: Task 5 in stage 0.0 failed 1 times, most recent failure: Lost task 5.0 in stage 0.0 (TID 5, localhost): java.lang.NoClassDefFoundError: Could not initialize class org.apache.cassandra.db.Keyspace
> at com.datastax.bdp.spark.writer.BulkTableWriter.writeSSTables(BulkTableWriter.scala:194)
> at com.datastax.bdp.spark.writer.BulkTableWriter.write(BulkTableWriter.scala:223)
> at com.datastax.bdp.spark.writer.BulkTableWriter$BulkSaveRDDFunctions$$anonfun$bulkSaveToCassandra$1.apply(BulkTableWriter.scala:280)
> at com.datastax.bdp.spark.writer.BulkTableWriter$BulkSaveRDDFunctions$$anonfun$bulkSaveToCassandra$1.apply(BulkTableWriter.scala:280)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:61)
> at org.apache.spark.scheduler.Task.run(Task.scala:56)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:200)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> As you see, this is not very informative.
> In fact, the real exception is:
> {noformat}
> java.lang.NoClassDefFoundError: Could not initialize class org.apache.cassandra.db.Keyspace
> at com.datastax.bdp.spark.writer.BulkTableWriter.writeSSTables(BulkTableWriter.scala:194)
> at com.datastax.bdp.spark.writer.BulkTableWriter.write(BulkTableWriter.scala:227)
> at com.datastax.bdp.spark.writer.BulkTableWriter$BulkSaveRDDFunctions$$anonfun$bulkSaveToCassandra$1.apply(BulkTableWriter.scala:284)
> at com.datastax.bdp.spark.writer.BulkTableWriter$BulkSaveRDDFunctions$$anonfun$bulkSaveToCassandra$1.apply(BulkTableWriter.scala:284)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:61)
> at org.apache.spark.scheduler.Task.run(Task.scala:56)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:200)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> java.lang.ExceptionInInitializerError
> at com.datastax.bdp.spark.writer.BulkTableWriter.writeSSTables(BulkTableWriter.scala:194)
> at com.datastax.bdp.spark.writer.BulkTableWriter.write(BulkTableWriter.scala:227)
> at com.datastax.bdp.spark.writer.BulkTableWriter$BulkSaveRDDFunctions$$anonfun$bulkSaveToCassandra$1.apply(BulkTableWriter.scala:284)
> at com.datastax.bdp.spark.writer.BulkTableWriter$BulkSaveRDDFunctions$$anonfun$bulkSaveToCassandra$1.apply(BulkTableWriter.scala:284)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:61)
> at org.apache.spark.scheduler.Task.run(Task.scala:56)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:200)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.cassandra.config.DatabaseDescriptor.createAllDirectories(DatabaseDescriptor.java:741)
> at org.apache.cassandra.db.Keyspace.<clinit>(Keyspace.java:72)
> ... 10 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org