You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Josh Rosen (JIRA)" <ji...@apache.org> on 2015/06/23 18:19:01 UTC
[jira] [Reopened] (SPARK-8498) Fix NullPointerException in
error-handling path in UnsafeShuffleWriter
[ https://issues.apache.org/jira/browse/SPARK-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Josh Rosen reopened SPARK-8498:
-------------------------------
Re-opening because I had to revert the branch-1.4 patch due to a Java 6 compilation issue.
> Fix NullPointerException in error-handling path in UnsafeShuffleWriter
> ----------------------------------------------------------------------
>
> Key: SPARK-8498
> URL: https://issues.apache.org/jira/browse/SPARK-8498
> Project: Spark
> Issue Type: Bug
> Components: Shuffle
> Affects Versions: 1.4.0
> Reporter: Josh Rosen
> Assignee: holdenk
> Fix For: 1.5.0
>
>
> This bug was reported by [~prudenko] on the dev list. When the {{tungsten-sort}} shuffle manager was enabled, an executor died with the following exception:
> {code}
> 15/06/19 17:53:35 WARN TaskSetManager: Lost task 38.0 in stage 41.0 (TID 3176, ip-10-50-225-214.ec2.internal): java.lang.NullPointerException
> at org.apache.spark.shuffle.unsafe.UnsafeShuffleWriter.write(UnsafeShuffleWriter.java:151)
> at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:70)
> at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
> at org.apache.spark.scheduler.Task.run(Task.scala:70)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> I think that this is actually due to an error-handling issue. In the stack trace, the NPE is being thrown from an error-handling branch of a `finally` block:
> {code}
> public void write(scala.collection.Iterator<Product2<K, V>> records) throws IOException {
> boolean success = false;
> try {
> while (records.hasNext()) {
> insertRecordIntoSorter(records.next());
> }
> closeAndWriteOutput();
> success = true;
> } finally {
> if (!success) {
> sorter.cleanupAfterError(); // <---- this is the line throwing the error
> }
> }
> }
> {code}
> I suspect that what's happening is that an exception is being thrown from user / upstream code in the initial call to records.next(), but the error-handling block is failing because sorter == null since we haven't initialized it yet.
> We should fix this bug with a {{sorter != null}} check and should also add a regression test to ShuffleSuite to ensure that exceptions thrown by user code at this step of the shuffle write path don't get masked by error-handling bugs inside of the shuffle code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org