You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "zuotingbing (JIRA)" <ji...@apache.org> on 2016/12/22 11:27:58 UTC
[jira] [Commented] (SPARK-18979) ShutdownHookManager：Exception while deleting Spark temp dir
[ https://issues.apache.org/jira/browse/SPARK-18979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15769827#comment-15769827 ]
zuotingbing commented on SPARK-18979:
-------------------------------------
my SPARK_LOCAL_DIRS value is setted like this:
SPARK_LOCAL_DIRS=/data2/zdh/spark/tmp,/data3/zdh/spark/tmp,/data4/zdh/spark/tmp,/data5/zdh/spark/tmp,/data6/zdh/spark/tmp,/data7/zdh/spark/tmp
From the worker log we find only 3 of 6 dirs delete failed ：
【
Search "Deleting directory" (6 hits in 1 files)
C:\Users\10159544\Desktop\logs\logs\spark-mr-worker-IDC-ZTECache-Logserver20-B12-U13.log (6 hits)
Line 909563: 2016-12-15 20:09:52,629 INFO ShutdownHookManager: Deleting directory /data2/zdh/spark/tmp/spark-67cf188b-5978-42d9-8ce6-e181f0ba4d0d
Line 910117: 2016-12-15 20:12:59,930 INFO ShutdownHookManager: Deleting directory /data6/zdh/spark/tmp/spark-688964cc-010d-44cd-b398-5f54fcee0b34
Line 910118: 2016-12-15 20:15:21,932 INFO ShutdownHookManager: Deleting directory /data3/zdh/spark/tmp/spark-f15e06d0-fedf-4618-8ff5-8c7b10a55cbe
Line 913560: 2016-12-15 20:17:49,990 INFO ShutdownHookManager: Deleting directory /data5/zdh/spark/tmp/spark-16647cd9-57b0-4893-a876-5a39ebc4078f
Line 915393: 2016-12-15 20:20:11,767 INFO ShutdownHookManager: Deleting directory /data7/zdh/spark/tmp/spark-18e24536-8739-4d1a-8381-c762b540addf
Line 915394: 2016-12-15 20:20:11,767 INFO ShutdownHookManager: Deleting directory /data4/zdh/spark/tmp/spark-e7f52941-3e2c-48d7-993c-1df891e24577
Search "Exception while deleting" (3 hits in 1 files)
C:\Users\10159544\Desktop\logs\logs\spark-mr-worker-IDC-ZTECache-Logserver20-B12-U13.log (3 hits)
Line 910098: 2016-12-15 20:12:59,930 ERROR ShutdownHookManager: Exception while deleting Spark temp dir: /data2/zdh/spark/tmp/spark-67cf188b-5978-42d9-8ce6-e181f0ba4d0d
Line 913541: 2016-12-15 20:17:49,990 ERROR ShutdownHookManager: Exception while deleting Spark temp dir: /data3/zdh/spark/tmp/spark-f15e06d0-fedf-4618-8ff5-8c7b10a55cbe
Line 1044408: 2016-12-15 20:22:40,501 ERROR ShutdownHookManager: Exception while deleting Spark temp dir: /data4/zdh/spark/tmp/spark-e7f52941-3e2c-48d7-993c-1df891e24577
】
> ShutdownHookManager：Exception while deleting Spark temp dir
> -------------------------------------------------------------
>
> Key: SPARK-18979
> URL: https://issues.apache.org/jira/browse/SPARK-18979
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.5.2
> Reporter: zuotingbing
>
> when i stop the worker process, the SPARK_LOCAL_DIRS should be delete recursively but failed. Exception info:
> 2016-12-15 20:12:59,930 ERROR ShutdownHookManager: Exception while deleting Spark temp dir: /data2/zdh/spark/tmp/spark-67cf188b-5978-42d9-8ce6-e181f0ba4d0d
> java.io.IOException: Failed to delete: /data2/zdh/spark/tmp/spark-67cf188b-5978-42d9-8ce6-e181f0ba4d0d
> at org.apache.spark.util.Utils$.deleteRecursively(Utils.scala:884)
> at org.apache.spark.util.ShutdownHookManager$$anonfun$1$$anonfun$apply$mcV$sp$3.apply(ShutdownHookManager.scala:63)
> at org.apache.spark.util.ShutdownHookManager$$anonfun$1$$anonfun$apply$mcV$sp$3.apply(ShutdownHookManager.scala:60)
> at scala.collection.mutable.HashSet.foreach(HashSet.scala:79)
> at org.apache.spark.util.ShutdownHookManager$$anonfun$1.apply$mcV$sp(ShutdownHookManager.scala:60)
> at org.apache.spark.util.SparkShutdownHook.run(ShutdownHookManager.scala:264)
> at org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ShutdownHookManager.scala:234)
> at org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1$$anonfun$apply$mcV$sp$1.apply(ShutdownHookManager.scala:234)
> at org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1$$anonfun$apply$mcV$sp$1.apply(ShutdownHookManager.scala:234)
> at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1699)
> at org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1.apply$mcV$sp(ShutdownHookManager.scala:234)
> at org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1.apply(ShutdownHookManager.scala:234)
> at org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1.apply(ShutdownHookManager.scala:234)
> at scala.util.Try$.apply(Try.scala:161)
> at org.apache.spark.util.SparkShutdownHookManager.runAll(ShutdownHookManager.scala:234)
> at org.apache.spark.util.SparkShutdownHookManager$$anon$2.run(ShutdownHookManager.scala:216)
> at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org