You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2023/03/30 04:50:56 UTC
[spark] branch branch-3.4 updated: [SPARK-42971][CORE] Change to print `workdir` if `appDirs` is null when worker handle `WorkDirCleanup` event
This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-3.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.4 by this push:
new 6e4fcf71851 [SPARK-42971][CORE] Change to print `workdir` if `appDirs` is null when worker handle `WorkDirCleanup` event
6e4fcf71851 is described below
commit 6e4fcf71851be64ac75fc3c50cb178e01d71368f
Author: yangjie01 <ya...@baidu.com>
AuthorDate: Thu Mar 30 13:50:27 2023 +0900
[SPARK-42971][CORE] Change to print `workdir` if `appDirs` is null when worker handle `WorkDirCleanup` event
### What changes were proposed in this pull request?
This pr change to print `workdir` if `appDirs` is null when worker handle `WorkDirCleanup` event.
### Why are the changes needed?
Print `appDirs` ause a NPE because `appDirs` is null and from the context, what should be printed is `workdir`
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Pass GitHub Actions
Closes #40597 from LuciferYang/SPARK-39296-FOLLOW.
Authored-by: yangjie01 <ya...@baidu.com>
Signed-off-by: Hyukjin Kwon <gu...@apache.org>
(cherry picked from commit f105fe82ab01fe787d00c6ad72f1d6dedb5f3a1b)
Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala b/core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala
index 04ef9cc0126..9fb66faef39 100755
--- a/core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala
+++ b/core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala
@@ -516,8 +516,7 @@ private[deploy] class Worker(
val cleanupFuture: concurrent.Future[Unit] = concurrent.Future {
val appDirs = workDir.listFiles()
if (appDirs == null) {
- throw new IOException(
- s"ERROR: Failed to list files in ${appDirs.mkString("dirs(", ", ", ")")}")
+ throw new IOException(s"ERROR: Failed to list files in $workDir")
}
appDirs.filter { dir =>
// the directory is used by an application - check that the application is not running
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org