You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@uniffle.apache.org by js...@apache.org on 2022/07/01 06:57:46 UTC
[incubator-uniffle] 02/17: [Bugfix] Fix spark2 executor stop NPE problem (#187)
This is an automated email from the ASF dual-hosted git repository.
jshao pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-uniffle.git
commit 7fa8b52e5739a0c2ded7f2eca84b086713765418
Author: roryqi <je...@gmail.com>
AuthorDate: Wed Jun 22 14:30:15 2022 +0800
[Bugfix] Fix spark2 executor stop NPE problem (#187)
backport 0.5.0
### What changes were proposed in this pull request?
We need to judge heartbeatExecutorService whether is null when we will stop it.
### Why are the changes needed?
#177 pr introduce this problem, when we run Spark applications on our cluster, the executor will throw NPE when method `stop` is called.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Manual test
---
.../src/main/java/org/apache/spark/shuffle/RssShuffleManager.java | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/client-spark/spark2/src/main/java/org/apache/spark/shuffle/RssShuffleManager.java b/client-spark/spark2/src/main/java/org/apache/spark/shuffle/RssShuffleManager.java
index 5d11c39..8a2c385 100644
--- a/client-spark/spark2/src/main/java/org/apache/spark/shuffle/RssShuffleManager.java
+++ b/client-spark/spark2/src/main/java/org/apache/spark/shuffle/RssShuffleManager.java
@@ -373,7 +373,9 @@ public class RssShuffleManager implements ShuffleManager {
@Override
public void stop() {
- heartBeatScheduledExecutorService.shutdownNow();
+ if (heartBeatScheduledExecutorService != null) {
+ heartBeatScheduledExecutorService.shutdownNow();
+ }
threadPoolExecutor.shutdownNow();
shuffleWriteClient.close();
}