You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2022/01/20 22:28:41 UTC
[spark] branch master updated: [SPARK-37806][K8S][FOLLOWUP] Use sc instead of sparkContext
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 9f95f46 [SPARK-37806][K8S][FOLLOWUP] Use sc instead of sparkContext
9f95f46 is described below
commit 9f95f46bd92d793c9d3f296e0a581795d3d11218
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Thu Jan 20 14:27:34 2022 -0800
[SPARK-37806][K8S][FOLLOWUP] Use sc instead of sparkContext
### What changes were proposed in this pull request?
This PR is a follow-up of https://github.com/apache/spark/pull/35096.
### Why are the changes needed?
To avoid NPE.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the CIs.
Closes #35263 from dongjoon-hyun/SPARK-37806-2.
Authored-by: Dongjoon Hyun <do...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
.../org/apache/spark/scheduler/cluster/k8s/ExecutorRollPlugin.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorRollPlugin.scala b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorRollPlugin.scala
index f6054a8..2a4d965 100644
--- a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorRollPlugin.scala
+++ b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorRollPlugin.scala
@@ -61,7 +61,7 @@ class ExecutorRollDriverPlugin extends DriverPlugin with Logging {
} else if (!sc.conf.get(DECOMMISSION_ENABLED)) {
logWarning(s"Disabled because ${DECOMMISSION_ENABLED.key} is false.")
} else {
- minTasks = sparkContext.conf.get(MINIMUM_TASKS_PER_EXECUTOR_BEFORE_ROLLING)
+ minTasks = sc.conf.get(MINIMUM_TASKS_PER_EXECUTOR_BEFORE_ROLLING)
// Scheduler is not created yet
sparkContext = sc
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org