You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2023/06/06 16:35:03 UTC
[spark] branch branch-3.4 updated: [SPARK-43976][CORE] Handle the case where modifiedConfigs doesn't exist in event logs
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-3.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.4 by this push:
new 778beb3bc67 [SPARK-43976][CORE] Handle the case where modifiedConfigs doesn't exist in event logs
778beb3bc67 is described below
commit 778beb3bc67ea784ef0c4314a1f998482eea2e5f
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Tue Jun 6 09:34:40 2023 -0700
[SPARK-43976][CORE] Handle the case where modifiedConfigs doesn't exist in event logs
### What changes were proposed in this pull request?
This prevents NPE by handling the case where `modifiedConfigs` doesn't exist in event logs.
### Why are the changes needed?
Basically, this is the same solution for that case.
- https://github.com/apache/spark/pull/34907
The new code was added here, but we missed the corner case.
- https://github.com/apache/spark/pull/35972
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the CIs.
Closes #41472 from dongjoon-hyun/SPARK-43976.
Authored-by: Dongjoon Hyun <do...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
(cherry picked from commit b4ab34bf9b22d0f0ca4ab13f9b6106f38ccfaebe)
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
.../scala/org/apache/spark/sql/execution/ui/ExecutionPage.scala | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/ExecutionPage.scala b/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/ExecutionPage.scala
index 498bb2a6c1c..c0e6b65d634 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/ExecutionPage.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/ExecutionPage.scala
@@ -84,15 +84,14 @@ class ExecutionPage(parent: SQLTab) extends WebUIPage("execution") with Logging
val metrics = sqlStore.executionMetrics(executionId)
val graph = sqlStore.planGraph(executionId)
+ val configs = Option(executionUIData.modifiedConfigs).getOrElse(Map.empty)
summary ++
planVisualization(request, metrics, graph) ++
physicalPlanDescription(executionUIData.physicalPlanDescription) ++
- modifiedConfigs(
- executionUIData.modifiedConfigs.filterKeys(
- !_.startsWith(pandasOnSparkConfPrefix)).toMap) ++
+ modifiedConfigs(configs.filterKeys(!_.startsWith(pandasOnSparkConfPrefix)).toMap) ++
modifiedPandasOnSparkConfigs(
- executionUIData.modifiedConfigs.filterKeys(_.startsWith(pandasOnSparkConfPrefix)).toMap)
+ configs.filterKeys(_.startsWith(pandasOnSparkConfPrefix)).toMap)
}.getOrElse {
<div>No information to display for query {executionId}</div>
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org