[spark] branch branch-3.3 updated: [SPARK-43976][CORE] Handle the case where modifiedConfigs doesn't exist in event logs

[do...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-3.3 by this push:
     new ababc572b66 [SPARK-43976][CORE] Handle the case where modifiedConfigs doesn't exist in event logs
ababc572b66 is described below

commit ababc572b66e0ccffa5fefed3c87b18e0f60cc50
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Tue Jun 6 09:34:40 2023 -0700

    [SPARK-43976][CORE] Handle the case where modifiedConfigs doesn't exist in event logs
    
    ### What changes were proposed in this pull request?
    
    This prevents NPE by handling the case where `modifiedConfigs` doesn't exist in event logs.
    
    ### Why are the changes needed?
    
    Basically, this is the same solution for that case.
    - https://github.com/apache/spark/pull/34907
    
    The new code was added here, but we missed the corner case.
    
    - https://github.com/apache/spark/pull/35972
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    Pass the CIs.
    
    Closes #41472 from dongjoon-hyun/SPARK-43976.
    
    Authored-by: Dongjoon Hyun <do...@apache.org>
    Signed-off-by: Dongjoon Hyun <do...@apache.org>
    (cherry picked from commit b4ab34bf9b22d0f0ca4ab13f9b6106f38ccfaebe)
    Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
 .../scala/org/apache/spark/sql/execution/ui/ExecutionPage.scala    | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/ExecutionPage.scala b/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/ExecutionPage.scala
index 498bb2a6c1c..c0e6b65d634 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/ExecutionPage.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/ExecutionPage.scala
@@ -84,15 +84,14 @@ class ExecutionPage(parent: SQLTab) extends WebUIPage("execution") with Logging
 
       val metrics = sqlStore.executionMetrics(executionId)
       val graph = sqlStore.planGraph(executionId)
+      val configs = Option(executionUIData.modifiedConfigs).getOrElse(Map.empty)
 
       summary ++
         planVisualization(request, metrics, graph) ++
         physicalPlanDescription(executionUIData.physicalPlanDescription) ++
-        modifiedConfigs(
-          executionUIData.modifiedConfigs.filterKeys(
-            !_.startsWith(pandasOnSparkConfPrefix)).toMap) ++
+        modifiedConfigs(configs.filterKeys(!_.startsWith(pandasOnSparkConfPrefix)).toMap) ++
         modifiedPandasOnSparkConfigs(
-          executionUIData.modifiedConfigs.filterKeys(_.startsWith(pandasOnSparkConfPrefix)).toMap)
+          configs.filterKeys(_.startsWith(pandasOnSparkConfPrefix)).toMap)
     }.getOrElse {
       <div>No information to display for query {executionId}</div>
     }


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org