[spark] branch master updated: [SPARK-27075] Remove duplicate execution tag parameters from the url, when accessing the execution table in the SQL page

[va...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

vanzin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 713646d  [SPARK-27075] Remove duplicate execution tag parameters from the url, when accessing the execution table in the SQL page
713646d is described below

commit 713646ddc20188331a142f48a4f4ec11cf4c52ed
Author: Shahid <sh...@gmail.com>
AuthorDate: Thu Mar 7 12:52:46 2019 -0800

    [SPARK-27075] Remove duplicate execution tag parameters from the url, when accessing the execution table in the SQL page
    
    ## What changes were proposed in this pull request?
    
    When we sort any columns in the execution table of the SQL page in the WEBUI, it throws IllegalArgumentException. The root cause is that,  in the url, we are duplicating the execution tag parameters in the 'parameterPath'. Actually we should filter out the executionTag related entries while getting the 'parameterOtherTable'
    https://github.com/apache/spark/blob/e9e8bb33ef9ad785473ded168bc85867dad4ee70/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/AllExecutionsPage.scala#L161-L163
    https://github.com/apache/spark/blob/e9e8bb33ef9ad785473ded168bc85867dad4ee70/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/AllExecutionsPage.scala#L241
    https://github.com/apache/spark/blob/e9e8bb33ef9ad785473ded168bc85867dad4ee70/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/AllExecutionsPage.scala#L263-L266
    
    ## How was this patch tested?
    Manually tested
    Test steps:
    Sort any column in the sql page execution table
    Before fix:
    ![screenshot from 2019-03-07 01-38-17](https://user-images.githubusercontent.com/23054875/53913261-f0b69580-4080-11e9-88ea-f238b47a21d5.png)
    
    After fix:
    ![screenshot from 2019-03-07 02-01-40](https://user-images.githubusercontent.com/23054875/53913285-01670b80-4081-11e9-81b6-78cdbf5a0817.png)
    
    Closes #23994 from shahidki31/SPARK-27075.
    
    Authored-by: Shahid <sh...@gmail.com>
    Signed-off-by: Marcelo Vanzin <va...@cloudera.com>
---
 .../org/apache/spark/sql/execution/ui/AllExecutionsPage.scala     | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/AllExecutionsPage.scala b/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/AllExecutionsPage.scala
index 05890de..43ff1e1 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/AllExecutionsPage.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/AllExecutionsPage.scala
@@ -158,9 +158,11 @@ private[ui] class AllExecutionsPage(parent: SQLTab) extends WebUIPage("") with L
     showSucceededJobs: Boolean,
     showFailedJobs: Boolean): Seq[Node] = {
 
-    val parameterOtherTable = request.getParameterMap().asScala.map { case (name, vals) =>
-      name + "=" + vals(0)
-    }
+    val parameterOtherTable = request.getParameterMap().asScala
+      .filterNot(_._1.startsWith(executionTag))
+      .map { case (name, vals) =>
+        name + "=" + vals(0)
+      }
 
     val parameterExecutionPage = request.getParameter(s"$executionTag.page")
     val parameterExecutionSortColumn = request.getParameter(s"$executionTag.sort")


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org