[spark] branch master updated: [SPARK-35086][SQL][CORE] --verbose should be passed to Spark SQL CLI too

[yu...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

yumwang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 9b2e0d6  [SPARK-35086][SQL][CORE] --verbose should be passed to Spark SQL CLI too
9b2e0d6 is described below

commit 9b2e0d6191003f7477266667b164497c248579d9
Author: Angerszhuuuu <an...@gmail.com>
AuthorDate: Thu Apr 15 12:59:20 2021 +0800

    [SPARK-35086][SQL][CORE] --verbose should be passed to Spark SQL CLI too
    
    ### What changes were proposed in this pull request?
    In current code, if we run spark sql with
    ```
    ./bin/spark-sql --verbose
    ```
    It won't be passed to end SparkSQLCliDriver, then the SessionState won't call `setIsVerbose`
    
    In the CLI option, it shows
    ```
    CLI options:
     -v,--verbose                     Verbose mode (echo executed SQL to the
                                      console)
    ```
    
    It's not consistent. This pr fix this issue
    ### Why are the changes needed?
    Fix bug
    
    ### Does this PR introduce _any_ user-facing change?
    when user call `-v` when run spark sql, sql will be echoed to console.
    
    ### How was this patch tested?
    Added UT
    
    Closes #32163 from AngersZhuuuu/SPARK-35086.
    
    Authored-by: Angerszhuuuu <an...@gmail.com>
    Signed-off-by: Yuming Wang <yu...@ebay.com>
---
 core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala      | 3 +++
 .../scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala    | 7 +++++++
 2 files changed, 10 insertions(+)

diff --git a/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala b/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala
index e5fd027..0e31fcf 100644
--- a/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala
+++ b/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala
@@ -852,6 +852,9 @@ private[spark] class SparkSubmit extends Logging {
     }
     sparkConf.set(SUBMIT_PYTHON_FILES, formattedPyFiles.split(",").toSeq)
 
+    if (args.verbose) {
+      childArgs ++= Seq("--verbose")
+    }
     (childArgs.toSeq, childClasspath.toSeq, sparkConf, childMainClass)
   }
 
diff --git a/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala b/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala
index 1a96012..b6e39a9 100644
--- a/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala
+++ b/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala
@@ -594,4 +594,11 @@ class CliSuite extends SparkFunSuite with BeforeAndAfterAll with Logging {
         -> "BroadcastHashJoin"
     )
   }
+
+  test("SPARK-35086: --verbose should be passed to Spark SQL CLI") {
+    runCliWithin(2.minute, Seq("--verbose"))(
+      "SELECT 'SPARK-35086' AS c1, '--verbose' AS c2;" ->
+        "SELECT 'SPARK-35086' AS c1, '--verbose' AS c2"
+    )
+  }
 }

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org