[spark] branch master updated: [SPARK-39400][SQL] spark-sql should remove hive resource dir in all case

[yu...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

yumwang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 4e4eb6f1ed5 [SPARK-39400][SQL] spark-sql should remove hive resource dir in all case
4e4eb6f1ed5 is described below

commit 4e4eb6f1ed5ff0d3caa7f424d2df23f186bf32a2
Author: Angerszhuuuu <an...@gmail.com>
AuthorDate: Thu Jun 9 08:06:13 2022 +0800

    [SPARK-39400][SQL] spark-sql should remove hive resource dir in all case
    
    ### What changes were proposed in this pull request?
    In current code, when we use `spark-sql` `-e` , `-f` or use `ctrl + c` to close  `spark-sql` session, will remain hive session resource dir under `/tmp` path, this pr help to clean this files
    
    ### Why are the changes needed?
    Clean remained files
    
    ### Does this PR introduce _any_ user-facing change?
    No
    
    ### How was this patch tested?
    Manuel tested
    
    Closes #36786 from AngersZhuuuu/SPARK-39400.
    
    Lead-authored-by: Angerszhuuuu <an...@gmail.com>
    Co-authored-by: AngersZhuuuu <an...@gmail.com>
    Signed-off-by: Yuming Wang <yu...@ebay.com>
---
 .../apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala b/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala
index fccb2a65273..d40cf73be63 100644
--- a/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala
+++ b/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala
@@ -103,10 +103,13 @@ private[hive] object SparkSQLCLIDriver extends Logging {
       sessionState.info = new PrintStream(System.err, true, UTF_8.name())
       sessionState.err = new PrintStream(System.err, true, UTF_8.name())
     } catch {
-      case e: UnsupportedEncodingException => System.exit(ERROR_PATH_NOT_FOUND)
+      case e: UnsupportedEncodingException =>
+        sessionState.close()
+        System.exit(ERROR_PATH_NOT_FOUND)
     }
 
     if (!oproc.process_stage2(sessionState)) {
+      sessionState.close()
       System.exit(ERROR_MISUSE_SHELL_BUILTIN)
     }
 
@@ -140,7 +143,10 @@ private[hive] object SparkSQLCLIDriver extends Logging {
     SessionState.setCurrentSessionState(sessionState)
 
     // Clean up after we exit
-    ShutdownHookManager.addShutdownHook { () => SparkSQLEnv.stop() }
+    ShutdownHookManager.addShutdownHook { () =>
+      sessionState.close()
+      SparkSQLEnv.stop()
+    }
 
     if (isRemoteMode(sessionState)) {
       // Hive 1.2 + not supported in CLI


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org