(spark) branch branch-3.4 updated: [SPARK-46417][SQL] Do not fail when calling hive.getTable and throwException is false

[ya...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

yao pushed a commit to branch branch-3.4
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-3.4 by this push:
     new 114754382b19 [SPARK-46417][SQL] Do not fail when calling hive.getTable and throwException is false
114754382b19 is described below

commit 114754382b19eded5488d63101fa4c520e8551ca
Author: Wenchen Fan <we...@databricks.com>
AuthorDate: Fri Dec 15 18:55:10 2023 +0800

    [SPARK-46417][SQL] Do not fail when calling hive.getTable and throwException is false
    
    ### What changes were proposed in this pull request?
    
    Uses can set up their own HMS and let Spark connects to it. We have no control over it and somtimes it's not even Hive but just a HMS-API-compatible service.
    
    Spark should be more fault-tolerant when calling HMS APIs. This PR fixes an issue in `hive.getTable` with `throwException = false`, to make sure we don't throw error when can't fetch the table.
    
    ### Why are the changes needed?
    
    avoid query failure caused by HMS bugs.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No
    
    ### How was this patch tested?
    
    in our product environment
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    No
    
    Closes #44364 from cloud-fan/hive.
    
    Lead-authored-by: Wenchen Fan <we...@databricks.com>
    Co-authored-by: Wenchen Fan <cl...@gmail.com>
    Signed-off-by: Kent Yao <ya...@apache.org>
    (cherry picked from commit 59488039f58b18617cd6dfd6dbe3bf014af222e7)
    Signed-off-by: Kent Yao <ya...@apache.org>
---
 .../main/scala/org/apache/spark/sql/hive/client/HiveShim.scala    | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala b/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
index 558af75ca8f5..c5b0ca2241cd 100644
--- a/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
+++ b/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
@@ -620,7 +620,13 @@ private[client] class Shim_v0_12 extends Shim with Logging {
       tableName: String,
       throwException: Boolean): Table = {
     recordHiveCall()
-    val table = hive.getTable(dbName, tableName, throwException)
+    val table = try {
+      hive.getTable(dbName, tableName, throwException)
+    } catch {
+      // Hive may have bugs and still throw an exception even if `throwException` is false.
+      case e: HiveException if !throwException =>
+        null
+    }
     if (table != null) {
       table.getTTable.setTableName(tableName)
       table.getTTable.setDbName(dbName)


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org