(spark) branch master updated: [SPARK-48808][SQL] Fix NPE when connecting thriftserver through Hive 1.2.1 and the result schema is empty

[ya...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

yao pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new c73c412b172e [SPARK-48808][SQL] Fix NPE when connecting thriftserver through Hive 1.2.1 and the result schema is empty
c73c412b172e is described below

commit c73c412b172ee1bfd7466e1222f0b1cc5c7ba4c8
Author: Kent Yao <ya...@apache.org>
AuthorDate: Thu Jul 4 21:58:54 2024 +0800

    [SPARK-48808][SQL] Fix NPE when connecting thriftserver through Hive 1.2.1 and the result schema is empty
    
    ### What changes were proposed in this pull request?
    
    Lower Hive JDBC/Thrift Client doesn't test whether columns are null or not before handling it. In this PR, we set an empty list to it if the result schema is empty
    
    ### Why are the changes needed?
    
    bugfix for lower hive clients
    
    ### Does this PR introduce _any_ user-facing change?
    
    no
    
    ### How was this patch tested?
    
    tested offline by wangyum
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    no
    
    Closes #47213 from yaooqinn/SPARK-48808.
    
    Lead-authored-by: Kent Yao <ya...@apache.org>
    Co-authored-by: Yuming Wang <yu...@ebay.com>
    Signed-off-by: Kent Yao <ya...@apache.org>
---
 .../scala/org/apache/spark/sql/hive/thriftserver/RowSetUtils.scala   | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/RowSetUtils.scala b/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/RowSetUtils.scala
index 66319fff2468..2cc774501816 100644
--- a/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/RowSetUtils.scala
+++ b/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/RowSetUtils.scala
@@ -76,11 +76,12 @@ object RowSetUtils {
     val tRowSet = new TRowSet(startRowOffSet, new java.util.ArrayList[TRow](rowSize))
     var i = 0
     val columnSize = schema.length
+    val columns = new java.util.ArrayList[TColumn](columnSize)
     while (i < columnSize) {
-      val tColumn = toTColumn(rows, i, schema(i), timeFormatters, binaryFormatter)
-      tRowSet.addToColumns(tColumn)
+      columns.add(i, toTColumn(rows, i, schema(i), timeFormatters, binaryFormatter))
       i += 1
     }
+    tRowSet.setColumns(columns)
     tRowSet
   }
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org