[spark] branch master updated: [SPARK-43876][SQL][FOLLOWUP] Add a unit test for fast hashmap for distinct queries

[do...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 744ba23027e [SPARK-43876][SQL][FOLLOWUP] Add a unit test for fast hashmap for distinct queries
744ba23027e is described below

commit 744ba23027ed906aaa22f257649c0419c5f69365
Author: Kun Wan <wa...@apache.org>
AuthorDate: Thu Jun 22 13:49:28 2023 -0700

    [SPARK-43876][SQL][FOLLOWUP] Add a unit test for fast hashmap for distinct queries
    
    ### What changes were proposed in this pull request?
    
    Add UT for https://github.com/apache/spark/pull/41374 change.
    
    ### Why are the changes needed?
    
    Add a proper UT to coverage the change.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No
    
    ### How was this patch tested?
    
    Add UT
    
    Closes #41685 from wankunde/fast_agg_for_distinct_followup.
    
    Authored-by: Kun Wan <wa...@apache.org>
    Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
 .../scala/org/apache/spark/sql/DataFrameAggregateSuite.scala | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala
index 28c7ee94b9c..87fa57a7192 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala
@@ -1987,6 +1987,18 @@ class DataFrameAggregateSuite extends QueryTest
     }
     assert(error7.toString contains "UnsupportedOperationException")
   }
+
+  test("SPARK-43876: Enable fast hashmap for distinct queries") {
+    withSQLConf(SQLConf.ENABLE_TWOLEVEL_AGG_MAP.key -> "true") {
+      val df = testData2.distinct()
+      checkAnswer(df, testData2)
+      val output = new java.io.ByteArrayOutputStream()
+      Console.withOut(output) {
+        df.explain("codegen")
+      }
+      assert(output.toString().contains("public class hashAgg_FastHashMap_0"))
+    }
+  }
 }
 
 case class B(c: Option[Double])


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org