[GitHub] rednaxelafx commented on a change in pull request #23731: [SPARK-26572][SQL] fix aggregate codegen result evaluation

[GitBox <gi...@apache.org>]

rednaxelafx commented on a change in pull request #23731: [SPARK-26572][SQL] fix aggregate codegen result evaluation
URL: https://github.com/apache/spark/pull/23731#discussion_r256145599
 
 

 ##########
 File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala
 ##########
 @@ -2110,4 +2112,28 @@ class DataFrameSuite extends QueryTest with SharedSQLContext {
       checkAnswer(res, Row("1-1", 6, 6))
     }
   }
+
+  test("SPARK-26572: fix aggregate codegen result evaluation") {
+    val baseTable = Seq((1), (1)).toDF("idx")
+
+    // BroadcastHashJoinExec with a HashAggregateExec child containing no aggregate expressions
+    val distinctWithId = baseTable.distinct().withColumn("id", monotonically_increasing_id())
 
 Review comment:
   I'm not sure how stable the results are going to be if you use `monotonically_increasing_id` here with an unspecified number of shuffle partitions. Since you're checking the exact value of the resulting id, if the number of shuffle partitions changes (let's say if someone decides to change the default shuffle partitions setting in all tests), this test can become fragile and fail unnecessarily.
   
   It might be worth setting the shuffle partition to 1 explicitly inside this test case. Or go back to grouping by `id` instead of checking the exact value of `id`, or just assert the `id`s are equal.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org