[GitHub] [spark] cloud-fan commented on a diff in pull request #41633: [SPARK-44082][SQL] Generate operator does not update reference set properly

["cloud-fan (via GitHub)" <gi...@apache.org>]

cloud-fan commented on code in PR #41633:
URL: https://github.com/apache/spark/pull/41633#discussion_r1235484958


##########
sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala:
##########
@@ -3613,6 +3613,21 @@ class DataFrameSuite extends QueryTest
     val df = Seq("0.5944910").toDF("a")
     checkAnswer(df.selectExpr("cast(a as decimal(7,7)) div 100"), Row(0))
   }
+
+  test("SPARK-44082: Generate operator does not update reference set properly") {
+    val schema = StructType(Array(
+      StructField("col1", StringType, true),
+      StructField("col2", StringType, true)
+    ))
+    val rowData = Seq(Row("100", "20000"),
+      Row("100", "100000"),
+      Row("300", "3000"))
+    val df1 = spark.createDataFrame(spark.sparkContext.parallelize(rowData), schema)
+    val df2 = spark.createDataFrame(spark.sparkContext.parallelize(rowData), schema)
+    df1.dropDuplicates(Seq("col1")).exceptAll(df2).collect.foreach { row =>

Review Comment:
   How is this test case related to the `Generate` operator?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org