[GitHub] [spark] cloud-fan commented on a change in pull request #32945: [SPARK-35769][SQL] Truncate java.time.Period by fields of year-month interval type

[GitBox <gi...@apache.org>]

cloud-fan commented on a change in pull request #32945:
URL: https://github.com/apache/spark/pull/32945#discussion_r654111649



##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGeneratorSuite.scala
##########
@@ -154,7 +156,15 @@ class RandomDataGeneratorSuite extends SparkFunSuite with SQLHelper {
         val data = generator.apply()
         val catalyst = toCatalyst(data)
         val convertedBack = toScala(catalyst)
-        assert(data == convertedBack)
+        dt match {

Review comment:
       shall we update the `RandomDataGenerator`? to make sure we generate correct data for `interval year`.

##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala
##########
@@ -284,7 +285,10 @@ object RandomDataGenerator {
         new CalendarInterval(months, days, ns)
       })
       case _: DayTimeIntervalType => Some(() => Duration.of(rand.nextLong(), ChronoUnit.MICROS))
-      case _: YearMonthIntervalType => Some(() => Period.ofMonths(rand.nextInt()).normalized())
+      case YearMonthIntervalType(_, YEAR) =>
+        val period = Period.ofMonths(rand.nextInt()).normalized()
+        Some(() => Period.ofYears(period.getYears).normalized())

Review comment:
       can we just do `Period.ofYears(rand.nextInt() / 12).normalized())`?

##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala
##########
@@ -284,7 +285,10 @@ object RandomDataGenerator {
         new CalendarInterval(months, days, ns)
       })
       case _: DayTimeIntervalType => Some(() => Duration.of(rand.nextLong(), ChronoUnit.MICROS))
-      case _: YearMonthIntervalType => Some(() => Period.ofMonths(rand.nextInt()).normalized())
+      case YearMonthIntervalType(_, YEAR) =>
+        val period = Period.ofMonths(rand.nextInt()).normalized()
+        Some(() => Period.ofYears(period.getYears).normalized())

Review comment:
       did you push the commit?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org