[GitHub] [spark] cloud-fan commented on a change in pull request #27498: [SPARK-30688][SQL] Week based dates not being parsed with TimestampFormatter

[GitBox <gi...@apache.org>]

cloud-fan commented on a change in pull request #27498:
URL: https://github.com/apache/spark/pull/27498#discussion_r427056244



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeFormatterHelper.scala
##########
@@ -99,12 +99,35 @@ private object DateTimeFormatterHelper {
     new DateTimeFormatterBuilder().parseCaseInsensitive()
   }
 
-  def toFormatter(builder: DateTimeFormatterBuilder, locale: Locale): DateTimeFormatter = {
-    builder
-      .parseDefaulting(ChronoField.MONTH_OF_YEAR, 1)
-      .parseDefaulting(ChronoField.DAY_OF_MONTH, 1)
-      .parseDefaulting(ChronoField.MINUTE_OF_HOUR, 0)
-      .parseDefaulting(ChronoField.SECOND_OF_MINUTE, 0)
+  def setDefaulting(builder: DateTimeFormatterBuilder,
+      locale: Locale,
+      isWeekBased: Boolean = false): DateTimeFormatterBuilder = {
+
+    val defaults = Seq(
+      (ChronoField.ERA, 1),
+      (ChronoField.MINUTE_OF_HOUR, 0),
+      (ChronoField.SECOND_OF_MINUTE, 0)
+    )
+
+    val weekDayBased = if (isWeekBased) {
+      Seq((ChronoField.DAY_OF_WEEK,
+        WeekFields.of(locale).getFirstDayOfWeek.getValue))
+    } else {
+      Seq(
+        (ChronoField.MONTH_OF_YEAR, 1),
+        (ChronoField.DAY_OF_MONTH, 1))

Review comment:
       I found these default values are very fragile to be set here, so I remove them all in https://github.com/apache/spark/pull/28576 . Can you check if it fixes your problem?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org