[GitHub] [spark] bart-samwel commented on pull request #28576: [SPARK-31755][SQL] allow missing year/hour when parsing date/timestamp string

[GitBox <gi...@apache.org>]

bart-samwel commented on pull request #28576:
URL: https://github.com/apache/spark/pull/28576#issuecomment-630696349


   Note that if we were designing this fresh, the year 1970 would not be a great choice when it would be used to parse arbitrary Month/Day combinations. The reason being that it is not a leap year, which means that it would never parse Feb 29.
   
   This behavior seems to be pretty broken. We should consider forbidding missing things in the middle / from the top, i.e., you have to have a prefix of (year, month, day, hour, minute, second, millis/micros/nanos). If anybody wants that, they can simulate the defaults by prefixing with their own stuff, e.g. if you want to parse hh:mm:ss then you have to parse `to_timestamp(concat('1970-01-01 ', the_thing_we_want_to_parse, 'yyyy-MM-dd hh:mm:ss')`. We can even hint this in the error message if we would forbid this by default, with a legacy config that people can use as a fallback.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org