[GitHub] [spark] HyukjinKwon commented on pull request #39239: [SPARK-41730][PYTHON] Set tz to UTC while converting of timestamps to python's datetime

["HyukjinKwon (via GitHub)" <gi...@apache.org>]

HyukjinKwon commented on PR #39239:
URL: https://github.com/apache/spark/pull/39239#issuecomment-1469830006

   Alright. Python's `datetime` is up to the program to deal with naive datetimes (https://docs.python.org/3/library/datetime.html#aware-and-naive-objects):
   
   > Whether a naive object represents Coordinated Universal Time (UTC), local time, or time in some other timezone is purely up to the program
   
   So if we argue that datetime with Timestanp NTZ is a datetime without timezone. ` df.collect()[0][0].utctimetuple()` has to be ` df.collect()[0][0].timetuple()`.
   
   So the problem here would be implementation detail. Previously, we get the physical timestamp assuming it's a local time. Now we should get the physical timestamp assuming the time is in UTC.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org