[GitHub] BryanCutler commented on a change in pull request #23795: [SPARK-26887][SQL][PYTHON] Create datetime.date directly instead of creating datetime64[ns] as intermediate data.

[GitBox <gi...@apache.org>]

BryanCutler commented on a change in pull request #23795: [SPARK-26887][SQL][PYTHON] Create datetime.date directly instead of creating datetime64[ns] as intermediate data.
URL: https://github.com/apache/spark/pull/23795#discussion_r257446732
 
 

 ##########
 File path: python/pyspark/sql/types.py
 ##########
 @@ -1681,38 +1681,53 @@ def from_arrow_schema(arrow_schema):
          for field in arrow_schema])
 
 
-def _check_series_convert_date(series, data_type):
-    """
-    Cast the series to datetime.date if it's a date type, otherwise returns the original series.
+def _arrow_column_to_pandas(column, data_type):
+    """ Convert Arrow Column to pandas Series.
+
+    If the given column is a date type column, creates a series of datetime.date directly instead
+    of creating datetime64[ns] as intermediate data.
 
 Review comment:
   It would be nice to say that for dates this will return `datetime.date`, but yeah maybe move the part about datetime[64] as intermediate to an internal comment.  `_arrow_table_to_pandas` has a comment that the reason for this is to match pyspark w/o arrow, but maybe it would be good to add here as well.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org