[GitHub] [spark] nchammas commented on a change in pull request #29935: [SPARK-33055][PYTHON][SQL] Add Python CalendarIntervalType

[GitBox <gi...@apache.org>]

nchammas commented on a change in pull request #29935:
URL: https://github.com/apache/spark/pull/29935#discussion_r562175621



##########
File path: python/pyspark/sql/types.py
##########
@@ -186,6 +186,30 @@ def fromInternal(self, ts):
             return datetime.datetime.fromtimestamp(ts // 1000000).replace(microsecond=ts % 1000000)
 
 
+class CalendarIntervalType(DataType, metaclass=DataTypeSingleton):
+    """Calendar Interval type
+    """
+    @classmethod
+    def typeName(cls):
+        return "interval"
+
+    def needConversion(self):
+        return True
+
+    def simpleString(self):
+        return "interval"
+
+    def toInternal(self, di):
+        raise NotImplementedError(
+            "Conversion from external Python types to interval not supported"
+        )

Review comment:
       I suppose in the future if we want to support conversion of Python's [`datetime.timedelta`](https://docs.python.org/3/library/datetime.html#datetime.timedelta), it would happen here, right?

##########
File path: python/pyspark/sql/types.py
##########
@@ -186,6 +186,30 @@ def fromInternal(self, ts):
             return datetime.datetime.fromtimestamp(ts // 1000000).replace(microsecond=ts % 1000000)
 
 
+class CalendarIntervalType(DataType, metaclass=DataTypeSingleton):

Review comment:
       Doesn't @zero323's example from the PR description show that Spark already exposes this type? 
   
   ```python
   spark.sql("SELECT current_date() - current_date()")
   ```
   
   For the record, btw, Postgres supports [an `interval` type](https://www.postgresql.org/docs/current/datatype-datetime.html) and done so since at least [version 7.1](https://www.postgresql.org/docs/7.1/datatype-datetime.html), which was released in 2001.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org