[GitHub] [arrow] jorgecarleitao commented on pull request #11138: ARROW-13977: [Format] clarify leap seconds and leap days for interval type

[GitBox <gi...@apache.org>]

jorgecarleitao commented on pull request #11138:
URL: https://github.com/apache/arrow/pull/11138#issuecomment-917565327


   If we do not account for leap seconds, isn't this type equivalent to `Duration[ms]`?
   
   I.e. Given `MS_IN_DAY = 60*60*24*1000`, if `Interval(1d,MS_IN_DAY) == Interval(2d,0)`, then `Interval(a,b) == Duration[ms](a*MS_IN_DAY + b)` which makes them semantically equivalent (and thus redundant?). This is how I would conclude that `Interval(1d,MS_IN_DAY) != Interval(2d,0)`, i.e. to capture different semantics.
   
   OTOH, [many systems do not handle leap seconds](https://github.com/chronotope/chrono/issues/198#issuecomment-354692880) and in 2012 [there were no plans to support leap seconds in Postgres](https://www.postgresql.org/message-id/23016.1327976502@sss.pgh.pa.us).
   
   So, my conclusion atm is that the decision tree is
   
   | care about leap  | Type                 |
   |------------------|----------------------|
   | days             | Interval MonthDaysNs |
   | seconds          | Interval DaysMs      |
   | days and seconds | Extension            |
   | None             | Duration             |
   
   If we ignore leap seconds in `DaysMs`, the table would become
   
   | care about leap  | Type                 |
   |------------------|----------------------|
   | days             | Interval MonthDaysNs |
   | seconds          | Extension            |
   | days and seconds | Extension            |
   | None             | Duration             |
   
   and we may deprecate `DaysMs` without loss of generality?
   
   -----------------
   
   Since `DataFusion` follows postgres, I shouldn't we stick to `MONTHS_DAYS_NS` so that we capture its behavior of storing months, days and (nano)seconds separately,
   
   ```
   postgres=# SELECT  interval '11 months + 28 days + 86399 seconds' + interval '1 month + 1 day + 1 second';
           ?column?         
   -------------------------
    1 year 29 days 24:00:00
   (1 row)
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org