You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Weston Pace (Jira)" <ji...@apache.org> on 2021/06/30 09:01:00 UTC
[jira] [Created] (ARROW-13218) [Doc] Document/clarify conventions
for timestamp storage
Weston Pace created ARROW-13218:
-----------------------------------
Summary: [Doc] Document/clarify conventions for timestamp storage
Key: ARROW-13218
URL: https://issues.apache.org/jira/browse/ARROW-13218
Project: Apache Arrow
Issue Type: Improvement
Components: Documentation
Reporter: Weston Pace
This is related to several mailing list discussions:
[https://lists.apache.org/thread.html/r017084eed74edbc95810fc049056570f45b0bb034d6eeadd647e8621%40%3Cdev.arrow.apache.org%3E]
[https://lists.apache.org/thread.html/re349b0b4971acb3e6612ce4fd0346c5a8504e2daf97864087306998c%40%3Cdev.arrow.apache.org%3E]
[https://lists.apache.org/thread.html/r8216e5de3efd2935e3907ad9bd20ce07e430952f84de69b36337e5eb%40%3Cdev.arrow.apache.org%3E]
and some votes:
[https://lists.apache.org/thread.html/rd6b460ffe8eff4f03e3e9d48ad6c796106dd0b3321529751f7c19743%40%3Cdev.arrow.apache.org%3E]
[https://lists.apache.org/thread.html/r1bdffc76537ae9c12c37396880087fee9c0eec9000bf6ed4c9850c44%40%3Cdev.arrow.apache.org%3E]
TBD
The consensus appears to be that there are (conceptually) at least three temporal types, (Java calls these Instant, LocalDateTime, and ZonedDateTime). Some of these types could map to multiple physical types (e.g. ZonedDateTime could be stored as a Timestamp or a Date32/Time64 pair). Some even have multiple encoding schemes for a given representation (there are multiple ways to encode a LocalDateTime into an 8 byte timestamp field).
In addition, it appears the consensus is that Arrow does not need three first-class types. Instead the documentation should be updated to clearly define a recommended convention for encoding the various conceptual types.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)