[jira] [Commented] (AVRO-739) Add Date/Time data types

["Matthew Willson (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AVRO-739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14117584#comment-14117584 ] 

Matthew Willson commented on AVRO-739:
--------------------------------------

Hi all

Definitely concur that there should at least be a standard option for serializing timestamps in a compact form (e.g. epoch millis). The reason I'm using avro is because it's an efficient binary format which can cut down on the IO bottleneck of running big analysis jobs.

Having some standard for storing local timezone offset information alongside a timestamp would be useful for "local-time-of-day"-based analyses, e.g. of web traffic.

Since the majority of analyses will not be local-time-of-day based though, I'd prefer to store all timestamps in UTC epoch millis, and store the local timezone offset in a separate field which can be used to correct it where required for time-of-day analysis.

I'd suggest storing a timezone offset in minutes, since it should then fit into 2 bytes. This is the choice made in the javascript date.getTimezoneOffset() API for example, and it appears to be a safe assumption that all timezone boundaries in use are aligned to minute boundaries (in fact 15 minute boundaries as it stands). But I'm not too picky if someone has another sensible suggestion.

Note this would mean you lose information about a logical timezone name, e.g. BST for british summertime, or "Europe/London" for whatever timezone is in force in London at this point in local time. For most purposes this is a good thing I think, since the definitions of these things can shift over time, whereas a UTC offset is pretty unambiguous.

> Add Date/Time data types
> ------------------------
>
>                 Key: AVRO-739
>                 URL: https://issues.apache.org/jira/browse/AVRO-739
>             Project: Avro
>          Issue Type: New Feature
>          Components: spec
>            Reporter: Jeff Hammerbacher
>             Fix For: 1.7.8
>
>         Attachments: AVRO-739-datetime-spec.xml.patch, AVRO-739-datetime-spec.xml.patch, AVRO-739-update-spec.diff, AVRO-739.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)