You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Russell Jurney <ru...@gmail.com> on 2010/07/02 08:28:11 UTC
DateTime Proposal Request for Comment
I though I would ping pig-user about this to get comments. I put a proposal
for builtin date functions I can have ready for Pig 0.8 at
http://issues.apache.org/jira/browse/PIG-1430
There are so many classes to add a type, given my time constraints I don't
see the benefit of a full datetime type over this proposal right now. These
functions should be backwards compatible with a full-blown datetime type.
Russ
Re: DateTime Proposal Request for Comment
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
Agreed that there should be a default.
On Tue, Jul 6, 2010 at 4:30 PM, Russell Jurney <ru...@gmail.com>wrote:
> I had an optional format argument in there on the first pass, I'll add
> it back. ISO time should be a convention because it is an
> international standard, human readable string you can evaluate for
> inequality, and because having a convention provides a simpler
> interface to beginners parsing longs and storing them in a string
> format for readability.
>
> Russell Jurney
> russell.jurney@gmail.com
> (404) 317-3620
> http://twitter.com/rjurney
> http://linkedin.com/in/russelljurney
>
> On Jul 6, 2010, at 4:19 PM, Dmitriy Ryaboy <dv...@gmail.com> wrote:
>
> > I am not sure why the ISO format has to be specifically favored over
> others.
> > Allow for a constructor that passes a string format?
> >
> > -D
> >
> > On Tue, Jul 6, 2010 at 10:17 AM, hc busy <hc...@gmail.com> wrote:
> >
> >> +1 to making more things built-in.
> >>
> >> I think defaulting to use the timezone on the machine on which the
> >> grunt/Pig
> >> is invoked, followed by an option to specify what timezone to use will
> be
> >> the best way to go.
> >>
> >> Because that way, at least all compute nodes will be computing uniformly
> >> using the same timezone. And if the timezone on the machine on which the
> >> frontend ran is off, then that's easier to track down.
> >>
> >> Finally, of course allow for it to be configured inside PigLatin as
> well.
> >>
> >>
> >>
> >> On Fri, Jul 2, 2010 at 8:16 PM, Corbin Hoenes <co...@tynt.com> wrote:
> >>
> >>> Nice!
> >>>
> >>> Sent from my iPhone
> >>>
> >>>
> >>> On Jul 2, 2010, at 6:42 PM, Russell Jurney <ru...@gmail.com>
> >>> wrote:
> >>>
> >>> I'm planning to get rid of piggybank datetime stuff and make jodatime a
> >>>> core
> >>>> dependency to these builtin functions. No REGISTER required.
> >>>>
> >>>> Russ
> >>>>
> >>>> On Fri, Jul 2, 2010 at 2:27 PM, Corbin Hoenes <co...@tynt.com>
> wrote:
> >>>>
> >>>> So is your plan to package the jodatime dependency as part of
> piggybank
> >>>>> or
> >>>>> will it require 2 REGISTER jar commands one for piggybank and one for
> >>>>> joda?
> >>>>>
> >>>>> On Jul 2, 2010, at 3:02 PM, Russell Jurney wrote:
> >>>>>
> >>>>> I was planning to rely completely on Jodatime there, and I believe
> its
> >>>>>> default behavior is to use the local timezone of the machine it is
> on.
> >>>>>>
> >>>>> ISO
> >>>>>
> >>>>>> time encodes the time zone, and jodatime does the conversions back
> and
> >>>>>> forth, taking it into account.
> >>>>>>
> >>>>>> Should this be a pig config field? You can specify a timezone in
> >>>>>>
> >>>>> jodatime,
> >>>>>
> >>>>>> so that should work out ok, it would limit hadoop node shenanigans.
> >>>>>>
> >>>>>> Russ
> >>>>>>
> >>>>>> On Fri, Jul 2, 2010 at 1:05 PM, hc busy <hc...@gmail.com> wrote:
> >>>>>>
> >>>>>> I wonder if we can pull timezone support into the system... So some
> >> day
> >>>>>>>
> >>>>>> I
> >>>>>
> >>>>>> can say things like
> >>>>>>>
> >>>>>>>
> >>>>>>> 'jiffies between 11am EST and 13:11 IST'
> >>>>>>>
> >>>>>>> and get back the correct answer in PigLatin.
> >>>>>>>
> >>>>>>> also, how does ISO time know the current time zone?
> >>>>>>>
> >>>>>>> I've seen people run into this situation where the compute node and
> >>>>>>> name
> >>>>>>> node are in different time zones (by accident of course, they're
> >>>>>>>
> >>>>>> physically
> >>>>>
> >>>>>> in the same room), and it produces some problems that are difficult
> to
> >>>>>>> track
> >>>>>>> down.
> >>>>>>>
> >>>>>>> so, at least your UnixToISO will need to have some kind of
> convention
> >>>>>>>
> >>>>>> for
> >>>>>
> >>>>>> knowing what timezone to send it into, right? What's the plan there?
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Thu, Jul 1, 2010 at 11:28 PM, Russell Jurney <
> >>>>>>>
> >>>>>> russell.jurney@gmail.com
> >>>>>
> >>>>>> wrote:
> >>>>>>>>
> >>>>>>>
> >>>>>>> I though I would ping pig-user about this to get comments. I put a
> >>>>>>>> proposal
> >>>>>>>> for builtin date functions I can have ready for Pig 0.8 at
> >>>>>>>> http://issues.apache.org/jira/browse/PIG-1430
> >>>>>>>>
> >>>>>>>> There are so many classes to add a type, given my time constraints
> I
> >>>>>>>>
> >>>>>>> don't
> >>>>>>>
> >>>>>>>> see the benefit of a full datetime type over this proposal right
> >> now.
> >>>>>>>> These
> >>>>>>>> functions should be backwards compatible with a full-blown
> datetime
> >>>>>>>>
> >>>>>>> type.
> >>>>>
> >>>>>>
> >>>>>>>> Russ
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>
> >>>>>
> >>
>
Re: DateTime Proposal Request for Comment
Posted by Russell Jurney <ru...@gmail.com>.
I had an optional format argument in there on the first pass, I'll add
it back. ISO time should be a convention because it is an
international standard, human readable string you can evaluate for
inequality, and because having a convention provides a simpler
interface to beginners parsing longs and storing them in a string
format for readability.
Russell Jurney
russell.jurney@gmail.com
(404) 317-3620
http://twitter.com/rjurney
http://linkedin.com/in/russelljurney
On Jul 6, 2010, at 4:19 PM, Dmitriy Ryaboy <dv...@gmail.com> wrote:
> I am not sure why the ISO format has to be specifically favored over others.
> Allow for a constructor that passes a string format?
>
> -D
>
> On Tue, Jul 6, 2010 at 10:17 AM, hc busy <hc...@gmail.com> wrote:
>
>> +1 to making more things built-in.
>>
>> I think defaulting to use the timezone on the machine on which the
>> grunt/Pig
>> is invoked, followed by an option to specify what timezone to use will be
>> the best way to go.
>>
>> Because that way, at least all compute nodes will be computing uniformly
>> using the same timezone. And if the timezone on the machine on which the
>> frontend ran is off, then that's easier to track down.
>>
>> Finally, of course allow for it to be configured inside PigLatin as well.
>>
>>
>>
>> On Fri, Jul 2, 2010 at 8:16 PM, Corbin Hoenes <co...@tynt.com> wrote:
>>
>>> Nice!
>>>
>>> Sent from my iPhone
>>>
>>>
>>> On Jul 2, 2010, at 6:42 PM, Russell Jurney <ru...@gmail.com>
>>> wrote:
>>>
>>> I'm planning to get rid of piggybank datetime stuff and make jodatime a
>>>> core
>>>> dependency to these builtin functions. No REGISTER required.
>>>>
>>>> Russ
>>>>
>>>> On Fri, Jul 2, 2010 at 2:27 PM, Corbin Hoenes <co...@tynt.com> wrote:
>>>>
>>>> So is your plan to package the jodatime dependency as part of piggybank
>>>>> or
>>>>> will it require 2 REGISTER jar commands one for piggybank and one for
>>>>> joda?
>>>>>
>>>>> On Jul 2, 2010, at 3:02 PM, Russell Jurney wrote:
>>>>>
>>>>> I was planning to rely completely on Jodatime there, and I believe its
>>>>>> default behavior is to use the local timezone of the machine it is on.
>>>>>>
>>>>> ISO
>>>>>
>>>>>> time encodes the time zone, and jodatime does the conversions back and
>>>>>> forth, taking it into account.
>>>>>>
>>>>>> Should this be a pig config field? You can specify a timezone in
>>>>>>
>>>>> jodatime,
>>>>>
>>>>>> so that should work out ok, it would limit hadoop node shenanigans.
>>>>>>
>>>>>> Russ
>>>>>>
>>>>>> On Fri, Jul 2, 2010 at 1:05 PM, hc busy <hc...@gmail.com> wrote:
>>>>>>
>>>>>> I wonder if we can pull timezone support into the system... So some
>> day
>>>>>>>
>>>>>> I
>>>>>
>>>>>> can say things like
>>>>>>>
>>>>>>>
>>>>>>> 'jiffies between 11am EST and 13:11 IST'
>>>>>>>
>>>>>>> and get back the correct answer in PigLatin.
>>>>>>>
>>>>>>> also, how does ISO time know the current time zone?
>>>>>>>
>>>>>>> I've seen people run into this situation where the compute node and
>>>>>>> name
>>>>>>> node are in different time zones (by accident of course, they're
>>>>>>>
>>>>>> physically
>>>>>
>>>>>> in the same room), and it produces some problems that are difficult to
>>>>>>> track
>>>>>>> down.
>>>>>>>
>>>>>>> so, at least your UnixToISO will need to have some kind of convention
>>>>>>>
>>>>>> for
>>>>>
>>>>>> knowing what timezone to send it into, right? What's the plan there?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jul 1, 2010 at 11:28 PM, Russell Jurney <
>>>>>>>
>>>>>> russell.jurney@gmail.com
>>>>>
>>>>>> wrote:
>>>>>>>>
>>>>>>>
>>>>>>> I though I would ping pig-user about this to get comments. I put a
>>>>>>>> proposal
>>>>>>>> for builtin date functions I can have ready for Pig 0.8 at
>>>>>>>> http://issues.apache.org/jira/browse/PIG-1430
>>>>>>>>
>>>>>>>> There are so many classes to add a type, given my time constraints I
>>>>>>>>
>>>>>>> don't
>>>>>>>
>>>>>>>> see the benefit of a full datetime type over this proposal right
>> now.
>>>>>>>> These
>>>>>>>> functions should be backwards compatible with a full-blown datetime
>>>>>>>>
>>>>>>> type.
>>>>>
>>>>>>
>>>>>>>> Russ
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>
Re: DateTime Proposal Request for Comment
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
I am not sure why the ISO format has to be specifically favored over others.
Allow for a constructor that passes a string format?
-D
On Tue, Jul 6, 2010 at 10:17 AM, hc busy <hc...@gmail.com> wrote:
> +1 to making more things built-in.
>
> I think defaulting to use the timezone on the machine on which the
> grunt/Pig
> is invoked, followed by an option to specify what timezone to use will be
> the best way to go.
>
> Because that way, at least all compute nodes will be computing uniformly
> using the same timezone. And if the timezone on the machine on which the
> frontend ran is off, then that's easier to track down.
>
> Finally, of course allow for it to be configured inside PigLatin as well.
>
>
>
> On Fri, Jul 2, 2010 at 8:16 PM, Corbin Hoenes <co...@tynt.com> wrote:
>
> > Nice!
> >
> > Sent from my iPhone
> >
> >
> > On Jul 2, 2010, at 6:42 PM, Russell Jurney <ru...@gmail.com>
> > wrote:
> >
> > I'm planning to get rid of piggybank datetime stuff and make jodatime a
> >> core
> >> dependency to these builtin functions. No REGISTER required.
> >>
> >> Russ
> >>
> >> On Fri, Jul 2, 2010 at 2:27 PM, Corbin Hoenes <co...@tynt.com> wrote:
> >>
> >> So is your plan to package the jodatime dependency as part of piggybank
> >>> or
> >>> will it require 2 REGISTER jar commands one for piggybank and one for
> >>> joda?
> >>>
> >>> On Jul 2, 2010, at 3:02 PM, Russell Jurney wrote:
> >>>
> >>> I was planning to rely completely on Jodatime there, and I believe its
> >>>> default behavior is to use the local timezone of the machine it is on.
> >>>>
> >>> ISO
> >>>
> >>>> time encodes the time zone, and jodatime does the conversions back and
> >>>> forth, taking it into account.
> >>>>
> >>>> Should this be a pig config field? You can specify a timezone in
> >>>>
> >>> jodatime,
> >>>
> >>>> so that should work out ok, it would limit hadoop node shenanigans.
> >>>>
> >>>> Russ
> >>>>
> >>>> On Fri, Jul 2, 2010 at 1:05 PM, hc busy <hc...@gmail.com> wrote:
> >>>>
> >>>> I wonder if we can pull timezone support into the system... So some
> day
> >>>>>
> >>>> I
> >>>
> >>>> can say things like
> >>>>>
> >>>>>
> >>>>> 'jiffies between 11am EST and 13:11 IST'
> >>>>>
> >>>>> and get back the correct answer in PigLatin.
> >>>>>
> >>>>> also, how does ISO time know the current time zone?
> >>>>>
> >>>>> I've seen people run into this situation where the compute node and
> >>>>> name
> >>>>> node are in different time zones (by accident of course, they're
> >>>>>
> >>>> physically
> >>>
> >>>> in the same room), and it produces some problems that are difficult to
> >>>>> track
> >>>>> down.
> >>>>>
> >>>>> so, at least your UnixToISO will need to have some kind of convention
> >>>>>
> >>>> for
> >>>
> >>>> knowing what timezone to send it into, right? What's the plan there?
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Thu, Jul 1, 2010 at 11:28 PM, Russell Jurney <
> >>>>>
> >>>> russell.jurney@gmail.com
> >>>
> >>>> wrote:
> >>>>>>
> >>>>>
> >>>>> I though I would ping pig-user about this to get comments. I put a
> >>>>>> proposal
> >>>>>> for builtin date functions I can have ready for Pig 0.8 at
> >>>>>> http://issues.apache.org/jira/browse/PIG-1430
> >>>>>>
> >>>>>> There are so many classes to add a type, given my time constraints I
> >>>>>>
> >>>>> don't
> >>>>>
> >>>>>> see the benefit of a full datetime type over this proposal right
> now.
> >>>>>> These
> >>>>>> functions should be backwards compatible with a full-blown datetime
> >>>>>>
> >>>>> type.
> >>>
> >>>>
> >>>>>> Russ
> >>>>>>
> >>>>>>
> >>>>>
> >>>
> >>>
>
Re: DateTime Proposal Request for Comment
Posted by hc busy <hc...@gmail.com>.
+1 to making more things built-in.
I think defaulting to use the timezone on the machine on which the grunt/Pig
is invoked, followed by an option to specify what timezone to use will be
the best way to go.
Because that way, at least all compute nodes will be computing uniformly
using the same timezone. And if the timezone on the machine on which the
frontend ran is off, then that's easier to track down.
Finally, of course allow for it to be configured inside PigLatin as well.
On Fri, Jul 2, 2010 at 8:16 PM, Corbin Hoenes <co...@tynt.com> wrote:
> Nice!
>
> Sent from my iPhone
>
>
> On Jul 2, 2010, at 6:42 PM, Russell Jurney <ru...@gmail.com>
> wrote:
>
> I'm planning to get rid of piggybank datetime stuff and make jodatime a
>> core
>> dependency to these builtin functions. No REGISTER required.
>>
>> Russ
>>
>> On Fri, Jul 2, 2010 at 2:27 PM, Corbin Hoenes <co...@tynt.com> wrote:
>>
>> So is your plan to package the jodatime dependency as part of piggybank
>>> or
>>> will it require 2 REGISTER jar commands one for piggybank and one for
>>> joda?
>>>
>>> On Jul 2, 2010, at 3:02 PM, Russell Jurney wrote:
>>>
>>> I was planning to rely completely on Jodatime there, and I believe its
>>>> default behavior is to use the local timezone of the machine it is on.
>>>>
>>> ISO
>>>
>>>> time encodes the time zone, and jodatime does the conversions back and
>>>> forth, taking it into account.
>>>>
>>>> Should this be a pig config field? You can specify a timezone in
>>>>
>>> jodatime,
>>>
>>>> so that should work out ok, it would limit hadoop node shenanigans.
>>>>
>>>> Russ
>>>>
>>>> On Fri, Jul 2, 2010 at 1:05 PM, hc busy <hc...@gmail.com> wrote:
>>>>
>>>> I wonder if we can pull timezone support into the system... So some day
>>>>>
>>>> I
>>>
>>>> can say things like
>>>>>
>>>>>
>>>>> 'jiffies between 11am EST and 13:11 IST'
>>>>>
>>>>> and get back the correct answer in PigLatin.
>>>>>
>>>>> also, how does ISO time know the current time zone?
>>>>>
>>>>> I've seen people run into this situation where the compute node and
>>>>> name
>>>>> node are in different time zones (by accident of course, they're
>>>>>
>>>> physically
>>>
>>>> in the same room), and it produces some problems that are difficult to
>>>>> track
>>>>> down.
>>>>>
>>>>> so, at least your UnixToISO will need to have some kind of convention
>>>>>
>>>> for
>>>
>>>> knowing what timezone to send it into, right? What's the plan there?
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Jul 1, 2010 at 11:28 PM, Russell Jurney <
>>>>>
>>>> russell.jurney@gmail.com
>>>
>>>> wrote:
>>>>>>
>>>>>
>>>>> I though I would ping pig-user about this to get comments. I put a
>>>>>> proposal
>>>>>> for builtin date functions I can have ready for Pig 0.8 at
>>>>>> http://issues.apache.org/jira/browse/PIG-1430
>>>>>>
>>>>>> There are so many classes to add a type, given my time constraints I
>>>>>>
>>>>> don't
>>>>>
>>>>>> see the benefit of a full datetime type over this proposal right now.
>>>>>> These
>>>>>> functions should be backwards compatible with a full-blown datetime
>>>>>>
>>>>> type.
>>>
>>>>
>>>>>> Russ
>>>>>>
>>>>>>
>>>>>
>>>
>>>
Re: DateTime Proposal Request for Comment
Posted by Corbin Hoenes <co...@tynt.com>.
Nice!
Sent from my iPhone
On Jul 2, 2010, at 6:42 PM, Russell Jurney <ru...@gmail.com>
wrote:
> I'm planning to get rid of piggybank datetime stuff and make
> jodatime a core
> dependency to these builtin functions. No REGISTER required.
>
> Russ
>
> On Fri, Jul 2, 2010 at 2:27 PM, Corbin Hoenes <co...@tynt.com> wrote:
>
>> So is your plan to package the jodatime dependency as part of
>> piggybank or
>> will it require 2 REGISTER jar commands one for piggybank and one
>> for joda?
>>
>> On Jul 2, 2010, at 3:02 PM, Russell Jurney wrote:
>>
>>> I was planning to rely completely on Jodatime there, and I believe
>>> its
>>> default behavior is to use the local timezone of the machine it is
>>> on.
>> ISO
>>> time encodes the time zone, and jodatime does the conversions back
>>> and
>>> forth, taking it into account.
>>>
>>> Should this be a pig config field? You can specify a timezone in
>> jodatime,
>>> so that should work out ok, it would limit hadoop node shenanigans.
>>>
>>> Russ
>>>
>>> On Fri, Jul 2, 2010 at 1:05 PM, hc busy <hc...@gmail.com> wrote:
>>>
>>>> I wonder if we can pull timezone support into the system... So
>>>> some day
>> I
>>>> can say things like
>>>>
>>>>
>>>> 'jiffies between 11am EST and 13:11 IST'
>>>>
>>>> and get back the correct answer in PigLatin.
>>>>
>>>> also, how does ISO time know the current time zone?
>>>>
>>>> I've seen people run into this situation where the compute node
>>>> and name
>>>> node are in different time zones (by accident of course, they're
>> physically
>>>> in the same room), and it produces some problems that are
>>>> difficult to
>>>> track
>>>> down.
>>>>
>>>> so, at least your UnixToISO will need to have some kind of
>>>> convention
>> for
>>>> knowing what timezone to send it into, right? What's the plan
>>>> there?
>>>>
>>>>
>>>>
>>>> On Thu, Jul 1, 2010 at 11:28 PM, Russell Jurney <
>> russell.jurney@gmail.com
>>>>> wrote:
>>>>
>>>>> I though I would ping pig-user about this to get comments. I
>>>>> put a
>>>>> proposal
>>>>> for builtin date functions I can have ready for Pig 0.8 at
>>>>> http://issues.apache.org/jira/browse/PIG-1430
>>>>>
>>>>> There are so many classes to add a type, given my time
>>>>> constraints I
>>>> don't
>>>>> see the benefit of a full datetime type over this proposal right
>>>>> now.
>>>>> These
>>>>> functions should be backwards compatible with a full-blown
>>>>> datetime
>> type.
>>>>>
>>>>> Russ
>>>>>
>>>>
>>
>>
Re: DateTime Proposal Request for Comment
Posted by Russell Jurney <ru...@gmail.com>.
I'm planning to get rid of piggybank datetime stuff and make jodatime a core
dependency to these builtin functions. No REGISTER required.
Russ
On Fri, Jul 2, 2010 at 2:27 PM, Corbin Hoenes <co...@tynt.com> wrote:
> So is your plan to package the jodatime dependency as part of piggybank or
> will it require 2 REGISTER jar commands one for piggybank and one for joda?
>
> On Jul 2, 2010, at 3:02 PM, Russell Jurney wrote:
>
> > I was planning to rely completely on Jodatime there, and I believe its
> > default behavior is to use the local timezone of the machine it is on.
> ISO
> > time encodes the time zone, and jodatime does the conversions back and
> > forth, taking it into account.
> >
> > Should this be a pig config field? You can specify a timezone in
> jodatime,
> > so that should work out ok, it would limit hadoop node shenanigans.
> >
> > Russ
> >
> > On Fri, Jul 2, 2010 at 1:05 PM, hc busy <hc...@gmail.com> wrote:
> >
> >> I wonder if we can pull timezone support into the system... So some day
> I
> >> can say things like
> >>
> >>
> >> 'jiffies between 11am EST and 13:11 IST'
> >>
> >> and get back the correct answer in PigLatin.
> >>
> >> also, how does ISO time know the current time zone?
> >>
> >> I've seen people run into this situation where the compute node and name
> >> node are in different time zones (by accident of course, they're
> physically
> >> in the same room), and it produces some problems that are difficult to
> >> track
> >> down.
> >>
> >> so, at least your UnixToISO will need to have some kind of convention
> for
> >> knowing what timezone to send it into, right? What's the plan there?
> >>
> >>
> >>
> >> On Thu, Jul 1, 2010 at 11:28 PM, Russell Jurney <
> russell.jurney@gmail.com
> >>> wrote:
> >>
> >>> I though I would ping pig-user about this to get comments. I put a
> >>> proposal
> >>> for builtin date functions I can have ready for Pig 0.8 at
> >>> http://issues.apache.org/jira/browse/PIG-1430
> >>>
> >>> There are so many classes to add a type, given my time constraints I
> >> don't
> >>> see the benefit of a full datetime type over this proposal right now.
> >>> These
> >>> functions should be backwards compatible with a full-blown datetime
> type.
> >>>
> >>> Russ
> >>>
> >>
>
>
Re: DateTime Proposal Request for Comment
Posted by Corbin Hoenes <co...@tynt.com>.
So is your plan to package the jodatime dependency as part of piggybank or will it require 2 REGISTER jar commands one for piggybank and one for joda?
On Jul 2, 2010, at 3:02 PM, Russell Jurney wrote:
> I was planning to rely completely on Jodatime there, and I believe its
> default behavior is to use the local timezone of the machine it is on. ISO
> time encodes the time zone, and jodatime does the conversions back and
> forth, taking it into account.
>
> Should this be a pig config field? You can specify a timezone in jodatime,
> so that should work out ok, it would limit hadoop node shenanigans.
>
> Russ
>
> On Fri, Jul 2, 2010 at 1:05 PM, hc busy <hc...@gmail.com> wrote:
>
>> I wonder if we can pull timezone support into the system... So some day I
>> can say things like
>>
>>
>> 'jiffies between 11am EST and 13:11 IST'
>>
>> and get back the correct answer in PigLatin.
>>
>> also, how does ISO time know the current time zone?
>>
>> I've seen people run into this situation where the compute node and name
>> node are in different time zones (by accident of course, they're physically
>> in the same room), and it produces some problems that are difficult to
>> track
>> down.
>>
>> so, at least your UnixToISO will need to have some kind of convention for
>> knowing what timezone to send it into, right? What's the plan there?
>>
>>
>>
>> On Thu, Jul 1, 2010 at 11:28 PM, Russell Jurney <russell.jurney@gmail.com
>>> wrote:
>>
>>> I though I would ping pig-user about this to get comments. I put a
>>> proposal
>>> for builtin date functions I can have ready for Pig 0.8 at
>>> http://issues.apache.org/jira/browse/PIG-1430
>>>
>>> There are so many classes to add a type, given my time constraints I
>> don't
>>> see the benefit of a full datetime type over this proposal right now.
>>> These
>>> functions should be backwards compatible with a full-blown datetime type.
>>>
>>> Russ
>>>
>>
Re: DateTime Proposal Request for Comment
Posted by Russell Jurney <ru...@gmail.com>.
I was planning to rely completely on Jodatime there, and I believe its
default behavior is to use the local timezone of the machine it is on. ISO
time encodes the time zone, and jodatime does the conversions back and
forth, taking it into account.
Should this be a pig config field? You can specify a timezone in jodatime,
so that should work out ok, it would limit hadoop node shenanigans.
Russ
On Fri, Jul 2, 2010 at 1:05 PM, hc busy <hc...@gmail.com> wrote:
> I wonder if we can pull timezone support into the system... So some day I
> can say things like
>
>
> 'jiffies between 11am EST and 13:11 IST'
>
> and get back the correct answer in PigLatin.
>
> also, how does ISO time know the current time zone?
>
> I've seen people run into this situation where the compute node and name
> node are in different time zones (by accident of course, they're physically
> in the same room), and it produces some problems that are difficult to
> track
> down.
>
> so, at least your UnixToISO will need to have some kind of convention for
> knowing what timezone to send it into, right? What's the plan there?
>
>
>
> On Thu, Jul 1, 2010 at 11:28 PM, Russell Jurney <russell.jurney@gmail.com
> >wrote:
>
> > I though I would ping pig-user about this to get comments. I put a
> > proposal
> > for builtin date functions I can have ready for Pig 0.8 at
> > http://issues.apache.org/jira/browse/PIG-1430
> >
> > There are so many classes to add a type, given my time constraints I
> don't
> > see the benefit of a full datetime type over this proposal right now.
> > These
> > functions should be backwards compatible with a full-blown datetime type.
> >
> > Russ
> >
>
Re: DateTime Proposal Request for Comment
Posted by hc busy <hc...@gmail.com>.
I wonder if we can pull timezone support into the system... So some day I
can say things like
'jiffies between 11am EST and 13:11 IST'
and get back the correct answer in PigLatin.
also, how does ISO time know the current time zone?
I've seen people run into this situation where the compute node and name
node are in different time zones (by accident of course, they're physically
in the same room), and it produces some problems that are difficult to track
down.
so, at least your UnixToISO will need to have some kind of convention for
knowing what timezone to send it into, right? What's the plan there?
On Thu, Jul 1, 2010 at 11:28 PM, Russell Jurney <ru...@gmail.com>wrote:
> I though I would ping pig-user about this to get comments. I put a
> proposal
> for builtin date functions I can have ready for Pig 0.8 at
> http://issues.apache.org/jira/browse/PIG-1430
>
> There are so many classes to add a type, given my time constraints I don't
> see the benefit of a full datetime type over this proposal right now.
> These
> functions should be backwards compatible with a full-blown datetime type.
>
> Russ
>