You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Marian Condurache <M....@bigpoint.net> on 2011/07/29 15:06:19 UTC
Unixtimestamp
Hi,
Any ideas how to convert unixtimestamp to some readable date format like YYYY-MM-DD ?
Is there a buit in function ?
Because I tried UnixToIso....but it expects unix time long ....or is there a way to convert timestamp in time long in Pig ?
LG, Marian
AW: Unixtimestamp
Posted by Marian Condurache <M....@bigpoint.net>.
Oau great ...THANKS !!!!!
-----Ursprüngliche Nachricht-----
Von: Norbert Burger [mailto:norbert.burger@gmail.com]
Gesendet: Freitag, 29. Juli 2011 16:32
An: user@pig.apache.org
Betreff: Re: Unixtimestamp
UnixToISO appears timestamps in milliseconds, not seconds, since the epoch.
The following fragment seems to generate what you're looking for:
file2 = LOAD 'timestamps.txt' USING PigStorage(',') AS (insertDate:long,
changeDate:long,a4);
myF3 = FOREACH file2 GENERATE ISOToYear(UnixToISO(insertDate*1000));
dump myF3;
On an input of:
1154518325,1216638895,0
1139353200,1232213159,0
It generates:
(2006-01-01T00:00:00.000Z)
(2006-01-01T00:00:00.000Z)
Norbert
On Fri, Jul 29, 2011 at 10:01 AM, Marian Condurache <
M.Condurache@bigpoint.net> wrote:
> Well i have some logs with unixtimestamps in
> Something like
>
> 1154518325,1216638895,0
> 1139353200,1232213159,0
> ...
> And so on
>
> Then I run:
>
> REGISTER /home/mojo/apps/pig/contrib/piggybank/java/piggybank.jar ;
> REGISTER /home/mojo/apps/pig/contrib/piggybank/java/lib/joda-time-1.6.jar;
>
> DEFINE UnixToISO
> org.apache.pig.piggybank.evaluation.datetime.convert.UnixToISO();
> DEFINE ISOToYear
> org.apache.pig.piggybank.evaluation.datetime.truncate.ISOToYear();
>
>
> file2 = load '/user/mojo/input/insStatsTest' USING PigStorage(',')
> AS (insertDate:long, changeDate:long,a4);
>
>
> myF2 = FOREACH file2 {
> x = UnixToISO(insertDate) ;
> y = ISOToYear(x) ;
> generate x,y;
> }
>
> myF3 = FOREACH file2 generate UnixToISO(insertDate);
>
> rm /user/mcondurache/output/testData
>
> STORE myF3 INTO '/user/mcondurache/output/testData' using PigStorage(',');
>
> And then I get back
>
>
> 1970-01-14T08:41:58.325Z,1970-01-01T00:00:00.000Z
> 1970-01-14T04:29:13.200Z,1970-01-01T00:00:00.000Z
> ...
>
> I am running a built version of pig from your repository and I downloaded
> the joda-time-1.6.jar from sourceforge.net website
>
> ,M
>
> -----Ursprüngliche Nachricht-----
> Von: Norbert Burger [mailto:norbert.burger@gmail.com]
> Gesendet: Freitag, 29. Juli 2011 15:33
> An: user@pig.apache.org
> Betreff: Re: Unixtimestamp
>
> Seems like the Piggybank's UnixToISO should do what you're looking for - if
> not long, in what datatype is your unixtimestamp arriving to your Pig
> script?
>
> Pig supports explicit casts [1], but I think the conversion you're
> referring
> to should be handled implicitly.
>
> [1] http://ofps.oreilly.com/titles/9781449302641/data_model.html#casts
>
> Norbert
>
> On Fri, Jul 29, 2011 at 9:06 AM, Marian Condurache <
> M.Condurache@bigpoint.net> wrote:
>
> > Hi,
> > Any ideas how to convert unixtimestamp to some readable date format like
> > YYYY-MM-DD ?
> > Is there a buit in function ?
> > Because I tried UnixToIso....but it expects unix time long ....or is
> there
> > a way to convert timestamp in time long in Pig ?
> >
> >
> >
> > LG, Marian
> >
>
Re: Unixtimestamp
Posted by Norbert Burger <no...@gmail.com>.
UnixToISO appears timestamps in milliseconds, not seconds, since the epoch.
The following fragment seems to generate what you're looking for:
file2 = LOAD 'timestamps.txt' USING PigStorage(',') AS (insertDate:long,
changeDate:long,a4);
myF3 = FOREACH file2 GENERATE ISOToYear(UnixToISO(insertDate*1000));
dump myF3;
On an input of:
1154518325,1216638895,0
1139353200,1232213159,0
It generates:
(2006-01-01T00:00:00.000Z)
(2006-01-01T00:00:00.000Z)
Norbert
On Fri, Jul 29, 2011 at 10:01 AM, Marian Condurache <
M.Condurache@bigpoint.net> wrote:
> Well i have some logs with unixtimestamps in
> Something like
>
> 1154518325,1216638895,0
> 1139353200,1232213159,0
> ...
> And so on
>
> Then I run:
>
> REGISTER /home/mojo/apps/pig/contrib/piggybank/java/piggybank.jar ;
> REGISTER /home/mojo/apps/pig/contrib/piggybank/java/lib/joda-time-1.6.jar;
>
> DEFINE UnixToISO
> org.apache.pig.piggybank.evaluation.datetime.convert.UnixToISO();
> DEFINE ISOToYear
> org.apache.pig.piggybank.evaluation.datetime.truncate.ISOToYear();
>
>
> file2 = load '/user/mojo/input/insStatsTest' USING PigStorage(',')
> AS (insertDate:long, changeDate:long,a4);
>
>
> myF2 = FOREACH file2 {
> x = UnixToISO(insertDate) ;
> y = ISOToYear(x) ;
> generate x,y;
> }
>
> myF3 = FOREACH file2 generate UnixToISO(insertDate);
>
> rm /user/mcondurache/output/testData
>
> STORE myF3 INTO '/user/mcondurache/output/testData' using PigStorage(',');
>
> And then I get back
>
>
> 1970-01-14T08:41:58.325Z,1970-01-01T00:00:00.000Z
> 1970-01-14T04:29:13.200Z,1970-01-01T00:00:00.000Z
> ...
>
> I am running a built version of pig from your repository and I downloaded
> the joda-time-1.6.jar from sourceforge.net website
>
> ,M
>
> -----Ursprüngliche Nachricht-----
> Von: Norbert Burger [mailto:norbert.burger@gmail.com]
> Gesendet: Freitag, 29. Juli 2011 15:33
> An: user@pig.apache.org
> Betreff: Re: Unixtimestamp
>
> Seems like the Piggybank's UnixToISO should do what you're looking for - if
> not long, in what datatype is your unixtimestamp arriving to your Pig
> script?
>
> Pig supports explicit casts [1], but I think the conversion you're
> referring
> to should be handled implicitly.
>
> [1] http://ofps.oreilly.com/titles/9781449302641/data_model.html#casts
>
> Norbert
>
> On Fri, Jul 29, 2011 at 9:06 AM, Marian Condurache <
> M.Condurache@bigpoint.net> wrote:
>
> > Hi,
> > Any ideas how to convert unixtimestamp to some readable date format like
> > YYYY-MM-DD ?
> > Is there a buit in function ?
> > Because I tried UnixToIso....but it expects unix time long ....or is
> there
> > a way to convert timestamp in time long in Pig ?
> >
> >
> >
> > LG, Marian
> >
>
AW: Unixtimestamp
Posted by Marian Condurache <M....@bigpoint.net>.
Well i have some logs with unixtimestamps in
Something like
1154518325,1216638895,0
1139353200,1232213159,0
...
And so on
Then I run:
REGISTER /home/mojo/apps/pig/contrib/piggybank/java/piggybank.jar ;
REGISTER /home/mojo/apps/pig/contrib/piggybank/java/lib/joda-time-1.6.jar;
DEFINE UnixToISO org.apache.pig.piggybank.evaluation.datetime.convert.UnixToISO();
DEFINE ISOToYear org.apache.pig.piggybank.evaluation.datetime.truncate.ISOToYear();
file2 = load '/user/mojo/input/insStatsTest' USING PigStorage(',')
AS (insertDate:long, changeDate:long,a4);
myF2 = FOREACH file2 {
x = UnixToISO(insertDate) ;
y = ISOToYear(x) ;
generate x,y;
}
myF3 = FOREACH file2 generate UnixToISO(insertDate);
rm /user/mcondurache/output/testData
STORE myF3 INTO '/user/mcondurache/output/testData' using PigStorage(',');
And then I get back
1970-01-14T08:41:58.325Z,1970-01-01T00:00:00.000Z
1970-01-14T04:29:13.200Z,1970-01-01T00:00:00.000Z
...
I am running a built version of pig from your repository and I downloaded the joda-time-1.6.jar from sourceforge.net website
,M
-----Ursprüngliche Nachricht-----
Von: Norbert Burger [mailto:norbert.burger@gmail.com]
Gesendet: Freitag, 29. Juli 2011 15:33
An: user@pig.apache.org
Betreff: Re: Unixtimestamp
Seems like the Piggybank's UnixToISO should do what you're looking for - if
not long, in what datatype is your unixtimestamp arriving to your Pig
script?
Pig supports explicit casts [1], but I think the conversion you're referring
to should be handled implicitly.
[1] http://ofps.oreilly.com/titles/9781449302641/data_model.html#casts
Norbert
On Fri, Jul 29, 2011 at 9:06 AM, Marian Condurache <
M.Condurache@bigpoint.net> wrote:
> Hi,
> Any ideas how to convert unixtimestamp to some readable date format like
> YYYY-MM-DD ?
> Is there a buit in function ?
> Because I tried UnixToIso....but it expects unix time long ....or is there
> a way to convert timestamp in time long in Pig ?
>
>
>
> LG, Marian
>
Re: Unixtimestamp
Posted by Norbert Burger <no...@gmail.com>.
Seems like the Piggybank's UnixToISO should do what you're looking for - if
not long, in what datatype is your unixtimestamp arriving to your Pig
script?
Pig supports explicit casts [1], but I think the conversion you're referring
to should be handled implicitly.
[1] http://ofps.oreilly.com/titles/9781449302641/data_model.html#casts
Norbert
On Fri, Jul 29, 2011 at 9:06 AM, Marian Condurache <
M.Condurache@bigpoint.net> wrote:
> Hi,
> Any ideas how to convert unixtimestamp to some readable date format like
> YYYY-MM-DD ?
> Is there a buit in function ?
> Because I tried UnixToIso....but it expects unix time long ....or is there
> a way to convert timestamp in time long in Pig ?
>
>
>
> LG, Marian
>