You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Muni mahesh <ma...@gmail.com> on 2013/09/20 16:00:52 UTC
ISOToUNix working in Pig 0.8.1 but not in Pig 0.11.0
Hi Hadoopers,
I did the same thing in Pig 0.8.1 but not Pig 0.11.0
register /usr/lib/pig/piggybank.jar;
register /usr/lib/pig/lib/joda-time-2.1.jar;
DEFINE CustomFormatToISO
org.apache.pig.piggybank.evaluation.datetime.convert.CustomFormatToISO();
DEFINE ISOToUnix
org.apache.pig.piggybank.evaluation.datetime.convert.ISOToUnix();
A = load '/home/user/Desktop/1.tsv' USING PigStorage('\t') AS
(dt1:chararray, dt2:chararray);
B = foreach A generate (long) ISOToUnix(CustomFormatToISO($0, 'yyyy-mm-dd
hh:mm:ss'));
*input *
2013-01-16 04:01:18 2013-01-16 04:01:36
2013-01-16 04:02:19 2013-01-16 04:03:11
*output* *expected*
(1358308878000,1358308896000)
(1358308939000,1358308991000)
Re: ISOToUNix working in Pig 0.8.1 but not in Pig 0.11.0
Posted by Pradeep Gollakota <pr...@gmail.com>.
Doh!
I think I made a mistake myself...
"yyyy-MM-dd HH:mm:ss"
Since you don't have AM/PM, I'm assuming that your time is 24-hr format.
So, you need to use the 24 hour format symbol of 'H' for hour instead of
'h'.
I really hate time.
On Fri, Sep 20, 2013 at 6:25 PM, Pradeep Gollakota <pr...@gmail.com>wrote:
> Be careful with your format definition... it looks like you might have a
> typo.
>
> I believe "yyyy-MM-dd hh:mm:ss" is the correct format.
> http://joda-time.sourceforge.net/apidocs/org/joda/time/format/DateTimeFormat.html
>
>
>
>
> On Fri, Sep 20, 2013 at 8:26 AM, Ruslan Al-Fakikh <me...@gmail.com>wrote:
>
>> What was the error?
>>
>> Not an issue, but why do you call the columns dt1, dt2, but not using the
>> name, using the ordinal number insted: $0?
>>
>>
>> On Fri, Sep 20, 2013 at 6:00 PM, Muni mahesh <mahesh87.hadoop@gmail.com
>> >wrote:
>>
>> > Hi Hadoopers,
>> >
>> > I did the same thing in Pig 0.8.1 but not Pig 0.11.0
>> >
>> > register /usr/lib/pig/piggybank.jar;
>> > register /usr/lib/pig/lib/joda-time-2.1.jar;
>> >
>> > DEFINE CustomFormatToISO
>> >
>> org.apache.pig.piggybank.evaluation.datetime.convert.CustomFormatToISO();
>> > DEFINE ISOToUnix
>> > org.apache.pig.piggybank.evaluation.datetime.convert.ISOToUnix();
>> >
>> > A = load '/home/user/Desktop/1.tsv' USING PigStorage('\t') AS
>> > (dt1:chararray, dt2:chararray);
>> > B = foreach A generate (long) ISOToUnix(CustomFormatToISO($0,
>> 'yyyy-mm-dd
>> > hh:mm:ss'));
>> >
>> >
>> > *input *
>> > 2013-01-16 04:01:18 2013-01-16 04:01:36
>> > 2013-01-16 04:02:19 2013-01-16 04:03:11
>> >
>> > *output* *expected*
>> > (1358308878000,1358308896000)
>> > (1358308939000,1358308991000)
>> >
>>
>
>
Re: ISOToUNix working in Pig 0.8.1 but not in Pig 0.11.0
Posted by Pradeep Gollakota <pr...@gmail.com>.
Be careful with your format definition... it looks like you might have a
typo.
I believe "yyyy-MM-dd hh:mm:ss" is the correct format.
http://joda-time.sourceforge.net/apidocs/org/joda/time/format/DateTimeFormat.html
On Fri, Sep 20, 2013 at 8:26 AM, Ruslan Al-Fakikh <me...@gmail.com>wrote:
> What was the error?
>
> Not an issue, but why do you call the columns dt1, dt2, but not using the
> name, using the ordinal number insted: $0?
>
>
> On Fri, Sep 20, 2013 at 6:00 PM, Muni mahesh <mahesh87.hadoop@gmail.com
> >wrote:
>
> > Hi Hadoopers,
> >
> > I did the same thing in Pig 0.8.1 but not Pig 0.11.0
> >
> > register /usr/lib/pig/piggybank.jar;
> > register /usr/lib/pig/lib/joda-time-2.1.jar;
> >
> > DEFINE CustomFormatToISO
> > org.apache.pig.piggybank.evaluation.datetime.convert.CustomFormatToISO();
> > DEFINE ISOToUnix
> > org.apache.pig.piggybank.evaluation.datetime.convert.ISOToUnix();
> >
> > A = load '/home/user/Desktop/1.tsv' USING PigStorage('\t') AS
> > (dt1:chararray, dt2:chararray);
> > B = foreach A generate (long) ISOToUnix(CustomFormatToISO($0, 'yyyy-mm-dd
> > hh:mm:ss'));
> >
> >
> > *input *
> > 2013-01-16 04:01:18 2013-01-16 04:01:36
> > 2013-01-16 04:02:19 2013-01-16 04:03:11
> >
> > *output* *expected*
> > (1358308878000,1358308896000)
> > (1358308939000,1358308991000)
> >
>
Re: ISOToUNix working in Pig 0.8.1 but not in Pig 0.11.0
Posted by Ruslan Al-Fakikh <me...@gmail.com>.
What was the error?
Not an issue, but why do you call the columns dt1, dt2, but not using the
name, using the ordinal number insted: $0?
On Fri, Sep 20, 2013 at 6:00 PM, Muni mahesh <ma...@gmail.com>wrote:
> Hi Hadoopers,
>
> I did the same thing in Pig 0.8.1 but not Pig 0.11.0
>
> register /usr/lib/pig/piggybank.jar;
> register /usr/lib/pig/lib/joda-time-2.1.jar;
>
> DEFINE CustomFormatToISO
> org.apache.pig.piggybank.evaluation.datetime.convert.CustomFormatToISO();
> DEFINE ISOToUnix
> org.apache.pig.piggybank.evaluation.datetime.convert.ISOToUnix();
>
> A = load '/home/user/Desktop/1.tsv' USING PigStorage('\t') AS
> (dt1:chararray, dt2:chararray);
> B = foreach A generate (long) ISOToUnix(CustomFormatToISO($0, 'yyyy-mm-dd
> hh:mm:ss'));
>
>
> *input *
> 2013-01-16 04:01:18 2013-01-16 04:01:36
> 2013-01-16 04:02:19 2013-01-16 04:03:11
>
> *output* *expected*
> (1358308878000,1358308896000)
> (1358308939000,1358308991000)
>