You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by jamal sasha <ja...@gmail.com> on 2012/10/25 23:46:57 UTC
Better formated.. Pig udf help
Hi,****
I am trying to write a pig udf function.. Basically the data is of format*
***
** **
Id,time****
What I am trying to do is … parse the time and then see whether its
breakfast, lunch or dinner.. based on the time stamp. Some entries wil be
null as well..****
** **
So here is the udf code for this.****
** **
*public* *class* time *extends* EvalFunc<String>{****
** **
*public* String exec(Tuple input) *throws* IOException {****
****
*if* ((*input* == *null*) || (input.size() == 0))****
*return* *null*;****
*try*{****
String time = (String) input.get(0) ;****
DateFormat df = *new* SimpleDateFormat("hh:mm:ss.000");****
Date date = df.parse(time);****
String timeOfDay = *getTimeOfDay*(date);****
*return* timeOfDay;****
} *catch* (ParseException e) {****
//how will I handle when df.parse(time) fails and throws
ParseException?****
//maybe:****
*return* *null*;****
}****
****
****
}****
** **
After this.. in eclipse.. I did the export of this as a jar called myudfs
and I have a jar file called myudfs.jar****
** **
Then I wrote the pig script as ****
Time.pig****
** **
REGISTER path/to/udf/myudfs.jar****
in = LOAD 'path/to/input' USING PigStorage(',') AS (id:long,
time:chararray);****
result = foreach in GENERATE myudfs.time(time);****
STORE result INTO 'path/to/output' using PigStorage(',');****
** **
** **
So.. as per the documentation.. http://wiki.apache.org/pig/UDFManual****
** **
Now I did this..****
java -cp path/to/pig.jar org.apache.pig.Main time.pig****
** **
Exception in thread "main" java.lang.NoClassDefFoundError:
org/pache/pig/Main****
Caused by: java.lang.ClassNotFoundException: org.pache.pig.Main****
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)****
at java.security.AccessController.doPrivileged(Native Method)****
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)****
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)****
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)****
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)****
Could not find the main class: org.pache.pig.Main. Program will exit.****
** **
** **
What am I doing wrong?****
Thanks****
** **
** **
** **
** **
** **
American Express made the following annotations on Thu Oct 25 2012 13:50:42
******************************************************************************
"This message and any attachments are solely for the intended recipient and
may contain confidential or privileged information. If you are not the
intended recipient, any disclosure, copying, use, or distribution of the
information included in this message and any attachments is prohibited. If
you have received this communication in error, please notify us by reply
e-mail and immediately and permanently delete this message and any
attachments. Thank you."
American Express a ajouté le commentaire suivant le Thu Oct 25 2012
13:50:42
Ce courrier et toute pièce jointe qu'il contient sont réservés au seul
destinataire indiqué et peuvent renfermer des renseignements confidentiels
et privilégiés. Si vous n'êtes pas le destinataire prévu, toute
divulgation, duplication, utilisation ou distribution du courrier ou de
toute pièce jointe est interdite. Si vous avez reçu cette communication par
erreur, veuillez nous en aviser par courrier et détruire immédiatement le
courrier et les pièces jointes. Merci.
******************************************************************************
-------------------------------------------------------------------------------
Re: Better formated.. Pig udf help
Posted by jamal sasha <ja...@gmail.com>.
Hi
So I followed the instructions.
Echo $PIG_CLASSPATH
Points to hadoop conf
On pig -f time.pig
I still get same error? ?
Problem resolving class version number
So this is what I did
1 write a program in Java as my udf and export it as udf.jar
Wrote a simple pig script and register that udf
Now trying to run it thru pig -f time.pig
On Friday, October 26, 2012, Prashant Kommireddi <pr...@gmail.com>
wrote:
> Do you see a "conf" dir within /path/Hadoop? If yes, point your
> PIG_CLASSPATH to it.
>
> export PIG_CLASSPATH=/path/hadoop/conf.
>
> Sent from my iPhone
>
> On Oct 26, 2012, at 10:35 AM, jamal sasha <ja...@gmail.com> wrote:
>
>> Hi
>> Great catch
>> Now I get an error
>> Cannot find hadoop configuration in class path ( neither hadoop site XML
>> etc)
>>
>> So I am running the file on a cluster which had say hadoop set up as
>>
>> /path/hadoop
>> /path/pig
>>
>> And I have account in it
>> So I cannot change the hadoop conf files as other users are also using
it.
>> How do I run this just for me ?
>> On Thursday, October 25, 2012, Bill Graham <bi...@gmail.com> wrote:
>>> Somewhere you have a typo, probably in the execution of your program:
>>>
>>> Exception in thread "main" java.lang.NoClassDefFoundError:
>>> org/pache/pig/Main****
>>>
>>> Caused by: java.lang.ClassNotFoundException: org.pache.pig.Main****
>>>
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)****
>>>
>>> Note that the 'a' in apache is missing.
>>>
>>>
>>> On Thu, Oct 25, 2012 at 2:46 PM, jamal sasha <ja...@gmail.com>
>> wrote:
>>>
>>>> Hi,****
>>>>
>>>> I am trying to write a pig udf function.. Basically the data is of
>>>> format*
>>>> ***
>>>>
>>>> ** **
>>>>
>>>> Id,time****
>>>>
>>>> What I am trying to do is … parse the time and then see whether its
>>>> breakfast, lunch or dinner.. based on the time stamp. Some entries wil
be
>>>> null as well..****
>>>>
>>>> ** **
>>>>
>>>> So here is the udf code for this.****
>>>>
>>>> ** **
>>>>
>>>> *public* *class* time *extends* EvalFunc<String>{****
>>>>
>>>> ** **
>>>>
>>>> *public* String exec(Tuple input) *throws* IOException {****
>>>>
>>>> ****
>>>>
>>>> *if* ((*input* == *null*) || (input.size() == 0))****
>>>>
>>>> *return* *null*;****
>>>>
>>>> *try*{****
>>>>
>>>> String time = (String) input.get(0) ;****
>>>>
>>>> DateFormat df = *new*
>> SimpleDateFormat("hh:mm:ss.000");****
>>>>
>>>> Date date = df.parse(time);****
>>>>
>>>> String timeOfDay = *getTimeOfDay*(date);****
>>>>
>>>> *return* timeOfDay;****
>>>>
>>>> } *catch* (ParseException e) {****
>>>>
>>>> //how will I handle when df.parse(time) fails and throws
>>>> ParseException?****
>>>>
>>>> //maybe:****
>>>>
>>>> *return* *null*;****
>>>>
>>>> }****
>>>>
>>>> ****
>>>>
>>>> ****
>>>>
>>>> }****
>>>>
>>>> ** **
>>>>
>>>> After this.. in eclipse.. I did the export of this as a jar called
myudfs
>>>> and I have a jar file called myudfs.jar****
>>>>
>>>> ** **
>>>>
>>>> Then I wrote the pig script as ****
>>>>
>>>> Time.pig****
>>>>
>>>> ** **
>>>>
>>>> REGISTER path/to/udf/myudfs.jar****
>>>>
>>>> in = LOAD 'path/to/input' USING PigStorage(',') AS (id:long,
>>>> time:chararray);****
>>>>
>>>> result = f
Re: Better formated.. Pig udf help
Posted by Prashant Kommireddi <pr...@gmail.com>.
Do you see a "conf" dir within /path/Hadoop? If yes, point your
PIG_CLASSPATH to it.
export PIG_CLASSPATH=/path/hadoop/conf.
Sent from my iPhone
On Oct 26, 2012, at 10:35 AM, jamal sasha <ja...@gmail.com> wrote:
> Hi
> Great catch
> Now I get an error
> Cannot find hadoop configuration in class path ( neither hadoop site XML
> etc)
>
> So I am running the file on a cluster which had say hadoop set up as
>
> /path/hadoop
> /path/pig
>
> And I have account in it
> So I cannot change the hadoop conf files as other users are also using it.
> How do I run this just for me ?
> On Thursday, October 25, 2012, Bill Graham <bi...@gmail.com> wrote:
>> Somewhere you have a typo, probably in the execution of your program:
>>
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/pache/pig/Main****
>>
>> Caused by: java.lang.ClassNotFoundException: org.pache.pig.Main****
>>
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)****
>>
>> Note that the 'a' in apache is missing.
>>
>>
>> On Thu, Oct 25, 2012 at 2:46 PM, jamal sasha <ja...@gmail.com>
> wrote:
>>
>>> Hi,****
>>>
>>> I am trying to write a pig udf function.. Basically the data is of
>>> format*
>>> ***
>>>
>>> ** **
>>>
>>> Id,time****
>>>
>>> What I am trying to do is … parse the time and then see whether its
>>> breakfast, lunch or dinner.. based on the time stamp. Some entries wil be
>>> null as well..****
>>>
>>> ** **
>>>
>>> So here is the udf code for this.****
>>>
>>> ** **
>>>
>>> *public* *class* time *extends* EvalFunc<String>{****
>>>
>>> ** **
>>>
>>> *public* String exec(Tuple input) *throws* IOException {****
>>>
>>> ****
>>>
>>> *if* ((*input* == *null*) || (input.size() == 0))****
>>>
>>> *return* *null*;****
>>>
>>> *try*{****
>>>
>>> String time = (String) input.get(0) ;****
>>>
>>> DateFormat df = *new*
> SimpleDateFormat("hh:mm:ss.000");****
>>>
>>> Date date = df.parse(time);****
>>>
>>> String timeOfDay = *getTimeOfDay*(date);****
>>>
>>> *return* timeOfDay;****
>>>
>>> } *catch* (ParseException e) {****
>>>
>>> //how will I handle when df.parse(time) fails and throws
>>> ParseException?****
>>>
>>> //maybe:****
>>>
>>> *return* *null*;****
>>>
>>> }****
>>>
>>> ****
>>>
>>> ****
>>>
>>> }****
>>>
>>> ** **
>>>
>>> After this.. in eclipse.. I did the export of this as a jar called myudfs
>>> and I have a jar file called myudfs.jar****
>>>
>>> ** **
>>>
>>> Then I wrote the pig script as ****
>>>
>>> Time.pig****
>>>
>>> ** **
>>>
>>> REGISTER path/to/udf/myudfs.jar****
>>>
>>> in = LOAD 'path/to/input' USING PigStorage(',') AS (id:long,
>>> time:chararray);****
>>>
>>> result = foreach in GENERATE myudfs.time(time);****
>>>
>>> STORE result INTO 'path/to/output' using PigStorage(',');****
>>>
>>> ** **
>>>
>>> ** **
>>>
>>> So.. as per the documentation.. http://wiki.apache.org/pig/UDFManual****
>>>
>>> ** **
>>>
>>> Now I did this..****
>>>
>>> java -cp path/to/pig.jar org.apache.pig.Main time.pig****
>>>
>>> ** **
>>>
>>> Exception in thread "main" java.lang.NoClassDefFoundError:
>>> org/pache/pig/Main****
>>>
>>> Caused by: java.lang.ClassNotFoundException: org.pache.pig.Main****
>>>
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)****
>>>
>>> at java.security.AccessController.doPrivileged(Native Method)****
>>>
>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)****
>>>
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)****
>>>
>>> at
>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)****
>>>
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)****
>>>
>>> Could not find the main class: org.pache.pig.Main. Program will
> exit.****
>>>
>>> ** **
>>>
>>> ** **
>>>
>>> What am I doing wrong?****
>>>
>>> Thanks****
>>>
>>> ** **
>>>
>>> ** **
>>>
>>> ** **
>>>
>>> ** **
>>>
>>> ** **
>>>
>>> American Express made the following annotations on Thu Oct 25 2012
> 13:50:42
>>>
>>>
> ******************************************************************************
>>>
>>> "This message and any attachments are solely for the intended recipient
> and
>>> may contain confidential or privileged information. If you are not the
>>> intended recipient, any disclosure, copying, use, or distribution of the
>>> information included in this message and any attachments is prohibited.
> If
>>> you have received this communication in error, please notify us by reply
>>> e-mail and immediately and permanently delete this message and any
>>> attachments. Thank you."
>>> American Express a ajouté le commentaire suivant le Thu Oct 25 2012
>>> 13:50:42
>>> Ce courrier et toute pièce jointe qu'il contient sont réservés au seul
>>> destinataire indiqué et peuvent renfermer des renseignements
> confidentiels
>>> et privilégiés. Si vous n'êtes pas le destinataire prévu, toute
>>> divulgation, duplication, utilisation ou distribution du courrier ou de
>>> toute pièce jointe est interdite. Si vous avez reçu cette communication
> par
>>> erreur, veuillez nous en aviser par courrier et détruire immédiatement le
>>> courrier et les pièces jointes. Merci.
>>>
>>>
> ******************************************************************************
>>>
>>>
>>>
>>>
> -------------------------------------------------------------------------------
>>>
>>
>>
>>
>> --
>> *Note that I'm no longer using my Yahoo! email address. Please email me at
>> billgraham@gmail.com going forward.*
>>
Re: Better formated.. Pig udf help
Posted by jamal sasha <ja...@gmail.com>.
Hi
Great catch
Now I get an error
Cannot find hadoop configuration in class path ( neither hadoop site XML
etc)
So I am running the file on a cluster which had say hadoop set up as
/path/hadoop
/path/pig
And I have account in it
So I cannot change the hadoop conf files as other users are also using it.
How do I run this just for me ?
On Thursday, October 25, 2012, Bill Graham <bi...@gmail.com> wrote:
> Somewhere you have a typo, probably in the execution of your program:
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/pache/pig/Main****
>
> Caused by: java.lang.ClassNotFoundException: org.pache.pig.Main****
>
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)****
>
> Note that the 'a' in apache is missing.
>
>
> On Thu, Oct 25, 2012 at 2:46 PM, jamal sasha <ja...@gmail.com>
wrote:
>
>> Hi,****
>>
>> I am trying to write a pig udf function.. Basically the data is of
>> format*
>> ***
>>
>> ** **
>>
>> Id,time****
>>
>> What I am trying to do is … parse the time and then see whether its
>> breakfast, lunch or dinner.. based on the time stamp. Some entries wil be
>> null as well..****
>>
>> ** **
>>
>> So here is the udf code for this.****
>>
>> ** **
>>
>> *public* *class* time *extends* EvalFunc<String>{****
>>
>> ** **
>>
>> *public* String exec(Tuple input) *throws* IOException {****
>>
>> ****
>>
>> *if* ((*input* == *null*) || (input.size() == 0))****
>>
>> *return* *null*;****
>>
>> *try*{****
>>
>> String time = (String) input.get(0) ;****
>>
>> DateFormat df = *new*
SimpleDateFormat("hh:mm:ss.000");****
>>
>> Date date = df.parse(time);****
>>
>> String timeOfDay = *getTimeOfDay*(date);****
>>
>> *return* timeOfDay;****
>>
>> } *catch* (ParseException e) {****
>>
>> //how will I handle when df.parse(time) fails and throws
>> ParseException?****
>>
>> //maybe:****
>>
>> *return* *null*;****
>>
>> }****
>>
>> ****
>>
>> ****
>>
>> }****
>>
>> ** **
>>
>> After this.. in eclipse.. I did the export of this as a jar called myudfs
>> and I have a jar file called myudfs.jar****
>>
>> ** **
>>
>> Then I wrote the pig script as ****
>>
>> Time.pig****
>>
>> ** **
>>
>> REGISTER path/to/udf/myudfs.jar****
>>
>> in = LOAD 'path/to/input' USING PigStorage(',') AS (id:long,
>> time:chararray);****
>>
>> result = foreach in GENERATE myudfs.time(time);****
>>
>> STORE result INTO 'path/to/output' using PigStorage(',');****
>>
>> ** **
>>
>> ** **
>>
>> So.. as per the documentation.. http://wiki.apache.org/pig/UDFManual****
>>
>> ** **
>>
>> Now I did this..****
>>
>> java -cp path/to/pig.jar org.apache.pig.Main time.pig****
>>
>> ** **
>>
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/pache/pig/Main****
>>
>> Caused by: java.lang.ClassNotFoundException: org.pache.pig.Main****
>>
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)****
>>
>> at java.security.AccessController.doPrivileged(Native Method)****
>>
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)****
>>
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)****
>>
>> at
>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)****
>>
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)****
>>
>> Could not find the main class: org.pache.pig.Main. Program will
exit.****
>>
>> ** **
>>
>> ** **
>>
>> What am I doing wrong?****
>>
>> Thanks****
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> American Express made the following annotations on Thu Oct 25 2012
13:50:42
>>
>>
******************************************************************************
>>
>> "This message and any attachments are solely for the intended recipient
and
>> may contain confidential or privileged information. If you are not the
>> intended recipient, any disclosure, copying, use, or distribution of the
>> information included in this message and any attachments is prohibited.
If
>> you have received this communication in error, please notify us by reply
>> e-mail and immediately and permanently delete this message and any
>> attachments. Thank you."
>> American Express a ajouté le commentaire suivant le Thu Oct 25 2012
>> 13:50:42
>> Ce courrier et toute pièce jointe qu'il contient sont réservés au seul
>> destinataire indiqué et peuvent renfermer des renseignements
confidentiels
>> et privilégiés. Si vous n'êtes pas le destinataire prévu, toute
>> divulgation, duplication, utilisation ou distribution du courrier ou de
>> toute pièce jointe est interdite. Si vous avez reçu cette communication
par
>> erreur, veuillez nous en aviser par courrier et détruire immédiatement le
>> courrier et les pièces jointes. Merci.
>>
>>
******************************************************************************
>>
>>
>>
>>
-------------------------------------------------------------------------------
>>
>
>
>
> --
> *Note that I'm no longer using my Yahoo! email address. Please email me at
> billgraham@gmail.com going forward.*
>
Re: Better formated.. Pig udf help
Posted by Bill Graham <bi...@gmail.com>.
Somewhere you have a typo, probably in the execution of your program:
Exception in thread "main" java.lang.NoClassDefFoundError:
org/pache/pig/Main****
Caused by: java.lang.ClassNotFoundException: org.pache.pig.Main****
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)****
Note that the 'a' in apache is missing.
On Thu, Oct 25, 2012 at 2:46 PM, jamal sasha <ja...@gmail.com> wrote:
> Hi,****
>
> I am trying to write a pig udf function.. Basically the data is of
> format*
> ***
>
> ** **
>
> Id,time****
>
> What I am trying to do is … parse the time and then see whether its
> breakfast, lunch or dinner.. based on the time stamp. Some entries wil be
> null as well..****
>
> ** **
>
> So here is the udf code for this.****
>
> ** **
>
> *public* *class* time *extends* EvalFunc<String>{****
>
> ** **
>
> *public* String exec(Tuple input) *throws* IOException {****
>
> ****
>
> *if* ((*input* == *null*) || (input.size() == 0))****
>
> *return* *null*;****
>
> *try*{****
>
> String time = (String) input.get(0) ;****
>
> DateFormat df = *new* SimpleDateFormat("hh:mm:ss.000");****
>
> Date date = df.parse(time);****
>
> String timeOfDay = *getTimeOfDay*(date);****
>
> *return* timeOfDay;****
>
> } *catch* (ParseException e) {****
>
> //how will I handle when df.parse(time) fails and throws
> ParseException?****
>
> //maybe:****
>
> *return* *null*;****
>
> }****
>
> ****
>
> ****
>
> }****
>
> ** **
>
> After this.. in eclipse.. I did the export of this as a jar called myudfs
> and I have a jar file called myudfs.jar****
>
> ** **
>
> Then I wrote the pig script as ****
>
> Time.pig****
>
> ** **
>
> REGISTER path/to/udf/myudfs.jar****
>
> in = LOAD 'path/to/input' USING PigStorage(',') AS (id:long,
> time:chararray);****
>
> result = foreach in GENERATE myudfs.time(time);****
>
> STORE result INTO 'path/to/output' using PigStorage(',');****
>
> ** **
>
> ** **
>
> So.. as per the documentation.. http://wiki.apache.org/pig/UDFManual****
>
> ** **
>
> Now I did this..****
>
> java -cp path/to/pig.jar org.apache.pig.Main time.pig****
>
> ** **
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/pache/pig/Main****
>
> Caused by: java.lang.ClassNotFoundException: org.pache.pig.Main****
>
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)****
>
> at java.security.AccessController.doPrivileged(Native Method)****
>
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)****
>
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)****
>
> at
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)****
>
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)****
>
> Could not find the main class: org.pache.pig.Main. Program will exit.****
>
> ** **
>
> ** **
>
> What am I doing wrong?****
>
> Thanks****
>
> ** **
>
> ** **
>
> ** **
>
> ** **
>
> ** **
>
> American Express made the following annotations on Thu Oct 25 2012 13:50:42
>
> ******************************************************************************
>
> "This message and any attachments are solely for the intended recipient and
> may contain confidential or privileged information. If you are not the
> intended recipient, any disclosure, copying, use, or distribution of the
> information included in this message and any attachments is prohibited. If
> you have received this communication in error, please notify us by reply
> e-mail and immediately and permanently delete this message and any
> attachments. Thank you."
> American Express a ajouté le commentaire suivant le Thu Oct 25 2012
> 13:50:42
> Ce courrier et toute pièce jointe qu'il contient sont réservés au seul
> destinataire indiqué et peuvent renfermer des renseignements confidentiels
> et privilégiés. Si vous n'êtes pas le destinataire prévu, toute
> divulgation, duplication, utilisation ou distribution du courrier ou de
> toute pièce jointe est interdite. Si vous avez reçu cette communication par
> erreur, veuillez nous en aviser par courrier et détruire immédiatement le
> courrier et les pièces jointes. Merci.
>
> ******************************************************************************
>
>
>
> -------------------------------------------------------------------------------
>
--
*Note that I'm no longer using my Yahoo! email address. Please email me at
billgraham@gmail.com going forward.*
Re: Better formated.. Pig udf help
Posted by jamal sasha <ja...@gmail.com>.
Hi
In this case I get an error
Problem resolving class version numbers for class myudfs.time??
On Thursday, October 25, 2012, pablomar <pa...@gmail.com>
wrote:
> to run your script you have to do
> pig -f time.pig
>
>
>
> On Thu, Oct 25, 2012 at 5:46 PM, jamal sasha <ja...@gmail.com>
wrote:
>
>> Hi,****
>>
>> I am trying to write a pig udf function.. Basically the data is of
>> format*
>> ***
>>
>> ** **
>>
>> Id,time****
>>
>> What I am trying to do is … parse the time and then see whether its
>> breakfast, lunch or dinner.. based on the time stamp. Some entries wil be
>> null as well..****
>>
>> ** **
>>
>> So here is the udf code for this.****
>>
>> ** **
>>
>> *public* *class* time *extends* EvalFunc<String>{****
>>
>> ** **
>>
>> *public* String exec(Tuple input) *throws* IOException {****
>>
>> ****
>>
>> *if* ((*input* == *null*) || (input.size() == 0))****
>>
>> *return* *null*;****
>>
>> *try*{****
>>
>> String time = (String) input.get(0) ;****
>>
>> DateFormat df = *new*
SimpleDateFormat("hh:mm:ss.000");****
>>
>> Date date = df.parse(time);****
>>
>> String timeOfDay = *getTimeOfDay*(date);****
>>
>> *return* timeOfDay;****
>>
>> } *catch* (ParseException e) {****
>>
>> //how will I handle when df.parse(time) fails and throws
>> ParseException?****
>>
>> //maybe:****
>>
>> *return* *null*;****
>>
>> }****
>>
>> ****
>>
>> ****
>>
>> }****
>>
>> ** **
>>
>> After this.. in eclipse.. I did the export of this as a jar called myudfs
>> and I have a jar file called myudfs.jar****
>>
>> ** **
>>
>> Then I wrote the pig script as ****
>>
>> Time.pig****
>>
>> ** **
>>
>> REGISTER path/to/udf/myudfs.jar****
>>
>> in = LOAD 'path/to/input' USING PigStorage(',') AS (id:long,
>> time:chararray);****
>>
>> result = foreach in GENERATE myudfs.time(time);****
>>
>> STORE result INTO 'path/to/output' using PigStorage(',');****
>>
>> ** **
>>
>> ** **
>>
>> So.. as per the documentation.. http://wiki.apache.org/pig/UDFManual****
>>
>> ** **
>>
>> Now I did this..****
>>
>> java -cp path/to/pig.jar org.apache.pig.Main time.pig****
>>
>> ** **
>>
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/pache/pig/Main****
>>
>> Caused by: java.lang.ClassNotFoundException: org.pache.pig.Main****
>>
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)****
>>
>> at java.security.AccessController.doPrivileged(Native Method)****
>>
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)****
>>
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)****
>>
>> at
>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)****
>>
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)****
>>
>> Could not find the main class: org.pache.pig.Main. Program will
exit.****
>>
>> ** **
>>
>> ** **
>>
>> What am I doing wrong?****
>>
>> Thanks****
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>> American Express made the following annotations on Thu Oct 25 2012
13:50:42
>>
>>
******************************************************************************
>>
>> "This message and any attachments are solely for the intended recipient
and
>> may contain confidential or privileged information. If you are not the
>> intended recipient, any disclosure, copying, use, or distribution of the
>> information included in this message and any attachments is prohibited.
If
>> you have received this communication in error, please notify us by reply
>> e-mail and immediately and permanently delete this message and any
>> attachments. Thank you."
>> American Express a ajouté le commentaire suivant le Thu Oct 25 2012
>> 13:50:42
>> Ce courrier et toute pièce jointe qu'il contient sont réservés au seul
>> destinataire indiqué et peuvent renfermer des renseignements
confidentiels
>> et privilégiés. Si vous n'êtes pas le destinataire prévu, toute
>> divulgation, duplication, utilisation ou distribution du courrier ou de
>> toute pièce jointe est interdite. Si vous avez reçu cette communication
par
>> erreur, veuillez nous en aviser par courrier et détruire immédiatement le
>> courrier et les pièces jointes. Merci.
>>
>>
******************************************************************************
>>
>>
>>
>>
-------------------------------------------------------------------------------
>>
>
Re: Better formated.. Pig udf help
Posted by pablomar <pa...@gmail.com>.
to run your script you have to do
pig -f time.pig
On Thu, Oct 25, 2012 at 5:46 PM, jamal sasha <ja...@gmail.com> wrote:
> Hi,****
>
> I am trying to write a pig udf function.. Basically the data is of
> format*
> ***
>
> ** **
>
> Id,time****
>
> What I am trying to do is … parse the time and then see whether its
> breakfast, lunch or dinner.. based on the time stamp. Some entries wil be
> null as well..****
>
> ** **
>
> So here is the udf code for this.****
>
> ** **
>
> *public* *class* time *extends* EvalFunc<String>{****
>
> ** **
>
> *public* String exec(Tuple input) *throws* IOException {****
>
> ****
>
> *if* ((*input* == *null*) || (input.size() == 0))****
>
> *return* *null*;****
>
> *try*{****
>
> String time = (String) input.get(0) ;****
>
> DateFormat df = *new* SimpleDateFormat("hh:mm:ss.000");****
>
> Date date = df.parse(time);****
>
> String timeOfDay = *getTimeOfDay*(date);****
>
> *return* timeOfDay;****
>
> } *catch* (ParseException e) {****
>
> //how will I handle when df.parse(time) fails and throws
> ParseException?****
>
> //maybe:****
>
> *return* *null*;****
>
> }****
>
> ****
>
> ****
>
> }****
>
> ** **
>
> After this.. in eclipse.. I did the export of this as a jar called myudfs
> and I have a jar file called myudfs.jar****
>
> ** **
>
> Then I wrote the pig script as ****
>
> Time.pig****
>
> ** **
>
> REGISTER path/to/udf/myudfs.jar****
>
> in = LOAD 'path/to/input' USING PigStorage(',') AS (id:long,
> time:chararray);****
>
> result = foreach in GENERATE myudfs.time(time);****
>
> STORE result INTO 'path/to/output' using PigStorage(',');****
>
> ** **
>
> ** **
>
> So.. as per the documentation.. http://wiki.apache.org/pig/UDFManual****
>
> ** **
>
> Now I did this..****
>
> java -cp path/to/pig.jar org.apache.pig.Main time.pig****
>
> ** **
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/pache/pig/Main****
>
> Caused by: java.lang.ClassNotFoundException: org.pache.pig.Main****
>
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)****
>
> at java.security.AccessController.doPrivileged(Native Method)****
>
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)****
>
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)****
>
> at
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)****
>
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)****
>
> Could not find the main class: org.pache.pig.Main. Program will exit.****
>
> ** **
>
> ** **
>
> What am I doing wrong?****
>
> Thanks****
>
> ** **
>
> ** **
>
> ** **
>
> ** **
>
> ** **
>
> American Express made the following annotations on Thu Oct 25 2012 13:50:42
>
> ******************************************************************************
>
> "This message and any attachments are solely for the intended recipient and
> may contain confidential or privileged information. If you are not the
> intended recipient, any disclosure, copying, use, or distribution of the
> information included in this message and any attachments is prohibited. If
> you have received this communication in error, please notify us by reply
> e-mail and immediately and permanently delete this message and any
> attachments. Thank you."
> American Express a ajouté le commentaire suivant le Thu Oct 25 2012
> 13:50:42
> Ce courrier et toute pièce jointe qu'il contient sont réservés au seul
> destinataire indiqué et peuvent renfermer des renseignements confidentiels
> et privilégiés. Si vous n'êtes pas le destinataire prévu, toute
> divulgation, duplication, utilisation ou distribution du courrier ou de
> toute pièce jointe est interdite. Si vous avez reçu cette communication par
> erreur, veuillez nous en aviser par courrier et détruire immédiatement le
> courrier et les pièces jointes. Merci.
>
> ******************************************************************************
>
>
>
> -------------------------------------------------------------------------------
>