You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Jay Vyas <ja...@gmail.com> on 2014/02/16 15:16:23 UTC
Running a Pig Jar (from hadoop)
Hi pig:
What is the common idiom for executing a Java application which runs pig
commands using the direct Java API (i.e. by creatiing a PigServer object,
etc...) ? There are a few ways i can think of:
1) using "hadoop jar" , but this will of course fail since pig is not in
the hadoop classpath.
2) using a "pig ..." command
3) Adding pig jars to the distributed cache at runtime in a standard java
hadoop program.
Not sure what is the "right" way to do it though?
Thanks !
--
Jay Vyas
http://jayunit100.blogspot.com
Re: Running a Pig Jar (from hadoop)
Posted by Shahab Yunus <sh...@gmail.com>.
I now it is does not directly answer your question but curious, have you
looked at PigRunner?
Regards,
Shahab
On Sun, Feb 16, 2014 at 9:16 AM, Jay Vyas <ja...@gmail.com> wrote:
> Hi pig:
>
> What is the common idiom for executing a Java application which runs pig
> commands using the direct Java API (i.e. by creatiing a PigServer object,
> etc...) ? There are a few ways i can think of:
>
> 1) using "hadoop jar" , but this will of course fail since pig is not in
> the hadoop classpath.
>
> 2) using a "pig ..." command
>
> 3) Adding pig jars to the distributed cache at runtime in a standard java
> hadoop program.
>
> Not sure what is the "right" way to do it though?
>
> Thanks !
> --
> Jay Vyas
> http://jayunit100.blogspot.com
>
Re: Running a Pig Jar (from hadoop)
Posted by choi jongwon <jo...@gmail.com>.
I think, PigServer is good to run.
But, the script code place in java code, and I want to run dynamic pig
script.
So, I use template engine like freemarker.
Make pig script using template engine, and read it using StringReader.
and modify the PigServer a little.
2014-02-17 6:10 GMT+09:00 Prashant Kommireddi <pr...@gmail.com>:
> This might help
> https://wiki.apache.org/pig/EmbeddedPig
>
> On Sunday, February 16, 2014, Jay Vyas <ja...@gmail.com> wrote:
>
> > Yeah that's what I think is the simplest: just exporting
> >
> > Hadoop_classpath=pig.jar:$hadoop_classpath
> >
> > Hadoop jar mymainclass arg1 arg2
> > > On Feb 16, 2014, at 3:04 PM, Prashant Kommireddi <prash1784@gmail.com
> <javascript:;>>
> > wrote:
> > >
> > > You could run it as a regular java application (with a main method)?
> You
> > > need to have pig and Hadoop on the classpath.
> > >
> > >> On Sunday, February 16, 2014, Jay Vyas <jayunit100@gmail.com
> <javascript:;>>
> > wrote:
> > >>
> > >> Hi pig:
> > >>
> > >> What is the common idiom for executing a Java application which runs
> pig
> > >> commands using the direct Java API (i.e. by creatiing a PigServer
> > object,
> > >> etc...) ? There are a few ways i can think of:
> > >>
> > >> 1) using "hadoop jar" , but this will of course fail since pig is not
> in
> > >> the hadoop classpath.
> > >>
> > >> 2) using a "pig ..." command
> > >>
> > >> 3) Adding pig jars to the distributed cache at runtime in a standard
> > java
> > >> hadoop program.
> > >>
> > >> Not sure what is the "right" way to do it though?
> > >>
> > >> Thanks !
> > >> --
> > >> Jay Vyas
> > >> http://jayunit100.blogspot.com
> > >>
> >
>
Re: Running a Pig Jar (from hadoop)
Posted by Prashant Kommireddi <pr...@gmail.com>.
This might help
https://wiki.apache.org/pig/EmbeddedPig
On Sunday, February 16, 2014, Jay Vyas <ja...@gmail.com> wrote:
> Yeah that's what I think is the simplest: just exporting
>
> Hadoop_classpath=pig.jar:$hadoop_classpath
>
> Hadoop jar mymainclass arg1 arg2
> > On Feb 16, 2014, at 3:04 PM, Prashant Kommireddi <prash1784@gmail.com<javascript:;>>
> wrote:
> >
> > You could run it as a regular java application (with a main method)? You
> > need to have pig and Hadoop on the classpath.
> >
> >> On Sunday, February 16, 2014, Jay Vyas <jayunit100@gmail.com<javascript:;>>
> wrote:
> >>
> >> Hi pig:
> >>
> >> What is the common idiom for executing a Java application which runs pig
> >> commands using the direct Java API (i.e. by creatiing a PigServer
> object,
> >> etc...) ? There are a few ways i can think of:
> >>
> >> 1) using "hadoop jar" , but this will of course fail since pig is not in
> >> the hadoop classpath.
> >>
> >> 2) using a "pig ..." command
> >>
> >> 3) Adding pig jars to the distributed cache at runtime in a standard
> java
> >> hadoop program.
> >>
> >> Not sure what is the "right" way to do it though?
> >>
> >> Thanks !
> >> --
> >> Jay Vyas
> >> http://jayunit100.blogspot.com
> >>
>
Re: Running a Pig Jar (from hadoop)
Posted by Jay Vyas <ja...@gmail.com>.
Yeah that's what I think is the simplest: just exporting
Hadoop_classpath=pig.jar:$hadoop_classpath
Hadoop jar mymainclass arg1 arg2
> On Feb 16, 2014, at 3:04 PM, Prashant Kommireddi <pr...@gmail.com> wrote:
>
> You could run it as a regular java application (with a main method)? You
> need to have pig and Hadoop on the classpath.
>
>> On Sunday, February 16, 2014, Jay Vyas <ja...@gmail.com> wrote:
>>
>> Hi pig:
>>
>> What is the common idiom for executing a Java application which runs pig
>> commands using the direct Java API (i.e. by creatiing a PigServer object,
>> etc...) ? There are a few ways i can think of:
>>
>> 1) using "hadoop jar" , but this will of course fail since pig is not in
>> the hadoop classpath.
>>
>> 2) using a "pig ..." command
>>
>> 3) Adding pig jars to the distributed cache at runtime in a standard java
>> hadoop program.
>>
>> Not sure what is the "right" way to do it though?
>>
>> Thanks !
>> --
>> Jay Vyas
>> http://jayunit100.blogspot.com
>>
Re: Running a Pig Jar (from hadoop)
Posted by Prashant Kommireddi <pr...@gmail.com>.
You could run it as a regular java application (with a main method)? You
need to have pig and Hadoop on the classpath.
On Sunday, February 16, 2014, Jay Vyas <ja...@gmail.com> wrote:
> Hi pig:
>
> What is the common idiom for executing a Java application which runs pig
> commands using the direct Java API (i.e. by creatiing a PigServer object,
> etc...) ? There are a few ways i can think of:
>
> 1) using "hadoop jar" , but this will of course fail since pig is not in
> the hadoop classpath.
>
> 2) using a "pig ..." command
>
> 3) Adding pig jars to the distributed cache at runtime in a standard java
> hadoop program.
>
> Not sure what is the "right" way to do it though?
>
> Thanks !
> --
> Jay Vyas
> http://jayunit100.blogspot.com
>