You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Siddhi Borkar <si...@persistent.co.in> on 2013/04/23 08:50:39 UTC
Invoking pig scripts from java
Hi ,
I need a way to invoke pig script from a java program and capture the output returned by the pig script. I was looking at the PigRunner api, however did not get much examples.
Is there any way how this can be achieved?Any working examples?
Thanks,
Sid
DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.
RE: Invoking pig scripts from java
Posted by Siddhi Borkar <si...@persistent.co.in>.
My java program is as follows
PigServer pigServer = new PigServer(ExecType.LOCAL);
pigServer.registerScript("processor.pig");
I need to capture the output (the stack trace ) generated by the pig script and save it in a file. Is there any way to get this done?
-----Original Message-----
From: Prashant Kommireddi [mailto:prash1784@gmail.com]
Sent: Wednesday, April 24, 2013 12:07 AM
To: user@pig.apache.org
Subject: Re: Invoking pig scripts from java
How do you plan on running the java process? You can redirect the output to a file?
On Tue, Apr 23, 2013 at 3:15 AM, Siddhi Borkar < siddhi_borkar@persistent.co.in> wrote:
> Thanks a lot Prashant.
> Is it possible to redirect the output generated by the pig script to a
> file? I need to do this in the java code and not in the pig scripts. I
> have some customer provided pig scripts which I am not allowed to
> modify, Hence I need to do this in java. Is there any way of doing this?
>
>
> -----Original Message-----
> From: Prashant Kommireddi [mailto:prash1784@gmail.com]
> Sent: Tuesday, April 23, 2013 2:03 PM
> To: user@pig.apache.org
> Subject: Re: Invoking pig scripts from java
>
> Both are used to run pig scripts. PigRunner uses a more direct way in
> using the exact grunt code path to execute a script.
>
> PigServer on the other hand provides a bunch of helper methods that
> could be useful (for eg, setting a job name, deleting/renaming files,
> job priority, running individual queries etc). So you don't have to
> set them as properties within your script.
>
> You would use PigRunner exactly the same way as running pig via CLI
> passing the args to suggest pig script name etc, additionally it takes
> in PigProgressNotificationListener as an argument if one is interested
> in providing custom implementations of it.
>
>
> On Tue, Apr 23, 2013 at 1:17 AM, Siddhi Borkar <
> siddhi_borkar@persistent.co.in> wrote:
>
> > Any idea what is the difference between Pigrunner and pigserver?
> >
> > -----Original Message-----
> > From: Prashant Kommireddi [mailto:prash1784@gmail.com]
> > Sent: Tuesday, April 23, 2013 12:28 PM
> > To: user@pig.apache.org
> > Subject: Re: Invoking pig scripts from java
> >
> > Have you looked at the PigServer API? Running scripts should be
> > pretty straightforward using it. Let us know if you face any issues.
> >
> >
> > http://pig.apache.org/docs/r0.11.1/api/org/apache/pig/PigServer.html
> > #r
> > egisterScript%28java.lang.String%29
> >
> >
> > On Mon, Apr 22, 2013 at 11:50 PM, Siddhi Borkar <
> > siddhi_borkar@persistent.co.in> wrote:
> >
> > > Hi ,
> > >
> > > I need a way to invoke pig script from a java program and capture
> > > the output returned by the pig script. I was looking at the
> > > PigRunner api, however did not get much examples.
> > > Is there any way how this can be achieved?Any working examples?
> > >
> > > Thanks,
> > > Sid
> > >
> > > DISCLAIMER
> > > ==========
> > > This e-mail may contain privileged and confidential information
> > > which is the property of Persistent Systems Ltd. It is intended
> > > only for the use of the individual or entity to which it is
> > > addressed. If you are not the intended recipient, you are not
> > > authorized to read, retain, copy, print, distribute or use this
> > > message. If you have received this communication in error, please
> > > notify the sender and delete all copies
> > of this message.
> > > Persistent Systems Ltd. does not accept any liability for virus
> > > infected mails.
> > >
> >
> > DISCLAIMER
> > ==========
> > This e-mail may contain privileged and confidential information
> > which is the property of Persistent Systems Ltd. It is intended only
> > for the use of the individual or entity to which it is addressed. If
> > you are not the intended recipient, you are not authorized to read,
> > retain, copy, print, distribute or use this message. If you have
> > received this communication in error, please notify the sender and
> > delete all copies
> of this message.
> > Persistent Systems Ltd. does not accept any liability for virus
> > infected mails.
> >
>
> DISCLAIMER
> ==========
> This e-mail may contain privileged and confidential information which
> is the property of Persistent Systems Ltd. It is intended only for the
> use of the individual or entity to which it is addressed. If you are
> not the intended recipient, you are not authorized to read, retain,
> copy, print, distribute or use this message. If you have received this
> communication in error, please notify the sender and delete all copies of this message.
> Persistent Systems Ltd. does not accept any liability for virus
> infected mails.
>
DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.
Re: Invoking pig scripts from java
Posted by Prashant Kommireddi <pr...@gmail.com>.
How do you plan on running the java process? You can redirect the output to
a file?
On Tue, Apr 23, 2013 at 3:15 AM, Siddhi Borkar <
siddhi_borkar@persistent.co.in> wrote:
> Thanks a lot Prashant.
> Is it possible to redirect the output generated by the pig script to a
> file? I need to do this in the java code and not in the pig scripts. I have
> some customer provided pig scripts which I am not allowed to modify,
> Hence I need to do this in java. Is there any way of doing this?
>
>
> -----Original Message-----
> From: Prashant Kommireddi [mailto:prash1784@gmail.com]
> Sent: Tuesday, April 23, 2013 2:03 PM
> To: user@pig.apache.org
> Subject: Re: Invoking pig scripts from java
>
> Both are used to run pig scripts. PigRunner uses a more direct way in
> using the exact grunt code path to execute a script.
>
> PigServer on the other hand provides a bunch of helper methods that could
> be useful (for eg, setting a job name, deleting/renaming files, job
> priority, running individual queries etc). So you don't have to set them as
> properties within your script.
>
> You would use PigRunner exactly the same way as running pig via CLI
> passing the args to suggest pig script name etc, additionally it takes in
> PigProgressNotificationListener as an argument if one is interested in
> providing custom implementations of it.
>
>
> On Tue, Apr 23, 2013 at 1:17 AM, Siddhi Borkar <
> siddhi_borkar@persistent.co.in> wrote:
>
> > Any idea what is the difference between Pigrunner and pigserver?
> >
> > -----Original Message-----
> > From: Prashant Kommireddi [mailto:prash1784@gmail.com]
> > Sent: Tuesday, April 23, 2013 12:28 PM
> > To: user@pig.apache.org
> > Subject: Re: Invoking pig scripts from java
> >
> > Have you looked at the PigServer API? Running scripts should be pretty
> > straightforward using it. Let us know if you face any issues.
> >
> >
> > http://pig.apache.org/docs/r0.11.1/api/org/apache/pig/PigServer.html#r
> > egisterScript%28java.lang.String%29
> >
> >
> > On Mon, Apr 22, 2013 at 11:50 PM, Siddhi Borkar <
> > siddhi_borkar@persistent.co.in> wrote:
> >
> > > Hi ,
> > >
> > > I need a way to invoke pig script from a java program and capture
> > > the output returned by the pig script. I was looking at the
> > > PigRunner api, however did not get much examples.
> > > Is there any way how this can be achieved?Any working examples?
> > >
> > > Thanks,
> > > Sid
> > >
> > > DISCLAIMER
> > > ==========
> > > This e-mail may contain privileged and confidential information
> > > which is the property of Persistent Systems Ltd. It is intended only
> > > for the use of the individual or entity to which it is addressed. If
> > > you are not the intended recipient, you are not authorized to read,
> > > retain, copy, print, distribute or use this message. If you have
> > > received this communication in error, please notify the sender and
> > > delete all copies
> > of this message.
> > > Persistent Systems Ltd. does not accept any liability for virus
> > > infected mails.
> > >
> >
> > DISCLAIMER
> > ==========
> > This e-mail may contain privileged and confidential information which
> > is the property of Persistent Systems Ltd. It is intended only for the
> > use of the individual or entity to which it is addressed. If you are
> > not the intended recipient, you are not authorized to read, retain,
> > copy, print, distribute or use this message. If you have received this
> > communication in error, please notify the sender and delete all copies
> of this message.
> > Persistent Systems Ltd. does not accept any liability for virus
> > infected mails.
> >
>
> DISCLAIMER
> ==========
> This e-mail may contain privileged and confidential information which is
> the property of Persistent Systems Ltd. It is intended only for the use of
> the individual or entity to which it is addressed. If you are not the
> intended recipient, you are not authorized to read, retain, copy, print,
> distribute or use this message. If you have received this communication in
> error, please notify the sender and delete all copies of this message.
> Persistent Systems Ltd. does not accept any liability for virus infected
> mails.
>
RE: Invoking pig scripts from java
Posted by Siddhi Borkar <si...@persistent.co.in>.
Thanks a lot Prashant.
Is it possible to redirect the output generated by the pig script to a file? I need to do this in the java code and not in the pig scripts. I have some customer provided pig scripts which I am not allowed to modify,
Hence I need to do this in java. Is there any way of doing this?
-----Original Message-----
From: Prashant Kommireddi [mailto:prash1784@gmail.com]
Sent: Tuesday, April 23, 2013 2:03 PM
To: user@pig.apache.org
Subject: Re: Invoking pig scripts from java
Both are used to run pig scripts. PigRunner uses a more direct way in using the exact grunt code path to execute a script.
PigServer on the other hand provides a bunch of helper methods that could be useful (for eg, setting a job name, deleting/renaming files, job priority, running individual queries etc). So you don't have to set them as properties within your script.
You would use PigRunner exactly the same way as running pig via CLI passing the args to suggest pig script name etc, additionally it takes in PigProgressNotificationListener as an argument if one is interested in providing custom implementations of it.
On Tue, Apr 23, 2013 at 1:17 AM, Siddhi Borkar < siddhi_borkar@persistent.co.in> wrote:
> Any idea what is the difference between Pigrunner and pigserver?
>
> -----Original Message-----
> From: Prashant Kommireddi [mailto:prash1784@gmail.com]
> Sent: Tuesday, April 23, 2013 12:28 PM
> To: user@pig.apache.org
> Subject: Re: Invoking pig scripts from java
>
> Have you looked at the PigServer API? Running scripts should be pretty
> straightforward using it. Let us know if you face any issues.
>
>
> http://pig.apache.org/docs/r0.11.1/api/org/apache/pig/PigServer.html#r
> egisterScript%28java.lang.String%29
>
>
> On Mon, Apr 22, 2013 at 11:50 PM, Siddhi Borkar <
> siddhi_borkar@persistent.co.in> wrote:
>
> > Hi ,
> >
> > I need a way to invoke pig script from a java program and capture
> > the output returned by the pig script. I was looking at the
> > PigRunner api, however did not get much examples.
> > Is there any way how this can be achieved?Any working examples?
> >
> > Thanks,
> > Sid
> >
> > DISCLAIMER
> > ==========
> > This e-mail may contain privileged and confidential information
> > which is the property of Persistent Systems Ltd. It is intended only
> > for the use of the individual or entity to which it is addressed. If
> > you are not the intended recipient, you are not authorized to read,
> > retain, copy, print, distribute or use this message. If you have
> > received this communication in error, please notify the sender and
> > delete all copies
> of this message.
> > Persistent Systems Ltd. does not accept any liability for virus
> > infected mails.
> >
>
> DISCLAIMER
> ==========
> This e-mail may contain privileged and confidential information which
> is the property of Persistent Systems Ltd. It is intended only for the
> use of the individual or entity to which it is addressed. If you are
> not the intended recipient, you are not authorized to read, retain,
> copy, print, distribute or use this message. If you have received this
> communication in error, please notify the sender and delete all copies of this message.
> Persistent Systems Ltd. does not accept any liability for virus
> infected mails.
>
DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.
Re: Invoking pig scripts from java
Posted by Prashant Kommireddi <pr...@gmail.com>.
Both are used to run pig scripts. PigRunner uses a more direct way in using
the exact grunt code path to execute a script.
PigServer on the other hand provides a bunch of helper methods that could
be useful (for eg, setting a job name, deleting/renaming files, job
priority, running individual queries etc). So you don't have to set them as
properties within your script.
You would use PigRunner exactly the same way as running pig via CLI passing
the args to suggest pig script name etc, additionally it takes in
PigProgressNotificationListener as an argument if one is interested in
providing custom implementations of it.
On Tue, Apr 23, 2013 at 1:17 AM, Siddhi Borkar <
siddhi_borkar@persistent.co.in> wrote:
> Any idea what is the difference between Pigrunner and pigserver?
>
> -----Original Message-----
> From: Prashant Kommireddi [mailto:prash1784@gmail.com]
> Sent: Tuesday, April 23, 2013 12:28 PM
> To: user@pig.apache.org
> Subject: Re: Invoking pig scripts from java
>
> Have you looked at the PigServer API? Running scripts should be pretty
> straightforward using it. Let us know if you face any issues.
>
>
> http://pig.apache.org/docs/r0.11.1/api/org/apache/pig/PigServer.html#registerScript%28java.lang.String%29
>
>
> On Mon, Apr 22, 2013 at 11:50 PM, Siddhi Borkar <
> siddhi_borkar@persistent.co.in> wrote:
>
> > Hi ,
> >
> > I need a way to invoke pig script from a java program and capture the
> > output returned by the pig script. I was looking at the PigRunner api,
> > however did not get much examples.
> > Is there any way how this can be achieved?Any working examples?
> >
> > Thanks,
> > Sid
> >
> > DISCLAIMER
> > ==========
> > This e-mail may contain privileged and confidential information which
> > is the property of Persistent Systems Ltd. It is intended only for the
> > use of the individual or entity to which it is addressed. If you are
> > not the intended recipient, you are not authorized to read, retain,
> > copy, print, distribute or use this message. If you have received this
> > communication in error, please notify the sender and delete all copies
> of this message.
> > Persistent Systems Ltd. does not accept any liability for virus
> > infected mails.
> >
>
> DISCLAIMER
> ==========
> This e-mail may contain privileged and confidential information which is
> the property of Persistent Systems Ltd. It is intended only for the use of
> the individual or entity to which it is addressed. If you are not the
> intended recipient, you are not authorized to read, retain, copy, print,
> distribute or use this message. If you have received this communication in
> error, please notify the sender and delete all copies of this message.
> Persistent Systems Ltd. does not accept any liability for virus infected
> mails.
>
RE: Invoking pig scripts from java
Posted by Siddhi Borkar <si...@persistent.co.in>.
Any idea what is the difference between Pigrunner and pigserver?
-----Original Message-----
From: Prashant Kommireddi [mailto:prash1784@gmail.com]
Sent: Tuesday, April 23, 2013 12:28 PM
To: user@pig.apache.org
Subject: Re: Invoking pig scripts from java
Have you looked at the PigServer API? Running scripts should be pretty straightforward using it. Let us know if you face any issues.
http://pig.apache.org/docs/r0.11.1/api/org/apache/pig/PigServer.html#registerScript%28java.lang.String%29
On Mon, Apr 22, 2013 at 11:50 PM, Siddhi Borkar < siddhi_borkar@persistent.co.in> wrote:
> Hi ,
>
> I need a way to invoke pig script from a java program and capture the
> output returned by the pig script. I was looking at the PigRunner api,
> however did not get much examples.
> Is there any way how this can be achieved?Any working examples?
>
> Thanks,
> Sid
>
> DISCLAIMER
> ==========
> This e-mail may contain privileged and confidential information which
> is the property of Persistent Systems Ltd. It is intended only for the
> use of the individual or entity to which it is addressed. If you are
> not the intended recipient, you are not authorized to read, retain,
> copy, print, distribute or use this message. If you have received this
> communication in error, please notify the sender and delete all copies of this message.
> Persistent Systems Ltd. does not accept any liability for virus
> infected mails.
>
DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.
Re: Invoking pig scripts from java
Posted by Prashant Kommireddi <pr...@gmail.com>.
Have you looked at the PigServer API? Running scripts should be pretty
straightforward using it. Let us know if you face any issues.
http://pig.apache.org/docs/r0.11.1/api/org/apache/pig/PigServer.html#registerScript%28java.lang.String%29
On Mon, Apr 22, 2013 at 11:50 PM, Siddhi Borkar <
siddhi_borkar@persistent.co.in> wrote:
> Hi ,
>
> I need a way to invoke pig script from a java program and capture the
> output returned by the pig script. I was looking at the PigRunner api,
> however did not get much examples.
> Is there any way how this can be achieved?Any working examples?
>
> Thanks,
> Sid
>
> DISCLAIMER
> ==========
> This e-mail may contain privileged and confidential information which is
> the property of Persistent Systems Ltd. It is intended only for the use of
> the individual or entity to which it is addressed. If you are not the
> intended recipient, you are not authorized to read, retain, copy, print,
> distribute or use this message. If you have received this communication in
> error, please notify the sender and delete all copies of this message.
> Persistent Systems Ltd. does not accept any liability for virus infected
> mails.
>