You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Andreas Paepcke <pa...@cs.stanford.edu> on 2011/03/15 06:40:30 UTC
DUMP or STORE Depending on Parameter Input
I want to do something really simple: I want to pass a string
into a Pig script. The string is either "stdout" or some target
file name. Say the string gets bound to $OUTPUT. That all
works fine.
After the script has computed a result R, depending on the
value of $OUTPUT, I want either to do a dump R (if
$OUTPUT == "stdout"), or a STORE of R into the given
file name that's held in $OUTPUT.
How do I do that conditional? I played with bincond, but
got stuck.
Thanks!
Andreas
Andreas
Re: DUMP or STORE Depending on Parameter Input
Posted by Andreas Paepcke <pa...@cs.stanford.edu>.
Great, Alan. I do know ahead of time which
output I want, so your solution will work. Didn't
know you could bind Pig statements to variables
and execute them.
Nonetheless: looking forward to the control flow
feature.
Andreas
On Tue, Mar 15, 2011 at 7:23 AM, Alan Gates <ga...@yahoo-inc.com> wrote:
> If you know before you start the script which you want, you can use
> parameter substitution:
>
> A = load 'foo';
> ...
> Z = foreach Y generate ...;
> $DO_OUTPUT
>
> Then, depending on which you want, run pig with
>
> pig -pDO_OUTPUT='dump Z;';
>
> or
>
> pig -pDO_OUTPUT="store Z into 'outfile';"
>
> If you want to decide which to do based on results of the script, then
> you'll need the new control flow integration features available in 0.9.
> These aren't yet released, but they are in trunk.
>
> Alan.
>
>
> On Mar 14, 2011, at 10:40 PM, Andreas Paepcke wrote:
>
> I want to do something really simple: I want to pass a string
>> into a Pig script. The string is either "stdout" or some target
>> file name. Say the string gets bound to $OUTPUT. That all
>> works fine.
>>
>> After the script has computed a result R, depending on the
>> value of $OUTPUT, I want either to do a dump R (if
>> $OUTPUT == "stdout"), or a STORE of R into the given
>> file name that's held in $OUTPUT.
>>
>> How do I do that conditional? I played with bincond, but
>> got stuck.
>>
>> Thanks!
>>
>> Andreas
>>
>>
>> Andreas
>>
>
>
Re: DUMP or STORE Depending on Parameter Input
Posted by Mridul Muralidharan <mr...@yahoo-inc.com>.
You could try using property file instead of cli param to pass the
name/value ...
Regards,
Mridul
On Tuesday 19 April 2011 05:29 AM, Andreas Paepcke wrote:
> I'm still struggling with parameter substitution.
> Below are six examples. Two work, the others don't.
> When they don't, I get this error message:
>
> ERROR org.apache.pig.Main - ERROR 2999: Unexpected internal error.
> Encountered unexpected arguments on command line - please check the
> command line.
>
> $ pig -param COLOR=blue script.pig -- Works
> $ pig -param COLOR="blue-green" script.pig -- Works
> $ pig -param COLOR="blue green" script.pig -- Fails
> $ pig -param COLOR="'blue green'" script.pig -- Fails
> $ pig -param COLOR="\'blue green\'" script.pig -- Fails
> $ pig -param COLOR="blue\ green" script.pig -- Fails
>
> How do I protect spaces so that I can pass multi-word
> parameters into Pig?
>
> Thanks,
>
> Andreas
>
>
> On Tue, Mar 15, 2011 at 7:23 AM, Alan Gates<ga...@yahoo-inc.com> wrote:
>
>> If you know before you start the script which you want, you can use
>> parameter substitution:
>>
>> A = load 'foo';
>> ...
>> Z = foreach Y generate ...;
>> $DO_OUTPUT
>>
>> Then, depending on which you want, run pig with
>>
>> pig -pDO_OUTPUT='dump Z;';
>>
>> or
>>
>> pig -pDO_OUTPUT="store Z into 'outfile';"
>>
>> If you want to decide which to do based on results of the script, then
>> you'll need the new control flow integration features available in 0.9.
>> These aren't yet released, but they are in trunk.
>>
>> Alan.
>>
>>
>> On Mar 14, 2011, at 10:40 PM, Andreas Paepcke wrote:
>>
>> I want to do something really simple: I want to pass a string
>>> into a Pig script. The string is either "stdout" or some target
>>> file name. Say the string gets bound to $OUTPUT. That all
>>> works fine.
>>>
>>> After the script has computed a result R, depending on the
>>> value of $OUTPUT, I want either to do a dump R (if
>>> $OUTPUT == "stdout"), or a STORE of R into the given
>>> file name that's held in $OUTPUT.
>>>
>>> How do I do that conditional? I played with bincond, but
>>> got stuck.
>>>
>>> Thanks!
>>>
>>> Andreas
>>>
>>>
>>> Andreas
>>>
>>
>>
Re: DUMP or STORE Depending on Parameter Input
Posted by Romain Rigaux <ro...@gmail.com>.
For me in bash it works:
-param COLOR="blue green"
But you can try with \u0020 instead of space:
-param COLOR="blue\u0020green"
Romain
On Mon, Apr 18, 2011 at 4:59 PM, Andreas Paepcke <pa...@cs.stanford.edu>wrote:
> I'm still struggling with parameter substitution.
> Below are six examples. Two work, the others don't.
> When they don't, I get this error message:
>
> ERROR org.apache.pig.Main - ERROR 2999: Unexpected internal error.
> Encountered unexpected arguments on command line - please check the
> command line.
>
> $ pig -param COLOR=blue script.pig -- Works
> $ pig -param COLOR="blue-green" script.pig -- Works
> $ pig -param COLOR="blue green" script.pig -- Fails
> $ pig -param COLOR="'blue green'" script.pig -- Fails
> $ pig -param COLOR="\'blue green\'" script.pig -- Fails
> $ pig -param COLOR="blue\ green" script.pig -- Fails
>
> How do I protect spaces so that I can pass multi-word
> parameters into Pig?
>
> Thanks,
>
> Andreas
>
>
> On Tue, Mar 15, 2011 at 7:23 AM, Alan Gates <ga...@yahoo-inc.com> wrote:
>
> > If you know before you start the script which you want, you can use
> > parameter substitution:
> >
> > A = load 'foo';
> > ...
> > Z = foreach Y generate ...;
> > $DO_OUTPUT
> >
> > Then, depending on which you want, run pig with
> >
> > pig -pDO_OUTPUT='dump Z;';
> >
> > or
> >
> > pig -pDO_OUTPUT="store Z into 'outfile';"
> >
> > If you want to decide which to do based on results of the script, then
> > you'll need the new control flow integration features available in 0.9.
> > These aren't yet released, but they are in trunk.
> >
> > Alan.
> >
> >
> > On Mar 14, 2011, at 10:40 PM, Andreas Paepcke wrote:
> >
> > I want to do something really simple: I want to pass a string
> >> into a Pig script. The string is either "stdout" or some target
> >> file name. Say the string gets bound to $OUTPUT. That all
> >> works fine.
> >>
> >> After the script has computed a result R, depending on the
> >> value of $OUTPUT, I want either to do a dump R (if
> >> $OUTPUT == "stdout"), or a STORE of R into the given
> >> file name that's held in $OUTPUT.
> >>
> >> How do I do that conditional? I played with bincond, but
> >> got stuck.
> >>
> >> Thanks!
> >>
> >> Andreas
> >>
> >>
> >> Andreas
> >>
> >
> >
>
Re: DUMP or STORE Depending on Parameter Input
Posted by Andreas Paepcke <pa...@cs.stanford.edu>.
I'm still struggling with parameter substitution.
Below are six examples. Two work, the others don't.
When they don't, I get this error message:
ERROR org.apache.pig.Main - ERROR 2999: Unexpected internal error.
Encountered unexpected arguments on command line - please check the
command line.
$ pig -param COLOR=blue script.pig -- Works
$ pig -param COLOR="blue-green" script.pig -- Works
$ pig -param COLOR="blue green" script.pig -- Fails
$ pig -param COLOR="'blue green'" script.pig -- Fails
$ pig -param COLOR="\'blue green\'" script.pig -- Fails
$ pig -param COLOR="blue\ green" script.pig -- Fails
How do I protect spaces so that I can pass multi-word
parameters into Pig?
Thanks,
Andreas
On Tue, Mar 15, 2011 at 7:23 AM, Alan Gates <ga...@yahoo-inc.com> wrote:
> If you know before you start the script which you want, you can use
> parameter substitution:
>
> A = load 'foo';
> ...
> Z = foreach Y generate ...;
> $DO_OUTPUT
>
> Then, depending on which you want, run pig with
>
> pig -pDO_OUTPUT='dump Z;';
>
> or
>
> pig -pDO_OUTPUT="store Z into 'outfile';"
>
> If you want to decide which to do based on results of the script, then
> you'll need the new control flow integration features available in 0.9.
> These aren't yet released, but they are in trunk.
>
> Alan.
>
>
> On Mar 14, 2011, at 10:40 PM, Andreas Paepcke wrote:
>
> I want to do something really simple: I want to pass a string
>> into a Pig script. The string is either "stdout" or some target
>> file name. Say the string gets bound to $OUTPUT. That all
>> works fine.
>>
>> After the script has computed a result R, depending on the
>> value of $OUTPUT, I want either to do a dump R (if
>> $OUTPUT == "stdout"), or a STORE of R into the given
>> file name that's held in $OUTPUT.
>>
>> How do I do that conditional? I played with bincond, but
>> got stuck.
>>
>> Thanks!
>>
>> Andreas
>>
>>
>> Andreas
>>
>
>
Re: DUMP or STORE Depending on Parameter Input
Posted by Alan Gates <ga...@yahoo-inc.com>.
If you know before you start the script which you want, you can use
parameter substitution:
A = load 'foo';
...
Z = foreach Y generate ...;
$DO_OUTPUT
Then, depending on which you want, run pig with
pig -pDO_OUTPUT='dump Z;';
or
pig -pDO_OUTPUT="store Z into 'outfile';"
If you want to decide which to do based on results of the script, then
you'll need the new control flow integration features available in
0.9. These aren't yet released, but they are in trunk.
Alan.
On Mar 14, 2011, at 10:40 PM, Andreas Paepcke wrote:
> I want to do something really simple: I want to pass a string
> into a Pig script. The string is either "stdout" or some target
> file name. Say the string gets bound to $OUTPUT. That all
> works fine.
>
> After the script has computed a result R, depending on the
> value of $OUTPUT, I want either to do a dump R (if
> $OUTPUT == "stdout"), or a STORE of R into the given
> file name that's held in $OUTPUT.
>
> How do I do that conditional? I played with bincond, but
> got stuck.
>
> Thanks!
>
> Andreas
>
>
> Andreas