You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by "Duckworth, Will" <wd...@comscore.com> on 2012/08/27 22:50:13 UTC
Parameterized Expression in Filter
I am trying to use a parameter as the expression in a filter.
Assuming:
colors_in = load ‘$in_path’ as (color:chararray);
flt = filter colors_in by color == ‘blue’ or color == ‘green’;
I would like to use this instead:
flt = filter colors_in by ‘$color_filter’;
But I get an unexpected symbol near ‘color’. I have tried a couple different formats for the string that I am passing in an none seem to work.
I can get this to work:
flt = filter colors_in by (color == ‘$color_filter1’ or color == ‘$color_filter2’);
But it means I have to know the complete filter for anything I want to run.
I know I could do this with embedded or dynamically building the pig but was trying to do it with just passing parameters to an existing pig script.
Any ideas?
Thanks.
Will Duckworth Senior Vice President, Software Engineering | comScore, Inc. (NASDAQ:SCOR)
o +1 (703) 438-2108 | m +1 (301) 606-2977 | wduckworth@comscore.com<ma...@comscore.com>
...........................................................................................................
Introducing Mobile Metrix 2.0 - The next generation of mobile behavioral measurement
www.comscore.com/MobileMetrix<http://www.comscore.com/Products_Services/Product_Index/Mobile_Metrix_2.0>
Re: Parameterized Expression in Filter
Posted by Jonathan Coveney <jc...@gmail.com>.
There are a couple ways to do this. One is to turn your filter into a
replicated join. IE put the colors you want in a separate file, and then
just join on color. Since it is an equijoin, it's essentially a big or
statement.
The second way would be the use the parameter approach, and then make a UDF
returning boolean that takes that parameter as it's argument
ie
DEFINE FilterUdf com.comscore.pig.MyFilterUdf('$color_param')
filt = filter colors_in by FilterUdf(color);
Make sense?
2012/8/27 Duckworth, Will <wd...@comscore.com>
> I am trying to use a parameter as the expression in a filter.
>
> Assuming:
>
> colors_in = load ‘$in_path’ as (color:chararray);
> flt = filter colors_in by color == ‘blue’ or color == ‘green’;
>
> I would like to use this instead:
>
> flt = filter colors_in by ‘$color_filter’;
>
> But I get an unexpected symbol near ‘color’. I have tried a couple
> different formats for the string that I am passing in an none seem to work.
>
> I can get this to work:
>
> flt = filter colors_in by (color == ‘$color_filter1’ or color ==
> ‘$color_filter2’);
>
> But it means I have to know the complete filter for anything I want to run.
>
> I know I could do this with embedded or dynamically building the pig but
> was trying to do it with just passing parameters to an existing pig script.
>
> Any ideas?
>
> Thanks.
>
>
>
> Will Duckworth Senior Vice President, Software Engineering | comScore,
> Inc. (NASDAQ:SCOR)
>
> o +1 (703) 438-2108 | m +1 (301) 606-2977 | wduckworth@comscore.com
> <ma...@comscore.com>
>
>
> ...........................................................................................................
>
> Introducing Mobile Metrix 2.0 - The next generation of mobile behavioral
> measurement
> www.comscore.com/MobileMetrix<
> http://www.comscore.com/Products_Services/Product_Index/Mobile_Metrix_2.0>
>
>
>
Re: Parameterized Expression in Filter
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
I think you just want this:
filt = filter colors_in by $color_filter;
(no quotes)
D
On Mon, Aug 27, 2012 at 1:50 PM, Duckworth, Will
<wd...@comscore.com> wrote:
> I am trying to use a parameter as the expression in a filter.
>
> Assuming:
>
> colors_in = load ‘$in_path’ as (color:chararray);
> flt = filter colors_in by color == ‘blue’ or color == ‘green’;
>
> I would like to use this instead:
>
> flt = filter colors_in by ‘$color_filter’;
>
> But I get an unexpected symbol near ‘color’. I have tried a couple different formats for the string that I am passing in an none seem to work.
>
> I can get this to work:
>
> flt = filter colors_in by (color == ‘$color_filter1’ or color == ‘$color_filter2’);
>
> But it means I have to know the complete filter for anything I want to run.
>
> I know I could do this with embedded or dynamically building the pig but was trying to do it with just passing parameters to an existing pig script.
>
> Any ideas?
>
> Thanks.
>
>
>
> Will Duckworth Senior Vice President, Software Engineering | comScore, Inc. (NASDAQ:SCOR)
>
> o +1 (703) 438-2108 | m +1 (301) 606-2977 | wduckworth@comscore.com<ma...@comscore.com>
>
> ...........................................................................................................
>
> Introducing Mobile Metrix 2.0 - The next generation of mobile behavioral measurement
> www.comscore.com/MobileMetrix<http://www.comscore.com/Products_Services/Product_Index/Mobile_Metrix_2.0>
>
>