You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Huo Zhu <zh...@gmail.com> on 2012/09/25 04:51:55 UTC
pig parameter substitution problem
hi, i recently find this problem when i using parameter substitution in
pig script
my test.pig
a = load ‘data' as (ch:chararray, num:int)
> b = filter by $FILTER
i execute this script with, ( i need to use positional notation to refer ch
)
pig -x local -dryrun -p FILTER=“ (\$0 == 'a') " -f test.pig
the test.pig.substituted is
b = filter by ($FILTER == 'a')
why is \$0 replaced by $FILTER even i escaped the '$'
--
Good Luck
Re: pig parameter substitution problem
Posted by Cheolsoo Park <ch...@cloudera.com>.
Opened a jira: https://issues.apache.org/jira/browse/PIG-2931
On Mon, Sep 24, 2012 at 9:07 PM, Cheolsoo Park <ch...@cloudera.com>wrote:
> Hi,
>
> This is a bug. What happens at line #236 in PreProcessorContext.java is
> basically as follows:
>
> $FILTER".replaceFirst("\\$FILTER", "($0 == 'a')"));
>
>
> This returns ($FILTER == 'a').
>
> The reason is explained in the Java doc for String class:
>
> Note that backslashes (\) and dollar signs ($) in the replacement string
>> may cause the results to be different than if it were being treated as a
>> literal replacement string. Dollar signs may be treated as references to
>> captured subsequences as described above, and backslashes are used to
>> escape literal characters in the replacement string.
>
>
> I guess that the fix is to escape the $ and \ chars in the replacement
> string before calling replaceFirst(). I will open a jira for this.
>
> Thanks,
> Cheolsoo
>
>
> On Mon, Sep 24, 2012 at 7:51 PM, Huo Zhu <zh...@gmail.com> wrote:
>
>> hi, i recently find this problem when i using parameter substitution in
>> pig script
>>
>> my test.pig
>>
>> a = load ‘data' as (ch:chararray, num:int)
>> > b = filter by $FILTER
>>
>>
>>
>> i execute this script with, ( i need to use positional notation to refer
>> ch
>> )
>>
>> pig -x local -dryrun -p FILTER=“ (\$0 == 'a') " -f test.pig
>>
>>
>> the test.pig.substituted is
>>
>> b = filter by ($FILTER == 'a')
>>
>>
>> why is \$0 replaced by $FILTER even i escaped the '$'
>>
>> --
>> Good Luck
>>
>
>
Re: pig parameter substitution problem
Posted by Cheolsoo Park <ch...@cloudera.com>.
Hi,
This is a bug. What happens at line #236 in PreProcessorContext.java is
basically as follows:
$FILTER".replaceFirst("\\$FILTER", "($0 == 'a')"));
This returns ($FILTER == 'a').
The reason is explained in the Java doc for String class:
Note that backslashes (\) and dollar signs ($) in the replacement string
> may cause the results to be different than if it were being treated as a
> literal replacement string. Dollar signs may be treated as references to
> captured subsequences as described above, and backslashes are used to
> escape literal characters in the replacement string.
I guess that the fix is to escape the $ and \ chars in the replacement
string before calling replaceFirst(). I will open a jira for this.
Thanks,
Cheolsoo
On Mon, Sep 24, 2012 at 7:51 PM, Huo Zhu <zh...@gmail.com> wrote:
> hi, i recently find this problem when i using parameter substitution in
> pig script
>
> my test.pig
>
> a = load ‘data' as (ch:chararray, num:int)
> > b = filter by $FILTER
>
>
>
> i execute this script with, ( i need to use positional notation to refer ch
> )
>
> pig -x local -dryrun -p FILTER=“ (\$0 == 'a') " -f test.pig
>
>
> the test.pig.substituted is
>
> b = filter by ($FILTER == 'a')
>
>
> why is \$0 replaced by $FILTER even i escaped the '$'
>
> --
> Good Luck
>