You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Cheolsoo Park (JIRA)" <ji...@apache.org> on 2012/09/25 07:41:09 UTC
[jira] [Created] (PIG-2931) $ signs in the replacement string make
parameter substitution fail
Cheolsoo Park created PIG-2931:
----------------------------------
Summary: $ signs in the replacement string make parameter substitution fail
Key: PIG-2931
URL: https://issues.apache.org/jira/browse/PIG-2931
Project: Pig
Issue Type: Bug
Affects Versions: 0.10.0
Reporter: Cheolsoo Park
Assignee: Cheolsoo Park
Fix For: 0.11
To reproduce the issue, use the following pig script:
{code:title=test.pig}
a = load 'data';
b = filter by $FILTER;
{code}
and run the following command:
{code}
pig -x local -dryrun -f test.pig -p FILTER="(\$0 == 'a')"
{code}
This generates the following script:
{code:title=test.pig.substituted}
a = load 'data';
b = filter by ($FILTER == 'a');
{code}
However this should be:
{code}
a = load 'data';
b = filter by ($0 == 'a');
{code}
This is because Pig calls replaceFirst() with a replacement string that include a $ sign as follows:
{code}
"$FILTER".replaceFirst("\\$FILTER", "($0 == 'a')"));
{code}
To treat $ signs as literals in the replacement string, we must escape them. Please see the [Java doc|http://docs.oracle.com/javase/6/docs/api/java/util/regex/Matcher.html#replaceFirst(java.lang.String)] for Matcher class for explanation:
{quote}
Note that backslashes (\) and dollar signs ($) in the replacement string may cause the results to be different than if it were being treated as a literal replacement string. Dollar signs may be treated as references to captured subsequences as described above, and backslashes are used to escape literal characters in the replacement string.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2931) $ signs in the replacement string make
parameter substitution fail
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheolsoo Park updated PIG-2931:
-------------------------------
Status: Patch Available (was: Open)
> $ signs in the replacement string make parameter substitution fail
> ------------------------------------------------------------------
>
> Key: PIG-2931
> URL: https://issues.apache.org/jira/browse/PIG-2931
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10.0
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: 0.11
>
> Attachments: PIG-2931.patch
>
>
> To reproduce the issue, use the following pig script:
> {code:title=test.pig}
> a = load 'data';
> b = filter by $FILTER;
> {code}
> and run the following command:
> {code}
> pig -x local -dryrun -f test.pig -p FILTER="(\$0 == 'a')"
> {code}
> This generates the following script:
> {code:title=test.pig.substituted}
> a = load 'data';
> b = filter by ($FILTER == 'a');
> {code}
> However this should be:
> {code}
> a = load 'data';
> b = filter by ($0 == 'a');
> {code}
> This is because Pig calls replaceFirst() with a replacement string that include a $ sign as follows:
> {code}
> "$FILTER".replaceFirst("\\$FILTER", "($0 == 'a')"));
> {code}
> To treat $ signs as literals in the replacement string, we must escape them. Please see the [Java doc|http://docs.oracle.com/javase/6/docs/api/java/util/regex/Matcher.html#replaceFirst(java.lang.String)] for Matcher class for explanation:
> {quote}
> Note that backslashes (\) and dollar signs ($) in the replacement string may cause the results to be different than if it were being treated as a literal replacement string. Dollar signs may be treated as references to captured subsequences as described above, and backslashes are used to escape literal characters in the replacement string.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2931) $ signs in the replacement string
make parameter substitution fail
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478589#comment-13478589 ]
Dmitriy V. Ryaboy commented on PIG-2931:
----------------------------------------
ok, +1. Please make a note about the highly unlikely backwards incompatibility.
> $ signs in the replacement string make parameter substitution fail
> ------------------------------------------------------------------
>
> Key: PIG-2931
> URL: https://issues.apache.org/jira/browse/PIG-2931
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10.0
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: 0.11
>
> Attachments: PIG-2931.patch
>
>
> To reproduce the issue, use the following pig script:
> {code:title=test.pig}
> a = load 'data';
> b = filter by $FILTER;
> {code}
> and run the following command:
> {code}
> pig -x local -dryrun -f test.pig -p FILTER="(\$0 == 'a')"
> {code}
> This generates the following script:
> {code:title=test.pig.substituted}
> a = load 'data';
> b = filter by ($FILTER == 'a');
> {code}
> However this should be:
> {code}
> a = load 'data';
> b = filter by ($0 == 'a');
> {code}
> This is because Pig calls replaceFirst() with a replacement string that include a $ sign as follows:
> {code}
> "$FILTER".replaceFirst("\\$FILTER", "($0 == 'a')"));
> {code}
> To treat $ signs as literals in the replacement string, we must escape them. Please see the [Java doc|http://docs.oracle.com/javase/6/docs/api/java/util/regex/Matcher.html#replaceFirst(java.lang.String)] for Matcher class for explanation:
> {quote}
> Note that backslashes (\) and dollar signs ($) in the replacement string may cause the results to be different than if it were being treated as a literal replacement string. Dollar signs may be treated as references to captured subsequences as described above, and backslashes are used to escape literal characters in the replacement string.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2931) $ signs in the replacement string
make parameter substitution fail
Posted by "Jonathan Coveney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478563#comment-13478563 ]
Jonathan Coveney commented on PIG-2931:
---------------------------------------
Dmitriy, I agree with Cheolsoo. Seems unlikely, and I'd rather fix it as it's a totally fair use of positionals. +1
> $ signs in the replacement string make parameter substitution fail
> ------------------------------------------------------------------
>
> Key: PIG-2931
> URL: https://issues.apache.org/jira/browse/PIG-2931
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10.0
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: 0.11
>
> Attachments: PIG-2931.patch
>
>
> To reproduce the issue, use the following pig script:
> {code:title=test.pig}
> a = load 'data';
> b = filter by $FILTER;
> {code}
> and run the following command:
> {code}
> pig -x local -dryrun -f test.pig -p FILTER="(\$0 == 'a')"
> {code}
> This generates the following script:
> {code:title=test.pig.substituted}
> a = load 'data';
> b = filter by ($FILTER == 'a');
> {code}
> However this should be:
> {code}
> a = load 'data';
> b = filter by ($0 == 'a');
> {code}
> This is because Pig calls replaceFirst() with a replacement string that include a $ sign as follows:
> {code}
> "$FILTER".replaceFirst("\\$FILTER", "($0 == 'a')"));
> {code}
> To treat $ signs as literals in the replacement string, we must escape them. Please see the [Java doc|http://docs.oracle.com/javase/6/docs/api/java/util/regex/Matcher.html#replaceFirst(java.lang.String)] for Matcher class for explanation:
> {quote}
> Note that backslashes (\) and dollar signs ($) in the replacement string may cause the results to be different than if it were being treated as a literal replacement string. Dollar signs may be treated as references to captured subsequences as described above, and backslashes are used to escape literal characters in the replacement string.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2931) $ signs in the replacement string
make parameter substitution fail
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13481819#comment-13481819 ]
Cheolsoo Park commented on PIG-2931:
------------------------------------
Hi Julien,
Thank you very much for pointing that out. In fact, I've just realized that I broke TestScriptLanguange for the reason that you're describing. Pig#bind() escapes '$', so my change makes bindLocalVariableTest2() fail.
Please let me open a jira under PIG-2972 and post a patch.
Thanks!
> $ signs in the replacement string make parameter substitution fail
> ------------------------------------------------------------------
>
> Key: PIG-2931
> URL: https://issues.apache.org/jira/browse/PIG-2931
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10.0
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: 0.11
>
> Attachments: PIG-2931.patch
>
>
> To reproduce the issue, use the following pig script:
> {code:title=test.pig}
> a = load 'data';
> b = filter by $FILTER;
> {code}
> and run the following command:
> {code}
> pig -x local -dryrun -f test.pig -p FILTER="(\$0 == 'a')"
> {code}
> This generates the following script:
> {code:title=test.pig.substituted}
> a = load 'data';
> b = filter by ($FILTER == 'a');
> {code}
> However this should be:
> {code}
> a = load 'data';
> b = filter by ($0 == 'a');
> {code}
> This is because Pig calls replaceFirst() with a replacement string that include a $ sign as follows:
> {code}
> "$FILTER".replaceFirst("\\$FILTER", "($0 == 'a')"));
> {code}
> To treat $ signs as literals in the replacement string, we must escape them. Please see the [Java doc|http://docs.oracle.com/javase/6/docs/api/java/util/regex/Matcher.html#replaceFirst(java.lang.String)] for Matcher class for explanation:
> {quote}
> Note that backslashes (\) and dollar signs ($) in the replacement string may cause the results to be different than if it were being treated as a literal replacement string. Dollar signs may be treated as references to captured subsequences as described above, and backslashes are used to escape literal characters in the replacement string.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2931) $ signs in the replacement string make
parameter substitution fail
Posted by "Jonathan Coveney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Coveney updated PIG-2931:
----------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Thanks Cheolsoo! It's in
> $ signs in the replacement string make parameter substitution fail
> ------------------------------------------------------------------
>
> Key: PIG-2931
> URL: https://issues.apache.org/jira/browse/PIG-2931
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10.0
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: 0.11
>
> Attachments: PIG-2931.patch
>
>
> To reproduce the issue, use the following pig script:
> {code:title=test.pig}
> a = load 'data';
> b = filter by $FILTER;
> {code}
> and run the following command:
> {code}
> pig -x local -dryrun -f test.pig -p FILTER="(\$0 == 'a')"
> {code}
> This generates the following script:
> {code:title=test.pig.substituted}
> a = load 'data';
> b = filter by ($FILTER == 'a');
> {code}
> However this should be:
> {code}
> a = load 'data';
> b = filter by ($0 == 'a');
> {code}
> This is because Pig calls replaceFirst() with a replacement string that include a $ sign as follows:
> {code}
> "$FILTER".replaceFirst("\\$FILTER", "($0 == 'a')"));
> {code}
> To treat $ signs as literals in the replacement string, we must escape them. Please see the [Java doc|http://docs.oracle.com/javase/6/docs/api/java/util/regex/Matcher.html#replaceFirst(java.lang.String)] for Matcher class for explanation:
> {quote}
> Note that backslashes (\) and dollar signs ($) in the replacement string may cause the results to be different than if it were being treated as a literal replacement string. Dollar signs may be treated as references to captured subsequences as described above, and backslashes are used to escape literal characters in the replacement string.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2931) $ signs in the replacement string
make parameter substitution fail
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466961#comment-13466961 ]
Dmitriy V. Ryaboy commented on PIG-2931:
----------------------------------------
The patch looks correct, but I can't help but wonder if anyone might be (ab)using the current behavior.. second opinion?
> $ signs in the replacement string make parameter substitution fail
> ------------------------------------------------------------------
>
> Key: PIG-2931
> URL: https://issues.apache.org/jira/browse/PIG-2931
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10.0
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: 0.11
>
> Attachments: PIG-2931.patch
>
>
> To reproduce the issue, use the following pig script:
> {code:title=test.pig}
> a = load 'data';
> b = filter by $FILTER;
> {code}
> and run the following command:
> {code}
> pig -x local -dryrun -f test.pig -p FILTER="(\$0 == 'a')"
> {code}
> This generates the following script:
> {code:title=test.pig.substituted}
> a = load 'data';
> b = filter by ($FILTER == 'a');
> {code}
> However this should be:
> {code}
> a = load 'data';
> b = filter by ($0 == 'a');
> {code}
> This is because Pig calls replaceFirst() with a replacement string that include a $ sign as follows:
> {code}
> "$FILTER".replaceFirst("\\$FILTER", "($0 == 'a')"));
> {code}
> To treat $ signs as literals in the replacement string, we must escape them. Please see the [Java doc|http://docs.oracle.com/javase/6/docs/api/java/util/regex/Matcher.html#replaceFirst(java.lang.String)] for Matcher class for explanation:
> {quote}
> Note that backslashes (\) and dollar signs ($) in the replacement string may cause the results to be different than if it were being treated as a literal replacement string. Dollar signs may be treated as references to captured subsequences as described above, and backslashes are used to escape literal characters in the replacement string.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2931) $ signs in the replacement string make
parameter substitution fail
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheolsoo Park updated PIG-2931:
-------------------------------
Attachment: (was: PIG-2931.patch)
> $ signs in the replacement string make parameter substitution fail
> ------------------------------------------------------------------
>
> Key: PIG-2931
> URL: https://issues.apache.org/jira/browse/PIG-2931
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10.0
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: 0.11
>
> Attachments: PIG-2931.patch
>
>
> To reproduce the issue, use the following pig script:
> {code:title=test.pig}
> a = load 'data';
> b = filter by $FILTER;
> {code}
> and run the following command:
> {code}
> pig -x local -dryrun -f test.pig -p FILTER="(\$0 == 'a')"
> {code}
> This generates the following script:
> {code:title=test.pig.substituted}
> a = load 'data';
> b = filter by ($FILTER == 'a');
> {code}
> However this should be:
> {code}
> a = load 'data';
> b = filter by ($0 == 'a');
> {code}
> This is because Pig calls replaceFirst() with a replacement string that include a $ sign as follows:
> {code}
> "$FILTER".replaceFirst("\\$FILTER", "($0 == 'a')"));
> {code}
> To treat $ signs as literals in the replacement string, we must escape them. Please see the [Java doc|http://docs.oracle.com/javase/6/docs/api/java/util/regex/Matcher.html#replaceFirst(java.lang.String)] for Matcher class for explanation:
> {quote}
> Note that backslashes (\) and dollar signs ($) in the replacement string may cause the results to be different than if it were being treated as a literal replacement string. Dollar signs may be treated as references to captured subsequences as described above, and backslashes are used to escape literal characters in the replacement string.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2931) $ signs in the replacement string
make parameter substitution fail
Posted by "Julien Le Dem (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13481788#comment-13481788 ]
Julien Le Dem commented on PIG-2931:
------------------------------------
sorry to reply late on this. This would break users that are already double escaping $ to insert values containing $
We should escape $ only if it is not escaped yet.
> $ signs in the replacement string make parameter substitution fail
> ------------------------------------------------------------------
>
> Key: PIG-2931
> URL: https://issues.apache.org/jira/browse/PIG-2931
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10.0
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: 0.11
>
> Attachments: PIG-2931.patch
>
>
> To reproduce the issue, use the following pig script:
> {code:title=test.pig}
> a = load 'data';
> b = filter by $FILTER;
> {code}
> and run the following command:
> {code}
> pig -x local -dryrun -f test.pig -p FILTER="(\$0 == 'a')"
> {code}
> This generates the following script:
> {code:title=test.pig.substituted}
> a = load 'data';
> b = filter by ($FILTER == 'a');
> {code}
> However this should be:
> {code}
> a = load 'data';
> b = filter by ($0 == 'a');
> {code}
> This is because Pig calls replaceFirst() with a replacement string that include a $ sign as follows:
> {code}
> "$FILTER".replaceFirst("\\$FILTER", "($0 == 'a')"));
> {code}
> To treat $ signs as literals in the replacement string, we must escape them. Please see the [Java doc|http://docs.oracle.com/javase/6/docs/api/java/util/regex/Matcher.html#replaceFirst(java.lang.String)] for Matcher class for explanation:
> {quote}
> Note that backslashes (\) and dollar signs ($) in the replacement string may cause the results to be different than if it were being treated as a literal replacement string. Dollar signs may be treated as references to captured subsequences as described above, and backslashes are used to escape literal characters in the replacement string.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2931) $ signs in the replacement string make
parameter substitution fail
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheolsoo Park updated PIG-2931:
-------------------------------
Attachment: PIG-2931.patch
Attaching a patch that fixes the issue by escaping $ signs in the replacement string.
I also added a test case to the TestParamSubPreproc suite and verified that it passes.
> $ signs in the replacement string make parameter substitution fail
> ------------------------------------------------------------------
>
> Key: PIG-2931
> URL: https://issues.apache.org/jira/browse/PIG-2931
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10.0
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: 0.11
>
> Attachments: PIG-2931.patch
>
>
> To reproduce the issue, use the following pig script:
> {code:title=test.pig}
> a = load 'data';
> b = filter by $FILTER;
> {code}
> and run the following command:
> {code}
> pig -x local -dryrun -f test.pig -p FILTER="(\$0 == 'a')"
> {code}
> This generates the following script:
> {code:title=test.pig.substituted}
> a = load 'data';
> b = filter by ($FILTER == 'a');
> {code}
> However this should be:
> {code}
> a = load 'data';
> b = filter by ($0 == 'a');
> {code}
> This is because Pig calls replaceFirst() with a replacement string that include a $ sign as follows:
> {code}
> "$FILTER".replaceFirst("\\$FILTER", "($0 == 'a')"));
> {code}
> To treat $ signs as literals in the replacement string, we must escape them. Please see the [Java doc|http://docs.oracle.com/javase/6/docs/api/java/util/regex/Matcher.html#replaceFirst(java.lang.String)] for Matcher class for explanation:
> {quote}
> Note that backslashes (\) and dollar signs ($) in the replacement string may cause the results to be different than if it were being treated as a literal replacement string. Dollar signs may be treated as references to captured subsequences as described above, and backslashes are used to escape literal characters in the replacement string.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2931) $ signs in the replacement string make
parameter substitution fail
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheolsoo Park updated PIG-2931:
-------------------------------
Release Note: This changes the behavior of parameter substitution so that dollar signs in the replacement string are no longer treated as references to captured subsequences, but they are now treated as a literal replacement string. Therefore, this will cause backward compatibility issues for users who rely on the previous behavior of parameter substitution.
Hadoop Flags: Incompatible change
> $ signs in the replacement string make parameter substitution fail
> ------------------------------------------------------------------
>
> Key: PIG-2931
> URL: https://issues.apache.org/jira/browse/PIG-2931
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10.0
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: 0.11
>
> Attachments: PIG-2931.patch
>
>
> To reproduce the issue, use the following pig script:
> {code:title=test.pig}
> a = load 'data';
> b = filter by $FILTER;
> {code}
> and run the following command:
> {code}
> pig -x local -dryrun -f test.pig -p FILTER="(\$0 == 'a')"
> {code}
> This generates the following script:
> {code:title=test.pig.substituted}
> a = load 'data';
> b = filter by ($FILTER == 'a');
> {code}
> However this should be:
> {code}
> a = load 'data';
> b = filter by ($0 == 'a');
> {code}
> This is because Pig calls replaceFirst() with a replacement string that include a $ sign as follows:
> {code}
> "$FILTER".replaceFirst("\\$FILTER", "($0 == 'a')"));
> {code}
> To treat $ signs as literals in the replacement string, we must escape them. Please see the [Java doc|http://docs.oracle.com/javase/6/docs/api/java/util/regex/Matcher.html#replaceFirst(java.lang.String)] for Matcher class for explanation:
> {quote}
> Note that backslashes (\) and dollar signs ($) in the replacement string may cause the results to be different than if it were being treated as a literal replacement string. Dollar signs may be treated as references to captured subsequences as described above, and backslashes are used to escape literal characters in the replacement string.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2931) $ signs in the replacement string
make parameter substitution fail
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467006#comment-13467006 ]
Cheolsoo Park commented on PIG-2931:
------------------------------------
Hi Dmitriy,
That's an interesting thought. But in this case, $0 and $1 will only be able to refer to parameter names. I can't think why someone would want to replace parameter names with strings that include themselves. I might not be seeing a useful case.
Thanks!
> $ signs in the replacement string make parameter substitution fail
> ------------------------------------------------------------------
>
> Key: PIG-2931
> URL: https://issues.apache.org/jira/browse/PIG-2931
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10.0
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: 0.11
>
> Attachments: PIG-2931.patch
>
>
> To reproduce the issue, use the following pig script:
> {code:title=test.pig}
> a = load 'data';
> b = filter by $FILTER;
> {code}
> and run the following command:
> {code}
> pig -x local -dryrun -f test.pig -p FILTER="(\$0 == 'a')"
> {code}
> This generates the following script:
> {code:title=test.pig.substituted}
> a = load 'data';
> b = filter by ($FILTER == 'a');
> {code}
> However this should be:
> {code}
> a = load 'data';
> b = filter by ($0 == 'a');
> {code}
> This is because Pig calls replaceFirst() with a replacement string that include a $ sign as follows:
> {code}
> "$FILTER".replaceFirst("\\$FILTER", "($0 == 'a')"));
> {code}
> To treat $ signs as literals in the replacement string, we must escape them. Please see the [Java doc|http://docs.oracle.com/javase/6/docs/api/java/util/regex/Matcher.html#replaceFirst(java.lang.String)] for Matcher class for explanation:
> {quote}
> Note that backslashes (\) and dollar signs ($) in the replacement string may cause the results to be different than if it were being treated as a literal replacement string. Dollar signs may be treated as references to captured subsequences as described above, and backslashes are used to escape literal characters in the replacement string.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2931) $ signs in the replacement string make
parameter substitution fail
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheolsoo Park updated PIG-2931:
-------------------------------
Attachment: PIG-2931.patch
> $ signs in the replacement string make parameter substitution fail
> ------------------------------------------------------------------
>
> Key: PIG-2931
> URL: https://issues.apache.org/jira/browse/PIG-2931
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10.0
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: 0.11
>
> Attachments: PIG-2931.patch
>
>
> To reproduce the issue, use the following pig script:
> {code:title=test.pig}
> a = load 'data';
> b = filter by $FILTER;
> {code}
> and run the following command:
> {code}
> pig -x local -dryrun -f test.pig -p FILTER="(\$0 == 'a')"
> {code}
> This generates the following script:
> {code:title=test.pig.substituted}
> a = load 'data';
> b = filter by ($FILTER == 'a');
> {code}
> However this should be:
> {code}
> a = load 'data';
> b = filter by ($0 == 'a');
> {code}
> This is because Pig calls replaceFirst() with a replacement string that include a $ sign as follows:
> {code}
> "$FILTER".replaceFirst("\\$FILTER", "($0 == 'a')"));
> {code}
> To treat $ signs as literals in the replacement string, we must escape them. Please see the [Java doc|http://docs.oracle.com/javase/6/docs/api/java/util/regex/Matcher.html#replaceFirst(java.lang.String)] for Matcher class for explanation:
> {quote}
> Note that backslashes (\) and dollar signs ($) in the replacement string may cause the results to be different than if it were being treated as a literal replacement string. Dollar signs may be treated as references to captured subsequences as described above, and backslashes are used to escape literal characters in the replacement string.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2931) $ signs in the replacement string make
parameter substitution fail
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheolsoo Park updated PIG-2931:
-------------------------------
Hadoop Flags: (was: Incompatible change)
> $ signs in the replacement string make parameter substitution fail
> ------------------------------------------------------------------
>
> Key: PIG-2931
> URL: https://issues.apache.org/jira/browse/PIG-2931
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10.0
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: 0.11
>
> Attachments: PIG-2931.patch
>
>
> To reproduce the issue, use the following pig script:
> {code:title=test.pig}
> a = load 'data';
> b = filter by $FILTER;
> {code}
> and run the following command:
> {code}
> pig -x local -dryrun -f test.pig -p FILTER="(\$0 == 'a')"
> {code}
> This generates the following script:
> {code:title=test.pig.substituted}
> a = load 'data';
> b = filter by ($FILTER == 'a');
> {code}
> However this should be:
> {code}
> a = load 'data';
> b = filter by ($0 == 'a');
> {code}
> This is because Pig calls replaceFirst() with a replacement string that include a $ sign as follows:
> {code}
> "$FILTER".replaceFirst("\\$FILTER", "($0 == 'a')"));
> {code}
> To treat $ signs as literals in the replacement string, we must escape them. Please see the [Java doc|http://docs.oracle.com/javase/6/docs/api/java/util/regex/Matcher.html#replaceFirst(java.lang.String)] for Matcher class for explanation:
> {quote}
> Note that backslashes (\) and dollar signs ($) in the replacement string may cause the results to be different than if it were being treated as a literal replacement string. Dollar signs may be treated as references to captured subsequences as described above, and backslashes are used to escape literal characters in the replacement string.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira