You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by Richard Ding <rd...@yahoo-inc.com> on 2011/03/05 01:01:54 UTC
Review Request: Pig tries to interpret UDF as Macro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/467/
-----------------------------------------------------------
Review request for pig and Julien Le Dem.
Summary
-------
The following script fail:
A = load '1.txt';
B = group A by $0;
C = foreach B {
C1 = filter A by $0 > -1;
C2 = distinct C1;
C3 = distinct A;
C4 = org.apache.pig.test.utils.Identity(*);
C5 = COUNT(C2);
C6 = SUM(C2.$1);
C7 = org.apache.pig.test.TestEvalPipeline\$TitleNGrams(C3);
C8 = MAX(C3.$1);
generate (int)group, C4, C5, C6, C7, C8, C2;
};
dump C;
Stack:
ERROR 2999: Unexpected internal error. Pig macro 'COUNT' must be defined before being invoked
java.lang.RuntimeException: Pig macro 'COUNT' must be defined before being invoked
at org.apache.pig.parser.MacroExpansion.mINLINE(MacroExpansion.java:509)
at org.apache.pig.parser.MacroExpansion.mTokens(MacroExpansion.java:1240)
at org.apache.pig.parser.MacroExpansion.nextToken(MacroExpansion.java:68)
at org.apache.pig.parser.ParserUtil.expandMacros(ParserUtil.java:53)
at org.apache.pig.parser.ParserUtil.getExpandedMacroAsBufferedReader(ParserUtil.java:89)
at org.apache.pig.Main.run(Main.java:505)
at org.apache.pig.Main.main(Main.java:108)
Seems Pig try to interpret UDF COUNT as a Macro.
This addresses bug PIG-1873.
https://issues.apache.org/jira/browse/PIG-1873
Diffs
-----
http://svn.apache.org/repos/asf/pig/trunk/build.xml 1078100
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/MacroExpansion.g 1078100
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/MacroRecursion.g 1078100
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/PigMacro.java 1078100
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestMacroExpansion.java 1078100
Diff: https://reviews.apache.org/r/467/diff
Testing
-------
Thanks,
Richard
Re: Review Request: Pig tries to interpret UDF as Macro
Posted by Julien Le Dem <ju...@ledem.net>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/467/#review318
-----------------------------------------------------------
Ship it!
Looks good!
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestMacroExpansion.java
<https://reviews.apache.org/r/467/#comment645>
does this path matter ?
- Julien
On 2011-03-04 16:01:54, Richard Ding wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/467/
> -----------------------------------------------------------
>
> (Updated 2011-03-04 16:01:54)
>
>
> Review request for pig and Julien Le Dem.
>
>
> Summary
> -------
>
>
> The following script fail:
>
> A = load '1.txt';
> B = group A by $0;
> C = foreach B {
> C1 = filter A by $0 > -1;
> C2 = distinct C1;
> C3 = distinct A;
> C4 = org.apache.pig.test.utils.Identity(*);
> C5 = COUNT(C2);
> C6 = SUM(C2.$1);
> C7 = org.apache.pig.test.TestEvalPipeline\$TitleNGrams(C3);
> C8 = MAX(C3.$1);
> generate (int)group, C4, C5, C6, C7, C8, C2;
> };
> dump C;
>
> Stack:
> ERROR 2999: Unexpected internal error. Pig macro 'COUNT' must be defined before being invoked
>
> java.lang.RuntimeException: Pig macro 'COUNT' must be defined before being invoked
> at org.apache.pig.parser.MacroExpansion.mINLINE(MacroExpansion.java:509)
> at org.apache.pig.parser.MacroExpansion.mTokens(MacroExpansion.java:1240)
> at org.apache.pig.parser.MacroExpansion.nextToken(MacroExpansion.java:68)
> at org.apache.pig.parser.ParserUtil.expandMacros(ParserUtil.java:53)
> at org.apache.pig.parser.ParserUtil.getExpandedMacroAsBufferedReader(ParserUtil.java:89)
> at org.apache.pig.Main.run(Main.java:505)
> at org.apache.pig.Main.main(Main.java:108)
>
> Seems Pig try to interpret UDF COUNT as a Macro.
>
>
> This addresses bug PIG-1873.
> https://issues.apache.org/jira/browse/PIG-1873
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/pig/trunk/build.xml 1078100
> http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/MacroExpansion.g 1078100
> http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/MacroRecursion.g 1078100
> http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/PigMacro.java 1078100
> http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestMacroExpansion.java 1078100
>
> Diff: https://reviews.apache.org/r/467/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Richard
>
>