You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Pradeep Kamath (JIRA)" <ji...@apache.org> on 2008/09/25 02:29:44 UTC
[jira] Issue Comment Edited: (PIG-443) Illustrate for the Types
branch
[ https://issues.apache.org/jira/browse/PIG-443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12634354#action_12634354 ]
pkamath edited comment on PIG-443 at 9/24/08 5:29 PM:
-------------------------------------------------------------
- A couple of unit tests - TestLogicalPlanBuilder and TestLogToPhyCompiler fail with the patch.
- The following script fails at the last illustrate.
{code}
a = load '/user/pig/tests/data/singlefile/studenttab10k' as (name, age, gpa);
b = load '/user/pig/tests/data/singlefile/votertab10k' as (name, age, registration, contributions);
c = filter a by age < 50;
illustrate c;
d = filter b by age < 50;
illustrate d;
e = cogroup c by (name, age), d by (name, age) ;
illustrate e;
f = foreach e generate flatten(c), flatten(d);
illustrate f;
g = group f by registration;
illustrate g;
h = foreach g generate group, SUM(f.d::contributions);
illustrate h;
i = order h by $1;
illustrate i;
{code}
Error:
{noformat}
...
--------------------------------------
| h | group: bytearray | double |
--------------------------------------
| | libertarian | 1697.08 |
| | republican | 1655.32 |
--------------------------------------
10118 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://clusterhost
10143 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: clusterhost
java.lang.NullPointerException
at org.apache.pig.pen.LineageTrimmingVisitor.PruneBaseDataConstrainedCoverage(LineageTrimmingVisitor.java:290)
at org.apache.pig.pen.LineageTrimmingVisitor.checkCompleteness(LineageTrimmingVisitor.java:458)
at org.apache.pig.pen.LineageTrimmingVisitor.processOperator(LineageTrimmingVisitor.java:416)
at org.apache.pig.pen.LineageTrimmingVisitor.visit(LineageTrimmingVisitor.java:251)
at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:105)
at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:40)
at org.apache.pig.pen.util.PreOrderDepthFirstWalker.depthFirst(PreOrderDepthFirstWalker.java:69)
at org.apache.pig.pen.util.PreOrderDepthFirstWalker.depthFirst(PreOrderDepthFirstWalker.java:71)
at org.apache.pig.pen.util.PreOrderDepthFirstWalker.walk(PreOrderDepthFirstWalker.java:55)
at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
at org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:108)
at org.apache.pig.PigServer.getExamples(PigServer.java:627)
at org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:278)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:183)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:83)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:64)
at org.apache.pig.Main.main(Main.java:306)
{noformat}
was (Author: pkamath):
- A couple of unit tests - TestLogicalPlanBuilder and TestLogToPhyCompiler fail with the patch.
- The following script fails at the last illustrate.
{code}
a = load '/user/pig/tests/data/singlefile/studenttab10k' as (name, age, gpa);
b = load '/user/pig/tests/data/singlefile/votertab10k' as (name, age, registration, contributions);
c = filter a by age < 50;
illustrate c;
d = filter b by age < 50;
illustrate d;
e = cogroup c by (name, age), d by (name, age) ;
illustrate e;
f = foreach e generate flatten(c), flatten(d);
illustrate f;
g = group f by registration;
illustrate g;
h = foreach g generate group, SUM(f.d::contributions);
illustrate h;
i = order h by $1;
illustrate i;
{code}
Error:
{noformat}
...
--------------------------------------
| h | group: bytearray | double |
--------------------------------------
| | libertarian | 1697.08 |
| | republican | 1655.32 |
--------------------------------------
10118 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://wilbur20.labs.corp.sp1.yahoo.com:802013
10143 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: wilbur20.labs.corp.sp1.yahoo.com:50020
java.lang.NullPointerException
at org.apache.pig.pen.LineageTrimmingVisitor.PruneBaseDataConstrainedCoverage(LineageTrimmingVisitor.java:290)
at org.apache.pig.pen.LineageTrimmingVisitor.checkCompleteness(LineageTrimmingVisitor.java:458)
at org.apache.pig.pen.LineageTrimmingVisitor.processOperator(LineageTrimmingVisitor.java:416)
at org.apache.pig.pen.LineageTrimmingVisitor.visit(LineageTrimmingVisitor.java:251)
at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:105)
at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:40)
at org.apache.pig.pen.util.PreOrderDepthFirstWalker.depthFirst(PreOrderDepthFirstWalker.java:69)
at org.apache.pig.pen.util.PreOrderDepthFirstWalker.depthFirst(PreOrderDepthFirstWalker.java:71)
at org.apache.pig.pen.util.PreOrderDepthFirstWalker.walk(PreOrderDepthFirstWalker.java:55)
at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
at org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:108)
at org.apache.pig.PigServer.getExamples(PigServer.java:627)
at org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:278)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:183)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:83)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:64)
at org.apache.pig.Main.main(Main.java:306)
{noformat}
> Illustrate for the Types branch
> -------------------------------
>
> Key: PIG-443
> URL: https://issues.apache.org/jira/browse/PIG-443
> Project: Pig
> Issue Type: New Feature
> Affects Versions: types_branch
> Reporter: Shubham Chopra
> Assignee: Shubham Chopra
> Fix For: types_branch
>
> Attachments: pig_types_22_09.patch, pig_types_23_09.patch, pig_types_24_09.patch
>
>
> This implementation of Illustrate uses visitors to traverse the plans.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.