You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Daniel Dai (JIRA)" <ji...@apache.org> on 2009/08/06 00:09:14 UTC
[jira] Commented: (PIG-910) PushUpFilter: Filter after skew LOJoin
generates an error
[ https://issues.apache.org/jira/browse/PIG-910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12739783#action_12739783 ]
Daniel Dai commented on PIG-910:
--------------------------------
I discussed with Santhosh, this issue is addressed by PIG-697 phase 5 patch (LOJoin patch). PIG-910-1.patch is a temporary solution before LOJoin patch, and is not intended to commit.
> PushUpFilter: Filter after skew LOJoin generates an error
> ---------------------------------------------------------
>
> Key: PIG-910
> URL: https://issues.apache.org/jira/browse/PIG-910
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.3.0
> Reporter: Daniel Dai
> Fix For: 0.4.0
>
> Attachments: PIG-910-1.patch
>
>
> The script is:
> A = load 'a' as (user:chararray, action, timespent, query_term, ip_addr, timestamp, estimated_revenue, page_info, page_links);
> alpha = load 'b' as (user:chararray, action, timespent, query_term, ip_addr, timestamp, estimated_revenue, page_info, page_links);
> C = join A by user, alpha by user using "skewed" parallel 40;
> D = filter C by A::user == 'aaa';
> store D into 'L5out';
> Stack trace is:
> ERROR 1093: Swap supports swap of operators with at most one input. Found first operator with 2 inputs.
> org.apache.pig.impl.plan.optimizer.OptimizerException: ERROR 2151:
> Internal error while pushing filters up.
> at org.apache.pig.impl.logicalLayer.optimizer.PushUpFilter.transform(PushUpFilter.java: 325)
> at org.apache.pig.impl.logicalLayer.optimizer.LogicalOptimizer.optimize(LogicalOptimizer.java:185)
> at org.apache.pig.PigServer.compileLp(PigServer.java:854)
> at org.apache.pig.PigServer.compileLp(PigServer.java:791)
> at org.apache.pig.PigServer.execute(PigServer.java:760)
> at org.apache.pig.PigServer.access$100(PigServer.java:95)
> at org.apache.pig.PigServer$Graph.execute(PigServer.java:943)
> at org.apache.pig.PigServer.executeBatch(PigServer.java:255)
> at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java: 112)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java: 169)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java: 141)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
> at org.apache.pig.Main.main(Main.java:320)
> Caused by: org.apache.pig.impl.plan.PlanException: ERROR 1093: Swap supports swap of operators with at most one input. Found first operator with 2 inputs.
> at org.apache.pig.impl.plan.OperatorPlan.swap(OperatorPlan.java:964)
> at org.apache.pig.impl.logicalLayer.optimizer.PushUpFilter.transform(PushUpFilter.java: 310)
> ... 12 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.