[jira] [Resolved] (PIG-2570) LimitOptimizer fails with dynamic LIMIT argument

["Daniel Dai (Resolved) (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-2570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Daniel Dai resolved PIG-2570.
-----------------------------

      Resolution: Fixed
        Assignee: Daniel Dai  (was: Bill Graham)
    Hadoop Flags: Reviewed

test-patch:
     [exec] -1 overall.  
     [exec] 
     [exec]     +1 @author.  The patch does not contain any @author tags.
     [exec] 
     [exec]     +1 tests included.  The patch appears to include 3 new or modified tests.
     [exec] 
     [exec]     -1 javadoc.  The javadoc tool appears to have generated 1 warning messages.
     [exec] 
     [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
     [exec] 
     [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
     [exec] 
     [exec]     -1 release audit.  The applied patch generated 535 release audit warnings (more than the trunk's current 532 warnings).

javadoc and release audit warning is unrelated. 

Patch committed to 0.10/trunk.
                
> LimitOptimizer fails with dynamic LIMIT argument
> ------------------------------------------------
>
>                 Key: PIG-2570
>                 URL: https://issues.apache.org/jira/browse/PIG-2570
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Bill Graham
>            Assignee: Daniel Dai
>             Fix For: 0.10, 0.11
>
>         Attachments: PIG-2570-1.patch, PIG-2570-2.patch, data1.txt, limit-fails.pig
>
>
> The following script fails with the exception shown below. Passing {{-t LimitOptimizer}} makes it work, as does adding an {{ORDER}} clause before the limit.
> {noformat}
> A = LOAD 'data1.txt' AS (owner:chararray,pet:chararray,age:int,phone:chararray);
> B = group A all; 
> C = foreach B generate SUM(A.age) as total; 
> D = foreach A generate owner, age/(double)C.total AS percentAge; 
> F = LIMIT D C.total/8;
> DUMP F;
> {noformat}
> {noformat}
> Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 2000: Error processing rule LimitOptimizer. Try -t LimitOptimizer
> 	at org.apache.pig.newplan.optimizer.PlanOptimizer.optimize(PlanOptimizer.java:122)
> 	at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:287)
> 	at org.apache.pig.PigServer.compilePp(PigServer.java:1317)
> 	at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1254)
> 	at org.apache.pig.PigServer.storeEx(PigServer.java:953)
> 	... 14 more
> Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 2243: Attempt to remove operator LOLimit that is still softly connected in the plan
> 	at org.apache.pig.newplan.BaseOperatorPlan.remove(BaseOperatorPlan.java:174)
> 	at org.apache.pig.newplan.BaseOperatorPlan.removeAndReconnect(BaseOperatorPlan.java:449)
> 	at org.apache.pig.newplan.logical.rules.LimitOptimizer$OptimizeLimitTransformer.transform(LimitOptimizer.java:124)
> 	at org.apache.pig.newplan.optimizer.PlanOptimizer.optimize(PlanOptimizer.java:110)
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira