You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Swati Jain <sw...@aggiemail.usu.edu> on 2010/07/28 21:48:19 UTC
Group by is not working with Filter
Hello Everyone,
I am trying to execute below mentioned script, but it is throwing error.
Script is:
A = load 'ex_groupby' USING PigStorage(',') as (a1:int,a2:int,a3:int);
G1 = GROUP A by (a1,a2);
describe G1;
*D = Filter G1 by group.$0 > 1;*
dump D;
Error is :
java.lang.ClassCastException: java.lang.Integer cannot be cast to
org.apache.pig.data.Tuple
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:389)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:138)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:276)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.GreaterThanExpr.getNext(GreaterThanExpr.java:72)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POFilter.getNext(POFilter.java:148)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:259)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:236)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
Any help would be appreciated.
Thanks,
Swati
Re: Group by is not working with Filter
Posted by Mridul Muralidharan <mr...@yahoo-inc.com>.
On Thursday 29 July 2010 01:18 AM, Swati Jain wrote:
> Hello Everyone,
>
> I am trying to execute below mentioned script, but it is throwing error.
> Script is:
>
> A = load 'ex_groupby' USING PigStorage(',') as (a1:int,a2:int,a3:int);
> G1 = GROUP A by (a1,a2);
> describe G1;
> *D = Filter G1 by group.$0> 1;*
What about
D = Filter G1 by $0.$0 > 1;
Does that work/help ?
Regards,
Mridul
> dump D;
>
> Error is :
>
> java.lang.ClassCastException: java.lang.Integer cannot be cast to
> org.apache.pig.data.Tuple
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:389)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:138)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:276)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.GreaterThanExpr.getNext(GreaterThanExpr.java:72)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POFilter.getNext(POFilter.java:148)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:259)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:236)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
>
> Any help would be appreciated.
>
> Thanks,
> Swati
Re: Group by is not working with Filter
Posted by Thejas M Nair <te...@yahoo-inc.com>.
This looks like a bug , can you please open a jira ?
Adding following statements to this script get it to work -
grunt> f = foreach G1 generate group.a1 as a1, group.a2 as a2, A;
grunt> fil = filter f by $0 > 1;
grunt> dump fil;
-thejas
On 7/28/10 12:48 PM, "Swati Jain" <sw...@aggiemail.usu.edu> wrote:
Hello Everyone,
I am trying to execute below mentioned script, but it is throwing error.
Script is:
A = load 'ex_groupby' USING PigStorage(',') as (a1:int,a2:int,a3:int);
G1 = GROUP A by (a1,a2);
describe G1;
*D = Filter G1 by group.$0 > 1;*
dump D;
Error is :
java.lang.ClassCastException: java.lang.Integer cannot be cast to
org.apache.pig.data.Tuple
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:389)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:138)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:276)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.GreaterThanExpr.getNext(GreaterThanExpr.java:72)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POFilter.getNext(POFilter.java:148)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:259)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:236)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
Any help would be appreciated.
Thanks,
Swati