You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by pob <pe...@gmail.com> on 2011/04/24 18:04:26 UTC
PIG + SUM
Hello,
Im trying these with pig 0.8 + c* 0.7.5 (branch).
Has anybody any idea?
Thanks.
x = foreach g2 generate group, data.(size);
dump x;
((drm,0),{(464868)})
((drm,1),{(464868)})
((snezz,0),{(8073),(8073)})
but:
x = foreach g2 generate group, SUM(data.size);
----
grunt> describe data;
data: {y::domain: chararray,y::spam: int,y::size: long,y::time: float}
grunt> describe g2
g2: {group: (y::domain: chararray,y::spam: int),data: {y::domain:
chararray,y::spam: int,y::size: long,y::time: float}}
grunt>
2011-04-24 18:02:18,910 [Thread-793] WARN
org.apache.hadoop.mapred.LocalJobRunner - job_local_0038
org.apache.pig.backend.executionengine.ExecException: ERROR 2106: Error
while computing sum in Initial
at org.apache.pig.builtin.LongSum$Initial.exec(LongSum.java:87)
at org.apache.pig.builtin.LongSum$Initial.exec(LongSum.java:65)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:229)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:273)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:343)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:291)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:256)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:236)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
Caused by: java.lang.ClassCastException: org.apache.pig.data.DataByteArray
cannot be cast to java.lang.Long
at org.apache.pig.builtin.LongSum$Initial.exec(LongSum.java:79)
... 14 more
2011-04-24 18:02:19,213 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- HadoopJobId: job_local_0038
2011-04-24 18:02:19,213 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 0% complete
2011-04-24 18:02:24,215 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- job job_local_0038 has failed! Stop running all dependent jobs
2011-04-24 18:02:24,216 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 100% complete
2011-04-24 18:02:24,216 [main] ERROR
org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
2011-04-24 18:02:24,216 [main] INFO org.apache.pig.tools.pigstats.PigStats
- Detected Local mode. Stats reported below may be incomplete
2011-04-24 18:02:24,216 [main] INFO org.apache.pig.tools.pigstats.PigStats
- Script Statistics:
Pig Stack Trace
---------------
ERROR 1066: Unable to open iterator for alias x
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
open iterator for alias x
at org.apache.pig.PigServer.openIterator(PigServer.java:754)
at
org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:76)
at org.apache.pig.Main.run(Main.java:465)
at org.apache.pig.Main.main(Main.java:107)
Caused by: java.io.IOException: Job terminated with anomalous status FAILED
at org.apache.pig.PigServer.openIterator(PigServer.java:744)
... 7 more
Re: PIG + SUM
Posted by pob <pe...@gmail.com>.
Even tho this simple thing doesnt work too:
grunt> z = foreach data generate time+size;
WARN org.apache.hadoop.mapred.LocalJobRunner - job_local_0004
java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be
cast to java.lang.Float
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Add.getNext(Add.java:97)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:364)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:291)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:236)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
2011/4/24 pob <pe...@gmail.com>
> Hello,
>
> Im trying these with pig 0.8 + c* 0.7.5 (branch).
>
>
> Has anybody any idea?
>
> Thanks.
>
>
>
> x = foreach g2 generate group, data.(size);
> dump x;
>
> ((drm,0),{(464868)})
> ((drm,1),{(464868)})
> ((snezz,0),{(8073),(8073)})
>
> but:
> x = foreach g2 generate group, SUM(data.size);
>
>
> ----
> grunt> describe data;
> data: {y::domain: chararray,y::spam: int,y::size: long,y::time: float}
> grunt> describe g2
> g2: {group: (y::domain: chararray,y::spam: int),data: {y::domain:
> chararray,y::spam: int,y::size: long,y::time: float}}
> grunt>
>
>
>
>
> 2011-04-24 18:02:18,910 [Thread-793] WARN
> org.apache.hadoop.mapred.LocalJobRunner - job_local_0038
> org.apache.pig.backend.executionengine.ExecException: ERROR 2106: Error
> while computing sum in Initial
> at org.apache.pig.builtin.LongSum$Initial.exec(LongSum.java:87)
> at org.apache.pig.builtin.LongSum$Initial.exec(LongSum.java:65)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:229)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:273)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:343)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:291)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:256)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:236)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
> Caused by: java.lang.ClassCastException: org.apache.pig.data.DataByteArray
> cannot be cast to java.lang.Long
> at org.apache.pig.builtin.LongSum$Initial.exec(LongSum.java:79)
> ... 14 more
> 2011-04-24 18:02:19,213 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - HadoopJobId: job_local_0038
> 2011-04-24 18:02:19,213 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 0% complete
> 2011-04-24 18:02:24,215 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - job job_local_0038 has failed! Stop running all dependent jobs
> 2011-04-24 18:02:24,216 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 100% complete
> 2011-04-24 18:02:24,216 [main] ERROR
> org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
> 2011-04-24 18:02:24,216 [main] INFO org.apache.pig.tools.pigstats.PigStats
> - Detected Local mode. Stats reported below may be incomplete
> 2011-04-24 18:02:24,216 [main] INFO org.apache.pig.tools.pigstats.PigStats
> - Script Statistics:
>
>
>
>
> Pig Stack Trace
> ---------------
> ERROR 1066: Unable to open iterator for alias x
>
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
> open iterator for alias x
> at org.apache.pig.PigServer.openIterator(PigServer.java:754)
> at
> org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:76)
> at org.apache.pig.Main.run(Main.java:465)
> at org.apache.pig.Main.main(Main.java:107)
> Caused by: java.io.IOException: Job terminated with anomalous status FAILED
> at org.apache.pig.PigServer.openIterator(PigServer.java:744)
> ... 7 more
>
>
>