You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Peter Connolly <pt...@yahoo.com> on 2013/05/03 21:01:26 UTC
ClassCastException with bincond operator
I'm new to pig and I'm getting a ClassCastException when I try to run the following script in pig 0.11.1:
A = LOAD 'test.log' AS (timestamp:long, pk_id:int, array_field:chararray, fk_id:int);
B = FOREACH A GENERATE timestamp, pk_id, FLATTEN(STRSPLIT(array_field,',')) AS (field1:chararray, field2:chararray, age:long), fk_id;
C = FOREACH B GENERATE timestamp-(age IS NULL ? (long) 0 : age) AS adjusted_timestamp:long, pk_id, fk_id;
DUMP C;
test.log contains only one row (I replaced tabs with \t):
1362178873552\t15404\ta,b,5\t6
I'm able to do a dump on B and get the results I expect, but when I do a dump/store on C I get this error:
java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Number
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Subtract.genericGetNext(Subtract.java:96)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Subtract.getNext(Subtract.java:119)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:348)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:372)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:297)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:283)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:278)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
I've tried casting every timestamp and age to a long in the definition of C and it still doesn't work.
Any ideas on how to fix the script?
Thanks,
Peter
Re: ClassCastException with bincond operator
Posted by Johnny Zhang <xi...@cloudera.com>.
Hi, Peter:
http://pig.apache.org/docs/r0.11.1/basic.html#cast
cast from String to long is not supported.
Johnny
On Fri, May 3, 2013 at 12:01 PM, Peter Connolly <pt...@yahoo.com> wrote:
> I'm new to pig and I'm getting a ClassCastException when I try to run the
> following script in pig 0.11.1:
>
>
> A = LOAD 'test.log' AS (timestamp:long, pk_id:int, array_field:chararray,
> fk_id:int);
>
> B = FOREACH A GENERATE timestamp, pk_id,
> FLATTEN(STRSPLIT(array_field,',')) AS (field1:chararray, field2:chararray,
> age:long), fk_id;
>
> C = FOREACH B GENERATE timestamp-(age IS NULL ? (long) 0 : age) AS
> adjusted_timestamp:long, pk_id, fk_id;
>
> DUMP C;
>
>
> test.log contains only one row (I replaced tabs with \t):
> 1362178873552\t15404\ta,b,5\t6
>
> I'm able to do a dump on B and get the results I expect, but when I do a
> dump/store on C I get this error:
> java.lang.ClassCastException: java.lang.String cannot be cast to
> java.lang.Number
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Subtract.genericGetNext(Subtract.java:96)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Subtract.getNext(Subtract.java:119)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:348)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:372)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:297)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:283)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:278)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
>
> I've tried casting every timestamp and age to a long in the definition of
> C and it still doesn't work.
>
> Any ideas on how to fix the script?
>
> Thanks,
> Peter
>