You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Alex Parvulescu (JIRA)" <ji...@apache.org> on 2010/02/11 09:30:28 UTC
[jira] Created: (PIG-1236) NPE when trying to cast an AVG null
value to int
NPE when trying to cast an AVG null value to int
-------------------------------------------------
Key: PIG-1236
URL: https://issues.apache.org/jira/browse/PIG-1236
Project: Pig
Issue Type: Bug
Components: grunt
Affects Versions: 0.6.0
Environment: Hadoop 0.20.1, Ubuntu 9.04 , Sun Java 1.6.0_16
Reporter: Alex Parvulescu
When you do AVG on a set of values that contains nulls it will return a null value. If you cast the AVG to int, not knowing that it will return null, you'll get a NPE.
What I did it is:
'C = FOREACH B GENERATE group, (int)AVG(A.v) as statsavg;'
The problem here is the AVG ran into some null values and returned null. And consequently the cast failed with a NPE.
This is the stacktrace
2010-02-09 11:14:36,444 [Thread-85] WARN org.apache.hadoop.mapred.LocalJobRunner - job_local_0006
java.lang.NullPointerException
at org.apache.pig.builtin.IntAvg.getValue(IntAvg.java:282)
at org.apache.pig.builtin.IntAvg.getValue(IntAvg.java:39)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:208)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:281)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast.getNext(POCast.java:182)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:352)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:277)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.runPipeline(PigMapReduce.java:423)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.processOnePackageOutput(PigMapReduce.java:391)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:371)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:239)
at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:463)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:411)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:215)
Now, because I'm not well aware how this works, I did not realize that the cast throws the NPE and not the computation of the average function on null values provided by the data set.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (PIG-1236) NPE when trying to cast an AVG null
value to int
Posted by "Alex Parvulescu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Parvulescu resolved PIG-1236.
----------------------------------
Resolution: Duplicate
> NPE when trying to cast an AVG null value to int
> -------------------------------------------------
>
> Key: PIG-1236
> URL: https://issues.apache.org/jira/browse/PIG-1236
> Project: Pig
> Issue Type: Bug
> Components: grunt
> Affects Versions: 0.6.0
> Environment: Hadoop 0.20.1, Ubuntu 9.04 , Sun Java 1.6.0_16
> Reporter: Alex Parvulescu
>
> When you do AVG on a set of values that contains nulls it will return a null value. If you cast the AVG to int, not knowing that it will return null, you'll get a NPE.
> What I did it is:
> 'C = FOREACH B GENERATE group, (int)AVG(A.v) as statsavg;'
> The problem here is the AVG ran into some null values and returned null. And consequently the cast failed with a NPE.
> This is the stacktrace
> 2010-02-09 11:14:36,444 [Thread-85] WARN org.apache.hadoop.mapred.LocalJobRunner - job_local_0006
> java.lang.NullPointerException
> at org.apache.pig.builtin.IntAvg.getValue(IntAvg.java:282)
> at org.apache.pig.builtin.IntAvg.getValue(IntAvg.java:39)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:208)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:281)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast.getNext(POCast.java:182)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:352)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:277)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.runPipeline(PigMapReduce.java:423)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.processOnePackageOutput(PigMapReduce.java:391)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:371)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:239)
> at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:463)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:411)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:215)
> Now, because I'm not well aware how this works, I did not realize that the cast throws the NPE and not the computation of the average function on null values provided by the data set.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.