You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Shubham Chopra (JIRA)" <ji...@apache.org> on 2011/07/08 20:03:16 UTC
[jira] [Created] (PIG-2155) NPE thrown by ABS
NPE thrown by ABS
-----------------
Key: PIG-2155
URL: https://issues.apache.org/jira/browse/PIG-2155
Project: Pig
Issue Type: Bug
Affects Versions: 0.9.0
Reporter: Shubham Chopra
Input data :
--file 'a' starts--
A|1
B|2
--file 'a' ends--
(Note the empty line at the end)
The following script does not work.
a = load 'a' using PigStorage('|') as (x:chararray, y:double);
b = foreach a generate *, ABS(y - 2*y) as test;
dump b;
The function ABS throws a NPE instead of giving out a null for the last line in the input:
java.lang.NullPointerException
at org.apache.pig.builtin.DoubleAbs.exec(DoubleAbs.java:45)
at org.apache.pig.builtin.DoubleAbs.exec(DoubleAbs.java:28)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:216)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:281)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:324)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:332)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:284)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:261)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:256)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:58)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2155) NPE thrown by ABS
Posted by "Prashant Kommireddi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269441#comment-13269441 ]
Prashant Kommireddi commented on PIG-2155:
------------------------------------------
Hi Allan,
Thanks for looking at this. I ran the same but I do not see an empty space.
(A,1.0,1.0)
(B,2.0,2.0)
(,,)
Here is my input file contents
grunt> cat data1
A|1
B|2
<empty line>
> NPE thrown by ABS
> -----------------
>
> Key: PIG-2155
> URL: https://issues.apache.org/jira/browse/PIG-2155
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: Shubham Chopra
> Labels: newbie, simple
>
> Input data :
> --file 'a' starts--
> A|1
> B|2
> --file 'a' ends--
> (Note the empty line at the end)
> The following script does not work.
> a = load 'a' using PigStorage('|') as (x:chararray, y:double);
> b = foreach a generate *, ABS(y - 2*y) as test;
> dump b;
> The function ABS throws a NPE instead of giving out a null for the last line in the input:
> java.lang.NullPointerException
> at org.apache.pig.builtin.DoubleAbs.exec(DoubleAbs.java:45)
> at org.apache.pig.builtin.DoubleAbs.exec(DoubleAbs.java:28)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:216)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:281)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:324)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:332)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:284)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:261)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:256)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:58)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2155) NPE thrown by ABS
Posted by "Allan Avendaño (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269175#comment-13269175 ]
Allan Avendaño commented on PIG-2155:
-------------------------------------
I was trying to reproduce same scenario, but while I tried to run a file with empty space and/or break lines at the end of the file, the result was something like this:
(1.0,A,1.0)
(2.0,B,2.0)
(,,) <-- break line
(, ,) <-- empty space
And no exceptions were thrown. Maybe because I'm working on latest version, but in any case this results are accepted?
> NPE thrown by ABS
> -----------------
>
> Key: PIG-2155
> URL: https://issues.apache.org/jira/browse/PIG-2155
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: Shubham Chopra
> Labels: newbie, simple
>
> Input data :
> --file 'a' starts--
> A|1
> B|2
> --file 'a' ends--
> (Note the empty line at the end)
> The following script does not work.
> a = load 'a' using PigStorage('|') as (x:chararray, y:double);
> b = foreach a generate *, ABS(y - 2*y) as test;
> dump b;
> The function ABS throws a NPE instead of giving out a null for the last line in the input:
> java.lang.NullPointerException
> at org.apache.pig.builtin.DoubleAbs.exec(DoubleAbs.java:45)
> at org.apache.pig.builtin.DoubleAbs.exec(DoubleAbs.java:28)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:216)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:281)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:324)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:332)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:284)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:261)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:256)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:58)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (PIG-2155) NPE thrown by ABS
Posted by "Gianmarco De Francisci Morales (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gianmarco De Francisci Morales resolved PIG-2155.
-------------------------------------------------
Resolution: Cannot Reproduce
Fix Version/s: 0.10.1
Assignee: Gianmarco De Francisci Morales
It looks like this was fixed, I can't reproduce the bug.
The spaces in the tuple are correct as pointed out by Prashant.
Thanks for having a look at this, Allan.
> NPE thrown by ABS
> -----------------
>
> Key: PIG-2155
> URL: https://issues.apache.org/jira/browse/PIG-2155
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: Shubham Chopra
> Assignee: Gianmarco De Francisci Morales
> Labels: newbie, simple
> Fix For: 0.10.1
>
>
> Input data :
> --file 'a' starts--
> A|1
> B|2
> --file 'a' ends--
> (Note the empty line at the end)
> The following script does not work.
> a = load 'a' using PigStorage('|') as (x:chararray, y:double);
> b = foreach a generate *, ABS(y - 2*y) as test;
> dump b;
> The function ABS throws a NPE instead of giving out a null for the last line in the input:
> java.lang.NullPointerException
> at org.apache.pig.builtin.DoubleAbs.exec(DoubleAbs.java:45)
> at org.apache.pig.builtin.DoubleAbs.exec(DoubleAbs.java:28)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:216)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:281)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:324)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:332)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:284)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:261)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:256)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:58)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2155) NPE thrown by ABS
Posted by "Prashant Kommireddi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13272159#comment-13272159 ]
Prashant Kommireddi commented on PIG-2155:
------------------------------------------
I think the <space> is coming from projecting all fields (*)
{code}
b = foreach a generate *, ABS(y - 2*y) as test;
{code}
Since the script is parsing on '|', it makes sense that space char is output.
> NPE thrown by ABS
> -----------------
>
> Key: PIG-2155
> URL: https://issues.apache.org/jira/browse/PIG-2155
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: Shubham Chopra
> Labels: newbie, simple
>
> Input data :
> --file 'a' starts--
> A|1
> B|2
> --file 'a' ends--
> (Note the empty line at the end)
> The following script does not work.
> a = load 'a' using PigStorage('|') as (x:chararray, y:double);
> b = foreach a generate *, ABS(y - 2*y) as test;
> dump b;
> The function ABS throws a NPE instead of giving out a null for the last line in the input:
> java.lang.NullPointerException
> at org.apache.pig.builtin.DoubleAbs.exec(DoubleAbs.java:45)
> at org.apache.pig.builtin.DoubleAbs.exec(DoubleAbs.java:28)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:216)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:281)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:324)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:332)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:284)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:261)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:256)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:58)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2155) NPE thrown by ABS
Posted by "Allan Avendaño (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13272155#comment-13272155 ]
Allan Avendaño commented on PIG-2155:
-------------------------------------
Hi Prashant,
My input file was this:
grunt> cat data/abs
cat data/abs
A|1
B|2
<break line>
<spaces>
and output was this
(A,1.0,1.0)
(B,2.0,2.0)
(,,)
( ,,)
and any exception was thrown as is described.
If there are spaces or break lines in the input, should be shown in the output or not?
> NPE thrown by ABS
> -----------------
>
> Key: PIG-2155
> URL: https://issues.apache.org/jira/browse/PIG-2155
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: Shubham Chopra
> Labels: newbie, simple
>
> Input data :
> --file 'a' starts--
> A|1
> B|2
> --file 'a' ends--
> (Note the empty line at the end)
> The following script does not work.
> a = load 'a' using PigStorage('|') as (x:chararray, y:double);
> b = foreach a generate *, ABS(y - 2*y) as test;
> dump b;
> The function ABS throws a NPE instead of giving out a null for the last line in the input:
> java.lang.NullPointerException
> at org.apache.pig.builtin.DoubleAbs.exec(DoubleAbs.java:45)
> at org.apache.pig.builtin.DoubleAbs.exec(DoubleAbs.java:28)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:216)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:281)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:324)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:332)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:284)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:261)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:256)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:58)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira