You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Chinna Rao Lalam (JIRA)" <ji...@apache.org> on 2011/05/25 08:17:47 UTC
[jira] [Created] (HIVE-2182) Avoid null pointer exception when
executing UDF
Avoid null pointer exception when executing UDF
-----------------------------------------------
Key: HIVE-2182
URL: https://issues.apache.org/jira/browse/HIVE-2182
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.5.0, 0.8.0
Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
Reporter: Chinna Rao Lalam
Assignee: Chinna Rao Lalam
For using UDF's executed following steps
{noformat}
add jar /home/udf/udf.jar;
create temporary function grade as 'udf.Grade';
select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
{noformat}
But from the above steps if we miss the first step (add jar) and execute remaining steps
{noformat}
create temporary function grade as 'udf.Grade';
select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
{noformat}
In tasktracker it is throwing this exception
{noformat}
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
... 18 more
Caused by: java.lang.RuntimeException: java.lang.NullPointerException
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
... 18 more
Caused by: java.lang.NullPointerException
at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
... 31 more
{noformat}
Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "Chinna Rao Lalam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13043356#comment-13043356 ]
Chinna Rao Lalam commented on HIVE-2182:
----------------------------------------
Hi He Yongqiang,
I feel this validation is not possible on client side because the client side validation cannot
ensure the jar presence in task tracker classpath.
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103803#comment-13103803 ]
jiraposter@reviews.apache.org commented on HIVE-2182:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1759/
-----------------------------------------------------------
(Updated 2011-09-13 17:32:55.124065)
Review request for hive and John Sichi.
Changes
-------
Patch rebased on trunk
Summary
-------
while executing UDF if the implementation jar is not present in MR class path it is throwing nullpointer exception instead of throwing nullpointer exception throwing meaning full exception with the required details.
This addresses bug HIVE-2182.
https://issues.apache.org/jira/browse/HIVE-2182
Diffs (updated)
-----
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBridge.java 1170044
trunk/ql/src/test/queries/clientnegative/udfnull.q PRE-CREATION
trunk/ql/src/test/results/clientnegative/udfnull.q.out PRE-CREATION
trunk/ql/src/test/results/compiler/plan/cast1.q.xml 1170044
trunk/ql/src/test/results/compiler/plan/groupby1.q.xml 1170044
trunk/ql/src/test/results/compiler/plan/groupby2.q.xml 1170044
trunk/ql/src/test/results/compiler/plan/groupby3.q.xml 1170044
trunk/ql/src/test/results/compiler/plan/groupby4.q.xml 1170044
trunk/ql/src/test/results/compiler/plan/groupby5.q.xml 1170044
trunk/ql/src/test/results/compiler/plan/groupby6.q.xml 1170044
trunk/ql/src/test/results/compiler/plan/input20.q.xml 1170044
trunk/ql/src/test/results/compiler/plan/input8.q.xml 1170044
trunk/ql/src/test/results/compiler/plan/join2.q.xml 1170044
trunk/ql/src/test/results/compiler/plan/sample1.q.xml 1170044
trunk/ql/src/test/results/compiler/plan/sample2.q.xml 1170044
trunk/ql/src/test/results/compiler/plan/sample3.q.xml 1170044
trunk/ql/src/test/results/compiler/plan/sample4.q.xml 1170044
trunk/ql/src/test/results/compiler/plan/sample5.q.xml 1170044
trunk/ql/src/test/results/compiler/plan/sample6.q.xml 1170044
trunk/ql/src/test/results/compiler/plan/sample7.q.xml 1170044
trunk/ql/src/test/results/compiler/plan/udf1.q.xml 1170044
trunk/ql/src/test/results/compiler/plan/udf4.q.xml 1170044
trunk/ql/src/test/results/compiler/plan/udf6.q.xml 1170044
Diff: https://reviews.apache.org/r/1759/diff
Testing
-------
Added unit tests
Thanks,
chinna
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.1.patch, HIVE-2182.2.patch, HIVE-2182.3.patch, HIVE-2182.4.patch, HIVE-2182.5.patch, HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "Chinna Rao Lalam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chinna Rao Lalam updated HIVE-2182:
-----------------------------------
Status: Patch Available (was: Open)
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.1.patch, HIVE-2182.2.patch, HIVE-2182.3.patch, HIVE-2182.4.patch, HIVE-2182.5.patch, HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13105025#comment-13105025 ]
John Sichi commented on HIVE-2182:
----------------------------------
Oops, looks like I typed in the wrong JIRA issue number in the commit message (I typed in HIVE-2181 instead of HIVE-2182), so the Hudson commit message went there instead. I've fixed it in the svn log though.
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Fix For: 0.9.0
>
> Attachments: HIVE-2182.1.patch, HIVE-2182.2.patch, HIVE-2182.3.patch, HIVE-2182.4.patch, HIVE-2182.5.patch, HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "Chinna Rao Lalam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chinna Rao Lalam updated HIVE-2182:
-----------------------------------
Attachment: HIVE-2182.3.patch
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.1.patch, HIVE-2182.2.patch, HIVE-2182.3.patch, HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "Chinna Rao Lalam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chinna Rao Lalam updated HIVE-2182:
-----------------------------------
Status: Patch Available (was: Open)
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.1.patch, HIVE-2182.2.patch, HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13100631#comment-13100631 ]
jiraposter@reviews.apache.org commented on HIVE-2182:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1759/
-----------------------------------------------------------
Review request for hive and John Sichi.
Summary
-------
while executing UDF if the implementation jar is not present in MR class path it is throwing nullpointer exception instead of throwing nullpointer exception throwing meaning full exception with the required details.
This addresses bug HIVE-2182.
https://issues.apache.org/jira/browse/HIVE-2182
Diffs
-----
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBridge.java 1166865
trunk/ql/src/test/queries/clientnegative/udfnull.q PRE-CREATION
trunk/ql/src/test/results/clientnegative/udfnull.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/1759/diff
Testing
-------
Added unit tests
Thanks,
chinna
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.1.patch, HIVE-2182.2.patch, HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "Chinna Rao Lalam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chinna Rao Lalam updated HIVE-2182:
-----------------------------------
Attachment: HIVE-2182.5.patch
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.1.patch, HIVE-2182.2.patch, HIVE-2182.3.patch, HIVE-2182.4.patch, HIVE-2182.5.patch, HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi updated HIVE-2182:
-----------------------------
Status: Open (was: Patch Available)
Can you add the test case back in? Also create a review board request?
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.1.patch, HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi updated HIVE-2182:
-----------------------------
Resolution: Fixed
Fix Version/s: 0.9.0
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Committed to trunk. Thanks Chinna!
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Fix For: 0.9.0
>
> Attachments: HIVE-2182.1.patch, HIVE-2182.2.patch, HIVE-2182.3.patch, HIVE-2182.4.patch, HIVE-2182.5.patch, HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "He Yongqiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
He Yongqiang updated HIVE-2182:
-------------------------------
Status: Open (was: Patch Available)
cancel this patch now, please resubmit after opening a new jira for client side processing or fix it in this jira. Thanks!
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13101508#comment-13101508 ]
John Sichi commented on HIVE-2182:
----------------------------------
+1. Will commit when tests pass.
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.1.patch, HIVE-2182.2.patch, HIVE-2182.3.patch, HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "Chinna Rao Lalam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chinna Rao Lalam updated HIVE-2182:
-----------------------------------
Status: Patch Available (was: Open)
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "Chinna Rao Lalam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chinna Rao Lalam updated HIVE-2182:
-----------------------------------
Status: Patch Available (was: Open)
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094195#comment-13094195 ]
John Sichi commented on HIVE-2182:
----------------------------------
I agree with Chinna that in general we don't know whether the client-side class availability matches the MR availability, so this extra check is useful.
However, can we improve the diagnostics? At a minimum, we want to know what class is missing; if we can also say which .jar we saw it in on the client-side, that would be great.
Code format nitpick: put opening curly brace on same line as if keyword.
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "Chinna Rao Lalam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13102808#comment-13102808 ]
Chinna Rao Lalam commented on HIVE-2182:
----------------------------------------
Sorry...TestParse.java is failing because of the plan xmls. In HIVE-2182.4 updated with the latest plan xmls. Pls check this patch.
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.1.patch, HIVE-2182.2.patch, HIVE-2182.3.patch, HIVE-2182.4.patch, HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13101501#comment-13101501 ]
John Sichi commented on HIVE-2182:
----------------------------------
Oops, sorry, ignore comment above...I misapplied the latest patch.
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.1.patch, HIVE-2182.2.patch, HIVE-2182.3.patch, HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "Chinna Rao Lalam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13092781#comment-13092781 ]
Chinna Rao Lalam commented on HIVE-2182:
----------------------------------------
client side validation we can do while creating the temporary function this scenario is already take care in the existing functionality. While creating temporary function it is throwing exception if the UDF jar not in class path(Hive Class path). If the requested jar in client class path and not in the executed MR class path this issue will come example if the UDF jar maintained in the HIVE/LIB and not executed "add jar" or not updated "hive.aux.jars.path" this issue will come.
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi updated HIVE-2182:
-----------------------------
Status: Open (was: Patch Available)
I got merge conflicts trying to apply the latest patch.
At revision 1170007.
(Stripping trailing CRs from patch.)
patching file ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBridge.java
(Stripping trailing CRs from patch.)
patching file ql/src/test/queries/clientnegative/udfnull.q
(Stripping trailing CRs from patch.)
patching file ql/src/test/results/clientnegative/udfnull.q.out
(Stripping trailing CRs from patch.)
patching file ql/src/test/results/compiler/plan/cast1.q.xml
Hunk #2 FAILED at 62.
Hunk #3 FAILED at 124.
Hunk #4 FAILED at 160.
Hunk #5 succeeded at 371 (offset 4 lines).
Hunk #7 succeeded at 455 (offset 4 lines).
Hunk #9 succeeded at 526 (offset 4 lines).
Hunk #11 succeeded at 622 (offset 4 lines).
Hunk #13 succeeded at 1066 (offset 4 lines).
Hunk #15 FAILED at 1131.
Hunk #16 FAILED at 1193.
5 out of 16 hunks FAILED -- saving rejects to file ql/src/test/results/compiler/plan/cast1.q.xml.rej
(Stripping trailing CRs from patch.)
patching file ql/src/test/results/compiler/plan/groupby1.q.xml
(Stripping trailing CRs from patch.)
patching file ql/src/test/results/compiler/plan/groupby2.q.xml
Hunk #13 succeeded at 1408 (offset 4 lines).
(Stripping trailing CRs from patch.)
patching file ql/src/test/results/compiler/plan/groupby3.q.xml
(Stripping trailing CRs from patch.)
patching file ql/src/test/results/compiler/plan/groupby4.q.xml
(Stripping trailing CRs from patch.)
patching file ql/src/test/results/compiler/plan/groupby5.q.xml
(Stripping trailing CRs from patch.)
patching file ql/src/test/results/compiler/plan/groupby6.q.xml
(Stripping trailing CRs from patch.)
patching file ql/src/test/results/compiler/plan/input20.q.xml
Hunk #1 FAILED at 1.
Hunk #2 FAILED at 62.
Hunk #3 FAILED at 124.
Hunk #6 FAILED at 850.
Hunk #7 FAILED at 862.
Hunk #8 FAILED at 919.
Hunk #9 FAILED at 981.
Hunk #10 FAILED at 1015.
8 out of 10 hunks FAILED -- saving rejects to file ql/src/test/results/compiler/plan/input20.q.xml.rej
(Stripping trailing CRs from patch.)
patching file ql/src/test/results/compiler/plan/input8.q.xml
Hunk #1 FAILED at 1.
Hunk #2 FAILED at 62.
Hunk #3 FAILED at 124.
Hunk #4 FAILED at 156.
Hunk #5 succeeded at 314 (offset 4 lines).
Hunk #7 succeeded at 403 (offset 4 lines).
Hunk #8 FAILED at 641.
Hunk #9 FAILED at 653.
Hunk #10 FAILED at 710.
Hunk #11 FAILED at 772.
8 out of 11 hunks FAILED -- saving rejects to file ql/src/test/results/compiler/plan/input8.q.xml.rej
(Stripping trailing CRs from patch.)
patching file ql/src/test/results/compiler/plan/join2.q.xml
(Stripping trailing CRs from patch.)
patching file ql/src/test/results/compiler/plan/sample1.q.xml
Hunk #5 succeeded at 555 (offset 4 lines).
Hunk #7 succeeded at 639 (offset 4 lines).
Hunk #9 succeeded at 885 (offset 4 lines).
Hunk #11 succeeded at 1021 (offset 4 lines).
(Stripping trailing CRs from patch.)
patching file ql/src/test/results/compiler/plan/sample2.q.xml
(Stripping trailing CRs from patch.)
patching file ql/src/test/results/compiler/plan/sample3.q.xml
(Stripping trailing CRs from patch.)
patching file ql/src/test/results/compiler/plan/sample4.q.xml
(Stripping trailing CRs from patch.)
patching file ql/src/test/results/compiler/plan/sample5.q.xml
(Stripping trailing CRs from patch.)
patching file ql/src/test/results/compiler/plan/sample6.q.xml
(Stripping trailing CRs from patch.)
patching file ql/src/test/results/compiler/plan/sample7.q.xml
(Stripping trailing CRs from patch.)
patching file ql/src/test/results/compiler/plan/udf1.q.xml
Hunk #5 succeeded at 510 (offset 4 lines).
Hunk #7 succeeded at 606 (offset 4 lines).
Hunk #9 succeeded at 702 (offset 4 lines).
Hunk #11 succeeded at 798 (offset 4 lines).
Hunk #13 succeeded at 894 (offset 4 lines).
Hunk #15 succeeded at 997 (offset 4 lines).
Hunk #17 succeeded at 1093 (offset 4 lines).
Hunk #19 succeeded at 1203 (offset 4 lines).
Hunk #21 succeeded at 1306 (offset 4 lines).
Hunk #23 succeeded at 1904 (offset 4 lines).
Hunk #25 succeeded at 2023 (offset 4 lines).
(Stripping trailing CRs from patch.)
patching file ql/src/test/results/compiler/plan/udf4.q.xml
Hunk #5 succeeded at 523 (offset 4 lines).
Hunk #7 succeeded at 585 (offset 4 lines).
Hunk #9 succeeded at 662 (offset 4 lines).
Hunk #11 succeeded at 717 (offset 4 lines).
Hunk #13 succeeded at 794 (offset 4 lines).
Hunk #15 succeeded at 849 (offset 4 lines).
Hunk #17 succeeded at 919 (offset 4 lines).
Hunk #19 succeeded at 996 (offset 4 lines).
Hunk #21 succeeded at 1051 (offset 4 lines).
Hunk #23 succeeded at 1126 (offset 4 lines).
Hunk #25 succeeded at 1212 (offset 4 lines).
Hunk #27 succeeded at 1296 (offset 4 lines).
Hunk #29 succeeded at 1846 (offset 4 lines).
Hunk #31 succeeded at 1965 (offset 4 lines).
(Stripping trailing CRs from patch.)
patching file ql/src/test/results/compiler/plan/udf6.q.xml
Hunk #5 succeeded at 357 (offset 4 lines).
Hunk #7 succeeded at 584 (offset 4 lines).
Hunk #9 succeeded at 703 (offset 4 lines).
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.1.patch, HIVE-2182.2.patch, HIVE-2182.3.patch, HIVE-2182.4.patch, HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "Chinna Rao Lalam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chinna Rao Lalam updated HIVE-2182:
-----------------------------------
Status: Patch Available (was: Open)
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.1.patch, HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "Chinna Rao Lalam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chinna Rao Lalam updated HIVE-2182:
-----------------------------------
Status: Patch Available (was: Open)
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.1.patch, HIVE-2182.2.patch, HIVE-2182.3.patch, HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "Chinna Rao Lalam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13100117#comment-13100117 ]
Chinna Rao Lalam commented on HIVE-2182:
----------------------------------------
While de serializing the udfClass variable is becoming null because the class is not present in MR classpath.
In user logs it is throwing the following exception
java.lang.ClassNotFoundException: com.samples.hive.udf.Grade
Continuing ...
So i have introduced one new variable to hold the udf class name like "udfClassName".
Now in the exception the class name will be displayed in the following way
Caused by: org.apache.hadoop.hive.ql.exec.UDFArgumentException: The UDF Implementation class 'com.samples.hive.udf.Grade' is Not in class path
at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:141)
at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:896)
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "Chinna Rao Lalam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13101396#comment-13101396 ]
Chinna Rao Lalam commented on HIVE-2182:
----------------------------------------
Sorry...Now updated the q.out file against the latest trunk
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.1.patch, HIVE-2182.2.patch, HIVE-2182.3.patch, HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi updated HIVE-2182:
-----------------------------
Status: Open (was: Patch Available)
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "Chinna Rao Lalam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chinna Rao Lalam updated HIVE-2182:
-----------------------------------
Attachment: HIVE-2182.patch
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13101499#comment-13101499 ]
John Sichi commented on HIVE-2182:
----------------------------------
It's still failing for me with the latest patch. Did you use -Doverwrite=true to regenerate the log?
{noformat}
[junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I grantTime -I [.][.][.] [0-9]* more -I job_[0-9]*_[0-9]* -I USING 'java -cp /data/users/jsichi/open/test-trunk/build/ql/test/logs/clientnegative/udfnull.q.out /data/users/jsichi/open/test-trunk/ql/src/test/results/clientnegative/udfnull.q.out
[junit] 18c18,27
[junit] < /data/users/jsichi/open/test-trunk/build/ql/tmp//hive.log
[junit] ---
[junit] > /home/opensrc/9thsep/build/ql/tmp//hive.log
[junit] > FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
[junit] > PREHOOK: query: CREATE TEMPORARY FUNCTION example_arraysum AS 'org.apache.hadoop.hive.contrib.udf.example.UDFExampleArraySum'
[junit] > PREHOOK: type: CREATEFUNCTION
[junit] > POSTHOOK: query: CREATE TEMPORARY FUNCTION example_arraysum AS 'org.apache.hadoop.hive.contrib.udf.example.UDFExampleArraySum'
[junit] > POSTHOOK: type: CREATEFUNCTION
[junit] > PREHOOK: query: SELECT example_arraysum(lint)FROM src_thrift
[junit] > PREHOOK: type: QUERY
[junit] > PREHOOK: Input: default@src_thrift
[junit] > PREHOOK: Output: file:/tmp/root/hive_2011-05-25_10-05-57_126_4632621650656424226/-mr-10000
{noformat}
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.1.patch, HIVE-2182.2.patch, HIVE-2182.3.patch, HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13101405#comment-13101405 ]
jiraposter@reviews.apache.org commented on HIVE-2182:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1759/
-----------------------------------------------------------
(Updated 2011-09-09 18:08:07.528830)
Review request for hive and John Sichi.
Summary
-------
while executing UDF if the implementation jar is not present in MR class path it is throwing nullpointer exception instead of throwing nullpointer exception throwing meaning full exception with the required details.
This addresses bug HIVE-2182.
https://issues.apache.org/jira/browse/HIVE-2182
Diffs (updated)
-----
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBridge.java 1167000
trunk/ql/src/test/queries/clientnegative/udfnull.q PRE-CREATION
trunk/ql/src/test/results/clientnegative/udfnull.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/1759/diff
Testing
-------
Added unit tests
Thanks,
chinna
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.1.patch, HIVE-2182.2.patch, HIVE-2182.3.patch, HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "He Yongqiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13043041#comment-13043041 ]
He Yongqiang commented on HIVE-2182:
------------------------------------
Instead of throwing the exception at runtime after starting the job, can we also do some pre-procssing in the client side to catch this exception? Is that easy to do?
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "Chinna Rao Lalam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chinna Rao Lalam updated HIVE-2182:
-----------------------------------
Attachment: HIVE-2182.1.patch
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.1.patch, HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13102821#comment-13102821 ]
jiraposter@reviews.apache.org commented on HIVE-2182:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1759/
-----------------------------------------------------------
(Updated 2011-09-12 17:27:12.064230)
Review request for hive and John Sichi.
Summary
-------
while executing UDF if the implementation jar is not present in MR class path it is throwing nullpointer exception instead of throwing nullpointer exception throwing meaning full exception with the required details.
This addresses bug HIVE-2182.
https://issues.apache.org/jira/browse/HIVE-2182
Diffs (updated)
-----
trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBridge.java 1169710
trunk/ql/src/test/queries/clientnegative/udfnull.q PRE-CREATION
trunk/ql/src/test/results/clientnegative/udfnull.q.out PRE-CREATION
trunk/ql/src/test/results/compiler/plan/cast1.q.xml 1169710
trunk/ql/src/test/results/compiler/plan/groupby1.q.xml 1169710
trunk/ql/src/test/results/compiler/plan/groupby2.q.xml 1169710
trunk/ql/src/test/results/compiler/plan/groupby3.q.xml 1169710
trunk/ql/src/test/results/compiler/plan/groupby4.q.xml 1169710
trunk/ql/src/test/results/compiler/plan/groupby5.q.xml 1169710
trunk/ql/src/test/results/compiler/plan/groupby6.q.xml 1169710
trunk/ql/src/test/results/compiler/plan/input20.q.xml 1169710
trunk/ql/src/test/results/compiler/plan/input8.q.xml 1169710
trunk/ql/src/test/results/compiler/plan/join2.q.xml 1169710
trunk/ql/src/test/results/compiler/plan/sample1.q.xml 1169710
trunk/ql/src/test/results/compiler/plan/sample2.q.xml 1169710
trunk/ql/src/test/results/compiler/plan/sample3.q.xml 1169710
trunk/ql/src/test/results/compiler/plan/sample4.q.xml 1169710
trunk/ql/src/test/results/compiler/plan/sample5.q.xml 1169710
trunk/ql/src/test/results/compiler/plan/sample6.q.xml 1169710
trunk/ql/src/test/results/compiler/plan/sample7.q.xml 1169710
trunk/ql/src/test/results/compiler/plan/udf1.q.xml 1169710
trunk/ql/src/test/results/compiler/plan/udf4.q.xml 1169710
trunk/ql/src/test/results/compiler/plan/udf6.q.xml 1169710
Diff: https://reviews.apache.org/r/1759/diff
Testing
-------
Added unit tests
Thanks,
chinna
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.1.patch, HIVE-2182.2.patch, HIVE-2182.3.patch, HIVE-2182.4.patch, HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi updated HIVE-2182:
-----------------------------
Status: Open (was: Patch Available)
I am getting the failure below when running the new test with latest trunk. Did you update the .q.out?
{noformat}
[junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I grantTime -I [.][.][.] [0-9]* more -I job_[0-9]*_[0-9]* -I USING 'java -cp /data/users/jsichi/open/test-trunk/build/ql/test/logs/clientnegative/udfnull.q.out /data/users/jsichi/open/test-trunk/ql/src/test/results/clientnegative/udfnull.q.out
[junit] 8,18c8
[junit] < PREHOOK: Output: file:/tmp/jsichi/hive_2011-09-08_16-48-29_269_6749666372366482183/-mr-10000
[junit] < Execution failed with exit status: 2
[junit] < Obtaining error information
[junit] <
[junit] < Task failed!
[junit] < Task ID:
[junit] < Stage-1
[junit] <
[junit] < Logs:
[junit] <
[junit] < /data/users/jsichi/open/test-trunk/build/ql/tmp//hive.log
[junit] ---
[junit] > PREHOOK: Output: file:/tmp/root/hive_2011-05-25_10-05-57_126_4632621650656424226/-mr-10000
[junit] Exception: Client execution results failed with error code = 1
[junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
[junit] Cleaning up TestNegativeCliDriver
[junit] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 5.496 sec
[junit] Test org.apache.hadoop.hive.cli.TestNegativeCliDriver FAILED
{noformat}
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.1.patch, HIVE-2182.2.patch, HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "Chinna Rao Lalam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chinna Rao Lalam updated HIVE-2182:
-----------------------------------
Attachment: HIVE-2182.2.patch
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.1.patch, HIVE-2182.2.patch, HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "Chinna Rao Lalam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chinna Rao Lalam updated HIVE-2182:
-----------------------------------
Attachment: HIVE-2182.4.patch
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.1.patch, HIVE-2182.2.patch, HIVE-2182.3.patch, HIVE-2182.4.patch, HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2182) Avoid null pointer exception when
executing UDF
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13102927#comment-13102927 ]
John Sichi commented on HIVE-2182:
----------------------------------
Yeah, I hit those failures too while testing. I'll rerun with the latest patch.
> Avoid null pointer exception when executing UDF
> -----------------------------------------------
>
> Key: HIVE-2182
> URL: https://issues.apache.org/jira/browse/HIVE-2182
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0, 0.8.0
> Environment: Hadoop 0.20.1, Hive0.8.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5)
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-2182.1.patch, HIVE-2182.2.patch, HIVE-2182.3.patch, HIVE-2182.4.patch, HIVE-2182.patch
>
>
> For using UDF's executed following steps
> {noformat}
> add jar /home/udf/udf.jar;
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> But from the above steps if we miss the first step (add jar) and execute remaining steps
> {noformat}
> create temporary function grade as 'udf.Grade';
> select m.userid,m.name,grade(m.maths,m.physics,m.chemistry) from marks m;
> {noformat}
> In tasktracker it is throwing this exception
> {noformat}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:126)
> at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:878)
> at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:904)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:107)
> ... 31 more
> {noformat}
> Instead of null pointer exception it should throw meaning full exception
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira