[jira] [Commented] (TEZ-1403) oah.mapred.Partitioner is not configured by JobConf

["Siddharth Seth (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-1403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14095001#comment-14095001 ] 

Siddharth Seth commented on TEZ-1403:
-------------------------------------

I think this is required, even though it'll be much better not having to create the JobConf instance, since the relevant values would have already been setup on the client side. To make mapred partitioners work however, I'm not sure we can do much else.

Will commit in a bit.

Thanks [~navis]

> oah.mapred.Partitioner is not configured by JobConf
> ---------------------------------------------------
>
>                 Key: TEZ-1403
>                 URL: https://issues.apache.org/jira/browse/TEZ-1403
>             Project: Apache Tez
>          Issue Type: Bug
>    Affects Versions: 0.4.0
>            Reporter: Navis
>         Attachments: TEZ-1403.1.patch.txt
>
>
> TotalOrderPartitioner is not initialized by JobConf, throwing NPE in runtime.
> {noformat}
> Caused by: java.lang.NullPointerException
> 	at org.apache.hadoop.mapreduce.lib.partition.TotalOrderPartitioner.getPartition(TotalOrderPartitioner.java:127)
> 	at org.apache.hadoop.mapreduce.lib.partition.TotalOrderPartitioner.getPartition(TotalOrderPartitioner.java:48)
> 	at org.apache.hadoop.hive.ql.exec.HiveTotalOrderPartitioner.getPartition(HiveTotalOrderPartitioner.java:41)
> 	at org.apache.hadoop.hive.ql.exec.HiveTotalOrderPartitioner.getPartition(HiveTotalOrderPartitioner.java:29)
> 	at org.apache.tez.mapreduce.partition.MRPartitioner.getPartition(MRPartitioner.java:85)
> 	at org.apache.tez.runtime.library.common.sort.impl.dflt.DefaultSorter.write(DefaultSorter.java:185)
> 	at org.apache.tez.runtime.library.output.OnFileSortedOutput$1.write(OnFileSortedOutput.java:116)
> 	at org.apache.tez.mapreduce.processor.map.MapProcessor$OldOutputCollector.collect(MapProcessor.java:359)
> 	at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.collect(ReduceSinkOperator.java:444)
> 	at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:319)
> 	... 16 more
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)