[jira] Commented: (MAPREDUCE-1743) conf.get("map.input.file") returns null when using MultipleInputs in Hadoop 0.20

["Ted Yu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862408#action_12862408 ] 

Ted Yu commented on MAPREDUCE-1743:
-----------------------------------

There is this check in updateJobWithSplit():
    if (inputSplit instanceof FileSplit) {

Since TaggedInputSplit isn't an instance of FileSplit, "map.input.file" isn't set by updateJobWithSplit().

According to the usage in DelegatingInputFormat.getSplits():
         splits.add(new TaggedInputSplit(pathSplit, conf, format.getClass(),
             mapperClass));
One solution is to let TaggedInputSplit extend FileSplit.


> conf.get("map.input.file") returns null when using MultipleInputs in Hadoop 0.20
> --------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1743
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1743
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 0.20.2
>            Reporter: Yuanyuan Tian
>
> There is a problem in getting the input file name in the mapper when uisng MultipleInputs in Hadoop 0.20. I need to use MultipleInputs to support different formats for my inputs to the my MapReduce job. And inside each mapper, I also need to know the exact input file that the mapper is processing. However, conf.get("map.input.file") returns null. Can anybody help me solve this problem? Thanks in advance.
> public class Test extends Configured implements Tool{
> 	static class InnerMapper extends MapReduceBase implements Mapper<Writable, Writable, NullWritable, Text>
> 	{
> 		................
> 		................
> 		public void configure(JobConf conf)
> 		{	
> 			String inputName=conf.get("map.input.file"));
> 			.......................................
> 		}
> 		
> 	}
> 	
> 	public int run(String[] arg0) throws Exception {
> 		JonConf job;
> 		job = new JobConf(Test.class);
> 		...........................................
> 		
> 		MultipleInputs.addInputPath(conf, new Path("A"), TextInputFormat.class);
> 		MultipleInputs.addInputPath(conf, new Path("B"), SequenceFileFormat.class);
> 		...........................................
> 	}
> }

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.