You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Yair Even-Zohar <ya...@revenuescience.com> on 2008/09/05 06:06:42 UTC

Can not retrieve input file name using mapred.input.file

Hi
I'm running an uploader to Hbase using mapreduce and in the map
configure method I use:

public void configure(JobConf job) {
        mapTaskId = job.get("mapred.task.id");
        inputFile = job.get("mapred.input.file");
        custId="";
        if (inputFile!=null) {
        	String splits[] = inputFile.split("_");
	        if (splits.length > 2)
	        	custId = splits[1];
        }
      }


I get the mapTaskId correctly but am getting null inputFile.
I previously set 

public JobConf createSubmittableJob(String[] args) {
	    JobConf c = new JobConf(getConf(), ClogUploader.class);
	    FileInputFormat.setInputPaths(c, new Path(args[0]));
}


Any suggestions?

Thanks
-Yair

RE: Re: Can not retrieve input file name using mapred.input.file

Posted by Yair Even-Zohar <ya...@revenuescience.com>.
The problem with this solution is that I get the exact match in
args[0].For example, when I enter a path like \tmp\files\* I get
inputFile=\tmp\files\*  instead of the actual file's name.
The reason I need the exact file name is that the file name contains
some information I need to use.

Thanks
-Yair

-----Original Message-----
From: news [mailto:news@ger.gmane.org] On Behalf Of Billy Pearson
Sent: Friday, September 05, 2008 3:24 AM
To: hbase-user@hadoop.apache.org
Subject: Re: Can not retrieve input file name using mapred.input.file

Try using

job.set(filename, args[0]);

then use

job.get(filename);

Billy




"Yair Even-Zohar" <ya...@revenuescience.com> 
wrote in message 
news:4B94F7D3090A974E94A9BD23E57BB143019CAA4D@corpdc-exch01.corp.digimin
e.com...
Hi
I'm running an uploader to Hbase using mapreduce and in the map
configure method I use:

public void configure(JobConf job) {
        mapTaskId = job.get("mapred.task.id");
        inputFile = job.get("mapred.input.file");
        custId="";
        if (inputFile!=null) {
        String splits[] = inputFile.split("_");
        if (splits.length > 2)
        custId = splits[1];
        }
      }


I get the mapTaskId correctly but am getting null inputFile.
I previously set

public JobConf createSubmittableJob(String[] args) {
    JobConf c = new JobConf(getConf(), ClogUploader.class);
    FileInputFormat.setInputPaths(c, new Path(args[0]));
}


Any suggestions?

Thanks
-Yair

Re: Can not retrieve input file name using mapred.input.file

Posted by Billy Pearson <sa...@pearsonwholesale.com>.
Try using

job.set(filename, args[0]);

then use

job.get(filename);

Billy




"Yair Even-Zohar" <ya...@revenuescience.com> 
wrote in message 
news:4B94F7D3090A974E94A9BD23E57BB143019CAA4D@corpdc-exch01.corp.digimine.com...
Hi
I'm running an uploader to Hbase using mapreduce and in the map
configure method I use:

public void configure(JobConf job) {
        mapTaskId = job.get("mapred.task.id");
        inputFile = job.get("mapred.input.file");
        custId="";
        if (inputFile!=null) {
        String splits[] = inputFile.split("_");
        if (splits.length > 2)
        custId = splits[1];
        }
      }


I get the mapTaskId correctly but am getting null inputFile.
I previously set

public JobConf createSubmittableJob(String[] args) {
    JobConf c = new JobConf(getConf(), ClogUploader.class);
    FileInputFormat.setInputPaths(c, new Path(args[0]));
}


Any suggestions?

Thanks
-Yair