You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Rares Vernica <rv...@gmail.com> on 2009/06/02 18:45:02 UTC
problem getting map input filename
Hello,
I have a problem getting the map input file name. Here is what I tried:
public class Map extends Mapper<Object, Text, LongWritable, Text> {
public void map(Object key, Text value, Context context)
throws IOException, InterruptedException {
Configuration conf = context.getConfiguration();
System.out.println(conf.get("map.input.file"));
System.out.println(conf.get("map.input.filename"));
System.out.println(conf.get("mapred.input.file"));
System.out.println(conf.get("mapred.input.filename"));
}
protected void setup(Context context) throws IOException,
InterruptedException {
Configuration conf = context.getConfiguration();
System.out.println(conf.get("map.input.file"));
System.out.println(conf.get("map.input.filename"));
System.out.println(conf.get("mapred.input.file"));
System.out.println(conf.get("mapred.input.filename"));
}
}
All the get is "null". What am I doing wrong?
Thanks,
Rares
Re: problem getting map input filename
Posted by Rares Vernica <rv...@gmail.com>.
On 6/2/09, Rares Vernica <rv...@gmail.com> wrote:
>
> I have a problem getting the map input file name. Here is what I tried:
>
> public class Map extends Mapper<Object, Text, LongWritable, Text> {
>
> public void map(Object key, Text value, Context context)
> throws IOException, InterruptedException {
> Configuration conf = context.getConfiguration();
> System.out.println(conf.get("map.input.file"));
> System.out.println(conf.get("map.input.filename"));
> System.out.println(conf.get("mapred.input.file"));
> System.out.println(conf.get("mapred.input.filename"));
> }
>
> protected void setup(Context context) throws IOException,
> InterruptedException {
> Configuration conf = context.getConfiguration();
> System.out.println(conf.get("map.input.file"));
> System.out.println(conf.get("map.input.filename"));
> System.out.println(conf.get("mapred.input.file"));
> System.out.println(conf.get("mapred.input.filename"));
> }
> }
>
> All the get is "null". What am I doing wrong?
I created https://issues.apache.org/jira/browse/HADOOP-5973
Re: problem getting map input filename
Posted by Rares Vernica <rv...@gmail.com>.
On 6/3/09, He Yongqiang <he...@software.ict.ac.cn> wrote:
>
> take a look at HADOOP-5368, :)
There you set the "map.input.file", I think it should already be set by Hadoop.
Re: problem getting map input filename
Posted by He Yongqiang <he...@software.ict.ac.cn>.
take a look at HADOOP-5368, :)
On 09-6-4 上午12:27, "Rares Vernica" <rv...@gmail.com> wrote:
> On 6/2/09, jason hadoop <ja...@gmail.com> wrote:
>>
>> you can always dump the entire property space and work it out that way.
>
> I dumped the property space and I could only find "mapred.input.dir".
> There was no "mapred.input.file".
>
> --
> Rares
>
>
Re: problem getting map input filename
Posted by Rares Vernica <rv...@gmail.com>.
On 6/2/09, jason hadoop <ja...@gmail.com> wrote:
>
> you can always dump the entire property space and work it out that way.
I dumped the property space and I could only find "mapred.input.dir".
There was no "mapred.input.file".
--
Rares
Re: problem getting map input filename
Posted by jason hadoop <ja...@gmail.com>.
you can always dump the entire property space and work it out that way.
I haven't used the 0.20 api's yet so I can't speak to them
On Tue, Jun 2, 2009 at 10:52 AM, Rares Vernica <rv...@gmail.com> wrote:
> On 6/2/09, randysch@comcast.net <ra...@comcast.net> wrote:
> >
> > Your Map class needs to have a configure method that can access the
> JobConf.
> > Like this:
>
> In Hadoop 0.20 the new Mapper class does no longer have "configure"
> and JobConf has been replaced with Job. In the Mapper methods, you now
> get a Context object.
>
> > public void configure(JobConf conf)
> > {
> > System.out.println(conf.get("map.input.file");
> > }
>
--
Alpha Chapters of my book on Hadoop are available
http://www.apress.com/book/view/9781430219422
www.prohadoopbook.com a community for Hadoop Professionals
Re: problem getting map input filename
Posted by Rares Vernica <rv...@gmail.com>.
On 6/2/09, randysch@comcast.net <ra...@comcast.net> wrote:
>
> Your Map class needs to have a configure method that can access the JobConf.
> Like this:
In Hadoop 0.20 the new Mapper class does no longer have "configure"
and JobConf has been replaced with Job. In the Mapper methods, you now
get a Context object.
> public void configure(JobConf conf)
> {
> System.out.println(conf.get("map.input.file");
> }
Re: problem getting map input filename
Posted by ra...@comcast.net.
Your Map class needs to have a configure method that can access the JobConf. Like this:
public void configure(JobConf conf)
{
System.out.println(conf.get("map.input.file");
}
randy
----- Original Message -----
From: "Rares Vernica" <rv...@gmail.com>
To: core-user@hadoop.apache.org
Sent: Tuesday, June 2, 2009 12:45:02 PM GMT -05:00 US/Canada Eastern
Subject: problem getting map input filename
Hello,
I have a problem getting the map input file name. Here is what I tried:
public class Map extends Mapper<Object, Text, LongWritable, Text> {
public void map(Object key, Text value, Context context)
throws IOException, InterruptedException {
Configuration conf = context.getConfiguration();
System.out.println(conf.get("map.input.file"));
System.out.println(conf.get("map.input.filename"));
System.out.println(conf.get("mapred.input.file"));
System.out.println(conf.get("mapred.input.filename"));
}
protected void setup(Context context) throws IOException,
InterruptedException {
Configuration conf = context.getConfiguration();
System.out.println(conf.get("map.input.file"));
System.out.println(conf.get("map.input.filename"));
System.out.println(conf.get("mapred.input.file"));
System.out.println(conf.get("mapred.input.filename"));
}
}
All the get is "null". What am I doing wrong?
Thanks,
Rares