You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Farhan Husain <ru...@gmail.com> on 2009/04/25 21:06:42 UTC
Re: File name to which mapper's key,value belongs to - is it
available?
For this purpose, I have written my own InputFormat class but I believe
there is a better way of doing that. JobCong may provide information of
input file.
On Sat, Apr 25, 2009 at 12:19 PM, Saptarshi Guha
<sa...@gmail.com>wrote:
> Hello,
> Is there a conf variable for getting the filename to which the current
> mapper's key,value belongs to?
> I have dir/dirA/part-X and dir/dirB/part-X
> i will process dir, but need to know whether the key,value is from
> dirA/part-* file or from a dirB/part-* file.
>
> I'd much rather not implement my own inputformat, since I'd like the
> method to be inputformat agnostic.
>
> Much thanks
> Saptarshi Guha
>
Re: File name to which mapper's key,value belongs to - is it
available?
Posted by Farhan Husain <ru...@gmail.com>.
In one occasion it did not work for me. I had to pass the same value to
jobconf with a new name.
On Sat, Apr 25, 2009 at 6:30 PM, Chuck Lam <ch...@gmail.com> wrote:
> yes, with the JobConf object try
>
> job.get("map.input.file");
>
>
>
>
> On Sat, Apr 25, 2009 at 12:06 PM, Farhan Husain <ru...@gmail.com> wrote:
>
> > For this purpose, I have written my own InputFormat class but I believe
> > there is a better way of doing that. JobCong may provide information of
> > input file.
> >
> > On Sat, Apr 25, 2009 at 12:19 PM, Saptarshi Guha
> > <sa...@gmail.com>wrote:
> >
> > > Hello,
> > > Is there a conf variable for getting the filename to which the current
> > > mapper's key,value belongs to?
> > > I have dir/dirA/part-X and dir/dirB/part-X
> > > i will process dir, but need to know whether the key,value is from
> > > dirA/part-* file or from a dirB/part-* file.
> > >
> > > I'd much rather not implement my own inputformat, since I'd like the
> > > method to be inputformat agnostic.
> > >
> > > Much thanks
> > > Saptarshi Guha
> > >
> >
>
Re: File name to which mapper's key,value belongs to - is it
available?
Posted by Chuck Lam <ch...@gmail.com>.
yes, with the JobConf object try
job.get("map.input.file");
On Sat, Apr 25, 2009 at 12:06 PM, Farhan Husain <ru...@gmail.com> wrote:
> For this purpose, I have written my own InputFormat class but I believe
> there is a better way of doing that. JobCong may provide information of
> input file.
>
> On Sat, Apr 25, 2009 at 12:19 PM, Saptarshi Guha
> <sa...@gmail.com>wrote:
>
> > Hello,
> > Is there a conf variable for getting the filename to which the current
> > mapper's key,value belongs to?
> > I have dir/dirA/part-X and dir/dirB/part-X
> > i will process dir, but need to know whether the key,value is from
> > dirA/part-* file or from a dirB/part-* file.
> >
> > I'd much rather not implement my own inputformat, since I'd like the
> > method to be inputformat agnostic.
> >
> > Much thanks
> > Saptarshi Guha
> >
>