File name to which mapper's key,value belongs to - is it available?

[Saptarshi Guha <sa...@gmail.com>]

Hello,
Is there a conf variable for getting the filename to which the current
mapper's key,value belongs to?
I have dir/dirA/part-X and dir/dirB/part-X
i will process dir, but need to know whether the key,value is from
dirA/part-* file or from a dirB/part-* file.

I'd much rather not implement my own inputformat, since I'd like the
method to be inputformat agnostic.

Much thanks
Saptarshi Guha

Re: File name to which mapper's key,value belongs to - is it available?

[Farhan Husain <ru...@gmail.com>]

In one occasion it did not work for me. I had to pass the same value to
jobconf with a new name.

On Sat, Apr 25, 2009 at 6:30 PM, Chuck Lam <ch...@gmail.com> wrote:

> yes, with the JobConf object try
>
> job.get("map.input.file");
>
>
>
>
> On Sat, Apr 25, 2009 at 12:06 PM, Farhan Husain <ru...@gmail.com> wrote:
>
> > For this purpose, I have written my own InputFormat class but I believe
> > there is a better way of doing that. JobCong may provide information of
> > input file.
> >
> > On Sat, Apr 25, 2009 at 12:19 PM, Saptarshi Guha
> > <sa...@gmail.com>wrote:
> >
> > > Hello,
> > > Is there a conf variable for getting the filename to which the current
> > > mapper's key,value belongs to?
> > > I have dir/dirA/part-X and dir/dirB/part-X
> > > i will process dir, but need to know whether the key,value is from
> > > dirA/part-* file or from a dirB/part-* file.
> > >
> > > I'd much rather not implement my own inputformat, since I'd like the
> > > method to be inputformat agnostic.
> > >
> > > Much thanks
> > > Saptarshi Guha
> > >
> >
>

Re: File name to which mapper's key,value belongs to - is it available?

[Chuck Lam <ch...@gmail.com>]

yes, with the JobConf object try

job.get("map.input.file");




On Sat, Apr 25, 2009 at 12:06 PM, Farhan Husain <ru...@gmail.com> wrote:

> For this purpose, I have written my own InputFormat class but I believe
> there is a better way of doing that. JobCong may provide information of
> input file.
>
> On Sat, Apr 25, 2009 at 12:19 PM, Saptarshi Guha
> <sa...@gmail.com>wrote:
>
> > Hello,
> > Is there a conf variable for getting the filename to which the current
> > mapper's key,value belongs to?
> > I have dir/dirA/part-X and dir/dirB/part-X
> > i will process dir, but need to know whether the key,value is from
> > dirA/part-* file or from a dirB/part-* file.
> >
> > I'd much rather not implement my own inputformat, since I'd like the
> > method to be inputformat agnostic.
> >
> > Much thanks
> > Saptarshi Guha
> >
>

Re: File name to which mapper's key,value belongs to - is it available?

[Farhan Husain <ru...@gmail.com>]

For this purpose, I have written my own InputFormat class but I believe
there is a better way of doing that. JobCong may provide information of
input file.

On Sat, Apr 25, 2009 at 12:19 PM, Saptarshi Guha
<sa...@gmail.com>wrote:

> Hello,
> Is there a conf variable for getting the filename to which the current
> mapper's key,value belongs to?
> I have dir/dirA/part-X and dir/dirB/part-X
> i will process dir, but need to know whether the key,value is from
> dirA/part-* file or from a dirB/part-* file.
>
> I'd much rather not implement my own inputformat, since I'd like the
> method to be inputformat agnostic.
>
> Much thanks
> Saptarshi Guha
>