Get information of input split from MapRunner?

[Qin Gao <qi...@cs.cmu.edu>]

Hi mailing,

I want to get information of current input split inside the MapRunner object
(or map function), however the only object I can get from the MapRunner is
the RecordReader, and I saw no method defined in RecordReader to fetch the
InputSplit object. Do you have any suggestions on this?

What I want to have is the start position of the input split and the length
of the split, as can be seen in the web interface. Because I am going to run
some task iteratively and some information need to be calculate for every
split and will not change in each iteration, if I can know which split I
need to deal with, then I can generate the information in the first
iteration and store them, then just load the right one in other iterations.

Best,
Qin

Re: Get information of input split from MapRunner?

[Kevin <kl...@gmail.com>]

Override "configure(JobConf job)" in your mapper class. Get the
"map.input.start" and "map.input.length" from the JobConf.

-Kevin



On Thu, Aug 21, 2008 at 2:14 PM, Qin Gao <qi...@cs.cmu.edu> wrote:
> Hi mailing,
>
> I want to get information of current input split inside the MapRunner object
> (or map function), however the only object I can get from the MapRunner is
> the RecordReader, and I saw no method defined in RecordReader to fetch the
> InputSplit object. Do you have any suggestions on this?
>
> What I want to have is the start position of the input split and the length
> of the split, as can be seen in the web interface. Because I am going to run
> some task iteratively and some information need to be calculate for every
> split and will not change in each iteration, if I can know which split I
> need to deal with, then I can generate the information in the first
> iteration and store them, then just load the right one in other iterations.
>
> Best,
> Qin
>