You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Xiaowei Li <se...@gmail.com> on 2012/12/14 19:45:51 UTC
how to differentiate which input directory current record comes from?
hi,
my MR job has multiple inputs, and I wanna how to differentiate which input
directory current row/record comes from in my mapper?
thanks!
-xw
Re: how to differentiate which input directory current record comes from?
Posted by Sigurd Spieckermann <si...@gmail.com>.
If you use the new API, you can access the MapContext object in the setup
method of the mapper. Then, you can get the input split with
MapContext#getInputSplit(), cast it to FileSplit and obtain the path of the
file the current split is part of through the FileSplit#getPath() method.
All records of the mapper instance will be part of that split so you only
need to get the information once per map task in the setup method.
Am 14.12.2012 19:46 schrieb "Xiaowei Li" <se...@gmail.com>:
> hi,
>
> my MR job has multiple inputs, and I wanna how to differentiate which
> input directory current row/record comes from in my mapper?
>
> thanks!
> -xw
>
Re: how to differentiate which input directory current record comes from?
Posted by Sigurd Spieckermann <si...@gmail.com>.
If you use the new API, you can access the MapContext object in the setup
method of the mapper. Then, you can get the input split with
MapContext#getInputSplit(), cast it to FileSplit and obtain the path of the
file the current split is part of through the FileSplit#getPath() method.
All records of the mapper instance will be part of that split so you only
need to get the information once per map task in the setup method.
Am 14.12.2012 19:46 schrieb "Xiaowei Li" <se...@gmail.com>:
> hi,
>
> my MR job has multiple inputs, and I wanna how to differentiate which
> input directory current row/record comes from in my mapper?
>
> thanks!
> -xw
>
Re: how to differentiate which input directory current record comes from?
Posted by Sigurd Spieckermann <si...@gmail.com>.
If you use the new API, you can access the MapContext object in the setup
method of the mapper. Then, you can get the input split with
MapContext#getInputSplit(), cast it to FileSplit and obtain the path of the
file the current split is part of through the FileSplit#getPath() method.
All records of the mapper instance will be part of that split so you only
need to get the information once per map task in the setup method.
Am 14.12.2012 19:46 schrieb "Xiaowei Li" <se...@gmail.com>:
> hi,
>
> my MR job has multiple inputs, and I wanna how to differentiate which
> input directory current row/record comes from in my mapper?
>
> thanks!
> -xw
>
Re: how to differentiate which input directory current record comes from?
Posted by Sigurd Spieckermann <si...@gmail.com>.
If you use the new API, you can access the MapContext object in the setup
method of the mapper. Then, you can get the input split with
MapContext#getInputSplit(), cast it to FileSplit and obtain the path of the
file the current split is part of through the FileSplit#getPath() method.
All records of the mapper instance will be part of that split so you only
need to get the information once per map task in the setup method.
Am 14.12.2012 19:46 schrieb "Xiaowei Li" <se...@gmail.com>:
> hi,
>
> my MR job has multiple inputs, and I wanna how to differentiate which
> input directory current row/record comes from in my mapper?
>
> thanks!
> -xw
>