Can mappers take local file as input?

[Zhenhua Guo <je...@gmail.com>]

I know mappers can take files in HDFS as input. I wonder whether they
can take local files as input.
Thanks.

Gerald

Re: Can mappers take local file as input?

[Owen O'Malley <om...@apache.org>]

Note that if you are running on a multi-node cluster, the "local" file
system needs to be NFS or some other distributed file system. If you have a
non-small cluster (> 10 machines), NFS will be very very busy trying to keep
up.

-- Owen

Re: Can mappers take local file as input?

[Harsh J <qw...@gmail.com>]

Hi,

On Thu, Nov 4, 2010 at 3:12 AM, Zhenhua Guo <je...@gmail.com> wrote:
> I know mappers can take files in HDFS as input. I wonder whether they
> can take local files as input.
> Thanks.
>
> Gerald
>

It can :)

Try with fs.default.name set to "file:///" to use the local filesystem
(and not HDFS). Or in your MR job, give the path as file://<path>
(assuming that this is available across all working nodes).

-- 
Harsh J
www.harshj.com