You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by shensl <sh...@163.com> on 2010/03/11 17:05:41 UTC

Can a map/reduce read local file system?

Can map/reduce read local file system? My map/reduce cannot read localhost file, is there any wrong in my code ? 

    protected void map(Object arg0, Object arg1, Context context)
            throws IOException, InterruptedException {
        Configuration conf = context.getConfiguration();
        LocalFileSystem lfs = FileSystem.getLocal(conf);
        FSDataOutputStream out = lfs.create(new Path("abc.txt"));
        out.writeUTF("abc");
        out.close();
    }

Thanks and Regards!

Re: Can a map/reduce read local file system?

Posted by Ninad Raut <hb...@gmail.com>.

If you donot have your configuration files e.g.hadoop-site.xml in the
classpath Mapreduce automatically reads from the Local file System. You can
also force the Mapred Engine to read from the local file system by adding
file:///<localpath> to your dfs.data.dir property in the configuration.

Cheers.
Ninad

2010/3/11 shensl <sh...@163.com>

> Can map/reduce read local file system? My map/reduce cannot read localhost
> file, is there any wrong in my code ?
>
>    protected void map(Object arg0, Object arg1, Context context)
>            throws IOException, InterruptedException {
>        Configuration conf = context.getConfiguration();
>        LocalFileSystem lfs = FileSystem.getLocal(conf);
>        FSDataOutputStream out = lfs.create(new Path("abc.txt"));
>        out.writeUTF("abc");
>        out.close();
>    }
>
> Thanks and Regards!