You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by santosh gandham <sa...@gmail.com> on 2009/11/09 19:25:53 UTC
using files in local file system
Hi,
I am new to Hadoop. I am trying yo use Hadoop streaming utility. In map
reduce program I want to use the files in Local file System. Should I need
to copy them into HDFS, if I want to use them? or is there any way that I
can access them from local file system itself? If I have to copy into HDFS
and use, how can I use or how can I specify in my program such that, my
program will access the file in HDFS? And if I can use directly from local
file system, how can I use? Can any one help me please. Thank you so much.
--
Best Regards,
Gandham Santhosh
Re： using files in local file system
Posted by Gang Luo <lg...@yahoo.com.cn>.
Hi Gandham,
I think you can use files on local filesystem if you import the right package in you program. But it doesn't make sense in an distributed environment. So, it is better to load the file into HDFS and use the corresponding IO api to access the files. The example at http://wiki.apache.org/hadoop/HadoopDfsReadWriteExample maybe helpful.
--Gang Luo
----- 原始邮件 ----
发件人： santosh gandham <sa...@gmail.com>
收件人： common-user@hadoop.apache.org
发送日期： 2009/11/9 (周一) 1:25:53 下午
主 题： using files in local file system
Hi,
I am new to Hadoop. I am trying yo use Hadoop streaming utility. In map
reduce program I want to use the files in Local file System. Should I need
to copy them into HDFS, if I want to use them? or is there any way that I
can access them from local file system itself? If I have to copy into HDFS
and use, how can I use or how can I specify in my program such that, my
program will access the file in HDFS? And if I can use directly from local
file system, how can I use? Can any one help me please. Thank you so much.
--
Best Regards,
Gandham Santhosh
___________________________________________________________
好玩贺卡等你发，邮箱贺卡全新上线！
http://card.mail.cn.yahoo.com/
Re: using files in local file system
Posted by Boris Shkolnik <bo...@yahoo-inc.com>.
When you create FileSystem object you may specify that you want a
LocalFileSystem. That will give you access to local files.
(e.g. FileContext.GetLocalFSFileContext())
Boris.
On 11/9/09 10:25 AM, "santosh gandham" <sa...@gmail.com> wrote:
> Hi,
> I am new to Hadoop. I am trying yo use Hadoop streaming utility. In map
> reduce program I want to use the files in Local file System. Should I need
> to copy them into HDFS, if I want to use them? or is there any way that I
> can access them from local file system itself? If I have to copy into HDFS
> and use, how can I use or how can I specify in my program such that, my
> program will access the file in HDFS? And if I can use directly from local
> file system, how can I use? Can any one help me please. Thank you so much.