You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Ashish Jain <as...@gmail.com> on 2014/01/07 08:03:00 UTC
Newbie: How to set up HDFS file system
Hello,
Can someone provide some pointers on how to set up a HDFS file system? I
tried searching in the document but could not find anything with respect to
this.
--Ashish
Re: Newbie: How to set up HDFS file system
Posted by Ashish Jain <as...@gmail.com>.
Thanks everyone. I was able to set up a single node hadoop. I was also able
to run few HDFS commands like -copyFromLocal. Also ran the word count
program. I have few more questions which I will put in another email.
++Ashish
On Tue, Jan 7, 2014 at 11:45 PM, Leonid Fedotov <lf...@hortonworks.com>wrote:
> Go to www.hortonworks.com and follow documentation to install HDP.
>
> *Thank you!*
>
> *Leonid Fedotov*
> Systems Architect - Professional Services
> lfedotov@hortonworks.com
> office: +1 855 846 7866 ext 292
> mobile: +1 650 430 1673
>
>
>
>
>
> On Jan 6, 2014, at 23:03, Ashish Jain <as...@gmail.com> wrote:
>
> Hello,
>
> Can someone provide some pointers on how to set up a HDFS file system? I
> tried searching in the document but could not find anything with respect to
> this.
>
> --Ashish
>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: Newbie: How to set up HDFS file system
Posted by Ashish Jain <as...@gmail.com>.
Thanks everyone. I was able to set up a single node hadoop. I was also able
to run few HDFS commands like -copyFromLocal. Also ran the word count
program. I have few more questions which I will put in another email.
++Ashish
On Tue, Jan 7, 2014 at 11:45 PM, Leonid Fedotov <lf...@hortonworks.com>wrote:
> Go to www.hortonworks.com and follow documentation to install HDP.
>
> *Thank you!*
>
> *Leonid Fedotov*
> Systems Architect - Professional Services
> lfedotov@hortonworks.com
> office: +1 855 846 7866 ext 292
> mobile: +1 650 430 1673
>
>
>
>
>
> On Jan 6, 2014, at 23:03, Ashish Jain <as...@gmail.com> wrote:
>
> Hello,
>
> Can someone provide some pointers on how to set up a HDFS file system? I
> tried searching in the document but could not find anything with respect to
> this.
>
> --Ashish
>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: Newbie: How to set up HDFS file system
Posted by Ashish Jain <as...@gmail.com>.
Thanks everyone. I was able to set up a single node hadoop. I was also able
to run few HDFS commands like -copyFromLocal. Also ran the word count
program. I have few more questions which I will put in another email.
++Ashish
On Tue, Jan 7, 2014 at 11:45 PM, Leonid Fedotov <lf...@hortonworks.com>wrote:
> Go to www.hortonworks.com and follow documentation to install HDP.
>
> *Thank you!*
>
> *Leonid Fedotov*
> Systems Architect - Professional Services
> lfedotov@hortonworks.com
> office: +1 855 846 7866 ext 292
> mobile: +1 650 430 1673
>
>
>
>
>
> On Jan 6, 2014, at 23:03, Ashish Jain <as...@gmail.com> wrote:
>
> Hello,
>
> Can someone provide some pointers on how to set up a HDFS file system? I
> tried searching in the document but could not find anything with respect to
> this.
>
> --Ashish
>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: Newbie: How to set up HDFS file system
Posted by Ashish Jain <as...@gmail.com>.
Thanks everyone. I was able to set up a single node hadoop. I was also able
to run few HDFS commands like -copyFromLocal. Also ran the word count
program. I have few more questions which I will put in another email.
++Ashish
On Tue, Jan 7, 2014 at 11:45 PM, Leonid Fedotov <lf...@hortonworks.com>wrote:
> Go to www.hortonworks.com and follow documentation to install HDP.
>
> *Thank you!*
>
> *Leonid Fedotov*
> Systems Architect - Professional Services
> lfedotov@hortonworks.com
> office: +1 855 846 7866 ext 292
> mobile: +1 650 430 1673
>
>
>
>
>
> On Jan 6, 2014, at 23:03, Ashish Jain <as...@gmail.com> wrote:
>
> Hello,
>
> Can someone provide some pointers on how to set up a HDFS file system? I
> tried searching in the document but could not find anything with respect to
> this.
>
> --Ashish
>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: Newbie: How to set up HDFS file system
Posted by Leonid Fedotov <lf...@hortonworks.com>.
Go to www.hortonworks.com and follow documentation to install HDP.
Thank you!
Leonid Fedotov
Systems Architect - Professional Services
lfedotov@hortonworks.com
office: +1 855 846 7866 ext 292
mobile: +1 650 430 1673
On Jan 6, 2014, at 23:03, Ashish Jain <as...@gmail.com> wrote:
> Hello,
>
> Can someone provide some pointers on how to set up a HDFS file system? I tried searching in the document but could not find anything with respect to this.
>
> --Ashish
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Newbie: How to set up HDFS file system
Posted by Leonid Fedotov <lf...@hortonworks.com>.
Go to www.hortonworks.com and follow documentation to install HDP.
Thank you!
Leonid Fedotov
Systems Architect - Professional Services
lfedotov@hortonworks.com
office: +1 855 846 7866 ext 292
mobile: +1 650 430 1673
On Jan 6, 2014, at 23:03, Ashish Jain <as...@gmail.com> wrote:
> Hello,
>
> Can someone provide some pointers on how to set up a HDFS file system? I tried searching in the document but could not find anything with respect to this.
>
> --Ashish
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Newbie: How to set up HDFS file system
Posted by Devin Suiter RDX <ds...@rdx.com>.
Installing Hadoop will install HDFS, and you will need to declare storage
directories on the host nodes, etc. There is also the question of what
setup you want to use, there is what is called "pseudo-distributed" mode
where all the Hadoop daemons are running in one host.
Are you a student looking to study, or are you wanting to set up something
for work? For study, the pseudo-distributed mode is easy and can run in
virtual machines easily. You will then have access to the HDFS utility, can
set up users and directories, but do not need much resources.
*Devin Suiter*
Jr. Data Solutions Software Engineer
100 Sandusky Street | 2nd Floor | Pittsburgh, PA 15212
Google Voice: 412-256-8556 | www.rdx.com
On Tue, Jan 7, 2014 at 8:50 AM, Chris Mawata <ch...@gmail.com> wrote:
> Read as much as you can about Hadoop. Installing Hadoop will install HDFS.
> On Jan 7, 2014 2:03 AM, "Ashish Jain" <as...@gmail.com> wrote:
>
>> Hello,
>>
>> Can someone provide some pointers on how to set up a HDFS file system? I
>> tried searching in the document but could not find anything with respect to
>> this.
>>
>> --Ashish
>>
>
Re: Newbie: How to set up HDFS file system
Posted by Devin Suiter RDX <ds...@rdx.com>.
Installing Hadoop will install HDFS, and you will need to declare storage
directories on the host nodes, etc. There is also the question of what
setup you want to use, there is what is called "pseudo-distributed" mode
where all the Hadoop daemons are running in one host.
Are you a student looking to study, or are you wanting to set up something
for work? For study, the pseudo-distributed mode is easy and can run in
virtual machines easily. You will then have access to the HDFS utility, can
set up users and directories, but do not need much resources.
*Devin Suiter*
Jr. Data Solutions Software Engineer
100 Sandusky Street | 2nd Floor | Pittsburgh, PA 15212
Google Voice: 412-256-8556 | www.rdx.com
On Tue, Jan 7, 2014 at 8:50 AM, Chris Mawata <ch...@gmail.com> wrote:
> Read as much as you can about Hadoop. Installing Hadoop will install HDFS.
> On Jan 7, 2014 2:03 AM, "Ashish Jain" <as...@gmail.com> wrote:
>
>> Hello,
>>
>> Can someone provide some pointers on how to set up a HDFS file system? I
>> tried searching in the document but could not find anything with respect to
>> this.
>>
>> --Ashish
>>
>
Re: Newbie: How to set up HDFS file system
Posted by Devin Suiter RDX <ds...@rdx.com>.
Installing Hadoop will install HDFS, and you will need to declare storage
directories on the host nodes, etc. There is also the question of what
setup you want to use, there is what is called "pseudo-distributed" mode
where all the Hadoop daemons are running in one host.
Are you a student looking to study, or are you wanting to set up something
for work? For study, the pseudo-distributed mode is easy and can run in
virtual machines easily. You will then have access to the HDFS utility, can
set up users and directories, but do not need much resources.
*Devin Suiter*
Jr. Data Solutions Software Engineer
100 Sandusky Street | 2nd Floor | Pittsburgh, PA 15212
Google Voice: 412-256-8556 | www.rdx.com
On Tue, Jan 7, 2014 at 8:50 AM, Chris Mawata <ch...@gmail.com> wrote:
> Read as much as you can about Hadoop. Installing Hadoop will install HDFS.
> On Jan 7, 2014 2:03 AM, "Ashish Jain" <as...@gmail.com> wrote:
>
>> Hello,
>>
>> Can someone provide some pointers on how to set up a HDFS file system? I
>> tried searching in the document but could not find anything with respect to
>> this.
>>
>> --Ashish
>>
>
Re: Newbie: How to set up HDFS file system
Posted by Devin Suiter RDX <ds...@rdx.com>.
Installing Hadoop will install HDFS, and you will need to declare storage
directories on the host nodes, etc. There is also the question of what
setup you want to use, there is what is called "pseudo-distributed" mode
where all the Hadoop daemons are running in one host.
Are you a student looking to study, or are you wanting to set up something
for work? For study, the pseudo-distributed mode is easy and can run in
virtual machines easily. You will then have access to the HDFS utility, can
set up users and directories, but do not need much resources.
*Devin Suiter*
Jr. Data Solutions Software Engineer
100 Sandusky Street | 2nd Floor | Pittsburgh, PA 15212
Google Voice: 412-256-8556 | www.rdx.com
On Tue, Jan 7, 2014 at 8:50 AM, Chris Mawata <ch...@gmail.com> wrote:
> Read as much as you can about Hadoop. Installing Hadoop will install HDFS.
> On Jan 7, 2014 2:03 AM, "Ashish Jain" <as...@gmail.com> wrote:
>
>> Hello,
>>
>> Can someone provide some pointers on how to set up a HDFS file system? I
>> tried searching in the document but could not find anything with respect to
>> this.
>>
>> --Ashish
>>
>
Re: Newbie: How to set up HDFS file system
Posted by Chris Mawata <ch...@gmail.com>.
Read as much as you can about Hadoop. Installing Hadoop will install HDFS.
On Jan 7, 2014 2:03 AM, "Ashish Jain" <as...@gmail.com> wrote:
> Hello,
>
> Can someone provide some pointers on how to set up a HDFS file system? I
> tried searching in the document but could not find anything with respect to
> this.
>
> --Ashish
>
Re: Newbie: How to set up HDFS file system
Posted by Chris Mawata <ch...@gmail.com>.
Read as much as you can about Hadoop. Installing Hadoop will install HDFS.
On Jan 7, 2014 2:03 AM, "Ashish Jain" <as...@gmail.com> wrote:
> Hello,
>
> Can someone provide some pointers on how to set up a HDFS file system? I
> tried searching in the document but could not find anything with respect to
> this.
>
> --Ashish
>
Re: Newbie: How to set up HDFS file system
Posted by Chris Mawata <ch...@gmail.com>.
Read as much as you can about Hadoop. Installing Hadoop will install HDFS.
On Jan 7, 2014 2:03 AM, "Ashish Jain" <as...@gmail.com> wrote:
> Hello,
>
> Can someone provide some pointers on how to set up a HDFS file system? I
> tried searching in the document but could not find anything with respect to
> this.
>
> --Ashish
>
Re: Newbie: How to set up HDFS file system
Posted by Chris Mawata <ch...@gmail.com>.
Read as much as you can about Hadoop. Installing Hadoop will install HDFS.
On Jan 7, 2014 2:03 AM, "Ashish Jain" <as...@gmail.com> wrote:
> Hello,
>
> Can someone provide some pointers on how to set up a HDFS file system? I
> tried searching in the document but could not find anything with respect to
> this.
>
> --Ashish
>
Re: Newbie: How to set up HDFS file system
Posted by Leonid Fedotov <lf...@hortonworks.com>.
Go to www.hortonworks.com and follow documentation to install HDP.
Thank you!
Leonid Fedotov
Systems Architect - Professional Services
lfedotov@hortonworks.com
office: +1 855 846 7866 ext 292
mobile: +1 650 430 1673
On Jan 6, 2014, at 23:03, Ashish Jain <as...@gmail.com> wrote:
> Hello,
>
> Can someone provide some pointers on how to set up a HDFS file system? I tried searching in the document but could not find anything with respect to this.
>
> --Ashish
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Newbie: How to set up HDFS file system
Posted by Leonid Fedotov <lf...@hortonworks.com>.
Go to www.hortonworks.com and follow documentation to install HDP.
Thank you!
Leonid Fedotov
Systems Architect - Professional Services
lfedotov@hortonworks.com
office: +1 855 846 7866 ext 292
mobile: +1 650 430 1673
On Jan 6, 2014, at 23:03, Ashish Jain <as...@gmail.com> wrote:
> Hello,
>
> Can someone provide some pointers on how to set up a HDFS file system? I tried searching in the document but could not find anything with respect to this.
>
> --Ashish
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.