You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@hadoop.apache.org by Mahmoud Al-Ewiwi <me...@gmail.com> on 2013/09/14 19:54:04 UTC
Unclear Hadoop 2.1X documentation
Hello,
I'm new to Hadoop and i want to learn it in order to do a project.
I'v started reading the documentation at this site:
http://hadoop.apache.org/docs/r2.1.0-beta/hadoop-project-dist/hadoop-common/SingleCluster.html
for setting a single node, but i could not figure a lot of things in these
documentation.
1."You should be able to obtain the MapReduce tarball from the release"
I could not find this tarball, where is it.
2."You will need protoc 2.5.0 installed"
what is that, there is no even a link for it or what is it
3."Assuming you have installed hadoop-common/hadoop-hdfs"
what also are these, and why you are assuming that. i have just downlaod
the hadoop-2.1.0-beta<http://ftp.itu.edu.tr/Mirror/Apache/hadoop/common/hadoop-2.1.0-beta/>and
extracted it
4. and exported *$HADOOP_COMMON_HOME*/*$HADOOP_HDFS_HOME*
!! that is strange, to where should these environment variables indicating
lastly as i know,the first step tutorial should give more details. or am i
searching the wrong side.
**
**
Re: Unclear Hadoop 2.1X documentation
Posted by Karthik Kambatla <ka...@cloudera.com>.
Moving general@ to bcc and redirecting this to the appropriate list -
user@hadoop.apache.org
On Mon, Sep 16, 2013 at 2:18 AM, Jagat Singh <ja...@gmail.com> wrote:
> Hello Mahmoud
>
> You can run on your machine also.
>
> I learnt everything on my 3gb 2ghz machine and recently got better machine.
>
> If you follow this post below you should be able to install and run hadoop
> in 30 mins.
>
> If your machine is not linux then i suggest you to download virtualbox ,
> give it 1400mb ram and start ubuntu in it.
>
> Then just follow steps here.
>
>
> http://jugnu-life.blogspot.com.au/2012/05/hadoop-20-install-tutorial-023x.html
>
> Thanks,
>
> Jagat
> On 16/09/2013 7:07 PM, "Mahmoud Al-Ewiwi" <me...@gmail.com> wrote:
>
> > Thanks Ted,
> >
> > for now i just need to learn the basics of the hadoop before going to ask
> > my university for more powerful machines.
> > i just want to know how to install and write some simple programs to ask
> my
> > supervisor for another server machines
> >
> > Best Regards
> >
> >
> > On Mon, Sep 16, 2013 at 3:57 AM, Ted Dunning <td...@maprtech.com>
> > wrote:
> >
> > > This is a very small amount of memory for running Hadoop + user
> programs.
> > >
> > > You might consider running your tests on a cloud provider like Amazon.
> > > That will give you access to decent sized machines for a relatively
> > small
> > > cost.
> > >
> > >
> > > On Sun, Sep 15, 2013 at 11:27 AM, Mahmoud Al-Ewiwi <me...@gmail.com>
> > > wrote:
> > >
> > > > Thanks to all, i'v tried to use some of these sandboxs, but
> > unfortunately
> > > > most of them a high amount of memory(3GB) for the guest machine and i
> > > have
> > > > only a 3GB on my machine (old machine), so i'm going to go along with
> > the
> > > > the normal installation (i have no choice)
> > > >
> > > > Thanks
> > > >
> > > >
> > > > On Sun, Sep 15, 2013 at 9:13 AM, Roman Shaposhnik <rv...@apache.org>
> > > wrote:
> > > >
> > > > > On Sat, Sep 14, 2013 at 10:54 AM, Mahmoud Al-Ewiwi <
> mewiwi@gmail.com
> > >
> > > > > wrote:
> > > > > > Hello,
> > > > > >
> > > > > > I'm new to Hadoop and i want to learn it in order to do a
> project.
> > > > > > I'v started reading the documentation at this site:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://hadoop.apache.org/docs/r2.1.0-beta/hadoop-project-dist/hadoop-common/SingleCluster.html
> > > > > >
> > > > > > for setting a single node, but i could not figure a lot of things
> > in
> > > > > these
> > > > > > documentation.
> > > > >
> > > > > For the first timer like yourself, perhaps using a Hadoop
> > distribution
> > > > > would be the best way to get started. Bigtop offers a 100%
> community
> > > > > driven distro, but there are, of course, vendor choices as well.
> > > > >
> > > > > Here's the info on Bigtop:
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+Hadoop+distribution+from+Bigtop+0.6.0
> > > > >
> > > > > Thanks,
> > > > > Roman.
> > > > >
> > > >
> > >
> >
>
Re: Unclear Hadoop 2.1X documentation
Posted by Karthik Kambatla <ka...@cloudera.com>.
Moving general@ to bcc and redirecting this to the appropriate list -
user@hadoop.apache.org
On Mon, Sep 16, 2013 at 2:18 AM, Jagat Singh <ja...@gmail.com> wrote:
> Hello Mahmoud
>
> You can run on your machine also.
>
> I learnt everything on my 3gb 2ghz machine and recently got better machine.
>
> If you follow this post below you should be able to install and run hadoop
> in 30 mins.
>
> If your machine is not linux then i suggest you to download virtualbox ,
> give it 1400mb ram and start ubuntu in it.
>
> Then just follow steps here.
>
>
> http://jugnu-life.blogspot.com.au/2012/05/hadoop-20-install-tutorial-023x.html
>
> Thanks,
>
> Jagat
> On 16/09/2013 7:07 PM, "Mahmoud Al-Ewiwi" <me...@gmail.com> wrote:
>
> > Thanks Ted,
> >
> > for now i just need to learn the basics of the hadoop before going to ask
> > my university for more powerful machines.
> > i just want to know how to install and write some simple programs to ask
> my
> > supervisor for another server machines
> >
> > Best Regards
> >
> >
> > On Mon, Sep 16, 2013 at 3:57 AM, Ted Dunning <td...@maprtech.com>
> > wrote:
> >
> > > This is a very small amount of memory for running Hadoop + user
> programs.
> > >
> > > You might consider running your tests on a cloud provider like Amazon.
> > > That will give you access to decent sized machines for a relatively
> > small
> > > cost.
> > >
> > >
> > > On Sun, Sep 15, 2013 at 11:27 AM, Mahmoud Al-Ewiwi <me...@gmail.com>
> > > wrote:
> > >
> > > > Thanks to all, i'v tried to use some of these sandboxs, but
> > unfortunately
> > > > most of them a high amount of memory(3GB) for the guest machine and i
> > > have
> > > > only a 3GB on my machine (old machine), so i'm going to go along with
> > the
> > > > the normal installation (i have no choice)
> > > >
> > > > Thanks
> > > >
> > > >
> > > > On Sun, Sep 15, 2013 at 9:13 AM, Roman Shaposhnik <rv...@apache.org>
> > > wrote:
> > > >
> > > > > On Sat, Sep 14, 2013 at 10:54 AM, Mahmoud Al-Ewiwi <
> mewiwi@gmail.com
> > >
> > > > > wrote:
> > > > > > Hello,
> > > > > >
> > > > > > I'm new to Hadoop and i want to learn it in order to do a
> project.
> > > > > > I'v started reading the documentation at this site:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://hadoop.apache.org/docs/r2.1.0-beta/hadoop-project-dist/hadoop-common/SingleCluster.html
> > > > > >
> > > > > > for setting a single node, but i could not figure a lot of things
> > in
> > > > > these
> > > > > > documentation.
> > > > >
> > > > > For the first timer like yourself, perhaps using a Hadoop
> > distribution
> > > > > would be the best way to get started. Bigtop offers a 100%
> community
> > > > > driven distro, but there are, of course, vendor choices as well.
> > > > >
> > > > > Here's the info on Bigtop:
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+Hadoop+distribution+from+Bigtop+0.6.0
> > > > >
> > > > > Thanks,
> > > > > Roman.
> > > > >
> > > >
> > >
> >
>
Re: Unclear Hadoop 2.1X documentation
Posted by Karthik Kambatla <ka...@cloudera.com>.
Moving general@ to bcc and redirecting this to the appropriate list -
user@hadoop.apache.org
On Mon, Sep 16, 2013 at 2:18 AM, Jagat Singh <ja...@gmail.com> wrote:
> Hello Mahmoud
>
> You can run on your machine also.
>
> I learnt everything on my 3gb 2ghz machine and recently got better machine.
>
> If you follow this post below you should be able to install and run hadoop
> in 30 mins.
>
> If your machine is not linux then i suggest you to download virtualbox ,
> give it 1400mb ram and start ubuntu in it.
>
> Then just follow steps here.
>
>
> http://jugnu-life.blogspot.com.au/2012/05/hadoop-20-install-tutorial-023x.html
>
> Thanks,
>
> Jagat
> On 16/09/2013 7:07 PM, "Mahmoud Al-Ewiwi" <me...@gmail.com> wrote:
>
> > Thanks Ted,
> >
> > for now i just need to learn the basics of the hadoop before going to ask
> > my university for more powerful machines.
> > i just want to know how to install and write some simple programs to ask
> my
> > supervisor for another server machines
> >
> > Best Regards
> >
> >
> > On Mon, Sep 16, 2013 at 3:57 AM, Ted Dunning <td...@maprtech.com>
> > wrote:
> >
> > > This is a very small amount of memory for running Hadoop + user
> programs.
> > >
> > > You might consider running your tests on a cloud provider like Amazon.
> > > That will give you access to decent sized machines for a relatively
> > small
> > > cost.
> > >
> > >
> > > On Sun, Sep 15, 2013 at 11:27 AM, Mahmoud Al-Ewiwi <me...@gmail.com>
> > > wrote:
> > >
> > > > Thanks to all, i'v tried to use some of these sandboxs, but
> > unfortunately
> > > > most of them a high amount of memory(3GB) for the guest machine and i
> > > have
> > > > only a 3GB on my machine (old machine), so i'm going to go along with
> > the
> > > > the normal installation (i have no choice)
> > > >
> > > > Thanks
> > > >
> > > >
> > > > On Sun, Sep 15, 2013 at 9:13 AM, Roman Shaposhnik <rv...@apache.org>
> > > wrote:
> > > >
> > > > > On Sat, Sep 14, 2013 at 10:54 AM, Mahmoud Al-Ewiwi <
> mewiwi@gmail.com
> > >
> > > > > wrote:
> > > > > > Hello,
> > > > > >
> > > > > > I'm new to Hadoop and i want to learn it in order to do a
> project.
> > > > > > I'v started reading the documentation at this site:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://hadoop.apache.org/docs/r2.1.0-beta/hadoop-project-dist/hadoop-common/SingleCluster.html
> > > > > >
> > > > > > for setting a single node, but i could not figure a lot of things
> > in
> > > > > these
> > > > > > documentation.
> > > > >
> > > > > For the first timer like yourself, perhaps using a Hadoop
> > distribution
> > > > > would be the best way to get started. Bigtop offers a 100%
> community
> > > > > driven distro, but there are, of course, vendor choices as well.
> > > > >
> > > > > Here's the info on Bigtop:
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+Hadoop+distribution+from+Bigtop+0.6.0
> > > > >
> > > > > Thanks,
> > > > > Roman.
> > > > >
> > > >
> > >
> >
>
Re: Unclear Hadoop 2.1X documentation
Posted by Karthik Kambatla <ka...@cloudera.com>.
Moving general@ to bcc and redirecting this to the appropriate list -
user@hadoop.apache.org
On Mon, Sep 16, 2013 at 2:18 AM, Jagat Singh <ja...@gmail.com> wrote:
> Hello Mahmoud
>
> You can run on your machine also.
>
> I learnt everything on my 3gb 2ghz machine and recently got better machine.
>
> If you follow this post below you should be able to install and run hadoop
> in 30 mins.
>
> If your machine is not linux then i suggest you to download virtualbox ,
> give it 1400mb ram and start ubuntu in it.
>
> Then just follow steps here.
>
>
> http://jugnu-life.blogspot.com.au/2012/05/hadoop-20-install-tutorial-023x.html
>
> Thanks,
>
> Jagat
> On 16/09/2013 7:07 PM, "Mahmoud Al-Ewiwi" <me...@gmail.com> wrote:
>
> > Thanks Ted,
> >
> > for now i just need to learn the basics of the hadoop before going to ask
> > my university for more powerful machines.
> > i just want to know how to install and write some simple programs to ask
> my
> > supervisor for another server machines
> >
> > Best Regards
> >
> >
> > On Mon, Sep 16, 2013 at 3:57 AM, Ted Dunning <td...@maprtech.com>
> > wrote:
> >
> > > This is a very small amount of memory for running Hadoop + user
> programs.
> > >
> > > You might consider running your tests on a cloud provider like Amazon.
> > > That will give you access to decent sized machines for a relatively
> > small
> > > cost.
> > >
> > >
> > > On Sun, Sep 15, 2013 at 11:27 AM, Mahmoud Al-Ewiwi <me...@gmail.com>
> > > wrote:
> > >
> > > > Thanks to all, i'v tried to use some of these sandboxs, but
> > unfortunately
> > > > most of them a high amount of memory(3GB) for the guest machine and i
> > > have
> > > > only a 3GB on my machine (old machine), so i'm going to go along with
> > the
> > > > the normal installation (i have no choice)
> > > >
> > > > Thanks
> > > >
> > > >
> > > > On Sun, Sep 15, 2013 at 9:13 AM, Roman Shaposhnik <rv...@apache.org>
> > > wrote:
> > > >
> > > > > On Sat, Sep 14, 2013 at 10:54 AM, Mahmoud Al-Ewiwi <
> mewiwi@gmail.com
> > >
> > > > > wrote:
> > > > > > Hello,
> > > > > >
> > > > > > I'm new to Hadoop and i want to learn it in order to do a
> project.
> > > > > > I'v started reading the documentation at this site:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://hadoop.apache.org/docs/r2.1.0-beta/hadoop-project-dist/hadoop-common/SingleCluster.html
> > > > > >
> > > > > > for setting a single node, but i could not figure a lot of things
> > in
> > > > > these
> > > > > > documentation.
> > > > >
> > > > > For the first timer like yourself, perhaps using a Hadoop
> > distribution
> > > > > would be the best way to get started. Bigtop offers a 100%
> community
> > > > > driven distro, but there are, of course, vendor choices as well.
> > > > >
> > > > > Here's the info on Bigtop:
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+Hadoop+distribution+from+Bigtop+0.6.0
> > > > >
> > > > > Thanks,
> > > > > Roman.
> > > > >
> > > >
> > >
> >
>
Re: Unclear Hadoop 2.1X documentation
Posted by Karthik Kambatla <ka...@cloudera.com>.
Moving general@ to bcc and redirecting this to the appropriate list -
user@hadoop.apache.org
On Mon, Sep 16, 2013 at 2:18 AM, Jagat Singh <ja...@gmail.com> wrote:
> Hello Mahmoud
>
> You can run on your machine also.
>
> I learnt everything on my 3gb 2ghz machine and recently got better machine.
>
> If you follow this post below you should be able to install and run hadoop
> in 30 mins.
>
> If your machine is not linux then i suggest you to download virtualbox ,
> give it 1400mb ram and start ubuntu in it.
>
> Then just follow steps here.
>
>
> http://jugnu-life.blogspot.com.au/2012/05/hadoop-20-install-tutorial-023x.html
>
> Thanks,
>
> Jagat
> On 16/09/2013 7:07 PM, "Mahmoud Al-Ewiwi" <me...@gmail.com> wrote:
>
> > Thanks Ted,
> >
> > for now i just need to learn the basics of the hadoop before going to ask
> > my university for more powerful machines.
> > i just want to know how to install and write some simple programs to ask
> my
> > supervisor for another server machines
> >
> > Best Regards
> >
> >
> > On Mon, Sep 16, 2013 at 3:57 AM, Ted Dunning <td...@maprtech.com>
> > wrote:
> >
> > > This is a very small amount of memory for running Hadoop + user
> programs.
> > >
> > > You might consider running your tests on a cloud provider like Amazon.
> > > That will give you access to decent sized machines for a relatively
> > small
> > > cost.
> > >
> > >
> > > On Sun, Sep 15, 2013 at 11:27 AM, Mahmoud Al-Ewiwi <me...@gmail.com>
> > > wrote:
> > >
> > > > Thanks to all, i'v tried to use some of these sandboxs, but
> > unfortunately
> > > > most of them a high amount of memory(3GB) for the guest machine and i
> > > have
> > > > only a 3GB on my machine (old machine), so i'm going to go along with
> > the
> > > > the normal installation (i have no choice)
> > > >
> > > > Thanks
> > > >
> > > >
> > > > On Sun, Sep 15, 2013 at 9:13 AM, Roman Shaposhnik <rv...@apache.org>
> > > wrote:
> > > >
> > > > > On Sat, Sep 14, 2013 at 10:54 AM, Mahmoud Al-Ewiwi <
> mewiwi@gmail.com
> > >
> > > > > wrote:
> > > > > > Hello,
> > > > > >
> > > > > > I'm new to Hadoop and i want to learn it in order to do a
> project.
> > > > > > I'v started reading the documentation at this site:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://hadoop.apache.org/docs/r2.1.0-beta/hadoop-project-dist/hadoop-common/SingleCluster.html
> > > > > >
> > > > > > for setting a single node, but i could not figure a lot of things
> > in
> > > > > these
> > > > > > documentation.
> > > > >
> > > > > For the first timer like yourself, perhaps using a Hadoop
> > distribution
> > > > > would be the best way to get started. Bigtop offers a 100%
> community
> > > > > driven distro, but there are, of course, vendor choices as well.
> > > > >
> > > > > Here's the info on Bigtop:
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+Hadoop+distribution+from+Bigtop+0.6.0
> > > > >
> > > > > Thanks,
> > > > > Roman.
> > > > >
> > > >
> > >
> >
>
Re: Unclear Hadoop 2.1X documentation
Posted by Mahmoud Al-Ewiwi <me...@gmail.com>.
Thanks to all,
and thanks to you Mr. Singh your tutorials are very good and now it works
perfectly.
Best Regards
On Mon, Sep 16, 2013 at 12:39 PM, Anshu Prateek <an...@gmail.com> wrote:
> Yes, docs might be unclear, but rather than taking the time to complaint
> "what is this?" , better thing to do is search and find answers + submit
> patches for hadoop docs.
>
> Assuming you have done xxx. (Implicit is if you have not done then go and
> do that).
>
> The good folks here have taken their time to reply to you but trust me, you
> would learn much more if you try to find the answers and help improve the
> broken docs.
>
> regards
> Anshu Prateek
>
>
> On Mon, Sep 16, 2013 at 2:48 PM, Jagat Singh <ja...@gmail.com> wrote:
>
> > Hello Mahmoud
> >
> > You can run on your machine also.
> >
> > I learnt everything on my 3gb 2ghz machine and recently got better
> machine.
> >
> > If you follow this post below you should be able to install and run
> hadoop
> > in 30 mins.
> >
> > If your machine is not linux then i suggest you to download virtualbox ,
> > give it 1400mb ram and start ubuntu in it.
> >
> > Then just follow steps here.
> >
> >
> >
> http://jugnu-life.blogspot.com.au/2012/05/hadoop-20-install-tutorial-023x.html
> >
> > Thanks,
> >
> > Jagat
> > On 16/09/2013 7:07 PM, "Mahmoud Al-Ewiwi" <me...@gmail.com> wrote:
> >
> > > Thanks Ted,
> > >
> > > for now i just need to learn the basics of the hadoop before going to
> ask
> > > my university for more powerful machines.
> > > i just want to know how to install and write some simple programs to
> ask
> > my
> > > supervisor for another server machines
> > >
> > > Best Regards
> > >
> > >
> > > On Mon, Sep 16, 2013 at 3:57 AM, Ted Dunning <td...@maprtech.com>
> > > wrote:
> > >
> > > > This is a very small amount of memory for running Hadoop + user
> > programs.
> > > >
> > > > You might consider running your tests on a cloud provider like
> Amazon.
> > > > That will give you access to decent sized machines for a relatively
> > > small
> > > > cost.
> > > >
> > > >
> > > > On Sun, Sep 15, 2013 at 11:27 AM, Mahmoud Al-Ewiwi <mewiwi@gmail.com
> >
> > > > wrote:
> > > >
> > > > > Thanks to all, i'v tried to use some of these sandboxs, but
> > > unfortunately
> > > > > most of them a high amount of memory(3GB) for the guest machine
> and i
> > > > have
> > > > > only a 3GB on my machine (old machine), so i'm going to go along
> with
> > > the
> > > > > the normal installation (i have no choice)
> > > > >
> > > > > Thanks
> > > > >
> > > > >
> > > > > On Sun, Sep 15, 2013 at 9:13 AM, Roman Shaposhnik <rv...@apache.org>
> > > > wrote:
> > > > >
> > > > > > On Sat, Sep 14, 2013 at 10:54 AM, Mahmoud Al-Ewiwi <
> > mewiwi@gmail.com
> > > >
> > > > > > wrote:
> > > > > > > Hello,
> > > > > > >
> > > > > > > I'm new to Hadoop and i want to learn it in order to do a
> > project.
> > > > > > > I'v started reading the documentation at this site:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://hadoop.apache.org/docs/r2.1.0-beta/hadoop-project-dist/hadoop-common/SingleCluster.html
> > > > > > >
> > > > > > > for setting a single node, but i could not figure a lot of
> things
> > > in
> > > > > > these
> > > > > > > documentation.
> > > > > >
> > > > > > For the first timer like yourself, perhaps using a Hadoop
> > > distribution
> > > > > > would be the best way to get started. Bigtop offers a 100%
> > community
> > > > > > driven distro, but there are, of course, vendor choices as well.
> > > > > >
> > > > > > Here's the info on Bigtop:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+Hadoop+distribution+from+Bigtop+0.6.0
> > > > > >
> > > > > > Thanks,
> > > > > > Roman.
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Unclear Hadoop 2.1X documentation
Posted by Anshu Prateek <an...@gmail.com>.
Yes, docs might be unclear, but rather than taking the time to complaint
"what is this?" , better thing to do is search and find answers + submit
patches for hadoop docs.
Assuming you have done xxx. (Implicit is if you have not done then go and
do that).
The good folks here have taken their time to reply to you but trust me, you
would learn much more if you try to find the answers and help improve the
broken docs.
regards
Anshu Prateek
On Mon, Sep 16, 2013 at 2:48 PM, Jagat Singh <ja...@gmail.com> wrote:
> Hello Mahmoud
>
> You can run on your machine also.
>
> I learnt everything on my 3gb 2ghz machine and recently got better machine.
>
> If you follow this post below you should be able to install and run hadoop
> in 30 mins.
>
> If your machine is not linux then i suggest you to download virtualbox ,
> give it 1400mb ram and start ubuntu in it.
>
> Then just follow steps here.
>
>
> http://jugnu-life.blogspot.com.au/2012/05/hadoop-20-install-tutorial-023x.html
>
> Thanks,
>
> Jagat
> On 16/09/2013 7:07 PM, "Mahmoud Al-Ewiwi" <me...@gmail.com> wrote:
>
> > Thanks Ted,
> >
> > for now i just need to learn the basics of the hadoop before going to ask
> > my university for more powerful machines.
> > i just want to know how to install and write some simple programs to ask
> my
> > supervisor for another server machines
> >
> > Best Regards
> >
> >
> > On Mon, Sep 16, 2013 at 3:57 AM, Ted Dunning <td...@maprtech.com>
> > wrote:
> >
> > > This is a very small amount of memory for running Hadoop + user
> programs.
> > >
> > > You might consider running your tests on a cloud provider like Amazon.
> > > That will give you access to decent sized machines for a relatively
> > small
> > > cost.
> > >
> > >
> > > On Sun, Sep 15, 2013 at 11:27 AM, Mahmoud Al-Ewiwi <me...@gmail.com>
> > > wrote:
> > >
> > > > Thanks to all, i'v tried to use some of these sandboxs, but
> > unfortunately
> > > > most of them a high amount of memory(3GB) for the guest machine and i
> > > have
> > > > only a 3GB on my machine (old machine), so i'm going to go along with
> > the
> > > > the normal installation (i have no choice)
> > > >
> > > > Thanks
> > > >
> > > >
> > > > On Sun, Sep 15, 2013 at 9:13 AM, Roman Shaposhnik <rv...@apache.org>
> > > wrote:
> > > >
> > > > > On Sat, Sep 14, 2013 at 10:54 AM, Mahmoud Al-Ewiwi <
> mewiwi@gmail.com
> > >
> > > > > wrote:
> > > > > > Hello,
> > > > > >
> > > > > > I'm new to Hadoop and i want to learn it in order to do a
> project.
> > > > > > I'v started reading the documentation at this site:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://hadoop.apache.org/docs/r2.1.0-beta/hadoop-project-dist/hadoop-common/SingleCluster.html
> > > > > >
> > > > > > for setting a single node, but i could not figure a lot of things
> > in
> > > > > these
> > > > > > documentation.
> > > > >
> > > > > For the first timer like yourself, perhaps using a Hadoop
> > distribution
> > > > > would be the best way to get started. Bigtop offers a 100%
> community
> > > > > driven distro, but there are, of course, vendor choices as well.
> > > > >
> > > > > Here's the info on Bigtop:
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+Hadoop+distribution+from+Bigtop+0.6.0
> > > > >
> > > > > Thanks,
> > > > > Roman.
> > > > >
> > > >
> > >
> >
>
Re: Unclear Hadoop 2.1X documentation
Posted by Jagat Singh <ja...@gmail.com>.
Hello Mahmoud
You can run on your machine also.
I learnt everything on my 3gb 2ghz machine and recently got better machine.
If you follow this post below you should be able to install and run hadoop
in 30 mins.
If your machine is not linux then i suggest you to download virtualbox ,
give it 1400mb ram and start ubuntu in it.
Then just follow steps here.
http://jugnu-life.blogspot.com.au/2012/05/hadoop-20-install-tutorial-023x.html
Thanks,
Jagat
On 16/09/2013 7:07 PM, "Mahmoud Al-Ewiwi" <me...@gmail.com> wrote:
> Thanks Ted,
>
> for now i just need to learn the basics of the hadoop before going to ask
> my university for more powerful machines.
> i just want to know how to install and write some simple programs to ask my
> supervisor for another server machines
>
> Best Regards
>
>
> On Mon, Sep 16, 2013 at 3:57 AM, Ted Dunning <td...@maprtech.com>
> wrote:
>
> > This is a very small amount of memory for running Hadoop + user programs.
> >
> > You might consider running your tests on a cloud provider like Amazon.
> > That will give you access to decent sized machines for a relatively
> small
> > cost.
> >
> >
> > On Sun, Sep 15, 2013 at 11:27 AM, Mahmoud Al-Ewiwi <me...@gmail.com>
> > wrote:
> >
> > > Thanks to all, i'v tried to use some of these sandboxs, but
> unfortunately
> > > most of them a high amount of memory(3GB) for the guest machine and i
> > have
> > > only a 3GB on my machine (old machine), so i'm going to go along with
> the
> > > the normal installation (i have no choice)
> > >
> > > Thanks
> > >
> > >
> > > On Sun, Sep 15, 2013 at 9:13 AM, Roman Shaposhnik <rv...@apache.org>
> > wrote:
> > >
> > > > On Sat, Sep 14, 2013 at 10:54 AM, Mahmoud Al-Ewiwi <mewiwi@gmail.com
> >
> > > > wrote:
> > > > > Hello,
> > > > >
> > > > > I'm new to Hadoop and i want to learn it in order to do a project.
> > > > > I'v started reading the documentation at this site:
> > > > >
> > > > >
> > > >
> > >
> >
> http://hadoop.apache.org/docs/r2.1.0-beta/hadoop-project-dist/hadoop-common/SingleCluster.html
> > > > >
> > > > > for setting a single node, but i could not figure a lot of things
> in
> > > > these
> > > > > documentation.
> > > >
> > > > For the first timer like yourself, perhaps using a Hadoop
> distribution
> > > > would be the best way to get started. Bigtop offers a 100% community
> > > > driven distro, but there are, of course, vendor choices as well.
> > > >
> > > > Here's the info on Bigtop:
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+Hadoop+distribution+from+Bigtop+0.6.0
> > > >
> > > > Thanks,
> > > > Roman.
> > > >
> > >
> >
>
Re: Unclear Hadoop 2.1X documentation
Posted by Mahmoud Al-Ewiwi <me...@gmail.com>.
Thanks Ted,
for now i just need to learn the basics of the hadoop before going to ask
my university for more powerful machines.
i just want to know how to install and write some simple programs to ask my
supervisor for another server machines
Best Regards
On Mon, Sep 16, 2013 at 3:57 AM, Ted Dunning <td...@maprtech.com> wrote:
> This is a very small amount of memory for running Hadoop + user programs.
>
> You might consider running your tests on a cloud provider like Amazon.
> That will give you access to decent sized machines for a relatively small
> cost.
>
>
> On Sun, Sep 15, 2013 at 11:27 AM, Mahmoud Al-Ewiwi <me...@gmail.com>
> wrote:
>
> > Thanks to all, i'v tried to use some of these sandboxs, but unfortunately
> > most of them a high amount of memory(3GB) for the guest machine and i
> have
> > only a 3GB on my machine (old machine), so i'm going to go along with the
> > the normal installation (i have no choice)
> >
> > Thanks
> >
> >
> > On Sun, Sep 15, 2013 at 9:13 AM, Roman Shaposhnik <rv...@apache.org>
> wrote:
> >
> > > On Sat, Sep 14, 2013 at 10:54 AM, Mahmoud Al-Ewiwi <me...@gmail.com>
> > > wrote:
> > > > Hello,
> > > >
> > > > I'm new to Hadoop and i want to learn it in order to do a project.
> > > > I'v started reading the documentation at this site:
> > > >
> > > >
> > >
> >
> http://hadoop.apache.org/docs/r2.1.0-beta/hadoop-project-dist/hadoop-common/SingleCluster.html
> > > >
> > > > for setting a single node, but i could not figure a lot of things in
> > > these
> > > > documentation.
> > >
> > > For the first timer like yourself, perhaps using a Hadoop distribution
> > > would be the best way to get started. Bigtop offers a 100% community
> > > driven distro, but there are, of course, vendor choices as well.
> > >
> > > Here's the info on Bigtop:
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+Hadoop+distribution+from+Bigtop+0.6.0
> > >
> > > Thanks,
> > > Roman.
> > >
> >
>
Re: Unclear Hadoop 2.1X documentation
Posted by Ted Dunning <td...@maprtech.com>.
This is a very small amount of memory for running Hadoop + user programs.
You might consider running your tests on a cloud provider like Amazon.
That will give you access to decent sized machines for a relatively small
cost.
On Sun, Sep 15, 2013 at 11:27 AM, Mahmoud Al-Ewiwi <me...@gmail.com> wrote:
> Thanks to all, i'v tried to use some of these sandboxs, but unfortunately
> most of them a high amount of memory(3GB) for the guest machine and i have
> only a 3GB on my machine (old machine), so i'm going to go along with the
> the normal installation (i have no choice)
>
> Thanks
>
>
> On Sun, Sep 15, 2013 at 9:13 AM, Roman Shaposhnik <rv...@apache.org> wrote:
>
> > On Sat, Sep 14, 2013 at 10:54 AM, Mahmoud Al-Ewiwi <me...@gmail.com>
> > wrote:
> > > Hello,
> > >
> > > I'm new to Hadoop and i want to learn it in order to do a project.
> > > I'v started reading the documentation at this site:
> > >
> > >
> >
> http://hadoop.apache.org/docs/r2.1.0-beta/hadoop-project-dist/hadoop-common/SingleCluster.html
> > >
> > > for setting a single node, but i could not figure a lot of things in
> > these
> > > documentation.
> >
> > For the first timer like yourself, perhaps using a Hadoop distribution
> > would be the best way to get started. Bigtop offers a 100% community
> > driven distro, but there are, of course, vendor choices as well.
> >
> > Here's the info on Bigtop:
> >
> >
> https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+Hadoop+distribution+from+Bigtop+0.6.0
> >
> > Thanks,
> > Roman.
> >
>
Re: Unclear Hadoop 2.1X documentation
Posted by Mahmoud Al-Ewiwi <me...@gmail.com>.
Thanks to all, i'v tried to use some of these sandboxs, but unfortunately
most of them a high amount of memory(3GB) for the guest machine and i have
only a 3GB on my machine (old machine), so i'm going to go along with the
the normal installation (i have no choice)
Thanks
On Sun, Sep 15, 2013 at 9:13 AM, Roman Shaposhnik <rv...@apache.org> wrote:
> On Sat, Sep 14, 2013 at 10:54 AM, Mahmoud Al-Ewiwi <me...@gmail.com>
> wrote:
> > Hello,
> >
> > I'm new to Hadoop and i want to learn it in order to do a project.
> > I'v started reading the documentation at this site:
> >
> >
> http://hadoop.apache.org/docs/r2.1.0-beta/hadoop-project-dist/hadoop-common/SingleCluster.html
> >
> > for setting a single node, but i could not figure a lot of things in
> these
> > documentation.
>
> For the first timer like yourself, perhaps using a Hadoop distribution
> would be the best way to get started. Bigtop offers a 100% community
> driven distro, but there are, of course, vendor choices as well.
>
> Here's the info on Bigtop:
>
> https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+Hadoop+distribution+from+Bigtop+0.6.0
>
> Thanks,
> Roman.
>
Re: Unclear Hadoop 2.1X documentation
Posted by Roman Shaposhnik <rv...@apache.org>.
On Sat, Sep 14, 2013 at 10:54 AM, Mahmoud Al-Ewiwi <me...@gmail.com> wrote:
> Hello,
>
> I'm new to Hadoop and i want to learn it in order to do a project.
> I'v started reading the documentation at this site:
>
> http://hadoop.apache.org/docs/r2.1.0-beta/hadoop-project-dist/hadoop-common/SingleCluster.html
>
> for setting a single node, but i could not figure a lot of things in these
> documentation.
For the first timer like yourself, perhaps using a Hadoop distribution
would be the best way to get started. Bigtop offers a 100% community
driven distro, but there are, of course, vendor choices as well.
Here's the info on Bigtop:
https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+Hadoop+distribution+from+Bigtop+0.6.0
Thanks,
Roman.
Re: Unclear Hadoop 2.1X documentation
Posted by Marco Shaw <ma...@gmail.com>.
Along with what Ted says... Find someone with a packaged demo or continue trying to manage your way through the install.
This is why some vendors are succesfull packaging Hadoop and offering support services: it's not always an easy task.
On Sat, Sep 14, 2013 at 2:54 PM, Mahmoud Al-Ewiwi <me...@gmail.com>
wrote:
> Hello,
> I'm new to Hadoop and i want to learn it in order to do a project.
> I'v started reading the documentation at this site:
> http://hadoop.apache.org/docs/r2.1.0-beta/hadoop-project-dist/hadoop-common/SingleCluster.html
> for setting a single node, but i could not figure a lot of things in these
> documentation.
> 1."You should be able to obtain the MapReduce tarball from the release"
> I could not find this tarball, where is it.
> 2."You will need protoc 2.5.0 installed"
> what is that, there is no even a link for it or what is it
> 3."Assuming you have installed hadoop-common/hadoop-hdfs"
> what also are these, and why you are assuming that. i have just downlaod
> the hadoop-2.1.0-beta<http://ftp.itu.edu.tr/Mirror/Apache/hadoop/common/hadoop-2.1.0-beta/>and
> extracted it
> 4. and exported *$HADOOP_COMMON_HOME*/*$HADOOP_HDFS_HOME*
> !! that is strange, to where should these environment variables indicating
> lastly as i know,the first step tutorial should give more details. or am i
> searching the wrong side.
> **
> **
Re: Unclear Hadoop 2.1X documentation
Posted by Ted Yu <yu...@gmail.com>.
It might be easier if you start with some sandbox environment for your
first setup.
Search 'hadoop sandbox' in google.
Cheers
On Sat, Sep 14, 2013 at 11:22 AM, Mahmoud Al-Ewiwi <me...@gmail.com> wrote:
> Thanks Mr. Ted
> what about 3 and 4 where are hadoop-common and hadoop-hdfs
>
>
>
> On Sat, Sep 14, 2013 at 9:09 PM, Ted Yu <yu...@gmail.com> wrote:
>
> > For #1, you can get the tar ball from
> > http://www.apache.org/dyn/closer.cgi/hadoop/common/
> > e.g. http://www.motorlogy.com/apache/hadoop/common/hadoop-2.1.0-beta/
> >
> > It is in maven too: http://mvnrepository.com/artifact/org.apache.hadoop/
> >
> > For #2, see https://code.google.com/p/protobuf/
> >
> >
> > On Sat, Sep 14, 2013 at 10:54 AM, Mahmoud Al-Ewiwi <me...@gmail.com>
> > wrote:
> >
> > > Hello,
> > >
> > > I'm new to Hadoop and i want to learn it in order to do a project.
> > > I'v started reading the documentation at this site:
> > >
> > >
> > >
> >
> http://hadoop.apache.org/docs/r2.1.0-beta/hadoop-project-dist/hadoop-common/SingleCluster.html
> > >
> > > for setting a single node, but i could not figure a lot of things in
> > these
> > > documentation.
> > >
> > > 1."You should be able to obtain the MapReduce tarball from the release"
> > >
> > > I could not find this tarball, where is it.
> > >
> > > 2."You will need protoc 2.5.0 installed"
> > >
> > > what is that, there is no even a link for it or what is it
> > >
> > > 3."Assuming you have installed hadoop-common/hadoop-hdfs"
> > >
> > > what also are these, and why you are assuming that. i have just
> downlaod
> > > the hadoop-2.1.0-beta<
> > > http://ftp.itu.edu.tr/Mirror/Apache/hadoop/common/hadoop-2.1.0-beta/
> >and
> > > extracted it
> > >
> > > 4. and exported *$HADOOP_COMMON_HOME*/*$HADOOP_HDFS_HOME*
> > >
> > > !! that is strange, to where should these environment variables
> > indicating
> > >
> > > lastly as i know,the first step tutorial should give more details. or
> am
> > i
> > > searching the wrong side.
> > > **
> > > **
> > >
> >
>
Re: Unclear Hadoop 2.1X documentation
Posted by Mahmoud Al-Ewiwi <me...@gmail.com>.
sorry, but i can't figure where
*$HADOOP_HDFS_HOME,* *$HADOOP_MAPRED_HOME,**$HADOOP_YARN_HOME and *
*$HADOOP_MAPRED_HOME *
pointing to*
*
On Sat, Sep 14, 2013 at 9:22 PM, Mahmoud Al-Ewiwi <me...@gmail.com> wrote:
> Thanks Mr. Ted
> what about 3 and 4 where are hadoop-common and hadoop-hdfs
>
>
>
> On Sat, Sep 14, 2013 at 9:09 PM, Ted Yu <yu...@gmail.com> wrote:
>
>> For #1, you can get the tar ball from
>> http://www.apache.org/dyn/closer.cgi/hadoop/common/
>> e.g. http://www.motorlogy.com/apache/hadoop/common/hadoop-2.1.0-beta/
>>
>> It is in maven too: http://mvnrepository.com/artifact/org.apache.hadoop/
>>
>> For #2, see https://code.google.com/p/protobuf/
>>
>>
>> On Sat, Sep 14, 2013 at 10:54 AM, Mahmoud Al-Ewiwi <me...@gmail.com>
>> wrote:
>>
>> > Hello,
>> >
>> > I'm new to Hadoop and i want to learn it in order to do a project.
>> > I'v started reading the documentation at this site:
>> >
>> >
>> >
>> http://hadoop.apache.org/docs/r2.1.0-beta/hadoop-project-dist/hadoop-common/SingleCluster.html
>> >
>> > for setting a single node, but i could not figure a lot of things in
>> these
>> > documentation.
>> >
>> > 1."You should be able to obtain the MapReduce tarball from the release"
>> >
>> > I could not find this tarball, where is it.
>> >
>> > 2."You will need protoc 2.5.0 installed"
>> >
>> > what is that, there is no even a link for it or what is it
>> >
>> > 3."Assuming you have installed hadoop-common/hadoop-hdfs"
>> >
>> > what also are these, and why you are assuming that. i have just downlaod
>> > the hadoop-2.1.0-beta<
>> > http://ftp.itu.edu.tr/Mirror/Apache/hadoop/common/hadoop-2.1.0-beta/
>> >and
>> > extracted it
>> >
>> > 4. and exported *$HADOOP_COMMON_HOME*/*$HADOOP_HDFS_HOME*
>> >
>> > !! that is strange, to where should these environment variables
>> indicating
>> >
>> > lastly as i know,the first step tutorial should give more details. or
>> am i
>> > searching the wrong side.
>> > **
>> > **
>> >
>>
>
>
Re: Unclear Hadoop 2.1X documentation
Posted by Mahmoud Al-Ewiwi <me...@gmail.com>.
Thanks Mr. Ted
what about 3 and 4 where are hadoop-common and hadoop-hdfs
On Sat, Sep 14, 2013 at 9:09 PM, Ted Yu <yu...@gmail.com> wrote:
> For #1, you can get the tar ball from
> http://www.apache.org/dyn/closer.cgi/hadoop/common/
> e.g. http://www.motorlogy.com/apache/hadoop/common/hadoop-2.1.0-beta/
>
> It is in maven too: http://mvnrepository.com/artifact/org.apache.hadoop/
>
> For #2, see https://code.google.com/p/protobuf/
>
>
> On Sat, Sep 14, 2013 at 10:54 AM, Mahmoud Al-Ewiwi <me...@gmail.com>
> wrote:
>
> > Hello,
> >
> > I'm new to Hadoop and i want to learn it in order to do a project.
> > I'v started reading the documentation at this site:
> >
> >
> >
> http://hadoop.apache.org/docs/r2.1.0-beta/hadoop-project-dist/hadoop-common/SingleCluster.html
> >
> > for setting a single node, but i could not figure a lot of things in
> these
> > documentation.
> >
> > 1."You should be able to obtain the MapReduce tarball from the release"
> >
> > I could not find this tarball, where is it.
> >
> > 2."You will need protoc 2.5.0 installed"
> >
> > what is that, there is no even a link for it or what is it
> >
> > 3."Assuming you have installed hadoop-common/hadoop-hdfs"
> >
> > what also are these, and why you are assuming that. i have just downlaod
> > the hadoop-2.1.0-beta<
> > http://ftp.itu.edu.tr/Mirror/Apache/hadoop/common/hadoop-2.1.0-beta/>and
> > extracted it
> >
> > 4. and exported *$HADOOP_COMMON_HOME*/*$HADOOP_HDFS_HOME*
> >
> > !! that is strange, to where should these environment variables
> indicating
> >
> > lastly as i know,the first step tutorial should give more details. or am
> i
> > searching the wrong side.
> > **
> > **
> >
>
Re: Unclear Hadoop 2.1X documentation
Posted by Ted Yu <yu...@gmail.com>.
For #1, you can get the tar ball from
http://www.apache.org/dyn/closer.cgi/hadoop/common/
e.g. http://www.motorlogy.com/apache/hadoop/common/hadoop-2.1.0-beta/
It is in maven too: http://mvnrepository.com/artifact/org.apache.hadoop/
For #2, see https://code.google.com/p/protobuf/
On Sat, Sep 14, 2013 at 10:54 AM, Mahmoud Al-Ewiwi <me...@gmail.com> wrote:
> Hello,
>
> I'm new to Hadoop and i want to learn it in order to do a project.
> I'v started reading the documentation at this site:
>
>
> http://hadoop.apache.org/docs/r2.1.0-beta/hadoop-project-dist/hadoop-common/SingleCluster.html
>
> for setting a single node, but i could not figure a lot of things in these
> documentation.
>
> 1."You should be able to obtain the MapReduce tarball from the release"
>
> I could not find this tarball, where is it.
>
> 2."You will need protoc 2.5.0 installed"
>
> what is that, there is no even a link for it or what is it
>
> 3."Assuming you have installed hadoop-common/hadoop-hdfs"
>
> what also are these, and why you are assuming that. i have just downlaod
> the hadoop-2.1.0-beta<
> http://ftp.itu.edu.tr/Mirror/Apache/hadoop/common/hadoop-2.1.0-beta/>and
> extracted it
>
> 4. and exported *$HADOOP_COMMON_HOME*/*$HADOOP_HDFS_HOME*
>
> !! that is strange, to where should these environment variables indicating
>
> lastly as i know,the first step tutorial should give more details. or am i
> searching the wrong side.
> **
> **
>
Re: Unclear Hadoop 2.1X documentation
Posted by "Ray DiGiacomo, Jr." <ra...@gmail.com>.
Hello Mahmoud,
You are not the only one who thinks the Apache Hadoop installation
documentation is unclear.
You may want to train with a Hadoop trainer instead of going the
pain-staking process of reading through Apache Hadoop setup documentation.
Check out Lion Data Systems. They have web-based and in-person Apache
Hadoop training courses to numb your pain. They also train people on R as
well:
liondatasystems.com/courses
- Ray
On Sat, Sep 14, 2013 at 10:54 AM, Mahmoud Al-Ewiwi <me...@gmail.com> wrote:
> Hello,
>
> I'm new to Hadoop and i want to learn it in order to do a project.
> I'v started reading the documentation at this site:
>
>
> http://hadoop.apache.org/docs/r2.1.0-beta/hadoop-project-dist/hadoop-common/SingleCluster.html
>
> for setting a single node, but i could not figure a lot of things in these
> documentation.
>
> 1."You should be able to obtain the MapReduce tarball from the release"
>
> I could not find this tarball, where is it.
>
> 2."You will need protoc 2.5.0 installed"
>
> what is that, there is no even a link for it or what is it
>
> 3."Assuming you have installed hadoop-common/hadoop-hdfs"
>
> what also are these, and why you are assuming that. i have just downlaod
> the hadoop-2.1.0-beta<
> http://ftp.itu.edu.tr/Mirror/Apache/hadoop/common/hadoop-2.1.0-beta/>and
> extracted it
>
> 4. and exported *$HADOOP_COMMON_HOME*/*$HADOOP_HDFS_HOME*
>
> !! that is strange, to where should these environment variables indicating
>
> lastly as i know,the first step tutorial should give more details. or am i
> searching the wrong side.
> **
> **
>