You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Steve Gao <st...@yahoo.com> on 2008/10/15 00:40:17 UTC
Are There Books of Hadoop/Pig?
Does anybody know if there are books about hadoop or pig? The wiki and manual are kind of ad-hoc and hard to comprehend, for example "I want to know how to apply patchs to my Hadoop, but can't find how to do it" that kind of things.
Would anybody help? Thanks.
Re: Need reboot the whole system if adding new datanodes?
Posted by Steve Loughran <st...@apache.org>.
Amit k. Saha wrote:
> On Wed, Oct 15, 2008 at 9:09 AM, David Wei <we...@kingsoft.com> wrote:
>> It seems that we need to restart the whole hadoop system in order to add new
>> nodes inside the cluster. Any solution for us that no need for the
>> rebooting?
>
> From what I know so far, you have to start the HDFS dameon (which
> reads the 'slaves' file) to 'let it know' which are the data nodes. So
> everytime you add a new DataNode, I believe you will have to restarted
> the daemon, which is like re-initiating the NameNode.
>
You don't need a slaves file; you can connect to a namenode without it.
So: no need to restart daemons. What you should do is decommission
datanodes, to shut them down cleanly and make sure all data is copied
off them, when taking them away deliberately. If you just kill it, the
namenode will notice, but some data may be underreplicated.
-steve
Re: Need reboot the whole system if adding new datanodes?
Posted by Paul <pa...@gmail.com>.
As long as the new node is in the slaves file on the master, just do a
start-all.sh and it will attempt to start everything. Nodes that are
already running will keep running and new nodes will be started.
Consider doing a rebalance after adding a new node for better
distribution.
-paul
On Oct 15, 2008, at 1:55 AM, "Amit k. Saha" <am...@gmail.com>
wrote:
> On Wed, Oct 15, 2008 at 9:09 AM, David Wei <we...@kingsoft.com>
> wrote:
>> It seems that we need to restart the whole hadoop system in order
>> to add new
>> nodes inside the cluster. Any solution for us that no need for the
>> rebooting?
>
> From what I know so far, you have to start the HDFS dameon (which
> reads the 'slaves' file) to 'let it know' which are the data nodes. So
> everytime you add a new DataNode, I believe you will have to restarted
> the daemon, which is like re-initiating the NameNode.
>
> Hope I am not very wrong :-)
>
> Best,
> Amit
>
> --
> Amit Kumar Saha
> http://blogs.sun.com/amitsaha/
> http://amitsaha.in.googlepages.com/
> Skype: amitkumarsaha
Re: Need reboot the whole system if adding new datanodes?
Posted by Prasad Pingali <pv...@research.iiit.ac.in>.
you can use the hadoop-daemon.sh script provided in bin folder. The following
will be the steps.
In the new machine to be added,
1.) ensure hadoop config is pointing to the right namenode.
2.) run bin/hadoop-daemon.sh start datanode
this should add datanode without needing a restart of complete cluster.
- Prasad.
On Wednesday 15 October 2008 11:25:29 am Amit k. Saha wrote:
> On Wed, Oct 15, 2008 at 9:09 AM, David Wei <we...@kingsoft.com> wrote:
> > It seems that we need to restart the whole hadoop system in order to add
> > new nodes inside the cluster. Any solution for us that no need for the
> > rebooting?
> >
> >From what I know so far, you have to start the HDFS dameon (which
>
> reads the 'slaves' file) to 'let it know' which are the data nodes. So
> everytime you add a new DataNode, I believe you will have to restarted
> the daemon, which is like re-initiating the NameNode.
>
> Hope I am not very wrong :-)
>
> Best,
> Amit
Re: Need reboot the whole system if adding new datanodes?
Posted by "Amit k. Saha" <am...@gmail.com>.
On Wed, Oct 15, 2008 at 9:09 AM, David Wei <we...@kingsoft.com> wrote:
> It seems that we need to restart the whole hadoop system in order to add new
> nodes inside the cluster. Any solution for us that no need for the
> rebooting?
>From what I know so far, you have to start the HDFS dameon (which
reads the 'slaves' file) to 'let it know' which are the data nodes. So
everytime you add a new DataNode, I believe you will have to restarted
the daemon, which is like re-initiating the NameNode.
Hope I am not very wrong :-)
Best,
Amit
--
Amit Kumar Saha
http://blogs.sun.com/amitsaha/
http://amitsaha.in.googlepages.com/
Skype: amitkumarsaha
Need reboot the whole system if adding new datanodes?
Posted by David Wei <we...@kingsoft.com>.
It seems that we need to restart the whole hadoop system in order to add
new nodes inside the cluster. Any solution for us that no need for the
rebooting?
PS: We just had one namenode in the cluster
Thx!
David
Need reboot the whole system if adding new datanodes?
Posted by David Wei <we...@kingsoft.com>.
It seems that we need to restart the whole hadoop system in order to add
new nodes inside the cluster. Any solution for us that no need for the
rebooting?
PS: We just had one namenode in the cluster
Thx!
David
Re: Are There Books of Hadoop/Pig?
Posted by "Amit k. Saha" <am...@gmail.com>.
On Wed, Oct 15, 2008 at 4:10 AM, Steve Gao <st...@yahoo.com> wrote:
> Does anybody know if there are books about hadoop or pig? The wiki and manual are kind of ad-hoc and hard to comprehend, for example "I want to know how to apply patchs to my Hadoop, but can't find how to do it" that kind of things.
>
> Would anybody help? Thanks.
http://oreilly.com/catalog/9780596521998/
HTH,
Amit
>
>
>
>
--
Amit Kumar Saha
http://blogs.sun.com/amitsaha/
http://amitsaha.in.googlepages.com/
Skype: amitkumarsaha