You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Hamed Ghavamnia <gh...@gmail.com> on 2012/01/02 13:22:19 UTC
hbase Multi Node Setup
Hi,
Does anyone have a link for a good tutorial regarding how to setup a fully
distributed hbase on top of a multi-node hadoop cluster?
Thanks,
Hamed
Re: hbase Multi Node Setup
Posted by Hamed Ghavamnia <gh...@gmail.com>.
Okay, I finally got it running :)
The changes I made were,
1)Using the exact same hbase-site.xml in all the machines
2)Keeping the zookeeper on the hbase-master (which has my namenode as well)
But I don't get why stopping the hbase-master takes so long, the zookeeper
is easily stopped, the regionservers are stopped quickly as well, but the
master takes too long to stop, and it's causing me problems.
On Tue, Jan 3, 2012 at 12:47 PM, Hamed Ghavamnia <gh...@gmail.com>wrote:
> Is it normal that stopping the hbase cluster is really slow?
> Is it different to start/stop the hbase service through
> /etc/init.d/hadoop-hbase-master or hbase/bin/start-hdfs.sh?
>
>
> On Tue, Jan 3, 2012 at 8:55 AM, Hamed Ghavamnia <gh...@gmail.com>wrote:
>
>> To TS.chia,
>> Thanks, but that's for running a hadoop cluster on multiple nodes, which
>> isn't the problem, I've already got that running, I want to know how to run
>> HBase on that multi-node cluster.
>>
>> To Alexander,
>> Thanks, I checked them out. I'm really having a hard time understanding
>> the relation between zookeeper and hbase, and how to configure the region
>> servers and the master. My actual question is, which configs should be done
>> on the master and which on the slaves. For example for hdfs, we would set
>> the namenode for all the datanodes in the hdfs-site.xml file, but hbase
>> doesn't seem to have something like that. Where is the master run, does the
>> master automatically run on the system which I run bin/hbase-start.sh?
>> On my simple test I'm running, I have a regionserver on one of my
>> machines (slave_two), and the zookeeper on another (slave_one), and I run
>> bin/start-hbase.sh on my hdfs datanode (master).
>> When I start hbase on the master, it starts the zookeeper successfully,
>> it can start the regionserver as well, but for the master it says the
>> master is already running, but I've stopped hbase on all of the machines
>> (with /etc/init.d/hadoop-hbase-master stop).
>>
>>
>> On Tue, Jan 3, 2012 at 7:19 AM, Alexander Lorenz <
>> wget.null@googlemail.com> wrote:
>>
>>> Hi,
>>>
>>> here two links for:
>>> http://wiki.apache.org/hadoop/Hbase/10Minutes
>>> https://ccp.cloudera.com/display/CDHDOC/HBase+Installation
>>>
>>> Install hbase and zookeeper, edit the configs (for single or multinode),
>>> pro host a unique ID (myid) for zookeeper, start zookeeper on all nodes as
>>> hbase too. Don't forget to identify your servers as master and slaves in
>>> hbase-site.xml and setup a path in hdfs (/hbase) as example. Use this path
>>> in your config.
>>> If the server does not start check the logs for errors.
>>>
>>> For testing I setup a image with 4 servers, you can download it here:
>>> http://mapredit.blogspot.com/p/all-in-one-hadoop-multi-node-appliance.html
>>>
>>> - Alex
>>>
>>> Alexander Lorenz
>>> http://mapredit.blogspot.com
>>>
>>> On Jan 2, 2012, at 5:07 AM, Hamed Ghavamnia <gh...@gmail.com>
>>> wrote:
>>>
>>> I've checked that out, but I don't understand what changes should be
>>> made to the config files on the hbase-master and the region servers. I've
>>> added my region servers(which are the same as my datanodes in hdfs) to the
>>> regionservers file, and I've changed the hbase-site.xml configuration file,
>>> but the hbase master web UI doesn't show any region servers.
>>>
>>> On Mon, Jan 2, 2012 at 4:02 PM, Sudharsan Sampath <su...@gmail.com>wrote:
>>>
>>>> If you are just looking for just steps The HBase Definitive Guide does
>>>> a decent brief. I am not sure of any automated tools handy though creating
>>>> one based on the steps is straight forward
>>>>
>>>> -Sudhan S
>>>>
>>>>
>>>> On Mon, Jan 2, 2012 at 5:52 PM, Hamed Ghavamnia <gh...@gmail.com>wrote:
>>>>
>>>>> Hi,
>>>>> Does anyone have a link for a good tutorial regarding how to setup a
>>>>> fully distributed hbase on top of a multi-node hadoop cluster?
>>>>>
>>>>> Thanks,
>>>>> Hamed
>>>>>
>>>>
>>>>
>>>
>>
>
Re: hbase Multi Node Setup
Posted by Hamed Ghavamnia <gh...@gmail.com>.
Is it normal that stopping the hbase cluster is really slow?
Is it different to start/stop the hbase service through
/etc/init.d/hadoop-hbase-master or hbase/bin/start-hdfs.sh?
On Tue, Jan 3, 2012 at 8:55 AM, Hamed Ghavamnia <gh...@gmail.com>wrote:
> To TS.chia,
> Thanks, but that's for running a hadoop cluster on multiple nodes, which
> isn't the problem, I've already got that running, I want to know how to run
> HBase on that multi-node cluster.
>
> To Alexander,
> Thanks, I checked them out. I'm really having a hard time understanding
> the relation between zookeeper and hbase, and how to configure the region
> servers and the master. My actual question is, which configs should be done
> on the master and which on the slaves. For example for hdfs, we would set
> the namenode for all the datanodes in the hdfs-site.xml file, but hbase
> doesn't seem to have something like that. Where is the master run, does the
> master automatically run on the system which I run bin/hbase-start.sh?
> On my simple test I'm running, I have a regionserver on one of my machines
> (slave_two), and the zookeeper on another (slave_one), and I run
> bin/start-hbase.sh on my hdfs datanode (master).
> When I start hbase on the master, it starts the zookeeper successfully, it
> can start the regionserver as well, but for the master it says the master
> is already running, but I've stopped hbase on all of the machines (with
> /etc/init.d/hadoop-hbase-master stop).
>
>
> On Tue, Jan 3, 2012 at 7:19 AM, Alexander Lorenz <wget.null@googlemail.com
> > wrote:
>
>> Hi,
>>
>> here two links for:
>> http://wiki.apache.org/hadoop/Hbase/10Minutes
>> https://ccp.cloudera.com/display/CDHDOC/HBase+Installation
>>
>> Install hbase and zookeeper, edit the configs (for single or multinode),
>> pro host a unique ID (myid) for zookeeper, start zookeeper on all nodes as
>> hbase too. Don't forget to identify your servers as master and slaves in
>> hbase-site.xml and setup a path in hdfs (/hbase) as example. Use this path
>> in your config.
>> If the server does not start check the logs for errors.
>>
>> For testing I setup a image with 4 servers, you can download it here:
>> http://mapredit.blogspot.com/p/all-in-one-hadoop-multi-node-appliance.html
>>
>> - Alex
>>
>> Alexander Lorenz
>> http://mapredit.blogspot.com
>>
>> On Jan 2, 2012, at 5:07 AM, Hamed Ghavamnia <gh...@gmail.com>
>> wrote:
>>
>> I've checked that out, but I don't understand what changes should be made
>> to the config files on the hbase-master and the region servers. I've added
>> my region servers(which are the same as my datanodes in hdfs) to the
>> regionservers file, and I've changed the hbase-site.xml configuration file,
>> but the hbase master web UI doesn't show any region servers.
>>
>> On Mon, Jan 2, 2012 at 4:02 PM, Sudharsan Sampath <su...@gmail.com>wrote:
>>
>>> If you are just looking for just steps The HBase Definitive Guide does a
>>> decent brief. I am not sure of any automated tools handy though creating
>>> one based on the steps is straight forward
>>>
>>> -Sudhan S
>>>
>>>
>>> On Mon, Jan 2, 2012 at 5:52 PM, Hamed Ghavamnia <gh...@gmail.com>wrote:
>>>
>>>> Hi,
>>>> Does anyone have a link for a good tutorial regarding how to setup a
>>>> fully distributed hbase on top of a multi-node hadoop cluster?
>>>>
>>>> Thanks,
>>>> Hamed
>>>>
>>>
>>>
>>
>
Re: hbase Multi Node Setup
Posted by Hamed Ghavamnia <gh...@gmail.com>.
To TS.chia,
Thanks, but that's for running a hadoop cluster on multiple nodes, which
isn't the problem, I've already got that running, I want to know how to run
HBase on that multi-node cluster.
To Alexander,
Thanks, I checked them out. I'm really having a hard time understanding the
relation between zookeeper and hbase, and how to configure the region
servers and the master. My actual question is, which configs should be done
on the master and which on the slaves. For example for hdfs, we would set
the namenode for all the datanodes in the hdfs-site.xml file, but hbase
doesn't seem to have something like that. Where is the master run, does the
master automatically run on the system which I run bin/hbase-start.sh?
On my simple test I'm running, I have a regionserver on one of my machines
(slave_two), and the zookeeper on another (slave_one), and I run
bin/start-hbase.sh on my hdfs datanode (master).
When I start hbase on the master, it starts the zookeeper successfully, it
can start the regionserver as well, but for the master it says the master
is already running, but I've stopped hbase on all of the machines (with
/etc/init.d/hadoop-hbase-master stop).
On Tue, Jan 3, 2012 at 7:19 AM, Alexander Lorenz
<wg...@googlemail.com>wrote:
> Hi,
>
> here two links for:
> http://wiki.apache.org/hadoop/Hbase/10Minutes
> https://ccp.cloudera.com/display/CDHDOC/HBase+Installation
>
> Install hbase and zookeeper, edit the configs (for single or multinode),
> pro host a unique ID (myid) for zookeeper, start zookeeper on all nodes as
> hbase too. Don't forget to identify your servers as master and slaves in
> hbase-site.xml and setup a path in hdfs (/hbase) as example. Use this path
> in your config.
> If the server does not start check the logs for errors.
>
> For testing I setup a image with 4 servers, you can download it here:
> http://mapredit.blogspot.com/p/all-in-one-hadoop-multi-node-appliance.html
>
> - Alex
>
> Alexander Lorenz
> http://mapredit.blogspot.com
>
> On Jan 2, 2012, at 5:07 AM, Hamed Ghavamnia <gh...@gmail.com> wrote:
>
> I've checked that out, but I don't understand what changes should be made
> to the config files on the hbase-master and the region servers. I've added
> my region servers(which are the same as my datanodes in hdfs) to the
> regionservers file, and I've changed the hbase-site.xml configuration file,
> but the hbase master web UI doesn't show any region servers.
>
> On Mon, Jan 2, 2012 at 4:02 PM, Sudharsan Sampath <su...@gmail.com>wrote:
>
>> If you are just looking for just steps The HBase Definitive Guide does a
>> decent brief. I am not sure of any automated tools handy though creating
>> one based on the steps is straight forward
>>
>> -Sudhan S
>>
>>
>> On Mon, Jan 2, 2012 at 5:52 PM, Hamed Ghavamnia <gh...@gmail.com>wrote:
>>
>>> Hi,
>>> Does anyone have a link for a good tutorial regarding how to setup a
>>> fully distributed hbase on top of a multi-node hadoop cluster?
>>>
>>> Thanks,
>>> Hamed
>>>
>>
>>
>
Re: hbase Multi Node Setup
Posted by Alexander Lorenz <wg...@googlemail.com>.
Hi,
here two links for:
http://wiki.apache.org/hadoop/Hbase/10Minutes
https://ccp.cloudera.com/display/CDHDOC/HBase+Installation
Install hbase and zookeeper, edit the configs (for single or multinode), pro host a unique ID (myid) for zookeeper, start zookeeper on all nodes as hbase too. Don't forget to identify your servers as master and slaves in hbase-site.xml and setup a path in hdfs (/hbase) as example. Use this path in your config.
If the server does not start check the logs for errors.
For testing I setup a image with 4 servers, you can download it here: http://mapredit.blogspot.com/p/all-in-one-hadoop-multi-node-appliance.html
- Alex
Alexander Lorenz
http://mapredit.blogspot.com
On Jan 2, 2012, at 5:07 AM, Hamed Ghavamnia <gh...@gmail.com> wrote:
> I've checked that out, but I don't understand what changes should be made to the config files on the hbase-master and the region servers. I've added my region servers(which are the same as my datanodes in hdfs) to the regionservers file, and I've changed the hbase-site.xml configuration file, but the hbase master web UI doesn't show any region servers.
>
> On Mon, Jan 2, 2012 at 4:02 PM, Sudharsan Sampath <su...@gmail.com> wrote:
> If you are just looking for just steps The HBase Definitive Guide does a decent brief. I am not sure of any automated tools handy though creating one based on the steps is straight forward
>
> -Sudhan S
>
>
> On Mon, Jan 2, 2012 at 5:52 PM, Hamed Ghavamnia <gh...@gmail.com> wrote:
> Hi,
> Does anyone have a link for a good tutorial regarding how to setup a fully distributed hbase on top of a multi-node hadoop cluster?
>
> Thanks,
> Hamed
>
>
Re: hbase Multi Node Setup
Posted by Hamed Ghavamnia <gh...@gmail.com>.
I've checked that out, but I don't understand what changes should be made
to the config files on the hbase-master and the region servers. I've added
my region servers(which are the same as my datanodes in hdfs) to the
regionservers file, and I've changed the hbase-site.xml configuration file,
but the hbase master web UI doesn't show any region servers.
On Mon, Jan 2, 2012 at 4:02 PM, Sudharsan Sampath <su...@gmail.com>wrote:
> If you are just looking for just steps The HBase Definitive Guide does a
> decent brief. I am not sure of any automated tools handy though creating
> one based on the steps is straight forward
>
> -Sudhan S
>
>
> On Mon, Jan 2, 2012 at 5:52 PM, Hamed Ghavamnia <gh...@gmail.com>wrote:
>
>> Hi,
>> Does anyone have a link for a good tutorial regarding how to setup a
>> fully distributed hbase on top of a multi-node hadoop cluster?
>>
>> Thanks,
>> Hamed
>>
>
>
Re: hbase Multi Node Setup
Posted by Sudharsan Sampath <su...@gmail.com>.
If you are just looking for just steps The HBase Definitive Guide does a
decent brief. I am not sure of any automated tools handy though creating
one based on the steps is straight forward
-Sudhan S
On Mon, Jan 2, 2012 at 5:52 PM, Hamed Ghavamnia <gh...@gmail.com>wrote:
> Hi,
> Does anyone have a link for a good tutorial regarding how to setup a fully
> distributed hbase on top of a multi-node hadoop cluster?
>
> Thanks,
> Hamed
>
Re: hbase Multi Node Setup
Posted by TS chia <th...@gmail.com>.
Hi
Here
http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/
Cheer
ts
On Mon, Jan 2, 2012 at 8:22 PM, Hamed Ghavamnia <gh...@gmail.com>wrote:
> Hi,
> Does anyone have a link for a good tutorial regarding how to setup a fully
> distributed hbase on top of a multi-node hadoop cluster?
>
> Thanks,
> Hamed
>