You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Roberto Alonso <ro...@gmail.com> on 2012/03/23 12:37:16 UTC
hbase.cluster.distributed set to true but it says false
Hello,
I have been experimenting some problems with the fully distributed version.
First of all I'll tell you my configuration:
I have 4 servers(server_{1,2,3,4}) with 6GB Ram and 2 cores. I installed
hadoop in all of them, this is the configuration:
- server_1 is namenode, datanode and secondarynamenode
- server_2, server_3, server_4: datanodes
The storage is aroung 500GB
On the other hand, I have installed hbase, and this is the configuration:
- server_1: master and regionserver
- server_2: zookeeper and regionserver
- server_3 and server_4: regionserver
hbase-site.xml for each server looks like this:
<property>
<name>hbase.zookeeper.quorum</name>
<value>server_2</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/hdfs/zookeeper</value>
</property>
<property>
<name>dfs.support.append</name>
<value>true</value>
</property>
<property>
<name>hbase.rootdir</name>
<value>hdfs://server_1:54310/hbase</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
So I have some problems you may help me:
1º Insertion is slow. I have an alphanumeric row with two column families.
It takes around 9 minutes to insert 200000 rows, but this is more or less
acceptable.
2º I have a map reduce job where I create a configuration:
Configuration config = HBaseConfiguration.create();
and then I ask for config.get("hbase.cluster.distributed"); and It says
"false", what do you think?
thanks a lot!
Re: hbase.cluster.distributed set to true but it says false
Posted by Roberto Alonso <ro...@gmail.com>.
Thanks for your quick repy. I am quite new in this field so:
1) I don't understand quite well what you mean, do you mean that is not
optimal the configuration? I have also this in hbase:
<!-- perfomance tunning -->
<property>
<name>hbase.client.scanner.caching</name>
<value>100</value>
</property>
<property>
<name>hbase.regionserver.handler.count</name>
<value>100</value>
</property>
2)any idea why it is getting default configuration when I put in the
config hbase.cluster.distributed to true?
On 23 March 2012 12:46, Michel Segel <mi...@hotmail.com> wrote:
> 1) your configuration is sub optimal so toss performance out the window.
> You appear to be on vms. Toss performance out the window.
>
> 2) sounds like you are picking up the default value and not your cluster's
> config.
>
> Sent from a remote device. Please excuse any typos...
>
> Mike Segel
>
> On Mar 23, 2012, at 6:37 AM, Roberto Alonso <ro...@gmail.com> wrote:
>
> > Hello,
> >
> > I have been experimenting some problems with the fully distributed
> version.
> > First of all I'll tell you my configuration:
> >
> > I have 4 servers(server_{1,2,3,4}) with 6GB Ram and 2 cores. I installed
> > hadoop in all of them, this is the configuration:
> > - server_1 is namenode, datanode and secondarynamenode
> > - server_2, server_3, server_4: datanodes
> > The storage is aroung 500GB
> >
> > On the other hand, I have installed hbase, and this is the configuration:
> > - server_1: master and regionserver
> > - server_2: zookeeper and regionserver
> > - server_3 and server_4: regionserver
> >
> > hbase-site.xml for each server looks like this:
> > <property>
> > <name>hbase.zookeeper.quorum</name>
> > <value>server_2</value>
> > </property>
> > <property>
> > <name>hbase.zookeeper.property.dataDir</name>
> > <value>/hdfs/zookeeper</value>
> > </property>
> > <property>
> > <name>dfs.support.append</name>
> > <value>true</value>
> > </property>
> > <property>
> > <name>hbase.rootdir</name>
> > <value>hdfs://server_1:54310/hbase</value>
> > </property>
> > <property>
> > <name>hbase.cluster.distributed</name>
> > <value>true</value>
> > </property>
> >
> > So I have some problems you may help me:
> > 1º Insertion is slow. I have an alphanumeric row with two column
> families.
> > It takes around 9 minutes to insert 200000 rows, but this is more or less
> > acceptable.
> > 2º I have a map reduce job where I create a configuration:
> > Configuration config = HBaseConfiguration.create();
> > and then I ask for config.get("hbase.cluster.distributed"); and It says
> > "false", what do you think?
> >
> > thanks a lot!
>
Re: hbase.cluster.distributed set to true but it says false
Posted by Michel Segel <mi...@hotmail.com>.
1) your configuration is sub optimal so toss performance out the window. You appear to be on vms. Toss performance out the window.
2) sounds like you are picking up the default value and not your cluster's config.
Sent from a remote device. Please excuse any typos...
Mike Segel
On Mar 23, 2012, at 6:37 AM, Roberto Alonso <ro...@gmail.com> wrote:
> Hello,
>
> I have been experimenting some problems with the fully distributed version.
> First of all I'll tell you my configuration:
>
> I have 4 servers(server_{1,2,3,4}) with 6GB Ram and 2 cores. I installed
> hadoop in all of them, this is the configuration:
> - server_1 is namenode, datanode and secondarynamenode
> - server_2, server_3, server_4: datanodes
> The storage is aroung 500GB
>
> On the other hand, I have installed hbase, and this is the configuration:
> - server_1: master and regionserver
> - server_2: zookeeper and regionserver
> - server_3 and server_4: regionserver
>
> hbase-site.xml for each server looks like this:
> <property>
> <name>hbase.zookeeper.quorum</name>
> <value>server_2</value>
> </property>
> <property>
> <name>hbase.zookeeper.property.dataDir</name>
> <value>/hdfs/zookeeper</value>
> </property>
> <property>
> <name>dfs.support.append</name>
> <value>true</value>
> </property>
> <property>
> <name>hbase.rootdir</name>
> <value>hdfs://server_1:54310/hbase</value>
> </property>
> <property>
> <name>hbase.cluster.distributed</name>
> <value>true</value>
> </property>
>
> So I have some problems you may help me:
> 1º Insertion is slow. I have an alphanumeric row with two column families.
> It takes around 9 minutes to insert 200000 rows, but this is more or less
> acceptable.
> 2º I have a map reduce job where I create a configuration:
> Configuration config = HBaseConfiguration.create();
> and then I ask for config.get("hbase.cluster.distributed"); and It says
> "false", what do you think?
>
> thanks a lot!