You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Ling Kun <lk...@qq.com> on 2012/12/20 07:04:29 UTC
Can Hbase run on NFS cluster
Dear all,
I am doing some test on Hbase running on a NFS cluster. Does anyone have any experience?
regionserver: slave1, slave2 and slave3 all mount the same NFS server /mnt/nfs_disk .
Theoretically, I can use the following configuration in hbase-site.xml to make it work in distributed mode, no matter what the performance is.
--------------------hbase-site.xml ------------------------
<property>
<name>hbase.rootdir</name>
<value>file:///mnt/nfs_disk/hbase/</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
------------------hbase-site.xml-------------------------
According to my testing, it is ok for small data write. While for huge amount of data write, which causes region split, it seems the daughter region can not be online correctly.
The following is my questions:
1. Does anyone have setup a hbase cluster on NFS disk experience?
2. Since the basic hbse operation works fine except the region split and daughter region online. could some one please show which part of the code is responsible for this type of works?
3. When HDFS + Hbase is used, it is easy to get high performance because the scan, read and write operations can be done locally when the hbase cluster is the same as HDFS cluster. Is this correct to explain the performance descending?
Thanks.
Ling Kun
RE: Can Hbase run on NFS cluster
Posted by Anoop Sam John <an...@huawei.com>.
Can you check for the logs at the region server side where the region split happens? Paste the logs at this point if some exception happens.
You can see the split code from CompactSpiltThread.java
Mean while not used HBase with NFS. I have not seen yet here in the mailing list some one using HBase with NFS.
-Anoop-
________________________________________
From: Ling Kun [lkun.lingcc@qq.com]
Sent: Thursday, December 20, 2012 11:34 AM
To: dev; user
Subject: Can Hbase run on NFS cluster
Dear all,
I am doing some test on Hbase running on a NFS cluster. Does anyone have any experience?
regionserver: slave1, slave2 and slave3 all mount the same NFS server /mnt/nfs_disk .
Theoretically, I can use the following configuration in hbase-site.xml to make it work in distributed mode, no matter what the performance is.
--------------------hbase-site.xml ------------------------
<property>
<name>hbase.rootdir</name>
<value>file:///mnt/nfs_disk/hbase/</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
------------------hbase-site.xml-------------------------
According to my testing, it is ok for small data write. While for huge amount of data write, which causes region split, it seems the daughter region can not be online correctly.
The following is my questions:
1. Does anyone have setup a hbase cluster on NFS disk experience?
2. Since the basic hbse operation works fine except the region split and daughter region online. could some one please show which part of the code is responsible for this type of works?
3. When HDFS + Hbase is used, it is easy to get high performance because the scan, read and write operations can be done locally when the hbase cluster is the same as HDFS cluster. Is this correct to explain the performance descending?
Thanks.
Ling Kun
RE: Can Hbase run on NFS cluster
Posted by Anoop Sam John <an...@huawei.com>.
Can you check for the logs at the region server side where the region split happens? Paste the logs at this point if some exception happens.
You can see the split code from CompactSpiltThread.java
Mean while not used HBase with NFS. I have not seen yet here in the mailing list some one using HBase with NFS.
-Anoop-
________________________________________
From: Ling Kun [lkun.lingcc@qq.com]
Sent: Thursday, December 20, 2012 11:34 AM
To: dev; user
Subject: Can Hbase run on NFS cluster
Dear all,
I am doing some test on Hbase running on a NFS cluster. Does anyone have any experience?
regionserver: slave1, slave2 and slave3 all mount the same NFS server /mnt/nfs_disk .
Theoretically, I can use the following configuration in hbase-site.xml to make it work in distributed mode, no matter what the performance is.
--------------------hbase-site.xml ------------------------
<property>
<name>hbase.rootdir</name>
<value>file:///mnt/nfs_disk/hbase/</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
------------------hbase-site.xml-------------------------
According to my testing, it is ok for small data write. While for huge amount of data write, which causes region split, it seems the daughter region can not be online correctly.
The following is my questions:
1. Does anyone have setup a hbase cluster on NFS disk experience?
2. Since the basic hbse operation works fine except the region split and daughter region online. could some one please show which part of the code is responsible for this type of works?
3. When HDFS + Hbase is used, it is easy to get high performance because the scan, read and write operations can be done locally when the hbase cluster is the same as HDFS cluster. Is this correct to explain the performance descending?
Thanks.
Ling Kun
Re: Can Hbase run on NFS cluster
Posted by Andrew Purtell <ap...@apache.org>.
I don't think this makes much sense. All of your cluster talks to one
filer. Surely it will be the bottleneck of anything but a trivial test.
Just use the filer.
On Wed, Dec 19, 2012 at 10:04 PM, Ling Kun <lk...@qq.com> wrote:
> Dear all,
> I am doing some test on Hbase running on a NFS cluster. Does anyone
> have any experience?
>
> regionserver: slave1, slave2 and slave3 all mount the same NFS server
> /mnt/nfs_disk .
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: Can Hbase run on NFS cluster
Posted by Andrew Purtell <ap...@apache.org>.
I don't think this makes much sense. All of your cluster talks to one
filer. Surely it will be the bottleneck of anything but a trivial test.
Just use the filer.
On Wed, Dec 19, 2012 at 10:04 PM, Ling Kun <lk...@qq.com> wrote:
> Dear all,
> I am doing some test on Hbase running on a NFS cluster. Does anyone
> have any experience?
>
> regionserver: slave1, slave2 and slave3 all mount the same NFS server
> /mnt/nfs_disk .
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)