You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by schausson <sc...@softera.fr> on 2016/11/25 08:53:13 UTC
Region server monitoring & data storage
Hi,
I would like to know if there is a way to monitor HBase cluster activity, in
order to check if all region servers works evenly when I try to write bulk
data from my java client application.
Is there a simple way to see that all region servers receive requests and
process data "evenly" ? Is there a way to check how/where data is stored in
HDFS by each region server/data node ?
HBase documentation
(http://hbase.apache.org/0.94/book/trouble.namenode.html#trouble.namenode.disk)
describes HDFS directory structure and I figured out that I should find one
directory per region per table in HDFS.
Unfortunately, I created my table with 11 pre-splitting regions (because my
cluster has 11 region servers/data nodes), but when I check HDFS content
after my writting operation, I can only see two sub-folders :
Example for my RAW_DATA_PARAMS_VALUES table in TEST namespace :
$ hdfs dfs -ls /apps/hbase/data/TEST/RAW_DATA_PARAMS_VALUES/
/Found 4 items
drwx------ - hbase hdfs 0 2016-11-24 17:44
/apps/hbase/data/TEST/RAW_DATA_PARAMS_VALUES/.tabledesc
drwx------ - hbase hdfs 0 2016-11-24 17:44
/apps/hbase/data/TEST/RAW_DATA_PARAMS_VALUES/.tmp
drwx------ - hbase hdfs 0 2016-11-24 17:46
/apps/hbase/data/TEST/RAW_DATA_PARAMS_VALUES/0142431be109d6c17a45335a8d7bde7a
drwx------ - hbase hdfs 0 2016-11-24 17:46
/apps/hbase/data/TEST/RAW_DATA_PARAMS_VALUES/6dff378a2e1c351837f687caa0b642c3
/
Thanks for any help !
Regards
Sebastien
--
View this message in context: http://apache-hbase.679495.n3.nabble.com/Region-server-monitoring-data-storage-tp4084683.html
Sent from the HBase User mailing list archive at Nabble.com.
Re: Region server monitoring & data storage
Posted by Ted Yu <te...@yahoo.com.INVALID>.
This means that the data you ingested only spread across these two regions, instead of 11 regions.
It was likely caused by the distribution of the row keys of ingested data.
Please examine the distribution and plan for different region boundaries if the distribution is typical for future data.
Cheers
On Friday, November 25, 2016 12:53 AM, schausson <sc...@softera.fr> wrote:
Hi,
I would like to know if there is a way to monitor HBase cluster activity, in
order to check if all region servers works evenly when I try to write bulk
data from my java client application.
Is there a simple way to see that all region servers receive requests and
process data "evenly" ? Is there a way to check how/where data is stored in
HDFS by each region server/data node ?
HBase documentation
(http://hbase.apache.org/0.94/book/trouble.namenode.html#trouble.namenode.disk)
describes HDFS directory structure and I figured out that I should find one
directory per region per table in HDFS.
Unfortunately, I created my table with 11 pre-splitting regions (because my
cluster has 11 region servers/data nodes), but when I check HDFS content
after my writting operation, I can only see two sub-folders :
Example for my RAW_DATA_PARAMS_VALUES table in TEST namespace :
$ hdfs dfs -ls /apps/hbase/data/TEST/RAW_DATA_PARAMS_VALUES/
/Found 4 items
drwx------ - hbase hdfs 0 2016-11-24 17:44
/apps/hbase/data/TEST/RAW_DATA_PARAMS_VALUES/.tabledesc
drwx------ - hbase hdfs 0 2016-11-24 17:44
/apps/hbase/data/TEST/RAW_DATA_PARAMS_VALUES/.tmp
drwx------ - hbase hdfs 0 2016-11-24 17:46
/apps/hbase/data/TEST/RAW_DATA_PARAMS_VALUES/0142431be109d6c17a45335a8d7bde7a
drwx------ - hbase hdfs 0 2016-11-24 17:46
/apps/hbase/data/TEST/RAW_DATA_PARAMS_VALUES/6dff378a2e1c351837f687caa0b642c3
/
Thanks for any help !
Regards
Sebastien
--
View this message in context: http://apache-hbase.679495.n3.nabble.com/Region-server-monitoring-data-storage-tp4084683.html
Sent from the HBase User mailing list archive at Nabble.com.