You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by "leiwangouc@gmail.com" <le...@gmail.com> on 2014/08/13 09:34:35 UTC
Any fast way to random access hbase data?
I have a hbase table with more than 2G rows.
Every hour there comes 5M~10M row ids and i must get all the row info from the hbase table.
But even I use the batch call(1000 row ids as a list) as described here
http://stackoverflow.com/questions/13310434/hbase-api-get-data-rows-information-by-list-of-row-ids
It takes about 1 hour.
Any other way to do this more quickly?
Thanks,
Lei
leiwangouc@gmail.com
Re: Re: Any fast way to random access hbase data?
Posted by "leiwangouc@gmail.com" <le...@gmail.com>.
Sorry, the real region server config is this:
<name>hfile.block.cache.size</name>
<value>0.25</value>
<source>hbase-site.xml</source>
leiwangouc@gmail.com
From: Esteban Gutierrez
Date: 2014-08-14 01:05
To: user@hbase.apache.org
Subject: Re: Re: Any fast way to random access hbase data?
Hi Lei,
Any chance for you to provide the value for hfile.block.cache.size from one
of the region servers? The HBase master disables the block cache (thats why
it shows 'programatically' as the source of the config)
cheers,
esteban.
--
Cloudera, Inc.
On Wed, Aug 13, 2014 at 6:41 AM, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:
> Like what Esteban said.
>
> Try to use more threads to query HBase. Start with 10 clients, each with 1K
> gets per batch, and adjust those numbers to see the impact on the
> performances.
>
> Any reason why your block cache is disabled? (hfile.block.cache.size = 0)
>
> JM
>
>
> 2014-08-13 5:23 GMT-04:00 leiwangouc@gmail.com <le...@gmail.com>:
>
> >
> > Haven't tried yet
> > only one thread
> > 10 regions servers, total 2555 regions.
> > I am just new to HBase and not sure what exactly the block cache mean,
> > here's the configuration i can see from the CDH HBase master UI:
> > <name>hbase.rs.cacheblocksonwrite</name>
> > <value>false</value>
> > <source>hbase-default.xml</source>
> >
> > <name>hbase.offheapcache.percentage</name>
> > <value>0</value>
> > <source>hbase-default.xml</source>
> >
> > <name>hfile.block.cache.size</name>
> > <value>0.0</value>
> > <source>programatically</source>
> > Table description:
> > {NAME => 'userdigest', coprocessor$3 =>
> > 'hdfs://agrant/user/tracking/userdigest/copro
> >
> >
> cessor/endpoint_0.0.17.jar|com.agrantsem.data.userdigest.endpoint.UserdigestEndPoint|
> > 1001|', coprocessor$2 =>
> > '|org.apache.hadoop.hbase.coprocessor.AggregateImplementatio
> > n||', FAMILIES => [{NAME => 'cf', DATA_BLOCK_ENCODING => 'NONE',
> > BLOOMFILTER => 'ROWC
> > OL', REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION => 'LZ4',
> > MIN_VERSIONS =>
> > '0', TTL => '2147483647', KEEP_DELETED_CELLS => 'false', BLOCKSIZE =>
> > '65536', IN_ME
> > MORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}]}
> >
> >
> >
> > leiwangouc@gmail.com
> >
> > From: Esteban Gutierrez
> > Date: 2014-08-13 15:59
> > To: user@hbase.apache.org
> > Subject: Re: Any fast way to random access hbase data?
> > Hello Lei,
> >
> > Have you tried a larger batch size? how many threads or tasks are you
> using
> > to fetch data? could you please describe a little bit more your HBase
> > cluster? e.g. how many region servers, how many regions per RS? whats the
> > hit ratio of the block cache? any chance for you to share the table
> schema?
> >
> > cheers,
> > esteban.
> >
> >
> >
> > --
> > Cloudera, Inc.
> >
> >
> >
> > On Wed, Aug 13, 2014 at 12:34 AM, leiwangouc@gmail.com <
> > leiwangouc@gmail.com
> > > wrote:
> >
> > >
> > > I have a hbase table with more than 2G rows.
> > > Every hour there comes 5M~10M row ids and i must get all the row info
> > from
> > > the hbase table.
> > > But even I use the batch call(1000 row ids as a list) as described here
> > >
> > >
> >
> http://stackoverflow.com/questions/13310434/hbase-api-get-data-rows-information-by-list-of-row-ids
> > >
> > > It takes about 1 hour.
> > > Any other way to do this more quickly?
> > >
> > > Thanks,
> > > Lei
> > >
> > >
> > > leiwangouc@gmail.com
> > >
> >
>
Re: Re: Any fast way to random access hbase data?
Posted by Esteban Gutierrez <es...@cloudera.com>.
Hi Lei,
Any chance for you to provide the value for hfile.block.cache.size from one
of the region servers? The HBase master disables the block cache (thats why
it shows 'programatically' as the source of the config)
cheers,
esteban.
--
Cloudera, Inc.
On Wed, Aug 13, 2014 at 6:41 AM, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:
> Like what Esteban said.
>
> Try to use more threads to query HBase. Start with 10 clients, each with 1K
> gets per batch, and adjust those numbers to see the impact on the
> performances.
>
> Any reason why your block cache is disabled? (hfile.block.cache.size = 0)
>
> JM
>
>
> 2014-08-13 5:23 GMT-04:00 leiwangouc@gmail.com <le...@gmail.com>:
>
> >
> > Haven't tried yet
> > only one thread
> > 10 regions servers, total 2555 regions.
> > I am just new to HBase and not sure what exactly the block cache mean,
> > here's the configuration i can see from the CDH HBase master UI:
> > <name>hbase.rs.cacheblocksonwrite</name>
> > <value>false</value>
> > <source>hbase-default.xml</source>
> >
> > <name>hbase.offheapcache.percentage</name>
> > <value>0</value>
> > <source>hbase-default.xml</source>
> >
> > <name>hfile.block.cache.size</name>
> > <value>0.0</value>
> > <source>programatically</source>
> > Table description:
> > {NAME => 'userdigest', coprocessor$3 =>
> > 'hdfs://agrant/user/tracking/userdigest/copro
> >
> >
> cessor/endpoint_0.0.17.jar|com.agrantsem.data.userdigest.endpoint.UserdigestEndPoint|
> > 1001|', coprocessor$2 =>
> > '|org.apache.hadoop.hbase.coprocessor.AggregateImplementatio
> > n||', FAMILIES => [{NAME => 'cf', DATA_BLOCK_ENCODING => 'NONE',
> > BLOOMFILTER => 'ROWC
> > OL', REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION => 'LZ4',
> > MIN_VERSIONS =>
> > '0', TTL => '2147483647', KEEP_DELETED_CELLS => 'false', BLOCKSIZE =>
> > '65536', IN_ME
> > MORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}]}
> >
> >
> >
> > leiwangouc@gmail.com
> >
> > From: Esteban Gutierrez
> > Date: 2014-08-13 15:59
> > To: user@hbase.apache.org
> > Subject: Re: Any fast way to random access hbase data?
> > Hello Lei,
> >
> > Have you tried a larger batch size? how many threads or tasks are you
> using
> > to fetch data? could you please describe a little bit more your HBase
> > cluster? e.g. how many region servers, how many regions per RS? whats the
> > hit ratio of the block cache? any chance for you to share the table
> schema?
> >
> > cheers,
> > esteban.
> >
> >
> >
> > --
> > Cloudera, Inc.
> >
> >
> >
> > On Wed, Aug 13, 2014 at 12:34 AM, leiwangouc@gmail.com <
> > leiwangouc@gmail.com
> > > wrote:
> >
> > >
> > > I have a hbase table with more than 2G rows.
> > > Every hour there comes 5M~10M row ids and i must get all the row info
> > from
> > > the hbase table.
> > > But even I use the batch call(1000 row ids as a list) as described here
> > >
> > >
> >
> http://stackoverflow.com/questions/13310434/hbase-api-get-data-rows-information-by-list-of-row-ids
> > >
> > > It takes about 1 hour.
> > > Any other way to do this more quickly?
> > >
> > > Thanks,
> > > Lei
> > >
> > >
> > > leiwangouc@gmail.com
> > >
> >
>
Re: Re: Any fast way to random access hbase data?
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Like what Esteban said.
Try to use more threads to query HBase. Start with 10 clients, each with 1K
gets per batch, and adjust those numbers to see the impact on the
performances.
Any reason why your block cache is disabled? (hfile.block.cache.size = 0)
JM
2014-08-13 5:23 GMT-04:00 leiwangouc@gmail.com <le...@gmail.com>:
>
> Haven't tried yet
> only one thread
> 10 regions servers, total 2555 regions.
> I am just new to HBase and not sure what exactly the block cache mean,
> here's the configuration i can see from the CDH HBase master UI:
> <name>hbase.rs.cacheblocksonwrite</name>
> <value>false</value>
> <source>hbase-default.xml</source>
>
> <name>hbase.offheapcache.percentage</name>
> <value>0</value>
> <source>hbase-default.xml</source>
>
> <name>hfile.block.cache.size</name>
> <value>0.0</value>
> <source>programatically</source>
> Table description:
> {NAME => 'userdigest', coprocessor$3 =>
> 'hdfs://agrant/user/tracking/userdigest/copro
>
> cessor/endpoint_0.0.17.jar|com.agrantsem.data.userdigest.endpoint.UserdigestEndPoint|
> 1001|', coprocessor$2 =>
> '|org.apache.hadoop.hbase.coprocessor.AggregateImplementatio
> n||', FAMILIES => [{NAME => 'cf', DATA_BLOCK_ENCODING => 'NONE',
> BLOOMFILTER => 'ROWC
> OL', REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION => 'LZ4',
> MIN_VERSIONS =>
> '0', TTL => '2147483647', KEEP_DELETED_CELLS => 'false', BLOCKSIZE =>
> '65536', IN_ME
> MORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}]}
>
>
>
> leiwangouc@gmail.com
>
> From: Esteban Gutierrez
> Date: 2014-08-13 15:59
> To: user@hbase.apache.org
> Subject: Re: Any fast way to random access hbase data?
> Hello Lei,
>
> Have you tried a larger batch size? how many threads or tasks are you using
> to fetch data? could you please describe a little bit more your HBase
> cluster? e.g. how many region servers, how many regions per RS? whats the
> hit ratio of the block cache? any chance for you to share the table schema?
>
> cheers,
> esteban.
>
>
>
> --
> Cloudera, Inc.
>
>
>
> On Wed, Aug 13, 2014 at 12:34 AM, leiwangouc@gmail.com <
> leiwangouc@gmail.com
> > wrote:
>
> >
> > I have a hbase table with more than 2G rows.
> > Every hour there comes 5M~10M row ids and i must get all the row info
> from
> > the hbase table.
> > But even I use the batch call(1000 row ids as a list) as described here
> >
> >
> http://stackoverflow.com/questions/13310434/hbase-api-get-data-rows-information-by-list-of-row-ids
> >
> > It takes about 1 hour.
> > Any other way to do this more quickly?
> >
> > Thanks,
> > Lei
> >
> >
> > leiwangouc@gmail.com
> >
>
Re: Re: Any fast way to random access hbase data?
Posted by lars hofhansl <la...@apache.org>.
>
> <name>hfile.block.cache.size</name>
> <value>0.0</value>
>
Yikes. Don't do that. :)
Even if your blocks are in the OS cache, upon each single Get HBase needs to re-allocate a new 64k block on the heap (including the index blocks).
If you see no chance that a working set of the data fits into the aggregate block cache across the region serves and your reads are truly random then you can try to reduce BLOCKSIZE => '65536', maybe to 8192 or even less (but that has other downsides).
Also disable Nagle's (i.e. enable tcpnodelay at both the HBase and HDFS layers).
________________________________
From: "leiwangouc@gmail.com" <le...@gmail.com>
To: user <us...@hbase.apache.org>
Sent: Wednesday, August 13, 2014 2:23 AM
Subject: Re: Re: Any fast way to random access hbase data?
Haven't tried yet
only one thread
10 regions servers, total 2555 regions.
I am just new to HBase and not sure what exactly the block cache mean, here's the configuration i can see from the CDH HBase master UI:
<name>hbase.rs.cacheblocksonwrite</name>
<value>false</value>
<source>hbase-default.xml</source>
<name>hbase.offheapcache.percentage</name>
<value>0</value>
<source>hbase-default.xml</source>
<name>hfile.block.cache.size</name>
<value>0.0</value>
<source>programatically</source>
Table description:
{NAME => 'userdigest', coprocessor$3 => 'hdfs://agrant/user/tracking/userdigest/copro
cessor/endpoint_0.0.17.jar|com.agrantsem.data.userdigest.endpoint.UserdigestEndPoint|
1001|', coprocessor$2 => '|org.apache.hadoop.hbase.coprocessor.AggregateImplementatio
n||', FAMILIES => [{NAME => 'cf', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROWC
OL', REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION => 'LZ4', MIN_VERSIONS =>
'0', TTL => '2147483647', KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '65536', IN_ME
MORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}]}
leiwangouc@gmail.com
From: Esteban Gutierrez
Date: 2014-08-13 15:59
To: user@hbase.apache.org
Subject: Re: Any fast way to random access hbase data?
Hello Lei,
Have you tried a larger batch size? how many threads or tasks are you using
to fetch data? could you please describe a little bit more your HBase
cluster? e.g. how many region servers, how many regions per RS? whats the
hit ratio of the block cache? any chance for you to share the table schema?
cheers,
esteban.
--
Cloudera, Inc.
On Wed, Aug 13, 2014 at 12:34 AM, leiwangouc@gmail.com <leiwangouc@gmail.com
> wrote:
>
> I have a hbase table with more than 2G rows.
> Every hour there comes 5M~10M row ids and i must get all the row info from
> the hbase table.
> But even I use the batch call(1000 row ids as a list) as described here
>
> http://stackoverflow.com/questions/13310434/hbase-api-get-data-rows-information-by-list-of-row-ids
>
> It takes about 1 hour.
> Any other way to do this more quickly?
>
> Thanks,
> Lei
>
>
> leiwangouc@gmail.com
>
Re: Re: Any fast way to random access hbase data?
Posted by "leiwangouc@gmail.com" <le...@gmail.com>.
Haven't tried yet
only one thread
10 regions servers, total 2555 regions.
I am just new to HBase and not sure what exactly the block cache mean, here's the configuration i can see from the CDH HBase master UI:
<name>hbase.rs.cacheblocksonwrite</name>
<value>false</value>
<source>hbase-default.xml</source>
<name>hbase.offheapcache.percentage</name>
<value>0</value>
<source>hbase-default.xml</source>
<name>hfile.block.cache.size</name>
<value>0.0</value>
<source>programatically</source>
Table description:
{NAME => 'userdigest', coprocessor$3 => 'hdfs://agrant/user/tracking/userdigest/copro
cessor/endpoint_0.0.17.jar|com.agrantsem.data.userdigest.endpoint.UserdigestEndPoint|
1001|', coprocessor$2 => '|org.apache.hadoop.hbase.coprocessor.AggregateImplementatio
n||', FAMILIES => [{NAME => 'cf', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROWC
OL', REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION => 'LZ4', MIN_VERSIONS =>
'0', TTL => '2147483647', KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '65536', IN_ME
MORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}]}
leiwangouc@gmail.com
From: Esteban Gutierrez
Date: 2014-08-13 15:59
To: user@hbase.apache.org
Subject: Re: Any fast way to random access hbase data?
Hello Lei,
Have you tried a larger batch size? how many threads or tasks are you using
to fetch data? could you please describe a little bit more your HBase
cluster? e.g. how many region servers, how many regions per RS? whats the
hit ratio of the block cache? any chance for you to share the table schema?
cheers,
esteban.
--
Cloudera, Inc.
On Wed, Aug 13, 2014 at 12:34 AM, leiwangouc@gmail.com <leiwangouc@gmail.com
> wrote:
>
> I have a hbase table with more than 2G rows.
> Every hour there comes 5M~10M row ids and i must get all the row info from
> the hbase table.
> But even I use the batch call(1000 row ids as a list) as described here
>
> http://stackoverflow.com/questions/13310434/hbase-api-get-data-rows-information-by-list-of-row-ids
>
> It takes about 1 hour.
> Any other way to do this more quickly?
>
> Thanks,
> Lei
>
>
> leiwangouc@gmail.com
>
Re: Any fast way to random access hbase data?
Posted by Esteban Gutierrez <es...@cloudera.com>.
Hello Lei,
Have you tried a larger batch size? how many threads or tasks are you using
to fetch data? could you please describe a little bit more your HBase
cluster? e.g. how many region servers, how many regions per RS? whats the
hit ratio of the block cache? any chance for you to share the table schema?
cheers,
esteban.
--
Cloudera, Inc.
On Wed, Aug 13, 2014 at 12:34 AM, leiwangouc@gmail.com <leiwangouc@gmail.com
> wrote:
>
> I have a hbase table with more than 2G rows.
> Every hour there comes 5M~10M row ids and i must get all the row info from
> the hbase table.
> But even I use the batch call(1000 row ids as a list) as described here
>
> http://stackoverflow.com/questions/13310434/hbase-api-get-data-rows-information-by-list-of-row-ids
>
> It takes about 1 hour.
> Any other way to do this more quickly?
>
> Thanks,
> Lei
>
>
> leiwangouc@gmail.com
>