You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Rita <rm...@gmail.com> on 2012/04/18 12:56:05 UTC
compression
How is data compressed in hbase? Is there data compressed after it hits the
disk or does the client do the compression and sent it to the region
server? It would be more efficient once it goes to the memstore and sees
what other data resembles it so compression is more effective?
--
--- Get your facts first, then you can distort them as you please.--
Re: compression
Posted by Harsh J <ha...@cloudera.com>.
Hey,
Data in HBase is compressed upon compaction/flushes (i.e. upon
creation of the storefiles). Hence the compression is also done over
blocks of data (akin to SequenceFiles) and is efficient. The memstore
isn't kept compressed nor is the WAL.
RPCs in Apache HBase aren't compressed yet, but
https://issues.apache.org/jira/browse/HBASE-5355 tracks it if you want
to take a look.
HLog (WAL) compression is also available now, via
https://issues.apache.org/jira/browse/HBASE-4608 (Its in 0.94)
On Wed, Apr 18, 2012 at 4:26 PM, Rita <rm...@gmail.com> wrote:
> How is data compressed in hbase? Is there data compressed after it hits the
> disk or does the client do the compression and sent it to the region
> server? It would be more efficient once it goes to the memstore and sees
> what other data resembles it so compression is more effective?
>
>
> --
> --- Get your facts first, then you can distort them as you please.--
--
Harsh J
Re: compression
Posted by Harsh J <ha...@cloudera.com>.
Hey,
Data in HBase is compressed upon compaction/flushes (i.e. upon
creation of the storefiles). Hence the compression is also done over
blocks of data (akin to SequenceFiles) and is efficient. The memstore
isn't kept compressed nor is the WAL.
RPCs in Apache HBase aren't compressed yet, but
https://issues.apache.org/jira/browse/HBASE-5355 tracks it if you want
to take a look.
HLog (WAL) compression is also available now, via
https://issues.apache.org/jira/browse/HBASE-4608 (Its in 0.94)
On Wed, Apr 18, 2012 at 4:26 PM, Rita <rm...@gmail.com> wrote:
> How is data compressed in hbase? Is there data compressed after it hits the
> disk or does the client do the compression and sent it to the region
> server? It would be more efficient once it goes to the memstore and sees
> what other data resembles it so compression is more effective?
>
>
> --
> --- Get your facts first, then you can distort them as you please.--
--
Harsh J