compressions and security

[Sam Seigal <se...@yahoo.com>]

Hi All,

A quick question on compression. I saw that HBase can use LZO compression
for storing data into the HFile.

Has anyone done experiments with using compressions at the application level
instead instead of letting HBase handle it ? Are there
advantages/disadvantages of this approach ?
Is it possible to use other compression libs in Hbase other than LZO ?

Also - are there best practices and/or conventions on encrypting data before
storing it in HBase ?

Thanks,

Sam

Re: compressions and security

[Stack <st...@duboce.net>]

What Doug said but also, if cells are of a size and/or data type that
is compressible (no point compressing small stuff, same for trying to
compress images types), then there are big wins all around if you have
the application compress the cells before putting them into hbase;
network traffic is less, the size of the data that traverses the hbase
server is less (and your caches' can carry more 'data'), etc.

stack

On Sat, Jul 16, 2011 at 4:38 AM, Doug Meil
<do...@explorysmedical.com> wrote:
>
> Hi there, see this in the book:
>
> http://hbase.apache.org/book.html#compression
>
> And this...
>
> http://hbase.apache.org/book.html#trouble.client.longpauseswithcompression
>
> And see this thread which was the original discussion on the long client
> pause entry.
>
> http://search-hadoop.com/m/WUnLM6ojHm1/Long+client+pauses+with+compression&
> subj=Long+client+pauses+with+compression
>
>
>
> On 7/16/11 5:23 AM, "Sam Seigal" <se...@yahoo.com> wrote:
>
>>Hi All,
>>
>>A quick question on compression. I saw that HBase can use LZO compression
>>for storing data into the HFile.
>>
>>Has anyone done experiments with using compressions at the application
>>level
>>instead instead of letting HBase handle it ? Are there
>>advantages/disadvantages of this approach ?
>>Is it possible to use other compression libs in Hbase other than LZO ?
>>
>>Also - are there best practices and/or conventions on encrypting data
>>before
>>storing it in HBase ?
>>
>>Thanks,
>>
>>Sam
>
>

Re: compressions and security

[Doug Meil <do...@explorysmedical.com>]

Hi there, see this in the book:

http://hbase.apache.org/book.html#compression

And this...

http://hbase.apache.org/book.html#trouble.client.longpauseswithcompression

And see this thread which was the original discussion on the long client
pause entry.

http://search-hadoop.com/m/WUnLM6ojHm1/Long+client+pauses+with+compression&
subj=Long+client+pauses+with+compression



On 7/16/11 5:23 AM, "Sam Seigal" <se...@yahoo.com> wrote:

>Hi All,
>
>A quick question on compression. I saw that HBase can use LZO compression
>for storing data into the HFile.
>
>Has anyone done experiments with using compressions at the application
>level
>instead instead of letting HBase handle it ? Are there
>advantages/disadvantages of this approach ?
>Is it possible to use other compression libs in Hbase other than LZO ?
>
>Also - are there best practices and/or conventions on encrypting data
>before
>storing it in HBase ?
>
>Thanks,
>
>Sam