You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@phoenix.apache.org by puneet <pu...@pubmatic.com> on 2014/07/02 11:29:56 UTC
How to Use Compression in Phoenix 4.0.0
Hi Team,
I need snappy compression to be used for the Hbase table but it is not
creating the table as SNAPPY compressed when I look through the DDL's in
the Hbase shell.
Also while loading the data the size of the data remains the same for
SNAPPY or NONE.
Please help if I am missing anything.
Details Below:-
CREATE TABLE IF NOT EXISTS adv_cat (
cat varchar ,
domain VARCHAR
CONSTRAINT PK PRIMARY KEY (cat,domain)
);
DDL:- describe 'ADV_CAT_GZ'
DESCRIPTION ENABLED
'ADV_CAT_GZ', {METHOD => 'table_att', coprocessor$1 =>
'|org.apache.phoenix.coprocessor.ScanRegionObser true
ver|1|', coprocessor$2 =>
'|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
coproc
essor$3 =>
'|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1|',
coprocessor$4 => '|org.
apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
coprocessor$5 => '|org.apache.phoenix.hbase.i
ndex.Indexer|1073741823|index.builder=org.apache.phoenix.index.PhoenixIndexBuilder,org.apache.hadoop.hb
ase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec'},
{NAME => '0', DATA_BLOCK_ENCODING =
> 'FAST_DIFF', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0',
VERSIONS => '3', COMPRESSION => 'NONE',
MIN_VERSIONS => '0', TTL => '2147483647', KEEP_DELETED_CELLS =>
'true', BLOCKSIZE => '65536', IN_MEMOR
Y => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}
CREATE TABLE IF NOT EXISTS adv_cat_snappy (
cat varchar ,
domain VARCHAR
CONSTRAINT PK PRIMARY KEY (cat,domain)
) COMPRESSION='SNAPPY';
describe 'ADV_CAT_SNAPPY'
DESCRIPTION ENABLED
'ADV_CAT_SNAPPY', {METHOD => 'table_att', coprocessor$1 =>
'|org.apache.phoenix.coprocessor.ScanRegionO true
bserver|1|', coprocessor$2 =>
'|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', co
processor$3 =>
'|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1|',
coprocessor$4 => '|
org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
coprocessor$5 => '|org.apache.phoenix.hba
se.index.Indexer|1073741823|index.builder=org.apache.phoenix.index.PhoenixIndexBuilder,org.apache.hadoo
p.hbase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec'}, {NAME => '0', DATA_BLOCK_ENCODI
NG => 'FAST_DIFF', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0',
VERSIONS => '3', COMPRESSION => 'NO
NE', MIN_VERSIONS => '0', TTL => '2147483647', KEEP_DELETED_CELLS =>
'true', BLOCKSIZE => '65536', IN_M
EMORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}
1 row(s) in 0.0660 seconds
--
------------------------------------------------------------------------
*Puneet Ojha* | Tech Lead: Data Analytics
6th Floor, Amar Paradigm | Baner Road, Pune 411045
o: +91-20-67285700 | m: +91-8605359898 | Skype: puneetkr.ojha
PubMatic
<http://www.pubmatic.com/index-eng.php?utm_source=Footer&utm_medium=Email&utm_content=PuneLogo&utm_campaign=Corporate>
Every ad. Every sales channel. Every screen. *One platform*
On the Web: Facebook
<http://www.pubmatic.com/social/facebook.php?utm_source=Footer&utm_medium=Email&utm_content=PuneFacebook&utm_campaign=Corporate>
Twitter
<http://www.pubmatic.com/social/twitter.php?utm_source=Footer&utm_medium=Email&utm_content=PuneTwitter&utm_campaign=Corporate>
LinkedIn
<http://www.pubmatic.com/social/linkedin.php?utm_source=Footer&utm_medium=Email&utm_content=PuneLinkedIn&utm_campaign=Corporate>
Find out more about our services: PubConnect
<http://www.pubmatic.com/pubconnect.php?utm_source=Footer&utm_medium=Email&utm_content=PunePubConnect&utm_campaign=Corporate>
| PubDirect
<http://www.pubmatic.com/pubdirect.php?utm_source=Footer&utm_medium=Email&utm_content=PunePubDirect&utm_campaign=Corporate>
| Solutions for Media Buyers
<http://www.pubmatic.com/media-buyers-overview.php?utm_source=Footer&utm_medium=Email&utm_content=PuneMediaBuyers&utm_campaign=Corporate>
What's New: PubMatic Launches Mobile SDK for app developers.
<http://www.pubmatic.com/press/2013/PubMatic-Launches-Self-Service-Open-SDK.php?utm_source=Footer&utm_medium=Email&utm_content=PuneSDKLink&utm_campaign=Corporate>
Latest Research: Real-Time Bidding in the United States and Worldwide,
2010-2017 (Q4 2013)
<http://www.pubmatic.com/reports-and-whitepapers.php?utm_source=Footer&utm_medium=Email&utm_content=PuneReportsLink&utm_campaign=Corporate>
Recent Videos: Ad Revenue 6
<http://www.pubmatic.com/ar6_videos.php?utm_source=Footer&utm_medium=Email&utm_content=NYCAR6Link&utm_campaign=Corporate>
| Ad Revenue Europe
<http://www.pubmatic.com/arE_videos.php?utm_source=Footer&utm_medium=Email&utm_content=NYCAR6Link&utm_campaign=Corporate>
Re: How to Use Compression in Phoenix 4.0.0
Posted by puneet <pu...@pubmatic.com>.
Hi Jeff,
It worked !!!
Thanks a lot.
On Thursday 03 July 2014 12:37 AM, Jeffrey Zhong wrote:
> Have you tried to set configuration "hfile.compression"= SNAPPY in
> hbase-site.xml when using bulkloadtool?
>
> From: puneet <pu...@pubmatic.com>
> Reply-To: <us...@phoenix.apache.org>
> Date: Wednesday, July 2, 2014 5:20 AM
> To: <us...@phoenix.apache.org>
> Subject: Re: How to Use Compression in Phoenix 4.0.0
>
>
> Hi James,
>
> Thanks for you quick reply.
> ------------------------------------
>
> I had created the table with COMPRESSION='SNAPPY' property but when I
> describe it through hbase shell then it shows COMPRESION='NONE'
>
> I did a workaround and performed alter from Hbase shell as below :-
>
> disable 'ADV_CAT_SNAPPY'
> 0 row(s) in 2.0320 seconds
>
> hbase(main):027:0> alter 'ADV_CAT_SNAPPY',{NAME=>'0',COMPRESSION=>'SNAPPY'}
> Updating all regions with the new schema...
> 1/1 regions updated.
> Done.
> 0 row(s) in 1.1440 seconds
>
> hbase(main):028:0> enable 'ADV_CAT_SNAPPY'
> 0 row(s) in 2.0970 seconds
> -----------------------------
> After the above steps the compression is modifed to SNAPPY.
>
> But the compression seems to be happening only when I am Loading via PSQL
> utility.
>
>
> When I run it from Loading via MaprReduce , then it is not complressing the
> data. I have Hbase env file modifed to use Snappy lib in Hadoop
>
>
> Need steps for enabling compression via mapreduce.
>
>
>
>
>
>
> On Wednesday 02 July 2014 03:42 PM, James Taylor wrote:
>
>
>>
>> If the table already exists, then CREATE TABLE IF NOT EXISTS is a noop.
>> Drop the table first and then create initially with the COMPRESSION='SNAPPY'
>> property.
>>
>> Thanks,
>> James
>>
>>
>> On Wed, Jul 2, 2014 at 11:29 AM, puneet <pu...@pubmatic.com>
>> <ma...@pubmatic.com> wrote:
>>
>>
>>>
>>> Hi Team,
>>>
>>> I need snappy compression to be used for the Hbase table but it is not
>>> creating the table as SNAPPY compressed when I look through the DDL's in
>>> the Hbase shell.
>>>
>>> Also while loading the data the size of the data remains the same for
>>> SNAPPY or NONE.
>>>
>>> Please help if I am missing anything.
>>>
>>> Details Below:-
>>>
>>> CREATE TABLE IF NOT EXISTS adv_cat (
>>> cat varchar ,
>>> domain VARCHAR
>>> CONSTRAINT PK PRIMARY KEY (cat,domain)
>>> );
>>>
>>> DDL:- describe 'ADV_CAT_GZ'
>>> DESCRIPTION ENABLED
>>> 'ADV_CAT_GZ', {METHOD => 'table_att', coprocessor$1 =>
>>> '|org.apache.phoenix.coprocessor.ScanRegionObser true
>>> ver|1|', coprocessor$2 =>
>>> '|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
>>> coproc
>>> essor$3 =>
>>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1|',
>>> coprocessor$4 => '|org.
>>> apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|', coprocessor$5
>>> => '|org.apache.phoenix.hbase.i
>>>
>>> ndex.Indexer|1073741823|index.builder=org.apache.phoenix.index.PhoenixIndexBu
>>> ilder,org.apache.hadoop.hb
>>>
>>> ase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec'}, {NAME
>>> => '0', DATA_BLOCK_ENCODING =
>>>> > 'FAST_DIFF', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0', VERSIONS
>>> => '3', COMPRESSION => 'NONE',
>>> MIN_VERSIONS => '0', TTL => '2147483647', KEEP_DELETED_CELLS => 'true',
>>> BLOCKSIZE => '65536', IN_MEMOR
>>> Y => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}
>>>
>>>
>>> CREATE TABLE IF NOT EXISTS adv_cat_snappy (
>>> cat varchar ,
>>> domain VARCHAR
>>> CONSTRAINT PK PRIMARY KEY (cat,domain)
>>> ) COMPRESSION='SNAPPY';
>>>
>>> describe 'ADV_CAT_SNAPPY'
>>> DESCRIPTION ENABLED
>>> 'ADV_CAT_SNAPPY', {METHOD => 'table_att', coprocessor$1 =>
>>> '|org.apache.phoenix.coprocessor.ScanRegionO true
>>> bserver|1|', coprocessor$2 =>
>>> '|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', co
>>> processor$3 =>
>>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1|',
>>> coprocessor$4 => '|
>>> org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>>> coprocessor$5 => '|org.apache.phoenix.hba
>>>
>>> se.index.Indexer|1073741823|index.builder=org.apache.phoenix.index.PhoenixInd
>>> exBuilder,org.apache.hadoo
>>>
>>> p.hbase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec'},
>>> {NAME => '0', DATA_BLOCK_ENCODI
>>> NG => 'FAST_DIFF', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0',
>>> VERSIONS => '3', COMPRESSION => 'NO
>>> NE', MIN_VERSIONS => '0', TTL => '2147483647', KEEP_DELETED_CELLS =>
>>> 'true', BLOCKSIZE => '65536', IN_M
>>> EMORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}
>>> 1 row(s) in 0.0660 seconds
>>>
>>> --
>>> ------------------------------
>>>
>>> *Puneet Ojha* | Tech Lead: Data Analytics
>>> 6th Floor, Amar Paradigm | Baner Road, Pune 411045
>>> o: +91-20-67285700 | m: +91-8605359898 | Skype: puneetkr.ojha
>>>
>>> [image: PubMatic]
>>> <http://www.pubmatic.com/index-eng.php?utm_source=Footer&utm_medium=Email&utm
>>> _content=PuneLogo&utm_campaign=Corporate>
>>> <http://www.pubmatic.com/index-eng.php?utm_source=Footer&utm_medium=Email&utm
>>> _content=PuneLogo&utm_campaign=Corporate>
>>> Every ad. Every sales channel. Every screen. *One platform*
>>>
>>> On the Web: [image: Facebook]
>>> <http://www.pubmatic.com/social/facebook.php?utm_source=Footer&utm_medium=Ema
>>> il&utm_content=PuneFacebook&utm_campaign=Corporate>
>>> <http://www.pubmatic.com/social/facebook.php?utm_source=Footer&utm_medium=Ema
>>> il&utm_content=PuneFacebook&utm_campaign=Corporate> [image:
>>> Twitter]
>>> <http://www.pubmatic.com/social/twitter.php?utm_source=Footer&utm_medium=Emai
>>> l&utm_content=PuneTwitter&utm_campaign=Corporate>
>>> <http://www.pubmatic.com/social/twitter.php?utm_source=Footer&utm_medium=Emai
>>> l&utm_content=PuneTwitter&utm_campaign=Corporate> [image:
>>> LinkedIn]
>>> <http://www.pubmatic.com/social/linkedin.php?utm_source=Footer&utm_medium=Ema
>>> il&utm_content=PuneLinkedIn&utm_campaign=Corporate>
>>> <http://www.pubmatic.com/social/linkedin.php?utm_source=Footer&utm_medium=Ema
>>> il&utm_content=PuneLinkedIn&utm_campaign=Corporate>
>>>
>>> Find out more about our services: PubConnect
>>> <http://www.pubmatic.com/pubconnect.php?utm_source=Footer&utm_medium=Email&ut
>>> m_content=PunePubConnect&utm_campaign=Corporate>
>>> <http://www.pubmatic.com/pubconnect.php?utm_source=Footer&utm_medium=Email&ut
>>> m_content=PunePubConnect&utm_campaign=Corporate>
>>> | PubDirect
>>> <http://www.pubmatic.com/pubdirect.php?utm_source=Footer&utm_medium=Email&utm
>>> _content=PunePubDirect&utm_campaign=Corporate>
>>> <http://www.pubmatic.com/pubdirect.php?utm_source=Footer&utm_medium=Email&utm
>>> _content=PunePubDirect&utm_campaign=Corporate>
>>> | Solutions for Media Buyers
>>> <http://www.pubmatic.com/media-buyers-overview.php?utm_source=Footer&utm_medi
>>> um=Email&utm_content=PuneMediaBuyers&utm_campaign=Corporate>
>>> <http://www.pubmatic.com/media-buyers-overview.php?utm_source=Footer&utm_medi
>>> um=Email&utm_content=PuneMediaBuyers&utm_campaign=Corporate>
>>> What's New: PubMatic Launches Mobile SDK for app developers.
>>> <http://www.pubmatic.com/press/2013/PubMatic-Launches-Self-Service-Open-SDK.p
>>> hp?utm_source=Footer&utm_medium=Email&utm_content=PuneSDKLink&utm_campaign=Co
>>> rporate>
>>> <http://www.pubmatic.com/press/2013/PubMatic-Launches-Self-Service-Open-SDK.p
>>> hp?utm_source=Footer&utm_medium=Email&utm_content=PuneSDKLink&utm_campaign=Co
>>> rporate>
>>> Latest Research: Real-Time Bidding in the United States and Worldwide,
>>> 2010-2017 (Q4 2013)
>>> <http://www.pubmatic.com/reports-and-whitepapers.php?utm_source=Footer&utm_me
>>> dium=Email&utm_content=PuneReportsLink&utm_campaign=Corporate>
>>> <http://www.pubmatic.com/reports-and-whitepapers.php?utm_source=Footer&utm_me
>>> dium=Email&utm_content=PuneReportsLink&utm_campaign=Corporate>
>>> Recent Videos: Ad Revenue 6
>>> <http://www.pubmatic.com/ar6_videos.php?utm_source=Footer&utm_medium=Email&ut
>>> m_content=NYCAR6Link&utm_campaign=Corporate>
>>> <http://www.pubmatic.com/ar6_videos.php?utm_source=Footer&utm_medium=Email&ut
>>> m_content=NYCAR6Link&utm_campaign=Corporate>
>>> | Ad Revenue Europe
>>> <http://www.pubmatic.com/arE_videos.php?utm_source=Footer&utm_medium=Email&ut
>>> m_content=NYCAR6Link&utm_campaign=Corporate>
>>> <http://www.pubmatic.com/arE_videos.php?utm_source=Footer&utm_medium=Email&ut
>>> m_content=NYCAR6Link&utm_campaign=Corporate>
>>>
>>>
>>>
>>> If the table already exists, then CREATE TABLE IF NOT EXISTS is a noop. Drop
>>> the table first and then create initially with the COMPRESSION='SNAPPY'
>>> property.
>>>
>>>
>>>
>>> Thanks,
>>>
>>> James
>>>
>>>
>>>
>>>
>>> On Wed, Jul 2, 2014 at 11:29 AM, puneet <pu...@pubmatic.com> wrote:
>>>
>>>>
>>>> Hi Team,
>>>>
>>>> I need snappy compression to be used for the Hbase table but it is not
>>>> creating the table as SNAPPY compressed when I look through the DDL's in the
>>>> Hbase shell.
>>>>
>>>> Also while loading the data the size of the data remains the same for
>>>> SNAPPY or NONE.
>>>>
>>>> Please help if I am missing anything.
>>>>
>>>> Details Below:-
>>>>
>>>> CREATE TABLE IF NOT EXISTS adv_cat (
>>>> cat varchar ,
>>>> domain VARCHAR
>>>> CONSTRAINT PK PRIMARY KEY (cat,domain)
>>>> );
>>>>
>>>> DDL:- describe 'ADV_CAT_GZ'
>>>> DESCRIPTION ENABLED
>>>> 'ADV_CAT_GZ', {METHOD => 'table_att', coprocessor$1 =>
>>>> '|org.apache.phoenix.coprocessor.ScanRegionObser true
>>>> ver|1|', coprocessor$2 =>
>>>> '|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
>>>> coproc
>>>> essor$3 =>
>>>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1|',
>>>> coprocessor$4 => '|org.
>>>> apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|', coprocessor$5
>>>> => '|org.apache.phoenix.hbase.i
>>>>
>>>> ndex.Indexer|1073741823|index.builder=org.apache.phoenix.index.PhoenixIndexB
>>>> uilder,org.apache.hadoop.hb
>>>> ase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec'}, {NAME
>>>> => '0', DATA_BLOCK_ENCODING =
>>>>> > 'FAST_DIFF', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0', VERSIONS
>>>>> => '3', COMPRESSION => 'NONE',
>>>> MIN_VERSIONS => '0', TTL => '2147483647 <tel:2147483647> ',
>>>> KEEP_DELETED_CELLS => 'true', BLOCKSIZE => '65536', IN_MEMOR
>>>> Y => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}
>>>>
>>>>
>>>> CREATE TABLE IF NOT EXISTS adv_cat_snappy (
>>>> cat varchar ,
>>>> domain VARCHAR
>>>> CONSTRAINT PK PRIMARY KEY (cat,domain)
>>>> ) COMPRESSION='SNAPPY';
>>>>
>>>> describe 'ADV_CAT_SNAPPY'
>>>> DESCRIPTION ENABLED
>>>> 'ADV_CAT_SNAPPY', {METHOD => 'table_att', coprocessor$1 =>
>>>> '|org.apache.phoenix.coprocessor.ScanRegionO true
>>>> bserver|1|', coprocessor$2 =>
>>>> '|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', co
>>>> processor$3 =>
>>>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1|',
>>>> coprocessor$4 => '|
>>>> org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>>>> coprocessor$5 => '|org.apache.phoenix.hba
>>>>
>>>> se.index.Indexer|1073741823|index.builder=org.apache.phoenix.index.PhoenixIn
>>>> dexBuilder,org.apache.hadoo
>>>> p.hbase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec'},
>>>> {NAME => '0', DATA_BLOCK_ENCODI
>>>> NG => 'FAST_DIFF', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0',
>>>> VERSIONS => '3', COMPRESSION => 'NO
>>>> NE', MIN_VERSIONS => '0', TTL => '2147483647 <tel:2147483647> ',
>>>> KEEP_DELETED_CELLS => 'true', BLOCKSIZE => '65536', IN_M
>>>> EMORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}
>>>> 1 row(s) in 0.0660 seconds
>>>>
>>>>
>>>> --
>>>>
>>>>
>>>>
>>>>
>>>> Puneet Ojha | Tech Lead: Data Analytics
>>>> 6th Floor, Amar Paradigm | Baner Road, Pune 411045
>>>> o: +91-20-67285700 <tel:%2B91-20-67285700> | m: +91-8605359898
>>>> <tel:%2B91-8605359898> | Skype: puneetkr.ojha
>>>>
>>>>
>>>>
>>>> <http://www.pubmatic.com/index-eng.php?utm_source=Footer&utm_medium=Email&ut
>>>> m_content=PuneLogo&utm_campaign=Corporate>
>>>> Every ad. Every sales channel. Every screen. One platform
>>>>
>>>>
>>>> On the Web:
>>>> <http://www.pubmatic.com/social/facebook.php?utm_source=Footer&utm_medium=Em
>>>> ail&utm_content=PuneFacebook&utm_campaign=Corporate>
>>>> <http://www.pubmatic.com/social/twitter.php?utm_source=Footer&utm_medium=Ema
>>>> il&utm_content=PuneTwitter&utm_campaign=Corporate>
>>>> <http://www.pubmatic.com/social/linkedin.php?utm_source=Footer&utm_medium=Em
>>>> ail&utm_content=PuneLinkedIn&utm_campaign=Corporate>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Find out more about our services: PubConnect
>>>> <http://www.pubmatic.com/pubconnect.php?utm_source=Footer&utm_medium=Email&u
>>>> tm_content=PunePubConnect&utm_campaign=Corporate> | PubDirect
>>>> <http://www.pubmatic.com/pubdirect.php?utm_source=Footer&utm_medium=Email&ut
>>>> m_content=PunePubDirect&utm_campaign=Corporate> | Solutions for Media
>>>> Buyers
>>>> <http://www.pubmatic.com/media-buyers-overview.php?utm_source=Footer&utm_med
>>>> ium=Email&utm_content=PuneMediaBuyers&utm_campaign=Corporate>
>>>> What's New: PubMatic Launches Mobile SDK for app developers.
>>>> <http://www.pubmatic.com/press/2013/PubMatic-Launches-Self-Service-Open-SDK.
>>>> php?utm_source=Footer&utm_medium=Email&utm_content=PuneSDKLink&utm_campaign=
>>>> Corporate>
>>>> Latest Research: Real-Time Bidding in the United States and Worldwide,
>>>> 2010-2017 (Q4 2013)
>>>> <http://www.pubmatic.com/reports-and-whitepapers.php?utm_source=Footer&utm_m
>>>> edium=Email&utm_content=PuneReportsLink&utm_campaign=Corporate>
>>>> Recent Videos: Ad Revenue 6
>>>> <http://www.pubmatic.com/ar6_videos.php?utm_source=Footer&utm_medium=Email&u
>>>> tm_content=NYCAR6Link&utm_campaign=Corporate> | Ad Revenue Europe
>>>> <http://www.pubmatic.com/arE_videos.php?utm_source=Footer&utm_medium=Email&u
>>>> tm_content=NYCAR6Link&utm_campaign=Corporate>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>>>
>>
>
>
>
>
>
> Have you tried to set configuration "hfile.compression"= SNAPPY in
> hbase-site.xml when using bulkloadtool?
>
> From: puneet <puneet.kumar@pubmatic.com
> <ma...@pubmatic.com>>
> Reply-To: <user@phoenix.apache.org <ma...@phoenix.apache.org>>
> Date: Wednesday, July 2, 2014 5:20 AM
> To: <user@phoenix.apache.org <ma...@phoenix.apache.org>>
> Subject: Re: How to Use Compression in Phoenix 4.0.0
>
> Hi James,
>
> Thanks for you quick reply.
> ------------------------------------
>
> I had created the table with COMPRESSION='SNAPPY' property but when I
> describe it through hbase shell then it shows COMPRESION='NONE'
>
> I did a workaround and performed alter from Hbase shell as below :-
>
> disable 'ADV_CAT_SNAPPY'
> 0 row(s) in 2.0320 seconds
>
> hbase(main):027:0> alter
> 'ADV_CAT_SNAPPY',{NAME=>'0',COMPRESSION=>'SNAPPY'}
> Updating all regions with the new schema...
> 1/1 regions updated.
> Done.
> 0 row(s) in 1.1440 seconds
>
> hbase(main):028:0> enable 'ADV_CAT_SNAPPY'
> 0 row(s) in 2.0970 seconds
> -----------------------------
> After the above steps the compression is modifed to SNAPPY.
>
> But the compression seems to be happening only when I am Loading via
> PSQL utility.
>
> When I run it from Loading via MaprReduce , then it is not
> complressing the data. I have Hbase env file modifed to use Snappy lib
> in Hadoop
>
>
> Need steps for enabling compression via mapreduce.
>
>
>
> On Wednesday 02 July 2014 03:42 PM, James Taylor wrote:
>> If the table already exists, then CREATE TABLE IF NOT EXISTS is a noop.
>> Drop the table first and then create initially with the COMPRESSION='SNAPPY'
>> property.
>>
>> Thanks,
>> James
>>
>>
>> On Wed, Jul 2, 2014 at 11:29 AM, puneet<pu...@pubmatic.com> wrote:
>>
>>> Hi Team,
>>>
>>> I need snappy compression to be used for the Hbase table but it is not
>>> creating the table as SNAPPY compressed when I look through the DDL's in
>>> the Hbase shell.
>>>
>>> Also while loading the data the size of the data remains the same for
>>> SNAPPY or NONE.
>>>
>>> Please help if I am missing anything.
>>>
>>> Details Below:-
>>>
>>> CREATE TABLE IF NOT EXISTS adv_cat (
>>> cat varchar ,
>>> domain VARCHAR
>>> CONSTRAINT PK PRIMARY KEY (cat,domain)
>>> );
>>>
>>> DDL:- describe 'ADV_CAT_GZ'
>>> DESCRIPTION ENABLED
>>> 'ADV_CAT_GZ', {METHOD => 'table_att', coprocessor$1 =>
>>> '|org.apache.phoenix.coprocessor.ScanRegionObser true
>>> ver|1|', coprocessor$2 =>
>>> '|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
>>> coproc
>>> essor$3 =>
>>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1|',
>>> coprocessor$4 => '|org.
>>> apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|', coprocessor$5
>>> => '|org.apache.phoenix.hbase.i
>>> ndex.Indexer|1073741823|index.builder=org.apache.phoenix.index.PhoenixIndexBuilder,org.apache.hadoop.hb
>>>
>>> ase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec'}, {NAME
>>> => '0', DATA_BLOCK_ENCODING =
>>> > 'FAST_DIFF', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0', VERSIONS
>>> => '3', COMPRESSION => 'NONE',
>>> MIN_VERSIONS => '0', TTL => '2147483647', KEEP_DELETED_CELLS => 'true',
>>> BLOCKSIZE => '65536', IN_MEMOR
>>> Y => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}
>>>
>>>
>>> CREATE TABLE IF NOT EXISTS adv_cat_snappy (
>>> cat varchar ,
>>> domain VARCHAR
>>> CONSTRAINT PK PRIMARY KEY (cat,domain)
>>> ) COMPRESSION='SNAPPY';
>>>
>>> describe 'ADV_CAT_SNAPPY'
>>> DESCRIPTION ENABLED
>>> 'ADV_CAT_SNAPPY', {METHOD => 'table_att', coprocessor$1 =>
>>> '|org.apache.phoenix.coprocessor.ScanRegionO true
>>> bserver|1|', coprocessor$2 =>
>>> '|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', co
>>> processor$3 =>
>>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1|',
>>> coprocessor$4 => '|
>>> org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>>> coprocessor$5 => '|org.apache.phoenix.hba
>>> se.index.Indexer|1073741823|index.builder=org.apache.phoenix.index.PhoenixIndexBuilder,org.apache.hadoo
>>>
>>> p.hbase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec'},
>>> {NAME => '0', DATA_BLOCK_ENCODI
>>> NG => 'FAST_DIFF', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0',
>>> VERSIONS => '3', COMPRESSION => 'NO
>>> NE', MIN_VERSIONS => '0', TTL => '2147483647', KEEP_DELETED_CELLS =>
>>> 'true', BLOCKSIZE => '65536', IN_M
>>> EMORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}
>>> 1 row(s) in 0.0660 seconds
>>>
>>> --
>>> ------------------------------
>>>
>>> *Puneet Ojha* | Tech Lead: Data Analytics
>>> 6th Floor, Amar Paradigm | Baner Road, Pune 411045
>>> o: +91-20-67285700 | m: +91-8605359898 | Skype: puneetkr.ojha
>>>
>>> [image: PubMatic]
>>> <http://www.pubmatic.com/index-eng.php?utm_source=Footer&utm_medium=Email&utm_content=PuneLogo&utm_campaign=Corporate>
>>> Every ad. Every sales channel. Every screen. *One platform*
>>>
>>> On the Web: [image: Facebook]
>>> <http://www.pubmatic.com/social/facebook.php?utm_source=Footer&utm_medium=Email&utm_content=PuneFacebook&utm_campaign=Corporate> [image:
>>> Twitter]
>>> <http://www.pubmatic.com/social/twitter.php?utm_source=Footer&utm_medium=Email&utm_content=PuneTwitter&utm_campaign=Corporate> [image:
>>> LinkedIn]
>>> <http://www.pubmatic.com/social/linkedin.php?utm_source=Footer&utm_medium=Email&utm_content=PuneLinkedIn&utm_campaign=Corporate>
>>>
>>> Find out more about our services: PubConnect
>>> <http://www.pubmatic.com/pubconnect.php?utm_source=Footer&utm_medium=Email&utm_content=PunePubConnect&utm_campaign=Corporate>
>>> | PubDirect
>>> <http://www.pubmatic.com/pubdirect.php?utm_source=Footer&utm_medium=Email&utm_content=PunePubDirect&utm_campaign=Corporate>
>>> | Solutions for Media Buyers
>>> <http://www.pubmatic.com/media-buyers-overview.php?utm_source=Footer&utm_medium=Email&utm_content=PuneMediaBuyers&utm_campaign=Corporate>
>>> What's New: PubMatic Launches Mobile SDK for app developers.
>>> <http://www.pubmatic.com/press/2013/PubMatic-Launches-Self-Service-Open-SDK.php?utm_source=Footer&utm_medium=Email&utm_content=PuneSDKLink&utm_campaign=Corporate>
>>> Latest Research: Real-Time Bidding in the United States and Worldwide,
>>> 2010-2017 (Q4 2013)
>>> <http://www.pubmatic.com/reports-and-whitepapers.php?utm_source=Footer&utm_medium=Email&utm_content=PuneReportsLink&utm_campaign=Corporate>
>>> Recent Videos: Ad Revenue 6
>>> <http://www.pubmatic.com/ar6_videos.php?utm_source=Footer&utm_medium=Email&utm_content=NYCAR6Link&utm_campaign=Corporate>
>>> | Ad Revenue Europe
>>> <http://www.pubmatic.com/arE_videos.php?utm_source=Footer&utm_medium=Email&utm_content=NYCAR6Link&utm_campaign=Corporate>
>>>
>>>
>>> If the table already exists, then CREATE TABLE IF NOT EXISTS is a
>>> noop. Drop the table first and then create initially with the
>>> COMPRESSION='SNAPPY' property.
>>>
>>> Thanks,
>>> James
>>>
>>>
>>> On Wed, Jul 2, 2014 at 11:29 AM, puneet <puneet.kumar@pubmatic.com
>>> <ma...@pubmatic.com>> wrote:
>>>
>>> Hi Team,
>>>
>>> I need snappy compression to be used for the Hbase table but it
>>> is not creating the table as SNAPPY compressed when I look
>>> through the DDL's in the Hbase shell.
>>>
>>> Also while loading the data the size of the data remains the
>>> same for SNAPPY or NONE.
>>>
>>> Please help if I am missing anything.
>>>
>>> Details Below:-
>>>
>>> CREATE TABLE IF NOT EXISTS adv_cat (
>>> cat varchar ,
>>> domain VARCHAR
>>> CONSTRAINT PK PRIMARY KEY (cat,domain)
>>> );
>>>
>>> DDL:- describe 'ADV_CAT_GZ'
>>> DESCRIPTION ENABLED
>>> 'ADV_CAT_GZ', {METHOD => 'table_att', coprocessor$1 =>
>>> '|org.apache.phoenix.coprocessor.ScanRegionObser true
>>> ver|1|', coprocessor$2 =>
>>> '|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
>>> coproc
>>> essor$3 =>
>>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1|',
>>> coprocessor$4 => '|org.
>>> apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>>> coprocessor$5 => '|org.apache.phoenix.hbase.i
>>> ndex.Indexer|1073741823|index.builder=org.apache.phoenix.index.PhoenixIndexBuilder,org.apache.hadoop.hb
>>>
>>> ase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec'},
>>> {NAME => '0', DATA_BLOCK_ENCODING =
>>> > 'FAST_DIFF', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0',
>>> VERSIONS => '3', COMPRESSION => 'NONE',
>>> MIN_VERSIONS => '0', TTL => '2147483647 <tel:2147483647>',
>>> KEEP_DELETED_CELLS => 'true', BLOCKSIZE => '65536', IN_MEMOR
>>> Y => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}
>>>
>>>
>>> CREATE TABLE IF NOT EXISTS adv_cat_snappy (
>>> cat varchar ,
>>> domain VARCHAR
>>> CONSTRAINT PK PRIMARY KEY (cat,domain)
>>> ) COMPRESSION='SNAPPY';
>>>
>>> describe 'ADV_CAT_SNAPPY'
>>> DESCRIPTION ENABLED
>>> 'ADV_CAT_SNAPPY', {METHOD => 'table_att', coprocessor$1 =>
>>> '|org.apache.phoenix.coprocessor.ScanRegionO true
>>> bserver|1|', coprocessor$2 =>
>>> '|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
>>> co
>>> processor$3 =>
>>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1|',
>>> coprocessor$4 => '|
>>> org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>>> coprocessor$5 => '|org.apache.phoenix.hba
>>> se.index.Indexer|1073741823|index.builder=org.apache.phoenix.index.PhoenixIndexBuilder,org.apache.hadoo
>>>
>>> p.hbase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec'},
>>> {NAME => '0', DATA_BLOCK_ENCODI
>>> NG => 'FAST_DIFF', BLOOMFILTER => 'NONE', REPLICATION_SCOPE =>
>>> '0', VERSIONS => '3', COMPRESSION => 'NO
>>> NE', MIN_VERSIONS => '0', TTL => '2147483647 <tel:2147483647>',
>>> KEEP_DELETED_CELLS => 'true', BLOCKSIZE => '65536', IN_M
>>> EMORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}
>>> 1 row(s) in 0.0660 seconds
>>>
>>> --
>>> ------------------------------------------------------------------------
>>>
>>> *Puneet Ojha* | Tech Lead: Data Analytics
>>> 6th Floor, Amar Paradigm | Baner Road, Pune 411045
>>> o: +91-20-67285700 <tel:%2B91-20-67285700> | m: +91-8605359898
>>> <tel:%2B91-8605359898> | Skype: puneetkr.ojha
>>>
>>> PubMatic
>>> <http://www.pubmatic.com/index-eng.php?utm_source=Footer&utm_medium=Email&utm_content=PuneLogo&utm_campaign=Corporate>
>>> Every ad. Every sales channel. Every screen. *One platform*
>>>
>>> On the Web: Facebook
>>> <http://www.pubmatic.com/social/facebook.php?utm_source=Footer&utm_medium=Email&utm_content=PuneFacebook&utm_campaign=Corporate>
>>> Twitter
>>> <http://www.pubmatic.com/social/twitter.php?utm_source=Footer&utm_medium=Email&utm_content=PuneTwitter&utm_campaign=Corporate>
>>> LinkedIn
>>> <http://www.pubmatic.com/social/linkedin.php?utm_source=Footer&utm_medium=Email&utm_content=PuneLinkedIn&utm_campaign=Corporate>
>>>
>>> Find out more about our services: PubConnect
>>> <http://www.pubmatic.com/pubconnect.php?utm_source=Footer&utm_medium=Email&utm_content=PunePubConnect&utm_campaign=Corporate>
>>> | PubDirect
>>> <http://www.pubmatic.com/pubdirect.php?utm_source=Footer&utm_medium=Email&utm_content=PunePubDirect&utm_campaign=Corporate>
>>> | Solutions for Media Buyers
>>> <http://www.pubmatic.com/media-buyers-overview.php?utm_source=Footer&utm_medium=Email&utm_content=PuneMediaBuyers&utm_campaign=Corporate>
>>> What's New: PubMatic Launches Mobile SDK for app developers.
>>> <http://www.pubmatic.com/press/2013/PubMatic-Launches-Self-Service-Open-SDK.php?utm_source=Footer&utm_medium=Email&utm_content=PuneSDKLink&utm_campaign=Corporate>
>>> Latest Research: Real-Time Bidding in the United States and
>>> Worldwide, 2010-2017 (Q4 2013)
>>> <http://www.pubmatic.com/reports-and-whitepapers.php?utm_source=Footer&utm_medium=Email&utm_content=PuneReportsLink&utm_campaign=Corporate>
>>> Recent Videos: Ad Revenue 6
>>> <http://www.pubmatic.com/ar6_videos.php?utm_source=Footer&utm_medium=Email&utm_content=NYCAR6Link&utm_campaign=Corporate>
>>> | Ad Revenue Europe
>>> <http://www.pubmatic.com/arE_videos.php?utm_source=Footer&utm_medium=Email&utm_content=NYCAR6Link&utm_campaign=Corporate>
>>>
>>>
>
> --
> ------------------------------------------------------------------------
>
> *Puneet Ojha* | Tech Lead: Data Analytics
> 6th Floor, Amar Paradigm | Baner Road, Pune 411045
> o: +91-20-67285700 | m: +91-8605359898 | Skype: puneetkr.ojha
>
> PubMatic
> <http://www.pubmatic.com/index-eng.php?utm_source=Footer&utm_medium=Email&utm_content=PuneLogo&utm_campaign=Corporate>
> Every ad. Every sales channel. Every screen. *One platform*
>
> On the Web: Facebook
> <http://www.pubmatic.com/social/facebook.php?utm_source=Footer&utm_medium=Email&utm_content=PuneFacebook&utm_campaign=Corporate>
> Twitter
> <http://www.pubmatic.com/social/twitter.php?utm_source=Footer&utm_medium=Email&utm_content=PuneTwitter&utm_campaign=Corporate>
> LinkedIn
> <http://www.pubmatic.com/social/linkedin.php?utm_source=Footer&utm_medium=Email&utm_content=PuneLinkedIn&utm_campaign=Corporate>
>
> Find out more about our services: PubConnect
> <http://www.pubmatic.com/pubconnect.php?utm_source=Footer&utm_medium=Email&utm_content=PunePubConnect&utm_campaign=Corporate>
> | PubDirect
> <http://www.pubmatic.com/pubdirect.php?utm_source=Footer&utm_medium=Email&utm_content=PunePubDirect&utm_campaign=Corporate>
> | Solutions for Media Buyers
> <http://www.pubmatic.com/media-buyers-overview.php?utm_source=Footer&utm_medium=Email&utm_content=PuneMediaBuyers&utm_campaign=Corporate>
> What's New: PubMatic Launches Mobile SDK for app developers.
> <http://www.pubmatic.com/press/2013/PubMatic-Launches-Self-Service-Open-SDK.php?utm_source=Footer&utm_medium=Email&utm_content=PuneSDKLink&utm_campaign=Corporate>
> Latest Research: Real-Time Bidding in the United States and Worldwide,
> 2010-2017 (Q4 2013)
> <http://www.pubmatic.com/reports-and-whitepapers.php?utm_source=Footer&utm_medium=Email&utm_content=PuneReportsLink&utm_campaign=Corporate>
> Recent Videos: Ad Revenue 6
> <http://www.pubmatic.com/ar6_videos.php?utm_source=Footer&utm_medium=Email&utm_content=NYCAR6Link&utm_campaign=Corporate>
> | Ad Revenue Europe
> <http://www.pubmatic.com/arE_videos.php?utm_source=Footer&utm_medium=Email&utm_content=NYCAR6Link&utm_campaign=Corporate>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or
> entity to which it is addressed and may contain information that is
> confidential, privileged and exempt from disclosure under applicable
> law. If the reader of this message is not the intended recipient, you
> are hereby notified that any printing, copying, dissemination,
> distribution, disclosure or forwarding of this communication is
> strictly prohibited. If you have received this communication in error,
> please contact the sender immediately and delete it from your system.
> Thank You.
--
------------------------------------------------------------------------
*Puneet Ojha* | Tech Lead: Data Analytics
6th Floor, Amar Paradigm | Baner Road, Pune 411045
o: +91-20-67285700 | m: +91-8605359898 | Skype: puneetkr.ojha
PubMatic
<http://www.pubmatic.com/index-eng.php?utm_source=Footer&utm_medium=Email&utm_content=PuneLogo&utm_campaign=Corporate>
Every ad. Every sales channel. Every screen. *One platform*
On the Web: Facebook
<http://www.pubmatic.com/social/facebook.php?utm_source=Footer&utm_medium=Email&utm_content=PuneFacebook&utm_campaign=Corporate>
Twitter
<http://www.pubmatic.com/social/twitter.php?utm_source=Footer&utm_medium=Email&utm_content=PuneTwitter&utm_campaign=Corporate>
LinkedIn
<http://www.pubmatic.com/social/linkedin.php?utm_source=Footer&utm_medium=Email&utm_content=PuneLinkedIn&utm_campaign=Corporate>
Find out more about our services: PubConnect
<http://www.pubmatic.com/pubconnect.php?utm_source=Footer&utm_medium=Email&utm_content=PunePubConnect&utm_campaign=Corporate>
| PubDirect
<http://www.pubmatic.com/pubdirect.php?utm_source=Footer&utm_medium=Email&utm_content=PunePubDirect&utm_campaign=Corporate>
| Solutions for Media Buyers
<http://www.pubmatic.com/media-buyers-overview.php?utm_source=Footer&utm_medium=Email&utm_content=PuneMediaBuyers&utm_campaign=Corporate>
What's New: PubMatic Launches Mobile SDK for app developers.
<http://www.pubmatic.com/press/2013/PubMatic-Launches-Self-Service-Open-SDK.php?utm_source=Footer&utm_medium=Email&utm_content=PuneSDKLink&utm_campaign=Corporate>
Latest Research: Real-Time Bidding in the United States and Worldwide,
2010-2017 (Q4 2013)
<http://www.pubmatic.com/reports-and-whitepapers.php?utm_source=Footer&utm_medium=Email&utm_content=PuneReportsLink&utm_campaign=Corporate>
Recent Videos: Ad Revenue 6
<http://www.pubmatic.com/ar6_videos.php?utm_source=Footer&utm_medium=Email&utm_content=NYCAR6Link&utm_campaign=Corporate>
| Ad Revenue Europe
<http://www.pubmatic.com/arE_videos.php?utm_source=Footer&utm_medium=Email&utm_content=NYCAR6Link&utm_campaign=Corporate>
Re: How to Use Compression in Phoenix 4.0.0
Posted by Jeffrey Zhong <jz...@hortonworks.com>.
Have you tried to set configuration "hfile.compression"= SNAPPY in
hbase-site.xml when using bulkloadtool?
From: puneet <pu...@pubmatic.com>
Reply-To: <us...@phoenix.apache.org>
Date: Wednesday, July 2, 2014 5:20 AM
To: <us...@phoenix.apache.org>
Subject: Re: How to Use Compression in Phoenix 4.0.0
Hi James,
Thanks for you quick reply.
------------------------------------
I had created the table with COMPRESSION='SNAPPY' property but when I
describe it through hbase shell then it shows COMPRESION='NONE'
I did a workaround and performed alter from Hbase shell as below :-
disable 'ADV_CAT_SNAPPY'
0 row(s) in 2.0320 seconds
hbase(main):027:0> alter 'ADV_CAT_SNAPPY',{NAME=>'0',COMPRESSION=>'SNAPPY'}
Updating all regions with the new schema...
1/1 regions updated.
Done.
0 row(s) in 1.1440 seconds
hbase(main):028:0> enable 'ADV_CAT_SNAPPY'
0 row(s) in 2.0970 seconds
-----------------------------
After the above steps the compression is modifed to SNAPPY.
But the compression seems to be happening only when I am Loading via PSQL
utility.
When I run it from Loading via MaprReduce , then it is not complressing the
data. I have Hbase env file modifed to use Snappy lib in Hadoop
Need steps for enabling compression via mapreduce.
On Wednesday 02 July 2014 03:42 PM, James Taylor wrote:
>
> If the table already exists, then CREATE TABLE IF NOT EXISTS is a noop.
> Drop the table first and then create initially with the COMPRESSION='SNAPPY'
> property.
>
> Thanks,
> James
>
>
> On Wed, Jul 2, 2014 at 11:29 AM, puneet <pu...@pubmatic.com>
> <ma...@pubmatic.com> wrote:
>
>
>>
>> Hi Team,
>>
>> I need snappy compression to be used for the Hbase table but it is not
>> creating the table as SNAPPY compressed when I look through the DDL's in
>> the Hbase shell.
>>
>> Also while loading the data the size of the data remains the same for
>> SNAPPY or NONE.
>>
>> Please help if I am missing anything.
>>
>> Details Below:-
>>
>> CREATE TABLE IF NOT EXISTS adv_cat (
>> cat varchar ,
>> domain VARCHAR
>> CONSTRAINT PK PRIMARY KEY (cat,domain)
>> );
>>
>> DDL:- describe 'ADV_CAT_GZ'
>> DESCRIPTION ENABLED
>> 'ADV_CAT_GZ', {METHOD => 'table_att', coprocessor$1 =>
>> '|org.apache.phoenix.coprocessor.ScanRegionObser true
>> ver|1|', coprocessor$2 =>
>> '|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
>> coproc
>> essor$3 =>
>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1|',
>> coprocessor$4 => '|org.
>> apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|', coprocessor$5
>> => '|org.apache.phoenix.hbase.i
>>
>> ndex.Indexer|1073741823|index.builder=org.apache.phoenix.index.PhoenixIndexBu
>> ilder,org.apache.hadoop.hb
>>
>> ase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec'}, {NAME
>> => '0', DATA_BLOCK_ENCODING =
>>> > 'FAST_DIFF', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0', VERSIONS
>> => '3', COMPRESSION => 'NONE',
>> MIN_VERSIONS => '0', TTL => '2147483647', KEEP_DELETED_CELLS => 'true',
>> BLOCKSIZE => '65536', IN_MEMOR
>> Y => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}
>>
>>
>> CREATE TABLE IF NOT EXISTS adv_cat_snappy (
>> cat varchar ,
>> domain VARCHAR
>> CONSTRAINT PK PRIMARY KEY (cat,domain)
>> ) COMPRESSION='SNAPPY';
>>
>> describe 'ADV_CAT_SNAPPY'
>> DESCRIPTION ENABLED
>> 'ADV_CAT_SNAPPY', {METHOD => 'table_att', coprocessor$1 =>
>> '|org.apache.phoenix.coprocessor.ScanRegionO true
>> bserver|1|', coprocessor$2 =>
>> '|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', co
>> processor$3 =>
>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1|',
>> coprocessor$4 => '|
>> org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>> coprocessor$5 => '|org.apache.phoenix.hba
>>
>> se.index.Indexer|1073741823|index.builder=org.apache.phoenix.index.PhoenixInd
>> exBuilder,org.apache.hadoo
>>
>> p.hbase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec'},
>> {NAME => '0', DATA_BLOCK_ENCODI
>> NG => 'FAST_DIFF', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0',
>> VERSIONS => '3', COMPRESSION => 'NO
>> NE', MIN_VERSIONS => '0', TTL => '2147483647', KEEP_DELETED_CELLS =>
>> 'true', BLOCKSIZE => '65536', IN_M
>> EMORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}
>> 1 row(s) in 0.0660 seconds
>>
>> --
>> ------------------------------
>>
>> *Puneet Ojha* | Tech Lead: Data Analytics
>> 6th Floor, Amar Paradigm | Baner Road, Pune 411045
>> o: +91-20-67285700 | m: +91-8605359898 | Skype: puneetkr.ojha
>>
>> [image: PubMatic]
>> <http://www.pubmatic.com/index-eng.php?utm_source=Footer&utm_medium=Email&utm
>> _content=PuneLogo&utm_campaign=Corporate>
>> <http://www.pubmatic.com/index-eng.php?utm_source=Footer&utm_medium=Email&utm
>> _content=PuneLogo&utm_campaign=Corporate>
>> Every ad. Every sales channel. Every screen. *One platform*
>>
>> On the Web: [image: Facebook]
>> <http://www.pubmatic.com/social/facebook.php?utm_source=Footer&utm_medium=Ema
>> il&utm_content=PuneFacebook&utm_campaign=Corporate>
>> <http://www.pubmatic.com/social/facebook.php?utm_source=Footer&utm_medium=Ema
>> il&utm_content=PuneFacebook&utm_campaign=Corporate> [image:
>> Twitter]
>> <http://www.pubmatic.com/social/twitter.php?utm_source=Footer&utm_medium=Emai
>> l&utm_content=PuneTwitter&utm_campaign=Corporate>
>> <http://www.pubmatic.com/social/twitter.php?utm_source=Footer&utm_medium=Emai
>> l&utm_content=PuneTwitter&utm_campaign=Corporate> [image:
>> LinkedIn]
>> <http://www.pubmatic.com/social/linkedin.php?utm_source=Footer&utm_medium=Ema
>> il&utm_content=PuneLinkedIn&utm_campaign=Corporate>
>> <http://www.pubmatic.com/social/linkedin.php?utm_source=Footer&utm_medium=Ema
>> il&utm_content=PuneLinkedIn&utm_campaign=Corporate>
>>
>> Find out more about our services: PubConnect
>> <http://www.pubmatic.com/pubconnect.php?utm_source=Footer&utm_medium=Email&ut
>> m_content=PunePubConnect&utm_campaign=Corporate>
>> <http://www.pubmatic.com/pubconnect.php?utm_source=Footer&utm_medium=Email&ut
>> m_content=PunePubConnect&utm_campaign=Corporate>
>> | PubDirect
>> <http://www.pubmatic.com/pubdirect.php?utm_source=Footer&utm_medium=Email&utm
>> _content=PunePubDirect&utm_campaign=Corporate>
>> <http://www.pubmatic.com/pubdirect.php?utm_source=Footer&utm_medium=Email&utm
>> _content=PunePubDirect&utm_campaign=Corporate>
>> | Solutions for Media Buyers
>> <http://www.pubmatic.com/media-buyers-overview.php?utm_source=Footer&utm_medi
>> um=Email&utm_content=PuneMediaBuyers&utm_campaign=Corporate>
>> <http://www.pubmatic.com/media-buyers-overview.php?utm_source=Footer&utm_medi
>> um=Email&utm_content=PuneMediaBuyers&utm_campaign=Corporate>
>> What's New: PubMatic Launches Mobile SDK for app developers.
>> <http://www.pubmatic.com/press/2013/PubMatic-Launches-Self-Service-Open-SDK.p
>> hp?utm_source=Footer&utm_medium=Email&utm_content=PuneSDKLink&utm_campaign=Co
>> rporate>
>> <http://www.pubmatic.com/press/2013/PubMatic-Launches-Self-Service-Open-SDK.p
>> hp?utm_source=Footer&utm_medium=Email&utm_content=PuneSDKLink&utm_campaign=Co
>> rporate>
>> Latest Research: Real-Time Bidding in the United States and Worldwide,
>> 2010-2017 (Q4 2013)
>> <http://www.pubmatic.com/reports-and-whitepapers.php?utm_source=Footer&utm_me
>> dium=Email&utm_content=PuneReportsLink&utm_campaign=Corporate>
>> <http://www.pubmatic.com/reports-and-whitepapers.php?utm_source=Footer&utm_me
>> dium=Email&utm_content=PuneReportsLink&utm_campaign=Corporate>
>> Recent Videos: Ad Revenue 6
>> <http://www.pubmatic.com/ar6_videos.php?utm_source=Footer&utm_medium=Email&ut
>> m_content=NYCAR6Link&utm_campaign=Corporate>
>> <http://www.pubmatic.com/ar6_videos.php?utm_source=Footer&utm_medium=Email&ut
>> m_content=NYCAR6Link&utm_campaign=Corporate>
>> | Ad Revenue Europe
>> <http://www.pubmatic.com/arE_videos.php?utm_source=Footer&utm_medium=Email&ut
>> m_content=NYCAR6Link&utm_campaign=Corporate>
>> <http://www.pubmatic.com/arE_videos.php?utm_source=Footer&utm_medium=Email&ut
>> m_content=NYCAR6Link&utm_campaign=Corporate>
>>
>>
>>
>> If the table already exists, then CREATE TABLE IF NOT EXISTS is a noop. Drop
>> the table first and then create initially with the COMPRESSION='SNAPPY'
>> property.
>>
>>
>>
>> Thanks,
>>
>> James
>>
>>
>>
>>
>> On Wed, Jul 2, 2014 at 11:29 AM, puneet <pu...@pubmatic.com> wrote:
>>
>>>
>>> Hi Team,
>>>
>>> I need snappy compression to be used for the Hbase table but it is not
>>> creating the table as SNAPPY compressed when I look through the DDL's in the
>>> Hbase shell.
>>>
>>> Also while loading the data the size of the data remains the same for
>>> SNAPPY or NONE.
>>>
>>> Please help if I am missing anything.
>>>
>>> Details Below:-
>>>
>>> CREATE TABLE IF NOT EXISTS adv_cat (
>>> cat varchar ,
>>> domain VARCHAR
>>> CONSTRAINT PK PRIMARY KEY (cat,domain)
>>> );
>>>
>>> DDL:- describe 'ADV_CAT_GZ'
>>> DESCRIPTION ENABLED
>>> 'ADV_CAT_GZ', {METHOD => 'table_att', coprocessor$1 =>
>>> '|org.apache.phoenix.coprocessor.ScanRegionObser true
>>> ver|1|', coprocessor$2 =>
>>> '|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
>>> coproc
>>> essor$3 =>
>>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1|',
>>> coprocessor$4 => '|org.
>>> apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|', coprocessor$5
>>> => '|org.apache.phoenix.hbase.i
>>>
>>> ndex.Indexer|1073741823|index.builder=org.apache.phoenix.index.PhoenixIndexB
>>> uilder,org.apache.hadoop.hb
>>> ase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec'}, {NAME
>>> => '0', DATA_BLOCK_ENCODING =
>>>> > 'FAST_DIFF', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0', VERSIONS
>>>> => '3', COMPRESSION => 'NONE',
>>> MIN_VERSIONS => '0', TTL => '2147483647 <tel:2147483647> ',
>>> KEEP_DELETED_CELLS => 'true', BLOCKSIZE => '65536', IN_MEMOR
>>> Y => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}
>>>
>>>
>>> CREATE TABLE IF NOT EXISTS adv_cat_snappy (
>>> cat varchar ,
>>> domain VARCHAR
>>> CONSTRAINT PK PRIMARY KEY (cat,domain)
>>> ) COMPRESSION='SNAPPY';
>>>
>>> describe 'ADV_CAT_SNAPPY'
>>> DESCRIPTION ENABLED
>>> 'ADV_CAT_SNAPPY', {METHOD => 'table_att', coprocessor$1 =>
>>> '|org.apache.phoenix.coprocessor.ScanRegionO true
>>> bserver|1|', coprocessor$2 =>
>>> '|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', co
>>> processor$3 =>
>>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1|',
>>> coprocessor$4 => '|
>>> org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>>> coprocessor$5 => '|org.apache.phoenix.hba
>>>
>>> se.index.Indexer|1073741823|index.builder=org.apache.phoenix.index.PhoenixIn
>>> dexBuilder,org.apache.hadoo
>>> p.hbase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec'},
>>> {NAME => '0', DATA_BLOCK_ENCODI
>>> NG => 'FAST_DIFF', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0',
>>> VERSIONS => '3', COMPRESSION => 'NO
>>> NE', MIN_VERSIONS => '0', TTL => '2147483647 <tel:2147483647> ',
>>> KEEP_DELETED_CELLS => 'true', BLOCKSIZE => '65536', IN_M
>>> EMORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}
>>> 1 row(s) in 0.0660 seconds
>>>
>>>
>>> --
>>>
>>>
>>>
>>>
>>> Puneet Ojha | Tech Lead: Data Analytics
>>> 6th Floor, Amar Paradigm | Baner Road, Pune 411045
>>> o: +91-20-67285700 <tel:%2B91-20-67285700> | m: +91-8605359898
>>> <tel:%2B91-8605359898> | Skype: puneetkr.ojha
>>>
>>>
>>>
>>> <http://www.pubmatic.com/index-eng.php?utm_source=Footer&utm_medium=Email&ut
>>> m_content=PuneLogo&utm_campaign=Corporate>
>>> Every ad. Every sales channel. Every screen. One platform
>>>
>>>
>>> On the Web:
>>> <http://www.pubmatic.com/social/facebook.php?utm_source=Footer&utm_medium=Em
>>> ail&utm_content=PuneFacebook&utm_campaign=Corporate>
>>> <http://www.pubmatic.com/social/twitter.php?utm_source=Footer&utm_medium=Ema
>>> il&utm_content=PuneTwitter&utm_campaign=Corporate>
>>> <http://www.pubmatic.com/social/linkedin.php?utm_source=Footer&utm_medium=Em
>>> ail&utm_content=PuneLinkedIn&utm_campaign=Corporate>
>>>
>>>
>>>
>>>
>>>
>>> Find out more about our services: PubConnect
>>> <http://www.pubmatic.com/pubconnect.php?utm_source=Footer&utm_medium=Email&u
>>> tm_content=PunePubConnect&utm_campaign=Corporate> | PubDirect
>>> <http://www.pubmatic.com/pubdirect.php?utm_source=Footer&utm_medium=Email&ut
>>> m_content=PunePubDirect&utm_campaign=Corporate> | Solutions for Media
>>> Buyers
>>> <http://www.pubmatic.com/media-buyers-overview.php?utm_source=Footer&utm_med
>>> ium=Email&utm_content=PuneMediaBuyers&utm_campaign=Corporate>
>>> What's New: PubMatic Launches Mobile SDK for app developers.
>>> <http://www.pubmatic.com/press/2013/PubMatic-Launches-Self-Service-Open-SDK.
>>> php?utm_source=Footer&utm_medium=Email&utm_content=PuneSDKLink&utm_campaign=
>>> Corporate>
>>> Latest Research: Real-Time Bidding in the United States and Worldwide,
>>> 2010-2017 (Q4 2013)
>>> <http://www.pubmatic.com/reports-and-whitepapers.php?utm_source=Footer&utm_m
>>> edium=Email&utm_content=PuneReportsLink&utm_campaign=Corporate>
>>> Recent Videos: Ad Revenue 6
>>> <http://www.pubmatic.com/ar6_videos.php?utm_source=Footer&utm_medium=Email&u
>>> tm_content=NYCAR6Link&utm_campaign=Corporate> | Ad Revenue Europe
>>> <http://www.pubmatic.com/arE_videos.php?utm_source=Footer&utm_medium=Email&u
>>> tm_content=NYCAR6Link&utm_campaign=Corporate>
>>>
>>>
>>>
>>
>>
>>
>>
>>
>
--
Puneet Ojha | Tech Lead: Data Analytics
6th Floor, Amar Paradigm | Baner Road, Pune 411045
o: +91-20-67285700 | m: +91-8605359898 | Skype: puneetkr.ojha
<http://www.pubmatic.com/index-eng.php?utm_source=Footer&utm_medium=Email&u
tm_content=PuneLogo&utm_campaign=Corporate>
Every ad. Every sales channel. Every screen. One platform
On the Web:
<http://www.pubmatic.com/social/facebook.php?utm_source=Footer&utm_medium=Em
ail&utm_content=PuneFacebook&utm_campaign=Corporate>
<http://www.pubmatic.com/social/twitter.php?utm_source=Footer&utm_medium=Ema
il&utm_content=PuneTwitter&utm_campaign=Corporate>
<http://www.pubmatic.com/social/linkedin.php?utm_source=Footer&utm_medium=Em
ail&utm_content=PuneLinkedIn&utm_campaign=Corporate>
Find out more about our services: PubConnect
<http://www.pubmatic.com/pubconnect.php?utm_source=Footer&utm_medium=Email&u
tm_content=PunePubConnect&utm_campaign=Corporate> | PubDirect
<http://www.pubmatic.com/pubdirect.php?utm_source=Footer&utm_medium=Email&ut
m_content=PunePubDirect&utm_campaign=Corporate> | Solutions for Media
Buyers
<http://www.pubmatic.com/media-buyers-overview.php?utm_source=Footer&utm_med
ium=Email&utm_content=PuneMediaBuyers&utm_campaign=Corporate>
What's New: PubMatic Launches Mobile SDK for app developers.
<http://www.pubmatic.com/press/2013/PubMatic-Launches-Self-Service-Open-SDK.
php?utm_source=Footer&utm_medium=Email&utm_content=PuneSDKLink&utm_campaign=
Corporate>
Latest Research: Real-Time Bidding in the United States and Worldwide,
2010-2017 (Q4 2013)
<http://www.pubmatic.com/reports-and-whitepapers.php?utm_source=Footer&utm_m
edium=Email&utm_content=PuneReportsLink&utm_campaign=Corporate>
Recent Videos: Ad Revenue 6
<http://www.pubmatic.com/ar6_videos.php?utm_source=Footer&utm_medium=Email&u
tm_content=NYCAR6Link&utm_campaign=Corporate> | Ad Revenue Europe
<http://www.pubmatic.com/arE_videos.php?utm_source=Footer&utm_medium=Email&u
tm_content=NYCAR6Link&utm_campaign=Corporate>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: How to Use Compression in Phoenix 4.0.0
Posted by puneet <pu...@pubmatic.com>.
Hi James,
Thanks for you quick reply.
------------------------------------
I had created the table with COMPRESSION='SNAPPY' property but when I
describe it through hbase shell then it shows COMPRESION='NONE'
I did a workaround and performed alter from Hbase shell as below :-
disable 'ADV_CAT_SNAPPY'
0 row(s) in 2.0320 seconds
hbase(main):027:0> alter 'ADV_CAT_SNAPPY',{NAME=>'0',COMPRESSION=>'SNAPPY'}
Updating all regions with the new schema...
1/1 regions updated.
Done.
0 row(s) in 1.1440 seconds
hbase(main):028:0> enable 'ADV_CAT_SNAPPY'
0 row(s) in 2.0970 seconds
-----------------------------
After the above steps the compression is modifed to SNAPPY.
But the compression seems to be happening only when I am Loading via
PSQL utility.
When I run it from Loading via MaprReduce , then it is not complressing
the data. I have Hbase env file modifed to use Snappy lib in Hadoop
Need steps for enabling compression via mapreduce.
On Wednesday 02 July 2014 03:42 PM, James Taylor wrote:
> If the table already exists, then CREATE TABLE IF NOT EXISTS is a noop.
> Drop the table first and then create initially with the COMPRESSION='SNAPPY'
> property.
>
> Thanks,
> James
>
>
> On Wed, Jul 2, 2014 at 11:29 AM, puneet <pu...@pubmatic.com> wrote:
>
>> Hi Team,
>>
>> I need snappy compression to be used for the Hbase table but it is not
>> creating the table as SNAPPY compressed when I look through the DDL's in
>> the Hbase shell.
>>
>> Also while loading the data the size of the data remains the same for
>> SNAPPY or NONE.
>>
>> Please help if I am missing anything.
>>
>> Details Below:-
>>
>> CREATE TABLE IF NOT EXISTS adv_cat (
>> cat varchar ,
>> domain VARCHAR
>> CONSTRAINT PK PRIMARY KEY (cat,domain)
>> );
>>
>> DDL:- describe 'ADV_CAT_GZ'
>> DESCRIPTION ENABLED
>> 'ADV_CAT_GZ', {METHOD => 'table_att', coprocessor$1 =>
>> '|org.apache.phoenix.coprocessor.ScanRegionObser true
>> ver|1|', coprocessor$2 =>
>> '|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
>> coproc
>> essor$3 =>
>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1|',
>> coprocessor$4 => '|org.
>> apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|', coprocessor$5
>> => '|org.apache.phoenix.hbase.i
>> ndex.Indexer|1073741823|index.builder=org.apache.phoenix.index.PhoenixIndexBuilder,org.apache.hadoop.hb
>>
>> ase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec'}, {NAME
>> => '0', DATA_BLOCK_ENCODING =
>> > 'FAST_DIFF', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0', VERSIONS
>> => '3', COMPRESSION => 'NONE',
>> MIN_VERSIONS => '0', TTL => '2147483647', KEEP_DELETED_CELLS => 'true',
>> BLOCKSIZE => '65536', IN_MEMOR
>> Y => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}
>>
>>
>> CREATE TABLE IF NOT EXISTS adv_cat_snappy (
>> cat varchar ,
>> domain VARCHAR
>> CONSTRAINT PK PRIMARY KEY (cat,domain)
>> ) COMPRESSION='SNAPPY';
>>
>> describe 'ADV_CAT_SNAPPY'
>> DESCRIPTION ENABLED
>> 'ADV_CAT_SNAPPY', {METHOD => 'table_att', coprocessor$1 =>
>> '|org.apache.phoenix.coprocessor.ScanRegionO true
>> bserver|1|', coprocessor$2 =>
>> '|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', co
>> processor$3 =>
>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1|',
>> coprocessor$4 => '|
>> org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>> coprocessor$5 => '|org.apache.phoenix.hba
>> se.index.Indexer|1073741823|index.builder=org.apache.phoenix.index.PhoenixIndexBuilder,org.apache.hadoo
>>
>> p.hbase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec'},
>> {NAME => '0', DATA_BLOCK_ENCODI
>> NG => 'FAST_DIFF', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0',
>> VERSIONS => '3', COMPRESSION => 'NO
>> NE', MIN_VERSIONS => '0', TTL => '2147483647', KEEP_DELETED_CELLS =>
>> 'true', BLOCKSIZE => '65536', IN_M
>> EMORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}
>> 1 row(s) in 0.0660 seconds
>>
>> --
>> ------------------------------
>>
>> *Puneet Ojha* | Tech Lead: Data Analytics
>> 6th Floor, Amar Paradigm | Baner Road, Pune 411045
>> o: +91-20-67285700 | m: +91-8605359898 | Skype: puneetkr.ojha
>>
>> [image: PubMatic]
>> <http://www.pubmatic.com/index-eng.php?utm_source=Footer&utm_medium=Email&utm_content=PuneLogo&utm_campaign=Corporate>
>> Every ad. Every sales channel. Every screen. *One platform*
>>
>> On the Web: [image: Facebook]
>> <http://www.pubmatic.com/social/facebook.php?utm_source=Footer&utm_medium=Email&utm_content=PuneFacebook&utm_campaign=Corporate> [image:
>> Twitter]
>> <http://www.pubmatic.com/social/twitter.php?utm_source=Footer&utm_medium=Email&utm_content=PuneTwitter&utm_campaign=Corporate> [image:
>> LinkedIn]
>> <http://www.pubmatic.com/social/linkedin.php?utm_source=Footer&utm_medium=Email&utm_content=PuneLinkedIn&utm_campaign=Corporate>
>>
>> Find out more about our services: PubConnect
>> <http://www.pubmatic.com/pubconnect.php?utm_source=Footer&utm_medium=Email&utm_content=PunePubConnect&utm_campaign=Corporate>
>> | PubDirect
>> <http://www.pubmatic.com/pubdirect.php?utm_source=Footer&utm_medium=Email&utm_content=PunePubDirect&utm_campaign=Corporate>
>> | Solutions for Media Buyers
>> <http://www.pubmatic.com/media-buyers-overview.php?utm_source=Footer&utm_medium=Email&utm_content=PuneMediaBuyers&utm_campaign=Corporate>
>> What's New: PubMatic Launches Mobile SDK for app developers.
>> <http://www.pubmatic.com/press/2013/PubMatic-Launches-Self-Service-Open-SDK.php?utm_source=Footer&utm_medium=Email&utm_content=PuneSDKLink&utm_campaign=Corporate>
>> Latest Research: Real-Time Bidding in the United States and Worldwide,
>> 2010-2017 (Q4 2013)
>> <http://www.pubmatic.com/reports-and-whitepapers.php?utm_source=Footer&utm_medium=Email&utm_content=PuneReportsLink&utm_campaign=Corporate>
>> Recent Videos: Ad Revenue 6
>> <http://www.pubmatic.com/ar6_videos.php?utm_source=Footer&utm_medium=Email&utm_content=NYCAR6Link&utm_campaign=Corporate>
>> | Ad Revenue Europe
>> <http://www.pubmatic.com/arE_videos.php?utm_source=Footer&utm_medium=Email&utm_content=NYCAR6Link&utm_campaign=Corporate>
>>
>>
>>
>> If the table already exists, then CREATE TABLE IF NOT EXISTS is a
>> noop. Drop the table first and then create initially with the
>> COMPRESSION='SNAPPY' property.
>>
>> Thanks,
>> James
>>
>>
>> On Wed, Jul 2, 2014 at 11:29 AM, puneet <puneet.kumar@pubmatic.com
>> <ma...@pubmatic.com>> wrote:
>>
>> Hi Team,
>>
>> I need snappy compression to be used for the Hbase table but it
>> is not creating the table as SNAPPY compressed when I look
>> through the DDL's in the Hbase shell.
>>
>> Also while loading the data the size of the data remains the same
>> for SNAPPY or NONE.
>>
>> Please help if I am missing anything.
>>
>> Details Below:-
>>
>> CREATE TABLE IF NOT EXISTS adv_cat (
>> cat varchar ,
>> domain VARCHAR
>> CONSTRAINT PK PRIMARY KEY (cat,domain)
>> );
>>
>> DDL:- describe 'ADV_CAT_GZ'
>> DESCRIPTION ENABLED
>> 'ADV_CAT_GZ', {METHOD => 'table_att', coprocessor$1 =>
>> '|org.apache.phoenix.coprocessor.ScanRegionObser true
>> ver|1|', coprocessor$2 =>
>> '|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
>> coproc
>> essor$3 =>
>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1|',
>> coprocessor$4 => '|org.
>> apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>> coprocessor$5 => '|org.apache.phoenix.hbase.i
>> ndex.Indexer|1073741823|index.builder=org.apache.phoenix.index.PhoenixIndexBuilder,org.apache.hadoop.hb
>>
>> ase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec'},
>> {NAME => '0', DATA_BLOCK_ENCODING =
>> > 'FAST_DIFF', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0',
>> VERSIONS => '3', COMPRESSION => 'NONE',
>> MIN_VERSIONS => '0', TTL => '2147483647 <tel:2147483647>',
>> KEEP_DELETED_CELLS => 'true', BLOCKSIZE => '65536', IN_MEMOR
>> Y => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}
>>
>>
>> CREATE TABLE IF NOT EXISTS adv_cat_snappy (
>> cat varchar ,
>> domain VARCHAR
>> CONSTRAINT PK PRIMARY KEY (cat,domain)
>> ) COMPRESSION='SNAPPY';
>>
>> describe 'ADV_CAT_SNAPPY'
>> DESCRIPTION ENABLED
>> 'ADV_CAT_SNAPPY', {METHOD => 'table_att', coprocessor$1 =>
>> '|org.apache.phoenix.coprocessor.ScanRegionO true
>> bserver|1|', coprocessor$2 =>
>> '|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
>> co
>> processor$3 =>
>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1|',
>> coprocessor$4 => '|
>> org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>> coprocessor$5 => '|org.apache.phoenix.hba
>> se.index.Indexer|1073741823|index.builder=org.apache.phoenix.index.PhoenixIndexBuilder,org.apache.hadoo
>>
>> p.hbase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec'},
>> {NAME => '0', DATA_BLOCK_ENCODI
>> NG => 'FAST_DIFF', BLOOMFILTER => 'NONE', REPLICATION_SCOPE =>
>> '0', VERSIONS => '3', COMPRESSION => 'NO
>> NE', MIN_VERSIONS => '0', TTL => '2147483647 <tel:2147483647>',
>> KEEP_DELETED_CELLS => 'true', BLOCKSIZE => '65536', IN_M
>> EMORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}
>> 1 row(s) in 0.0660 seconds
>>
>> --
>> ------------------------------------------------------------------------
>>
>> *Puneet Ojha* | Tech Lead: Data Analytics
>> 6th Floor, Amar Paradigm | Baner Road, Pune 411045
>> o: +91-20-67285700 <tel:%2B91-20-67285700> | m: +91-8605359898
>> <tel:%2B91-8605359898> | Skype: puneetkr.ojha
>>
>> PubMatic
>> <http://www.pubmatic.com/index-eng.php?utm_source=Footer&utm_medium=Email&utm_content=PuneLogo&utm_campaign=Corporate>
>> Every ad. Every sales channel. Every screen. *One platform*
>>
>> On the Web: Facebook
>> <http://www.pubmatic.com/social/facebook.php?utm_source=Footer&utm_medium=Email&utm_content=PuneFacebook&utm_campaign=Corporate>
>> Twitter
>> <http://www.pubmatic.com/social/twitter.php?utm_source=Footer&utm_medium=Email&utm_content=PuneTwitter&utm_campaign=Corporate>
>> LinkedIn
>> <http://www.pubmatic.com/social/linkedin.php?utm_source=Footer&utm_medium=Email&utm_content=PuneLinkedIn&utm_campaign=Corporate>
>>
>> Find out more about our services: PubConnect
>> <http://www.pubmatic.com/pubconnect.php?utm_source=Footer&utm_medium=Email&utm_content=PunePubConnect&utm_campaign=Corporate>
>> | PubDirect
>> <http://www.pubmatic.com/pubdirect.php?utm_source=Footer&utm_medium=Email&utm_content=PunePubDirect&utm_campaign=Corporate>
>> | Solutions for Media Buyers
>> <http://www.pubmatic.com/media-buyers-overview.php?utm_source=Footer&utm_medium=Email&utm_content=PuneMediaBuyers&utm_campaign=Corporate>
>> What's New: PubMatic Launches Mobile SDK for app developers.
>> <http://www.pubmatic.com/press/2013/PubMatic-Launches-Self-Service-Open-SDK.php?utm_source=Footer&utm_medium=Email&utm_content=PuneSDKLink&utm_campaign=Corporate>
>> Latest Research: Real-Time Bidding in the United States and
>> Worldwide, 2010-2017 (Q4 2013)
>> <http://www.pubmatic.com/reports-and-whitepapers.php?utm_source=Footer&utm_medium=Email&utm_content=PuneReportsLink&utm_campaign=Corporate>
>> Recent Videos: Ad Revenue 6
>> <http://www.pubmatic.com/ar6_videos.php?utm_source=Footer&utm_medium=Email&utm_content=NYCAR6Link&utm_campaign=Corporate>
>> | Ad Revenue Europe
>> <http://www.pubmatic.com/arE_videos.php?utm_source=Footer&utm_medium=Email&utm_content=NYCAR6Link&utm_campaign=Corporate>
>>
>>
--
------------------------------------------------------------------------
*Puneet Ojha* | Tech Lead: Data Analytics
6th Floor, Amar Paradigm | Baner Road, Pune 411045
o: +91-20-67285700 | m: +91-8605359898 | Skype: puneetkr.ojha
PubMatic
<http://www.pubmatic.com/index-eng.php?utm_source=Footer&utm_medium=Email&utm_content=PuneLogo&utm_campaign=Corporate>
Every ad. Every sales channel. Every screen. *One platform*
On the Web: Facebook
<http://www.pubmatic.com/social/facebook.php?utm_source=Footer&utm_medium=Email&utm_content=PuneFacebook&utm_campaign=Corporate>
Twitter
<http://www.pubmatic.com/social/twitter.php?utm_source=Footer&utm_medium=Email&utm_content=PuneTwitter&utm_campaign=Corporate>
LinkedIn
<http://www.pubmatic.com/social/linkedin.php?utm_source=Footer&utm_medium=Email&utm_content=PuneLinkedIn&utm_campaign=Corporate>
Find out more about our services: PubConnect
<http://www.pubmatic.com/pubconnect.php?utm_source=Footer&utm_medium=Email&utm_content=PunePubConnect&utm_campaign=Corporate>
| PubDirect
<http://www.pubmatic.com/pubdirect.php?utm_source=Footer&utm_medium=Email&utm_content=PunePubDirect&utm_campaign=Corporate>
| Solutions for Media Buyers
<http://www.pubmatic.com/media-buyers-overview.php?utm_source=Footer&utm_medium=Email&utm_content=PuneMediaBuyers&utm_campaign=Corporate>
What's New: PubMatic Launches Mobile SDK for app developers.
<http://www.pubmatic.com/press/2013/PubMatic-Launches-Self-Service-Open-SDK.php?utm_source=Footer&utm_medium=Email&utm_content=PuneSDKLink&utm_campaign=Corporate>
Latest Research: Real-Time Bidding in the United States and Worldwide,
2010-2017 (Q4 2013)
<http://www.pubmatic.com/reports-and-whitepapers.php?utm_source=Footer&utm_medium=Email&utm_content=PuneReportsLink&utm_campaign=Corporate>
Recent Videos: Ad Revenue 6
<http://www.pubmatic.com/ar6_videos.php?utm_source=Footer&utm_medium=Email&utm_content=NYCAR6Link&utm_campaign=Corporate>
| Ad Revenue Europe
<http://www.pubmatic.com/arE_videos.php?utm_source=Footer&utm_medium=Email&utm_content=NYCAR6Link&utm_campaign=Corporate>
Re: How to Use Compression in Phoenix 4.0.0
Posted by James Taylor <ja...@apache.org>.
If the table already exists, then CREATE TABLE IF NOT EXISTS is a noop.
Drop the table first and then create initially with the COMPRESSION='SNAPPY'
property.
Thanks,
James
On Wed, Jul 2, 2014 at 11:29 AM, puneet <pu...@pubmatic.com> wrote:
> Hi Team,
>
> I need snappy compression to be used for the Hbase table but it is not
> creating the table as SNAPPY compressed when I look through the DDL's in
> the Hbase shell.
>
> Also while loading the data the size of the data remains the same for
> SNAPPY or NONE.
>
> Please help if I am missing anything.
>
> Details Below:-
>
> CREATE TABLE IF NOT EXISTS adv_cat (
> cat varchar ,
> domain VARCHAR
> CONSTRAINT PK PRIMARY KEY (cat,domain)
> );
>
> DDL:- describe 'ADV_CAT_GZ'
> DESCRIPTION ENABLED
> 'ADV_CAT_GZ', {METHOD => 'table_att', coprocessor$1 =>
> '|org.apache.phoenix.coprocessor.ScanRegionObser true
> ver|1|', coprocessor$2 =>
> '|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
> coproc
> essor$3 =>
> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1|',
> coprocessor$4 => '|org.
> apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|', coprocessor$5
> => '|org.apache.phoenix.hbase.i
> ndex.Indexer|1073741823|index.builder=org.apache.phoenix.index.PhoenixIndexBuilder,org.apache.hadoop.hb
>
> ase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec'}, {NAME
> => '0', DATA_BLOCK_ENCODING =
> > 'FAST_DIFF', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0', VERSIONS
> => '3', COMPRESSION => 'NONE',
> MIN_VERSIONS => '0', TTL => '2147483647', KEEP_DELETED_CELLS => 'true',
> BLOCKSIZE => '65536', IN_MEMOR
> Y => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}
>
>
> CREATE TABLE IF NOT EXISTS adv_cat_snappy (
> cat varchar ,
> domain VARCHAR
> CONSTRAINT PK PRIMARY KEY (cat,domain)
> ) COMPRESSION='SNAPPY';
>
> describe 'ADV_CAT_SNAPPY'
> DESCRIPTION ENABLED
> 'ADV_CAT_SNAPPY', {METHOD => 'table_att', coprocessor$1 =>
> '|org.apache.phoenix.coprocessor.ScanRegionO true
> bserver|1|', coprocessor$2 =>
> '|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', co
> processor$3 =>
> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1|',
> coprocessor$4 => '|
> org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
> coprocessor$5 => '|org.apache.phoenix.hba
> se.index.Indexer|1073741823|index.builder=org.apache.phoenix.index.PhoenixIndexBuilder,org.apache.hadoo
>
> p.hbase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec'},
> {NAME => '0', DATA_BLOCK_ENCODI
> NG => 'FAST_DIFF', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0',
> VERSIONS => '3', COMPRESSION => 'NO
> NE', MIN_VERSIONS => '0', TTL => '2147483647', KEEP_DELETED_CELLS =>
> 'true', BLOCKSIZE => '65536', IN_M
> EMORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}
> 1 row(s) in 0.0660 seconds
>
> --
> ------------------------------
>
> *Puneet Ojha* | Tech Lead: Data Analytics
> 6th Floor, Amar Paradigm | Baner Road, Pune 411045
> o: +91-20-67285700 | m: +91-8605359898 | Skype: puneetkr.ojha
>
> [image: PubMatic]
> <http://www.pubmatic.com/index-eng.php?utm_source=Footer&utm_medium=Email&utm_content=PuneLogo&utm_campaign=Corporate>
> Every ad. Every sales channel. Every screen. *One platform*
>
> On the Web: [image: Facebook]
> <http://www.pubmatic.com/social/facebook.php?utm_source=Footer&utm_medium=Email&utm_content=PuneFacebook&utm_campaign=Corporate> [image:
> Twitter]
> <http://www.pubmatic.com/social/twitter.php?utm_source=Footer&utm_medium=Email&utm_content=PuneTwitter&utm_campaign=Corporate> [image:
> LinkedIn]
> <http://www.pubmatic.com/social/linkedin.php?utm_source=Footer&utm_medium=Email&utm_content=PuneLinkedIn&utm_campaign=Corporate>
>
> Find out more about our services: PubConnect
> <http://www.pubmatic.com/pubconnect.php?utm_source=Footer&utm_medium=Email&utm_content=PunePubConnect&utm_campaign=Corporate>
> | PubDirect
> <http://www.pubmatic.com/pubdirect.php?utm_source=Footer&utm_medium=Email&utm_content=PunePubDirect&utm_campaign=Corporate>
> | Solutions for Media Buyers
> <http://www.pubmatic.com/media-buyers-overview.php?utm_source=Footer&utm_medium=Email&utm_content=PuneMediaBuyers&utm_campaign=Corporate>
> What's New: PubMatic Launches Mobile SDK for app developers.
> <http://www.pubmatic.com/press/2013/PubMatic-Launches-Self-Service-Open-SDK.php?utm_source=Footer&utm_medium=Email&utm_content=PuneSDKLink&utm_campaign=Corporate>
> Latest Research: Real-Time Bidding in the United States and Worldwide,
> 2010-2017 (Q4 2013)
> <http://www.pubmatic.com/reports-and-whitepapers.php?utm_source=Footer&utm_medium=Email&utm_content=PuneReportsLink&utm_campaign=Corporate>
> Recent Videos: Ad Revenue 6
> <http://www.pubmatic.com/ar6_videos.php?utm_source=Footer&utm_medium=Email&utm_content=NYCAR6Link&utm_campaign=Corporate>
> | Ad Revenue Europe
> <http://www.pubmatic.com/arE_videos.php?utm_source=Footer&utm_medium=Email&utm_content=NYCAR6Link&utm_campaign=Corporate>
>