You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Ashish Nigam <as...@gmail.com> on 2013/04/03 20:34:14 UTC
HBase Column Family TTL and cell deletions
Hi,
I have a table "service_counters" that has service id as row id.
There is a column family "stats-hourly" that stores counters related to
that service per hour.
I have set TTL to be 7 days for the column family.
{NAME => 'service_counters', FAMILIES => [{NAME => 'stats-hourly',
COMPRESSION => 'LZO', TTL => '604800'}]}
Now the problem is that I still see columns that are more than 7 days old.
I have also run major compactions but I still see old rows.
Any pointers as to why these columns are not being deleted?
Thanks
Ashish
Re: HBase Column Family TTL and cell deletions
Posted by Ted Yu <yu...@gmail.com>.
I used http://www.ruddwire.com/handy-code/date-to-millisecond-calculators/and
got same result.
I think the initial confusion was due to TTL being in seconds and timestamp
being in milliseconds.
On Wed, Apr 3, 2013 at 8:07 PM, lars hofhansl <la...@apache.org> wrote:
> 1364928886015 is Tue Apr 02 11:54:46 PDT 2013, no?
> So it's less than a day old.
>
> (Sorry if I am missing something)
>
>
> -- Lars
>
>
>
> ________________________________
> From: Ashish Nigam <as...@gmail.com>
> To: user@hbase.apache.org; dev@hbase.apache.org
> Sent: Wednesday, April 3, 2013 3:19 PM
> Subject: Re: HBase Column Family TTL and cell deletions
>
> Any thoughts?
>
> Here's one cell that should have been deleted ( I am putting HBase get
> repines in shell)
>
> hbase(main):001:0> get 'service_counters','3141',{COLUMN =>
> 'stats-hourly:2013032014-C-U-4092276'}
> 13/04/03 21:57:17 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> COLUMN CELL
>
>
>
>
> stats-hourly:2013032014-C-U-4092276
> timestamp=1364928886015, value=\x00\x00\x00\x00\x00\x00\x00\x06
>
>
>
> 1 row(s) in 0.1040 seconds
>
>
> On Wed, Apr 3, 2013 at 12:03 PM, Ashish Nigam <ashnigamtech@gmail.com
> >wrote:
>
> > minVersion is set to 0
> > Here's detailed info on this CF -
> >
> > {NAME => 'stats-hourly', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER =>
> > 'NONE', REPLICATION_SCOPE => '0', VERSIONS => '3', COMPRESSION => 'LZO',
> > MIN_VERSIONS => '0', TTL => '604800', KEEP_DELETED_CELLS => 'false',
> > BLOCKSIZE => '65536', IN_MEMORY => 'false', ENCODE_ON_DISK => 'true',
> > BLOCKCACHE => 'true'}
> >
> > HBase version is 0.94.2-cdh4.2.0,
> >
> >
> >
> > On Wed, Apr 3, 2013 at 11:48 AM, Ted Yu <yu...@gmail.com> wrote:
> >
> >> Is it possible that minVersions was set for your column family ?
> >>
> >> What version of HBase are you using ?
> >>
> >> Thanks
> >>
> >>
> >> On Wed, Apr 3, 2013 at 11:34 AM, Ashish Nigam <ashnigamtech@gmail.com
> >> >wrote:
> >>
> >> > Hi,
> >> > I have a table "service_counters" that has service id as row id.
> >> > There is a column family "stats-hourly" that stores counters related
> to
> >> > that service per hour.
> >> > I have set TTL to be 7 days for the column family.
> >> >
> >> > {NAME => 'service_counters', FAMILIES => [{NAME => 'stats-hourly',
> >> > COMPRESSION => 'LZO', TTL => '604800'}]}
> >> >
> >> > Now the problem is that I still see columns that are more than 7 days
> >> old.
> >> > I have also run major compactions but I still see old rows.
> >> > Any pointers as to why these columns are not being deleted?
> >> >
> >> > Thanks
> >> > Ashish
> >> >
> >>
> >
> >
>
Re: HBase Column Family TTL and cell deletions
Posted by lars hofhansl <la...@apache.org>.
1364928886015 is Tue Apr 02 11:54:46 PDT 2013, no?
So it's less than a day old.
(Sorry if I am missing something)
-- Lars
________________________________
From: Ashish Nigam <as...@gmail.com>
To: user@hbase.apache.org; dev@hbase.apache.org
Sent: Wednesday, April 3, 2013 3:19 PM
Subject: Re: HBase Column Family TTL and cell deletions
Any thoughts?
Here's one cell that should have been deleted ( I am putting HBase get
repines in shell)
hbase(main):001:0> get 'service_counters','3141',{COLUMN =>
'stats-hourly:2013032014-C-U-4092276'}
13/04/03 21:57:17 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
COLUMN CELL
stats-hourly:2013032014-C-U-4092276
timestamp=1364928886015, value=\x00\x00\x00\x00\x00\x00\x00\x06
1 row(s) in 0.1040 seconds
On Wed, Apr 3, 2013 at 12:03 PM, Ashish Nigam <as...@gmail.com>wrote:
> minVersion is set to 0
> Here's detailed info on this CF -
>
> {NAME => 'stats-hourly', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER =>
> 'NONE', REPLICATION_SCOPE => '0', VERSIONS => '3', COMPRESSION => 'LZO',
> MIN_VERSIONS => '0', TTL => '604800', KEEP_DELETED_CELLS => 'false',
> BLOCKSIZE => '65536', IN_MEMORY => 'false', ENCODE_ON_DISK => 'true',
> BLOCKCACHE => 'true'}
>
> HBase version is 0.94.2-cdh4.2.0,
>
>
>
> On Wed, Apr 3, 2013 at 11:48 AM, Ted Yu <yu...@gmail.com> wrote:
>
>> Is it possible that minVersions was set for your column family ?
>>
>> What version of HBase are you using ?
>>
>> Thanks
>>
>>
>> On Wed, Apr 3, 2013 at 11:34 AM, Ashish Nigam <ashnigamtech@gmail.com
>> >wrote:
>>
>> > Hi,
>> > I have a table "service_counters" that has service id as row id.
>> > There is a column family "stats-hourly" that stores counters related to
>> > that service per hour.
>> > I have set TTL to be 7 days for the column family.
>> >
>> > {NAME => 'service_counters', FAMILIES => [{NAME => 'stats-hourly',
>> > COMPRESSION => 'LZO', TTL => '604800'}]}
>> >
>> > Now the problem is that I still see columns that are more than 7 days
>> old.
>> > I have also run major compactions but I still see old rows.
>> > Any pointers as to why these columns are not being deleted?
>> >
>> > Thanks
>> > Ashish
>> >
>>
>
>
Re: HBase Column Family TTL and cell deletions
Posted by Ashish Nigam <as...@gmail.com>.
My bad. False alarm. It is working fine and TTL is in seconds.
I was able to reduce TTL in that column family and saw number of rows go
down after major compaction.
On Wed, Apr 3, 2013 at 4:56 PM, Andrew Purtell <ap...@apache.org> wrote:
> TTL is in seconds.
>
> When we put this in I couldn't envision any valid use case for sub-second
> expiration.
>
>
> On Wed, Apr 3, 2013 at 4:22 PM, Ian Varley <iv...@salesforce.com> wrote:
>
> > I don't know if it's right (haven't checked source just now) but
> according
> > to this:
> >
> > http://hbase.apache.org/book/ttl.html
> >
> > Column family TTL is in seconds, not milliseconds. Could that be the
> > problem?
> >
> > (If not, we should fix that page in the ref guide.)
> >
> > On Apr 3, 2013, at 5:19 PM, Ashish Nigam wrote:
> >
> > Any thoughts?
> >
> > Here's one cell that should have been deleted ( I am putting HBase get
> > repines in shell)
> >
> > hbase(main):001:0> get 'service_counters','3141',{COLUMN =>
> > 'stats-hourly:2013032014-C-U-4092276'}
> > 13/04/03 21:57:17 WARN util.NativeCodeLoader: Unable to load
> native-hadoop
> > library for your platform... using builtin-java classes where applicable
> > COLUMN CELL
> >
> >
> >
> >
> > stats-hourly:2013032014-C-U-4092276
> > timestamp=1364928886015, value=\x00\x00\x00\x00\x00\x00\x00\x06
> >
> >
> >
> > 1 row(s) in 0.1040 seconds
> >
> >
> > On Wed, Apr 3, 2013 at 12:03 PM, Ashish Nigam <ashnigamtech@gmail.com
> > <ma...@gmail.com>>wrote:
> >
> > minVersion is set to 0
> > Here's detailed info on this CF -
> >
> > {NAME => 'stats-hourly', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER =>
> > 'NONE', REPLICATION_SCOPE => '0', VERSIONS => '3', COMPRESSION => 'LZO',
> > MIN_VERSIONS => '0', TTL => '604800', KEEP_DELETED_CELLS => 'false',
> > BLOCKSIZE => '65536', IN_MEMORY => 'false', ENCODE_ON_DISK => 'true',
> > BLOCKCACHE => 'true'}
> >
> > HBase version is 0.94.2-cdh4.2.0,
> >
> >
> >
> > On Wed, Apr 3, 2013 at 11:48 AM, Ted Yu <yuzhihong@gmail.com<mailto:
> > yuzhihong@gmail.com>> wrote:
> >
> > Is it possible that minVersions was set for your column family ?
> >
> > What version of HBase are you using ?
> >
> > Thanks
> >
> >
> > On Wed, Apr 3, 2013 at 11:34 AM, Ashish Nigam <ashnigamtech@gmail.com
> > <ma...@gmail.com>
> > wrote:
> >
> > Hi,
> > I have a table "service_counters" that has service id as row id.
> > There is a column family "stats-hourly" that stores counters related to
> > that service per hour.
> > I have set TTL to be 7 days for the column family.
> >
> > {NAME => 'service_counters', FAMILIES => [{NAME => 'stats-hourly',
> > COMPRESSION => 'LZO', TTL => '604800'}]}
> >
> > Now the problem is that I still see columns that are more than 7 days
> > old.
> > I have also run major compactions but I still see old rows.
> > Any pointers as to why these columns are not being deleted?
> >
> > Thanks
> > Ashish
> >
> >
> >
> >
> >
> >
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
Re: HBase Column Family TTL and cell deletions
Posted by Andrew Purtell <ap...@apache.org>.
TTL is in seconds.
When we put this in I couldn't envision any valid use case for sub-second
expiration.
On Wed, Apr 3, 2013 at 4:22 PM, Ian Varley <iv...@salesforce.com> wrote:
> I don't know if it's right (haven't checked source just now) but according
> to this:
>
> http://hbase.apache.org/book/ttl.html
>
> Column family TTL is in seconds, not milliseconds. Could that be the
> problem?
>
> (If not, we should fix that page in the ref guide.)
>
> On Apr 3, 2013, at 5:19 PM, Ashish Nigam wrote:
>
> Any thoughts?
>
> Here's one cell that should have been deleted ( I am putting HBase get
> repines in shell)
>
> hbase(main):001:0> get 'service_counters','3141',{COLUMN =>
> 'stats-hourly:2013032014-C-U-4092276'}
> 13/04/03 21:57:17 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> COLUMN CELL
>
>
>
>
> stats-hourly:2013032014-C-U-4092276
> timestamp=1364928886015, value=\x00\x00\x00\x00\x00\x00\x00\x06
>
>
>
> 1 row(s) in 0.1040 seconds
>
>
> On Wed, Apr 3, 2013 at 12:03 PM, Ashish Nigam <ashnigamtech@gmail.com
> <ma...@gmail.com>>wrote:
>
> minVersion is set to 0
> Here's detailed info on this CF -
>
> {NAME => 'stats-hourly', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER =>
> 'NONE', REPLICATION_SCOPE => '0', VERSIONS => '3', COMPRESSION => 'LZO',
> MIN_VERSIONS => '0', TTL => '604800', KEEP_DELETED_CELLS => 'false',
> BLOCKSIZE => '65536', IN_MEMORY => 'false', ENCODE_ON_DISK => 'true',
> BLOCKCACHE => 'true'}
>
> HBase version is 0.94.2-cdh4.2.0,
>
>
>
> On Wed, Apr 3, 2013 at 11:48 AM, Ted Yu <yuzhihong@gmail.com<mailto:
> yuzhihong@gmail.com>> wrote:
>
> Is it possible that minVersions was set for your column family ?
>
> What version of HBase are you using ?
>
> Thanks
>
>
> On Wed, Apr 3, 2013 at 11:34 AM, Ashish Nigam <ashnigamtech@gmail.com
> <ma...@gmail.com>
> wrote:
>
> Hi,
> I have a table "service_counters" that has service id as row id.
> There is a column family "stats-hourly" that stores counters related to
> that service per hour.
> I have set TTL to be 7 days for the column family.
>
> {NAME => 'service_counters', FAMILIES => [{NAME => 'stats-hourly',
> COMPRESSION => 'LZO', TTL => '604800'}]}
>
> Now the problem is that I still see columns that are more than 7 days
> old.
> I have also run major compactions but I still see old rows.
> Any pointers as to why these columns are not being deleted?
>
> Thanks
> Ashish
>
>
>
>
>
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: HBase Column Family TTL and cell deletions
Posted by Ian Varley <iv...@salesforce.com>.
I don't know if it's right (haven't checked source just now) but according to this:
http://hbase.apache.org/book/ttl.html
Column family TTL is in seconds, not milliseconds. Could that be the problem?
(If not, we should fix that page in the ref guide.)
On Apr 3, 2013, at 5:19 PM, Ashish Nigam wrote:
Any thoughts?
Here's one cell that should have been deleted ( I am putting HBase get
repines in shell)
hbase(main):001:0> get 'service_counters','3141',{COLUMN =>
'stats-hourly:2013032014-C-U-4092276'}
13/04/03 21:57:17 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
COLUMN CELL
stats-hourly:2013032014-C-U-4092276
timestamp=1364928886015, value=\x00\x00\x00\x00\x00\x00\x00\x06
1 row(s) in 0.1040 seconds
On Wed, Apr 3, 2013 at 12:03 PM, Ashish Nigam <as...@gmail.com>>wrote:
minVersion is set to 0
Here's detailed info on this CF -
{NAME => 'stats-hourly', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER =>
'NONE', REPLICATION_SCOPE => '0', VERSIONS => '3', COMPRESSION => 'LZO',
MIN_VERSIONS => '0', TTL => '604800', KEEP_DELETED_CELLS => 'false',
BLOCKSIZE => '65536', IN_MEMORY => 'false', ENCODE_ON_DISK => 'true',
BLOCKCACHE => 'true'}
HBase version is 0.94.2-cdh4.2.0,
On Wed, Apr 3, 2013 at 11:48 AM, Ted Yu <yu...@gmail.com>> wrote:
Is it possible that minVersions was set for your column family ?
What version of HBase are you using ?
Thanks
On Wed, Apr 3, 2013 at 11:34 AM, Ashish Nigam <as...@gmail.com>
wrote:
Hi,
I have a table "service_counters" that has service id as row id.
There is a column family "stats-hourly" that stores counters related to
that service per hour.
I have set TTL to be 7 days for the column family.
{NAME => 'service_counters', FAMILIES => [{NAME => 'stats-hourly',
COMPRESSION => 'LZO', TTL => '604800'}]}
Now the problem is that I still see columns that are more than 7 days
old.
I have also run major compactions but I still see old rows.
Any pointers as to why these columns are not being deleted?
Thanks
Ashish
Re: HBase Column Family TTL and cell deletions
Posted by lars hofhansl <la...@apache.org>.
1364928886015 is Tue Apr 02 11:54:46 PDT 2013, no?
So it's less than a day old.
(Sorry if I am missing something)
-- Lars
________________________________
From: Ashish Nigam <as...@gmail.com>
To: user@hbase.apache.org; dev@hbase.apache.org
Sent: Wednesday, April 3, 2013 3:19 PM
Subject: Re: HBase Column Family TTL and cell deletions
Any thoughts?
Here's one cell that should have been deleted ( I am putting HBase get
repines in shell)
hbase(main):001:0> get 'service_counters','3141',{COLUMN =>
'stats-hourly:2013032014-C-U-4092276'}
13/04/03 21:57:17 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
COLUMN CELL
stats-hourly:2013032014-C-U-4092276
timestamp=1364928886015, value=\x00\x00\x00\x00\x00\x00\x00\x06
1 row(s) in 0.1040 seconds
On Wed, Apr 3, 2013 at 12:03 PM, Ashish Nigam <as...@gmail.com>wrote:
> minVersion is set to 0
> Here's detailed info on this CF -
>
> {NAME => 'stats-hourly', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER =>
> 'NONE', REPLICATION_SCOPE => '0', VERSIONS => '3', COMPRESSION => 'LZO',
> MIN_VERSIONS => '0', TTL => '604800', KEEP_DELETED_CELLS => 'false',
> BLOCKSIZE => '65536', IN_MEMORY => 'false', ENCODE_ON_DISK => 'true',
> BLOCKCACHE => 'true'}
>
> HBase version is 0.94.2-cdh4.2.0,
>
>
>
> On Wed, Apr 3, 2013 at 11:48 AM, Ted Yu <yu...@gmail.com> wrote:
>
>> Is it possible that minVersions was set for your column family ?
>>
>> What version of HBase are you using ?
>>
>> Thanks
>>
>>
>> On Wed, Apr 3, 2013 at 11:34 AM, Ashish Nigam <ashnigamtech@gmail.com
>> >wrote:
>>
>> > Hi,
>> > I have a table "service_counters" that has service id as row id.
>> > There is a column family "stats-hourly" that stores counters related to
>> > that service per hour.
>> > I have set TTL to be 7 days for the column family.
>> >
>> > {NAME => 'service_counters', FAMILIES => [{NAME => 'stats-hourly',
>> > COMPRESSION => 'LZO', TTL => '604800'}]}
>> >
>> > Now the problem is that I still see columns that are more than 7 days
>> old.
>> > I have also run major compactions but I still see old rows.
>> > Any pointers as to why these columns are not being deleted?
>> >
>> > Thanks
>> > Ashish
>> >
>>
>
>
Re: HBase Column Family TTL and cell deletions
Posted by Ian Varley <iv...@salesforce.com>.
I don't know if it's right (haven't checked source just now) but according to this:
http://hbase.apache.org/book/ttl.html
Column family TTL is in seconds, not milliseconds. Could that be the problem?
(If not, we should fix that page in the ref guide.)
On Apr 3, 2013, at 5:19 PM, Ashish Nigam wrote:
Any thoughts?
Here's one cell that should have been deleted ( I am putting HBase get
repines in shell)
hbase(main):001:0> get 'service_counters','3141',{COLUMN =>
'stats-hourly:2013032014-C-U-4092276'}
13/04/03 21:57:17 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
COLUMN CELL
stats-hourly:2013032014-C-U-4092276
timestamp=1364928886015, value=\x00\x00\x00\x00\x00\x00\x00\x06
1 row(s) in 0.1040 seconds
On Wed, Apr 3, 2013 at 12:03 PM, Ashish Nigam <as...@gmail.com>>wrote:
minVersion is set to 0
Here's detailed info on this CF -
{NAME => 'stats-hourly', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER =>
'NONE', REPLICATION_SCOPE => '0', VERSIONS => '3', COMPRESSION => 'LZO',
MIN_VERSIONS => '0', TTL => '604800', KEEP_DELETED_CELLS => 'false',
BLOCKSIZE => '65536', IN_MEMORY => 'false', ENCODE_ON_DISK => 'true',
BLOCKCACHE => 'true'}
HBase version is 0.94.2-cdh4.2.0,
On Wed, Apr 3, 2013 at 11:48 AM, Ted Yu <yu...@gmail.com>> wrote:
Is it possible that minVersions was set for your column family ?
What version of HBase are you using ?
Thanks
On Wed, Apr 3, 2013 at 11:34 AM, Ashish Nigam <as...@gmail.com>
wrote:
Hi,
I have a table "service_counters" that has service id as row id.
There is a column family "stats-hourly" that stores counters related to
that service per hour.
I have set TTL to be 7 days for the column family.
{NAME => 'service_counters', FAMILIES => [{NAME => 'stats-hourly',
COMPRESSION => 'LZO', TTL => '604800'}]}
Now the problem is that I still see columns that are more than 7 days
old.
I have also run major compactions but I still see old rows.
Any pointers as to why these columns are not being deleted?
Thanks
Ashish
Re: HBase Column Family TTL and cell deletions
Posted by Ashish Nigam <as...@gmail.com>.
Any thoughts?
Here's one cell that should have been deleted ( I am putting HBase get
repines in shell)
hbase(main):001:0> get 'service_counters','3141',{COLUMN =>
'stats-hourly:2013032014-C-U-4092276'}
13/04/03 21:57:17 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
COLUMN CELL
stats-hourly:2013032014-C-U-4092276
timestamp=1364928886015, value=\x00\x00\x00\x00\x00\x00\x00\x06
1 row(s) in 0.1040 seconds
On Wed, Apr 3, 2013 at 12:03 PM, Ashish Nigam <as...@gmail.com>wrote:
> minVersion is set to 0
> Here's detailed info on this CF -
>
> {NAME => 'stats-hourly', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER =>
> 'NONE', REPLICATION_SCOPE => '0', VERSIONS => '3', COMPRESSION => 'LZO',
> MIN_VERSIONS => '0', TTL => '604800', KEEP_DELETED_CELLS => 'false',
> BLOCKSIZE => '65536', IN_MEMORY => 'false', ENCODE_ON_DISK => 'true',
> BLOCKCACHE => 'true'}
>
> HBase version is 0.94.2-cdh4.2.0,
>
>
>
> On Wed, Apr 3, 2013 at 11:48 AM, Ted Yu <yu...@gmail.com> wrote:
>
>> Is it possible that minVersions was set for your column family ?
>>
>> What version of HBase are you using ?
>>
>> Thanks
>>
>>
>> On Wed, Apr 3, 2013 at 11:34 AM, Ashish Nigam <ashnigamtech@gmail.com
>> >wrote:
>>
>> > Hi,
>> > I have a table "service_counters" that has service id as row id.
>> > There is a column family "stats-hourly" that stores counters related to
>> > that service per hour.
>> > I have set TTL to be 7 days for the column family.
>> >
>> > {NAME => 'service_counters', FAMILIES => [{NAME => 'stats-hourly',
>> > COMPRESSION => 'LZO', TTL => '604800'}]}
>> >
>> > Now the problem is that I still see columns that are more than 7 days
>> old.
>> > I have also run major compactions but I still see old rows.
>> > Any pointers as to why these columns are not being deleted?
>> >
>> > Thanks
>> > Ashish
>> >
>>
>
>
Re: HBase Column Family TTL and cell deletions
Posted by Ashish Nigam <as...@gmail.com>.
Any thoughts?
Here's one cell that should have been deleted ( I am putting HBase get
repines in shell)
hbase(main):001:0> get 'service_counters','3141',{COLUMN =>
'stats-hourly:2013032014-C-U-4092276'}
13/04/03 21:57:17 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
COLUMN CELL
stats-hourly:2013032014-C-U-4092276
timestamp=1364928886015, value=\x00\x00\x00\x00\x00\x00\x00\x06
1 row(s) in 0.1040 seconds
On Wed, Apr 3, 2013 at 12:03 PM, Ashish Nigam <as...@gmail.com>wrote:
> minVersion is set to 0
> Here's detailed info on this CF -
>
> {NAME => 'stats-hourly', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER =>
> 'NONE', REPLICATION_SCOPE => '0', VERSIONS => '3', COMPRESSION => 'LZO',
> MIN_VERSIONS => '0', TTL => '604800', KEEP_DELETED_CELLS => 'false',
> BLOCKSIZE => '65536', IN_MEMORY => 'false', ENCODE_ON_DISK => 'true',
> BLOCKCACHE => 'true'}
>
> HBase version is 0.94.2-cdh4.2.0,
>
>
>
> On Wed, Apr 3, 2013 at 11:48 AM, Ted Yu <yu...@gmail.com> wrote:
>
>> Is it possible that minVersions was set for your column family ?
>>
>> What version of HBase are you using ?
>>
>> Thanks
>>
>>
>> On Wed, Apr 3, 2013 at 11:34 AM, Ashish Nigam <ashnigamtech@gmail.com
>> >wrote:
>>
>> > Hi,
>> > I have a table "service_counters" that has service id as row id.
>> > There is a column family "stats-hourly" that stores counters related to
>> > that service per hour.
>> > I have set TTL to be 7 days for the column family.
>> >
>> > {NAME => 'service_counters', FAMILIES => [{NAME => 'stats-hourly',
>> > COMPRESSION => 'LZO', TTL => '604800'}]}
>> >
>> > Now the problem is that I still see columns that are more than 7 days
>> old.
>> > I have also run major compactions but I still see old rows.
>> > Any pointers as to why these columns are not being deleted?
>> >
>> > Thanks
>> > Ashish
>> >
>>
>
>
Re: HBase Column Family TTL and cell deletions
Posted by Ashish Nigam <as...@gmail.com>.
minVersion is set to 0
Here's detailed info on this CF -
{NAME => 'stats-hourly', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER =>
'NONE', REPLICATION_SCOPE => '0', VERSIONS => '3', COMPRESSION => 'LZO',
MIN_VERSIONS => '0', TTL => '604800', KEEP_DELETED_CELLS => 'false',
BLOCKSIZE => '65536', IN_MEMORY => 'false', ENCODE_ON_DISK => 'true',
BLOCKCACHE => 'true'}
HBase version is 0.94.2-cdh4.2.0,
On Wed, Apr 3, 2013 at 11:48 AM, Ted Yu <yu...@gmail.com> wrote:
> Is it possible that minVersions was set for your column family ?
>
> What version of HBase are you using ?
>
> Thanks
>
>
> On Wed, Apr 3, 2013 at 11:34 AM, Ashish Nigam <ashnigamtech@gmail.com
> >wrote:
>
> > Hi,
> > I have a table "service_counters" that has service id as row id.
> > There is a column family "stats-hourly" that stores counters related to
> > that service per hour.
> > I have set TTL to be 7 days for the column family.
> >
> > {NAME => 'service_counters', FAMILIES => [{NAME => 'stats-hourly',
> > COMPRESSION => 'LZO', TTL => '604800'}]}
> >
> > Now the problem is that I still see columns that are more than 7 days
> old.
> > I have also run major compactions but I still see old rows.
> > Any pointers as to why these columns are not being deleted?
> >
> > Thanks
> > Ashish
> >
>
Re: HBase Column Family TTL and cell deletions
Posted by Ashish Nigam <as...@gmail.com>.
minVersion is set to 0
Here's detailed info on this CF -
{NAME => 'stats-hourly', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER =>
'NONE', REPLICATION_SCOPE => '0', VERSIONS => '3', COMPRESSION => 'LZO',
MIN_VERSIONS => '0', TTL => '604800', KEEP_DELETED_CELLS => 'false',
BLOCKSIZE => '65536', IN_MEMORY => 'false', ENCODE_ON_DISK => 'true',
BLOCKCACHE => 'true'}
HBase version is 0.94.2-cdh4.2.0,
On Wed, Apr 3, 2013 at 11:48 AM, Ted Yu <yu...@gmail.com> wrote:
> Is it possible that minVersions was set for your column family ?
>
> What version of HBase are you using ?
>
> Thanks
>
>
> On Wed, Apr 3, 2013 at 11:34 AM, Ashish Nigam <ashnigamtech@gmail.com
> >wrote:
>
> > Hi,
> > I have a table "service_counters" that has service id as row id.
> > There is a column family "stats-hourly" that stores counters related to
> > that service per hour.
> > I have set TTL to be 7 days for the column family.
> >
> > {NAME => 'service_counters', FAMILIES => [{NAME => 'stats-hourly',
> > COMPRESSION => 'LZO', TTL => '604800'}]}
> >
> > Now the problem is that I still see columns that are more than 7 days
> old.
> > I have also run major compactions but I still see old rows.
> > Any pointers as to why these columns are not being deleted?
> >
> > Thanks
> > Ashish
> >
>
Re: HBase Column Family TTL and cell deletions
Posted by Ted Yu <yu...@gmail.com>.
Is it possible that minVersions was set for your column family ?
What version of HBase are you using ?
Thanks
On Wed, Apr 3, 2013 at 11:34 AM, Ashish Nigam <as...@gmail.com>wrote:
> Hi,
> I have a table "service_counters" that has service id as row id.
> There is a column family "stats-hourly" that stores counters related to
> that service per hour.
> I have set TTL to be 7 days for the column family.
>
> {NAME => 'service_counters', FAMILIES => [{NAME => 'stats-hourly',
> COMPRESSION => 'LZO', TTL => '604800'}]}
>
> Now the problem is that I still see columns that are more than 7 days old.
> I have also run major compactions but I still see old rows.
> Any pointers as to why these columns are not being deleted?
>
> Thanks
> Ashish
>